Test Suites & Reports
Maestro can run a suite of tests and generate a test report at the end.
To run a suite, point
maestro test
to a folder that contains the flowsmaestro test myFolderWtihTests/
Maestro will run every flow from the directory excluding subfolders. The command will complete successfully if and only if all the flows have completed successfully.
To generate a report, add a
--format
parameter to a test
command:maestro test --format junit myFolderWithTests/
Or if you are using Maestro Cloud:
maestro cloud --format junit myFolderWithTests/
Once execution completes, the report will be stored in a
report.xml
file in a JUnit-compatible format that is supported by most platforms.junit
- JUnit XML format
--output {file}
allows to override the report filename
There are multiple mechanisms to control what flows to run when running a test suite.
Flow tags are covered extensively in the following section
By default, when running a test suite only flows from the top level of a given directory will be executed. Consider the following folder structure:
workspace/
flowA.yaml
subFolder/
flowB.yaml
subSubFolder/
flowC.yaml
When running a
test
or cloud
command on a workspace
folder, only flowA.yaml
will be executed by default (though it is still able to refer to subFolder/flowB.yaml
and subFolder/subSubFolder/flowC.yaml
using runFlow
command).This behaviour can be customised by using inclusion patterns. To do that, update your
config.yaml
(create the file if missing) as following:flows:
- "*" # the default behaviour
- "subFolder/*"
In such case, both flowA and flowB will be included into the test suite but not flowC.
Tests can also be included recursively:
flows:
- "**"
In this example, all flows A, B and C will be included into the test suite.
We discourage usage of deterministic ordering as this will block you from parallelising test execution.
This option is not supported for Maestro Cloud tests. All tests in a suite are going to be executed in parallel regardless of this setting.
Normally, tests in a suite are executed in a non-deterministic order. In cases where a fixed order of execution is required, you can force your tests to run in alphabetical order. Update your
config.yaml
(or create the file if missing) to enable this behaviour:local:
deterministicOrder: true
Last modified 1mo ago