[[!meta title="Running the automated test suite"]]
Use the `run_test_suite` script found in the Tails source root to run
all automated Cucumber test features. See the [[setup
documentation|test/setup]] in case you don't have a testing
It's important to note that some features only depend on the Tails
sources, and some on the actual product of the sources, i.e. a Tails
ISO image. These features are tagged `@source` and `@product`,
respectively. The arguments passed to `run_test_suite` may only affect
one of these types of features and not the other.
A typical example run of a few `@product` features could be:
./run_test_suite --view --capture test-0.17.webm \
--iso path/to/tails.iso \
which will test only the `apt` and `erase_memory` features (if
no feature paths are given, all features in `features/cucumber` will
be tested) of the given ISO image `tails.iso` while showing the test
session in a VNC viewer (`--view`) and also capturing it into a video
called `test-0.17.web` (`--capture`). Similarly, to test a `@source`
feature, we'd simply run something like:
For full instructions, see its `--help`.
Note: since the test suite itself uses `virt-viewer` to interact with
the Tails guest you cannot use it as that will steal the session from
the test suite, leaving it with a blank screen.
The test suite can be configured in the following ways:
1. `run_test_suite` parameters, which takes precedence over
2. the local configuration file `features/config/local.yml`, which
takes precedence over
3. the default configuration file `features/config/defaults.yml`.
However, some values are treated as secrets and have no
defaults. These secrets are generally information about online sevices
to be used in certain features, like host, port and credentials --
stuff we don't want to make public. These must be set explicitly in
order for those features to run.
Here's a list of all non-secret key-value pairs that can be supported
by the local configuration file:
* `DEBUG`: Boolean value. If set to `true`, various debugging info
will be printed. Defaults to `false`.
* `PAUSE_ON_FAIL`: Boolean value. If set to `true`, the test suite run
is suspended on failure until ENTER is pressed. This is useful for
investigating the state of the VM guest to see exactly why a test
failed. Defaults to `false`.
* `SIKULI_RETRY_FINDFAILED`: Boolean value. If set to `true`, print a
warning whenever Sikuli fails to find an image and allow *one* retry
after pressing ENTER. This is useful for updating outdated images,
or when extracting new images. Defaults to `false`.
* `TEMP_DIR`: String value. Directory where various temporary files
are written during a test, e.g. VM snapshots and memory dumps,
failure screenshots, pcap files and disk images. Defaults to