path: root/wiki/src/contribute/release_process/test
diff options
Diffstat (limited to 'wiki/src/contribute/release_process/test')
4 files changed, 123 insertions, 3 deletions
diff --git a/wiki/src/contribute/release_process/test/automated_tests.mdwn b/wiki/src/contribute/release_process/test/automated_tests.mdwn
index 22db600..e637935 100644
--- a/wiki/src/contribute/release_process/test/automated_tests.mdwn
+++ b/wiki/src/contribute/release_process/test/automated_tests.mdwn
@@ -25,6 +25,8 @@ the Sikuli programmatic interface from our step definitions.
See [[contribute/release_process/test/setup]] and [[contribute/release_process/test/usage]].
+Core developers can also run it [[usage/on_lizard]].
## Features
With this tool, it is possible to:
diff --git a/wiki/src/contribute/release_process/test/setup.mdwn b/wiki/src/contribute/release_process/test/setup.mdwn
index d9b35d4..5c9008a 100644
--- a/wiki/src/contribute/release_process/test/setup.mdwn
+++ b/wiki/src/contribute/release_process/test/setup.mdwn
@@ -20,14 +20,22 @@ wheezy-backports sources added:
echo -e "Package: *\nPin: release o=Debian,a=testing\nPin-Priority: 500" \
> /etc/apt/preferences.d/Debian_testing && \
apt-get update &&
- apt-get install git xvfb virt-viewer libsikuli-script-java \
+ apt-get install git i18nspector xvfb virt-viewer libsikuli-script-java \
libxslt1-dev tcpdump unclutter radvd x11-apps syslinux \
libcap2-bin devscripts libvirt-ruby ruby-rspec gawk ntp ovmf/testing \
ruby-json x11vnc xtightvncviewer ffmpeg libavcodec-extra-53 \
- libvpx1 dnsmasq-base openjdk-7-jre && \
+ libvpx1 dnsmasq-base openjdk-7-jre ruby-guestfs && \
apt-get -t wheezy-backports install qemu-kvm qemu-system-x86 libvirt0 \
libvirt-dev libvirt-bin seabios ruby-rjb ruby-packetfu cucumber && \
- service libvirt-bin restart
+ service libvirtd restart
+In addition, if `libguestfs` doesn't work by default you probably have
+hit [[!debbug 741203]], which still may affect Wheezy systems but it's
+verified to have been fixed in Debian Jessie. Then you must run (as
+ export excludes="--exclude ^openrc"
+ update-guestfs-appliance
Other requirements
diff --git a/wiki/src/contribute/release_process/test/usage.mdwn b/wiki/src/contribute/release_process/test/usage.mdwn
index 1535360..9376cee 100644
--- a/wiki/src/contribute/release_process/test/usage.mdwn
+++ b/wiki/src/contribute/release_process/test/usage.mdwn
@@ -1,5 +1,10 @@
[[!meta title="Running the automated test suite"]]
+[[!toc levels=2]]
+Basic usage
Use the `run_test_suite` script found in the Tails source root to run
all automated Cucumber test features. See the [[setup
documentation|test/setup]] in case you don't have a testing
@@ -31,3 +36,79 @@ For full instructions, see its `--help`.
Note: since the test suite itself uses `virt-viewer` to interact with
the Tails guest you cannot use it as that will steal the session from
the test suite, leaving it with a blank screen.
+The test suite can be configured in the following ways:
+1. `run_test_suite` parameters, which takes precedence over
+2. the local configuration file `features/config/local.yml`, which
+takes precedence over
+3. the default configuration file `features/config/defaults.yml`.
+However, some values are treated as secrets and have no
+defaults. These secrets are generally information about online sevices
+to be used in certain features, like host, port and credentials --
+stuff we don't want to make public. These must be set explicitly in
+order for those features to run.
+## Non-secret configuration
+Here's a list of all non-secret key-value pairs that can be supported
+by the local configuration file:
+* `DEBUG`: Boolean value. If set to `true`, various debugging info
+ will be printed. Defaults to `false`.
+* `PAUSE_ON_FAIL`: Boolean value. If set to `true`, the test suite run
+ is suspended on failure until ENTER is pressed. This is useful for
+ investigating the state of the VM guest to see exactly why a test
+ failed. Defaults to `false`.
+* `SIKULI_RETRY_FINDFAILED`: Boolean value. If set to `true`, print a
+ warning whenever Sikuli fails to find an image and allow *one* retry
+ after pressing ENTER. This is useful for updating outdated images,
+ or when extracting new images. Defaults to `false`.
+* `TEMP_DIR`: String value. Directory where various temporary files
+ are written during a test, e.g. VM snapshots and memory dumps,
+ failure screenshots, pcap files and disk images. Defaults to
+ `"/tmp/TailsToaster"`.
+## "Secret" configuration
+This section describes the formats for all secret configurations that
+must be configured in the local configuration file for certain
+features or scenarios to work. If any of these are omitted, parts of
+the test suite will fail.
+### Tor pluggable transports
+The format is:
+ Tor:
+ Transports:
+ $TYPE:
+ - ipv4_address: ""
+ ipv4_port: 443
+ fingerprint: "01234567890abcdef01234567890abcdef012345"
+ extra:
+ - ipv4_address: ""
+ [...]
+ - ipv4_address: ""
+ [...]
+where the type `$TYPE` (and `$ANOTHER_TYPE`) should be something like
+`Obfs4` or `Bridge` (the first type) or whatever Tor calls them. Both
+`fingerprint` and `extra` are optional and can be left empty (or
+skipped completely), but e.g. `extra` is necessary for `Obfs4` type
+bridges, for the `cert=... iat-mode=...` stuff, and the same for
+`Scramblesuite`'s `password=...`.
+This setting is required for `tor_bridges.feature` (requires types
+`Bridge`, `Obfs2`, `Obfs3` and `Obfs4`) and `time_syncing.feature`
+(requires type `Bridge` only).
diff --git a/wiki/src/contribute/release_process/test/usage/on_lizard.mdwn b/wiki/src/contribute/release_process/test/usage/on_lizard.mdwn
new file mode 100644
index 0000000..9d4d09d
--- /dev/null
+++ b/wiki/src/contribute/release_process/test/usage/on_lizard.mdwn
@@ -0,0 +1,29 @@
+[[!meta title="Running the automated test suite on lizard"]]
+The isotester1 VM on lizard is configured to run our automated test
+suite. Core Tails developers can run it there, e.g. on ISO images
+built by Jenkins.
+[[!toc levels=2]]
+# Entering the system
+As a core developer, you have SSH access to `isotester1.lizard`.
+The connection details you need live in our internal Git repository.
+# Getting an ISO image
+You can quickly retrieve ISO images built by Jenkins from
+<>, e.g. using `wget` or `curl`.
+# Running the test suite
+Use `sudo su -` to enter a root session.
+A clone of the Tails Git repository can be found in `/srv/git/`.
+When using the `--capture` option, please pass it a filename within
+`/tmp/TailsToaster`, for storage management reasons.
+You can access the VNC display of the system under testing using
+a SSH tunnel.