summaryrefslogtreecommitdiffstats
path: root/wiki/src/contribute/build.mdwn
blob: 4e0cead290efaf1ea7d8eda14bfb08424bcaf223 (plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
[[!meta title="Building a Tails image"]]

[[!toc levels=3]]

Git repository and branches
===========================

You will need to clone the Tails Git repository, and to checkout the
branch that you want to build (most likely, _not_ `master`): learn
more about [[our Git branches layout|contribute/git#main-repo]].

<a id="vagrant"></a>

Using Vagrant
=============

Tails can be built easily in a virtual machine using [Rake], [Vagrant]
and [vagrant-libvirt]. The process requires the
[[!wikipedia Kernel-based_Virtual_Machine desc="KVM"]] virtual machine
hypervisor to be available, a minimum of 512 MiB of free memory and a
maximum of 20 GB of free storage.

[Rake]: http://rake.rubyforge.org/
[Vagrant]: http://vagrantup.com/
[vagrant-libvirt]: https://github.com/vagrant-libvirt/vagrant-libvirt/

For details how Vagrant is setup, see its
[[design page|build/vagrant-setup]].

## Installing the dependencies in Debian

If you run Debian Jessie, you must install a few dependencies from
Debian Stretch, and one of Tails' repos until [!debbug 823395]] is fixed:

    sudo tee /etc/apt/sources.list.d/stretch.list <<EOF
    deb http://ftp.us.debian.org/debian/ stretch main
    deb-src http://ftp.us.debian.org/debian/ stretch main
    EOF
    sudo tee /etc/apt/preferences.d/vagrant-libvirt-from-stretch.pref <<EOF
    Package: *
    Pin: release o=Debian,n=stretch
    Pin-Priority: 100
    
    Package: vagrant vagrant-libvirt ruby-excon ruby-fog-core ruby-fog-libvirt ruby-fog-xml
    Pin: release o=Debian,n=stretch
    Pin-Priority: 990
    EOF

    sudo apt-key add config/chroot_sources/tails.chroot.gpg
    sudo tee /etc/apt/sources.list.d/builder-jessie.list <<EOF
    deb http://deb.tails.boum.org/ builder-jessie main
    deb-src http://deb.tails.boum.org/ builder-jessie main
    EOF

    sudo apt-get update

Now we can install all the dependencies:

    sudo apt-get install \
        git \
        rake \
        libvirt-daemon-system \
        dnsmasq-base \
        ebtables \
        qemu-system-x86 \
        qemu-utils \
        vagrant \
        vagrant-libvirt
    sudo systemctl restart libvirtd

Make sure that the user that is supposed to initiate the build command
is part of the `kvm`, `libvirt` and `libvirt-qemu` groups:

    for group in kvm libvirt libvirt-qemu; do sudo adduser $USER $group; done

Then reboot the system (alternatively, if you know what you are doing,
run `newgrp` and reload the `kvm` module(s)).

## Known issues and workarounds

* If Vagrant failed to start the Tails builder VM the first time
  (e.g. because of permission issues or the `kvm` module not veing
  loaded) it will not automatically run the provisioning script, so
  you must run `rake vm:provision` yourself before attempting your
  first `rake build`. If that fails, run `rake vm:destroy`, which
  removes this half-broken VM, and then start from scratch with `rake
  build` or similar.

* `ruby-fog-libvirt <= 0.0.3` has a
  [bug](https://github.com/fog/fog-libvirt/issues/21) that will make
  Vagrant crash when trying to start the VM if you have any libvirt
  domain defined with a floppy (`fd`) as part of the boot order, set
  via `<os>` tag; please remove all such occurrences! If you really
  need to have a floppy as part of some domain's boot order, please
  set it via `<disk>` by adding `<boot order='1'/>` which does not
  trigger the bug.

* If the system you are setting this up on ever had the old
  (Virtualbox-based) Vagrant setup, you will get errors unless you
  clean up these parts:

  - Remove any old pinnings for `vagrant` and `ruby-net-ssh`. If you
    followed our old instructions to the point, that should be:

        sudo rm /etc/apt/preferences.d/tails-build-vagrant

  - Clearing the Vagrant configuration seems to also often solve
    issues:

        rm -r ~/.vagrant.d

## Building Tails using Vagrant

Once all dependencies are installed, get the Tails sources and
checkout the development branch:

    git clone https://git-tails.immerda.ch/tails
    cd tails
    git checkout devel

Build Tails using Vagrant:

    rake build

The first time, this can take a little while to download the base virtual
machine from Tails mirror (around 300 MB). It will then boot the machine,
set it up and start the build process. When done, several `tails-*` files
should appear in the current directory.

After you are done working on Tails, do not forget to shut the virtual
machine down:

    rake vm:halt

One may also want to [[contribute/customize]] their image before building.

To know all available Rake tasks, please run `rake -T`.

Local HTTP proxy
----------------

If you have a local HTTP proxy, the build system will use it as long as
you properly set the `http_proxy` environment variable. The easiest way to
do so is to run:

    http_proxy=http://proxy.lan:3142

... and then, pass the `extproxy` build option (see below).

This needs to be done before any other operations.

Build options
-------------

Options regarding the build process can be set using the
`TAILS_BUILD_OPTIONS` environment variable. Multiple options must be
separated by whitespaces.

The following options are available:

### Memory build settings

Tails builds way faster when everything is done in memory. If your computer
runs Linux and happens to have more than 7 GB of free memory before you
start the virtual machine, it will automatically switch to 'build in RAM'
mode.

To force a specific behaviour please set:

 * **ram**: start the virtual machine with 7 GB of memory, build Tails
   inside a `tmpfs`. Build fails if the system is not in a proper state to
   do so.
 * **noram**: start the virtual machine with 512 MB of memory if not already
   done, build Tails using the virtual machine hard disk.

### HTTP proxy settings

Building Tails requires downloading a little bit more than 1 GB of Debian
packages. To preserve bandwidth and developer sanity, using a HTTP proxy is
nearly a must. Tails virtual machine contains a fully configured local HTTP
proxy that will be used if no other local proxy is defined.

The following flags can be used to force a specific behaviour:

 * **extproxy**: use the proxy configured through the `http_proxy`
   environment variable. Fail if it is not set.
 * **vmproxy**: use the local proxy configured in the virtual machine even
   if a local HTTP proxy is set.
 * **noproxy**: do not use any HTTP proxy.

### SquashFS compression settings

One of the most expensive operations when building Tails is the creation
of the final SquashFS. It also depends on the compression algorithm used.
When working on the `stable` or `testing` branch, the image will be made
using the slow but efficient default. Any other setup will switch to the
faster *gzip*.

Forcing a specific behaviour can be done using:

 * **gzipcomp**: always use *gzip* to create the SquashFS.
 * **defaultcomp**: always use the default compression algorithm.

### Clean-up settings

Some operations are preserved accross builds. Currently they are:

* The wiki (for documentation).

In case you want to delete all these, the following option is available:

 * **cleanall**: force a clean up before starting the build.

### Virtual CPUs settings

The number of virtual CPUs that are allocated in the virtual machine can be
customized through:

 * **cpus=_n_**: allocate _n_ CPUs to the virtual machine.

Obviously you should not allocate more virtual CPUs than the number of cores
available to the host system. When using Linux, the number of CPUs allocated
will default to be the same as the host system.

### Git settings

The build system can only work on files that have been *committed* to the Git
repository. By default, it will refuse to start a build in presence of
uncommitted changes. This behaviour can be controlled by:

 * **ignorechanges**: allow to make a build that will ignore changes in the Git
   repository.

### Example

The fastest build you could pretend to get can be done by setting:

    export TAILS_BUILD_OPTIONS="ram cache extproxy gzipcomp"

This will force the build to happen in RAM and allow skipping the
boostrap stage if one is cached, and will use use an HTTP proxy
external to the virtual machine, and SquashFS compression will be done
using *gzip*.

<a id="manual"></a>

Building manually
=================

In order to build Tails manually, you need a running Debian Jessie
system and some [backports](http://backports.debian.org/). Anything
else will fail.

Dependencies
------------

The following Debian packages need to be installed:

* our `live-build` 2.x package, adapted for Wheezy and later. Its version is
  something like *3.0.5+really+is+2.0.12-0.tails2*. One can install it
  from:

      deb http://deb.tails.boum.org/ builder-jessie main

  This APT repository's signing key can be found:

  - in our Git tree (that you have cloned already, right?):
    `config/chroot_sources/tails.chroot.gpg`
  - at <http://deb.tails.boum.org/key.asc>
  - on the keyservers.

  It is certified by the
  [[Tails signing key|doc/about/openpgp_keys#signing]], and its
  fingerprint is:

      221F 9A3C 6FA3 E09E 182E  060B C798 8EA7 A358 D82E

  You should pin that repository, so that live-build isn't upgraded to
  the version of jessie.

      #/etc/apt/preferences.d/00-builder-jessie-pinning
      Package: *
      Pin: release o=Debian,a=stable
      Pin-Priority: 700

      Package: *
      Pin: origin deb.tails.boum.org
      Pin-Priority: 800


  Then install these dependencies from Jessie:

      apt-get install \
        dpkg-dev \
        eatmydata \
        gettext \
        intltool \
        libfile-slurp-perl \
        liblist-moreutils-perl \
        libyaml-libyaml-perl \
        libyaml-perl \
        libyaml-syck-perl \
        perlmagick \
        po4a \
        syslinux-utils \
        time \
        whois

  And install these dependencies from jessie-backports (please verify
  manually that the following command actually does install the
  expected versions):

      apt-get install \
        debootstrap/jessie-backports \
        ikiwiki/jessie-backports

Configure live-build
--------------------

Remove any line matching `/^\[[:space:]]*LB.*MIRROR.*=/` in
`/etc/live/build.conf`.

Build process
-------------

Every build command must be run as `root`, at the root of a clone of the
[[`tails` repository|git]].

In short, a build shall be done using:

    lb clean --all && lb config && lb build

Running `lb config` or `lb build` in an environment that wasn't full
cleaned first is not supported.

### Customize the build process if needed

If you need to set custom build settings that are specific to your
local environment, such as a custom Debian mirror or APT proxy, you
probably want to configure live-build a bit.

The most common customizations are documented on this wiki:

* to avoid compressing the SquashFS using XZ (efficient, but very
  slow), `export MKSQUASHFS_OPTIONS='-comp gzip'` in your
  build environment;
* to avoid downloading lots of Debian packages during every build, you
  can use [[!debpts apt-cacher-ng]]; however, the build system
  constantly switches APT sources for our
  [[APT repositories|contribute/APT_repository]], so some custom
  configuration is needed to make `apt-cacher-ng` useful: see the
  bits about `apt-cacher-ng` in
  [[!tails_gitweb vagrant/provision/assets/build-tails]].

More documentation about this can be found in the [Debian Live
Manual](http://live.debian.net/manual-2.x/html/live-manual.en.html).

More information
================

More documentation about the build process can be found in the [Debian
Live Manual](http://live.debian.net/manual/oldstable/html/live-manual.en.html).

Related pages
=============

[[!map pages="contribute/build/*"]]