summaryrefslogtreecommitdiffstats
path: root/wiki/src/contribute/release_process/tor-browser.mdwn
blob: ea4c920a950024caee86008acaacce875538041c (plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
[[!meta title="Upgrading the Tor Browser"]]

[[!toc levels=2]]

The big picture
===============

The Tails ISO build system [[!tails_gitweb
config/chroot_local-hooks/10-tbb desc="downloads"]] a set of Tor
Browser tarballs from a location specified in [[!tails_gitweb
config/chroot_local-includes/usr/share/tails/tbb-dist-url.txt]], and
compares their hash with previously verified ones found in
[[!tails_gitweb
config/chroot_local-includes/usr/share/tails/tbb-sha256sums.txt]].

Once released officially, Tor Browser tarballs can be found in
a [permanent (?)
location](http://archive.torproject.org/tor-package-archive/torbrowser/).
However, when upgrading Tor Browser for an imminent Tails release, we
generally have to use Tor Browser tarballs that are under QA and not
officially released yet. So, we have to retrieve them from another,
temporary location, such as
<http://people.torproject.org/~mikeperry/builds/>. If we hard-coded
this temporary URL in `tbb-dist-url.txt`, then our release tag would
only be buildable for as long the tarballs stay in that place, which
at best is a few months.

To solve this, we host ourselves the Tor Browser tarballs we need, and
point to [this permanent
location](http://torbrowser-archive.tails.boum.org/) for anything that
we tag.

Still, one can set an arbitrary download location in
`tbb-dist-url.txt`, which should provide all the flexibility needed
for development purposes.

Upgrade Tor Browser in Tails
============================

Have a look at

* <https://archive.torproject.org/tor-package-archive/torbrowser/>
* <https://www.torproject.org/dist/torbrowser/>
* <https://people.torproject.org/~mikeperry/builds/>
* <https://people.torproject.org/~gk/builds/>
* <https://people.torproject.org/~boklm/builds/>
* <https://people.torproject.org/~linus/builds/>

and see if the desired version is available. Set `TBB_DIST_URL` to the
chosen URL, and set `TBB_VERSION` to the desired Tor Browser version, for
example:

    TBB_DIST_URL=https://people.torproject.org/~mikeperry/builds/4.5-build5/
    TBB_VERSION=4.5-build5

Fetch the version's hash file and its detached signature, and verify
with GnuPG:

    wget ${TBB_DIST_URL}/sha256sums-unsigned-build.txt{.asc,} && \
    gpg --verify sha256sums-unsigned-build.txt{.asc,}

Filter the tarballs we want and make them available at build time,
when the tarballs are fetched:

    grep --color=never "\<tor-browser-linux32-.*\.tar.xz$" sha256sums-unsigned-build.txt > \
         config/chroot_local-includes/usr/share/tails/tbb-sha256sums.txt

Then update the URL to the one chosen above:

    echo "${TBB_DIST_URL}" | sed "s,^https://,http://," > \
         config/chroot_local-includes/usr/share/tails/tbb-dist-url.txt

<div class="note">
<p>
We cannot use HTTPS due to limitations/bugs in
<code>apt-cacher-ng</code>, which often is used in Tails build
environments. However, it is of no consequence since we verify the
checksum file.
</p>
</div>

Lastly, commit:

    git commit config/chroot_local-includes/usr/share/tails/tbb-*.txt \
        -m "Upgrade Tor Browser to ${TBB_VERSION}."

Then do the same operation for the 64-bit tarballs:

    git checkout feature/stretch && \
    grep --color=never "\<tor-browser-linux64-.*\.tar.xz$" sha256sums-unsigned-build.txt > \
         config/chroot_local-includes/usr/share/tails/tbb-sha256sums.txt && \
    echo "${TBB_DIST_URL}" | sed "s,^https://,http://," > \
         config/chroot_local-includes/usr/share/tails/tbb-dist-url.txt && \
    git commit config/chroot_local-includes/usr/share/tails/tbb-*.txt \
        -m "Upgrade Tor Browser to ${TBB_VERSION}."

<div class="caution">
<p>
If this new Tor Browser is meant to be included in a Tails
release, then that's not enough: as explained above, we need to host
the corresponding tarballs ourselves, so read on the next section.
</p>
</div>

Sync with the start-tor-browser script
======================================

Adapt our `config/chroot_local-includes/usr/local/bin/tor-browser`
and/or
`config/chroot_local-includes/usr/local/lib/tails-shell-library/tor-browser.sh`
for recent changes in `RelativeLink/start-tor-browser` in the
[Tor Browser Bundle Git repo](https://git.torproject.org/builders/tor-browser-bundle.git). Look
in the Git history:

    git log -p RelativeLink/start-tor-browser

and take note of changes to environment variables (or newly added
ones) and the commandline options passed to the `firefox` executable,
etc.

Self-hosted Tor Browser tarballs archive
========================================

Initial setup
-------------

First, install git-annex.

Then, make sure you have an entry for `git.puppet.tails.boum.org` in
your `~/.ssh/config`. See `systems/ISO_history.mdwn` in the internal Git repo
for details.

Then, clone the metadata repository and initialize git-annex:

	git clone gitolite@git.puppet.tails.boum.org:torbrowser-archive.git && \
	cd torbrowser-archive && \
	git annex init 

You now have a lot of (dangling) symlinks in place of the files that are
available in this git-annex repo.

To synchronize your local git-annex metadata with the remote, run:

	git annex sync

Set up environment variables
----------------------------

1. Make sure you still have the environment variables defined in the
   previous section set.

2. Make `TAILS_GIT_REPO` point to the main Tails Git repository
   checkout where `tbb-dist-url.txt` is being worked on, for example:

       TAILS_GIT_REPO="$HOME/tails/git"

3. Make `TBB_ARCHIVE` point to your local git annex working
   copy of our Tor Browser archive, for example:

       TBB_ARCHIVE="$HOME/tails/torbrowser-archive"

4. Make `TBB_IMPORT_BRANCH` point to the branch where you want to
   import the new Tor Browser's metadata, for example:

       TBB_IMPORT_BRANCH=feature/123456-torbrowser-42.3.4

Import a new set of Tor Browser tarballs
----------------------------------------

1. Download and verify all the tarballs we need:

       TMPDIR=$(mktemp -d)
       CHROOT_INCLUDES="config/chroot_local-includes"
       TBB_SHA256SUMS_FILE="${CHROOT_INCLUDES}/usr/share/tails/tbb-sha256sums.txt"
       TBB_DIST_URL_FILE="${CHROOT_INCLUDES}/usr/share/tails/tbb-dist-url.txt"
       cd "$TAILS_GIT_REPO" && git checkout "$TBB_IMPORT_BRANCH"
       TBB_TARBALLS_BASE_URL="$(cat "${TBB_DIST_URL_FILE}" | sed "s,^http://,https://,")"
       current_branch=$(git -C "$TAILS_GIT_REPO" branch | awk '/^\* / { print $2 }')
       for branch in "$current_branch" feature/stretch ; do
          git -C "$TAILS_GIT_REPO" show "$branch:$TBB_SHA256SUMS_FILE" \
          | while read expected_sha256 tarball; do
             (
                cd "$TMPDIR"
                echo "Retrieving '${TBB_TARBALLS_BASE_URL}/${tarball}'..."
                curl --remote-name --continue-at - \
                   "${TBB_TARBALLS_BASE_URL}/${tarball}"
             )
          done
          (
             cd "$TMPDIR" && \
             git -C "$TAILS_GIT_REPO" show "$branch:$TBB_SHA256SUMS_FILE" \
                | sha256sum -c -
          )
       done

3. Move the tarballs into your local Git annex:

       cd "$TBB_ARCHIVE" && \
       mkdir "$TBB_VERSION" && cd "$TBB_VERSION" && \
       git annex import --duplicate "$TMPDIR/"* "$TAILS_GIT_REPO/"sha256sums-*

Commit and push your changes
----------------------------

	cd "$TBB_ARCHIVE" && \
	git commit -m "Add Tor Browser ${TBB_VERSION}." && \
	git annex sync && \
	git annex copy --to origin -- "${TBB_VERSION}"

Wait for the synchronization
----------------------------

Once you've gone through these steps, a cronjob that runs every
5 minutes will download the tarballs and make them available on
<http://torbrowser-archive.tails.boum.org/>.

Wait for this to happen before you proceed with the next steps.

In the meantime, you might want to import the new Tor Browser tarballs
into your `apt-cacher-ng` local cache.

Adjust the URL in the main Git repository
-----------------------------------------

    cd "$TAILS_GIT_REPO" && \
    git checkout "$TBB_IMPORT_BRANCH"
    current_branch=$(git branch | awk '/^\* / { print $2 }')
    for branch in "$current_branch" feature/stretch ; do
       git checkout "$branch" && \
       echo "http://torbrowser-archive.tails.boum.org/${TBB_VERSION}/" > \
            config/chroot_local-includes/usr/share/tails/tbb-dist-url.txt && \
       git commit config/chroot_local-includes/usr/share/tails/tbb-dist-url.txt \
           -m "Fetch Tor Browser from our own archive."
    done