summaryrefslogtreecommitdiffstats
path: root/vagrant
Commit message (Collapse)AuthorAgeFilesLines
* Rubocop: enable Style/MethodCallWithArgsParenthesesintrigeri2020-05-111-6/+7
| | | | | … with a fairly relaxed configuration. Fix a few remaining offenses that did not feel worth making additional exceptions for.
* Rubocop: enable Metrics/BlockLengthintrigeri2020-05-101-1/+1
| | | | | … with slightly relaxed maximum value, and dropping the ball on the few worst offenders for now.
* Rubocop: fix Style/MutableConstant offensesintrigeri2020-05-101-3/+3
| | | | | | Changes generated with: `rubocop --only Style/MutableConstant --auto-correct`
* Rubocop: manually fix Style/MultilineTernaryOperator offensesintrigeri2020-04-231-2/+5
|
* Rubocop: fix Style/ConditionalAssignment offensesintrigeri2020-04-231-5/+1
| | | | | Initially done via --auto-correct, then manually linted the resulting code (that would otherwise introduce a bunch of offenses).
* Rubocop: manually fix Naming/HeredocDelimiterNaming offensesintrigeri2020-04-231-2/+2
|
* Rubocop: fix Style/Encoding offensesintrigeri2020-04-221-2/+0
| | | | | | Changes generated with: `rubocop --only Style/Encoding --auto-correct`
* Rubocop: fix Style/RedundantReturn offensesintrigeri2020-04-221-1/+1
| | | | | | Changes generated with: `rubocop --only Style/RedundantReturn --auto-correct`
* Rubocop: fix Style/StringLiterals offensesintrigeri2020-04-212-6/+6
| | | | | | Changes generated with: `rubocop --only Style/StringLiterals --auto-correct`
* Rubocop: manually fix remaining Lint/UnusedBlockArgument offensesintrigeri2020-04-211-1/+1
|
* Rubocop: fix Layout/SpaceAroundOperators offensesintrigeri2020-04-211-2/+2
| | | | | | Changes generated with: `rubocop --only Layout/SpaceAroundOperators --auto-correct`
* Rubocop: fix Layout/SpaceAfterComma offensesintrigeri2020-04-211-1/+1
| | | | | | Changes generated with: `rubocop --only Layout/SpaceAfterComma --auto-correct`
* Rubocop: fix Layout/MultilineMethodCallIndentation offensesintrigeri2020-04-211-1/+1
| | | | | | Changes generated with: `rubocop --only Layout/MultilineMethodCallIndentation --auto-correct`
* Rubocop: fix Layout/EmptyLineAfterMagicComment offensesintrigeri2020-04-211-0/+1
| | | | | | Changes generated with: `rubocop --only Layout/EmptyLineAfterMagicComment --auto-correct`
* Rubocop: fix Layout/SpaceInsideReferenceBrackets offensesintrigeri2020-04-211-2/+2
| | | | | | Changes generated with: `rubocop --only Layout/SpaceInsideReferenceBrackets --auto-correct`
* Vagrant box: upgrade to po4a 0.55 (refs: #17005)intrigeri2020-04-121-7/+1
|
* Freeze APT snapshots for 4.5~rc1.Cyril Brulebois2020-03-263-3/+3
|
* Merge remote-tracking branch 'origin/stable' into ↵intrigeri2020-03-052-1/+3
|\ | | | | | | feature/8415-overlayfs+force-all-tests
| * Merge remote-tracking branch 'origin/stable' into ↵intrigeri2020-02-071-3/+0
| |\ | | | | | | | | | feature/17386-vagrant-disable-cpu-vuln-mitigations
| * \ Merge remote-tracking branch 'origin/stable' into ↵intrigeri2020-01-144-100/+119
| |\ \ | | | | | | | | | | | | feature/17386-vagrant-disable-cpu-vuln-mitigations
| * | | Vagrant build box: disable mitigation features for CPU vulnerabilities ↵intrigeri2019-12-301-0/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | (refs: #17386) Given the kind of things we do in our Vagrant build box, it seems very unlikely that vulnerabilities such as Spectre and Meltdown can be exploited in there. Let's reclaim some of the performance cost of the corresponding mitigation features. As of Linux 4.9.189, mitigations=off is equivalent to: nopti nospectre_v1 nospectre_v2 spectre_v2_user=off spec_store_bypass_disable=off l1tf=off mds=off As of Linux 5.4.5, these extra options are added to the above list: kpti=0 nobp=0 ssbd=force-off tsx_async_abort=off kvm.nx_huge_pages=off
| * | | Vagrant build box: remove obsolete duplicate assignment to config.vm.box_urlintrigeri2019-12-301-1/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This line was overridden by the next one: config.vm.box_url = "http://127.0.0.1/dev/null" … which is there to ensure we never download any basebox.
* | | | Vagrant build-tails script: drop obsolete reference to aufs (refs: #17451)intrigeri2020-02-211-1/+1
| |_|/ |/| | | | | | | | We've ditched aufs usage from our build code in Tails 2.4.
* | | Merge remote-tracking branch ↵intrigeri2020-02-071-3/+0
|\ \ \ | |_|/ |/| | | | | 'johanbluecreek/bugfix/17179-proper-chroot-for-syslinux' into stable (Closes: #17179)
| * | syslinux is not used in tails builder anymoreJohan Blåbäck2020-01-091-3/+0
| |/
* | Build system: always use a fresh Git cloneintrigeri2020-01-011-24/+23
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | I often see build aborting with this command failing: git checkout --force "${GIT_REF}" … because for whatever reason, the branch I'm building is not present in $TAILS_GIT_DIR. Some of the problems reported on #17289 also seem to come from an outdated $TAILS_GIT_DIR. I think that's because after the initial cloning, we run: git config remote.origin.fetch +refs/remotes/origin/*:refs/remotes/origin/* … which hides origin/* refs that are present on the host system but were not pushed to the real "origin" remote yet. IMO a slightly slower, but more robust, build process is better for developer experience than a brittle one: several of our team-mates have grown quite frustrated by our Vagrant build system due to this sort of problems, and have to regularly resort to heavy hammer workarounds such as "rake vm:destroy && rake basebox:clean_old", because it's too annoying to figure out which exact failure mode one they're in and which more pinpointed workaround would work in that case.
* | Add missing word in commentintrigeri2020-01-011-1/+1
| |
* | Build system: shrink the apt-cacher-ng cache after a successful build too ↵intrigeri2020-01-011-12/+18
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | (refs: #17288) 7f0a1f0e42792cc715f28f8566c98fa1e6bacda3 moved this cache cleanup operation from the provisioning script to build-tails, so it's run before each build. This is great and improves things in most cases. But there's one situation in which this change makes things worse: 1. Before build N, the cache was approaching the 10G limit. 2. Build N has filled the cache. 3. While we provision the VM for build N+1, APT operations fail: apt-cacher-ng returns a "503 OUT OF DISK SPACE" error. To mitigate this, let's do garbage collection both before and after a build.
* | Build system: make indentation style internally consistent in build-tailsintrigeri2020-01-011-3/+4
| |
* | Build system: make indentation style internally consistent in ↵intrigeri2020-01-011-14/+15
| | | | | | | | setup-tails-builder
* | Build system: make the code less order-dependentintrigeri2020-01-011-2/+1
| |
* | Build system: behave correctly when disabling a previously set "offline" or ↵intrigeri2020-01-011-0/+1
| | | | | | | | | | | | | | "vmproxy+extproxy" build option Previously, setting one of these build options *once* would taint the Vagrant box forever with the resulting apt-cacher-ng configuration.
* | Split long lineintrigeri2020-01-011-1/+2
| |
* | Build system: fully provision the Vagrant box every time it starts, and ↵intrigeri2020-01-011-13/+23
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | partially re-provision it for every build Commit 5b2a3af4af533af8244f7c9a8e6cf028c18b7f0b said it would "provision a new Vagrant VM for every ISO build" (except in some cases), but I'm not sure I understand: we did exactly this before and that commit removed the very line where we did run_vagrant('provision'). I understand the rest of that commit was supposed to ensure we provision the VM "for every ISO build", but I've seen many weird failure modes in the last couple years, on my system and coworkers', such as builds failing because build-tails was not installed (which is supposed to have been done at provisioning time), but Vagrant skipped the provisioning step because it was under the wrong impression that the VM had been provisioned already¹. I suspect our mechanism to make Vagrant forget that it did already provision a given box is not fully reliable. So, I'm bringing back an explicit provisioning step, this time in the vm:up task. This should help guarantee that the provisioning was done before we start the :build task. Additionally, in order to mitigate the resulting performance regression for subsequent builds when using the "rescue" or "keeprunning" build option, let's skip unneeded provisioning steps when building for the second time (or more) since the Vagrant box started: while it makes sense to run some of the provisioning steps for every build, some other steps are not necessary for the second build and the following ones. [1] According to https://www.vagrantup.com/docs/provisioning/, provisioning is meant to be done when creating a VM for the first time, not every time Vagrant starts a VM. It looks like we're piggy-backing on a concept that means something else, so I would not be surprised if Vagrant did not do what we need by default.
* | Build system: key the website build cache on the input parameters that ↵intrigeri2020-01-013-21/+34
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | primarily determine its output (refs: #15342) As said when I reviewed anonym's initial implementation (5d07d53a0a70a51bfb43d9fa7f317855362a0706), I was not satisfied by a cache that came with no automated invalidation mechanism at all, thus leaving up to each individual developer the "is this weird behavior caused by the cache?" analysis, and the manual invalidation of the cache. So, IMO we need to know when we can invalidate the cache, which is equivalent to: we need to know when a fresh build of the website would yield a significantly different output. This is what this commit does, by maintaining multiple cache directories, keyed to the input parameters that primarily determine the output of the website build. Note that I'm definitely not aiming for perfection here: for example, I did not list all the ikiwiki dependencies recursively. Finally, I'm also moving this cache to a dedicated storage volume, in order to support caching the website even when not using the 'vmproxy' option: - To start with, I don't use 'vmproxy' myself, but I would like to benefit from caching the website. - Furthermore, once we're confident enough in this caching mechanism, we could consider enabling it for our CI builds, that don't use 'vmproxy' either.
* | Build system: remove 5 years old transition codeintrigeri2019-12-311-24/+0
| |
* | Build system: don't install eatmydata (obsolete dependency)intrigeri2019-12-311-1/+0
| |
* | Build system: don't install whois (obsolete dependency)intrigeri2019-12-311-2/+1
| |
* | Build system: install libimage-magick-perl instead of the perlmagick ↵intrigeri2019-12-311-1/+1
| | | | | | | | | | | | transitional package This package was renamed back in Stretch (and possibly earlier).
* | Build system: don't install libyaml-perl (obsolete dependency)intrigeri2019-12-311-1/+0
| | | | | | | | | | This was needed back when we built our own ikiwiki from Git… a looong time ago.
* | Build system: don't manually install libyaml-libyaml-perl (ikiwiki dependency)intrigeri2019-12-311-1/+1
| | | | | | | | We install ikiwiki from Debian and it depends on this package already.
* | Build system: don't install libxml-simple-perl (ikiwiki build-dependency)intrigeri2019-12-311-1/+0
| | | | | | | | | | This was needed back when we built our own ikiwiki from Git… a looong time ago.
* | Build system: don't manually install libhtml-parser-perl nor liburi-perl ↵intrigeri2019-12-311-1/+0
| | | | | | | | | | | | (ikiwiki dependencies) We install ikiwiki from Debian and it depends on these packages already.
* | Build system: don't manually install libhtml-scrubber-perl nor ↵intrigeri2019-12-311-2/+0
| | | | | | | | | | | | libhtml-template-perl (ikiwiki dependencies) We install ikiwiki from Debian and it depends on these packages already.
* | Build system: don't install libtext-multimarkdown-perl (unused dependency)intrigeri2019-12-311-1/+0
| | | | | | | | | | We've never enabled the multimarkdown ikiwiki option and I doubt we ever will.
* | Build system: don't install libfile-chdir-perl (obsolete dependency)intrigeri2019-12-311-1/+0
| | | | | | | | | | This was needed back when we built our own ikiwiki from Git… a looong time ago.
* | Replace all new occurrences of "wiki" with "website" (refs: #15342)intrigeri2019-12-311-8/+8
| | | | | | | | | | Our website has not exactly been a wiki for many years now, and thus we've been slowly moving from this terminology to "website".
* | Merge remote-tracking branch 'origin/stable' into feature/15342-cache-wikiintrigeri2019-12-3110-52/+123
|\ \ | |/
| * Vagrant: install po4a from Stretch in the basebox (refs: #16868)intrigeri2019-10-202-2/+8
| | | | | | | | | | | | We install it from our builder-jessie (sic) APT suite. Given we use time-based APT snapshots even for our own overlay APT suites, this requires bumping the corresponding snapshot to a version that includes the desired version of po4a.
| * build-tails: wait for NTP to be disabled before setting the desired date ↵intrigeri2019-10-191-0/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | (refs: #16868) "timedatectl set-ntp false" merely queues up a job: it does not wait until NTP is actually disabled. So far we did not notice that this was racing against us setting a custom date, but systemd 241 it becomes obvious: systemd does not allow other operations while one did not finish anymore. Disabling NTP services can take a bit of time, so with a Buster Vagrant box, when $TAILS_DATE_OFFSET is set (which we do for our reproducibly_build_Tails_ISO_* Jenkins jobs), the build would fail like this: + as_root_do timedatectl set-ntp false + sudo http_proxy=http://192.168.122.10:3142 APT_SNAPSHOTS_SERIALS={"torproject":"2019100904","debian":"2019100904","debian-security":"2019101901"} TAILS_MERGE_BASE_BRANCH=1 GIT_COMMIT=844ef7072f5856be50635aba45d6ebd820e70205 GIT_REF=feature/16868-buster-vagrant-box+force-all-tests BASE_BRANCH_GIT_COMMIT=b2559c50ba6c3f839da89d475964160889f8fbad timedatectl set-ntp false + date --utc --date=+8 days +%F %T Setting system time to 2019-10-27 09:52:54 + DESIRED_DATE=2019-10-27 09:52:54 + echo Setting system time to 2019-10-27 09:52:54 + as_root_do timedatectl set-time 2019-10-27 09:52:54 + sudo http_proxy=http://192.168.122.10:3142 APT_SNAPSHOTS_SERIALS={"torproject":"2019100904","debian":"2019100904","debian-security":"2019101901"} TAILS_MERGE_BASE_BRANCH=1 GIT_COMMIT=844ef7072f5856be50635aba45d6ebd820e70205 GIT_REF=feature/16868-buster-vagrant-box+force-all-tests BASE_BRANCH_GIT_COMMIT=b2559c50ba6c3f839da89d475964160889f8fbad timedatectl set-time 2019-10-27 09:52:54 Failed to set time: Previous request is not finished, refusing. So let's wait for the NTP service to be disabled before using timedatectl to configure stuff again.