summaryrefslogtreecommitdiffstats
path: root/wiki/src/blueprint/GitLab.mdwn
blob: 336f3587358114353eb335be2e16200cd71a2fb1 (plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
This is about migrating our data and workflow from Redmine to GitLab,
which is tracked as [[!tails_ticket 15878]].

[[!toc levels=3]]

# Issues tracking

See also the [GitLab doc on issues](https://docs.gitlab.com/ce/user/project/issues/).

## Private issues

One can make an issue
[confidential](https://docs.gitlab.com/ce/user/project/issues/confidential_issues.html)
when creating it; confidentiality can later be toggled on/off at any
time. A confidential issue is visible only by whoever created it and
by project members with at least
[Reporter](https://docs.gitlab.com/ce/user/permissions.html#project-members-permissions) access.

Given "Reporter" access is needed for lots of relatively basic
operations, such as labelling and assigning issues, presumably most
active Tails contributors would get at least this access level on the
`tails-team/tails` project, and then in turn access to this project's
confidential issues. So Tails teams that need to further restrict
access to their confidential issues will need to track their issues
under their own `tails-team/$project`.

## Relationships

On Redmine we heavily use relationships between fine-grained issues:
parent/subtasks, Related, Blocks, etc. There's no such thing in
GitLab FOSS ("CE") edition ([feature
request](https://gitlab.com/gitlab-org/gitlab-foss/issues/64186)) so
we'll need to significantly change our workflow here.

Below we describe potential solutions.

### Parent/subtask and Blocks relationship
 
A GitLab issue can have a list of tasks (checklist).
GitLab displays "X of Y tasks completed" prominently.
 
Describing each subtask in free-form text, directly as an item on this
list of tasks, should work fine to emulate a set of subtasks that are
all assigned to the same person. For example, see [[!gnome_gitlab
nautilus/issues/667]].

For more complex cases, e.g. when non-trivial subtasks are done by
different people (GitLab CE does not support multiple assignees per
issue), each subtask could be an item on this list of tasks, that
links to another issue. The downside is that after resolving one of
the subtasks, one will also need to mark the corresponding item as
completed on the task list of the "parent" issue.

This should also replace most of our current usage of the Blocks
relationship.

Alternatively, one can add a comment with a list of the blocker
issues. Comments can be updated and links will indicate which of the
blockers are closed (strike-through).

### Related to

We can write this information directly either in the description of
one of the issues that are related to each other or in a comment.
Either way, this adds a message in the Activity stream of the
referenced issue, with a link to the other issue. Granted, on issues
with one single long discussion thread, as we're used to on Redmine,
such messages will be lost, so better cross-reference issues in the
description of each related issue, or use labels, or get used to
"Start Discussion" to separate multiple sub-threads that can be marked
as resolved independently from each other.

### Duplicates

We can close duplicates with a comment that references the duplicated
issue. It adds a message to the Activity stream of the referenced
issue, which allows one to find duplicates later on if needed.

And to ensure we can list issues that have really been resolved,
add a "Duplicate" label.

## Status

Each open issue must have one of these labels:

 - "1. To do" (previously: "Confirmed")
 - "2. Doing" ("In progress" was too vague: it could mean anything
   between "someone did the first 2% of the work 5 years ago" to "this is
   what I'm focused on today")
 - "3. Needs Validation"

… except issues that were just created and need to be triaged
(previously: "New").

This lends itself to issue boards with 4 columns: "1. To do", "2.
Doing", "3. To review", and "Closed".

Closing an issue means one of:

 - The fix or feature the issue is about was merged and will be in
   a future release (previously: "Fix committed" for the next release,
   "Resolved" for 4.0).

   To list these issues: closed issues whose milestone is a version
   was not released yet.

 - The fix or feature the issue is about is already available to
   our users (previously: "Resolved").

   To list these issues: closed issues whose milestone is a version
   that's been released already.

 - We've rejected it or marked it as a duplicate (previously:
   "Rejected" and "Duplicate")

   To list these issues: closed issues with respectively the "Rejected"
   or "Duplicate" label.

Most closed issues will still have the "3. To review" label.
That should not cause any problem in practice. Worst case this can be
fixed automatically, either via a webhook or a scheduled batch job.

## Other issues metadata

- Target version → Milestone
- Feature branch: GitLab will automatically link a branch that mentions
  an issue.
- Category, Affected Tool, Priority, Type of work → use a set of
  labels, each with a prefix, for each of them; and maybe simplify
  a bit. For example:
  - Qubes uses "P: $priority", "C: $category", etc.
  - One can set multiple labels so we could perhaps merge "Category"
    and "Affected Tool". For example, a ticket about Thunderbird
    persistence could have the two "C: email" and "C: persistence" labels.
- Log time: was enabled as an experiment, not actively used anymore;
  contributors can use GitLab's Time tracking if they wish so.
- Due date → Due date
- Starter → dedicated label
- Tracker: drop it (we've never really taken advantage of the fine
  distinction between describing a problem as a bug vs. describing
  its solution as a feature)
- % Done: drop it (we don't use this field enough to provide any value)
- Estimated time → description
- Attachments: any issue/MR description or comment can have file
  attachments

## Project management

Currently we use the Redmine "Deliverable for" custom field.
A "Deliverable for SponsorX" label should do the job for tracking
global progress on a grant: then one can use the issues list or an
issues board to visualize progress (despite the doc saying otherwise
while I'm writing this, one can now have
[multiple issue boards](https://docs.gitlab.com/ee/user/project/issue_board.html#multiple-issue-boards-starter)
in the community edition of GitLab).

An issues list can be ordered by due date, milestone due date, etc.,
which should emulate the most important aspects of the Redmine custom
queries we use, except naming the query (see "Custom queries" below).

We can track progress on a specific project (be it a grant deliverable
or not) by adding another label, e.g. "Project XYZ". For example,
<https://redmine.tails.boum.org/code/projects/tails/issues?query_id=298>
lists issues that have "Deliverable for" = "SponsorX" and whose parent
task contains [[!tails_ticket 14568]]. We would have labels
"Deliverable for SponsorX" and "Project XYZ". And if an additional
intermediary level of tracking is needed between "this is part of
SponsorX" and "this is about Additional Software" we can create
tracking issues that each have list of tasks.

## Personal self-management

For planning, the same solutions as for project management apply,
modulo adding a filter for the assignee.

And for more lightweight tracking, GitLab's Todos are great. Todos are
fully orthogonal to "tickets assigned to me"; they are listed on
a different page and can be marked as done with one single click.
For example:

 - Every time one mentions me somewhere, I get a Todo item.
   This allows me to track the questions I've been asked, that is, to
   replace our past usage of "Info Needed" and current usage of
   `@nick` "mentions", without the need to reassign an issue nor to
   create a subtask. As a bonus, for many of us, a list of Todos
   made prominent by GitLab will be easier to keep track of than
   mere email notifications.

 - I can click "Add todo" on an issue and it will appear on my list
   of Todos (regardless of whether I'm the assignee or not).

## Core team (self-)management

XXX: how to replace e.g.
<https://redmine.tails.boum.org/code/projects/tails/issues?query_id=307>?

Use a dedicated set of labels?

## Custom queries

We use Redmine custom queries to have easy access to named searches
and visualizations that we either often need, or that we want to share
within a team or the overall Tails project.

In GitLab, the closest equivalent to Redmine custom queries is the URL
of an issues list or issues board filtered by assignee, milestone,
and/or label.

We will need ways to share these URLs and ideally, to name them.
We can do that on the GitLab project's description (home page),
on parent tracking tickets, on blueprints, on a team's
page on our website, and possibly in GitLab's own
[wiki](https://docs.gitlab.com/ce/user/project/wiki/) if we decide to
use it (either only for this use case or [[!tails_ticket 9174
desc="for our blueprints"]]).

# Merge requests

The [Protected branch
flow](https://docs.gitlab.com/ce/user/project/merge_requests/authorization_for_merge_requests.html#protected-branch-flow)
is probably the most adequate for regular contributors, since it's the
least disruptive in terms of workflow and habits and requires less
work to adjust our Jenkins CI setup:

 - We mark our main branches (`stable`, `testing`, `devel`,
   `feature/buster`) as "Protected". We give "Maintainer" access to
   people allowed to push to these branches, i.e. what we previously
   called "commit access".
 - The Git workflow remains unchanged for regular developers who are
   not granted full commit access: they get "Developer" access, can
   push a topic branch to the canonical Git repository and our CI will
   pick it up. The only difference is that they are not restricted to
   pushing to their own `$nickname/*` namespace, which makes things
   simpler and has other advantages, e.g. they can use the `wip/`
   prefix (so our Jenkins CI ignores the branch) and collaborate with
   others on shared branches.
 - Other contributors get access strictly lower than "Developer".
   They push topic branches to their own fork of the repository and
   create merge requests.
 - Our current Jenkins CI jobs generation process remains unchanged.
   (Technically, we could adjust it so it generates CI jobs for _any_
   merge request (based on `refs/merge-requests/*/head`), but this
   would give arbitrary code execution on our CI to _anyone_.
   Our infrastructure is not currently equipped to cope with this.)

# Wiki

It's out of scope for the first iteration but at some point, we might
want to migrate our blueprints to GitLab's
[wiki](https://docs.gitlab.com/ce/user/project/wiki/):
[[!tails_ticket 9174]]).


# Other specification

* guest accounts?

# Importing data from Redmine

## Specification

Note: some of what follows still needs to be negotiated with the
Tails community.

### Scope

* issues in the [Tails Redmine
  project](https://redmine.tails.boum.org/code/projects/tails/issues);
  data from other projects does not matter to us

### MUST

* For every existing Redmine issue, the meaning of `#NNNN` MUST be
  preserved unambiguously. (This probably implies that we can't have
  a GitLab issue with the same ID.)

* [[!tails_ticket NNNN]] and
  <https://redmine.tails.boum.org/code/issues/NNNN> links MUST point
  or — possibly transitively, via a placeholder issue — redirect to the
  relevant GitLab issue.

* Preserve issue private/public status. Given GitLab's permissions
  model, this implies that some issues MUST be migrated to another
  GitLab sub-project than the default one; for details, see the
  "Private issues" section above.

* Preserve parent/subtask relationship in a way that GitLab will
  enforce, i.e. prevent closing the GitLab issue corresponding to the
  Redmine parent issue, as long as one of its subtasks is
  not completed.

  Implementation idea: a GitLab issue imported from a Redmine issue
  that has subtasks could have a list of tasks (checklist), with each
  subtask being an item on that list; ideally, the status of each such
  Redmine subtask (closed or open) should be reflected as the status
  of the corresponding GitLab checklist item (completed or not).

* Preserve the "Blocks" and "Related to" information.

  It's OK if the "Blocks" semantics from Redmine (if X blocks Y, one
  cannot close Y until X is resolved) is not enforced by GitLab.

  Implementation idea: add the "X blocks Y" information in the
  description of issues X and Y and/or as comments on X and Y.

* Preserve the "Related to" relationship.

* Preserve issue status:

  - "Confirmed" → open, "1. To do" label
  - "In Progress" → open, "2. Doing" label
  - "Needs Validation" → open, "3. Needs Validation" label
  - "New" → open
  - "Duplicates" → closed, "Duplicate" label
  - "Rejected" → closed, "Rejected" label
  - "Fix committed": assume this status does not exist on Redmine anymore

* Preserve "Target version" → Milestone

* Preserve the  "Feature branch" information.

  Adding this information as-is in a comment would be good enough.

* Preserve Category, Affected Tool, Priority, Type of work: XXX
  (probably with labels, see "Other issues metadata" section above)

* Preserve any true "Starter" boolean value.

  Implementation idea: dedicated label.

* Preserve "Deliverable for" values for open issues.

  Implementation idea: "Deliverable for SponsorXYZ" label.

* Preserve attachments

* Convert Textile issue description and comments to Markdown
  (pandoc can do most of the work see <https://pandoc.org/try/?from=textile&to=markdown>)
  ([[!tails_ticket 17241 desc="examples"]]):
  - `commit:c6f4d0fd`
  - bold
  - italic
  - strike-through
  - `<pre>` blocks
  - `@code@`
  - bullet lists, including nested ones

* Convert public notes to GitLab comments.

* Private comments (notes): migrate their contents somewhere,
  and make it accessible from the corresponding issue,
  in the right place in the discussion.

  It looks like GitLab does not support private comments.

### SHOULD

* Convert more Textile issue description and comments to Markdown
  ([[!tails_ticket 17241 desc="examples"]]):
  - links to an issue note: `#124-6`, or `#124#note-6`
  - links with specific text: `"Redmine web site":http://www.redmine.org`
  - tables, if supported by GitLab
  - h1 and h2 headings
  - `source:some/file`

* Preserve watchers → participants.

### Information that can be lost during the migration

See "Other issues metadata" section above for some discussion about why.

* Log time
* Due date
* Tracker
* % Done
* Estimated time

### To be triaged

Relevant features — to be triaged as MUST/SHOULD/MAY:

* Does not require global GitLab administration token.
* Does not require SSH access to the GitLab machine.

## Tools

A number of tools are available online.

These data migration tools come in various shape and some don't
support GitLab API v4, but generally there's a fork on GitHub that
fixes the most critical problems. Start there, explore the network
of forks, and follow the white GitHub rabbit(-hole):

* <https://github.com/redmine-gitlab-migrator/redmine-gitlab-migrator>
* <https://github.com/ultreia-io/migrate-redmine-to-gitlab>

# Issues and merge requests triaging

* One can define policies (YAML) and have them applied automatically.
  - <https://gitlab.com/gitlab-org/quality/triage-ops>
  - <https://gitlab.com/gitlab-org/gitlab-triage>

# Resources

* Tor's [migration plan](https://nc.riseup.net/s/SnQy3yMJewRBwA7)
  from Trac to GitLab.
  - Rough timeline defined on 2019-10-15:
    1. Have a full migration into legacy in Mid November
    2. Ask people to find errors the last two weeks of November
    3. Migration early december
  - They are considering using GitLab EE Ultimate, which includes
    proprietary components.
  - Meetings:
    - [agenda and notes](https://pad.riseup.net/p/e-q1GP43W4gsY_tYUNxf)
    - 2019-10-15 meeting
      [logs](http://meetbot.debian.net/tor-meeting/2019/tor-meeting.2019-10-15-17.01.html)
    - 2019-10-01 meeting
      [logs](http://meetbot.debian.net/tor-meeting/2019/tor-meeting.2019-10-01-18.00.html)
* KDE migration to Gitlab:
  - <https://gitlab.com/gitlab-org/gitlab-foss/issues/57338/designs>
  - <https://gitlab.com/gitlab-org/gitlab/issues/24900>