summaryrefslogtreecommitdiffstats
path: root/wiki/src/blueprint/GitLab.mdwn
blob: 1b1468503e6f181ddfa311d16e68641724b39359 (plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
This is about migrating our data and workflow from Redmine to GitLab,
which is tracked as [[!tails_ticket 15878]].

[[!toc levels=3]]

# Issues tracking

See also the [GitLab doc on issues](https://docs.gitlab.com/ce/user/project/issues/).

## Private issues

One can make an issue
[confidential](https://docs.gitlab.com/ce/user/project/issues/confidential_issues.html)
when creating it; confidentiality can later be toggled on/off at any
time. A confidential issue is visible only by whoever created it and
by project members with at least
[Reporter](https://docs.gitlab.com/ce/user/permissions.html#project-members-permissions) access.

Given "Reporter" access is needed for lots of relatively basic
operations, such as labelling and assigning issues, presumably most
active Tails contributors would get at least this access level on the
`tails-team/tails` project, and then in turn access to this project's
confidential issues. So Tails teams that need to further restrict
access to their confidential issues will need to track their issues
under their own `tails-team/$project`.

## Relationships

On Redmine we heavily use relationships between fine-grained issues:
parent/subtasks, Related, Blocks, etc. There's no such thing in
GitLab FOSS ("CE") edition ([feature
request](https://gitlab.com/gitlab-org/gitlab-foss/issues/64186)) so
we'll need to significantly change our workflow here.

Below we describe potential solutions.

### Parent/subtask and Blocks relationship
 
A GitLab issue can have a list of tasks (checklist).
GitLab displays "X of Y tasks completed" prominently.
 
Describing each subtask in free-form text, directly as an item on this
list of tasks, should work fine to emulate a set of subtasks that are
all assigned to the same person. For example, see [[!gnome_gitlab
nautilus/issues/667]].

For more complex cases, e.g. when non-trivial subtasks are done by
different people (GitLab CE does not support multiple assignees per
issue), each subtask could be an item on this list of tasks, that
links to another issue. The downside is that after resolving one of
the subtasks, one will also need to mark the corresponding item as
completed on the task list of the "parent" issue.

This should also replace most of our current usage of the Blocks
relationship.

Alternatively, one can add a comment with a list of the blocker
issues. Comments can be updated and links will indicate which of the
blockers are closed (strike-through).

### Related to

We can write this information directly either in the description of
one of the issues that are related to each other or in a comment.
Either way, this adds a message in the Activity stream of the
referenced issue, with a link to the other issue. Granted, on issues
with one single long discussion thread, as we're used to on Redmine,
such messages will be lost, so better cross-reference issues in the
description of each related issue, or use labels, or get used to
"Start Discussion" to separate multiple sub-threads that can be marked
as resolved independently from each other.

### Duplicates

We can close duplicates with a comment that references the duplicated
issue. It adds a message to the Activity stream of the referenced
issue, which allows one to find duplicates later on if needed.

And to ensure we can list issues that have really been resolved,
add a "Duplicate" label.

## Status

Each open issue must have one of these labels:

 - "1. To do" (previously: "Confirmed")
 - "2. Doing" ("In progress" was too vague: it could mean anything
   between "someone did the first 2% of the work 5 years ago" to "this is
   what I'm focused on today")
 - "3. Needs Validation"

… except issues that were just created and need to be triaged
(previously: "New").

This lends itself to issue boards with 4 columns: "1. To do", "2.
Doing", "3. To review", and "Closed".

Closing an issue means one of:

 - The fix or feature the issue is about was merged and will be in
   a future release.

   To list these issues: closed issues whose milestone is a version
   was not released yet.

 - The fix or feature the issue is about is already available to
   our users (previously: "Resolved").

   To list these issues: closed issues whose milestone is a version
   that's been released already.

 - We've rejected it or marked it as a duplicate (previously:
   "Rejected" and "Duplicate")

   To list these issues: closed issues with respectively the "Rejected"
   or "Duplicate" label.

Most closed issues will still have the "3. To review" label.
That should not cause any problem in practice. Worst case this can be
fixed automatically, either via a webhook or a scheduled batch job.

## Other issues metadata

- Target version → Milestone
- Feature branch: GitLab will automatically link a branch that mentions
  an issue.
- Category, Affected Tool, Priority, Type of work → use a set of
  labels, each with a prefix, for each of them; and maybe simplify
  a bit. For example:
  - Qubes uses "P: $priority", "C: $category", etc.
  - One can set multiple labels so we could perhaps merge "Category"
    and "Affected Tool". For example, a ticket about Thunderbird
    persistence could have the two "C: email" and "C: persistence" labels.
- Log time: was enabled as an experiment, not actively used anymore;
  contributors can use GitLab's Time tracking if they wish so.
- Due date → Due date
- Starter → dedicated label
- Tracker: drop it (we've never really taken advantage of the fine
  distinction between describing a problem as a bug vs. describing
  its solution as a feature)
- % Done: drop it (we don't use this field enough to provide any value)
- Estimated time → description
- Attachments: any issue/MR description or comment can have file
  attachments

## Project management

Currently we use the Redmine "Deliverable for" custom field.
A "Deliverable for SponsorX" label should do the job for tracking
global progress on a grant: then one can use the issues list or an
issues board to visualize progress (despite the doc saying otherwise
while I'm writing this, one can now have
[multiple issue boards](https://docs.gitlab.com/ee/user/project/issue_board.html#multiple-issue-boards-starter)
in the community edition of GitLab).

An issues list can be ordered by due date, milestone due date, etc.,
which should emulate the most important aspects of the Redmine custom
queries we use, except naming the query (see "Custom queries" below).

We can track progress on a specific project (be it a grant deliverable
or not) by adding another label, e.g. "Project XYZ". For example,
<https://redmine.tails.boum.org/code/projects/tails/issues?query_id=298>
lists issues that have "Deliverable for" = "SponsorX" and whose parent
task contains [[!tails_ticket 14568]]. We would have labels
"Deliverable for SponsorX" and "Project XYZ". And if an additional
intermediary level of tracking is needed between "this is part of
SponsorX" and "this is about Additional Software" we can create
tracking issues that each have list of tasks.

## Personal self-management

For planning, the same solutions as for project management apply,
modulo adding a filter for the assignee.

And for more lightweight tracking, GitLab's Todos are great. Todos are
fully orthogonal to "tickets assigned to me"; they are listed on
a different page and can be marked as done with one single click.
For example:

 - Every time one mentions me somewhere, I get a Todo item.
   This allows me to track the questions I've been asked, that is, to
   replace our past usage of "Info Needed" and current usage of
   `@nick` "mentions", without the need to reassign an issue nor to
   create a subtask. As a bonus, for many of us, a list of Todos
   made prominent by GitLab will be easier to keep track of than
   mere email notifications.

 - I can click "Add todo" on an issue and it will appear on my list
   of Todos (regardless of whether I'm the assignee or not).

## Core team (self-)management

XXX: how to replace e.g.
<https://redmine.tails.boum.org/code/projects/tails/issues?query_id=307>?

Use a dedicated set of labels?

## Custom queries

We use Redmine custom queries to have easy access to named searches
and visualizations that we either often need, or that we want to share
within a team or the overall Tails project.

In GitLab, the closest equivalent to Redmine custom queries is the URL
of an issues list or issues board filtered by assignee, milestone,
and/or label.

We will need ways to share these URLs and ideally, to name them.
We can do that on the GitLab project's description (home page),
on parent tracking tickets, on blueprints, on a team's
page on our website, and possibly in GitLab's own
[wiki](https://docs.gitlab.com/ce/user/project/wiki/) if we decide to
use it (either only for this use case or [[!tails_ticket 9174
desc="for our blueprints"]]).

# Merge requests

The [Protected branch
flow](https://docs.gitlab.com/ce/user/project/merge_requests/authorization_for_merge_requests.html#protected-branch-flow)
is probably the most adequate for regular contributors, since it's the
least disruptive in terms of workflow and habits and requires less
work to adjust our Jenkins CI setup:

 - We mark our main branches (`stable`, `testing`, `devel`,
   `feature/buster`) as "Protected". We give "Maintainer" access to
   people allowed to push to these branches, i.e. what we previously
   called "commit access".
 - The Git workflow remains unchanged for regular developers who are
   not granted full commit access: they get "Developer" access, can
   push a topic branch to the canonical Git repository and our CI will
   pick it up. The only difference is that they are not restricted to
   pushing to their own `$nickname/*` namespace, which makes things
   simpler and has other advantages, e.g. they can use the `wip/`
   prefix (so our Jenkins CI ignores the branch) and collaborate with
   others on shared branches.
 - Other contributors get access strictly lower than "Developer".
   They push topic branches to their own fork of the repository and
   create merge requests.
 - Our current Jenkins CI jobs generation process remains unchanged.
   (Technically, we could adjust it so it generates CI jobs for _any_
   merge request (based on `refs/merge-requests/*/head`), but this
   would give arbitrary code execution on our CI to _anyone_.
   Our infrastructure is not currently equipped to cope with this.)

# Wiki

It's out of scope for the first iteration but at some point, we might
want to migrate our blueprints to GitLab's
[wiki](https://docs.gitlab.com/ce/user/project/wiki/):
[[!tails_ticket 9174]]).


# Other specification

* guest accounts?

# Importing data from Redmine

## Specification

Note: some of what follows still needs to be negotiated with the
Tails community.

### Scope

* issues in the [Tails Redmine
  project](https://redmine.tails.boum.org/code/projects/tails/issues);
  data from other projects does not matter to us
* target versions (milestones)

### Out of scope

* Redmine wiki: we don't use it

### MUST

* For every existing Redmine issue, the meaning of its `#NNNN` ID MUST
  be preserved unambiguously. This implies that we can't reuse
  existing Redmine issue IDs in our main GitLab project, where most
  issues will live.)

* [[!tails_ticket NNNN]] and
  <https://redmine.tails.boum.org/code/issues/NNNN> links MUST point
  or — possibly transitively — redirect to the relevant GitLab issue.

  Implementation idea: start with a static web redirector, and then:

   - This is sufficient to solve the problem for issues imported into
     our main GitLab project, if they have the same ID as they used to.

   - For issues imported into another GitLab project, we either need
     extra redirection rules based on a mapping from Redmine issue IDs
     to GitLab (project, issue ID) tuples, or placeholder issues on
     GitLab that handle a second redirection step (GitLab might handle
     the redirection iself when an issue is moved from one project to
     another)

* Preserve issue private/public status. Given GitLab's permissions
  model, this implies that some issues MUST be migrated to another
  GitLab project than the default one where most of our issues will
  live; for details, see the "Private issues" section above.

  Example: [[!tails_ticket 10346]]

* Preserve parent/subtask relationship in a way that GitLab will
  enforce, i.e. prevent closing the GitLab issue corresponding to the
  Redmine parent issue, as long as one of its subtasks is
  not completed.

  Examples: [[!tails_ticket 15878]], [[!tails_ticket 12213]]

  Implementation idea: a GitLab issue imported from a Redmine issue
  that has subtasks could have a list of tasks (checklist), with each
  subtask being an item on that list; ideally, the status of each such
  Redmine subtask (closed or open) should be reflected as the status
  of the corresponding GitLab checklist item (completed or not).

* Preserve the "Blocks" and "Related to" information.

  It's OK if the "Blocks" semantics from Redmine (if X blocks Y, one
  cannot close Y until X is resolved) is not enforced by GitLab.

  Examples: [[!tails_ticket 15878]], [[!tails_ticket 12213]]

  Implementation idea: add the "X blocks Y" information in the
  description of issues X and Y and/or as comments on X and Y.

* Preserve issue status:

  - "New" → open
  - "Confirmed" → open, "1. To do" label
  - "In Progress" → open, "2. Doing" label
  - "Needs Validation" → open, "3. Needs Validation" label
  - "Duplicates" → closed, "Duplicate" label
  - "Rejected" → closed, "Rejected" label
  - "Resolved" → closed

* Preserve "Target version" → Milestone

* Preserve the  "Feature branch" information.

  Adding this information as-is in a comment would be good enough.

  Example: [[!tails_ticket 16173]]

* Preserve Category, Affected Tool, Priority, Type of work… somehow.

  Implementation idea: with labels, see the "Other issues metadata" section above.

* Preserve any true "Starter" boolean value.

  Implementation idea: dedicated label.

* Preserve "Deliverable for" values for open issues.

  Implementation idea: "Deliverable for SponsorXYZ" label.

* Preserve attachments

  Example: [[!tails_ticket 7763]]

* Convert Textile issue description and comments to Markdown
  (pandoc can do most of the work see <https://pandoc.org/try/?from=textile&to=markdown>):
  - `commit:c6f4d0fd`
  - bold
  - italic
  - strike-through
  - `<pre>` blocks
  - `@code@`
  - bullet lists, including nested ones

  Examples: [[!tails_ticket 17241]]

  Textile: <https://www.redmine.org/projects/redmine/wiki/RedmineTextFormatting>
  GitLab Flavored Markdown (GFM): <https://docs.gitlab.com/ee/user/markdown.html>

* Convert public notes to GitLab comments.

  Example: [[!tails_ticket 17121]]

* Private comments (notes): migrate their contents somewhere,
  and make it accessible from the corresponding issue,
  in the right place in the discussion.

  Example: [[!tails_ticket 16875#note-3]]

  It looks like GitLab does not support private comments.

* Preserve issue assignee

  It is acceptable to require assignees to manually create their
  GitLab user account themselves before the final migration.

* Existing issues assigned to a group must remain trackable, somehow.

  This feature is not intensively used. We can leave it up to the
  teams who use it to decide how they want to track these tickets on
  GitLab. A temporary label to not lose track of these tickets would
  be good enough as far as the migration is concerned.

  We have two such groups:
  - _Help Desk_: recently created, not used yet
  - _Sysadmins_: 15 tickets as of 2019-11-18

* Preserve the value of the _Blueprint_ custom field

  Adding this information to the issue description is good enough.

### SHOULD

* Convert more Textile issue description and comments to Markdown
  ([[!tails_ticket 17241 desc="examples"]]):
  - links to an issue note: `#124-6`, or `#124#note-6`
  - links with specific text: `"Redmine web site":http://www.redmine.org`
  - tables, if supported by GitLab
  - h1 and h2 headings
  - `source:some/file`

  Examples: [[!tails_ticket 17241]]

* Preserve watchers → participants.

  Example: [[!tails_ticket 16875]]

### Information that can be lost during the migration

See "Other issues metadata" section above for some discussion about why.

* Log time
* Start date
* Due date
* Tracker
* % Done
* Estimated time
* Code syntax highlighting

### To be triaged

Relevant features — to be triaged as MUST/SHOULD/MAY:

* Metadata changes in issue notes that also have free-form text.
* Issue notes that only change metadata.
* Does not require global GitLab administration token.
* Does not require SSH access to the GitLab machine.
* Which user:
  - created an issue
  - wrote a note
  - modified the metadata of an issue
* Existing user accounts: if it's OK to do all operations as a dedicated
  migration bot user (with free-form text saying who did it on Redmine),
  and if it's OK to ask active contributors to create their GitLab account
  themselves, maybe don't need to migrate user accounts.

## Tools

A number of tools are available online.

These data migration tools come in various shape and some don't
support GitLab API v4, but generally there's a fork on GitHub that
fixes the most critical problems. Start there, explore the network
of forks, and follow the white GitHub rabbit(-hole):

* <https://github.com/redmine-gitlab-migrator/redmine-gitlab-migrator>
* <https://github.com/ultreia-io/migrate-redmine-to-gitlab>

# Issues and merge requests triaging

* One can define policies (YAML) and have them applied automatically.
  - <https://gitlab.com/gitlab-org/quality/triage-ops>
  - <https://gitlab.com/gitlab-org/gitlab-triage>

# Resources

* Tor's [migration plan](https://nc.riseup.net/s/SnQy3yMJewRBwA7)
  from Trac to GitLab.
  - Rough timeline defined on 2019-10-15:
    1. Have a full migration into legacy in Mid November
    2. Ask people to find errors the last two weeks of November
    3. Migration early december
  - They are considering using GitLab EE Ultimate, which includes
    proprietary components.
  - Meetings:
    - [agenda and notes](https://pad.riseup.net/p/e-q1GP43W4gsY_tYUNxf)
    - 2019-10-15 meeting
      [logs](http://meetbot.debian.net/tor-meeting/2019/tor-meeting.2019-10-15-17.01.html)
    - 2019-10-01 meeting
      [logs](http://meetbot.debian.net/tor-meeting/2019/tor-meeting.2019-10-01-18.00.html)
* KDE migration to Gitlab:
  - <https://gitlab.com/gitlab-org/gitlab-foss/issues/57338/designs>
  - <https://gitlab.com/gitlab-org/gitlab/issues/24900>