summaryrefslogtreecommitdiffstats
path: root/wiki/src/blueprint/better_metrics.mdwn
blob: c3364cf5430740b6fcb83e6ebb71df01ce8358e1 (plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
[[!meta title="Better metrics"]]

## Current status

- Metrics are collected:
  - Manually
  - Only by sajolida and intrigeri
  - Into text files
  - In the internal Git repo

- We are collecting:
  - Community metrics
    - Subscribers to amnesia-news
    - Messages on tails-dev
    - Subscribers on tails-dev
    - Twitter followers
  - Help desk
    - Messages on tails-bug
    - WhisperBack reports
  - Releases
    - Date
    - Size
  - Usage
    - Boots
    - Downloads of the OpenPGP signature

## Concerns

- The workload is not shared
- The data is not systematically published (some are in the monthly reports)
- Manual process

## Taking a step back...

- Why are we collecting these metrics?
- What shall we do with them?
- What metrics are most important to collect?
- What kind of tools and infrastructure shall we use to manage these metrics?
- Where do we store them? Who has access to them?

## Ideas of abstract goals behind the numbers

From a brainstorming session, with no priorities or check on whether
these are possible and worth it to gather.

- Find how many users we do have? What are the trends?
- How many bugs? Per category? Are we carrying around old bugs? Are we adding new bugs?
- Are we reducing errors for users ?
- Are we reducing the time needed for task completion?
- Help desk reports
- How technically skilled do you need to be to use Tails? How technically skilled are our users?
- Donations
  - How many donations do we get?
  - How much are people donation?
  - When do we get donation?
- Incoming links? Who is endorsing Tails?
- Where are the users located?
- How big, diverse and active is our community?
- Statistics on private and public meetings:
  - How many people attended?
  - How many meetings?
  - How many newcomers?
- Conferences
  - Attended?
  - How many mentions?
- How fast to we solve problems?
  - Security issues
- How organized and accountable we are?
- Conversion rate
  - User agents
- Heat map of the website usage
  - Are people coming to download the image?
  - Are people coming to read the documentation?
- Why are people interested in Tails?

## Possible ways to gather this data

- Ticket tracker for help desk
- Web analytic
- Surveys

## References

- [Translating UX Goals into Analytics Measurement Plans](https://www.nngroup.com/articles/ux-goals-analytics/)