Collect and analyze download statistics

[str4d]

We already indirectly collect download statistics in a variety of forms, but rarely look at most of them. Properly collecting and analyzing them (possibly with anonymizing/noise) would provide some insight into network size and install/update paths, as well as potentially (rough) OS stats that could help to inform future development.

Steps for completion:

1. Identify all possible sources of download statistics
2. Identify ways to collect these statistics (with anonymization if necessary)
3. Write scripts to pull (or push) these statistics
4. Write a server / Jenkins plugin / other means of collecting, analyzing and presenting these statistics

[str4d]

Add a subticket #1120.

[hodgepodge]

If someone could point me to where we are already aggregating download statistics, I'd love to pick up this ticket. I've been working on a similar project for a few months now that's geared towards aggregating, and summarizing statistics collected on Maven builds in a distributed way.

Which types of nodes are we currently collecting metrics from? If we can collect anonymous metrics from the reseed servers, that might be a good place to start.

[str4d]

We are not currently directly collecting or aggregating metrics anywhere - that is what this ticket is for. But we do have a bunch of sources that we can leverage (step 1).

To contribute to step 1, here are all the sources of download stats I can think of:

• Website server logs (specifically for the download re-director)
  • Installers, updates, Android
  • Can distinguish between Windows and non-Windows via URLs
  • Maybe can distinguish further via User Agents
• Download server logs
  • Should theoretically be a superset of the website server logs
• ​http://tracker2.postman.i2p/
  • General update stats
• kytv's Debian repo
• Launchpad
  • Indication of Debian / Ubuntu users
• Google Code (at least historically)
  • IIRC this once showed a large Windows component
• Google Play
  • For Android stats, already aggregate
• Main F-Droid repo
  • I2P Android
  • Download numbers were recently removed as misleading, they may create a replacement
• Our F-Droid repo
  • Android stats
• GitHub
  • i2pd stats?
• Reseed servers
  • If users only bootstrap on first start, these stats should roughly equal (after de-duplication) the installer download stats. In reality these stats will be higher.