wiki:OpenITPReview/Criteria

Version 18 (modified by zzz, 6 years ago) (diff)

adversaries/audience

Peer Review Board Selection Criteria

The questions below are taken directly from the draft criteria pages linked on http://wiki.openitp.org/peerreviewboard:start

Initial Filtering

Criterion Our Answer
Is it open source? Yes
Is it relevant to our field? Should be (see below)
Is it a circumvention tool? Yes
Is it a host security tool? No?
Is it a secure communications tool? Yes
Is it aimed at a high-risk population? Yes?
Is it an evidence gathering tool for human rights? No
Is it a disaster response or democracy facilitation tool? No
Is it an end-user or middle-tier tool? End-user(?)
Do we have a coverage gap that this tool fits into? ?
Is this tool real or just vaporware? Real
Is this tool shipping? Yes
Is this tool in production? Yes
Does this tool have actual users? Yes
Does this tool fill a currently unmet need? Yes - a decentralised low-latency anonymous network
Does this have unique advantages such as usability or localization? Yes? - Built in eepsite webserver?
Who nominated this tool? Us
Does your community have the resources to audit this tool independently? No
Does your project have funding for the audit? No - most of our funds are in volatile BTC and at this time are insufficient to fund an extensive audit.
Has this project been audited in the past, either by us or anyone else? No

Project Selection

Threat Model

Criterion Our response Do we fulfil this?
Does the tool have a threat model? Yes (Needs reviewing) - http://www.i2p2.i2p/how_threatmodel Yes
Does the tool have one or more clearly defined use contexts? Maybe? I2P is an anonymous overlay network, and has many potential use contexts. ?
Does the threat model follow a clearly established methodology? TODO Needs research
Is the threat model formally specified? Define "formal" Probably not formal enough for them

User Experience

Criterion Our response Do we fulfil this?
Does the user experience compromise the secure use of the tool? No?
Has there been a professional designer involved in the tool development process? Anonymous designers have donated their time and effort. No money has been paid towards improving the design of the I2P software UI.
Has there been user experience testing involved in the design process and if so, what? Sampling of the opinions of users on IRC (a very small percentage of the estimated userbase). No
Is their a style guide or set of design guidelines for the tool? No

Documentation

Criterion Our response Do we fulfil this?
Is the tool documentation sufficient to guide the intended audience through using the tool properly?
Is the documentation translated into the same set of languages as the tool? A small subset of languages, and not completely. No
Is the documentation up to date, regularly maintained, and accurate? Not entirely; not as often as it should be; reasonably. No
Does the documentation correctly describe the security caveats and use cases of the tool? Yes
Does the documentation make clear security claims, and are those claims supported by the tool? Yes? - We need a review to find out ;)
For tools intended for end-users, is there a set of introductory documentation for inexperienced users? No

Audience and Adversary Definition

Criterion Our response Do we fulfil this?
Is the tool actively designed with the needs of at-risk users in mind? Yes Yes
Does the project understand who their users are? Yes, in general, however specifics are difficult to gather due to the nature of anonymity tools. Yes
Does the project understand who their user's adversaries are? Yes, in general, however specifics are difficult to gather. Yes
Is the tool actively designed with their user's adversary's capabilities in mind? Yes, in general, however we could always do better. Yes
Is the tool being used for contexts outside of those that it was designed for? Probably. At least one user uses I2P as a dynamic DNS with NAT traversal capability. I2P is an open source tool that is modified and packaged by several third parties.
Was the tool designed with a realistic awareness of the needs of its intended user community? Yes, in general. See e.g. http://www.i2p2.de/_static/pdf/i2p_philosophy.pdf and old meeting logs. Yes

Development Process Transparency

Criterion Our response Do we fulfil this?
Does the project plan development in public fora (such as a mailing list or IRC channel)? Yes: IRC2P and freenode/#i2p-dev, http://zzz.i2p, http://lists.i2p2.i2p (primarily IRC and zzz.i2p) Yes
Does the project have an accurate roadmap that is up to date and has a history of use? http://trac.i2p2.i2p/wiki/Roadmaps/1.0 would be the most up-to-date; http://www.i2p2.i2p/roadmap and http://www.i2p2.i2p/todo also exist. None of these have a recent history of use. No
Does the project have a public bug tracker? Has the project used their bug tracker over time and kept it accurate? Yes to both. While we can't claim we have always kept it perfectly accurate, we have kept it current and accurate in recent years and plan to continue doing so. Yes
Does the project have a public source repository that is in use for mainline development? Monotone (see e.g. http://viewmtn.meeh.i2p/ and public repos mtn://mtn.i2p2.de and mtn://mtn.i2p-projekt.i2p); See also git mirror https://github.com/i2p/i2p.i2p Yes
Has the project added new developers or other volunteers over the life of the project? Yes Yes

Vulnerability Response Process Maturity and Transparency

Criterion Our response Do we fulfil this?
Does the project have documented criteria for determining what is a security issue? No
Does the project have a documented process for classifying the impact of security vulnerability reports? No TODO Define or set up Yes
Does the project have a documented response process for security vulnerability reports? Partial. Documented at http://zzz.i2p/topics/780 No
What is the project history of responding to security incidents and is it documented? Generally fixed in the next release. Release schedule is accelerated if necessary. Our typical release cycle is 6-10 weeks, or about 7 releases per year. History is documented at http://zzz.i2p/forums/13
Does the project have an internal responsible disclosure policy and is it used? Yes to both. Documented at http://zzz.i2p/topics/780 and we generally disclose issues in release notes, see http://www.i2p2.de/announcements - We also use the project news feed which is delivered to users in the tool, as well as other channels such as Twitter and http://forum.i2p/ Yes
What timeline does the project have around responding to vulnerabilities? Next release. Release schedule is accelerated if necessary. Our typical release cycle is 6-10 weeks, or about 7 releases per year.

Project License and IP Disposition

Criterion Our response Do we fulfil this?
Does the project have a license on the Open Source Initiative's list of free software licenses? Several - http://www.i2p2.i2p/licenses Yes
Is the project functionally open source such that the PRB could independently fix vulnerabilities without project team cooperation if it became necessary? This includes situations where, for example, a free software project depends on a proprietary library. Yes?
Is the project aware of any potential patent or copyright issues with the project that would limit distribution of any part of their project or interfere with the audit process? No

Privacy and Terms of Service Disposition

Criterion Our response Do we fulfil this?
To what degree does the project (as opposed to tool) come into contact with confidential information? The I2P network operates on is that there are no "trusted" routers/servers, so the project has no direct contact with confidential information. Some router statistics are publicly published to the netDB for diagnostic purposes, and users sometimes post potentially-deanonymizing information as part of bug reports. What about IPs, router IDs etc. in website logs from updates? - In-network update, also anyone can set up an update server Yes?
Does the project understand what data they gather about their users and what its privacy and security impacts are? TODO Check this
What do project policies permit the project to do with the data they gather? TODO Check this
What attitude toward users do any privacy or terms of service statements present? Not applicable?

Project Continuity

Criterion Our response Do we fulfil this?
Does the project have an active developer base? Small, but active. Yes
Does the project have a meaningful revenue or funding model sufficient to cover its costs in the long term? Donations cover server costs and provide for bounties; many services are run by volunteers. See http://www.i2p2.i2p/halloffame for revenue details. Yes?
Does the project have an accurate roadmap that is up to date and has a history of use? See Development Process Transparency above.
Does the project have a public bug tracker? Has the project used their bug tracker over time and kept it accurate? See Development Process Transparency above.
Does the project have a public source repository that is in use for mainline development? See Development Process Transparency above.
Is the project documentation up to date in all the languages the project supports? No, but the translation tagging of the website revamp will enable more accurate coverage. No
Does the project have a test framework? Both JUnit and ScalaTest Yes
Does the project have significant regression testing coverage? About 30% - see http://jenkins.killyourtv.i2p/job/cobertura/ No

Project Impact

Criterion Our response Do we fulfil this?
How large is the project's user base? Impossible to know for sure. Our best estimate is 20-25K daily unique users and about 30K monthly uniques. Yes?
Does this project benefit an at-risk population directly or indirectly? Directly Yes
Are there any alternatives for this functionality on the platforms it serves? Tor provides hidden services, but unmaintained and tangential to Tor's target functionality.
Is this tool recommended by trainers or others in the field? Unknown
How security-critical is the tool's functionality? Being an anonymous network, other tools are dependent on its security.
Is this project infrastructure that other tools depend on? Yes, e.g. eepsites, torrent software, http://nightweb.net Yes
What does the project's growth curve look like? Exponential - see http://stats.i2p/cgi-bin/total_routers_year.cgi ?
Is this tool localized for significant at-risk populations? We have translations for Arabic and Chinese (among others). Yes?
Is localization applied consistently? Localization of the routerconsole is mostly done via gettext. Inconsistencies do occur in the separate-page translations. Maybe?

Project Auditing Need

Criterion Our response Do we fulfil this?
Has the project been audited before, and if so how code base changed since the previous audit? No Yes?
Are their significant known security concerns or has the project had public exploit(s)? Future cryptographic weakness (see #856). No known public exploits of I2P itself (TODO Check this).
Is this project implicated in the harm of an at-risk population? No Yes
Is the project written in a high-risk language like C? No (Java) Yes
Is the project's development team relatively inexperienced, especially with security? The team has a wide range of technical training and experience, ranging from college students to those with 30+ years of engineering / programming background. No active developers have indicated having any formal security background or education. Yes?