Changes between Initial Version and Version 1 of OpenITPReview/Criteria


Ignore:
Timestamp:
May 8, 2013 9:00:03 AM (6 years ago)
Author:
str4d
Comment:

Legend:

Unmodified
Added
Removed
Modified
  • OpenITPReview/Criteria

    v1 v1  
     1= Peer Review Board Selection Criteria =
     2
     3The questions below are taken directly from the draft criteria pages linked on http://wiki.openitp.org/peerreviewboard:start
     4
     5== Initial Filtering ==
     6|| '''Criterion''' || '''Our Answer''' ||
     7|| Is it open source? || Yes ||
     8|| Is it relevant to our field? || Should be (see below) ||
     9|| ''Is it a circumvention tool?'' || Yes? ||
     10|| ''Is it a host security tool?'' || No? ||
     11|| ''Is it a secure communications tool?'' || Yes ||
     12|| ''Is it aimed at a high-risk population?'' || Yes? ||
     13|| ''Is it an evidence gathering tool for human rights?'' || No ||
     14|| ''Is it a disaster response or democracy facilitation tool?'' || No ||
     15|| ''Is it an end-user or middle-tier tool?'' || End-user(?) ||
     16|| Do we have a coverage gap that this tool fits into? || ? ||
     17|| Is this tool real or just vaporware? || Real ||
     18|| Is this tool shipping? || Yes ||
     19|| Is this tool in production? || Yes ||
     20|| Does this tool have actual users? || Yes ||
     21|| Does this tool fill a currently unmet need? || ? ||
     22|| Does this have unique advantages such as usability or localization? || ? ||
     23|| Who nominated this tool? || Us ||
     24|| Does your community have the resources to audit this tool independently? || No ||
     25|| Does your project have funding for the audit? || Possibly? ||
     26|| Has this project been audited in the past, either by us or anyone else? || No ||
     27
     28== Project Selection ==
     29
     30=== Threat Model ===
     31|| '''Criterion''' || '''Our response''' || '''Do we fulfil this?''' ||
     32|| Does the tool have a threat model?|| Yes - http://www.i2p2.i2p/threatmodel || Yes ||
     33|| Does the tool have one or more clearly defined use contexts? || Maybe? || ? ||
     34|| Does the threat model follow a clearly established methodology? ||  ||  ||
     35|| Is the threat model formally specified? || Define "formal" ||  ||
     36
     37=== User Experience ===
     38|| '''Criterion''' || '''Our response''' || '''Do we fulfil this?''' ||
     39|| Does the user experience compromise the secure use of the tool? ||  ||  ||
     40|| Has there been a professional designer involved in the tool development process? ||  ||  ||
     41|| Has there been user experience testing involved in the design process and if so, what? ||  ||  ||
     42|| Is their a style guide or set of design guidelines for the tool? ||  ||  ||
     43
     44=== Documentation ===
     45|| '''Criterion''' || '''Our response''' || '''Do we fulfil this?''' ||
     46|| Is the tool documentation sufficient to guide the intended audience through using the tool properly? ||  ||  ||
     47|| Is the documentation into the same set of languages as the tool? ||  ||  ||
     48|| Is the documentation up to date, regularly maintained, and accurate? ||  ||  ||
     49|| Does the documentation correctly describe the security caveats and use cases of the tool? ||  ||  ||
     50|| Does the documentation make clear security claims, and are those claims supported by the tool? ||  ||  ||
     51|| For tools intended for end-users, is there a set of introductory documentation for inexperienced users? ||  ||  ||
     52
     53=== Audience and Adversary Definition ===
     54|| '''Criterion''' || '''Our response''' || '''Do we fulfil this?''' ||
     55|| Is the tool actively designed with the needs of at-risk users in mind? ||  ||  ||
     56|| Does the project understand who their users are? ||  ||  ||
     57|| Does the project understand who their user's adversaries are? ||  ||  ||
     58|| Is the tool actively designed with their user's adversary's capabilities in mind? ||  ||  ||
     59|| Is the tool being used for contexts outside of those that it was designed for? ||  ||  ||
     60|| Was the tool designed with a realistic awareness of the needs of its intended user community? ||  ||  ||
     61
     62=== Development Process Transparency ===
     63|| '''Criterion''' || '''Our response''' || '''Do we fulfil this?''' ||
     64|| Does the project plan development in public fora (such as a mailing list or IRC channel)? || Yes: IRC2P/#i2p-dev, http://zzz.i2p, http://lists.i2p2.i2p || Yes ||
     65|| Does the project have an accurate roadmap that is up to date and has a history of use? ||  || No ||
     66|| Does the project have a public bug tracker? Has the project used their bug tracker over time and kept it accurate? || Yes - here (though not always entirely current due to small developer base) || Yes ||
     67|| Does the project have a public source repository that is in use for mainline development? ||  ||  ||
     68|| Has the project added new developers or other volunteers over the life of the project? || Yes, though only a few long-term developers. || Yes ||
     69
     70=== Vulnerability Response Process Maturity and Transparency ===
     71|| '''Criterion''' || '''Our response''' || '''Do we fulfil this?''' ||
     72|| Does the project have documented criteria for determining what is a security issue? || '''TODO: Check this''' ||  ||
     73|| Does the project have a documented process for classifying the impact of security vulnerability reports? ||  ||  ||
     74|| Does the project have a documented response process for security vulnerability reports? ||  ||  ||
     75|| What is the project history of responding to security incidents and is it documented? ||  ||  ||
     76|| Does the project have an internal responsible disclosure policy and is it used? ||  ||  ||
     77|| What timeline does the project have around responding to vulnerabilities? ||  ||  ||
     78
     79=== Project License and IP Disposition ===
     80|| '''Criterion''' || '''Our response''' || '''Do we fulfil this?''' ||
     81|| Does the project have a license on the Open Source Initiative's list of free software licenses? || Several - http://www.i2p2.i2p/licenses || Yes ||
     82|| Is the project functionally open source such that the PRB could independently fix vulnerabilities without project team cooperation if it became necessary? This includes situations where, for example, a free software project depends on a proprietary library. ||  || Yes? ||
     83|| Is the project aware of any potential patent or copyright issues with the project that would limit distribution of any part of their project or interfere with the audit process? || No ||  ||
     84
     85=== Privacy and Terms of Service Disposition ===
     86|| '''Criterion''' || '''Our response''' || '''Do we fulfil this?''' ||
     87|| To what degree does the project (as opposed to tool) come into contact with confidential information? || Some router statistics are publicly published to the netDB for diagnostic purposes; '''what about IPs, router IDs etc. in website logs from updates?''' ||  ||
     88|| Does the project understand what data they gather about their users and what its privacy and security impacts are? || '''TODO: Check this''' ||  ||
     89|| What do project policies permit the project to do with the data they gather? || '''TODO: Check this''' ||  ||
     90|| What attitude toward users do any privacy or terms of service statements present? || Not applicable? ||  ||
     91
     92=== Project Continuity ===
     93|| '''Criterion''' || '''Our response''' || '''Do we fulfil this?''' ||
     94|| Does the project have an active developer base? || Small, but active. ||  ||
     95|| Does the project have a meaningful revenue or funding model sufficient to cover its costs in the long term? || Donations cover server costs and provide for bounties; many services are run by volunteers. ||  ||
     96|| Does the project have an accurate roadmap that is up to date and has a history of use? || See above. ||  ||
     97|| Does the project have a public bug tracker? Has the project used their bug tracker over time and kept it accurate? || See above. ||  ||
     98|| Does the project have a public source repository that is in use for mainline development? || Monotone || Yes ||
     99|| Is the project documentation up to date in all the languages the project supports? || No, but the translation tagging of the website revamp will enable more accurate coverage. || No ||
     100|| Does the project have a test framework? || Both JUnit and ScalaTest || Yes ||
     101|| Does the project have significant regression testing coverage? ||  ||  ||
     102
     103=== Project Impact ===
     104|| '''Criterion''' || '''Our response''' || '''Do we fulfil this?''' ||
     105|| How large is the project's user base? || http://stats.i2p shows ~30,000 users ||  ||
     106|| Does this project benefit an at-risk population directly or indirectly? ||  ||  ||
     107|| Are there any alternatives for this functionality on the platforms it serves? || Tor provides hidden services, but unmaintained and tangential to Tor's target functionality. ||  ||
     108|| Is this tool recommended by trainers or others in the field? || Unknown ||  ||
     109|| How security-critical is the tool's functionality? ||  ||  ||
     110|| Is this project infrastructure that other tools depend on? ||  ||  ||
     111|| What does the project's growth curve look like? || Slow growth for the first 8 years, then large (exponential?) growth over the last few years. ||  ||
     112|| Is this tool localized for significant at-risk populations? ||  ||  ||
     113|| Is localization applied consistently? ||  ||  ||
     114
     115=== Project Auditing Need ===
     116|| '''Criterion''' || '''Our response''' || '''Do we fulfil this?''' ||
     117|| Has the project been audited before, and if so how code base changed since the previous audit? || No ||  ||
     118|| Are their significant known security concerns or has the project had public exploit(s)? || Nothing known? ||  ||
     119|| Is this project implicated in the harm of an at-risk population? || No ||  ||
     120|| Is the project written in a high-risk language like C? || Written in Java, so... no? ||  ||
     121|| Is the project's development team relatively inexperienced, especially with security? ||  ||  ||