Changes between Version 3 and Version 4 of OpenITPReview/Criteria


Ignore:
Timestamp:
May 8, 2013 12:34:39 PM (6 years ago)
Author:
str4d
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • OpenITPReview/Criteria

    v3 v4  
    3939|| Does the user experience compromise the secure use of the tool? ||  ||  ||
    4040|| Has there been a professional designer involved in the tool development process? || Anonymous designers have donated their time and effort. No money has been paid towards improving the design of the I2P software UI. ||  ||
    41 || Has there been user experience testing involved in the design process and if so, what? || Sampling of the opinions of users on IRC (a very small percentage of the estimated userbase). || No ||
    42 || Is their a style guide or set of design guidelines for the tool? ||  || No ||
     41|| Has there been user experience testing involved in the design process and if so, what? || Sampling of the opinions of users on IRC (a very small percentage of the estimated userbase). || '''No''' ||
     42|| Is their a style guide or set of design guidelines for the tool? ||  || '''No''' ||
    4343
    4444=== Documentation ===
    4545|| '''Criterion''' || '''Our response''' || '''Do we fulfil this?''' ||
    4646|| Is the tool documentation sufficient to guide the intended audience through using the tool properly? ||  ||  ||
    47 || Is the documentation into the same set of languages as the tool? ||  || No ||
    48 || Is the documentation up to date, regularly maintained, and accurate? || No, but the translation tagging of the website revamp will enable more accurate coverage. || No ||
     47|| Is the documentation translated into the same set of languages as the tool? || A small subset of languages, and not completely. || '''No''' ||
     48|| Is the documentation up to date, regularly maintained, and accurate? || Not entirely; not as often as it should be; reasonably. || '''No''' ||
    4949|| Does the documentation correctly describe the security caveats and use cases of the tool? ||  ||  ||
    5050|| Does the documentation make clear security claims, and are those claims supported by the tool? ||  ||  ||
     
    5353=== Audience and Adversary Definition ===
    5454|| '''Criterion''' || '''Our response''' || '''Do we fulfil this?''' ||
    55 || Is the tool actively designed with the needs of at-risk users in mind? ||  || No? ||
     55|| Is the tool actively designed with the needs of at-risk users in mind? ||  || '''No?''' ||
    5656|| Does the project understand who their users are? || Difficult (impossible?) to get these statistics. ||  ||
    5757|| Does the project understand who their user's adversaries are? || See previous criterion. ||  ||
     
    6363|| '''Criterion''' || '''Our response''' || '''Do we fulfil this?''' ||
    6464|| Does the project plan development in public fora (such as a mailing list or IRC channel)? || Yes: IRC2P/#i2p-dev, http://zzz.i2p, http://lists.i2p2.i2p || Yes ||
    65 || Does the project have an accurate roadmap that is up to date and has a history of use? || http://trac.i2p2.i2p/wiki/Roadmaps/1.0 would be the most up-to-date; http://www.i2p2.i2p/roadmap and http://www.i2p2.i2p/todo also exist. None of these have a recent history of use. || No ||
     65|| Does the project have an accurate roadmap that is up to date and has a history of use? || http://trac.i2p2.i2p/wiki/Roadmaps/1.0 would be the most up-to-date; http://www.i2p2.i2p/roadmap and http://www.i2p2.i2p/todo also exist. None of these have a recent history of use. || '''No''' ||
    6666|| Does the project have a public bug tracker? Has the project used their bug tracker over time and kept it accurate? || Yes to both (though not always entirely current due to the small developer base). || Yes ||
    6767|| Does the project have a public source repository that is in use for mainline development? || Monotone (see e.g. http://viewmtn.meeh.i2p/ ) || Yes ||
     
    7171|| '''Criterion''' || '''Our response''' || '''Do we fulfil this?''' ||
    7272|| Does the project have documented criteria for determining what is a security issue? || '''TODO: Check this''' ||  ||
    73 || Does the project have a documented process for classifying the impact of security vulnerability reports? || '''TODO: Define or set up''' || No ||
    74 || Does the project have a documented response process for security vulnerability reports? || '''TODO: Define or set up''' || No ||
     73|| Does the project have a documented process for classifying the impact of security vulnerability reports? || '''TODO: Define or set up''' || '''No''' ||
     74|| Does the project have a documented response process for security vulnerability reports? || '''TODO: Define or set up''' || '''No''' ||
    7575|| What is the project history of responding to security incidents and is it documented? || '''TODO: Check history''' ||  ||
    76 || Does the project have an internal responsible disclosure policy and is it used? ||  || No ||
     76|| Does the project have an internal responsible disclosure policy and is it used? ||  || '''No''' ||
    7777|| What timeline does the project have around responding to vulnerabilities? || As soon as possible? ||  ||
    7878
     
    8181|| Does the project have a license on the Open Source Initiative's list of free software licenses? || Several - http://www.i2p2.i2p/licenses || Yes ||
    8282|| Is the project functionally open source such that the PRB could independently fix vulnerabilities without project team cooperation if it became necessary? This includes situations where, for example, a free software project depends on a proprietary library. ||  || Yes? ||
    83 || Is the project aware of any potential patent or copyright issues with the project that would limit distribution of any part of their project or interfere with the audit process? || No ||  ||
     83|| Is the project aware of any potential patent or copyright issues with the project that would limit distribution of any part of their project or interfere with the audit process? || No. '''TODO: Check this''' ||  ||
    8484
    8585=== Privacy and Terms of Service Disposition ===
     
    9797|| Does the project have a public bug tracker? Has the project used their bug tracker over time and kept it accurate? || See ''Development Process Transparency'' above. ||
    9898|| Does the project have a public source repository that is in use for mainline development? || See ''Development Process Transparency'' above. ||
    99 || Is the project documentation up to date in all the languages the project supports? || See ''Documentation'' above. ||
     99|| Is the project documentation up to date in all the languages the project supports? || No, but the translation tagging of the website revamp will enable more accurate coverage. || '''No''' ||
    100100|| Does the project have a test framework? || Both JUnit and !ScalaTest || Yes ||
    101 || Does the project have significant regression testing coverage? || About 30% - see http://jenkins.killyourtv.i2p/job/cobertura/ || No ||
     101|| Does the project have significant regression testing coverage? || About 30% - see http://jenkins.killyourtv.i2p/job/cobertura/ || '''No''' ||
    102102
    103103=== Project Impact ===
    104104|| '''Criterion''' || '''Our response''' || '''Do we fulfil this?''' ||
    105 || How large is the project's user base? || http://stats.i2p shows ~30,000 routers || ||
     105|| How large is the project's user base? || http://stats.i2p shows ~30,000 routers || Yes? ||
    106106|| Does this project benefit an at-risk population directly or indirectly? ||  || Yes ||
    107107|| Are there any alternatives for this functionality on the platforms it serves? || Tor provides hidden services, but unmaintained and tangential to Tor's target functionality. ||  ||
     
    109109|| How security-critical is the tool's functionality? || Being an anonymous network, other tools are dependent on its security. ||  ||
    110110|| Is this project infrastructure that other tools depend on? || Yes, e.g. eepsites, torrent software, http://nightweb.net || Yes ||
    111 || What does the project's growth curve look like? || Slow growth for the first 8 years, then large (exponential?) growth over the last few years. || ||
    112 || Is this tool localized for significant at-risk populations? || We have translations for Arabic and Chinese (among others). || ||
    113 || Is localization applied consistently? || Localization of the routerconsole is mostly done via gettext. Inconsistencies do occur in the separate-page translations. || ||
     111|| What does the project's growth curve look like? || Slow growth for the first 8 years, then large (exponential?) growth over the last few years. || ? ||
     112|| Is this tool localized for significant at-risk populations? || We have translations for Arabic and Chinese (among others). || Yes? ||
     113|| Is localization applied consistently? || Localization of the routerconsole is mostly done via gettext. Inconsistencies do occur in the separate-page translations. || Maybe? ||
    114114
    115115=== Project Auditing Need ===
    116116|| '''Criterion''' || '''Our response''' || '''Do we fulfil this?''' ||
    117 || Has the project been audited before, and if so how code base changed since the previous audit? || No || ||
     117|| Has the project been audited before, and if so how code base changed since the previous audit? || No || Yes? ||
    118118|| Are their significant known security concerns or has the project had public exploit(s)? || Nothing known? ||  ||
    119 || Is this project implicated in the harm of an at-risk population? || No || ||
    120 || Is the project written in a high-risk language like C? || Written in Java, so... no? || ||
     119|| Is this project implicated in the harm of an at-risk population? || No || Yes ||
     120|| Is the project written in a high-risk language like C? || Written in Java, so... no? || Maybe? ||
    121121|| Is the project's development team relatively inexperienced, especially with security? ||  ||  ||