Article feedback/Extended review

Several groups and organizations have created ad-hoc tools to create and store quality reviews of Wikipedia content.

The Open wiki review system aims to provide a standard, customizable framework for such organizations to conduct open quality review of Wikipedia content, and integrate the results through the Quality indicators API.

This system is primarily intended for Wikipedia.

System type

 * What form does the system take?


 * Recommendation: A MediaWiki extension will facilitate the integration with the wiki, and a third-party software will facilitate the integration with the partner organization's existing system. From a scalability and community perception point of view, a third-party system is preferable. However, if the goal is to move quickly, a MediaWiki extension may be simpler and faster to implement, even if it's not the ideal model.


 * Ways to mitigate the weaknesses: If the MediaWiki extension model is chosen, particular attention must be given not to present reviewers as people with editorial power or undue influence on the content of articles. Wikipedia is an egalitarian environment, and reviewers must acknowledge this reality in order to avoid a conflict, even if they're not used to their authority or comments being questioned.


 * Features recommended independently of the choice:
 * native integration with the Quality indicators API


 * Other possible features: (none at present)

Authentication

 * How does a credentialed reviewer authenticate?


 * Recommendation: A full-fledged authentication system integrated with the partner organization's accounts base is the most scalable and flexible model, but will require a lot of development work. As a simpler alternative, an on-wiki affiliation system would allow accounts to belong to specific groups they're affiliated. The credentials or affiliation of the user would be verified by a group manager. The "e-mail token" model, although simple, seems unreliable and hackish; it is not recommended for this use, even as a temporary solution.


 * Ways to mitigate the weaknesses: (none at present)


 * Features recommended independently of the choice: (none at present)


 * Other possible features: (none at present)

Review submission

 * Who decides who reviews what?


 * Recommendation: A voluntary model is the simplest way to go as a first step, and the most likely to fit within the existing article feedback infrastructure. However, the need for more collaboration features may arise and will need to be addressed.


 * Ways to mitigate the weaknesses:
 * The simple voluntary model can be extended later with additional coordination, collaboration and assignation features, based on the feedback received.
 * Existing expert review systems show that reviewers usually stick to their field of expertise when their name is publicly associated with the review.


 * Features recommended independently of the choice:
 * The real name of the reviewer must be known.
 * Personal queue for pending reviews
 * E-mail notifications
 * API to submit reviews from an existing third-party review system


 * Other possible features:
 * Restriction of reviewable articles (depending on category, selection by review coordinator, etc.)

Review content

 * What is the content of the actual review?


 * Recommendations:


 * Ways to mitigate the weaknesses: (none at present)


 * Features recommended independently of the choice:
 * Reviewers are invited to edit the page they're reviewing and fix the errors they notice.
 * Reviewers are invited to disclose conflicts of interest about the article itself (e.g. the reviewer has significantly edited the article) or the topic (e.g. the reviewer is a known critic of some of the points made); if their reviewer account is the same as, or linked to, their Wikipedia user account, they are prompted to disclose an article COI if they've edited it.


 * Other possible features:

Review publication, access and reuse

 * Where and how is the review accessible?


 * Recommendation: Ideally, the reviews would be published on the organization's website (to benefit from the strengths listed above), and could be fetched from there to be integrated in the wiki's discussion system, where editors could see them and comment. Realistically, mirroring the reviews and comments might prove difficult in the short term. The most important requirement seems to be the proximity of the review to the editors who can improve the entry, and the discussion between the editors and the reviewers; as a consequence, it's probably better to publish the reviews on Wikipedia


 * Ways to mitigate the weaknesses:
 * Integrate reviews with the discussion system (e.g. LiquidThreads), so that 1. Reviews are similar to other on-wiki discussions, and 2. editors can easily comment on them, and discuss with the reviewer.


 * Features recommended independently of the choice:
 * Public list of one's reviews
 * API to access the entirety of the reviews and their specifics
 * Notification to users who watch the reviewed page
 * Ability to comment on the review