This page is currently a draft.
The open nature of wikis makes quality management a serious challenge. This page gives a general overview of quality review systems and their characteristics.
Review processes by types
A review process can be:
- internal: the process is mainly handled locally by the wiki's community, and the data is stored in the wiki's pages or its database;
- external: the process is mainly handled by an external group, and the data is stored outside the wiki. Integration with the wiki is optional.
- mixed: e.g., an external group posts reviews on the wiki.
Some authority-based review processes distinguish between "experts" and others. In this case, reviewers can be classified whether they are:
- people self-identifying as experts (no credentials verification)
- people identifying as experts whose credentials have been verified.
Example review processes
- A more detailed description is available for some projects.
|Review process||Wiki||Type||Reviewers||Review location||Review tool||Visibility on page||Notes|
|WikiProject assessments||Wikipedia||internal||local community (thematic "WikiProjects")||wiki talk pages||optional (JS)||optional (JS)|
|WikiProject Medicine / Google||Wikipedia||mixed||professional medical editors||wiki talk pages||no||no||Example|
|Public Policy Initiative||Wikipedia||mixed||local community and subject matter experts||Google document||no||no|
|Article feedback pilots||Wikipedia||mixed||anyone (readers & authors)||wiki database||yes (MW ext.)||yes|
|Encyclopedia of Life||Wikipedia||external||Individual EOL curators||external database & Wikimedia toolserver||?||optional (JS)|
|Rfam / Pfam||Wikipedia||external||academics from the Wellcome Trust Sanger Institute||external database? & Wikimedia Toolserver||?||optional (JS)|
|APS Wikipedia Initiative||Wikipedia||external||technically, anyone, but filtered a posteriori||external database||yes (PHP)||no||Harnessing the Power of Wikipedia for Scientific Psychology: A Call to Action|
- People with academic degrees or publications in the relevant science (RNA/protein biology)
- Reviews are filtered a posteriori based on some basic level information provided by reviewers about their education level and whether they are a member of APS or not.
|Review process||Type||Binary flag||Metrics||Metrics values||Free-form comments allowed||Notes|
|WikiProject assessments||exclusive||no||quality, importance||quality: 7 (stub, start, C, B, GA, A, FA); importance: 4 (low, mid, high, top)||yes|
|WikiProject Medicine / Google||cumulative system, but exclusive in practice||no||none (qualitative)||n/a||yes||checked against good article criteria|
|Public Policy Initiative||no||comprehensiveness, sourcing, neutrality, readability, formatting, illustrations||resp. 1-10, 0-6, 0-3, 0-3, 0-2, 0-2|
|Article feedback pilots||cumulative||no||well-sourced, neutral, complete, readable||1-5 (stars)||no||possibility to skip metrics (value = 0)|
|Encyclopedia of Life||exclusive||yes (trusted/untrusted)||misidentified, incorrect/misleading information, poor writing/image/sound quality, redundant/duplicate, other||2 (boolean)||yes||user flags issues rather than assessing metrics|
|Rfam / Pfam|
|APS Wikipedia Initiative||cumulative||no||trustworthy, unbiased, complete, well-written, accurate||5 (strongly disagree / disagree / neutral / agree / strongly agree)||yes|
|Article feedback extended review||cumulative||TBD||TBD||TBD||yes||planning phase|
- Exclusive: only one master review/rating possible; cumulative: multiple reviews/ratings possible.
- For example to select articles for publication in other media like books.
In the context of quality review, there are two kinds of users: reviewers and review readers.
From a reviewer point of view
- What system, how it works, what the goals are, what they expect
- Do they want to communicate with review readers?
From a review reader point of view
- What are their goals by reading a review? (i.e. what is useful for them)
- What do they expect from a review?
- Do they want to communicate with reviewers?
People who have already created review processes might be able to explain:
- what were their goals when they created these tools & processes
- whether some design decisions were intentional (e.g. type/scale of rating)
- Article feedback/UX Research
- Quality assessment tools for Wikipedia readers
- m:Expert review & m:Talk:Expert review
- Survey: Expert barriers to Wikipedia (includes relevant questions such as: (1) whether respondents volunteer to be interviewed; (2) whether they would be willing to review wiki articles)