Quality review

From mediawiki.org

The open nature of wikis makes quality management a serious challenge. This page gives a general overview of quality review systems and their characteristics.

Review processes by types[edit]

A review process can be:

  • internal: the process is mainly handled locally by the wiki's community, and the data is stored in the wiki's pages or its database;
  • external: the process is mainly handled by an external group, and the data is stored outside the wiki. Integration with the wiki is optional.
  • mixed: e.g., an external group posts reviews on the wiki.

Some authority-based review processes distinguish between "experts" and others. In this case, reviewers can be classified whether they are:

  • "non-experts"
  • people self-identifying as experts (no credentials verification)
  • people identifying as experts whose credentials have been verified.

Example review processes[edit]

A more detailed description is available for some projects.
  1. People with academic degrees or publications in the relevant science (RNA/protein biology)
  2. Reviews are filtered a posteriori based on some basic level information provided by reviewers about their education level and whether they are a member of APS or not.

Review systems[edit]

  1. Exclusive: only one master review/rating possible; cumulative: multiple reviews/ratings possible.
  2. For example to select articles for publication in other media like books.

User research[edit]

In the context of quality review, there are two kinds of users: reviewers and review readers.

From a reviewer point of view

  • What system, how it works, what the goals are, what they expect
  • Do they want to communicate with review readers?

From a review reader point of view

  • What are their goals by reading a review? (i.e. what is useful for them)
  • What do they expect from a review?
  • Do they want to communicate with reviewers?

People who have already created review processes might be able to explain:

  • what were their goals when they created these tools & processes
  • whether some design decisions were intentional (e.g. type/scale of rating)

Domain research[edit]