Article feedback

The Article feedback project is an initiative of the Wikimedia Foundation to engage Wikimedia readers in the assessment of article quality, one of the five priorities defined in the strategic plan.

It is based on the WMF-developed ArticleFeedback MediaWiki extension and currently deployed on a subset of pages on the English Wikipedia.

This page serves as a central hub to document research, project updates, and development timelines, and to allow for centralized community conversation. Frequently asked questions on this project can be found on this page.

June 22, 2011: First anonymized data dumps for the Article Feedback deployment on the English Wikipedia are available. (read more)

Project History and Background

 * For a detailed log of Article Feedback milestones and releases, please refer to this page

The Wikimedia Foundation has been experimenting with a feature to capture reader quality assessments of articles since September 2010. Originally designed to support the Public Policy Initiative, the first phase of the feature was rolled out last September on the English Wikipedia. In November, the feature was put on another 50-60 articles in addition to those improved through the Public Policy Initiative.



In March 2011, v2.0 of the feature was released to approximately 3,000 English Wikipedia articles. The user interface of the feature was redesigned.



We also added calls to action. These are invitations presented to the rater, after the completion of the rating process, to take an additional action.



The three calls we’ve tested so far are invitations to create an account, to take a survey, or to edit the page.

Finally, version 2 includes a checkbox (currently A/B tested to 50% of users) that can be used to qualify the expertise of the rater. The checkbox includes the following options:
 * I am highly knowledgeable about this topic (optional):
 * I have a relevant college/university degree
 * It is part of my profession
 * It is a deep personal passion
 * The source of my knowledge is not listed here
 * I would like to help improve Wikipedia, send me an e-mail (optional)

The e-mail option was recently added to test systematic e-mail calls-to-action targeting self-identified expert raters.

Since launching the tool in September, we’ve continually analyzed the results and made small modifications based on quantitative and qualitative data. We’re still learning from the feature, and based on the work so far, we think that reader feedback has a lot of potential:
 * Quality assessment
 * article feedback can help complement internal quality assessment of Wikipedia articles with a new source of data on quality, surfacing content of potentially very high or very low quality, and measuring change over time.


 * Reader engagement
 * article feedback represents a way to encourage participation from the reader community.

Current Plan (June 2011)
Given the encouraging results of the trial so far, we rolled out the feedback tool to approximately 100,000 articles of the English Wikipedia in the week of May 9, 2011. Articles were selected at random (versus the stratified-random sampling we used for the 3,000 articles). The primary goal of this rollout was to test the scalability of the feature. We will continue to analyze the ratings, survey, and click-through data, and we are working on an experimental dashboard.

The idea behind the dashboard is to surface articles to the editing community that are being rated very highly or very lowly. The use of this information, if it is used at all, is up to the community. Please provide feedback on this feature on the talk page of the feature.

One June 30, 2011 the feature was enhanced to add explanatory tooltips for each rating (i.e., "What does one star mean?").

We're planning to ramp-up use of the feature starting July 12 (at the earliest) due to a dependency on some upd2log work that will help with the performance of this and other features. This schedule may still change.

In the longer term, we hope to be able to explore some more complex ideas, including:


 * enabling readers to leave free-text comments specifying issues with the article
 * enabling readers to praise the authors of an article, and enabling authors to receive praise
 * enabling editors to collaboratively filter rater comments and issue reports, and to promote comments of special significance to the talk page
 * enabling raters to attach credentials to a Wikipedia user account, which will then be associated with their rating.
 * improving the user interface integration of the tool to reduce initial screen real estate usage while increasing visibility/discoverability.

These ideas are described in more detail on the extended review page, including first wireframes for a more comprehensive rating and feedback tool.

Quantitative research

 * Quantitative Research
 * Article feedback/Research/February 2011
 * Call to action
 * Rater expertise

Anonymized rating data are available for download from the Wikimedia Toolserver (see data format).

Initial UX research

 * Article feedback/UX Research

Usability testing
Usability testing videos from May 2011:

Future
As we are still learning about the use of this feature, no decisions on future enhancements have been made.


 * Ideas log: a log of feature requests and possible improvements
 * Article feedback/Extended review: A possible evolution of the feature to conduct open peer quality review.

Other documents

 * Frequently asked questions