Article feedback/FAQ


 * What is the "Article Feedback Tool"?


 * The Article Feedback Tool is an experimental tool that allows any reader of an article (whether they're also an editor or not) to quickly and easily assess the sourcing, completeness, neutrality, and readability of a Wikipedia article on a five-point scale. It's a way to increase reader engagement by getting feedback from readers on how they view the article, and it gives editors an easy way to see where an article needs improvement. In general, the pilot of this tool also reflects a shift in the Wikimedia Foundation's development processes towards more systematic experimentation and trials with new technology. We believe small experiments like this can be very useful in helping Wikimedia to innovate and learn.


 * Where is it being rolled out?


 * We've scheduled a pilot roll-out on 400 English Wikipedia articles that are part of WikiProject United States Public Policy for September 22, as part of the Wikimedia Foundation's Public Policy Initiative, a grant-funded article improvement project. The duration of the experiment is from September to December 2010, which corresponds to the first term of the schools participating in the Public Policy Initiative.


 * What will this new tool help the Wikimedia community achieve?


 * The Wikimedia community cares a great deal about quality, and employs a number of mechanisms for article self-assessment (varying by project and language edition). The English Wikipedia, for example, utilizes processes for nominating and selecting the best articles, and for tagging articles associated with specific WikiProjects by quality class.


 * There is, to date, no standard mechanism by which large numbers of readers can easily engage in quality assessment. We believe that such a mechanism provides multiple potential advantages:
 * A simple rating process could be an entry point to provide strong invitations to readers to edit, discuss, or participate in other way.
 * Extremely high or extremely low ratings may be useful indicators that support community cleanup or article nomination processes.
 * We may be able to build upon this tool to develop a strong, standardized rating framework for content.
 * Large numbers of readers who rate an article to be of high quality provide an element of external validation of our self-assessments.
 * Even if the ratings are imperfect, they may reveal useful trends over time.
 * An easy-to-use tool is likely to scale well to a large number of articles.


 * What are some anticipated obstacles?
 * We'll keep a close look at some possible issues:
 * There may be attempts to game the system. We therefore will carefully analyze rating behavior in this first pilot.
 * The tool may be used by people whose knowledge on the topic they are assessing is limited. We will survey the users of the system, and will also compare the rating information we will receive against other rating instruments.
 * Some rating designs may reduce the incentive to edit or discuss issues. In this pilot, we will not get things exactly right, but we are well aware of this potential issue, and plan to carefully study how ratings influence user interaction as a whole.
 * Low-traffic articles may receive an insufficient number of ratings, or ratings may date too quickly to be useful as articles change.


 * How do I use the tool?


 * Just click on the star you want to assign each article and press the submit button.


 * Won't these ratings measure information a computer could easily predict (e.g. number of citations)?


 * We suspect that the ratings will correlate very strongly with heuristics that could be developed to predict the same quality characteristics of an article, such as standardized readability tests. Aside from some toolserver scripts, we don't currently have standardized heuristics of this type, and this may be an area of future development and exploration. However, we also hypothesize that the most interesting human-generated ratings are those that substantially deviate from what the heuristics would tell us. For example, if a large number of readers rate an article as poorly sourced, even though it has many citations, that is likely an article worthy of further examination.


 * Do you have plans to roll it out to a larger number of articles?


 * This is an initial pilot. The Wikimedia Foundation will carefully study the interaction characteristics of the system and the data obtained, and will use this data to inform its proposed design for future iterations of such a system. We would like to work with a group of Wikimedians to evaluate the pilot and think about the future of content assessment in Wikimedia projects.


 * Are you suggesting that a high or low rating means that an article needs to be improved?


 * No. At this point, we're experimenting with this tool. It's possible, for example, that reader-driven quality assessment will only be useful on some types of articles, or will not be useful at all.


 * How will the software deal with multiple votes from the same account or IP (i.e., ballot stuffing)?


 * Only the most recent ratings from a single user will be used in the calculation. If you rate an article with three stars, and then later rate it four stars, only the four star rating will be included in the calculations.


 * How will out-of-date ratings be handled?


 * Your rating will become "stale" after five revisions have occurred since you last rated the article. This will be denoted by pale blue stars and a message below advising you that you may wish to re-rate the article. Simply click stars again to submit your new rating.