Article feedback/Public Policy Pilot

This page lists the Functional Requirements for the Public Policy Pilot of the Article feedback feature. This is a work in progress.

Feedback from the Public Policy Initiative has been and will continue to be incorporated into this feature. The Public Policy Initiative will decide whether to use the data as part of their project evaluation.

Objectives

 * Encourage user participation by giving readers the ability to assess an article
 * Give readers and editors a glanceable average of (user generated) ratings

Timeline
We aim to have this feature in production by September 15, 2010.

Brief Use Cases/Scenarios

 * Reader A - an occasional wikipedia reader
 * Reader B - frequent wikipedia reader
 * Expert Reader - subject matter expert reader
 * Editor A - copyeditor
 * Editor B - contributor to that article
 * Expert Editor - one of the main contributors to the article

Feature Requirements
General requirements
 * This feature will exist only on a subset of pages designated by the Public Policy Initiative
 * This feature will be available to all users (not just logged in)

Reader Feedback: Input and Display
 * There will be the following 4 categories, each with a five-star rating:
 * Well-sourced
 * Complete
 * Neutral
 * Readable


 * There will be tooltips to describe each category of rating (Amy to provide descriptions)
 * There will be a link to provide feedback on the feature. This link will go to a survey (see  proposed questions and survey page (not enabled yet) )


 * For a given user and article, there are the following states for review:
 * Haven't reviewed at all
 * Have reviewed current revision, let user change review
 * Haven't reviewed current revision, but reviewed previous revision, let user re-review, give message that their review is stale


 * There will be two areas (boxes) upon page load:
 * Rate this article box: for reader to provide ratings:
 * Averages will not be reflected in stars
 * Once user submits ratings, page will reload and reflect action just taken
 * Number of ratings (below) increments
 * Average ratings box: displays average of ratings + total number or ratings per dimension in graphical form
 * Averages calculated by taking average of last 10 ratings. If there are fewer than 5 ratings, we will display "Not enough ratings to provide average" message.
 * There will be links to view the average ratings in graphical form.


 * Re-rating an article (stale ratings)
 * Once a user has rated an article, they will be prevented from rating it again unless the stale requirement has been met
 * Stale requirement: if there has been at least 5 edits from other users since the last review, we will inform the user that their review may be stale (messaging needs to be soft and cannot imply that the article is stale). The user may then re-rate the article.


 * Updating Article Averages (stale averages)

Analytics
 * Ideally, we would have page views per article so that we know the ratings rate = (number of ratings) / (total views of page)
 * We also need some measurement of ratings completion = (users that complete the rating process) / (users that start the rating process)
 * Ways of analyzing ratings distributions in aggregate (vs. one article at a time).

Known Issues
 * Re-rating current revisions should be allowed. This is a bug in the existing software.

Deleted Requirements
 * THere will be 1 "overall" rating with a five category rating
 * There will be an open comment field. What happens with the information entered in the comments field is TBD. Options are:
 * The comment goes into some database, but not published anywhere (I think this is where we are leaning)
 * The comments go a special page which is linked from somewhere either on the article page or rating control
 * We'll likely see spam quickly. Restricting addition of URLs would help. Note that WP also has a fair share of aggressive trolls/vandals who may post inappropriate or personal information. If we don't want to implement a basic deletion mechanism, we will probably need to restrict views.

Design
Available at Article feedback/Public Policy Pilot/Design

Feedback Form Questions
First phase available at Article feedback/Public Policy Pilot/Feedback_Survey

Second phase available at Article feedback/Public Policy Pilot/Feedback Survey Phase2

Technical Requirements
Available at Article feedback/Public Policy Pilot/Technical

Metrics

 * Early Data: Metrics from the first week of use.