Article feedback/Version 5/Feature Requirements

This page describes new features to be developed for the Article Feedback Tool Version 5 (AFT V5), during phase 1 of this project (Oct.-Dec. 2011). Other features for phase 2 (Jan.-Mar. 2012) are only referred to here in outline form. Check the new AFT V5 project page for more information about this initiative. For a preview of what these forms and pages might look like, read below, or, which include simple wireframes for key touchpoints, as well a project plan and other useful exhibits.

The technical design of the AFT can be found here.

Overview
Key features for phase 1 of AFT V5 will include:
 * new feedback forms
 * calls to action
 * feedback page
 * moderation tools
 * expanded feedback form

In the first phase of this project (Oct.-Dec. 2011), we will create and test four different types of feedback forms:
 * Option 1: Share your feedback - Did you find what you were looking for? Add a comment
 * Option 2: Make a suggestion Suggest an improvement, ask a question, report a problem or give praise to the editors
 * Option 3: Review this page - Rate this article. Add a comment or suggestion for improvement.
 * Option 4: Edit this page - Help improve Wikipedia (call to action, for comparison purposes)

We plan to A/B test these four options against the current rating tool, to find out which is most effective for engaging readers, supporting editors and improving article quality. We will also test them against the absence of any feedback form, to be further described in the metrics section.

We plan to develop more features in a second phase (Jan.-March 2012), based on first phase results. We are starting a wish-list of features for that second phase and will link it here shortly.



Technical (non-functional) requirements

 * The AFT V5 shall be developed on top of the existing AFT implementation (MediaWiki extension), preserving its current functionality, e.g.:
 * Ratings from feedback items are not counted after 30 days (but continue to be stored so that they can be retrieved on the feedback form)
 * Feedback tool can be disabled in user preferences for registered users
 * Source control requirements:
 * Core - use v1.18 (REL1_18)
 * Extensions - from trunk
 * AFT being developed as an extension, forked (not branched) from trunk



Feedback functionality
For each article, the following feedback items shall exist:
 * Text comment (we may also want to track number of words or characters)
 * Comment tag, one and only one of:
 * Suggestion
 * Question
 * Problem
 * Praise
 * Ratings - numerical, on a scale between 0 and 5. The following rating criteria shall exist:
 * Overall rating
 * Credibility
 * Neutrality
 * Thoroughness
 * Well-written
 * Yes/No Answers - those may also be mapped to the overall rating (yes = 4, no = 2), though we need to track whether the Yes/No buttons were used
 * Commenter's source of knowledge on the subject (pre-defined options - TBD: text entry for a generic "not listed here" option? - FF: Sure, though we are de-emphasizing this for now, as results for this have been inconclusive so far. We will only keep it for the current version)

For logged in users, feedback should be posted under their user account. Any registered user may only have one feedback record per article. New feedback from a user on an article he already posted feedback on should replace the previous feedback record for that user and that article.

For anonymous or logged out users, feedback shall be posted with their IP address. ''TBD: how to limit additional feedback from anonymous users? Time window? Session-based? None? - FF: we don't know yet how to deal with this. Also note that we plan to store IP addresses for all users, not just logged out, and that the last two digits of the IP address may be used to decide which form to display to the user.''

All feedback records shall be time-stamped. All feedback records will also include the number of yes or no votes from other readers rating each post, as well as whether or not that post has been featured, hidden or just listed.



Feedback Form interface
The feedback interface appears on every article, as the current AFT does (unless disabled in user preferences). There shall be several options of the user interface, for A/B testing. Ultimately, one option will be selected for production use. The following 4 options shall be implemented (A, B, C, D), in addition to the currently existing one (E). We will also test a sixth option (F), where we will not display any feedback form at all.
 * TBD: which articles do/do not display the feedback form (criteria). FF: Articles that are newly created, or which do not include any significant content. Also, 1 out of 6 times, we may display no feedback form at all.
 * The interface option to display shall be selected randomly. (This would be done so that there is equal chance for each of the 6 options, which would be determined by looking at the last two digits of the IP address)



Option 1


This variation is a feedback form. The important parts of it are: Additional elements:
 * A yes/no question. (the yes/no shall be mapped to 4 and 2 overall rating, respectively - we will also record the yes/no answer in a separate field).
 * A comment box (multiline).
 * Help button (link to static page or FAQ with tips on how to give feedback)
 * Links to feedback page and privacy policy
 * Post button



Option 2


This variation of the feedback form focuses on the text comment and its label (tag).
 * The title of the form shall reflect the currently selected label ("Make a suggestion"/"Ask a question"/etc)
 * Label selection (default selected is suggestion)
 * Comments box (multiline, with different prompt depending on selected label)
 * Post button (caption shall reflect the selected label).
 * Links to the feedback page and privacy policy



Option 3


This version of the feedback form provides the users with the ability to give an overall rating to the article (see rating functionality above), and a comment. Labels/tags input are not provided on this form.



Option 4


This variation does not provide feedback options. Instead its a direct call for the readers to edit the article. The "Edit" button takes the reader to the article's edit page. The "learn how to edit" link takes the reader to editing help hub. Only registered and logged in and anonymous users shall be able to edit the page. For anonymous users the edit button shall be replaced by the usual login/register options, while still maintaining the "Edit" call to action. This option will not be displayed for articles that are protected or semi-protected.



Option 5


The current version of the Article Feedback Tool will be one of the options we will give to users during the test in December.

Option 6
The last option we will give to users during the test in December will be not feedback form at all.

Editing your feedback
The user shall be able to go back and edit their feedback for the article, as per current functionality and with accordance to the requirements outlined in the feedback functionality section above. Only one feedback post per user per article.

Image to be added later on.



Calls to Action
After readers post their feedback, they will see one of these calls to action:
 * edit this article (for unprotected pages only - anyone can edit, even if they are not logged in)
 * take a survey (link to survey page) - this call to action may be postponed for a week
 * sign up or login (if logged out) - this call to action may be postponed for a week
 * get email notifications (if my post is used) - this call to action may be postponed for a week

We may start the test in December with only the 'Edit' call to action, but will plan to add more later on.

Edit this page


This is one of several possible calls to action to be displayed after the feedback had been submitted. We may start the test in December with only this call to action, but will plan to add more later on.

Take a Survey


New image will be added later (thumbnail of current version is provided here for reference). This call to action may be postponed for a week, so we can focus on the first call to edit in our first tests.

Sign up or login
Image to be added later on. This call to action may be postponed for a week, so we can focus on the first call to edit in our first tests.

Get email notifications
Image to be added later on. This call to action may be postponed for a week, so we can focus on the first call to edit in our first tests.

Other calls to action
Other optional calls to action we are considering for phase 1.5 include:


 * discuss this article
 * view feedback page
 * learn how to edit

Specs to be added later on, if we decide to include any of these in phase 1.5.



Feedback page


The shall display a list of the feedback items for an article. Registered users will be able to see all the posts for this article, as well as sort or filter them, and vote on whether or not they are helpful. Editors and administrators will also be able to moderate the feedback page, and feature, hide or list feedback posts.

The actual nature of the comments we receive from readers (e.g., signal-to-noise ratio, what users actually comment on, etc.) will influence the design of the feedback page, so final development of the feedback page will likely occur once the first feedback forms have been tested.

The feedback page shall support navigation functionality for the comments list:
 * Filtering (show: ...) - predefined set of options:
 * All - unfiltered
 * ''TBD - more options. FF: This could include 'Most Recent', 'Top Rated', 'Featured Posts', 'By Editors', 'Comments-only', etc. One of these is likely to be the default, instead of 'All'."
 * Sorting (sort by: ...) - predefined set of options:
 * Date
 * Comment rating (up/down votes by other members)
 * Overall article rating

The comments list shall display a configurable number (TBD: configurable how? per-user/systemwide? what is the default? this may require MediaWiki core changes. FF: Good questions, need to discuss this some more. I think starting with 50 posts by default might be a good start. Not sure we need to give user control of number of posts right now.) of items by default. The "show more comments" button at the end of the list shall load additional items, same amount, and, again, display the "show more comments" button at the end of the list.

Each item in the comments list shall display:
 * Logged user name/IP address
 * Overall article rating
 * Collapsable article rating details (collapsed by default)
 * Timestamp
 * Comment
 * Votes on feedback posts (+1/-1), such as up/down arrows (or 'Is this feedback helpful?' with 'Yes/No' buttons) - providing the ability to rate individual comments by other users.
 * Moderation tools (see below) - available only to moderators/administrators.

The feedback page shall support the following moderation tools for each comment, visible only to moderators and administrators:
 * Feature (Star icon?) - in phase 1 of the project this will promote the item to top of page, as well as include a 'Featured' label next to post.
 * Hide (Trash icon?) - mark the comment as hidden. Functionally, the comment is not deleted from the database, just not displayed unless a specific filter is selected, as well as include a 'Hidden' label next to post.
 * List (for changing featured hidden posts back to normal status)

Nested comments - the commenting functionality shall provide for nested comments (that should be reflected both in the database schema and in code infrastructure), but this functionality should not be available in the interface. The infrastructure should not impose limits on the depth of the comments tree. If such limits are to be introduced at later phases of the project, they shall be set in the code/configuration.

Expanded Feedback
(preliminary) (optional) - Tell us more: suggest improvements, rate article qualities or share your expertise.