Article feedback/Version 5/Feature Requirements

This page describes new features to be developed for the Article Feedback Tool Version 5 (AFT V5), during phases 1.0 and 1.5 of this project (Oct.-Dec. 2011). Other features for phase 2 (Jan.-Mar. 2012) are only referred to here in outline form. Check the new AFT V5 project page for more information about this initiative. For a preview of what these forms and pages might look like, read below, or, which include simple wireframes for key touchpoints, as well a project plan and other useful exhibits.

See also: technical design page, as well as data and metrics plan.

Overview
Key features for phases 1.0 and 1.5 of AFT V5 will include:
 * new feedback forms
 * calls to action
 * feedback page
 * administration tools
 * expanded feedback form

In the first phase of this project (Oct.-Dec. 2011), we will create and test four different types of feedback forms:
 * Option 1: Share your feedback - Did you find what you were looking for? Add a comment
 * Option 2: Make a suggestion Suggest an improvement, ask a question, report a problem or give praise to the editors
 * Option 3: Review this page - Rate this article. Add a comment or suggestion for improvement.
 * Option 4: Edit this page - Help improve Wikipedia (call to action, for comparison purposes)

We plan to A/B test these options against each other, to find out which is most effective for engaging readers, supporting editors and improving article quality. We will also test them against the absence of any feedback form, to be further described in the data and metrics plan (option 0).

We plan to develop more features in a second phase (Jan.-March 2012), based on first phase results. We are starting a wish-list of features for that second phase and will link it here shortly (for now, we have outlined key phase 2 features and labeled them as such below).



Technical (non-functional) requirements

 * The AFT v5 shall be developed using much of the existing AFT implementation (MediaWiki extension). However, the AFT v4 tool will remain a separate code module, and the two will co-exist on the same server. Some of the current functionality v4 functionality will continue to be supported in v5, e.g.:
 * Feedback tool can be disabled in user preferences for registered users
 * Source control requirements:
 * Core - use v1.18 (REL1_18)
 * Extensions - from trunk
 * AFT being developed as an extension, forked (not branched) from trunk
 * The AFTv5 shall be able to install together with AFTv4. Only one (i.e. either AFTv4 or AFTv5) shall be displayed on the page. To further clarify, the AFTv5 shall replace AFTv4 on a predefined subset of articles for the period of testing.



Feedback inputs
For each article, the following feedback user inputs shall exist:
 * Text comment (we may also want to track number of words or characters)
 * Yes/No Answers (option 1 only)
 * Comment tag, one and only one of: (option 2 only)
 * Suggestion
 * Question
 * Problem
 * Praise
 * Ratings (option 3 only)
 * Overall rating
 * Trustworthiness (expanded forms only)
 * Objectivity (expanded forms only)
 * Completeness (expanded forms only)
 * Writing quality (expanded forms only)



Access and permissions

 * Any user can post feedback about an article, whether they are logged in or not.
 * Users can post multiple comments per article, if they have several suggestions.
 * For logged in users, feedback is posted under their user account.
 * For anonymous or logged out users, feedback is posted under their IP address.
 * Anyone can view the feedback page
 * Anyone can vote up or down any post on the feedback page (yes/no)
 * Administrators and editors only can access admin tools on the feedback page (hide, feature, etc.)



Other data
Feedback records will also include for each post:
 * time-stamp
 * yes or no votes given by other users for this post
 * abuse flags from other users about this post
 * status: whether this post has been hidden or featured by moderators
 * comments made by other users in response to this post (phase 2)

Other feedback items from expanded forms may include, if applicable (phase 1.5 or 2):
 * Checked suggestions for improvement (e.g.: "Need more links")
 * Commenter's source of knowledge on the subject (pre-defined options TBD)
 * Section which the user is giving feedback about (if that option is available in expanded feedback, or recorded through link placement next to edit button)



Feedback form
The feedback form interface will appear on every article selected for our phase 1 test, as the current AFT does now on Wikipedia (unless disabled in user preferences). There will be several options of the user interface, for A/B testing. Ultimately, one option will be selected for production use on all articles. The following 4 options shall be implemented, along with a fifth option 5, where we will not display any feedback form at all.

Note: For final text copy, please refer to the text descriptions below, rather than the graphic wireframes and mockups, which could be out of date.
 * During the first testing phase, new feedback forms will only be shown on a subset of articles in the english encyclopedia (e.g.: Obama). They will not be shown for articles that are newly created, or which do not include any significant content.
 * Feedback forms will not be shown either if the user has selected to hide the AFT (checked 'Don't show the Article feedback widget' in Preferences > Appearance >)
 * The feedback form option to display shall be selected according to the reader's IP address, to give an equal chance to all options to be selected. (This would be determined by looking at the last two digits of the IP address, so we can place users into buckets for longitudinal studies. For example, in our first test IP addresses ending with 00 to 33 would be assigned to bucket 1, 34 to 66 to bucket 2, etc.)
 * The "bucketing" mechanism shall not be configurable, but should be easily changeable in code.
 * For testing purposes, we will need a way for a tester to specify which form they want to test, regardless of their IP address (special URL tag? temporary text links at the bottom of the forms?)
 * All feedback forms will be displayed at the bottom of the article pages.
 * We are also planning to test three different 'Feedback' text links higher up in the browser window, as described in the Placement section below. This would display the form as an overlay at the same level as the text link (or at the very least provide a jump link to the form at the bottom, if overlays cannot be implemented for phase 1).
 * The feedback form interface shall follow the Mediawiki style guide
 * All links will be underlined on hover (as per the style guide). External links and links that open in another window shall have an appropriate icon next to them.
 * For returning users (registered & logged in, already provided feedback for this article): the form will be blank, even if you provided feedback earlier. But this text link will be shown: 'See your last post >>' with a link to your last post(s) on the feedback page, sorted by user. Unlike with the AFT v4 form, users will not be able to edit previous feedback (do we want to allow a grace period, allowing you to edit for 10 minutes if they make a mistake?
 * when calculating overall ratings (or yes/no answers), only count the last post rating, not an average of all ratings by that user for that article



Option 1


(See also for this feedback tool.)

This feedback form includes these design elements, from top to bottom, left to right: 'Your comment will be shared on this [feedback page]. By posting, you agree to transparency under these [terms]'
 * Title: 'Help improve this article' (instead of earlier titles: 'What do you think?' or 'Share your feedback')
 * Help button (link to static page or FAQ with tips on how to give feedback)
 * Question: 'Did you find what you were looking for?'
 * Yes/No Buttons:
 * The yes/no buttons shall have a button appearance, but function as radio buttons.
 * Clicking a yes/no button shall toggle the yes/no switch state, but not submit the form.
 * The yes/no answers shall be stored in a separate field, as well as mapped to 4 and 2 overall ratings
 * Clicking on yes/no buttons will cause a context-sensitive text prompt to appear in the comment box (see below)
 * A comment box (multiline text area)
 * A gray text prompt inside the comment box (after user clicks yes or no, e.g.: 'What's missing? Any suggestions for improvement?' - see below)
 * Small disclaimer text with links:
 * Post button: '[ Post your feedback ]'

Context-sensitive text prompts: The prompts in the text area are context-sensitive, so you get a different message if you click on Yes, rather than No. 
 * Yes: "What did you like most? Share your praise with the editors."
 * No: 'What's missing? Any suggestions for improvement?' (as shown in this wireframe)
 * If neither Yes/No button is selected, do not show text prompt (as shown in this wireframe)

Option 2


This second variation of the feedback form focuses on the text comment and its label (tag). Interface functional details:
 * The title of the form shall be "Help improve this article" (used to be context-sensitive)
 * Label selection (default selected is 'suggestion')
 * Comments box (multiline, with different prompt depending on selected label)
 * Post button (caption shall reflect the selected label).
 * Links to the feedback page and privacy policy
 * The labels selection shall function as tags selection, only one can be selected at any time.
 * Selecting a label shall change the default placeholder text for the comments box, only if no text had been entered yet
 * If text had been entered in the comment box, it shall persist through label changes.
 * To further clarify, there is only one text input box.

Note: This form design is inspired in part by GetSatisfaction.com. 

Option 3


This version of the feedback form provides the users with the ability to give an overall rating to the article (see rating functionality above), and a comment. Labels/tags input are not provided on this form. The visual appearance shall be the same as option 1, but have 5-star rating widget instead of the yes/no buttons.



Option 4


This variation does not provide feedback options. Instead its a direct call for the readers to edit the article. The "Edit" button takes the reader to the article's edit page. The "learn how to edit" link takes the reader to editing help hub.
 * Both registered and anonymous users shall be able to edit the page, as per current Mediawiki functionality.
 * If the article is not editable by the user (according to its protection level), another interface option is to be displayed instead (options 1-3 or 5)



Option 5
The last option we will give to users during the test in December will be no feedback form at all.

Multiple posts / Editing your feedback
Anyone can post as many feedback items as they want for a given article (TBD: to prevent spam, do we want to set a limit of 10 posts per user per day?). We may also give the users 30 minutes or so to go back and edit their feedback, in case they make a mistake they want to correct. This may be done by adding an edit link next to your posts on the feedback page.



Calls to Action
After readers post their feedback, they will see one of these calls to action (CTA):
 * edit this article (for unprotected pages only - anyone can edit, even if they are not logged in)
 * take a survey (link to survey page) - this call to action may be postponed for a week
 * sign up or login (if logged out) - this call to action may be postponed for a week
 * get email notifications (if my post is used) - this call to action may be postponed for a week

Call to action selection criteria:
 * We will start the test in December with only the 'Edit' call to action for the first week, but expect to add more calls to action later on.
 * After a week, when we have enough edit conversion data, we will introduce more CTAs, on a random basis, with equal selection chances
 * If a page cannot be edited, we will display the 'get email notifications' CTA instead (or the survey CTA, if 'get email' CTA is not ready yet).

Edit this page


This is one of several possible calls to action to be displayed after the feedback had been submitted. We will start the test in December with only this call to action, but will plan to add more later on.

We are proposing a new design for this Edit CTA here, inspired by an earlier design by Brandon Harris.

Take a Survey


New image will be added later (thumbnail of current version is provided here for reference).

This call to action will be postponed for a week, so we can focus on the first call to edit in our first tests.

We will start with the current version of this CTA from AFT v4, with some changes to the wording of the survey, to reflect some of your new project goals and user interface.

Sign up or login
We will start with the current version of this CTA from AFT v4, with some minor changes to the wording.

Image to be added later on. This call to action will be postponed for a week, so we can focus on the first call to edit in our first tests.

Get email notifications
This CTA would only be displayed to anonymous users ("anons") or logged in members for whom we don't already have an email address. If such a user posts a comment, we will invite them to enter their email address, so we can notify them if their comment is used. We should also have a checkbox to make this message dismissable (e.g., "Don't show this again").

This is a new call to action, which doesn't already exist, though we may be able to adapt the post-survey CTA from AFT v4, with some minor changes to the wording. (We think the current CTA functionality allows dismissing, but only for a certain period of time.)

Image to be added later on. This call to action will be postponed for a week, so we can focus on the first call to edit in our first tests.

Other calls to action
Other optional calls to action we are considering for phase 2 include:


 * discuss this article
 * view feedback page
 * learn how to edit

Specs to be added later on, if we decide to include any of these in phase 1.5.



Placement


To increase feedback rates, we are discussing adding a 'Feedback' link higher up on the page. (see thumbnail to the right)

Feedback links on article pages
Here are some of the options we are considering for a feedback link higher up on the article pages:
 * Option A: add feedback link in the section bars (next to edit link)
 * Option B: add feedback link in the article title bar (top right corner of page)
 * Option C: add vertical button in the right margin of the browser window

Feedback links on talk pages
We are also discussing placement options for a new feedback link on article Talk pages. This shows three different ways in which a feedback link could be placed on the Talk page, linking to the feedback page where all reader posts are listed.

Overlay form
When users click on that feedback link, we would ideally like to display the feedback form as an overlay next to the link (as shown in the mockups -- see thumbnails to the right). If this proves difficult, a fall-back option is to simply include a jump link to the feedback form at the bottom of the page.



Feedback page


The shall display a list of the feedback items for an article, as shown in the preliminary wireframe image above (which doesn't show moderation tools).

Registered users will be able to see all the posts for this article, as well as sort or filter them, and vote on whether or not they are helpful. Editors and administrators will also be able to moderate the feedback page, and feature, hide or list feedback posts (see next section).

The actual nature of the comments we receive from readers (e.g., signal-to-noise ratio, what users actually comment on, etc.) will influence the design of the feedback page, so final development of the feedback page will likely occur once the first feedback forms have been tested.

the feedback page shall be linked to from the talk page. ''TBD: investigate if this can be implemented via hooks, and if no Mediawiki core changes are required for this. If core changes are required, the requirement may be changed/adjusted''.

The feedback page shall support navigation functionality for the comments list:
 * Filtering ('Show: ...') - predefined set of options:
 * All - unfiltered
 * With Comments (this would be default listing in phase 1.5)
 * Most Recent only
 * Top Rated only
 * Featured (phase 2)
 * Most Relevant (this would be default listing in phase 2 - show Featured first, then Top Rated with comments, then Most Recent with comments)
 * Option 1 form feedback only
 * Option 2 form feedback only
 * Option 3 form feedback only
 * Feedback flagged for abuse
 * Feedback by Editors/Admins only
 * Feedback by Registered Members
 * Feedback by Anonymous Users
 * Feedback posted on Talk page (phase 2)
 * Feedback that has been addressed (phase 2)
 * Feedback posted on Talk page (phase 2)
 * Feedback that has been addressed (phase 2)


 * Sorting ('Sort by: ...') - predefined set of options:
 * Date (similar to most recent, show the latest post first)
 * Article rating (if available, including Yes/No from option 1, rating from option 3, and average from current version)
 * Feedback rating (yes/no votes by other members rating this post)

The list would display up to 25? feedback posts by default (this number could be changed by Wikimedia based on user feedback, but no need to give user control of number of posts right now.) A "Show more" button at the end of the list shall load additional posts, same amount, and, again, display the "show more" button at the end of the list. The number of comments to display per page can be easily changed in code, for phase 1.

Each item in the feedback page list shall display:
 * Logged user name (or IP address) (link to user page)
 * Overall article rating (or 'Yes/No' answer: 'found what they were looking for' from option 3)
 * Timestamp (show 'x days, hours, minutes ago' -- all the way up to 48 hours, then just show date and time)
 * Comment (up to 500? words, then click 'More' to see the rest)
 * 'More' button showing feedback details (from expanded or current form, collapsed by default)
 * Vote on this feedback post, with prompt such as 'Is this feedback helpful?' with 'Yes/No' buttons (or up/down arrows = +1/-1), providing the ability to vote individual posts by other users up or down. Also need to show the number of up and down votes.
 * Flag as abuse - a text link for any user to flag offensive comments, spam, etc. Also need to show if abuse has been reported for this post and how many times.
 * Diff link (e.g.: '152 edits since this post' - links to a comparison page showing the difference between the article at the time the feedback was posted and its current version.)
 * Moderation tools in right margin (see below) - only visible to moderators/administrators. (see next section)

Nested comments - For future versions of this tool, this page's functionality shall support nested comments (that should be reflected both in the database schema and in code infrastructure), but this functionality should not be available in the interface for the first phase. The infrastructure should not impose limits on the depth of the comments tree. If such limits are to be introduced at later phases of the project, they shall be set in the code/configuration.

What's changed since last update:
 * we'd like to be able to filter that page by feedback form option, to show only posts from option 1, option 2, etc. (or use CSV data dumps for each article and option)
 * after 'Was this feedback helpful? Yes/No', need to show number of votes in parenthesis (with link that shows names of members who found it helpful or not)
 * say "Flag as abuse" (instead of "Report abuse")
 * show number of flags in parenthesis (with link that shows names of members who flagged it)
 * say 'Admin. Tools' (instead of 'Moderation tools') -- the word 'moderation' is not recommended for WIkipedia
 * say "Show this post" (instead of "List this post") for reverting an earlier decision to 'hide' a post -- in the administrative tools
 * say 'Modified by ' or 'Hidden by ' (instead of 'Last moderated by')

Admin. Tools


This administrator version of the includes admin./moderation tools, which would only be visible to editors and administrators.

These moderation tools would include these features for each post:
 * Hide (Trash icon?) - mark the post as hidden. (functionally, the comment is not deleted from the database, just not displayed unless a specific filter is selected, as well as including a 'Hidden' label next to post for editors.)
 * Show (for changing the status of 'featured' or 'hidden posts' back to normal 'listed' status)
 * Feature (Star icon?) - in phase 1 of the project this will promote the post to top of page, as well as include a 'Featured' label next to post. - Phase 2


 * Check-box to indicate important items that have been posted manually to talk page ([ ] 'Posted to talk page') - Phase 2
 * Check-box to indicate items that have been resolved (e.g.: '[  ]  Issue has been solved.') - Phase 2
 * Last modified by (showing last editor name and date)

Here is an older wireframe that shows together (only focus on tools in the right sidebar)

Expanded Feedback


For later stages of this project, we are considering an expanded feedback form, which could let readers suggest specific improvements, rate article qualities or share their expertise. The preliminary wireframe above assumes that an expandable/collapsable text link enables users to display this form ('Tell us more').

If time allows, we will try to include such an expanded form in phase 1.5. Otherwise, it will be addressed in phase 2 of this project.

Data collection
The feedback forms will collect a variety of data for later analysis, including:
 * Article title
 * User name (if logged in)
 * User IP address
 * Original post date/time stamp
 * Modified post date/time stamp
 * Feedback form version last used
 * Yes/No answer (if using option 1)
 * Overall rating (if using option 3 -- or average if using current form)
 * Detailed ratings (if using current or expanded form)
 * Comment (in its entirety)
 * Number of words in comment
 * Feedback type (suggestion, question, problem or praise, if using option 2)
 * Email address (if provided in email call to action)
 * Edit status (attempted or completed, indicating if user edited the page using option 4 or call to action)
 * User status (visitor, anonymous, registered, editor or admin at the time of first post)

This section will be expanded in a separate Metrics requirement page in coming days. For now, see Metrics outline on main AFT V5 page.