Article feedback/Version 5

This is the project hub page for Version 5 of the Article Feedback Tool. The Wikimedia Foundation is developing this new feature as an "on-ramp" to engage readers to contribute to Wikipedia -- and become editors over time.

Version 5 of this Article Feedback Tool (AFT5) is now being tested on 10% of the English Wikipedia, towards a full release in the first quarter of 2013. For an update on this project, check out this blog post, which includes recent research findings — as well as an update on final features in development for this reader engagement tool.

Here’s how you can learn more about Article Feedback v5:
 * watch this quick video tour
 * read our walkthrough tutorial
 * check our help page

You are invited to try out Article Feedback now, on one of these pages:
 * article feedback form
 * article feedback page
 * central feedback page
 * more articles with feedback

Related Pages

 * Blog post on Article Feedback V5 Next Steps
 * Join the discussion on the Article Feedback Talk page
 * Share this link with your community: Wikipedia article on AFT5
 * Useful links to prototypes and reports
 * Feature Requirements
 * Metrics Dashboard
 * Data and Metrics Plan
 * Technical Design

Project update
Here's our latest update on Article Feedback version 5:

Goals
In October 2011, Wikimedia kicked off another round of product development on new and alternative methods of providing feedback regarding the quality of articles, including ideas like a moderated free-text comment queue for suggestions. To that end, the foundation hired Fabrice Florin as product manager for new editor engagement, as well as OmniTI in Maryland for the software development of these new features.

The overall goal of this project is to find ways for readers to contribute productively to building the encyclopedia. Version 5 will also continue to test different methods for measuring quality, through both implicit and explicit methods. We invite the Wikipedia community (as well as all Wikimedians), to participate in this experiment. Together, we hope to create and test new collaborative tools towards these objectives:
 * engage readers to participate more on Wikipedia
 * give editors new tools to improve article quality
 * encourage readers to become editors over time
 * invite a collaboration between editors and readers
 * experiment with outsourcing web development

The first implementations of the Article Feedback Tool (Versions 1-4) were focused on the dual objectives of participation and quality. A new Version 5 of the Article Feedback Tool (AFT V5) is now being tested, to take into account both community feedback and data analysis for the current version (AFTv4). The main goal is to find ways for readers to contribute productively to building the encyclopedia. Version 5 will also continue to test different methods for measuring quality, through both implicit and explicit methods.

For Version 5, the Foundation is replacing the current ratings system with "comment" boxes, to invite constructive suggestions for improvement, rather than judgmental ratings. A reader would be able to provide useful feedback (such as "you're missing a section on that person's childhood") that editors could use to improve the article (as opposed to simply rating the article "3 stars" for completeness). This would allow readers to better engage with editors, who could then actively improve articles if they find the feedback helpful.

The first option that was tested for Version 5 (see image) was a simple feedback form that asks the readers: "Did you find what you were looking for?", with 'Yes' or 'No' buttons. Readers are then invited to write a comment and suggest improvements to this article. This approach may provide implicit measurements of quality that could be as effective as explicit ratings in the current form. For example, with the "Did you find what you're looking for" question, the percentage of "yes"'s could be an indicator of article quality.

Another feedback form that was tested for V5 enabled users to post different types of feedback: make a suggestion, ask a question, report a problem or give praise to the authors. A third option was also tested, which allowed users to rate articles and add a comment. Test results showed a slight overall preference for Option 1 (even though the other two options were also found useful). For that reason, the first option was selected for the next phase of development and new variations of that feedback form will undergo further testing. To learn more, read the first report about the first phase of this experiment.

These new article feedback forms are being tested on a small group of randomly-selected articles (now 5% percent of the English Encyclopedia). The Wikimedia Foundation is collecting detailed metrics for each of these feedback formats and is working with the community to evaluate their effectiveness. As usual, the relevant data will be made public so that anyone can dig into the numbers.

A feedback page lists all the user posts for each article. All users have access to this feedback page, and can filter or sort that list in a variety of ways. Experienced editors and administrators have the option to feature posts more prominently, or hide offensive posts. Tools enabling the community to mark posts as helpful are also be available, along with the ability to flag abuse. The actual nature of the comments we received from readers (e.g., signal-to-noise ratio, what users actually comment on, etc.) has influenced the design of the feedback page, as well as recommendations from the editor community.

For a preview of what some of these features might look like, check the for Version 5.

Features


Key features for AFT V5 include: (see also full feature requirements page)
 * new feedback forms
 * calls to action
 * feedback page
 * monitoring tools

In the first phase of this project, we developed and tested a variety of different features, including: In July 2012, we started to release some of these new features more widely on Wikipedia, and plan to deploy to all of the English Wikipedia in October 2012, with deployment to other Wiki projects later in the year.
 * Option 1: Did you find what you were looking for?
 * Feedback links
 * New calls to action
 * A stand-alone call to edit
 * Feedback page

To see what all these features look like, check the links at the top of this page.

Get involved
We are developing this new tool in collaboration with Wikipedia editors, with whom we meet regularly over IRC and other channels. We are looking for more volunteers to help improve this article feedback tool.

Talk page
You're invited to give feedback about the Foundation's current plans for Version 5 and let us know what you think on the Talk page: what community concerns aren't being considered? Are there flaws in the current plans? How would you make the current ideas better? do you have any of your own to share? This is open to everyone - just drop your thoughts on the Talk page.

IRC chats
To invite community participation during the development of this tool, the Foundation also hosts frequent IRC chats during office hours, as outlined in the schedule below. We hope you will join us on future chats. In the meantime, here are logs of our earlier IRC chats on Oct. 27, Nov. 3, Nov. 10, Nov. 18 and Dec. 16, 2011 -- Jan. 6, Jan. 13, Jan. 20 , Jan. 27 and Feb. 16, 2012.

Metrics


AFT v.5
 * For live results of our current tests, check the metrics dashboard and feedback stream, as well as data dumps.
 * Metrics and research questions that will be used to test AFT v.5, as well as a detailed plan with the breakdown of the different tests that we will run, can be found on this page.

AFT v.4
 * We collected a number of high-level usage metrics (November 2011) from the current version of AFT as a baseline before starting the deployment of AFT v.5.
 * Several dashboards with real-time data collected from AFT v.4 are available from the toolserver:
 * Global daily ratings and conversions
 * Volume of daily ratings per article
 * Most frequently rated articles
 * Detailed reports from the analysis of data collected via AFT v.4 are available on this page.

Schedule
Version 5's development milestones are outlined below. For more details, check this timeline. These dates are tentative and subject to change, so please check this page often for schedule updates.

All dates above subject to change, based on potential bugs or slowdowns and Wikimedia's overall plans for this project.

Team
Here are the Wikimedia team members who are working on this project at this time:
 * VP Engineering / Product	Erik Moeller
 * Senior Product Manager	Howie Fung
 * Product Manager, Editor Engagement	Fabrice Florin (project lead and primary author of this page}
 * Features Engineering Directors	Terry Chay, Alolita Sharma
 * Senior Research Analyst	Dario Taraborelli
 * Senior Designer	Brandon Harris
 * UX Designer Pau Giner
 * Lead Engineer Matthias Mullie
 * Code Review / Deployer	Roan Kattouw
 * Community Liaison 	Oliver Keyes
 * Research Consultant Aaron Halfaker

(note that Oliver and Aaron are Wikimedia contractors -- all others are WMF employees)

The first phases of development for AFT5 were outsourced to OmniTI, our development partner. Their team included Leon Fayer, Yoni Shostak, Reha Sterbin, Elizabeth Smith, Greg Chiasson, Mike Jackson and Sean Heavey.

Community members Here are some of the Wikipedia community members who have joined our workgroup to help design this product together:
 * Bensin
 * Dougweller
 * Looie496
 * RJHall
 * Utar
 * and many more

We’d like to give special thanks and recognition to these community members who have already done so much to develop this new tool: we are grateful for your insights and commitment to this cause!