Article feedback/Version 5

This is the project hub page for Version 5 of the Article Feedback Tool. The Wikimedia Foundation is developing this new feature as an "on-ramp" to engage readers to contribute to Wikipedia -- and become editors over time.

Here’s how you can learn more about Article Feedback v5:
 * watch this quick video tour
 * read our walkthrough tutorial
 * check our help page

You are invited to try out Article Feedback now, on one of these pages:
 * article feedback form
 * article feedback page
 * central feedback page
 * more articles with feedback

Related Pages

 * Blog post on Article Feedback V5 Deployment
 * Join the discussion on the Article Feedback Talk page
 * Share this link with your community: Wikipedia article on AFT5
 * Useful links to prototypes and reports
 * Feature Requirements
 * Metrics Dashboard
 * Data and Metrics Plan
 * Technical Design

Goals
In October 2011, Wikimedia kicked off another round of product development on new and alternative methods of providing feedback regarding the quality of articles, including ideas like a moderated free-text comment queue for suggestions. To that end, the foundation hired Fabrice Florin as product manager for new editor engagement, as well as OmniTI in Maryland for the software development of these new features.

The overall goal of this project is to find ways for readers to contribute productively to building the encyclopedia. Version 5 will also continue to test different methods for measuring quality, through both implicit and explicit methods. We invite the Wikipedia community (as well as all Wikimedians), to participate in this experiment. Together, we hope to create and test new collaborative tools towards these objectives:
 * engage readers to participate more on Wikipedia
 * give editors new tools to improve article quality
 * encourage readers to become editors over time
 * invite a collaboration between editors and readers
 * experiment with outsourcing web development

The first implementations of the Article Feedback Tool (Versions 1-4) were focused on the dual objectives of participation and quality. A new Version 5 of the Article Feedback Tool (AFT V5) is now being tested, to take into account both community feedback and data analysis for the current version (AFTv4). The main goal is to find ways for readers to contribute productively to building the encyclopedia. Version 5 will also continue to test different methods for measuring quality, through both implicit and explicit methods.

For Version 5, the Foundation is replacing the current ratings system with "comment" boxes, to invite constructive suggestions for improvement, rather than judgmental ratings. A reader would be able to provide useful feedback (such as "you're missing a section on that person's childhood") that editors could use to improve the article (as opposed to simply rating the article "3 stars" for completeness). This would allow readers to better engage with editors, who could then actively improve articles if they find the feedback helpful.

The first option that was tested for Version 5 (see image) was a simple feedback form that asks the readers: "Did you find what you were looking for?", with 'Yes' or 'No' buttons. Readers are then invited to write a comment and suggest improvements to this article. This approach may provide implicit measurements of quality that could be as effective as explicit ratings in the current form. For example, with the "Did you find what you're looking for" question, the percentage of "yes"'s could be an indicator of article quality.

Another feedback form that was tested for V5 enabled users to post different types of feedback: make a suggestion, ask a question, report a problem or give praise to the authors. A third option was also tested, which allowed users to rate articles and add a comment. Test results showed a slight overall preference for Option 1 (even though the other two options were also found useful). For that reason, the first option was selected for the next phase of development and new variations of that feedback form will undergo further testing. To learn more, read the first report about the first phase of this experiment.

These new article feedback forms are being tested on a small group of randomly-selected articles (now 5% percent of the English Encyclopedia). The Wikimedia Foundation is collecting detailed metrics for each of these feedback formats and is working with the community to evaluate their effectiveness. As usual, the relevant data will be made public so that anyone can dig into the numbers.

A feedback page lists all the user posts for each article. All users have access to this feedback page, and can filter or sort that list in a variety of ways. Experienced editors and administrators have the option to feature posts more prominently, or hide offensive posts. Tools enabling the community to mark posts as helpful are also be available, along with the ability to flag abuse. The actual nature of the comments we received from readers (e.g., signal-to-noise ratio, what users actually comment on, etc.) has influenced the design of the feedback page, as well as recommendations from the editor community.

For a preview of what some of these features might look like, check the for Version 5.

Features


Key features for AFT V5 include: (see also full feature requirements page)
 * new feedback forms
 * calls to action
 * feedback page
 * monitoring tools

In the first phase of this project (Dec.-Feb. 2011), we developed and started testing three different feedback forms to extend the current rating tool: We tested these options to find out which is most effective for engaging readers, supporting editors and improving article quality (see [our report on first test results).
 * Option 1: Did you find what you were looking for?
 * Option 2: Make a suggestion
 * Option 3: Review this page

In the next phases of this project, we developed and tested these additional options: In the July 2012, we started to release some of these new features more widely on Wikipedia, and plan to deploy to all of the English Wikipedia in September 2012, with deployment to other Wiki projects later in the year.
 * Feedback links
 * New calls to action
 * A stand-alone call to edit
 * Feedback page

To see what all these features look like, check the links at the top of this page.

Get involved
We are developing this new tool in collaboration with Wikipedia editors, with whom we meet regularly over IRC and other channels. We are looking for more volunteers to help improve this article feedback tool.

Talk page
You're invited to give feedback about the Foundation's current plans for Version 5 and let us know what you think on the Talk page: what community concerns aren't being considered? Are there flaws in the current plans? How would you make the current ideas better? do you have any of your own to share? This is open to everyone - just drop your thoughts on the Talk page.

IRC chats
To invite community participation during the development of this tool, the Foundation also hosts frequent IRC chats during office hours, as outlined in the schedule below. We hope you will join us on future chats. In the meantime, here are logs of our earlier IRC chats on Oct. 27, Nov. 3, Nov. 10, Nov. 18 and Dec. 16, 2011 -- Jan. 6, Jan. 13, Jan. 20 , Jan. 27 and Feb. 16, 2012.

Feedback Evaluation
The Foundation is looking for experienced editors to help evaluate the quality of the feedback collected by the new AFT5 tools in 2012. To learn more about this project, visit the Feedback evaluation page. A special task force of editors is being trained to review feedback posts for this evaluation.

Schedule
The first phase of Version 5's development is timetabled below; those segments marked in green are the ones where we will be asking for community feedback and participation. These dates are still tentative and subject to change, so please check this page often for schedule updates. Even if you cannot participate right away, there will be more opportunities to contribute in future stages of the project. All dates above subject to change, based on agile development goals as well as Wikimedia's overall plans for this project.

Weekly Update
Last week: (July 9)

Development:
 * Deployed to 3% of English Wikipedia
 * Updated and deployed My Contributions
 * Updated and deployed My Last Post for all CTAs
 * Updated and deployed activity log (now shows helpful tags, color-coded)
 * Developed and tested IE7 fixes and deployed IE8 update
 * Investigated Oversighter email fix (dependent on Ops)
 * Deployed more features and bug fixes based on community/testing feedback
 * Worked with Chris and Jeremy on unit testing and quality assurance
 * Wrote list of bugs and feature requests following outreach

Outreach:
 * Continued wider socialization plan, coordinated tasks and schedule with other teams
 * Completed, published and promoted short blog post about wider deployment
 * Completed, published and promoted video tour (including YouTube and Vimeo)
 * Completed and published tutorial walkthrough and landing page (redirected WP:AFT to it)
 * Updated copy for project hub pages on en-wiki (WP:AFT5) and on mediawiki
 * Completed and published AFT5 slides -- also used for Wikimania presentation
 * Presented AFT5 findings at Wikimania, and discussed it with many stakeholders
 * Discussed internationalization requirements with localization team
 * Discussed local version of AFT5 with French and German Wikipedia organizers

This week: (July 16)

Development:
 * Deploy to 5% of English Wikipedia
 * Develop, test (and deploy?) IE7 fixes and other configuration issues
 * Develop, test (and deploy?) Oversighter email fix (dependent on Ops)
 * Develop and test overlay replacement for toolbar 'Improve this page' link
 * Update Abuse filters to disallow obscenities (instead of auto-flag), test warning message
 * Investigate, design, develop and test more features and bug fixes based on feedback from outreach / Wikimania
 * Address requests from internationalization team that are easy to do right away
 * Continue work with Chris and Jeremy on unit testing and quality assurance
 * Continue work on scalability and productization plan with engineering team

Outreach:
 * Continue wider socialization plan (central notice, IRC chats, talk page)
 * Deploy central notice banner, linking to landing page
 * Hold 4+ IRC chats this week (Wed. 11am PT + 3 x chats on Friday)* Participate on talk page discussions, respond
 * Follow up on Wikimania discussions, respond to questions, suggestions
 * Update copy for current WP:AFT and related pages on en-wiki / mediawiki
 * Follow up on discussions with localization team, as well as French and German Wikipedia organizers
 * Fabrice will be taking two days off this week (Thursday & Friday)

Q3 Timeline: Here's our timeline for AFT deployments this summer:
 * Tuesday 7/17 to 5% of en-wiki
 * Tuesday, 7/24 to 10% of en-wiki
 * Productization through the summer
 * Deploy to 100% of en-wiki in mid-September
 * Facilitate adoption by other Wiki projects in Q4

All dates above subject to change, based on agile development goals as well as Wikimedia's overall plans for this project.

Metrics
AFT v.5
 * For live results of our current tests, check the metrics dashboard and feedback stream, as well as data dumps.
 * Metrics and research questions that will be used to test AFT v.5, as well as a detailed plan with the breakdown of the different tests that we will run, can be found on this page.

AFT v.4
 * We collected a number of high-level usage metrics (November 2011) from the current version of AFT as a baseline before starting the deployment of AFT v.5.
 * Several dashboards with real-time data collected from AFT v.4 are available from the toolserver:
 * Global daily ratings and conversions
 * Volume of daily ratings per article
 * Most frequently rated articles
 * Detailed reports from the analysis of data collected via AFT v.4 are available on this page.

Team
Here are the Wikimedia team members who are working on this project at this time:
 * VP Engineering / Product	Erik Moeller
 * Senior Product Manager	Howie Fung
 * Product Manager, Editor Engagement	Fabrice Florin (project lead and primary author of this page}
 * Features Engineering Directors	Terry Chay, Alolita Sharma
 * Senior Research Analyst	Dario Taraborelli
 * Senior Designer	Brandon Harris
 * UX Designer Pau Giner
 * Developer Matthias Mullie
 * Code Review / Deployer	Roan Kattouw
 * Community Liaison 	Oliver Keyes
 * Research Consultant Aaron Halfaker

(note that Oliver and Aaron are Wikimedia contractors -- all others are WMF employees)

Developers Here are the team members from OmniTI, our development partner:
 * VP Business Development  Leon Fayer
 * Project Manager  Yoni Shostak
 * Developer  Reha Sterbin
 * Developer  Greg Chiasson
 * UI Developer/Designer  Sean Heavey

Community members Here are some of the Wikipedia community members who have joined our workgroup to help design this product together:
 * Bensin
 * Dougweller
 * Looie496
 * RJHall
 * Utar
 * and many more

We’d like to give special thanks and recognition to these community members who have already done so much to develop this new tool: we are grateful for your insights and commitment to this cause!