Article feedback/status

Last update on: 2013-01-monthly

2010-09-01
We’re in the beginning phase of this project, figuring out requirements and generally determining the scope of our near-term and long-term efforts in this area. We are currently working on a pilot rating system which will be available as part of the Public Policy pilot program in late September.

2010-10-01
Phase 1, deployed last week in a pilot experiment for the Public Policy Initiative. The goal is to gather metrics and learn about how this feature should work. We plan to use this information to help develop requirements for the next phase in the first quarter of 2011.

2010-11-01
The first version of this feature was rolled out at the end of September, and we spent the month of October collecting and analyzing the feedback from the tool. We are currently designing the second version of this feature, and starting development later this month. We plan to roll out the next phase in sync with the Public Policy Initiative Phase 2 in January.

2010-12-01
In development. The initial version (phase 1) has been in production since the end of September on English Wikipedia. Phase 2, focusing on usability improvements, performance improvements and wider deployment, is slated for release in January 2011.

2011-01-01
In development. The initial version (phase 1) has been in production since the end of September on English Wikipedia. Phase 2, slated for release in January 2011, has been under development. Planning for phase 3 will begin soon.

2011-02-01
Phase 2 is currently in development. The requirements have validated against the current codebase and UI. This feature will be dark launched on prototype on February 4. We'll wait until after the 1.17 deployment before making this feature live due to Resource Loader dependencies.

2011-03-01
The deployment to our prototype has surfaced additional feature requirements that we've now addressed. Now that MediaWiki 1.17 has been successfully deployed, we can release the latest version of the Article feedback tool on the English Wikipedia, as part of our pilot experiment this quarter. Requirements for the next version (3.0) are being drafted.

2011-04-01
The second phase of this feature was released on the English Wikipedia in mid March. A major change in the interface is the ability for reviewers to specify the source of their knowledge, e.g. if they have an academic degree in a related field (see screenshot). Experiments to encourage user engagement are being performed as well. Dario Taraborelli also published an analysis of the first phase experiment. We're currently expanding the scope of the experiment to include several thousand articles, in order to get results that are more meaningful statistically.

2011-05-01
<section begin=2011-05-01 /> <section end=2011-05-01 />
 * Phase 2: Trevor Parscal implemented the expiration of ratings, added error handling mechanisms and fixed IE bugs. Roan Kattouw reviewed and deployed the code to production.
 * Phase 3: Timo Tijhof and Trevor Parscal implemented the EmailCapture extension, a tool that allows unregistered users rating an article to leave their e-mail address if they want to be contacted later by the Community department. Weekly deployments were performed by Roan Kattouw. The team started to work on the dashboard, a summary page to surface general rating trends.

2011-06-01
<section begin=2011-06-01 />Version 3 was deployed to the English Wikipedia on May 9 with new features, like the article feedback dashboard, a summary page showing general rating trends. The experiment was expanded to 100,000 articles and may be expanded further after analysis of the results. The next version of the Article Feedback feature is currently in development.<section end=2011-06-01 />

2011-07-01
<section begin=2011-07-01 />Additional features were added in June, like a dashboard tracking articles receiving low ratings. Roan Kattouw started to implement the UDP back-end to provide clicktracking metrics to assess user engagement. The community provided feedback and bug reports, and the development team addressed the concerns raised, for example by implementing a user preference to hide the tool. Tooltips were also added to provide more information on the meaning of the star ratings. Dario Taraborelli continued to evaluate the data provided by the articles already showing the feature. The incremental roll-out to all articles on the English Wikipedia is planned to be completed by mid-July.<section end=2011-07-01 />

2011-08-01
<section begin=2011-08-01 />Roan Kattouw completed the UDP logger (for clicktracking metrics) and deployed it to production. The Article feedback feature was incrementally rolled out to all articles on the English Wikipedia, and the Product research team continued to analyze its impact (read more).<section end=2011-08-01 />

2011-08-31
<section begin=2011-08-31 />Development was completed in July; August was mostly devoted to data analysis. Dario Taraborelli analyzed the volume of edits, and couldn't yet find any statistically significant difference in edits before and after the activation of the feature on English Wikipedia articles. In order to clarify licensing and privacy policies regarding the data (and to facilitate its reuse by external tools & researchers), an explanation was published, stating that user feedback data (from Article feedback and MoodBar, for example) were considered public contributions just like any edit.<section end=2011-08-31 />

2011-09-30
<section begin=2011-09-30 />Dario Taraborelli continued to analyze the data recorded. Howie Fung assessed proposals from vendors in response to the RfP to build an extended review system, that would notably integrate free-text comments to the Article feedback feature.<section end=2011-09-30 />

2011-10-03
<section begin=2011-10-03 />Planning and design work begins on a new version 5 of the Article Feedback Tool. Fabrice Florin is engaged as product consultant to lead the development of AFT V5.<section end=2011-10-03 />

2011-10-31
<section begin=2011-10-31 />Work has started on Version 5 of this feature, which will focus on reader engagement, for example by offering new feedback forms and free-text comment fields; a comment moderation system is also planned. This project is led by contractor Fabrice Florin, and will be implemented by an external contracting company, OmniTI.<section end=2011-10-31 />

2011-11-30
<section begin=2011-11-30 />Fabrice Florin completed feature requirements, mockups and project plan for Version 5, and led development of first beta code by OmniTI. Dario Taraborelli started to work on the version 5 data model, metrics requirements and data analysis plan. He also worked with Fabrice and Howie Fung on guidelines and requirements for marking of AFT-generated edits and account tracking. A series of real-time dashboards for AFT data were uploaded to the Toolserver. Roan Kattouw reviewed the proposed database schema, and showed the OmniTI team how to deploy code to the prototype server.<section end=2011-11-30 />

2011-12-31
<section begin=2011-12-31/>Aaron Halfaker, Oliver Keyes and Dario Taraborelli have finished gathering valuable data from the community about the usefulness of comments coming in from each of the three forms launched in December. A survey to get comments from readers about the effectiveness and attractiveness of each design was also introduced, and the team has been compiling the various sets of data to produce a report on the pros and cons of each form. Fabrice Florin is leading development on the next round of features, including a new feedback page, to be implemented by OmniTI, our development partner. <section end=2011-12-31/>

2012-01-31
<section begin=2012-01-31/>Fabrice Florin led development on the next round of Article Feedback Tool v5 features, including a new feedback page, to be released in early February by OmniTI, our development partner. Aaron Halfaker, Oliver Keyes and Dario Taraborelli continued to collect valuable data from the community about the usefulness of comments coming in from each of the three forms launched in December. A survey to get comments from readers about the effectiveness and attractiveness of each design was also launched, and the team has been compiling the various sets of data to produce a report on the pros and cons of each forms in early February. The target date for an expanded feedback page is Feb. 15 for pre-deployment testing on en-labs, then wider deployment on Feb. 22.<section end=2012-01-31/>

2012-02-29
<section begin=2012-02-29/>Fabrice Florin is working with our development partner OmniTi on version 5 of this tool, with the help of community liaison Oliver Keyes and analyst Dario Taraborelli. This month, the team created a new feedback page, with special features to be tested with oversighters and rollbackers in early March. Roan Kattouw has completed code review for compatibility with 1.19 and will deploy a new release on March 8, as well as a new working test environment in coming weeks. Final reports for phase 1 of this project will be published in early March. <section end=2012-02-29/>

2012-03-31
<section begin=2012-03-31/>Fabrice Florin worked with OmniTi to develop new features for version 5 of the Article Feedback Tool (AFT5). This month, the team created new feedback links, new monitoring tools for editors and for oversighters, and started development on an abuse filter and a relevance filter. Brandon Harris and Heather Walls helped enhance the article feedback design, creating new icons for the monitoring tools. The AFT5 team also published its first report on phase 1 of AFT5, co-authored by Fabrice and Dario Taraborelli, with Howie Fung, Oliver Keyes and Aaron Halfaker. Roan Kattouw helped solve some tricky technical issues and deployed several new releases with the team this month. Current goals for this project are to complete feature development by the end of May, with full deployment in the summer. <section end=2012-03-31/>

2012-04-monthly
<section begin=2012-04-monthly/>Fabrice Florin worked with OmniTi to develop a range of new features for version 5 of the Article Feedback Tool (AFT5). This month, the team deployed the first versions of the abuse filter and the relevance filter, as well as new monitoring tools for editors, to help surface useful suggestions and reduce the noise on the feedback page where posts are listed for each article. Pau Giner started designing a new look and feel for that feedback page, as well as a final version of the feedback form, with guidance from Brandon Harris. We also finalized a set of special tools for oversighters, as well as new feedback links. Dario Taraborelli, Oliver Keyes and Aaron Halfaker collected and analyzed data on how prominent feedback links impact both volume and quality of user feedback. Roan Kattouw continued to review our code and help deploy weekly releases, and Matthias Mullie contributed new code for this project. We expect to complete feature development by the end of May, with full deployment in the summer.<section end=2012-04-monthly/>

2012-05-monthly
<section begin="2012-05-monthly"/>Fabrice Florin worked with OmniTI and new WMF engineer Matthias Mullie to develop a range of new features for version 5 of the Article Feedback Tool (AFT5). This month, the team deployed a new look and feel for the article feedback page (based on a streamlined design by Pau Giner) as well as a central feedback page, where editors can monitor posts from all articles on Wikipedia. We also developed a [//en.wikipedia.org/wiki/San_Francisco?aftv5_form=6 final feedback form] (scroll to bottom of page), which gradually engages users to contribute to the encyclopedia. Dario Taraborelli, Oliver Keyes and Aaron Halfaker collected and analyzed data on how posting feedback impacts both conversion and newcomer quality. Based on this analysis, we now project over 2 million feedback posts per month on the English Encyclopedia when the tool is widely deployed later this year (on par with the total number of edits per month). Our research suggests that posting feedback encourages a substantial number of users to productively edit articles on Wikipedia, which is expected to help reverse the recent decline in both new and existing editors. Roan Kattouw continued to review our code and deploy weekly releases, while training our team to deploy their own code over time. We are planning for a wider deployment by the end of June, with full deployment a couple months later.<section end="2012-05-monthly"/>

2012-06-monthly
<section begin="2012-06-monthly"/>Fabrice Florin worked with new WMF engineer Matthias Mullie and OmniTI to develop a range of new features for version 5 of the Article Feedback Tool (AFT5). This month, the team completed primary feature development for this tool, including the article feedback page, the central feedback page, and the [//en.wikipedia.org/wiki/San_Francisco?aftv5_form=6 final feedback form] (scroll to bottom of page). We started writing and publishing new documentation about this project, including this help page. Dario Taraborelli, Aaron Halfaker and Oliver Keyes published a full report that suggests that people who post feedback are more likely to edit articles afterwards. Roan Kattouw continued to review our code and trained our team to start deploying code on their own. We have started a wider deployment of AFT, which will gradually increase our coverage to 10% of the English encyclopedia by the end of July, with full deployment a couple months later.<section end="2012-06-monthly"/>

2012-07-monthly
<section begin="2012-07-monthly"/>Fabrice Florin and Matthias Mullie led the deployment of Article Feedback on 10% of the English Encyclopedia, in collaboration with Pau Giner, Ryan Kaldari, Roan Kattouw, Oliver Keyes, Chris McMahon, Benny Situ, Heather Walls, Howie Fung and Terry Chay. This month, the team developed final features for this tool, including the article feedback page, the central feedback page, and the [//en.wikipedia.org/wiki/San_Francisco?aftv5_form=6 final feedback form] (scroll to bottom of page). To guide users of this tool, we also published a new video tour, a walkthrough tutorial and various help pages. We have received a very positive response to article feedback from the Wikipedia community through a variety of channels, from talk pages to IRC chats and Wikimania presentations. Community members typically find the tool useful and well-thought out, and many editors have told us they have already made improvements to articles based on feedback from readers — which is exactly the behavior we were hoping to encourage. We have started our productization phase (more platforms, scalability, code re-factoring, localization, metrics, mobile). We are now aiming for a full release to 100% of English Wikipedia by October 2012 — with other wiki projects starting later this year. <section end="2012-07-monthly"/>

2012-08-monthly
<section begin="2012-08-monthly"/>This month, we developed a range of new features for Article Feedback, which is now deployed on 10% of the English Encyclopedia. Improvements include the ability to view feedback from my watched pages, hide my posts, give feedback on help pages — as well as enable editors to clear all flags and administrators to protect articles to limit feedback on controversial pages. These and other features can be tested on this sample article feedback page or on the central feedback page (please report any bugs on Bugzilla). For more information about this tool, check our project overview. We are now in our productization phase (support for more platforms, scalability, code re-factoring, localization, metrics, mobile) and are aiming for a full release to 100% of English Wikipedia by the end of October 2012 — with other wiki projects starting later this year. This new tool was created in collaboration with the Wikipedia community and developed by Fabrice Florin, Matthias Mullie, Pau Giner, Ryan Kaldari, Roan Kattouw, Oliver Keyes, Chris McMahon, Benny Situ, Dario Taraborelli, Howie Fung and Terry Chay, in association with OmniTI. <section end="2012-08-monthly"/>

2012-09-monthly
<section begin="2012-09-monthly"/>This month, we deployed more new features for Article Feedback, which is still being tested on 10% of the English Wikipedia. Improvements include iPad support, updating special abuse filters to automatically disallow swear words and common vandalism, as well as developing new automated filtering features to reduce the workload for editors, administrators and oversighters. These and other features can be tested on this sample article feedback page or on the central feedback page (please report any bugs on Bugzilla). We are also in the process of re-factoring our code and 'sharding' our database to make this extension more scalable, prototyping a mobile web interface, as well as collecting more data to track how many readers who post feedback end up becoming editors as a result. As we complete these final improvements and confirm that this tool is converting readers into editors as intended, we are preparing for a full release to 100% of English Wikipedia in coming weeks — with other wiki projects starting later this year. For more information about this tool, check our project overview. This new tool was created in collaboration with the Wikipedia community and developed by Fabrice Florin, Matthias Mullie, Pau Giner, Ryan Kaldari, Oliver Keyes, Chris McMahon, Benny Situ, Dario Taraborelli, Howie Fung and Terry Chay. <section end="2012-09-monthly"/>

2012-10-monthly
<section begin="2012-10-monthly"/>This month, we developed a few final features for Article Feedback, which is being tested on 10% of the English Wikipedia. Though our lead developer was loaned out to the WikiVoyage project for most of the month, we deployed a couple improvements to reduce the moderation workload for editors, including: better abuse filters to automatically disallow swear words; new ways for moderators to filter the feedback page; and a check list to help users request oversight. These and other features can be tested on this sample article feedback page or on the central feedback page (please report any bugs on Bugzilla). We also started to collect new research data to track how moderators use the feedback page, as well as measure how many readers who post feedback become editors or registered users. Next month, we plan to make a few final improvements to address these findings, as well as complete code re-factoring to improve database performance. Once these tasks are done in coming weeks, we expect to release Article Feedback v5 to 100% of the English Wikipedia by the end of the year, and to other projects in 2013. For more information about this tool, check our project overview.<section end="2012-10-monthly"/>

2012-11-monthly
<section begin="2012-11-monthly"/>This month, we continued to develop final features for Article Feedback, and researched how people are using this tool on the English Wikipedia. With the help of community members, we designed new features to reduce the editor workload, including improved moderation tools and a more prominent feedback link. These features will be developed next month, once we've completed code re-factoring to improve database performance. We also analyzed new research data to track how moderators use the feedback page, and measure how many readers who post feedback become editors or registered users. Next month, we will invite Wikipedians to evaluate the usefulness of feedback posts and the effectiveness of our new moderation tools. Once these tasks are done, we plan to release Article Feedback v5 to 100% of the English Wikipedia in early 2013. For more information about this tool, check our project overview.<section end="2012-11-monthly"/>

2012-12-monthly
<section begin="2012-12-monthly"/>We made good progress on Article Feedback version 5 this month. We completed a research study on the English Wikipedia, confirming that many readers use this feature and a sizable number of them go on to register and become editors. Based on that research and editor suggestions, we started development on new features to reduce the editor workload through better filters and simpler moderation tools. We also continued to refactor our code, to support millions of comments on a dedicated database cluster to be deployed in coming months. Once this work is complete, we plan to release Article Feedback v5 to 100% of the English Wikipedia in March, and to other Wikimedia sites later this year. The German Wikipedia has already started a pilot to evaluate this tool, and a similar initiative is also under discussion on the French Wikipedia.<section end="2012-12-monthly"/>

2013-01-monthly
<section begin="2013-01-monthly"/>This month, our team prepared an updated version of Article Feedback version 5 (AFT) and discussed its release with communities in the English, French and German Wikipedias. Lead developer Matthias Mullie completed a major refactoring of the AFT code, which is now being reviewed by Terry Chay, Luke Welling, S Page and Asher Feldman, among others. He is now completing development on a final set of new features, including simpler moderation tools and better filters, to be tested and deployed next month. Dario Taraborelli and Aaron Halfaker presented the results on their feedback evaluation with experienced Wikipedians, which suggests that about 39% of the feedback collected on the English Wikipedia can be useful for improving articles (see also their recent research study results). Oliver Keyes responded to community questions in a request for comments about whether this tool should be deployed widely on the English Wikipedia, with a final decision expected next month. Fabrice Florin led product planning for this tool and continued to discuss its possible deployment on the French Wikipedia (which is now wrapping up its community vote) -- and with the German Wikipedia, which is evaluating the tool in this ongoing pilot, with a vote expected in May. Once our development is complete and as each project reaches a decision, we plan to release Article Feedback v5 on these and other Wikimedia sites in coming months. For more info about Article feedback, check our recent blog post. <section end="2013-01-monthly"/>