Article feedback/Version 5/Release Plan 2013
Here is our proposed 2013 release plan for these article feedback tools on Wikipedia:
- Article Feedback v5 project (AFT5)
- Article Feedback v4 project (AFT4)
- ReaderFeedback (an earlier feedback tool)
In coming weeks, the Wikimedia Foundation's WMF Editor Engagement team plans to:
- deploy an updated version of AFT5 for testing on English, French and German Wikipedias
- support the AFT5 tool during these limited 3-6-month pilots in 2013
- support a full deployment of AFT5 on these sites, if requested by their communities
- discuss more AFT5 releases on other sites, based on the success of the first pilots
- encourage upgrades to AFT5 for sites using older AFT4 and ReaderFeedback tools
Our overall goals for Article Feedback v5 in 2013 are to:
- engage readers to contribute productively to Wikipedia
- help editors improve articles based on reader feedback
- test this tool in multiple languages and regions
- help projects deploy this tool more widely
We will first remove AFT4 and AFT5 lottery articles on English Wikipedia, at the community's request. We will then deploy and test a new version of Article Feedback v5 on three large projects: English, French and German Wikipedias. If these limited pilots are successful, we expect to support more projects that wish to use to AFT5 in the second half of 2013.
We propose to accomplish these goals in these incremental phases:
Phase 1: First pilots
- test and debug new features and database cluster for AFT5
- switch to AFT5 'opt-in' version and remove AFT4 on English Wikipedia
- deploy the updated version of AFT5 on the English, French and German Wikipedias
- test this new software in limited pilots throughout 2013
- monitor community discussions about wider releases of AFT5 on their sites
- support wider releases if requested by communities (or remove software after pilots end)
- start discussions with more international projects interested in AFT5
Phase 2: More projects
- after a full release on a phase 1 project, select a second group of projects that have reached consensus about adding AFT5 on their sites
- priority will be given to projects that have old software that can be replaced by AFT5 (AFT4 or ReaderFeedback)
- support the release of AFT5 for that second group of international sites
Phase 1 is already under way and is expected to continue until Q4 2013, when the French Wikipedia plans to vote on a full release. Phase 2 is likely to start in Q3 2013 if the German Wikipedia votes for a full release in May 2013 -- or in Q4 2013 if the French Wikipedia votes for a full release in September 2013.
Note that we will support these first pilots with bug fixes and simple feature tweaks, but are not planning to develop any more new features this year. Based on the outcome of the first two phases, we may consider providing a turnkey version of Article Feedback (with full documentation, so sites can deploy on their own, with minimal support from us. This could enable wider releases on a third group of international sites, using such a turnkey version. In the meantime, projects are welcome to enable AFT5 on their own, once the tool has been tested for a month in the first pilots, but we recommend that they contact us with an estimate of how many comments they expect to generate per month, as described in the first paragraph of the 'Checklist' section below.
This project timeline provides more details when we expect to complete these goals in this fiscal year.
For this 2013 plan, we will discuss AFT5 releases with different groups of projects:
- Large Wikipedia projects (e.g. English, French and German Wikipedias)
- Projects with old software (AFT4 or ReaderFeedback)
- Other projects (e.g. Wikivoyage, Wikinews)
Here is more information about the first projects which we plan to support for phase 1 of this release plan: English, French and German Wikipedias.
The English Wikipedia community recently held a Request for Comments (RfC) about the future of AFT5 on their site. They support continued experimentation with this tool on an opt-in basis, but are not ready for a full roll-out of the current versions at this time. At the community's request, we plan to remove feedback forms on all articles with AFT4 installed, as well as articles selected by lottery for AFT5 (which will be considerably less than the existing level of deployment). Editors who wish to get feedback for their articles can do so by simply adding this special 'Category:Article_Feedback_5' category on those pages. We will support these editors and monitor the usage of this tool for the rest of 2013. Our goal is to share our findings with the community later in the year, and discuss at that time whether to remove this tool entirely -- or continue to make it available beyond 2013.
The French Wikipedia community recently held a about adding AFT5 on their site. They decided to test the tool on a limited basis during a 6-month pilot ending in September 2013, then vote again about a full release on their entire site. As a result, we plan to release the software on 42,000 articles selected by the community in early April 2013. We will provide limited support during the pilot -- and help with a wider deployment if the community decides to do this in the fall.
The German Wikipedia community held a about adding AFT5 on their site in fall 2012. They decided to test the tool on a limited basis during a 4-month pilot ending in May 2013, then vote again about a full release on their entire site. As a result, we released on 14,000 articles selected by the community last December, and plan to update this pilot with new features in March 2013. We will provide limited support during the pilot -- and will help with a wider deployment if the community votes to do this in June 2013.
If these first pilots are successful, we will consider supporting a second group of projects, particularly if they are using old software that could be replaced with AFT5, as outlined below.
Projects with AFT4:
- Chinese Wikipedia (AFT4)
- English Wikipedia (AFT4)
- Hindi Wikipedia (AFT4)
- Hungarian Wikipedia (AFT4, consensus reached)
- Portuguese Wikipedia (AFT4)
- Serbian Wikipedia (AFT4)
- Spanish Wikipedia (AFT4)
- Spanish Wikinews (AFT4)
- Meta Wiki (AFT4)
Projects with ReaderFeedback:
- Chinese Wikipedia (ReaderFeedback)
- English Wikibooks (ReaderFeedback)
- English Wikinews (ReaderFeedback)
- Russian Wikinews (ReaderFeedback)
- Turkish Wikinews (ReaderFeedback)
Other interested projects:
- Kannada Wikipedia (consensus reached)
- Swedish Wikipedia (help pages)
- Wikinews (ru, en)
- Wikivoyage (en, it)
Here are key milestones we are considering in coming weeks, for discussion purposes:
Week of March 4:
- remove AFT4 on the English Wikipedia
- remove feedback from AFT5 lottery articles
- switch AFT5 to 'opt-in' (show feedback for 'Article Feedback 5' category)
Week of March 11:
- deploy an updated version of the tool on a new database cluster for both English and German Wikipedias
- discuss decisions about Article feedback settings (e.g.: inappropriate posts, feedback link, abuse filter)
Week of March 18:
- deploy Article Feedback on a few articles on the French Wikipedia (for a technical test for the first two weeks).
- test the new version with a small group of volunteers on English, French and German Wikipedias + fix bugs
Week of March 25:
- deploy and test bug fixes with a larger group of volunteers
- finalize decisions about Article feedback settings for each wiki.
Week of April 1:
- deploy a final release version on English, French and German Wikipedias
- increase the pilot footprint on the French Wikipedia to 42,000 articles
- publish a blog post about these new releases
- socialize all 3 releases with a broader community
- track and study feedback metrics
Here are the tasks which need to be completed in coming weeks, listed in rough chronological order:
- complete code review
- make any required code changes
- update help pages
- remove AFT4 tool
- remove AFT5 tool for 'lottery pages'
- enable AFT5 for 'Article Feedback 5' category
- remove AFT5 DB + archive
- upload AFT5 DB with new schema
- deploy new features
- test and fix bugs
- remove AFT5 DB + archive
- upload AFT5 DB with new schema
- test and fix bugs
- deploy AFT5 extension + related code
- enable AFT5 on a few articles for testing
- test and fix bugs as needed
- enable on 42,000 articles
- set up metrics dashboards
- set up abuse filters
- run usability study, as needed
- adjust local configuration settings, as needed
- encourage replacement of AFT4 + ReaderFeedback worldwide
Please add to this list any important tasks we may have missed, as needed.
Here is a checklist of actions to consider for any release of Article Feedback v5 on a new Wikipedia site:
Contact us about this release
- Contact one of our team members to discuss your interest in releasing this tool on your project (email is preferred as a communication channel):
- Fabrice Florin (fflorin-at-wikimedia-org) or
- Oliver Keyes (okeyes-at-wikimedia-org)
- We recommend that you identify an english-speaking community representative to act as a liaison with our team (and provide us with their email address)
Discuss this release with your community
- We recommend that you start an online conversation (or request for comments) to discuss this tool with your community
- Make your community aware of the tool's benefits (e.g.: engage readers to participate, give editors useful suggestions for improving articles) ...
- Also make them aware of the its costs (e.g.: feedback needs to be moderated, only about 40% of the feedback appears useful)
- Try to reach consensus to confirm that the community is prepared to use this tool and moderate comments
Consider a small pilot
- We recommend starting with a small pilot on a limited number of articles, to confirm that the tool works for your community
- The pilot duration should be at least a month or two, so that you have time to make decisions about settings and tweak the tool as needed
- At this time, the Foundation will not be able to support new pilots until the summer of 2013 (except for English, French and German Wikipedia pilots)
Plan your release
- How many articles would be enabled with AFT5 for the pilot? for a wide across your entire site?
- How many feedback posts are expected on a monthly basis for the wide release? (this number is needed so WMF can provide database storage)
- Contact the WMF's editor engagement team with this information, as well as special needs for your project
Namespaces and categories
- What namespaces should AFTv5 be enabled on? (e.g. only main article namespace, also help pages, etc.)
- What percentage of the articles should AFTv5 be enabled at? (note: this is configurable per namespace)
- What categories should be used to blacklist/whitelist AFTv5 on a certain page? (Eg: 'Category:Article_Feedback_5')
- Check access and permissions matrix defaults for each user right
- Confirm that the right tools are available for your user rights
- Change access configurations if needed
Mask over inappropriate comments
At the request of English Wikipedia community members, we plan to display masks over 'inappropriate' comments (formerly called 'hidden'). This is consistent with the policy for rev/del, but slows down the moderation process by requiring moderators to click on every inappropriate post to view its contents. Would you like to keep that mask or not? For projects that choose to keep it, we propose to add this 'Show contents' button (to unmask all inappropriate posts for moderators who want to check them more quickly).
Feedback link on articles
This link shows up on article pages if there is useful feedback for an article, to make it visible to editors. This link can be turned off for any project, if needed (but editors may not know there is feedback for their article without it).
This feature removes comments that are not moderated after a while, to reduce the moderation workload. Some settings like timings can be changed in configuration.
Discuss on talk page
This new feature lets you share useful feedback with other editors on the talk page. It is ready for testing on our prototype site. Please let us know if you have any comments.
The purpose of this feature is to reduce the noise on the feedback page, by automatically disallowing inappropriate posts that include swear words or other objectionable content. This requires that the latest version of the Abuse filter extension be included on each site, and also requires the addition of about a dozen new 'feedback filters', which of course would need to be translated to disallow offensive words in the correct language (rather than English).
Calls to action
After readers post their feedback, they see a call to action (CTA), which can be customized as needed. We recommend using the ''Sign up' CTA 4 to invite unregistered readers to create an account or login (while showing all other users the 'View feedback' CTA 5), as currently implemented on the English Wikipedia. Note that this can be changed to show a 'Take a survey' CTA3, asking all participants to fill in a quick survey for your final evaluation.
Note that when we introduce the new moderation tools and filters, we will need to update this reader and editor help pages, as well as these feedback moderation guidelines. Let's discuss the best way to do this collaboratively in coming weeks. We will start this process by creating an updated version of these help pages in English first, then having them translated and updated on the German and French Wikipedias in coming weeks.
Metrics dashboard update
Once we've deployed the new features on the German and French Wikipedia, we will need to integrate data from new moderation tools and new data cluster into latest feedback volume dashboard and moderation dashboard ('useful', 'resolved', 'no action needed' and 'inappropriate').
Editor Engagement Data
After the next release, we can provide editor engagement data to track how many AFT users end up creating new accounts as a result. To that end, we propose to add an AFT campaign tag in a 'user bucket' cookie when an AFT user creates a new account from the sign up CTA. If we can get resources to do this, we would like to implement this metrics program by early April, so we can give the results to the German community in late April, ahead of their vote in May.
There may be other important localization action items which should be added to this list. Please add to this list anything we missed.
More Info 
To learn more about Article Feedback v5, visit one of these pages:
- AFT5 project hub
- feature requirements page
- testing page (for trying out new features on prototype site)