Edit check/th

In the 2022–2023 fiscal year, the Editing Team is working on a set of improvements for the visual editor to help new volunteers understand and follow some of the policies and guidelines necessary to make constructive changes to Wikipedia projects.

Below, you can find information about the goals of this project, the history that has informed it, and why the Wikimedia Foundation's Product Department is prioritizing this work. Watch Editing team/Community Conversations for scheduled meetings about this project.

Objectives

 * 1) Newcomers and Junior Contributors from Sub-Saharan Africa will feel safe and confident enough while editing to publish changes they are proud of and that experienced volunteers consider useful.
 * 2) Moderators at the English and French Wikipedias will notice improvements in the quality of edits newcomers are making and be motivated to configure how Edit Check presents policies to them.

Baseline metrics Two of the metrics the Editing Team is planning to use to evaluate the impact of the initial reference check are:


 * 1) A decrease in the proportion of new content edits that are reverted
 * 2) An increase in the proportion of new content edits that include references

To help set targets for the two metrics named above, we recently completed a baseline analysis. Here's some of what we learned:


 * Across all Wikipedias, new content edits that include a reference are ~2x less likely to be reverted (6.2%) than edits that do not include a reference (11.5%)


 * Across all Wikipedias, newcomers and junior editors are less likely to include a new reference with new content edits compared to more senior editors.
 * Of all the new content edits newcomers make across Wikipedias, 12% of these edits include a reference.
 * Of all the new content edits people who have made >500 cumulative edits across Wikipedias, 26% of these edits edit include a reference

You can see per wiki breakdowns in the full report here.

Storing and show decline responses

In March, we shared plans to present people who decline to add a source when Edit Check prompts them to do so with a way to share why they made this decision.

This past week, we converged on how Edit Check will initially make these responses available to experienced volunteers...

To start, the reason someone selects for declining to add a reference when Edit Check invites them to do so will get logged as an edit tag that is "appended" to that edit. Note: the definitions for these yet-to-be defined tags will eventually be stored here: Edit check/Tags.

Two new change tags

This week, two new Edit Check-related change tags became available that you can use to filter Special:RecentChanges.

These tags will help us collectively evaluate the extent to which the reference Edit Check increases the likelihood that people accompany the new content they're adding with a reference. Note: the logic that determine when the two tags get applied is the same logic that is used to decide whether people should be presented with the reference Edit Check.

Edit Check Prototype (mobile) ready

A prototype for the first Edit Check is ready! Now, we need your help identifying how it might need to be fixed and improved before being enabled in production as a beta feature.

You can find instructions for trying out the Edit Check prototype and sharing feedback about on the talk page: Seeking Feedback: Edit Check Prototype.

For context, this first Edit Check that will prompt newcomers who are contributing new content without including a corresponding reference to consider doing so.

First version

The first version of Edit Check is almost ready for you all to try!

Within the next week, you can expect us to share a link to a test wiki where you can try the Edit Check prototype.

This first iteration will invite people who add more than 50 new characters to an article in the main namespace to include a reference in the they're making, IF they have not already done so themselves.

In the meantime, you can see the kinds of edits EditCheck currently thinks warrant a reference, by filtering Recent changes using the newly-introduced  tag. View the tag on en.wiki and fr.wiki.

Informed by community conversations (still ongoing)   and a series of technical and design investigations  , during February the Editing Team became clear about the first version of Edit Check on mobile...


 * User Experience: the first version of Edit Check will introduce a new step within the mobile visual editor's publishing workflow that people will see if/when they add new content without a reference. Design for the desktop user experience is still underway. See T329579.
 * Usability Testing: to learn whether people understand and can intuitively navigate the mobile Edit Check workflow, we will soon begin a series of usability tests. See T327356.
 * Technical Investigation: Edit Check will use a "transaction-based" approach for determining what new content is added within a given edit session. Work on developing a way to detect individual sentences is ongoing in T324363.
 * Initial Heuristic: To start, the initial Edit Check heuristic will be relatively straightforward in so far as it will prompt people to decide whether the change they are making warrants a reference if/when they are adding a new paragraph and that paragraph does not already contain a reference. See T324730 and T329988#8654867.

Next up: the Editing Team will be implementing the initial Edit Check heuristic (T324730) and a corresponding hidden change tag (T324733) so that we – volunteers and members of the Editing Team – can evaluate the extent to which the reference check heuristic is getting initiated in expected cases.

Work on Edit Check is underway! Below you will find an overview of what the Editing Team is actively working on…


 * Community conversations: Between October 2022 and January 2023, the Editing Team hosted seven community conversations to learn what contributing to Wikipedia has been like for people living in and from Sub-Saharan Africa. Next week, you can expect the team to publish the findings from these conversations and how they will inform the work we do on this project.
 * Initial Focus: The first feature the team will be introducing is one that checks whether the new content people are attempting to add includes a reference. Learn more in the Strategy and Approach section below.
 * Design: The team is actively working on a proposal for what the mobile user experience for the first reference check could be like. In the coming weeks, we will be inviting volunteers to help us revise and refine these designs. In the meantime, you can follow along with this work in Phabricator.
 * Talking with experienced volunteers: for the "reference check" to be useful to inexperienced and experienced volunteers alike, it will need to guide people to cite references in ways that projects expect. In the coming weeks, we'll begin conversations with experienced volunteers to learn what these expectations are so that we can ensure Edit Check is configured in ways that align with them.
 * Technical investigations: For the "reference check" to work, the software will need to know when people are attempting to add new content, whether that new content warrants a reference, and whether it currently contains a reference. The Editing Engineering team is currently doing a series of technical investigations to decide how we will approach building this functionality.

Strategy and Approach
To equip newcomers and Junior Contributors from Sub-Saharan Africa with the know-how and tools to publish changes they are proud of and that experienced volunteers consider useful, the Editing Team will be introducing new functionality within the visual editor (desktop and mobile) that will check the changes people are attempting to make and present them with actions they can take to improve these changes in ways that will align with established Wikipedia policies and guidelines.

The first "check" the Editing Team will be introducing is one that will detect when people are attempting to add new content to an existing article without a corresponding reference and prompt them to do so. The functionality will be accompanied by a complimentary set of features that will enable moderators to configure the user experience newcomers and Junior Contributors will see to ensure the software is guiding them to take actions that align with project policies and conventions.

Challenges
The visual editor's growing popularity among people who are new to editing Wikipedia leads us to think that the editing experience has been reasonably successful at helping inexperienced volunteers learn the technical skills necessary to publish changes to Wikipedia.

The trouble is that the visual editor and other editing interfaces do not make people aware of the Wikipedia policies and guidelines they are expected to follow.

As a result, the changes inexperienced volunteers publish often break established best practices and lead to undesirable outcomes for inexperienced volunteers, experienced volunteers, and Wikipedia projects as a whole:


 * 1) Inexperienced volunteers become disappointed and frustrated when the good-faith change(s) they arrived to the wiki seeking to make are undone (read: reverted), deleted, and/or scrutinized in inequitable ways. These poor interactions are demotivating and drive these could-be volunteers and community members, and the knowledge that are uniquely positioned to offer, away.
 * 2) Experienced volunteers/moderators need to do more work reverting low-quality edits and posting messages on inexperienced volunteers' talk pages to make them aware of the policies and/or guidelines they are likely to have unknowingly broken. Continually needing to educate inexperienced volunteers and undo their changes can lead to experienced volunteers becoming skeptical of inexperienced volunteers and impatient with them.
 * 3) Wikipedia projects struggle to grow and diversify their volunteer populations and shrink the knowledge gaps present within Wikimedia wikis.

This project seeks to address the challenges above by:


 * 1) Offering inexperienced volunteers relevant and actionable feedback about Wikipedia policies in the precious moments when they are in the midst of making a change using the visual editor.
 * 2) Equipping moderators with a new ability to specify the feedback inexperienced volunteers are presented with while they are editing

Theory of change
This project is built on the belief that by surfacing relevant guidance in the precious moments when inexperienced volunteers are in the midst of making a change to Wikipedia and equipping them with the know-how and tools necessary to apply this guidance, they will make changes they are proud of and that experienced volunteers value.

In the longer term, the Editing Team thinks that people who are new, particularly people who have historically been excluded from and harmed by established power structures, will feel safe and motivated making changes to Wikipedia if they can accurately predict whether the changes they are attempting to make are aligned with existing Wikipedia policies, guidelines, and/or cultural conventions.

More broadly, the Editing Team thinks that to evolve towards a future where wikis' policies and cultural norms – and ultimately, content – reflect the diverse experiences of the people these projects are intended to serve, we first need to make the norms and standards that are currently in place legible and actionable to people while they are editing. This way, volunteers can develop shared awareness of cases where these norms and standards are not having the impacts they were intended to have and decide what – if any – changes they think are worth making to them in response.

Primary audiences
The Editing Team is centering the needs of people in this work who are:
 * 1) Experience:  Learning the basics of contributing to Wikipedia 
 * 2) *  In the context of this project, we are considering people who are still "learning the basics" to be people who have published <100 cumulative edits to a single, or multiple, Wikipedias. This includes people who are editing Wikipedia for the first time. 
 * 3) Location:  Living in Sub-Saharan Africa 
 * 4) Projects:  Contributing to the English and French Wikipedias 
 * 5) Motivation:  Seeking to fill gaps they notice within Wikipedia 

The four focus criteria listed above are outgrowths of:


 * Newcomers are two times more likely to live in Africa or Asia.
 * The movement struggles to retain editors who live outside Europe and North America.
 * People from Sub-Saharan Africa are underrepresented within the movement: people from Sub-Saharan Africa represent only 1% of active unique editors, despite representing 15% of the global population and 7% of the global internet population.
 * 80% of registered editors in Sub-Saharan Africa contribute to English or French Wikipedia.

Reference Detection
To start, the Editing Team is pursuing an approach with Edit Check that minimizes the likelihood of false positives and is implemented in ways that empower volunteers, on a per-project basis, to evolve the heuristic to become more robust over time.

This strategy amounts to the initial reference Edit Check becoming activated if/when all of the following conditions are met:

The conditions above are implemented and maintained in code here: editcheck/init.js.
 * 1) A minimum of one new paragraph of text is added to the article someone is editing
 * 2) The "new paragraph(s) of text" someone has added does NOT include a reference
 * 3) The changes described in "1." and "2." are happening on a page within the main namespace (NS:0)

The Editing Team arrived at the decision to start with a relatively limited and straightforward set of rules in order to:
 * 1) Increase the likelihood that newcomers and Junior Contributors find the guidance Edit Check is presenting them with, and the editing experience more broadly, to be intuitive and straightforward so that they feel encourage to return to edit again
 * 2) Decrease the likelihood that Edit Check is creating more work for experienced volunteers by prompting newcomers and Junior Contributors to add sources when they are not needed

You can learn more about the assumptions that informed the thinking above in T329988.

Configurability
The Editing Team thinks it is crucial that moderators be empowered to configure when, and for whom, Edit Check becomes activated. This way, they can be confident the software is promoting behavior they deem to be productive and modify the software when it is not.

In line with the above, and drawing inspiration from how the Edit filter and Growth Team Community configuration systems afford volunteers the ability to audit and configure how they function on-wiki, Edit Check will enable volunteers, on a per project basis to:


 * Audit and edit the logic that determines when the reference Edit Check becomes activated and
 * Review the edits people who are shown Edit Check are making

Work to implement the above is ongoing in T327959.

Mobile
The first version of Edit Check will introduce a new step within the mobile visual editor's publishing workflow that people will see if/when they add new content without a reference.

Desktop
'' Design for the desktop user experience is still underway. See T329579. ''

Evaluating impact
The viability of the features introduced as part of the Edit Check project depends on the impacts it causes and averts.

This section describes the:


 * 1) Impacts the features introduced as part of the Edit Check are intended to cause and avert
 * 2) Data we will use to help determine the extent to which a feature has/has not caused a particular impact
 * 3) Evaluation methods we will use to gather the data necessary to determine the impact of a given feature

Deployment sequence
Before volunteers and staff can consider whether a featured introduced as part of the Edit Check project is fit for being made available at all projects, we need evidence that confirms:


 * 1) Experienced volunteers are optimistic the check will be activated within edits when they would expect it to
 * 2) Members of the Editing Team are optimistic newcomers and Junior Contributors are finding the user experience intuitive and helpful
 * 3) Staff and volunteers agree the feature has net positive impact on projects and the volunteers who build and maintain them

To gather the evidence needed to confirm the above, the Editing Team will follow a deployment sequence made up of phases listed below.

Background
Volunteers throughout the movement have a long history of working to:

The Editing Team and this project have been inspired by these efforts, some of which are listed below. If there is a project or resource you think we should be aware of, please add it here!
 * Proactively educate and guide newcomers to make changes they feel proud of and changes that improve Wikipedia
 * Prevent people from publishing destructive changes, and
 * React to and moderate changes to Wikipedia articles.