Edit check/Peacock check

Wikimedia Foundation projects Peacock Check Prompt people to write using a neutral tone. Group: Editing + Machine Learning Start: 2025-01-29 Team members: Aiko Chou, Benoît Evellin, Diego Saez-Trumper, Georgios Kyziridis, Megan Neisler​, David Chan, David Lynch, Esther Akinloose, Marielle Volz, Rummana Yasmeen, Nicolas Ayoub, Benoît Evellin, Ed Sanders, Zoe, Backlog: #EditCheck Lead: Sucheta Salgaonkar, Peter Pelberg, Management: Valerie Puffet-Michel (engineering), Ilias Sarantopoulos

Tracked in Phabricator
Task T365301

This page holds the work the Editing Team is doing in collaboration with the Machine Learning Team to develop Peacock Check.

Peacock Check is an Edit Check that prompts people to consider the neutrality of the new content they are venturing to add to Wikipedia.

A notable aspect of this project: Peacock Check is the first Edit Check that uses artificial intelligence. In this case, to identify biased language within the new text people are attempting to publish to Wikipedia. The model used is .

To participate in and follow this project's development, we recommend adding this page to your watchlist. 

Status

Development underway: Peacock Check is under active development.

Please visit Edit check#Status to gain a more granular understanding of where the development stands.

Objectives

Peacock Check is intended to:

1. Cause newer volunteers to add new information to Wikipedia's main namespace that is written in a neutral tone
2. Reduce the effort and attention experienced volunteers need to allocate towards ensuring text in the main namespace is written from a neutral point of view.

Evaluating impact

Maybe this section ends up including the artifact we are venturing to create to model the end-to-end data flow that will drive the feature.

Background

Writing in a neutral tone is a pillar of Wikipedia.

Writing in a neutral tone is also a practice many new volunteers find to be unintuitive. An October 2024 analysis of the new content edits newer volunteers published to English Wikipedia found:

• 56% of the new content edits newer volunteers published contained peacock words.
• 22% of the new content edits newer volunteers published that contained peacock words were reverted
• New content edits containing peacock words were 46.7% more likely to be reverted than new content edits without peacock words

Edit Check

This initiative sits within the larger Edit Check project – an effort to meet people while they are editing with actionable feedback about Wikipedia policies.

Edit Check is intended to deliver impact for two key groups of people.

Experienced volunteers who need:

1. Relief from repairing preventable damage
2. Capacity to confront complexity

New(er) volunteers who need:

1. Actionable feedback
2. Compelling opportunities to contribute
3. Clarity about what is expected of them

History

This section will include the projects/initiatives this project is decedent of. Think: Model Diego wrote.

FAQ

Why AI?

AI increases Wikipedia projects' ability to detect promotional/non-neutral language before people publish it.

Which AI model do you use?

We use an open-source model called BERT. The model we use is not a large language model (LLM). It is actually a smaller language model which the Machine learning team prefers, because it tells us how probable each of its predictions is, and it's easier to adapt to our custom data.

What language(s) does/will Peacock Check support?

Work to decide the languages Peacock Check will support to start is underway in T388471 and T387925.

What – if any – on-wiki logging will be in place so volunteers can see when Peacock Check was shown?

To start, we're planning for an edit tag to be appended to all edits in which ≥1 Peacock Check is shown.

This approach follows what was implemented for Reference Check.

What control do volunteers have over how the model behaves?