ORES/FAQ

= Beginner Questions =

What does ORES stand for?
ORES Stands for Objective Revision Evaluation Service

Predicting edit and article quality
ORES is an artificial intelligence service, which helps human editors improve the quality of edits and articles on Wikipedia. ORES uses a combination of open data and open source machine learning algorithms to train models and create scores that help predict the quality of edits as they are made. Learn more about the background and basics of ORES.

Tools and services that use ORES scores
Many tools use ORES scores. These tools automate time-consuming workflows that would otherwise be done manually by human editors.

ORES tools have been used to predict the quality of new edits and articles, quickly identify and address damaging edits (sometimes called "vandalism"), check for copyright violations, and patrol recent changes to articles.

What's an ORES score?
An ORES score is a score assigned to individual edits to articles on (Wikipedia). ORES scores help humans and machines describe the quality of an edit.

ORES scores allow humans and machines to generate better article content and to improve existing content.

ORES generates 2 types of scores “article” and “edit” “quality.

The “edit” ORES score helps to determine which edits are damaging to an article and which edits were made in good faith. This makes it easier to identify well intended folks who might need extra support to make better edits to articles. It can also help identify and revert clear incidents of vandalism.

What is a patroller?
ORES patrollers use specific filters to determine if new edits may be damaging. Potentially damaging edits are flagged and brought to the attention of human editors to make a final determination.

What are damaging edits (sometimes called "vandalism")?
The word "vandalism" implies an edit was made to create damage on purpose.

Sometimes folks make edits that have damaging effects, even if they make them with the best intentions. A patroller's job is to look for "damaging" edits, whether the damage was intended or not.

Additional information on ORES review tools page.

What is an article?
A wiki article is a webpage you can edit. The contents of an article change over time, so when ORES scores things about the article, we will either use the current article content, or historical content from an earlier edit.

What is an edit?
Wiki articles can be changed by anyone clicking on the "Edit" button, as explained on this help page. Each edit is made by one user, who opens the article, edits and saves. Edits cannot be changed after the fact, although you can make a new edit to correct mistakes.

When ORES scores an edit, we are analyzing the change to guess whether it was helpful or damaging to the article, and whether the change was made in good faith.

Edits are also known as "revisions". Reading the info page for an edit (example) will show you two things, the change made in that edit, and the contents of the article once that edit is applied.

What does quality mean?
Article content is scored using scales such as the Wikipedia 1.0 assessment, which are designed to give an idea of the completeness and readability of an article, as well as the richness of citations to supporting material. Each wiki language and project will use their own scale, see the English Wikipedia 1.0 scale for more information.

How do I get support for my wiki in ORES?
https://www.mediawiki.org/wiki/ORES/Get_support

= Intermediate Questions =

How do I use ORES?
ORES is built into the RecentChanges, Watchlist, and Contributions pages for supported wikis. We recommend using the new filter interface, which gives more flexible searching and highlighting for several thresholds of damaging and good-faith prediction.

If your wiki isn't supported yet, please help us put together the language assets and let us know that you're interested in working with us to develop support. See "How to I get support for my wiki" above.

What tools are available that make use of ORES?
A number of tools are available that make use of ORES.

What does ORES' architecture look like?
Architecture diagram (conceptual)

= Expert Questions =

What installations of ORES are there (WMFLabs & Production) and what's the difference between them?

 * ORES in WMFLabs: https://ores.wmflabs.org
 * ORES in Production: https://ores.wikimedia.org

The difference between WMFLabs and Product comes down to new features and stability. You can expect ORES in Production to be stable, highly available, and performant. ORES in WMFLabs is our experimental installation. It should be mostly stable, mostly available, and relatively performant. We will deploy new models and new features to ORES in WMFLabs first. We suggest you use the WMFLabs installation for testing out new models, or for running large batch processing jobs (up to 4 parallel requests per second) without affecting Production. Production should be able to handle moderate batch processing jobs (up to 2 parallel requests per second), but it more suited for real-time usage (i.e. requesting scores for changes as they happen).

Who should I contact about ORES stuff?
ORES swagger: https://ores.wikimedia.org/v2/ or https://ores.wmflabs.org/v2/ (Audience: Tool devs)

Where should I set my thresholds for filtering/highlighting
Requires: https://phabricator.wikimedia.org/T140364

Defined at https://github.com/wiki-ai/ores/blob/master/ores/wsgi/routes/v2/swagger.yaml

Link to sources in the swagger template, if it's not already.
ORES repo: https://github.com/wiki-ai/ores (Audience: Contributors)

Where do I get more information about what ORES is and how it is used?
ORES on MediaWiki

ORES MediaWiki Extension

PythonHosted: https://pythonhosted.org (Uses python's sphinx doc framework) (Audience: Developers / contributors)