ORES review tool



The ORES review tool is the key user-facing feature of the ext>Special:MyLanguage/Extension:ORES|ORES extension. The review interface integrates the scores generated by the ORES service into MediaWiki's interface.

ORES provides automated scoring of revisions in order to aid editors. For example, ORES can predict whether or not an edit is vandalism as well as the quality level of an article.

See ORES' documentation for more information about what types of scoring is available.

We deliberately set the default threshold so low to capture all vandalism cases so false positives are expected unlike anti-vandalism bot that set the threshold so high to capture only vandalism cases (and don't have false positives). If you don't want to see the flag for most edits, you can simply change ORES sensitivity (see below).

Using ORES
If the ORES extension is activated, you can enable the review tool within your user account by looking under the "beta features" section of Special:Preferences.

The review tool will augment Special:RecentChanges and Special:Watchlist by highlighting and flagging edits (with an r ) that need review because ORES' prediction model judges them to be be "damaging".

You'll also be able to filter these lists by selecting the "Hide good edits" option.

When you select this option, the review tool will hide any edits that ORES judges to be unlikely to be damaging.

If you reviewed an edit and realized it's not vandalism, you can simply mark it as patrolled and the highlighting and flag will be removed.

You can change the sensitivity of ORES in your preferences (under the "Recent changes" tab) to "High (flags more edits)" or "Low (flags fewer edits)". You can also choose to make "Hide good edits" selected by default.

[[File:ORES_extension.highlighted_change.png|thumb|350px|

Changes that "need review" and may be damaging are highlighted and flagged in Special:RecentChanges. ]] [[File:ORES_extension.legend.png|thumb|350px|

A "needs review" flag is described in the Special:RecentChanges legend. ]] [[File:ORES_extension.recent_changes_prefs.png|thumb|350px|

Configuration settings are available via Special:Preferences. ]]

How does ORES detect damaging edits?
ORES uses machine learning strategies to "learn" what damaging edits look like by reviewing examples created by Wikipedians through Wiki labels.

These predictions are inherently imperfect because ORES can't be as smart as a real human.

However, ORES can help make the work of RecentChanges-patrolling easier by flagging edits that *might* be damaging.

This is why the review interface states that flagged edits "may be damaging and should be reviewed".

Ultimately, human judgement is necessary for determining which edits are damaging and which edits are not.

See m:Objective Revision Evaluation Service for more information about how "edit quality" is evaluated in ORES.

Why use the term "damaging" instead of "vandalism"?
The word "vandalism" implies malicious intent. However, a patroller's job is to look for damaging edits whether or not they were made intentionally or not. So referring to the edits that the review tool flags as "damaging" is more true to the kind of work the system is designed to support.