ORES review tool



The ORES review tool is the key user-facing feature of the ext>Special:MyLanguage/Extension:ORES|ORES extension. The review interface in integrates the scores generated by the ORES service and into MediaWiki's interface.

ORES provides automated scoring of revisions in order to aid editors. For example, ORES can predict whether or not an edit is vandalism as well as the quality level of an article.

See ORES' documentation for more information about what types of scoring is available.

Using ORES
If the ORES extension activated, you can enable the review tool within your user account by looking under the "beta features" section of Special:Preferences.

The review tool will augment Special:RecentChanges and Special:Watchlist by highlighting and flagging edits (with an r ) that need review because ORES' prediction model judges them to be be "damaging".

You'll also be able to filter these lists by selecting the "Hide good edits" option.

When you select this option, the review tool will hide any edits that ORES judges to be unlikely to be damaging.

If you reviewed an edit and realized it's not vandalism, you can simply mark it as patrolled and the highlighting and flag will be removed.

You can change the sensitivity of ORES in your preferences (under the "Recent changes" tab) to "High (flags more edits)" or "Low (flags fewer edits)". You can also choose to make "Hide good edits" selected by default.

[[File:ORES_extension.highlighted_change.png|thumb|350px|

Changes that "need review" and may be damaging are highlighted and flagged in Special:RecentChanges. ]] [[File:ORES_extension.legend.png|thumb|350px|

A "needs review" flag is described in the Special:RecentChanges legend. ]] [[File:ORES_extension.recent_changes_prefs.png|thumb|350px|

Configuration settings are available via Special:Preferences. ]]

How does ORES detect damaging edits?
ORES uses machine learning strategies to "learn" what damaging edits look like by reviewing examples created by Wikipedians through Wiki labels.

These predictions are inherently imperfect because ORES can't be as smart as a real human.

However, ORES can help make the work of RecentChanges-patrolling easier by flagging edits that *might* be damaging.

This is why the review interface states that flagged edits "may be damaging and should be reviewed".

Ultimately, human judgement is necessary for determining which edits are damaging and which edits are not.

See m:Objective Revision Evaluation Service for more information about how "edit quality" is evaluated in ORES.

Why use the term "damaging" instead of "vandalism"?
The word "vandalism" implies malicious intent. However, a patroller's job is to look for damaging edits whether or not they were made intentionally or not. So referring to the edits that the review tool flags as "damaging" is more true to the kind of work the system is designed to support.