Extension:ORES

From MediaWiki.org
Jump to: navigation, search
MediaWiki extensions manualManual:Extensions
Crystal Clear action run.png
ORES

Release status:Extension status beta

ORES extension screenshot.png
DescriptionTemplate:Extension#description This MediaWiki extension integrates data from the ORES project into the RecentChanges view.
Author(s)Template:Extension#username Kunal Mehta, Amir Sarabadani, Adam Roses Wight
LicenseTemplate:Extension#license GNU General Public License 3.0 or later
Download
Hooks usedTemplate:Extension#hook
BeforePageDisplayManual:Hooks/BeforePageDisplay
ChangesListSpecialPageFiltersManual:Hooks/ChangesListSpecialPageFilters
ChangesListSpecialPageQueryManual:Hooks/ChangesListSpecialPageQuery
EnhancedChangesListModifyBlockLineDataManual:Hooks/EnhancedChangesListModifyBlockLineData
EnhancedChangesListModifyLineDataManual:Hooks/EnhancedChangesListModifyLineData
GetBetaFeaturePreferencesManual:Hooks/GetBetaFeaturePreferences
GetPreferencesManual:Hooks/GetPreferences
LoadExtensionSchemaUpdatesManual:Hooks/LoadExtensionSchemaUpdates
OldChangesListRecentChangesLineManual:Hooks/OldChangesListRecentChangesLine
RecentChange_saveManual:Hooks/RecentChange_save
ChangesListSpecialPageStructuredFiltersManual:Hooks/ChangesListSpecialPageStructuredFilters

Translate the ORES extension if it is available at translatewiki.net

Check usage and version matrix.
Vagrant roleMediaWiki-Vagrant#Using roles ores

IssuesPhabricator

Open tasks · Report a bug

The ORES MediaWiki extension integrates data from an ORES service into the RecentChanges view.

Currently, the ORES backend service is only set up for Wikimedia wikis, and significant work would be required to set it up for third-party MediaWiki installations.

Screenshots[edit]

Changes that "need review" and may be damaging are highlighted in Special:RecentChanges.
A "needs review" flag is added to the Special:RecentChanges legend.

Installation[edit]

  • Download and place the file(s) in a directory called ORES in your extensions/ folder.
  • Add the following code at the bottom of your LocalSettings.php:
    wfLoadExtension( 'ORES' );
    
  • YesY Done - Navigate to Special:Version on your wiki to verify that the extension is successfully installed.

You need to run CheckModelVersions.php maintenance script once it's deployed (and after that you can run PopulateDatabase.php too)

Config variables[edit]

Here's the config variables and their default values and a little description about them.

// URL of the service
$wgOresBaseUrl = 'https://ores.wikimedia.org/';
// Either to exclude edits made by bot to score
$wgOresExcludeBots = true;
// Models to score
$wgOresModels = [
    'damaging' => true,
    'goodfaith' => true,
    'reverted' => true,
    'wp10' => true
];
// Thresholds of different sensitivies in ORES
$wgOresDamagingThresholds = [ 'soft' => 0.7, 'hard' => 0.5 ];
// Namespaces the ORES should score. Empty array means all namespaces.
// If not empty, it will only works on the given namespaces.
// Determine namespaces like [ 0 => true, 120 => true ].
$wgOresEnabledNamespaces = [];
// Database id for ORES service. If not determined, it'll use database name.
// You can choose 'testwiki' that ORES service sends last two digits of rev_id flipped.
// For example: https://ores.wikimedia.org/v1/scores/testwiki/damaging/12345
$wgOresWikiId = null;

ORES service responses[edit]

ORES extension is merely more than an interface to the ORES service. The service returns a probability score of edits being damaging like this (API v1):

{
  "724030089": {
    "damaging": {
      "prediction": false,
      "probability": {
        "false": 0.8917716518085119,
        "true": 0.10822834819148802
      }
    }
  }
}

It means this edit (diff=724030089) is 10% likely to have caused damage. Note that 90% likely doesn't mean 9 out of ten cases will be vandalism. Choosing thresholds should be done via analysing recall (percentage of vandalism it can catch) or false positive rate. In ORES the "soft" threshold is when recall is 75% (meaning it will include 75% of all damaging edits) and the "hard" threshold is when recall is 90%. You can get the thresholds from model info (an example).

Database schema[edit]

ORES extension introduces two new tables: ores_model and ores_classification. See the full database schema description.

Extension workflow[edit]

Scores[edit]

Once an edit is made the extension triggers a job to hit the service and store the results in the ores_classification. It means it will not include scores for edits made before the deployment. In order to fill the database you can run the maintenance script PopulateDatabase.php. It will hits the service and keeps the score for the last 5,000 edits. You can run it several times if needed.

Once a model gets updated to a newer version CheckModelVersions.php maintenance script needs to be ran to update the ores_model table which will cause to scores stored in ores_classification become deprecated. You can clean these obsolete scores by running PurgeScoreCache.php maintenance script.

Interface[edit]

The extension won't show anything when deployed but it will add itself as a beta feature (Extension:BetaFeatures is a dependency of this extension) and once it's enabled by the user it will use hooks in ChangesList (RecentChanges, Watchlist, and RelatedChanges) in both old and enhanced mode and highlights when score exceeds the given threshold.