ORES makes mistakes. It's not as smart as humans are. In order to make sure ORES is as useful as possible despite its limitations, documentation around its issues and mistakes is critical. This page is intended to be a central reference for reporting and discussing issues with ORES.
- See /Edit_quality for reporting misclassifications.
Bias against anonymous editors
The edit quality models have a known bias against anonymous editors. In some ways, this is desirable since most vandalism does come from anonymous editors. However we've been able to mitigate some of the issue. See the original reports (T118982, T129624), our task for identifying the problem (T120138 and analysis. This issue was presentation by EpochFail at a mw:Wikimedia Research/Showcase (slides).
We're working to mitigate the problem by increasing signal from non-user-related features.
- T144636 -- [Epic] Implement PCFG features for editquality and draftquality
- T145812 -- Implement ~100 most important hash vector features in editquality models
- T162617 -- Use 'informals', 'badwords', etc. in Wikidata feature set
- See /Article quality for reporting misclassifications.
- See /Draft quality for reporting misclassifications.
- T148700 -- The tracking task for building a queryable system for tracking misclassifications