JADE

This page introduces a new software initiative to provide human feedback about AI judgments.

What is Meta-ORES?
The Wikimedia Foundation Scoring Platform team is proposing Meta-ORES as a way to attach human judgments and dialogue to any wiki entity, similar to how the Talk namespace relates to articles, but with associated structured data such as an article quality rating to challenge ratings made by automated scoring. It will serve as a robust false-positive and feedback gathering system, to tune the AI and encourage democratic oversight. This lets users refute automated scoring, or independently review a wiki entity.

The first iteration will support judgments targeting articles and revisions, in order to refute ORES scores, but can be expanded later to target more artifact types.

These discussions can be curated and vandalism suppressed through the usual mechanisms, and may integrate with Extension:AbuseFilter. We'll be asking the volunteer community to take on a new workload in patrolling these artifacts, so we'll make accessible in all the usual dashboards.

TODO: A UI mockup would go a long way towards explaining what's going on here.

Why is Meta-ORES important?
TODO.

Meta-ORES will facilitate a community of people overseeing the AI, perhaps even in "partnership" with the AI. Meta-ORES is needed so that editors can effectively challenge the AIs' automated judgments.

Why is it called "Meta-ORES"?
Actually, the team doesn't like the name either. Help by suggesting more colorful names in an upcoming community naming contest.