JADE/Use cases

From MediaWiki.org
Jump to navigation Jump to search

User/roles[edit]

Editors

An editor is a contributor to a wiki where JADE is available. An editor uses the MediaWiki UIs to build, curate, and maintain content. Editors discuss their views and arrive at consensus decisions.

Patrollers

Patrollers review streams of new content for vandalism, damage, spam, and other types of problematic content. They often use a mixture of Special:Recentchanges and 3rd party tools to do their work.

Tool developers

Tool developers build tools for other Wikipedians to use. Their tools may aid in patrolling, in task routing, or otherwise help Wikipedians manage their activities. Tool developers generally apply 5 approaches to delivery of their technologies: gadgets/user-scripts, MediaWiki extensions, robots, 3rd party web tools, and 3rd party stand-alone tools. Rarely, tool developers build secondary services for other tools to use such as APIs and streams.

Researchers

The term "researcher" is often reserved for a professional researcher, but in this case, it refers to anyone who is trying to build knowledge. See en:WP:OR. Researchers review activities on wiki and use data analysis strategies (e.g. m:Quarry and m:Database dumps) to make sense of what is happening and what should happen. Data quality issues are of great concern to researchers.

Model builders

Model builders train artificial intelligences using human judgment as training examples. (Arguably, they are a subset of "researchers".) Model builders are concerned with the representativeness and availability of training data.

Admins

Admins and other users with advanced rights are able to suppress content that is damaging to individuals or otherwise needs to be purged from a wikis history. They are often involved in the resolution of harassment issues (e.g. en:DOXing) and copyright issues.

Use cases[edit]

Judge entity[edit]

Users
Editor, Patroller, Admin
Description
Subjective opinions on wiki entities are recorded. E.g. "Is this edit damaging?"
Statement
I want to record the subjective judgments that I make.
Feature
Either directly or (more likely) using a purpose-built tool, the user can create a judgment. The judgment would include the aspect judged, the opinion of the user, and freeform text that provides further justification. There is also a "preferred" flag meaning "there is consensus that this is the 'correct' judgment," and this flag should be set by the user.

Disagree[edit]

Users
Editor, Patroller
Description
Sometimes users will disagree on the appropriate judgement. That disagreement is can be filed as the beginning of a consensus discussion.
Statement
I disagree with other editor patrollers on some judgment calls. I would like to file my disagreement in a productive way.
Feature
A user who disagrees has different options. They can post to the talk page to register disagreement or they can directly edit the judgment page to add a new judgment. While adding the new judgment, the user may also change the "preferred" flag to reflect their preference.

Signal consensus[edit]

Users
Editor
Description
Editors work together to determine the best judgment and then flag it as the current best.
Statement
When there is disagreement and we're able to identify a consensus, I would like flag the best judgment for re-use and consideration by others.
Feature
At some point, the discussion concludes and an action is decided upon. A neutral editor, representing the consensus achieved, can set the preferred flag on the judgment deemed to be preferred.

Change consensus[edit]

Users
Editor, Patroller
Description
Editors can change the consensus signal based on new understandings (or to revert vandalism)
Statement
Sometimes consensus changes, so I would like to change how consensus is recorded to reflect that.
Feature
At some point, the discussion concludes and an action is decided upon. A neutral editor, representing the consensus achieved, changes the preferred flag to be on a different judgment instead.

Describe judgment[edit]

Users
Editor, Patroller
Description
Users can describe the reasoning behind a judgment in a flexible way.
Statement
I'd like to explain my reasoning. In some cases, a judgment isn't obvious and I want to know what criteria I applied when considering something.
Feature
While endorsing a judgment, a user can fill out a rationale explaining their preference.

Edit judgement reasoning[edit]

Users
Editor
Description
Typos, missing information, and non-standard language can be fixed after a judgment is stored.
Statement
I'd like to fix typos, add relevant information, or normalize language used in judgment descriptions.
Feature
Judgments are wiki pages and can be revised as needed.

Read judgments for entity[edit]

Users
Editor, Patroller, Researcher, Model builder
Description
Given an entity, are there any judgments? Is there consensus?
Statement
I'd like to know if an entity has been judged. I'd also like to know if there is consensus on a specific judgment.
Feature
There will be an API that provides judgment data for entities; clients can also fall back on the vanilla MediaWiki API and look up judgments using the page title structure "Judgment:{Page|Revision|Diff}/{page_id|rev_id}". For end users, we are working on areas in the user interface to surface judgment information.

Assess judgment quality[edit]

Users
Patroller, Researcher, Model builder
Description
Quality is captured in process. What kind of process lead to a given judgment? Given an entity and a judgement, one can determine who agrees with what judgment and what experience level they have. Who agrees with the consensus? Is the only editor involved currently in good standing (i.e. not banned)? Do any experienced editors agree with consensus? How many users agree? Is there any dissent recorded? Is there a description? Does the description match the entity in some way? Does the description use standard language?
Statement
I want to use my own exploratory strategies to determine which judgments are appropriate for my work. Most important to me are the who, where, and why. Who provided the judgment. Where were they (what were they looking at?) And what is their justification (if any)?
Feature
Judgment pages include plenty of information that should help derive context. Users who are using a third-party client integrated with JADE will specify an origin in their judgments, which can be used to filter out (or filter in) particular curation tools, including those that use ORES. Users can also provide rationales and notes with their judgments, which can be analyzed by researchers.

Bulk download[edit]

Users
Researcher, Model builder
Description
In order to analyze trends and gather train/test sets, researchers and model builders will want to gather large sets of judgments for processing and aggregation.
Statement
I want to be able to get all of the judgments at once. Sometimes I just want to get judgments for a specific schema, but I can do my own filtering if the dataset isn't too big. I'll use the smallest bulk dataset I can that serves my purpose.
Feature
JADE will offer either a specialized dump created with research in mind, or failing that, JADE data will appear in regular Wikimedia dumps.

API for performing editor/patroller actions[edit]

Users
Tool developers
Description
In order to enable their users to interact with JADE data, tool developers will need to develop integrations with JADE via a programmable interface. This programmable interface must allow 3rd party users to perform basic actions in JADE, but advanced actions may need to happen via the web interface.
Statement
I want to be able to let my users submit judgments to JADE. In unusual cases where their judgments are against consensus, I can link them to the advanced UI, but I'd like to do that rarely.
Feature
JADE offers an API for interacting with judgment pages. This API should make it possible for tool developers to integrate JADE with their applications.

Suppress changes[edit]

Users
Admins
Description
If you give people the space for it, they will harass and DOX each other. So Admins can suppress and delete historical changes to make sure they are removed from the record.
Statement
I need to be able to suppress any changes that are seriously offensive, copyrighted, or otherwise would cause harm to people.
Feature
Human-generated content in JADE is treated like any other MediaWiki content and can be suppressed accordingly.

Patrol for damage[edit]

Users
Editors, Admins
Description
People may use JADE to post petty vandalism that doesn't rise to the level of needing to be suppressed. Editors may want to investigate vandalism.
Statement
I need to be able to patrol JADE content for vandalism in order to remove it and maintain the integrity of the wiki.
Feature
Edits to JADE content appear in recent changes if the Judgment namespace is enabled. This allows integration with other patrolling tools that rely on the recent changes feed.

Revert damage[edit]

Users
Editors, Admins
Description
If petty vandalism is spotted on a JADE page, it should be reverted.
Statement
I need to be able to revert vandalism that is posted on a wiki page in order to preserve the integrity of the wiki.
Feature
The usual edit, rollback, and undo functionalities in MediaWiki can be used on JADE pages to undo vandalism.

Track user contributions[edit]

Users
Editors, Patrollers, Admins
Description
Looking through a user's contributions is an important way of self-policing in Wikipedia. It also is a great way for users to remember what they have been working on recently. Collecting activities that a user performed is essential for maintaining and querying a user's reputation.
Statement
I want to be able to see what a user is up to. If they are going particularly great work, I might send them a barnstar. If they are consistently behaving badly, I might warn them or block them. I'd also like to review all of their activities to make sure I clean up the damage they have caused.
Feature
Edits to Judgment pages show up in recent changes, watchlists, etc. Since judgment pages are in their own namespace, it is possible to easily filter them.