Page Triage

WMF engineering activity

Page Triage

Interface for triaging pages created in MediaWiki

Group:	Features
Start:	2012-03
End:	ongoing
Management:	Fabrice Florin, Ian Baker
Team:	Howie Fung, Brandon Harris, Benny Situ, Ryan Kaldari, Andrew Garrett, Terry Chay

This document is a work in progress. Comments are appreciated but this is not a final draft.

A mockup of the "list" interface for Page Triage

A mockup of the "list" interface for Page Triage showing the "filters" dialog

This document describes the design of a new interface for triaging pages in MediaWiki. This document is a work in progress. Feedback is welcome on the talk page.

This project is envisioned with multiple phases. By necessity, this bulk of this document focuses on what is currently targeted as "release one" of the Page Triage project. Notes about additional releases follow.

Status

• [edit status] • [add new]
  2012-04-monthly: The editor engagement team continued to develop its prototype of Page Triage, which provides an enhanced list of pages to be reviewed by community patrollers. Benny Situ, Ian Baker and Ryan Kaldari developed new features for the list view of this feature, while Brandon Harris designed a new curation bar to appear on the article pages. Oliver Keyes acted as community liaison and Fabrice Florin managed this project with Howie Fung. Current goals are to complete development of the new curation toolbar in May, and deploy an integrated release version the following month, along with the Article Creation landing system. The latest prototype is available on Wikimedia Labs; our first incremental deployment to the English Wikipedia is planned for the first week in May.

Notes on Nomenclature

This document has developed a new term, "triaged". We believe that "triage" is a more descriptive term than "patrolled." Further, it does not evoke feelings of militarism or police work; rather, that of a doctor trying to save patients rather than prevent them from treatment.

This document will refer to "New Pages" as any page that has not been marked as "Patrolled". Edit count on the page is not taken into consideration.

For the sake of verbiage, this document will assume two states of an article:

• Untriaged - The article has not been marked triaged (old "Patrolled").
• Triaged - The article has been marked as "triaged" (old Patrolled). Tags requesting improvement may or may not have been applied.
• Nominated for Deletion - The article has been marked for deletion.

Rationale

• "New Page Patrol" is a complicated process that is poorly supported by the MediaWiki software itself.
  • No two patrollers seem to utilize the same process.
• Users who perform New Page Patrol report high levels of frustration and burn out due to feeling overworked for these reasons:
  • Inexperienced patrollers aggressively over-template, requiring work to be rechecked;
  • They often don't identify or fix major problems with new articles, requiring work to be rechecked
  • Too few users choose to become patrollers or triagers.
    • Because education about the triaging process is difficult
    • Because optimizing a system for page patrol is a "Power User" job, requiring greater-than average computer savvy as well as (oftentimes) downloading of third party software

Hypotheses

A native, easy-to-use interface for New Page Patrol would:

• increase the number of users who choose to become patrollers, reducing workload.
• help establish better education about the process as is, resulting in lower "false positive" rates.
• allow expansion and modification of the system to support different backend systems and logic screens.
• serve as an engagement point for mobile and tablet users, for whom editing is currently not feasible.
• utilize positive messaging features to reduce new editor bite, thus promoting editor retention.

Phase 1 (List View)

Feature Requirements

• Track the users who have triaged a page and the dates that they did so.
• Provide a list view of New Pages.
  • This list must be filterable.
  • This list must easily show the state of a page, whether or not it has been triaged.
  • This list view must provide as much useful information as possible about an article.
• Eliminate the 30 day "expiry" to the New Pages queue.
• Provide a "Mark as Triaged" link on every page that has not been triaged without requiring the user to have first visted Special:PageTriage
• Optionally mark every un triaged page as NOINDEX
• Optionally mark every Nominated for Deletion' page as NOINDEX

Easement of 30 Day Expiration

A major feature of Page Triage phase 1 is the removal and replacement of the current New Pages queue. The current implementation of the queue is an artifact of the Recent Changes queue, which is only kept for thirty days.

With Page Triage, a new queuing infrastructure will be implemented. This queue will operate thus:

• When a new page is created, it will be added to this queue with the "triaged" bit set to "false".
• When a page is marked "triaged", the "triaged" bit will flip to "true", and the date of the event as well as the user who did the action will be stored.
• Pages that are marked as "triaged" will remain in the queue for an additional sixty days, after which they will be removed from the queue.
• Pages that have not been marked as "triaged" will continue to remain in the queue indefinately.
• Pages that have been nominated for deletion will naturally fall out of the queue when and if they are deleted.

Issues with Queue Modification

A driving factor for the continued triaging of new pages is the existence of the 30 day expiry. This expiration date serves as a motivator: triagers are encouraged to clear the queue.

It is unknown what the removal of this limit will do with regards to triager motivation. Other motivations will be experimented with. If statistics show that the removal of the expiration date radically impacts the performance of the triaging system, it is suggested that it be re-enabled until such time as better systems can be found.

Addition of "Mark as Triaged" link

With the modification of the queuing infrastructure, it should be possible to add a "mark as triaged" link to any untriaged page and have it be persistant without causing cache fragmentation.

This control will be hidden by default. There will be a user preference created that, when enabled, will cause the control to appear.

Alternatively, this can be a system-set cookie: the controls will continue to appear as long as the user visits Page Triage or utilizes the control for a set time (say, seven days).

NOINDEX Flag on Un-Triaged Pages

Since a major problem with new pages is the proliferation of potentially illegal content being indexed by search engines, a default stance of marking pages as NOINDEX until they have been triaged is an option.

When an article is marked as "triaged", the NOINDEX flag will be removed from the article.

Issues with NOINDEX

Of primary concern with the addition of an automatic NOINDEX flag on untriaged edits is that of "Current Events" articles. It is desirable for these articles to be quickly indexed by search engines; hiding them until they have been triaged may be undesirable.

User Experience: List View

A mockup of the "list" interface for Page Triage

The proposed List View interface explodes the current "untriaged" list into a more readable and scannable format.

Log:Nominated for Deletion

A new system log, "Nominated for deletion", will be created. This will be a "standard" MediaWiki log.

When a deletion nomination template is added to a page, an entry will be inserted into the log with the page name and the user who nominated the article for deletion. This functionality works regardless of how the template is injected into the page: it is an "on save" hook, and is not a special function of the Page Triage software.

If a deletion template is removed, an additional entry will be included in the log, indicating that the tag was deleted.

This log may be viewed in a global context or as one of the user's logs (like contributions, blocks, etc.)

For Discussion: Mark as Unreviewed

It may be desirable to provide a mechanism to flip an article back into an "untriaged" state. While such a feature would (in theory) be easy to implement, questions have arisen as to its desirability in the workflow.

The reason for this is that articles that are incorrectly marked as triaged are often headed for the "delete" pile anyway, and it is a simple matter to PROD such an article.

Header Section

The header section will provide controls for filtering as well as the ability to switch queue direction (back or front).

The header primarily shows the currently engaged filters. If too many filters are enabled for full display, the list will be truncated with a "..." indicator that is "hot". Hovering over this will reveal the full list of filters.

Clicking on the "set filters" link will open a tooltip dialog for setting any and all filters.

By default, the only enabled filters are "show triaged edits" and "show nominated for deletion".

A count of the number of untriaged articles will also be displayed.

Filter Mechanisms

All filters are additive. There is no "Or" concept.

The following ways will exist for filtering the List Interface:

• Show Triaged Pages
• Show Articles nominated for deletion
• Show Bot pages
• Show redirects
• By Creator Username
• By Namespace
• By Tag
• Only orphans
• Only no categories
• Only by blocked users
• Only by non-autoconfirmed users

Footer

The footer of the list interface will show statistic information about the performance of the queue:

• Top 5 patrollers
• Median age of unpatrolled article
• Age of oldest article
• A link/button that leads to a more detailed analysis of queue performance

Individual Entries

A mockup showing the hover state of a "triaged" marker

A mockup showing the hover state of a list element

Each entry within the List View contains the following elements:

• A quick-scannable "triaged/not triaged/nominated for deletion" badge/notification

• If a page has been triaged, a green checkmark will appear
• If the page has not been triaged, a red alert mark will appear
• If the page has been nominated for deletion, a black trashcan mark will appear.
• When the user hovers over the icon, if the article has been triaged, a hover will appear showing who did the triaging and when.

• The Page title, along with a link to its history, its size and number of edits
• A count of categories and images

• If there are no categories, this shall be called out in bold and red
• If the page is an orphan, this too shall be called out boldly

• The date the page was created
• The user name of the page creator, his or her edit count, and when he or she started editing;
• The first 500 characters of the page/article (not the edit summary)
• A "Triage" button (in phase one this will open the article in a new tab)

The List Interface is envisioned to be infinitely scrolling. The filter controls will persist at the top of the screen and the statistics pane will persist at the bottom of the screeen. Scrolling within the page will infinite scroll through the entire queue.

Phase 2 (Curation Toolbar)

Feature Requirements

• Provide an, easy-to-use, and intuitive "toolbar" interface that allows page examination and tagging in situ
  • This interface must provide meta-data about the article
  • This interface must show the article in the interface
  • This interface must allow for editing in situ
  • This interface must work with Twinkle
  • This interface must promote positive feedback/welcoming actions
  • This interface must be pageable without leaving the interface
  • Ideally, the interface's "paging queue" will be smart and modify itself according to behaviors of other triagers and their work.
    • This helps to prevent a race condition wherein two triagers work on the same article simultaneously, and generate edit conflicts.

User Experience: Curation Toolbar

This section is a work in progress.

A mockup of the Curation Toolbar, in its most simple format

A mockup of the Curation Toolbar, showing a notifications badge

The Curation Toolbar attempts to address several issues involving the general page review and curation workflow. It is designed to be extensible, context-aware, and allow for modification by the greater community on wikis where it has been enabled.

The Curation Toolbar allows users to review pages, mark them as "reviewed/patrolled", tag them with various templates (including deletion templates), and provide gratitude to the authors of articles.

The Curation Toolbar requires Javascript.

Curation Mode

The Curation Toolbar introduces a new concept to the page patrol/triage workflow: Curation Mode. When a user is in Curation Mode, the toolbar will appear on all pages (though various functions will be disabled, depending upon context).

A User can enter Curation Mode through the following ways:

• Accessing a page via a link from the New Page Triage "list view"
• Accessing a page via a link from Special:NewPages
• Activating "Curation Mode" from a link the sidebar Toolbox, "Enter Curation Mode".

Exiting Curation Mode is handled by clicking the "X" control at the top of the Curation Bar. This will prompt the user for confirmation.

First Time in Curation Mode

The first time a user enters Curation Mode, a simple modal dialog box will appear to the user that explains what Curation Mode is, how it works, and how to exit Curation Mode.

Toolbar Buttons

The following buttons will exist in the first release of the Curation Toolbar:

• Close Control - this will be a small button that will cause the user to exit Curation Mode when clicked. There will be a confirmation dialog.
• Information - This will activate the "Information" flyout.
• WikiLove - This will activate the "WikiLove" flyout
• Mark Reviewed - This will activate the "Mark Reviewed" flyout.
• Tag - This will activate the "Tag" flyout.
• Delete - This will activate the "Nominate for Deletion" flyout.
• Skip - This will skip the current article and move the user to the next article in the queue.

Specific details about the buttons and their behaviors will be included in their corresponding flyout sections, below.

Flyout Behaviors

Clicking on the various buttons on the toolbar will activate "flyout" dialogs with additional information or options.

Note that not all buttons will activate flyouts:

• The "Close" control will cause the user to exit Curation Mode
• The "Skip" button will simply advance the user to the next article in the queue.

Flyout: Information

A mockup of the Curation Toolbar, showing the examination panel

The information flyout provides the user with meta information about the page being viewed, including a system-generated "possible problems list".

If the system detects that there are "possible problems" with the article, a red notifications badge will be displayed on the button with the number of detected problems highlighted.

The flyout panel itself can be broken up into three sections:

Metadata

This section displays several basic bits of information about the page being viewed:

• It's reviewed status (as an icon)
• Article size and number of edits
• Date of creation
• Author name, including the number of edits the author has.

Possible Problems

This section displays a list of system determined "meta data" problems that the article may have (such as "no images", "no references," etc.). Each entry will have a short description as to what the list entry means and why it may or may not be important.

Simple History View

This is section is a simplified view of the articles history, bundled by date. This section will scroll independently.

History entries will be collated by date, and then listed by time, user name, and commit summary.

Flyout: Tag

A mockup of the Curation Toolbar, showing the tagging interface.

The tag flyout allows the reviewer to select multiple tags and add them to the article in bulk.

Various tags will be organized into tabs based on type or category. The user can flip between tabs, selecting tags as needed. The user does not need to click the "add selected tags" button until they are finished: the system remembers which tags have been selected in which categories.

If tags are selected in "deep" tabs, this information is provided to the user by markers in each tab indicating how many tags have been selected in that tab. Further, a "total" number of tags selected will be displayed in the pane next to the "commit" button.

For each tag, a simplified name is used. For example, the "Needs Cleanup" template is listed as "Cleanup" (the better for alphabetization). Each tag includes a short (max 200 character) description as to what it is to be used for.

When a tag requires parameters, checking it will open a sub-dialog indicating that such parameters are required and that the system cannot continue until the parameters have been filled out.

Clicking the "Add Selected Tags" button will add the selected tags to the article in a single edit, cause the page to refresh, and reopen the tag dialog.

The structure of the tag dialog should be definable within MediaWiki (similar to the way that WikiLove handles configurations). This will allow local communities to add or remove tags as needed or desired.

Flyout: Mark Reviewed

A mockup of the Curation Toolbar, showing the "mark reviewed" panel

A mockup of the Curation Toolbar, showing the "mark unreviewed" panel

The "Mark Reviewed" flyout is a simple dialog. It simply requires confirmation that the user intends to mark the page as reviewed.

If the article has already been marked as reviewed, the icon will change state indicating such.

If a "mark as unreviewed" feature is implemented, this dialog will allow for the article to be marked for "unreview" if it is already reviewed.

Flyout: WikiLove

A mockup of the Curation Toolbar, showing the WikiLove panel

The WikiLove panel is a two-stage system. Activating the panel will show a list of all the users who have edited the article, in a roll-up format (e.g., showing a total of edits). The sort order should be based on the degree of contributions with the page creator first and then all other editors listed by the number of contributions, descending.

Next to each user will be a checkbox. Checking this box will include them in stage two of the system. A "check all" link will check everyone in the list.

Once the user has selected the users they desire, they can click the "send wikilove" button. This will then open the WikiLove dialog box, whereupon they can go about constructing the WikiLove award of their choice.

Note that the WikiLove extension must be modified in order to allow for multiple recipients.

Flyout: Nominate for Deletion

Draft workflow for Deletion Wizard

The deletion flyout will have two modes. For advanced users, it will look much like the tag flyout, but with only deletion-related tags.

For beginning users or those who don't yet understand the nuances of Wikipedia's deletion procedures, it will be a wizard that attempts to walk the user through common deletion deletion cases, and properly direct an article to the correct deletion procedure with the appropriate rationale.

Preferences

Eventually, we should create an interface for settings PageTriage preferences. These preferences could include:

• Use infinite scrolling
• Save my filter settings
• Open pages in new tabs
• Float nav bars

To Do

Screen Mockups for:

• Tag flyout behavior when reason is required
• Deletion flyout
• Exit Curation Mode confirmation
• Welcome to Curation Mode