Reading/Web/Projects/Mobile Page Issues

Templates that show messages regarding page content do not display on mobile Wikimedia sites. This leaves readers unaware of the reliability of the pages they are reading. Hiding this information from users can be problematic. This is especially true in cases where important issues, such as hoax articles or articles considered for deletion, are hidden.

The readersteam>Reading/Web|Readers web team would like to improve the treatment of these page issue templates on the mobile website (starting with the ambox templates and their cross-wiki variants). The goal is to include a description of the nature of the issue or notice itself, as well as its severity. These changes will help users make better judgements on the reliability of the pages they are reading. Displaying these issues and notices could also make readers curious about how Wikimedia projects work. This could potentially increase the likelihood of their involvement as contributors.

Below are some examples of how this improvement may look. Feedback is welcome.

January 21st, 2019
We have published a of our full report with the results of our A/B tests.

Overall, we saw a large increase in awareness with the new treatment for page issues.

We also noticed that users tend to have higher engagement with issues of higher severity than those of medium or low severity.

We will be continuing with the last series of deployments on English wikipedia later this week.

December 17, 2018

 * We have deployed the feature to all Wikipedias, except English Wikipedia

December 16, 2018

 * We have finished the final stages of the analysis of the results from our A/B tests: https://phabricator.wikimedia.org/T200794

October 1, 2018
We will be launching an A/B test of the feature on the following wikis:


 * The test will run for two weeks


 * We will be tracking the performance of the feature quantitatively but also hope to receive some feedback from readers and editors as well.

September 24, 2018
We have launched an A/B test of the feature on Latvian Wikipedia

Introduction
When articles have issues, they often feature a large coloured box above the article notifying readers and editors of the issues at hand.

These notices are actually special templates inserted into the content of the article.

English Wikipedia
Each notice in the article has its own template, and each template uses a meta-template called   (Article message box).

Ambox uses a Lua module called Module:Message box.

Other MediaWiki namespaces like categories and talk pages have notices that are specific to their namespace as well.

These notices have existed for over a decade and have many conventions around their usage. Contributors maintain these notices. Every language community is free to adopt or invent their own notices, specific to the needs of their project. For example, Commons has a wide array of licensing notices while Wiktionary uses entirely different notices.

The broad range and diversity of these notices makes them hard to standardize. One happened in 2007 on the English Wikipedia.

This refresh occurred before the modern mobile web even existed. Despite later efforts by the WMF to bring these notices into the mobile web, they are still not very mobile friendly. As a starting point, this proposal will focus on improving notices in the article namespace that based on the template.

Current mobile treatment
On the current mobile Wikimedia sites we don't render the entire template. Instead we display a small grey link with the text "page issues" below the title of an article. When clicked, an overlay with a list of the issues appears. Some notices have text for a "compact" version of the template. When available, only the text for the compact version is displayed.

Limitations
The current implementation depends on modifying the HTML generated by the templates. Slight changes to the template HTML can break this feature on mobile. The current implementation doesn't work on all wikis. Templates are different across languages. For example, French Wikipedia refreshed their message boxes in 2016. They now use different templates than English Wikipedia. frmodules>w:fr:Aide:Modèles de maintenance|Here is a list of their maintenance modules. Because of this change, French alerts are not visible on mobile Wikipedia.

English Wikipedia Village Pump proposal
There was enwikiProposal>w:Wikipedia:Village pump (proposals)/Archive 137#Make certain warning templates visible on mobile (requesting formal close)|a proposal on the English Wikipedia in 2016. The proposal argued that the the small "page issues" link doesn't convey the importance of certain notices.

In September 2016, the community approved this proposal to expose page issues on the mobile website. There was clear consensus that some warning templates should be visible on mobile.

""If an article is up for AfD and flagged as a possible hoax with insufficient medical sourcing, any reader visiting that article on a computer is greeted by three large red and orange boxes at the top of the page, one with a cautionary stop sign... If a reader instead visits the same hoax medical article on their phone... they just get two tiny grey words "Page issues" under the article title...""

Objective
The goal of this project is to improve awareness of particular issues within an article on the mobile web. We will do this by changing the visual styling of page issues.

The team will work with communities to provide guidelines for for styling templates. This work will result in the optimal formatting of issues on mobile site without changing the formatting on desktop.

This goal maps to annualplan>meta:Wikimedia Foundation Annual Plan/2017-2018/Final/Programs/Product#Program 2: Better Encyclopedia|Program 2, Objective 1 of the annual plan for Readers. "Enable readers to gauge the quality and reliability of an article during their reading experience".

Page issues definition
For the extent of this project (as well as in the documentation below), we will define a page as having page issues if it is within the main article namespace (NS_MAIN, 0) and [https://gerrit.wikimedia.org/r/plugins/gitiles/mediawiki/skins/MinervaNeue/+/refs/heads/master/resources/skins.minerva.scripts/cleanuptemplates.js#97 contains] HTML table with the CSS class  etc.

In English Wikipedia this means transcluding the following template (also via nested transclusion):


 * Ambox (article message box)

Interwiki equivalents with localized names will not be affected yet.

The following templates are not classed as "page issues", as they are only used in the non-article-namespace (NS_MAIN, 0) but are related to this project, in that they are collapsed for readability on mobile:


 * Tmbox (talk page message box) when in the "Talk" namespace


 * Cmbox (category message box) when in the "Category" namespace


 * Fmbox (file message box) when in the "File" namespace.

Note, any template that does not identify itself as an "ambox" is not an issue, this explicitly includes:


 * w:Template:Asbox (article stub box) - these are suggestions for improvement (not issues)

A note on "type" and "issue" parameters
We will only be displaying issue severity (and associated icons) for templates that contain the "type" parameter.

More about the type parameter can be seen on the ambox template pages on each project and wiki (here is the page for enwiki).

Issues that do not contain the type parameter within the template will still appear on the page using the default styling.

Similarly, short descriptions will only be available if the "issue" parameter is available within the template itself.

An overview of the templates that already have this text can be found on this phabricator task.

Workflow

 * 1) A user visits a page with a page issue.


 * 1) The user interacts with a "Page issue" element.


 * 1) The user is directed to details on the page issue.

General
Mobile page issues will display the following:


 * Issue description


 * Short descriptions for the issue will be used where available.


 * If no short description is available, we will display the beginning of the long-form issue description.


 * Issues length for both short and long descriptions will be restricted based on the space constraints.


 * Issue severity


 * Severity will be set according to ambox type as follows (this means they will inherit the color of the template type):


 * Severe: type=speedy, type=delete


 * Medium: type=content


 * Low: type = style


 * Notice: type = notice, type = move, type = protection


 * All other templates will display with normal severity.


 * Each severity level will have a custom appearance and position within the page.


 * If an issue contains links to other pages, the links will be removed. The links will appear on the page issue detail modal.

Examples:







Single issue

 * For pages that have a single issue, severity will be derived from the template as defined above.


 * An example. If template has severity level "Medium", the article will have a single page issue appear at the Medium level. ([https://en.wikipedia.org/w/index.php?title=Free_China_(Second_Sino-Japanese_War)&oldid=777751038 example]).

Multiple issues

 * Issue severity levels will be derived from the severity levels of the individual issues.


 * If an article contains the template, the issue level of the article will be the highest issue level available.


 * For example, if template has level "High", the article will have issues appear at High level.(example).


 * Each multiple issue level will receive unique copy, appearance, and position within the page.

Example:



Multiple issues without Template:Multiple issues
We do not support the case where the multiple issues template is not used.

If several issues are adjacent to one another but are not wrapped in the multiple issues template, then there are potential problems with the display - in particular with icon display.

See phab:T202349 for further discussion.

Section issues
The initial implementation of this project will display issues within individual sections.

Potential future improvements
Once we put these styles in place, we would like to look at being able to provide them in a more centralized way by delivering mobile-friendly CSS using the TemplateStyles extension. This will allow us to cover a larger percentage of templates with specialized treatment and give more power to template editors to select the specialized treatment for each template.

Initial updates to the page issue modal
To start, the Readers team made the following changes to the page issue modal. This change happened in late March 2018. The goal of this change is to expose more detail about the page issue itself. It also provides information to readers on how to fix these issues.

Feature evaluation
The Readers team wants to understand the effect increasing the awareness and prominence of page issues on the mobile website has, particularly on the perceived quality and reliability of pages by readers. Our goal is to help readers better gauge the reliability and quality of the article they are reading. Our hypothesis is that making them more aware of page issues will contribute to this goal.

We'd like to answer the following questions:

Does the new treatment for page issue notices increase the awareness among readers of page issues?


 * Qualitative research:


 * Do readers notice the new page issue treatment more than the current treatment?


 * Do readers notice version B (with titles) of the new treatment more than version A (without titles)?


 * Do readers notice page issue notifications located after the lead paragraph more than if they are located at the top of the page?


 * Quantitative research:


 * Is there an increase in click-through based on the new issue treatments (from the article page to the issues modal)?


 * Is there any correlation between severity of the issue and click-through rate?

'''How do users feel about being informed of page issues? How does awareness of page issues affect their perception of Wikipedia?'''


 * Qualitative research:


 * RQ1 - Do page issues make sense to readers?


 * RQ2 - Do readers care about page issues? Do they find them useful? Important?


 * RQ3 - Are readers familiar with page issues already? Have they seen them on other articles?


 * RQ4 - Do readers understand how page issues work? In other words, how they appear on a page?


 * RQ5 - Does becoming aware of page issues change readers’ perception of Wikipedia?

Additional research questions:


 * How do readers form opinions about the quality and reliability of Wikipedia pages in general?


 * Do readers care more about issues considered by Wikipedia to be of higher severity than issues considered to be of lower severity?


 * What feedback loops (if any) get activated as a result of increased awareness of page issues? For example, do mobile edits increase with page issues as referrer? Does the new issue treatment changes affect issue removal rates?

Technical challenges
The technical challenges of improving Ambox templates have a few areas of concern. To summarize, templates are inconsistent.

The output of is inconsistent in many ways.


 * across languages


 * across different types of templates


 * location inside the article

There is no consistent machine-readable way of extracting the template content.

Areas of concern

 * Text length


 * The length of many Ambox messages are very long and not suitable for mobile devices.


 * Desktop specific HTML


 * Templates using make heavy use of tables for layout, which doesn't work on mobile.


 * Language specific HTML


 * The HTML output is different across languages, making it nearly impossible to consistently extract the message content and repurpose it for mobile.

Proposed technical solutions
Use the TemplateStyles extension to create mobile friendly template CSS

To be discussed for future iteration.

Use Page Content Service to display page issues

https://phabricator.wikimedia.org/T172002

Modify MobileFrontend/Minerva CSS to accomodate existing template markup

This solution would work within the existing template structure and probably only on English Wikipedia.

Many templates contain a CSS class that marks text as. Using CSS, we could hide this extra text and only present users with the "summary" text. This is the same text that is currently presented in the page issues overlay. Many templates, notably deletion templates, do not have this CSS class. The issue of text length would have to be addressed on a per template basis.

Here is a demo of what modifying the CSS might look like.

Add structured HTML attributes to Ambox templates

This solution requires modifying templates.

As mentioned before, the HTML produced by the templates varies. It can change at any time and in any language. We shouldn't hinder these changes, but make sure that contributors can adapt them to new contexts, such as mobile.

Adding semantic attributes to the HTML could provide us with the necessary "hooks" to extract the Ambox content. We could then present it in an appropriate way on mobile. We could provide a standard set of attributes that template editors can insert into their templates. This would work regardless of the template used or language. These attributes could be "machine readable" which means they can be parsed or extracted. For example like the Page Content Service for consumption by the mobile apps. These semantic attributes would be invisible to readers. They would not effect the appearance of existing templates.

Semantic HTML attributes fall into the category of "micro>w:Microformats|microformats". The idea is to extend HTML with the ability to express very specific types of data, in a way that is machine readable. We could extend the Ambox HTML with attributes to identify the templates as "message boxes" with various properties. These properties could include things like priority, date, and actionable steps. This information could then be parsed. Consumers could include the mobile site and apps. Also presented to end-users in a mobile friendly fashion.

We could create our own standard, but there already exists a W3C recommended specification called RDFa>w:RDFa|RDFa. RDFa standardizes on how to use attributes to add rich meta-data to HTML.

RDFa defines rich data structures as "vocabularies" or sets of properties that define an data structure. We can define a custom vocabulary that describes the properties of a "message box", and use those properties in the HTML attributes.

As an example, RDFa uses HTML attributes such as,  ,  , and. The following example shows how to add them to templates.

Inventory of mobile-friendly page issues
The changes proposed above will improve most page issue templates. Two kinds of templates will receive specialized treatment. One are templates with defined short descriptions (those that contain text marked as "hide when compact"). The other kind are templates with defined ambox type. These templates will display the shorter more mobile-friendly form of the text. They will include visual indicators for the severity of the issue.

Hide when compact
To determine coverage for these changes we performed an inventory of templates. We looked at templates that contain text marked "hide when compact". The following were our results. Further documented in T189132.

Method

An extensive inventory of "page issue" templates was taken across a variety of languages using the MediaWiki API and this script. The script parses templates that belong to a specific category. For example, on English Wikipedia it parses all the templates that are members of the category Category:Article message templates. The templates are then rendered in an HTML table with CSS added to see which can be made compact or not. A sum of the compact templates is also generated.

Disclaimer

This method works for languages that actually have a category for all "page issues". Not all languages do, some only have more specific categories. Only the top-ten Wikipedias were used in this report.

High level design goals:

 * 1) Compact: page issues come in a wide variety of lengths, but screen space on phones is limited and we want to respect the reader's desire to access the content, so we decided the banners needed to be compact (two lines of text, or roughly 62px tall max).  We hope that in turn template editors will further refine page issue descriptions to (more) succinctly describe the issue.


 * 1) Easy to understand: there are over 300 unique page issues on English Wikipedia, many with beautiful, custom icons.  While editors may be familiar with each individual issue we wanted a way to simplify them for readers.  In a Vector-like manner we chose severity (i.e. type) as the main concept to communicate visually using 5 generic, globally understandable icons.

The consistent placement of the blue Learn more link makes it easy for anyone to further engage with an issue they discover.
 * 1) Easy to engage with: we think that it's important for readers to have the opportunity to easily access additional information about a page issue, and perhaps even be intrigued enough to try fixing it.  While many page issues have various links embedded in their descriptions, we wanted to focus the reader's attention on a single action: Learn more.

Here is an example of an ideal page issue banner:



Compatibility across Wikis:
Since page issues are not structured in any way across the hundreds of Wiki projects we knew there would be a large compatibility challenge.

In other words, there was no simple way for us to render all page issues from all projects in a consistent manner on mobile devices that met our design goals listed above.

Since we are most familiar English Wikipedia we decided to anchor the rendering code around how English page issues are formatted.

Page issues on some other Wikis work quite well out of the box (e.g. Japanese, Chinese), however for other projects reformatting of page issue templates will be necessary in order to get them to play nicely with how we render them on mobile.

Here are a few examples from various Wikis that explain what kinds of fixes we recommend:

Making page issues (ambox templates) mobile friendly
See