VisualEditor on mobile report

https://wikimediafoundation.org/about/mission/This page contains a rough outline of a report on mobile editing using the visual editor that is being produced as part of the Editing team's work from the annual plan, as described at m:Wikimedia Foundation Annual Plan/2018-2019/Audiences#Outcome 3: Mobile Contribution.

Strategic direction
The Wikimedia Foundation 2030 strategic direction calls on the Foundation to invest in knowledge equity, which “means focusing on the knowledge and communities that have been left out by structures of power and privilege, and welcoming people from every background to build strong and diverse communities."

The | Annual Plan for FY2018/2019 lays out how the Foundation aims to achieve these goals by focusing on the diversity of contributors and content as well as the need to ensure that mobile devices are fully supported in order to empower people in underrepresented regions and languages.

The Audiences department is focused on improving the mobile editing experience. Within the department, the Editing team is working to improve the experience of visual-based mobile editing by: Conducting research to understand the current visual-based mobile editing experience and define areas for improvement (Q1) Working with the community to determine a strategic approach to building tools (Q2-Q4) Developing and testing solutions to ensure pain points have been lessened or removed (Q1-Q4)

Objective of this report
The objective of this report is to document the current state of visual-based mobile web editing, and to identify any challenges that exist. This report will:


 * explore and document the end-to-end workflow for current users
 * define pain points in the workflow and larger scale problems outside the workflow
 * make proposals for addressing identified pain points
 * validate thinking and stimulate discussion with both senior management and the communities

This report is intended to define the current visual-based mobile editing experience and identify areas of challenge. The report will be updated based on feedback, but ultimately it’s intended to be static documentation of the current understanding of mobile editing (as of Q1 2018), and not to be a moving target to document of ALL the things. The Audience team will build upon this work and ultimately all action items will go into Phabricator tasks.

Definitions

 * mobile editing – Mobile Editing refers to any kind of contribution made to Wikimedia through a phone or tablet. Mobile content creators and editors access Wikimedia through native or web apps. The number of people who are using mobile devices to access Wikimedia projects is constantly increasing. Many users access Wikimedia sites exclusively this way. In particular, emerging Wikimedia communities have many more primary and exclusive mobile users compared to existing communities.


 * visual editor – the current technical implementation in the browser. Editing using wikitext is a barrier for many mobile contributors, due to the unfamiliar markup system and the difficulties created by the use of mobile devices in general. For example, smaller screens allow contributors to see and preview only a fraction of a page at a time, and the absence of common markup characters, such as  and , on the devices' main keyboard views raise the bar for technical know-how required to easily contribute. The visual editor within the mobile web experience solves many of the pain points that contributors have with wikitext on mobile, however, it has usability, user experience, and performance issues. ..


 * visual-based mobile editing – An experience that allows an editor to see what the end result will look like while the edit is being created. This is the general term for the approach to creating this kind of edit.


 * mobile web – The use of the internet through a handheld mobile device such as a phone or tablet.

History of visual editing on the mobile web
Visual editing on Wikimedia has evolved significantly over the years. Some highlights include:
 * Summer 2012: Wikitext editing on mobile Web was developed initially as a beta, and then was released for all logged-in users in July 2013.
 * Autumn 2013: Visual editing on mobile Web was developed alongside the desktop release. It was initially released to tablet-based logged-in beta users in October 2013, with a "switcher" to let users move back and forth between editors; pressing the edit pencil would always initially open the wikitext editor.
 * 2014: Visual editing was rolled out more widely, and has been opt-in available to all users on all devices for years up to September 2018.


 * November 2014: Mobile Web wikitext and visual editing was expanded to logged-out editors, initially on the Italian Wikipedia


 * April 2015: Logged-out visual editing on mobile Web was expanded to all wikis.

Quantitative metrics
To be written: Explanation of why these metrics are important to us, what we learn from them, and how that informs the work that we're doing.

See T202132.

Usage of common editing features
See T202133 and T202148. (sum of each item in the columns should add up to 100%) (the sum of each item in the columns shouldn't add up to 100% since an editor may use more than one feature in an edit)

Edit completion rate
See T202134.

On desktop, broken down by experience and editor. On phone, broken down by experience and editor:

Breakdown of users per-platform
See T202135. (sum of each item in the column should add up to 100%)

Median time to save an edit
See T202137.

How many edits are made using the visual and wikitext editors?
See T202138.

Monthly retention rates
See T202146.

Which parts of the edit process cause people to abort edits?
See T202147.

Explore the end-to-end workflow for current users
The editing process begins when the user taps edit. Upon tapping edit, the user is taken to a modal dialogue that presents them with three options: edit anonymously, create account, or sign up.

Upon choosing one of the actions and completing it, the user is immediately taken to the source editor, which shows them the wikitext for the article. On many wikis, the user's viewport is filled with template syntax for the infobox, so a new user can't immediately understand how what they're seeing now relates to what they were reading.

The visual editor is available on mobile web, but it is not exposed to the user in a clear way. The edit pencil at the top of the viewport allows users to switch to visual editing, and when they do so, they are presented with the mobile web version of the visual editor. This version of the visual editor has stripped down functionality compared to the desktop version:


 * certain things have been disabled, such as image editing
 * some editing tasks are only available in a simpler form, e.g. math formula editing

Define pain points in the workflow and larger scale problems outside the workflow
An important part of improving products is understanding where problems and pain points currently exist—once the problems are understood, improvements can be made. One of the traps it is easy to fall into is to assume that the problems you have with the product are the problems that everyone has—whilst this is a useful first approximation, product teams carry out processes in order to gather data that is more rigorous, more comprehensive, and more reliable.

Process
Visual Editor Expert Review Framework – This framework was developed by the team designer in conjunction with the Audiences Design team. It's based on The 10 Heuristics for User Interface Design, the Wikimedia Foundation values, and design principles.

Methods

 * 1) Expert review – UX, UI, Accessibility, Language and Engineering professionals will evaluate the tasks provided against the visual editor expert review framework.
 * 2) Usability test usertesting.com research method + demographics screener + protocol – Mobile Web Contributors will attempt to complete the tasks provided.

Personas


Although we strive to design features for all humans, in the scope of this project we have identified a type of editor that represents the "persona" of users who we believe that we are in the best position to support and delight.

These users have one or more of the following attributes:


 * are using the browser on their mobile device to edit
 * are using the visual editor within Wikipedia.org to edit
 * have interest/motivation to fix or improve existing pages (not create new pages or perform administrative functions)

This focus allows the product team to craft a rubric for prioritizing attention to contributor attributes.

The prioritization is:


 * 1) Experienced editors who are experience on mobile
 * 2) Experienced editors who are inexperienced on mobile
 * 3) Inexperienced editors who are inexperienced on mobile
 * 4) Inexperienced editors who are inexperienced on mobile

Usability test


Visual Editor on Mobile Web Usability Testing via Usertesting.com:

Of the 6 tasks tested (getting to edit mode, adding and formatting text, saving edits, adding internal links and adding citations), only the first 3 were performed without much difficulty or confusion. Though the participants (who hadn’t previously edited Wikipedia) gave generally positive feedback about the editing process, one participant gave up on the last task in frustration. Not only did participants experience bugs and various feature elements needing improvement, they were sometimes entirely unaware that they didn’t complete a task or completed it incorrectly. Given that the tasks were relatively basic/MVP-level, further design/research iteration is recommended.

Heuristic analysis
In addition to performing a usability study, a study performed by internal (within the Wikimedia Foundation) and industry experts provided insight into the quality of the tool. A dozen experts were chosen in the fields of product, engineering, accessibility, design and RTL scripts. The experts took the usability test and then rated the tool against a values-based rubric that the product designer created based off of Jakob Neilsen's 10 Heuristics for User Interface.

Overall, the product did not perform well against the criteria in the rubric. Here is a report-card themed summary of that information. Each report card mirrors a value presented in the Wikimedia Mission.



Report Card 1: We Welcome and Cherish Our Differences
* Letter case is only relevant for languages that are written in Latin, Cyrillic, Greek, and Armenian alphabet.

Make proposals for addressing identified pain points
Two broad categories for ways to address the problem:*

1. Look at the UX issues in the end-to-end workflow and improve (and, possibly, remove) steps: e.g. is it correct to have the first step be a full-screen three option modal? do we have evidence that it's bad, or just intuition?

2. Come up with alternative visual-based editing experiences, and expose those to users'''

Technical research

 * What are the technical constraints that make the current VE mobile experience difficult to improve or change?


 * What could we do to make that easier?


 * What things have been persistent challenges due to those technical constraints?


 * Would a different visual-based system have different technical constraints?

Current software constraints

This section includes issues that exist due to the current state of visual editing software and have the potential to impact mobile editing.

Platform


 * Mobile Safari does strange viewport scrolling when the keyboard is opened, mostly based around the assumption that you are focussing a short input field. MobileFrontend has some hacks to work around these, but they may need more tweaking, especially to work with the OOUI dialog system
 * Current Action: improve viewport scrolling (Phabricator ticket to be added shortly)
 * Mobile OSes often unload pages from memory if you switch tab or application, especially iOS. This would result in the editor being reloaded when you return to that tab. Fortunately with auto-save this is somewhat less catastrophic, but there may be more work to remember other user state (e.g. if the user was in the middle of creating a citation)
 * Current Action: research the rate at which this happens by adding logging for autosave restoration, which will help us determine how high of a priority it is to address (T202656 and T202148)
 * Screen space constraints affect certain editing tasks, e.g. gallery editing: currently thumbnails and editing toolbars are squashed onto a small screen; difficult to solve without rethinking the workflow
 * Current Action: improve consistency of mobile dialogs (T202575)

Code Repositories

 * The existing mobile wikitext editor code is spread between two repositories (Minerva and MobileFrontend), which makes making and reviewing changes more difficult. Some code specific to mobile visual editor is in yet another repo, VisualEditor. Streamlining the repositories will improve performance for all users
 * Current Action: move mobile editor code (T198765)

Performance

 * Mobile web is much slower than web apps, so we may need to anticipate actions not happening immediately and guard against it in the UI (e.g. prevent buttons from being clicked twice)
 * Current Action: current user testing will determine whether this is a concern we need to address

Input and language support
Types of input method can vary significantly


 * Input via hardware keyboard / software keyboard / full-screen handwriting / voice / ...
 * Candidates inserted as single code units / grapheme clusters / words / phrases
 * Different paradigms for text correction

Current Action: see VisualEditor mobile IME analysis for more information on research to better understand these constraints

Mobile development limitations
These limitations are concerns we must bear in mind as we make design and product decisions.

Platform-specific limitations

 * Mobile Safari (iOS) does not track the visible viewport size, which makes it extremely challenging to attach toolbars to the bottom of the viewport (which the Google Docs App does, for example)
 * Some browsers don't use read/write from/to the clipboard programmatically, so we can't make our own copy/paste buttons in the UI
 * Safari's webkit engine in general is considered much buggier than Chrome's blink
 * We have less control over text input than in apps. For example in Mobile Safari a context menu with [B/I/U/Copy/Paste/...] is shown on text selections, which we can't suppress

Performance

 * Devices in the wide variety of targeted markets are extremely variable in form both factor and performance. We will need to account for that variability in planning
 * Bandwidth, latency and intermittent connections will vary greatly across target audiences. Variability in performance means that certain intensive processes (e.g. visual diffs of long documents) will have particularly variable user experience

Target wikis
To measure and quantify the impact of the interventions that are better being made, the team has identified a set of target wikis where we will focus our efforts.

The criteria are Wikipedias sorted by majority-mobile editors percentage with over 200 monthly editors. From this list, some larger wikis have been removed in order to target medium-sized wikis that are likely to benefit from the changes we hope to make. The list of target wikis is below.

This breaks down into 3 WikiGroups. WikiGroups are wikis that have some common attributes that we are associating for a specific reason. For the purpose of this work, the wikis can be grouped by language.

Next steps

 * Remote team brainstorms
 * Team shareout 1


 * Design intervention planning
 * Technical feasibility audit
 * Team shareout 2
 * Call for community input
 * Triage Phabricator to reflect current thinking

Validate thinking and stimulate discussion with both senior management and the communities
We are asking for thoughts and reactions on these pain points from as many communities as possible:


 * Have you experienced similar frustrations with the visual editor on the mobile web?
 * What do you think works well with the visual editor on the mobile web?
 * Did we miss anything?

Please join the conversation on the Talk page.