VisualEditor on mobile report

This page contains a rough outline of a report on mobile editing using the visual editor that is being produced as part of the Editing team's work from the annual plan, as described at m:Wikimedia Foundation Annual Plan/2018-2019/Audiences#Outcome 3: Mobile Contribution.

Background
The 2018–2019 Annual Plan for the Wikimedia Foundation focuses on growing new contributors and content, with a major focus on mobile editing.

The number of people using mobile devices to access Wikimedia sites is constantly increasing, and there are many users who access Wikimedia sites exclusively this way. In particular, emerging Wikimedia communities have many more primary and exclusive mobile users compared to existing communities. Editing using wikitext is a barrier for many of these users, due to both the unfamiliar markup system and the additional difficulties created by the use of mobile devices, such as the smaller screens, which allow contributors to see and preview only a fraction of a page at a time, and the absence of common markup characters, such as  and , on the devices' main keyboard views, as well as many other factors.

The visual editor is available on the mobile web, and solves many of the above complexities. However, the mobile visual editor has usability, user experience, and performance issues, and is difficult for users to find in the interface. The reasons for these issues involve complex interactions between different technical-, design-, and community-based challenges. In addition to the visual editor, there is also the potential to build new visual-based forms of contribution that do not involve the visual editor itself.

The target audience for this work to improve visual-based mobile editing is new contributors, with the aim of converting these contributors into lifelong community members so that they continue to create new content.

Objective of this report
Since the official launch of mobile editing in July 2013, improvements to the mobile editing experience have been done by multiple different teams at the Wikimedia Foundation and by multiple different groups of volunteer editors and developers. Due to the piecemeal nature of the mobile editing improvements, there is little understanding or documentation of what the experience is for users, from starting their first edits to becoming regular contributors.

The objective of this report is to document the current state of mobile editing, and the challenges that exist, so that improvements can be made to the highest impact areas. In particular, the aim of this report is to:


 * list what the Editing team currently believes the main use cases for mobile editing are.
 * document the end-to-end existing workflow for users.
 * document "bigger problems" not directly related to specific steps of the workflow.
 * propose some ways of improving visual-based mobile editing.
 * stimulate discussion with both senior management and the communities about ways things can be improved.

The report will be updated based on feedback, but ultimately the report is intended to be a static documentation of the current understanding of mobile editing, and not to be a moving target to document of ALL the things. Further reports and documents will be produced, and ultimately action items will go in Phabricator tasks.

A timeline of mobile editing
Mobile editing has evolved significantly over the years.


 * July 2013: wikitext editing on mobile web for registered users is released
 * October 2013: visual editing on mobile web (as non-default) is released to production
 * June 2014: wikitext editing for anonymous and registered users is released on new native Android app
 * July 2014: wikitext editing for anonymous and registered users is released on new native iOS app
 * August 2014: anonymous editing on mobile web is released to alpha


 * March 2015: anonymous editing on mobile web is rolled out to production
 * December 2016: RFC for Wikidata description editing on Android goes out
 * February 2017: Wikidata description editing released to beta and then production on Android for Russian, Hebrew and Catalan
 * April 2017: Wikidata description editing rolled out to production on more languages on Android
 * July 2017: Wikidata description editing released to production on Android for all languages except English
 * May 2018: Android app updated to respect the shortdescription template (no longer allow editing Wikidata descriptions if a local description is present)

Current state of mobile web editing workflow
The editing process begins when the user taps edit. Upon tapping edit, the user is taken to a modal dialogue that presents them with three options: edit anonymously, create account,or sign up.

** PLACEHOLDER FOR FLOW DIAGRAM (Jess to add)**


 * Upon choosing one of the actions and completing it, the user is immediately taken to the source editor, which shows them the wikitext for the article. On many wikis, the user's viewport is filled with template syntax for the infobox, so a new user can't immediately understand how what they're seeing now relates to what they were reading.
 * The visual editor is available on mobile web, but it is not exposed to the user in a clear way. The edit pencil at the top of the viewport allows users to switch to visual editing, and when they do so, they are presented with the mobile web version of the visual editor.
 * This version of the visual editor has stripped down functionality compared to the desktop version:
 * certain things have been disabled, such as image editing
 * some editing tasks are only available in a simpler form, e.g. math formula editing
 * add more

Evaluation of current state of Visual Editor on mobile web
Our goal is to understand the landscape of concerns and opportunities for the Visual Editor. As part of the discovery work that we are doing to explore and define the right problem to solve for with our focus on new editors in the Annual Plan. We will conduct a usability study and an expert review.

Tasks
For the purpose of this evaluation, expert and novice testers were asked to perform the following tasks using the Visual Editor on a mobile browser.

1. Add new text and then format part it.

2. Link to another Wikipedia article.

3. Add new text with a citation.

Research questions
TBD

Principles used to evaluate
Visual Editor Expert Review Framework - This framework was developed by the team designer in conjunction with the Audiences Design team. It's based on The 10 Heuristics for User Interface Design, the Wikimedia Foundation values, and design principles.

Research Methods

 * 1) Expert Review - UX, UI, Accessibility, Language and Engineering professionals will evaluate the tasks provided against the Visual Editor Expert Review Framework.
 * 2) Usability Test usertesting.com research method + demographics screener + protocol - Mobile Web Contributors will attempt to complete the tasks provided.

Findings
[ link to the slidedeck findings report ]

Understanding users
- who's using VE for mobile web now?

Personas
Although we strive to design features for all humans, in the scope of this project we have identified a type of editor that represents the "persona" of users who we believe that we are in the best position to support and delight.

These users have one or more of the following attributes:


 * are using the browser on their mobile device to edit
 * are using Visual Editor within Wikipedia.org to edit
 * have interest/motivation to fix or improve existing pages (not create new pages or perform administrative functions)

- secondary persona framework for user analysis:

User Journeys
Discussion of mobile-heavy wikis -- what's the % of mobile vs desktop edits on those wikis.

Mobile personas from the New Editors report.

Quantitative metrics
See T202132.

WARNING: This section is a very early draft, and metrics may be added, changed, or completely removed from this section.

Usage of common editing features
See T202133 and T202148. (sum of each item in the columns should add up to 100%) (the sum of each item in the columns shouldn't add up to 100% since an editor may use more than one feature in an edit)

Edit success rate
See T202134. (sum of each item in the columns should add up to 100%) (sum of each item in the columns should add up to 100%) (sum of each item in the columns should add up to 100%)

Breakdown of users per-platform
See T202135. (sum of each item in the column should add up to 100%)

Median time to save an edit
See T202137.

How many edits are made using the visual and wikitext editors?
See T202138. (sum of each item in the rows should add up to 100%)

Monthly retention rates
See T202146.

Which parts of the edit process cause people to abort edits?
See T202147.

Technical concerns
[Ed/Marcella]

What are the technical constraints that make the current VE mobile experience difficult to improve or change? What could we do to make that easier?

What things have been persistent challenges due to those technical constraints?

Would a different visual-based system have different technical constraints? See below -- Would it take more work to build something from scratch, and then have to maintain that as well?

Platform issues

 * On Mobile Safari (iOS) we don't know the visible viewport size, this makes it hard to do things like attach toolbars to the bottom of the viewport (which the Google Docs App does, for example).
 * How hard vs impossible? Is there work that we could do to make this happen? Does Jess think this is likely to be a problem, and therefore something we ought to investigate now?


 * Safari's webkit engine in general is considered much buggier than Chrome's blink.


 * In general we have less control over text input than in apps. For example in Mobile Safari a context menu with [B/I/U/Copy/Paste/...] is shown on text selections, which we can't suppress.
 * Other examples? Should we catalog the problems?


 * Mobile Safari does strange viewport scrolling when the keyboard is opened, mostly based around the assumption that you are focussing a short input field. MobileFrontend has some hacks to work around these, but they may need more tweaking, especially to work with the OOUI dialog system.
 * What hacks? Do they help? Should we/can we come up with a less hacky solution?


 * Some browsers don't let use read/write from/to the clipboard programmatically, so we can't make our own copy/paste buttons in the UI.
 * Does Jess think this is likely to be a problem, and therefore something we ought to investigate further? How could we get around this?


 * Mobile OSes like to unload pages from memory if you switch tab or application, especially iOS. This would result in the editor being reloaded when you return to that tab. Fortunately with auto-save this is somewhat less catastrophic, but there may be more work to remember other user state (e.g. if the user was in the middle of creating a citation).
 * How important is this? What work will we need to do? Should we start on it?


 * There is a lot of l10n variety to support and test:
 * Certain issues will affect particular languages and scripts.
 * Types of input method can vary wildly:
 * Input via hardware keyboard / software keyboard / full-screen handwriting / voice / ...
 * Candidates inserted as single code units / grapheme clusters / words / phrases
 * Different paradigms for text correction
 * etc.
 * Interplay with viewport scrolling and text input issues (mentioned above)
 * See VisualEditor mobile IME analysis.
 * currently being investigated


 * Screen space constraints affect certain editing tasks, e.g. gallery editing: currently thumbnails and editing toolbars are squashed onto a small screen; difficult to solve without rethinking the workflow.
 * Can we list the other editing tasks that are affected? Which ones are going to be important for us?

Performance

 * Mobile web is much slower than web apps, so we may need to anticipate actions not happening immediately and guard against it in the UI (e.g. prevent buttons from being clicked twice)
 * Can we investigate how to do this?


 * Devices are very variable, in form factor and performance. We can have a good experience on iPhones and Samsung's high-end devices, but there's a lot of underpowered Android phones out there which might not cope as well.
 * It'll be important in our target wikis that we work with underpowered phones -- a lot of people in emerging markets don't have powerful phones. What are the constraints that we need to be aware of?


 * Variability in performance means that certain intensive processes (e.g. visual diffs of long documents) will have particularly variable user experience.
 * Which processes are most affected?


 * Bandwidth, latency and intermittent connections will really affect the user experience.
 * How? What should we do about it?

Tech debt

 * The existing mobile wikitext editor code is spread between two repositories (Minerva and MobileFrontend), which makes making and reviewing changes more difficult (T198765). Some code specific to mobile visual editor is in yet another repo, VisualEditor.
 * What should we do about this?

Target wikis
In order to better measure and quantify the changes that are better being made, the team has identified a set of target wikis where we will focus our efforts.

The criteria are Wikipedias sorted by majority-mobile editors percentage with over 200 monthly editors. From this list, some larger wikis have been removed in order to target medium-sized wikis that are likely to benefit from the changes we hope to make. The list of target wikis is below.

Options for improvements
Two broad categories for ways to address the problem:


 * 1) Improve VE and expose it more to users:
 * 2) Look at the UX issues in the end-to-end workflow and improve (and, possibly, remove) steps
 * 3) e.g. is it correct to have the first step be a full-screen three option modal? do we have evidence that it's bad, or just intuition?
 * 4) MEASURE MEASURE MEASURE
 * 5) Look at performance, measure it, figure out how to improve it, or indeed if it can be improved to a suitable level
 * 6) Performance breakdowns by common devices, etc.
 * 7) Does section-level editing help solve these issues?
 * 8) Look at VE proper, and whether the UX is actually optimised properly for mobile
 * 9) do we remove buttons, strip functionality?
 * 10) How do we convince communities that it's something worth using?
 * 11) Most communities, including many large communities, have VE as the default experience on desktop (German, Portuguese, Russian WPs, etc.)
 * 12) Some large communities do not (English, Dutch, French WPs, etc.)
 * 13) None have VE as the default on mobile - how do we change that, and do we even want to change it?


 * 1) Come up with alternative visual-based editing experiences, and expose those to users
 * 2) Things like sentence-level editing allow a visual-based editing experience that is more lightweight
 * 3) How would such experiences be exposed to people?
 * 4) Would it take more work to build something from scratch, and then have to maintain that as well?

It's not necessarily an either-or to these options, but trying to do both simultaneously would almost certainly be a mistake...? Perhaps one could be tackled, then the other.

Next steps
List specific action items and timelines:


 * Heuristic evaluation
 * Comparative review of mobile editing UIs.
 * Usability test
 * Soliciting feedback from management in the Foundation (e.g. Katherine, Toby, etc.)
 * [add timeline]
 * Soliciting feedback from users
 * [add timeline]