Reading/Web/Projects/NewMobileWebsite

= Introduction =

Each day, more than 30 million users access Wikipedia through our mobile website. Mobile data traffic is expected to increase 7-fold worldwide from 2016-2021. This growth rate is twice as fast as fixed IP traffic over the same time period. Mobile data traffic is growing fastest in the developing regions of the Middle East and Africa, Asia Pacific, and Latin America.

The mobile web version of MediaWiki has been around for 6 years, and in that time, devices have evolved. The internet has fundamentally changed to support more people all over the world on an enormous variety of devices with varied connection, screen size, languages, and more. The internet has fundamentally changed and with it, so have the expectations of its users and their needs.

Our current infrastructure limits our ability to reach and connect to these users, both as readers and as potential future contributors, not only in terms numbers, but in terms of background. In other words, we’re limiting our community to a closed set instead of exploring the globally diverse population our movement strives to support.

Reaching More People Worldwide
Through research in 2016, we learned that many people who are interested in using Wikipedia are not currently using it due to cost and access barriers. For example, in Nigeria, the cost of internet would need to decrease 97% to be widely affordable - and that impacts how people use MediaWiki sites. Over a quarter of people we surveyed in Nigeria who are interested in reading Wikipedia reported that they weren’t able to due to the cost of data. In a more recent phone survey in Iraq, 81% of Wikipedia readers reported using the internet less due to the cost of data. In the last few years, we’ve made great strides towards decreasing the size and load time of the site, but we believe that we can do more with a more significant architectural change.

Beyond the issue of affordability is poor connectivity. People all over the world lose connection from their mobile phones for any number of reasons - they go into a tunnel, leave a wifi access point, or the power goes out. New technologies can enable us to continue to support people in those situations and more - we could even imagine providing access to articles completely offline.

Restriction of access was also a main conclusion of the New Voices Research in Brazil and Indonesia. Users in many emerging communities face difficulties in accessing the internet, including high bandwidth costs, inconsistent internet connections (which disrupt downloads), and limited storage on their phones (often less than 16GB). For many, data and device memory are precious commodities that must be carefully rationed.

Supporting varied audiences
Our users cover the spectrum of almost every device and browser, yet our list of fully supported browsers is limited. The content and user experience we currently provide to these browsers is often faulty and the level of specificity of the fixes so high that we often let users live with the bugs rather than spending the effort of fixing them on a per-browser, per-bug basis.

Furthermore, younger users who already spend many hours on commercial social media platforms are not joining Wikipedia communities in large numbers to demand that Wikimedia projects keep up with their changing technological needs. This gap extends to our communities, especially in terms of technical volunteers. Many new to the movement feel that the current barriers to entry are too high. The perceived culture of exclusivity and lack of support for newcomers is demotivating.

Cost of development and design
Our current approach often relies on duplicating efforts on the backend and frontend, as well as balancing adding functionality to mobile and desktop separately on the same stack. This has lead us to large amounts of technical debt and increased risk when making changes to support new and existing users. These risks slow down development over time, while increasing the amount of cleanup we are accountable for. It’s inefficient. We are interested in exploring architecture changes that will allow us to focus on building the right features for all platforms the first time around, rather than building for a single platform, then altering to fit the rest.

Creating a responsive experience for larger screens
Currently, it is difficult to experiment and test ideas for an improved tablet or desktop experience without creating large changes for all of our users. We would like to provide an experience which caters to users with larger screens without significantly affecting mobile-only features.

Technical Issues
Over the January and February of 2017, our engineers focused on discussing the main technical issues with the current mobile website. Below is a sampling of the discussion.

https://phabricator.wikimedia.org/T156259

https://phabricator.wikimedia.org/T156263

Identifying Potential Solutions
Over the past few years, we collected the above problems and began the brainstorming process of creating a solution which fits most if not all of the identified issues: Next steps
 * The Reading Web team set identifying solutions as a goal for Q3 of the fiscal year 2016-2017
 * Over the course of the quarter, two solutions were proposed - a refactor of the website and the building of a new website
 * Each solution was weighed against the problems identified above
 * Each proposed solution was discussed in terms of pros and cons within the reading web team as well as other WMF teams
 * The rough formulation of this proposal was drafted
 * We would like to reach out to our communities for thoughts and ideas on the proposal, including alterations and additions

= Proposal =

We propose to rebuild the mobile website with a focus on:
 * Optimization for a modern browsing experience
 * Connection management - smooth transition for short bursts of low connection
 * Data efficiency
 * Providing the ability for a fully offline experience
 * Creating an architecture that would inform decisions across Wikimedia platforms in the future

Connectivity Management
We are proposing to build an offline first application, which focuses on users with low connectivity and intermittent connections. This means that, while browsing and reading articles on Wikipedia, users will have a smooth transition when they experience intermittent connectivity - they will be able to continue reading articles and to navigate within their session. However, they will not be able to access any articles they have not previously viewed within that session. In particular:
 * Readers must be able to read articles when in conditions with low connectivity or no connectivity
 * Readers must be able to navigate through the articles they have viewed within a session in conditions with low connectivity or no connectivity
 * Readers must be aware when articles do not load properly due to low connectivity
 * Readers must be able to access the menu when not connected
 * When users attempt to select articles while offline, the system must display the appropriate messaging indicating that the user has no/low connectivity and might not be able to access the article

Large Screens
The mobile website must be optimized for flexibility, allowing reading on screens of all sizes, including all tablets and desktop screens:
 * The mobile website must be modular and responsive
 * Screen size must be taken into consideration when designing and developing new feature or altering features from mobilefrontend as it is currently

Standardized Service Layer

 * We would like to focus on standardizing our service layer for content consumption
 * Sharing infrastructure and efforts with Apps teams
 * Using services to improve performance of pages

Changes to editing functionality

 * We will be working with the editing team to create a smoother mobile editing experience:
 * All editing features currently available on the mobile website will be available on the new mobile website
 * In the future, a goal for the new mobile website would be to integrate any future improvements to editing functionality

Saved pages and lists

 * We would like to allow users to flag and save articles of interest for later consumption. Ultimately, our goal is to allow these articles to be accessible through all platforms, with the possibility of allowing the articles to be accessible offline:
 * Saved articles will not be available for offline reading upon first iteration
 * Users will be able to flag articles for later consumption
 * Users will be able to download articles in PDF format
 * Users will be able to access their saved articles when logged out or logged in
 * Users will be informed that their list of saved articles when logged out may be lost

Cost and performance

 * Must be more efficient than currently with data usage, to make browsing relatively cheaper

Other changes

 * Account creation - focus on the reader as a logged-in user
 * Reading lists syncing will require more users to create accounts, which will necessitate a reevaluation of the account creation workflow and the current method of distinguishing between readers and editors (currently we use logged in status)

Lite Experience
While the above solution is fitting for many users with low connectivity, it can only be implemented on modern browsers. We would like to explore creating or adapting mobilefrontend into a lite solution for browsers which will not support the new mobile website. This lighter experience must:
 * include all relevant content
 * be available to users based on their connection and preference
 * be opt-in
 * be fast and cost-efficient
 * be stylistically consistent with the full-featured experience

Future Offline Functionality
After building list functionality, we would like to test and explore the opportunity to deliver content while users are fully offline:
 * The mobile website must provide offline support for user selected articles
 * Users may access offline articles by flagging each article they would like to read offline from the article page
 * The article will be saved to a list of saved (offline) pages
 * Users must be prompted to save a link to their offline articles to their home screen
 * Offline support must be available for all browsers that support service workers

High-Level Technical Requirements
See Reading/Web/Projects/NewMobileWebsite/Technical_overview for a high level overview on the technical side.

As we progress we'll define more the approach and make RFCs when we need to make specific decisions.

Roadmap Outline
All changes will be tracked in the phabricator project currently named Marvin.

Milestone 1: Product plan and architecture decisions

Milestone 2: Initial development stage
 * Development plan


 * Requirements for first iteration


 * Overview of all current functionality for the mobile website


 * (if time) requirements for additional/new functionality


 * Define MVP features

Milestone 3
 * Setting up initial Phabricator tasks
 * Set up development infrastructure


 * Repo


 * Test infrastructure


 * Deployment


 * Scaffold


 * Define MVP


 * Build out application chrome and design components


 * Build out article namespace


 * Set up production deployments


 * Set up performance metrics

Milestone 4
 * Complete prototype for production deployment - page issues, mobile PDFs, search, related pages
 * Begin work on connection management workflows


 * Set up analytics


 * User testing on new features

Milestone 5
 * Deploy to test group of users
 * Iterate
 * Deploy connection management workflows
 * User testing with New Readers

Milestone 6
 * Continue building functionality - main page, special pages, login, settings
 * Deploy all reading features to beta

Milestone 7
 * Deploy editing functionality (VE and text editor) to beta and test group
 * Continue building out special pages
 * Small wiki test for full experience

Milestone 8
 * Small wiki deployment
 * Large wiki test

Milestone 9
 * Large wiki deployments

Who is Marvin?
Marvin is the internal codename for this project. We have used it for the name of the phabricator board and the ongoing prototype. We do not plan on using this name for any user-facing features or in external communication.

How will these changes impact our communities?
As noted before, the outcome of this change will be widespread. We are hoping to create a faster website that is reliable under conditions of low connectivity, which will benefit readers and editors alike. While this effort is currently contained to the reading team, we will be communicating with the editing team to draft out the implications for editing on mobile. For our first iteration, we are not expecting to develop changes to the current editing experience on mobile. As the project grows, we will be exploring ways to integrate improvements to the editing experience with the new website. We will be updating this page with any planned improvements to editing connected to this effort as we begin building and testing.

In addition, we are hoping that modernizing our infrastructure will bring an influx of technical volunteers interested in participating in this effort and future improvements to the new mobile website. Of course, at this time, this is just a theory, but reach out if you're interested in helping out.

Through this process, we will be reaching out to communities for feedback, ideas, and questions, as well as identifying communities that are interested in testing the changes.

How will this impact 3rd party mediawiki users and developers?
At this stage - we’re not quite sure, but we have many options. One of the current possibilities is to work with the current version of mobilefrontend, strip it of some or many of it’s Wikimedia-specific features, and pass it over to our volunteers and community as the default 3rd party mobile website. While the services we will be creating may be for Wikimedia-specific use cases, they will also be available for 3rd parties.

How will this project affect the desktop experience?
While we are currently considering this approach to be a mobile-only solution, we hope that these changes will inform and highlight any architecture decisions we make in the future, on both mobile and desktop web with the potential of creating a new architecture which functions smoothly across all platforms. However, we are early in the process and will be focusing on efforts that cover the above-identified problems.

In terms of functionality, we recognize that features that are useful on desktop may not necessarily translate to mobile and vice versa. Through the process of building the new website, we are hoping to identify these features and select which ones deserve replication, which may require specialized functionality, and which do not make sense to have on both platforms.

What is the rollout plan?
We plan to begin deploying the feature as an opt-in feature to a small percentage of users. As we increase the functionality and get closer to parity with mobilefrontend, we will be increasing the percentage of users with the ability to opt-in. Upon full parity, we will begin switching smaller wikis to the new version of the mobile website and deploy the new version for a small percentages of larger projects. After feedback and iteration, we will be switching all projects to the new version of the website, while giving them the opportunity to navigate back to mobilefrontend or the desktop versions of the website from any page. As functionality updates in the new version will be minimal outside of the connection management workflows, most users will not perceive any significant changes in their experience.