Reading/Web/Desktop Improvements/Repository/Research and design: Phase 1/en

Our research and design process for the Desktop improvements project began in May of 2019. At that point in time the project was not yet well defined (in terms of scope, requirements, timeline) so in part this research and design was intended to help give definition to the project, however that was not the primary goal. Phase I was mainly about investigation and research, figuring out where we can create value, and finding focus.

We began by considering the current default experience on desktop (Vector) and asking ourselves: in what ways can we improve upon this? Where are opportunities to modify the interface in order to create a better experience for all readers and editors? How can we make it easier for people to do the things they want to do? How can we create a more pleasing reading environment?

To help ground conversation here is a basic map of the main elements in Vector that a logged-out person sees:



Research and design activities
The research and design activities we engaged in in order to explore these questions (see lead paragraph) included:


 * Understanding the history of the desktop interface
 * Reading previous Wikipedia research conducted by the Wikimedia Foundation or other research institutions and individuals
 * Discussions at our team offsite to develop a shared understanding of the project and generate ideas
 * Reading about redesigns/updates of other large websites (Reddit, Twitter, etc.)
 * Conducting an audit of other large websites to try and glean common structural elements

Intended output
The output from phase 1 was: a better understanding of the desktop interface, and proposed focus areas for improvements. We didn't have strict criteria for what a focus area could be. Generally speaking it was an idea of an improvement we could make, though at varying levels of specificity, such as: a less cluttered reading experience, or language switching ease.

Focus areas
The proposed focus areas are:


 * Creating a more focused and "quiet" reading environment by consolidating or optionally collapsing navigational links, including:
 * Main sidebar navigation
 * Article tools
 * User tools
 * Language switching
 * Search
 * Article navigation / table of contents

Additional, more feature-specific, ideas that came up: reading preferences (e.g. dark mode), share button, larger edit button / add new article button (for smaller wikis) / making it more obvious how to "get involved", article stats / activity summary.

History of the desktop interface
The results of this investigation can be found here: /A History of Wiki Skins.

New editor experiences (2017)

 * Link to research


 * “Finding 5. The complexity and separation of how Wikipedia is made, and the community behind it, make it difficult to convert readers to editors, and new editors to experienced editors. Many new editors were confused about how Wikipedia works, or were not aware that their understanding of the model was incorrect. Some thought that Wikipedia was edited only by experts or a small group, until they noticed the edit function or learned that anyone could edit outside of Wikipedia (e.g., through news articles, friends, or social media). Once they started editing, most new editors did not understand Wikipedia’s policies and the rationale behind them, and were not aware of or had not interacted with other editors.” (p19)
 * “Finding 9. Editing processes and the mechanisms that support them (e.g., communication with other editors, help pages) are not intuitive or discoverable, making it difficult for new editors to learn and progress. Many non-retained new editors struggled to remember the basics of editing (e.g., how to login, their username, where the editing interface is located) making it difficult for them to replicate their edits and become repeat editors. Many new editors, both retained and non-retained, were unaware of or confused about the more advanced editing functions and the processes that support them (e.g., how to talk to or where to receive messages from other users, where to find help or ask questions).” (p25)

Readership survey (2011)

 * Link to research


 * Broad survey about reading
 * Finding B: “Search is the most desired improvement to Wikipedia About 32% of our readers said they were extremely/very likely (score of 9+ out of 10) to use Wikipedia more if the search functionality was improved. Better search emerged as the most desired feature, over others like more multimedia content, a better mobile site, a more simplistic design, and so on.” … “Some of the search features desired by these readers were: auto-completion for search terms and better keyword search for both Portuguese and Indic languages, and also transliteration plugins for Indic languages.”
 * This proves that search is relevant. While we don’t plan to focus on functional improvements to search it’s helpful to know that it’s an important feature to folks and therefore increasing prominence seems like a good idea
 * They measured trust and other “quality” metrics
 * “Readers compare Wikipedia favorably with most major websites” “Readers from all 16 countries in our sample compared Wikipedia's interface and ease of navigation to other Internet properties. If we look at the sample as whole, Wikipedia (8.09 on 10) was rated a close second to Google (8.44) on these measures.”

New readership data: Some things we've been learning recently about how Wikipedia is read (2016)

 * Link to research


 * Similar conclusions as How the structure of Wikipedia articles influences user navigation (2017) - basically people spend more time in the top of the article
 * Interestingly, the top of the article is where the most surrounding clutter us

How the structure of Wikipedia articles influences user navigation (2017)

 * Link to research


 * This paper is focused on how people find the information they’re looking for within “information networks”, specifically focusing on navigating via links. Their main conclusion is that people mostly click the links in the lead paragraph or infobox of an article:
 * “Our results suggest that article structure has a strong influence on navigation. We find evidence that a large share of user clicks are to links in the lead section or an infobox. For free-form Wikipedia navigation, navigation decisions can be best explained by a bias towards the article structure, favoring links located near the top of the article.”
 * They discuss how people initially scan a page to figure out whether or not the page contains the information they’re looking for:
 * “Web users have been found to quickly decide whether a page is worth their interest. Weinreich et al. (2006) report that users stay on most pages only for a short time span and that 52% of all visits are shorter than 10 seconds. Web users frequently skim a page at first to determine its relevancy (Liu, White, and Dumais, 2010; Liu et al., 2014). This behavior also shows in the analysis of click locations: in a study, 76.5% of clicks were made in the area visible without scrolling and 45% on links located near top left corner (Weinreich et al., 2006).”
 * Based on this, we can draw the conclusion that displaying all of these additional links in the sidebar might not be very effective and in fact might make it more difficult for readers to find what they need

A large-scale study of Wikipedia users’ quality of experience (2019)

 * Link to research


 * This study focuses on the “quality of experience”, determined exclusively by loading speed. They surveyed 62k people from 59k distinct IP addresses across 46k Wikipedia pages. They used quick survey to ask people if the site loaded fast enough.
 * The findings were that the vast majority of people were happy with the loading time and therefore perceive the site to be a quality experience — the breakdown was positive (84.8%), neutral (7.7%), negative (7.5%).
 * "The main takeaways in our analysis are that users are consistently satisfied, and that scores do not exhibit seasonality at circa- dian or weekly timescales, which is unexpected. Quite surprisingly, scores are also not affected by network-related events (e.g. data center switchover) happening during the period, nor by Wikipedia- related events (e.g., banner campaigns that alter the page rendering) nor by known browsers events (e.g., Chrome 69 first paint regres- sion"
 * "Additionally, we find that scores are, as expected, heavily influenced by user-level expertise and equipment (e.g., device, OS and browser), as well as network and country-level characteristics (including access technologies, ISP and economical factors). Interestingly, scores are not affected by the Wikipedia page size, nor by the device price (unless economical factors are also weighted in)." One explanation for why people with greater user-level expertise are more likely to be satisfied with the loading time is that they've become accustomed to it and aren't necessarily comparing it with other websites (as less frequent visitors might be doing).
 * Potential implications on this project: 1) the site is already very performant which might suggest it's not important to try and further improve, 2) performance plays a large role in quality perception and therefore we should be mindful of maintaining the current status quo

Quick Dive into Reader Behavior (2015)

 * Link to research


 * Small study (12 participants), only focused on mobile
 * Notes that people don’t use Wikipedia search very much
 * Part of this is because it doesn’t work well with keywords - “Strong search mental models involve keywords. It can be challenging to find a relevant Wikipedia articles using keywords to search in EN:wiki” (link to slide)
 * Also notes that one person had a difficult time finding search, and that this came up in past testing - “Hard time finding wiki search...have seen this before in guerrilla testing)” (link to slide)

Discussions at team offsite


Our team had an offsite in Prague during May 12th–15th, 2019. We had three sessions regarding different aspects of the project. The general goal was to develop a shared (emergent) understanding of the project, and begin to generate ideas for the kinds of work that needed to be done. The first session was a review of the history of the desktop interface. The second was focused on framing and design, and included a brainstorming activity to generate possible focus areas for improvements.

[TBD - Alex to add images of the framing concepts from the presentation — inclusivity, flexible container, populations graph]

Here is the output grouped into categories:

The third session was focused on technical aspects of the project, as well as scope and timeline.

Other redesigns
[TBD - to be filled in from external research doc]

Audit of other websites
[TBD - to be filled in from external research doc]