Reading/Web/Desktop Improvements/Repository/First Prototype Feedback Report

In December we published a prototype of the first few features of the desktop improvements project for community feedback. The prototype presented a collapsible version of the sidebar, a max-width constraint on the content, and a more prominent location for the language switcher. We received detailed, thoughtful feedback from over 200 logged-in users, across five languages. The feedback is mostly positive, with the majority of users seeing the changes as an improvement over the current design. However there were also some areas of concern. Many of the issues raised were due to bugs in the prototype (particularly with the language switching menu), while others exposed areas for improvement that we will iterate on and/or keep an eye on during development. This report highlights the main points raised, both positive and negative, and our plans going forward in response to this feedback.

= Feedback method = The prototype was shown to logged-in users using central notice on the following wikis:


 * Basque Wikipedia
 * French Wikipedia
 * French Wiktionary
 * Hebrew Wikipedia
 * Persian Wikipedia
 * Polish Wikipedia
 * English Wikipedia
 * Portuguese Wikiversity

Replies were collected using a pre-populated form with specialized questions around the changes that were presented. We also asked for general thoughts and opinions around the changes in the prototype. Responders were also given the option to submit their thoughts via email. The results presented contain both the feedback we received on the page as well as via email.

We would like to note that we received a lot of comments related to bugs in the prototype, especially around the language switching functionality and the Universal Language Selector (ULS). We apologize for not providing enough information on the constraints of the prototype itself and for not catching these earlier.

= Summary of results = We only bothered to ask 230 logged-in users to give us feedback on the prototype. The majority of the responses were in English (180), followed by French (38) and Hebrew (7).

Positive


 * The majority of the logged-in users who tested the prototype preferred the new location of the language selector over the current implementation in the sidebar
 * The majority of editors liked the ability to collapse the sidebar; specifically the more focused reading view that results from the sidebar being collapsed and a max-width constraint on the content

Room for improvement


 * There were a number of concerns raised with the language switcher, most of which were related to prototype bugs as well as issues relating to language switching requiring two clicks instead of one, the new language switcher being too prominent, and questions around internationalizing the languages names in the list
 * There were some concerns around the amount of white space introduced with a collapsible sidebar and max-width constraint on the content

Feature requests


 * Dark/night mode
 * Make the table of contents always available on the page

= Results =

Positive
Overall, most users preferred the new location of the language selector - they reported it was easier to find than the previous location. People who used the language selector often reported that it would be faster for them to switch languages. A number of users also reported that the new location is more intuitive as it follows a pattern that other multilingual sites use.

Room for improvement and iteration ideas

 * One-click language switching: currently you are able to switch to a different language Wiki with a single click (and potentially some scrolling depending on your screen height). In the prototype it takes two clicks to switch languages (plus potentially needing to scroll as well). While many people noted that the new position of the language switcher was easier to find, they also expressed concern about having an extra click in order to switch languages, especially for cases where people expect  to switch frequently. There are two ways we are thinking about this:
 * We will be collecting usage data and performing A/B tests on our test wikis to determine whether language switching increases, decreases, or stays relatively the same. It is our hope that despite the extra click, the increased prominence of the language switcher will lead to greater discoverability and ultimately more language switching.
 * We’ve sketched out an option that includes links for switching languages directly on the page, adjacent to the general language selector as shown below.04_-_DIP_-_Language_links_in_article_title_bar_alt.png

However since language switching is not relevant to all people we would have to figure out some logic that handles when to show these direct links, otherwise they could be distracting. Perhaps they could appear once someone has switched languages once, or if we think they are likely to switch languages (e.g. due to a discrepancy between their operating system language and the language of the Wiki they are reading). We have not yet settled on an exact approach and plan on revisiting this idea once the initial implementation of the language switching is in place. Any feedback on this or the above mockups would be greatly appreciated


 * Issues with finding a given language within the Universal Language Selector (ULS): A lot of users liked the position of the new language selector but expressed concerns with the language selector itself.
 * Some of these concerns were around the relative difficulty of finding a given language within the selector and were in part due to the following bugs within the prototype itself:  the list of frequently used and suggested languages was missing, the language search was broken, and the menu was being rendered too small.
 * There were also concerns raised around the ordering of languages by region, and the spacing between the items within the selector leading to too much scrolling.

We will prioritize the suggestions and issues and are hoping to work with the Language team to make some improvements to the Universal Language Selector. We will update more on this soon.


 * Location & prominence: The following concerns were expressed about the location of the language switcher in the prototype:
 * Some people preferred the current location over the new location with varying reasons.  The main theme seemed to be the difficulty of using the Universal Language Selector and the loss of a one-click access, both of which are addressed below.
 * The new location is too prominent since many people won’t ever switch languages. A main goal of this project is to increase the prominence of frequently used functionality.  Currently, the language switching links are the most frequently used links within the sidebar. In addition, we believe that allowing multilingual users to switch languages is a crucial way of promoting knowledge equity and, due to the current location, many are unaware this is a possibility.
 * The new location is good but the button is too big — we have sketched out several other options for the button treatment, shown belowShowing four treatments for the interlanguage links menu.png


 * Looks like a translation: A couple of users mentioned that placing the language selector within the article view might cause casual readers to think that when they switch languages they are merely viewing a translation of the article, rather than switching to a different wiki.  Clarifying the relationship between different language Wikis is important and we are currently conducting user testing to investigate this and understand if this confusion is occurring, and how much of an issue this would present to readers.

Other notes

 * There were some requests on displaying the English names of languages in addition to the native name (i.e. displaying both Bengali and বাংলা, rather than just বাংলা).
 * Translate the word “language” A number of users mentioned that the word “language” should be translated to the language of the wiki.  This was a mistake of the prototype. The interface will be translated into the language of the wiki in the final version.
 * Some people mentioned adding some kind of indicator in the sidebar that points people towards the new location of the language switcher — we hope to explore this.
 * Some people were wondering what the language switcher will look like on articles with no other languages available — sketch coming soon.

Positive
The majority of users who provided feedback liked the collapsed sidebar for personal use, and especially for the purposes of reading. People mentioned that it removes distractions and makes reading a more pleasant experience. One user with reading disabilities mentioned that removing the sidebar makes it easier for them to focus on the content.

Room for improvement and iteration ideas

 * Do not collapse by default for logged-out users: Some users did not like the idea of the sidebar being collapsed by default for logged-out users, and expressed a preference to the current status quo.  Their reasoning was that the items in the sidebar could raise interest in editing and the inner workings of Wikimedia projects by readers. We acknowledge that with the sidebar collapsed there are fewer total entry points to editing/contributing related pages (e.g. Recent changes, etc.), however our hope is that by reducing the amount of links on the page the ones that remain (Edit, Talk, History, etc.) will be more noticeable and hopefully see more engagement. While we don’t plan on iterating the current design based on this feedback, we will be testing A/B clicks on links to the sidebar to compare how a collapsible sidebar compares to a non-collapsible one.  We will also be monitoring whether the sidebar changes have any effects on account creation (as a proxy for conversion from readers to editors).


 * Collapse by default to everyone: Conversely, a number of users mentioned that they would prefer the collapsed sidebar to be default for readers and editors, or to remove the sidebar altogether.  Currently, we don’t have plans to pursue either of these ideas, but we welcome individual communities to discuss sidebar links and their usefulness further.


 * Refine and/or relocate the open/close trigger for the menu: While many people commented that the hamburger icon wasn’t an intuitive way to open/close the menu, only one of the 220+ users were unable to open the sidebar when prompted (link to feedback). The two most common suggestions in terms of possible improvements were: 1) when the sidebar is open the icon should change to something like an “X” or “<<”, and 2) try moving the icon closer to the menu itself, rather than having it in the site header. Sketches for these two ideas are below.


 * Don’t hide random: A few users mentioned that the random page link is important to readers and a core part of the Wikimedia wiki experience.  We agree that this is an interesting idea and are exploring moving the link outside of the sidebar and into the search bar.  Search with random article button.png


 * Collapse the sections within the sidebar as well, or reduce the amount of links in the sidebar: Currently the sidebar contains many links that are not frequently used. By making the sidebar collapsible some people worry that we are ignoring the bigger challenge which would be to remove the unused links in the sidebar. Some have suggested making the sections within the sidebar collapsible. Others have suggested cleaning up the links. The links in the sidebar are determined on a per-Wiki basis, so while we agree that folks should remove unnecessary links from the sidebar this type of effort is outside the scope of our project.

Open questions, new ideas, and other notes

 * Where did the edit button and categories go? A few users mentioned that they were not able to access the edit button and list of categories within the prototype.  This was a bug of the prototype. We apologize for not mentioning it in the directions.

Context
One of the bugs in the prototype was that the content had a max-width constraint only when the sidebar was closed. Many people commented on how this was odd, and how much the content jumped across the page when you closed the sidebar. This was a mistake, not how we intended it to be — please see this updated prototype to understand how the max-width constraint will work: https://di-collapsible-sidebar-5.firebaseapp.com/Tea

Positive
The majority of people responded positively to the max-width layout, noting that it resulted in a more comfortable reading experience. Several people mentioned that having margins around the content makes it easier to focus, and having shorter line-lengths is a more optimal experience for reading. This is consistent with professional research that has been conducted (see Computer text line lengths affect reading and learning by By Peter Orton, Ph.D. IBM Center for Advanced Learning).

Our rationale
There are two reasons why we believe a layout with a max-width constraint will be beneficial to readers and editors:

1. Readability
The primary reason is to improve readability of Wikimedia wiki pages. So perhaps the first question is: how can we know what the optimal width of the content area is? There are research-based recommendations regarding optimal line lengths for readability of text, so we should probably start there. But then again Wikimedia wiki articles are different from common articles or web pages in certain ways. They are unique both in how long they are and in the variability of layout from one article to the next. Both of these factors may lead to a larger than usual need for scanning and searching for content (rather than linear reading). Our design must take into account these distinctions. We need to therefore ask, are Wikimedia wiki pages unique enough to warrant a line-length different from what is generally recommended? Below we explain how we’ve arrived at our design recommendation for what the max-width should be.

Without studying readability of Wikimedia wiki pages directly we can’t know what is optimal, but in an attempt to make an educated guess we start with the research on optimal line length for readable text. The research and recommendations in this area seem to be well established. The Wikipedia page on Line Length provides a good overview, as does the essay Size Matters: Balancing Line Length And Font Size In Responsive Web Design by Professor Laura Franz. The research study Computer text line lengths affect reading and learning by By Peter Orton, Ph.D. IBM Center for Advanced Learning is a more rigorous, academic study. The popular recommendation is that there should be between 40 and 75 characters per line. The findings of multiple studies conclude that "short line lengths are easier to read", and furthermore regarding learning and information retention "Subjects reading the narrow paragraphs had better retention than those reading the wide paragraphs". One can find many popular sites that conform to these guidelines. Articles on the online science journal Nature have a max-width resulting in ~76 characters per line, New York Times articles are ~64 characters per line, Times of India articles are ~100 characters (Hindi), Oxford Academic journal articles are ~75, and articles on the World Health Organization’s website are ~96 (Latin alphabet), ~46 (Chinese alphabet), and ~85 (Cyrillic alphabet). It is also worth noting that when using reading mode in Safari or Firefox text is rendered at ~73 and ~77 characters per line respectively (Latin alphabet).

In comparison, a Wikimedia wiki page on a browser window at 1280px* has a character count of ~170 characters per line, and that’s at the small end of the screen size spectrum. (*The most common computer screen size, accounting for 22% of users, is 1366px wide according to StatCounter; imagining a browser window at nearly full width you end up with ~1280px). Then factor in that on Wikimedia wiki the character count per line grows as the screen width grows (whereas on the other sites mentioned the character count per line remains the same, the result of max-width constraints). So on the second most popular screen-size, 1920px (21% of users), the character count per line is ~262 (again assuming a browser window at nearly full-width), more than three times the recommended value. So as a starting point we know that for paragraphs of uninterrupted text we are well over the recommended limit.

The question then becomes: why not limit the width of Wikimedia content such that we achieve the recommended line length, as other online content sites seem to? The short answer is: because our pages are different, and therefore people read them differently. Wikimedia wiki pages are very long, contain a large amount of information, and they are not uniform from one page to the next. As a result, people have a greater need to skim and search within pages than they would when reading a typical online article or book (this is supported by our research around reading time on Wikipedia – link). So while the line length recommendations provide a good starting point, we also must consider that the more narrow we make the content, the longer the page gets, and perhaps the more difficult scanning becomes (involves more scrolling, etc.) (for more information regarding different types of online reading please see this 2006 study conducted by the Nielsen Norman Group: link). Additionally, because Wikimedia wiki pages contain many elements that are floated inline alongside text it is not straightforward to achieve a specific number of text characters per line.

So how do we find a width for the content that accommodates both focused/engaged reading (shorter line lengths, less density) and scanning/searching/skimming (longer line lengths, more density)? Based on the above information it seems safe to assume we should limit the width by some amount, while still enabling readers to skim and search around, obtaining a visual map of the page without having to scroll too much. As stated earlier, without studying this directly (which we hope to be able to do during the course of this project) it is impossible to know what the optimal width is, but we can make an educated guess. While Wikimedia wiki pages are not uniform and contain a huge amount of variation in terms of layout, there are two common experiences we might want to anchor around when making this consideration.


 * 1) The top of an article, a paragraph of text situated next to an infobox
 * 2) The middle of an article, a paragraph with no elements interrupting it

We can consider these two experiences at various widths, counting the character length per line for each: Based exclusively on the recommended line length it seems like somewhere around 700px is a reasonable width. However we are trying to find a width that strikes a balance, and retains some of the density of the page to allow for easier scanning (with less scrolling). At 1000px wide an uninterrupted paragraph of text is ~154 characters long, just about double the upper limit of the recommended range. Other things we might want to factor in here are: sometimes there are floated elements that are wider than infoboxes, resulting in more narrow columns of text next to them. Also up until now there has not been a max-width, so while some editors might edit on narrower screens (or check how pages look on narrower screens) there’s likely content on pages that won’t look great at a narrower width (yet), simply because it might not have been a consideration (e.g. large tables).

Another approach that might inform our approach here is thinking about a grid-based layout (for an overview of the topic please see Building Better UI Designs With Layout Grids). This is an approach that aims for both visual harmony on the page, and making decisions about spacing, widths, etc. easier. While the Vector skin does not currently use a grid, something we could do is think about the width of the infobox as a grid column (since they are such common elements), and then use a multiple of that to determine the content width.

2. Establishing a common reading experience
The second reason we think introducing a max-width could be beneficial to the reading experience is because it would work towards establishing a common experience, which hopefully would be helpful to editors when making decisions about page layouts (note: 1024px is mentioned as a minimum size to consider in the WP:Manual of Style/Layout page, though that’s not quite the same thing). Currently an editor might be editing a page at a width of 1500px, while a reader reads it at a width of 1200px. By implementing a max-width we don’t remove this discrepancy completely (because there would still be variation below the fixed-width, for people with narrower screens), however we would be greatly limiting the range of variation.

Conclusion
After thinking all of that through we’ve come to two conclusions:


 * 1) It seems that a max-width in the range of 800–1000px is a sensible starting point. We will center the content on the page to ensure that it looks good with the sidebar both open and closed.
 * 2) It seems worthwhile to conduct a study focusing on the readability of Wikipedia articles specifically. We hope to be able to find the resources to do this.

Additional notes

A note on breaking templates / content / special pages / etc: Part of what makes Wikipedia, and other Wikimedia wikis, a powerful tool for sharing knowledge is that there are very few constraints on how information is presented. The result of this is a wide variety of different elements on the pages: tables, image galleries, diagrams, panoramic images, graphs, forms, maps, category boxes, and more. Having dealt with the challenges of designing the mobile site, and getting the content to look good, we recognize that there are going to be some situations where page content doesn’t look great given the max-with. Our plan currently is:


 * Work with our test Wiki communities to identify issues and discuss solutions using template styles or other existing tools.
 * Not to implement the max-width on Special pages. Special pages are not necessarily intended for “reading”, they often function more as lists or dashboards, and until we have time to work through the intricacies of more responsive layouts for these pages we will be leaving them alone. Here is an initial prototype of how this would work — you can switch between "View history" and "Read" to get a sense for it: https://di-collapsible-sidebar-5.firebaseapp.com/Tea

Previous conversations: this topic has been discussed in the past. Please feel free to add additional links to past conversations here.


 * 2014 – discussion from the Typography Refresh project (link)

Overall feedback and other feature ideas

 * Build a night mode: A significant amount of users made requests for a dark/night mode to be added to the scope of the project. While we were not considering this in the initial scope, we realize this is a request we’ve seen multiple times in the past as well.  We are currently in discussions on whether we can potentially add dark mode to the scope of the project and how this might affect our timeline. We will update once we know more.


 * The table of contents should always remain in view: Many people commented that having access to the table of contents regardless of how far down the page they had scrolled would be, in one person’s words, “extremely helpful” (link). We are planning to explore this functionality as part of this project.


 * Build a sticky header: A lot of users expressed interest in having constant access for commonly used actions at the top of the page.  Users requested to have access to commonly used pages and actions (talk, history, edit) as well as other functionality such as the search bar.  A similar request was the need to return to the top of the page and/or to switch sections in a quick manner. We plan on exploring this option as a part of this project.  Our current thinking is that we would like to introduce a sticky header with commonly used actions as well as a persistent table of contents.  Below are some examples of our ideas so far.  We’d appreciate any feedback around this you might have.


 * Clean up the top bar: There were some thoughts and ideas around cleaning up the links at the top of the page by collapsing them or moving them under dropdowns.  This is also something we’re considering for the project.




 * The logo is too small.