May 2021: User testing report now available
In early 2021, the Web team and Design Research team contracted three independent research groups to study the usage of two proposed features: a fixed "sticky" header and a persistent table of contents. Our goal was to allow frequently used functionality to be available throughout the page, thus reducing the time people scrolled up and down the page looking for the tools they need. We tested new readers, casual readers, and editors in three different countries - Ghana, Indonesia, and Argentina and three different languages - English, Bahasa Indonesia, and Spanish. This report shows the findings of our studies. Overall, both of the proposed features were positively received by the study participants.
March 2021: New search widget live on pilot wikis, A/B test in progress for logged-in users
We are happy to announce that our new search widget is now available on all of our current pilot wikis by default. The new search functionality optimizes the search experience by providing context on search results, such as images and descriptions, making it easier to find the correct results.
We are also performing an A/B test on pilot wikis, for logged-in users only. The A/B test provides the new widget for 50% of logged-in users and compares it to the old search for the remaining 50%. The results of the test will allow us to measure the success of the feature and to identify areas for improvement and iteration. The test will run for 2 weeks, after which, we will analyze and publish the results.
January 2021: New search widget and other general updates
Over the past few months, we have been working on the new search widget, which will optimize the current experience by providing context, such as images and descriptions, for searches. While we have experienced some delays with this deployment, we are currently scheduled to deploy some time in February.
We have also begun reviewing some of the data on the Desktop Improvements project as a whole, as well as on individual features. We have published a report on the usage of the collapsible sidebar. According to the data, logged-out users are more likely to keep the sidebar collapsed than logged-in users. These results gave us confidence that our eventual default will be keeping the sidebar open for logged-in users and closed for logged-out users.
In addition, we also have begun looking at the rates at which logged-in users are opting out of the desktop improvements on our pilot wikis. The average opt-out rates for all logged-in users were between 1.78% on euwiki and 4.09% on hewiki. For active editors, the range was between 5% and 14%. We will be looking at this data in more detail in the weeks to come.
Finally, we wanted to give a quick timeline of the next few months:
- February 2021 - new search widget deployed to pilot wikis. The pilot wiki list will expand to include Portuguese Wikipedia, Turkish Wikipedia, Korean Wikipedia, Serbian Wikipedia, and German Wikivoyage.
- February 2021 - Second round of prototypes for logged in users. We will be running a second round of prototypes for the sticky header and user menu with logged-in users over the month of February. We encourage everyone to review these and give us feedback!
- March 2021 - We are currently focused on improving our language switching capabilities. We plan on deploying the new functionality this March to our pilot wikis.
September 2020: New location of search bar now available on all wikis
We have deployed the new location of the search functionality to all projects. The new location is available by default for anonymous users on our early adopter wikis, and by preference for all other users.
We are also performing an A/B test of the new location with logged-in users on our early adopter wikis. 50% of logged-in users are seeing the new experience, while the other 50% are seeing the old experience. This test will last two weeks. Our hypothesis is that the group with the new experience will search more frequently. Results will be available in approximately one month.
Basque, Farsi, and Hebrew Wikipedias, as well as French Wiktionary and Portuguese Wikiversity have now received the new version of the Vector skin by default. For now, these changes include the collapsible sidebar, maximum width, and new header. For feedback and questions - please head to our talk page.
We have now deployed our collapsible sidebar, maximum width, and new header to all projects as a user preference. To opt-in - go to the appearance tab on your preferences page and uncheck the "legacy vector" option. Once opted in, you will receive all future updates as soon as they are ready. For feedback and questions - please head to our talk page.
Our first change, a collapsible sidebar, allows users to collapse the lengthy menu on the left side of the page. We believe this change improves usability by allowing people to focus on the content itself - on reading, editing, or moderating.
Our second change introduces a maximum line width to our content on pages such as article pages and discussion pages. Studies have shown that limiting the width can lead to better retention of content, as well as a decrease in eye strain. (please review our FAQ for a list of the literature reviewed)
We deployed our collapsible sidebar on officewiki and testwiki. You can see it by going to either of those projects, or by appending the url parameter ?useskinversion=2 to the url on any project.
March 2020: User Research with Readers - phase 1 report
Starting in January 2020 we have been working with Hureo, a user research firm based in India, to perform a user study on how new and casual readers use the desktop interface of Wikipedia. The outcome of the first phase of the study is a report detailing their observations from 24 user interviews.
The study was separated in two phases, with the first phase focusing on primarily English readers, and the second phase on bilingual and non-English readers. The goal of the study was to understand the experience of new and casual readers, both in terms of how they feel about the site and how they use the site, focusing in particular on the concepts of trust and welcomeness, as well as on the usability of commonly used features on the site. In addition, we were also interested in uncovering potential areas of improvement in terms of overall and feature-specific usability. We will be using these results to inform future feature development for desktop improvements project.
March 2020: Full results from prototype feedback
In December, 2019 we published a prototype of the first few features of the desktop improvements project for community feedback. We received detailed, thoughtful feedback from over 200 logged-in users, across five languages. We have published a report which highlights the main points raised, both positive and negative, and our plans going forward in response to this feedback. The prototype presented a collapsible version of the sidebar, a fixed-width layout, and a more prominent location for the language switcher. The feedback was mostly positive, with the majority of users seeing the proposed changes as an improvement over the current design. However there were also some areas of concern. Many of the issues raised were due to bugs in the prototype (particularly with the language switching menu), while others exposed areas for improvement that we will iterate on and/or keep an eye on during development.
February 2020: Update on initial features and overall feature sequence
- We have began building the opting-in and opting-out structure for the project. The setting for turning the improvements on will be within the user preferences list for logged-in users. For test wikis, the improvements will be on by default, but logged-in users will be able to turn them off anytime via their preferences or a button in the sidebar. Please see the Opting in and Release Plan page for more details and mockups.
- We have also published the current list of features considered for the project as well as the sequence we plan on building them in. As we will be testing each individual feature prior to building, this list is subject to change based on the feedback we receive.
February 2020: Feedback Round 1 Summary
We have finished our first round of feedback for the prototype of the first few features of the desktop improvements project: the new header, collapsible sidebar, and improved language switching. So far, the results have been mostly positive, with the majority of users seeing the changes as a significant improvement over the current design. However, we also found a few areas for improvement that we will iterate on and consider during development. Here are a few highlights of the results so far:
- A majority of the editors who tested the prototype really liked the new location of the language switcher.
- A few editors raised concerns around internationalization and the ability to switch languages using one click.
- A majority of the editors liked the collapsibility of the sidebar, especially for readers
- There were some concerns around the amount of white space introduced with a collapsible sidebar and fixed-width layout
- We saw many requests for a dark/night mode for the site
We are currently running the second feedback round on English and Polish Wikipedias. If you haven’t had a chance yet, please let us know your thoughts on the prototype page. Once both feedback rounds are completed, we will be publishing a more in-depth report.
January 2020: Language Switching Users Tests
In December we did some usability testing to compare the current placement of the Universal Language Selector (in the sidebar) with the proposed location in the article header. We tested a prototype with 21 users in order to determine if people have an easier time switching languages given the new location of the language switcher. The results of the test confirmed our hypothesis — participants in the test group (new location) were able to switch languages more quickly than participants in the control group (old location). Based on these results we plan on continuing to explore moving the language selector to this location as a part of the desktop improvements project.
December 2019: Prototype testing
Between December 2019 and February 2020, we will be performing tests on a prototype for the first few features of the desktop improvements project. We will be gathering feedback from a variety of test wikis by encouraging editors to participate using a central notice banner. We ran the banners and received feedback from the majority of our test wikis during December 2019. In February 2020, we will continue running the banners on English and Polish Wikipedias. So far, the feedback we have received is mostly positive but we have also identified some areas of our prototype that we will iterate on based on the feedback. We will be publishing the results of the first round of feedback over the next few weeks, and of the second, sometime in February. In the meantime, we encourage you to give us feedback (if you haven't yet) on the prototype page.
October 2019: Technical Research pt 2
We have published the results of our technical research in preparation for beginning the work on the project. In particular, we have focused on gathering more information on the following:
- The possibility of serving a different version of the site to logged-out users (for opt-in/opt-out and A/B testing purposes)
- Search and the differences between the current search widgets
- Language switching and possibilities on improving the language switcher
- T234907 - A technical RfC on what code-base to start with
- Changes to EventLogging and how they will impact our work
September 2019: Wikimania research report
During Wikimania 2019, we interviewed editors with the goal of sharing the plans for our upcoming Desktop improvements project, and collecting valuable feedback on a number of preliminary design ideas. Our research consisted of user interviews, a free-form feedback exercise, and a presentation with breakout groups for more focused discussion. We have published a report and a PDF summary (in English) of the feedback we received. Overall, we received positive feedback on the focus areas selected, as well as the individual prototypes for ideas. However, we were also able to identify areas for improvements.
We will be iterating over this feedback over the next few weeks and plan on developing a prototype that we can test with a wider audience across wikis.
In the slideshow below is a sample of 19 of the ideas we tested. For more context, please read the full report and then give us feedback on the talk page!
September 2019: Desktop usage and behavior data analysis
As a part of our research process, we wanted to learn more about the way people currently use the site. In particular, how often they use available functionality such as links in the sidebar, language switching, and search. We have published our results. Overall, usage of sidebar links is low - only about 0.5% of all logged out users and 1.6% of all logged-in desktop users clicked on one of the pages linked in the sidebar. Language switching usage varied, generally based on the size of the wiki, with smaller wikis switching languages more often.
August 2019: Research and brainstorming at Wikimania
Wikimania provided us with the opportunity to speak with experienced members of our communities. Over the five days of the conference we were able to share the plans for the project, and collect valuable feedback on a number of design ideas. Our research consisted of user interviews, a free-form feedback exercise, and a presentation + brainstorming session (you can see the slide-deck we used at the side). We will soon be publishing the summary of the feedback, as well as interview session results.
August 2019: Technical Research
To explore different technical possibilities for the project, the team spent a week hacking on different approaches to a single problem - how to enable the sidebar in the desktop experience to be collapsible. We are hoping to use the results from these experiments to determine the technical architecture for the improvements, as well as the skin we would like to build these improvements within. Here is a list of the experiments themselves:
- Building a collapsible sidebar using a fork of the vector skin
- Building the desktop sidebar within the Minerva skin. An exploration in the question: "is there a world where we have the same skin for desktop and mobile?"
- Building a collapsible sidebar using user styles only