Wikimedia Research/Design Research/Reading Team UX Research/Top Chrome Usability

Study authors: Sherah Smith & Nirzar Pangarkar

Stakeholders: Mobile Web

Phabricator Ticket: https://phabricator.wikimedia.org/T147641

Background
In ongoing efforts to reshape the mobile web browsing experience, some branding changes have been taking place. A series of tests are going to measure the usability of said changes.

Goals

 * Gather usability feedback: We want to make sure that the Menu and Search buttons on the left side of the top bar on mobile web are easily switched between, and that the user’s goals for each action are readily and easily achieved on the first try. This will inform how we iterate the layout of the top bar on mobile web.
 * Gather branding feedback: Do users recognize the brand? Do they recognize they are inside Wikipedia?
 * Another side-goal is to test out the execution of smaller usability tests, in order to streamline the process and make it easier for teams to rely on simple, quick feedback from the UserZoom platform.

Research Questions

 * Do users recognize that they are inside Wikipedia? (Assess brand awareness level)
 * Can a user navigate to the side menu on first attempt?
 * Can a user navigate to a search on first attempt?
 * Which is more common: users try to tap the magnifying glass to initiate a search, or users tap inside the search box to initiate a search?

Methodology
UserZoom remote unmoderated usability test. 8 participants.

Participants
Readers, half men, half women. Minimum 6 participants.

Schedule

 * Gather info/assets: 10/17-10/19
 * Build test inside UZ & get designer signoff: 10/20
 * Recruiting: 10/20-10/21
 * Test results come in/QA’d: 10/24-10/26
 * Analysis/Synthesis: 10/27-10/28
 * Delivery: 10/31 or first available date w/stakeholders

Protocol
Follow up Questions
 * 1) (Starts with this page: https://trending.wmflabs.org/en.wikipedia/Halloween ) Tell me, out loud, a little about what you’re seeing here on this page. Take a minute to poke around a little.
 * 2) Thanks for your descriptions. Next, please search for the word “Sloth”. Say aloud what you are doing, and describe what happens when you have performed the search for “Sloth.”
 * 3) Now search for “Octopus.” Say aloud what is happening, and describe what you see after you have performed the search.
 * 4) Now go to the main menu. What are the options listed in the main menu, and (without selecting the options) what do you think they would lead you to?
 * 5) Now close the menu. Click on any interesting link in the page you see when you close the menu. Describe where you are on the internet, and what the pages you’ve been viewing are for.
 * 6) Open the main menu once again, and then close it. What do you think about the menu?
 * 1) Did you think it was easy to search for something?
 * 2) What did you think you were testing today?
 * 3) Do you have any feedback about what you saw today?

Do users recognize that they are inside Wikipedia? (Assess brand awareness level)
Based on the feedback from the video recordings as well as survey responses, we feel confident that participants do understand that they are using Wikipedia, rather than browsing "the Internet" or "a search engine."

Can a user navigate to the side menu on first attempt?
Yes. There were no issues locating the menu. One out of 16 people had issues understanding how to close the menu, so we are rating its usability as an A+.

Can a user navigate to a search on first attempt?
Yes. There were zero issues locating and using the search bar throughout the tests.

Which is more common: users try to tap the magnifying glass to initiate a search, or users tap inside the search box to initiate a search?
Only one person tapped the magnifying glass inside the search bar on Prototype A. There were no usability issues with tapping into and using the search bar throughout the tests.

Final Questionnaire Answers
There are two batches of answers. The first tables come from Prototype version A tests, the second from Prototype version B.

1. Did you think it was easy to search for something? 2. What did you think you were testing today? 3. Do you have any feedback about what you saw today?