Wikimedia Apps/Team/Android/Communication/UsertestingJuly2021

Background
The Android team is working to improve communication systems in the Android app. We currently support all notifications and alerts in the Android app.

However, when users are in the app, notifications are not easy to find. We are planning to make them more obvious.

Our July 2021-September 2021 update will make it easier to find notifications in the app and make them more obvious on the lock screen. By December 2021, we will build an intuitive inbox and interface in the notification home center, with the Android system elements in mind.

July 2021 through December 2021 we will focus on logged-in users that have made at least one edit. We will build on our preliminary work from earlier months to make alert awareness for logged-out users better in the beginning of 2022. We plan to explore onboarding experiences to communication tools for users that are logged out – help them find them. This will be in addition to other positive reinforcements and onboarding workflows for logged-in users.

Before making more changes to notifications and messaging in the app, we want to gain an understanding of intuitive placements, features and workflows for our users in the app. This project page collects initial user research. We ask that participants in our research comment on this discussion talk page with their responses to the protocol below or email us at android-support@wikimedia.org.

Research questions

 * When asking a user to check their notifications, in the app, where do they click?
 * Find out about how discoverable the various entry points presented are for notifications (how easy they are to find)
 * What flows and actions do users expect when they have push notifications turned on compared to those that have it turned off?
 * What are the expectations of what a user will see if they click a bell compared to an inbox and what are the drawbacks of having just one set of notifications instead?
 * Do users believe elements of the user talk page should be in the notification center?

Protocol
1) Discoverability: Imagine that you opened your Wikipedia app to check your notifications. Which version of the app makes it clearest to you that you have a new notification? Please describe where on the screen you would tap to get your notifications. Please rank the variants below by how easy it is to find the notification icon/button.

Variant A 👇

9) We currently plan to incorporate an interface for quick reactions / default replies after tapping the aforementioned “Reply” field. In the screen below, you’ll see one of the explorations representing the concept of quick reactions. We are not planning to use emojis but would like to instead leverage common phrases, similar to how we e.g. have templates for welcoming newcomers in various ways or how admins can choose common reasons to block someone. What are some common replies or phrases you use when interacting with other editors on-wiki?

10) We’re currently working to make sure important Wikipedia notifications are seen inside and outside of the Wikipedia app. We are looking at notification reminders (e.g. receive weekly notification reminders about a revert until the notification is marked as read). Notification reminders will be modifiable through user settings. Please select your reaction to this proposed functionality.

A) Yes, I’d love to see this functionality.

B) No, I don’t like this functionality.

C) I am indifferent.

Results
We received feedback from an English Wikipedia editor, Arabic Wikipedia editor, and Indonesian Wikipedia editor.

1) Variant A is the winner The first question we wanted to understand which icon was better for discoverability. Our Arabic and English users chose A (Bell Icon top right) as their first choice, and our Indonesian user chose A as their second choice. B (top right overflow menu) was the first choice for the Indonesian user, the second for the English Wikipedia editor and the third choice for the Arabic user. C (Bottom navigation) was ranked second, and the last choice for our Indonesian and English Wikipedia user.

2) Bell Icon is the winner

All three users associate the bell icon for notifications. The inbox was understood as a link to messages. The person iconography was associated with a profile, user page or ping.

3) We are going ahead with our assumptions made in T288064, as the the consistent notifications that were prioritized by respondents were: mentions, talk page messages, email from other users, edit reverts and user right changes.

4) All respondents were in favor of the idea of having mentions separately and that mentioned they would appreciate some sort of grouping within the 'All' tab

The idea of separating the notification types on the Android app is a good one. The information is presented in an understandable and efficient way but the other notifications might be jumbled up under 'All'. Quotes from respondents: That's an excellent idea. However, it would be nice to separate some of the notifications into several categories. The 'All' section is still confusing. I think separating notifications is a good idea, but having All might bury important notifications inside, like what's happening with FB, even with the iconography used. Mentions might be merged with emails to give more sense


 * Our thoughts:* Grouping could make it easy to miss notifications, so we will see how filtering addresses the organization challenge

5) Two users prefer mentions first and all secondarily. One user prefer all first. To reduce risk we will have 'All' first and review metrics around people clicking on mentions.

Answers:

“I would prefer all notifications as the default tab. If I had to organize my notification home center, I would have tabs for each notification type and a button/text to 'view all notifications' which would remove/hide the tabs and just show all the notifications in a list.” “I suggest adding the 'Edits' section in the notification. This section contains notifications about edit reverts, page links, thanks, and translation. For the default tab, I prefer mentions because when admins give warnings, users can see them instantly.” “I'd like to see Mentions and emails merged under one name and then All. But in the All, I want to be able to distinguish easily the list of notifications, maybe use a slightly different color variation for the background + the icon.”

6) Filtering by type: Ranking (point scale from 1-9)

D (22 points): Grouping of notifications (e.g. if someone received several “Thanks” for an article edit, should it be grouped) [Grouping by Mentions]. C (21 points):** Filtering by type [action item for design] B (17 points):** Filter by project, task created: T288068 E (14 points):** Search for new notifications: Yes. Search for archived notifications: No (as technically not ready / out of scope) H (13 points):** Customizable swipe left and right gestures (No, since design feedback from experienced Android users advocated for one swipe gesture [Mark as read]   A (12 points):** Filter by language, task created (T288068)    F (11 points):** Better preferences, task created (T287477)    G (11 points):** Notifications for Watchlist (Not now, as technically not ready / out of scope)    I (11 points):** Easy access to archived notifications. New designs have been improved and it’s easy to access it

7) Page links was the only thing mentioned as missing, and it has been added in the designs (T287477) 8) One user said: More context is needed for edit reverts or article talk. Otherwise people are ok with the suggestion 9) "Thank you". "I'll fix that", "I agree", "Thanks for pointing that out" "Well done" "Very good point, it will be updated" (Task created: T288105) 10) People would like to see notification reminders (V2 option)

Research Questions

When asking a user to check their notifications, in the app, where do they click? What flows and actions do users expect when they have push turned on vs. those that have it turned off? What are the expectations of what a user will see if they click a bell vs. an inbox and what are the tradeoffs for consolidation? Do users believe elements of the user talk page should be in the notification center?

Primary feedback audience
We welcome feedback from all current and prospective Wikipedia editors.

However, we are especially interested in hearing how our tools potentially impact:
 * English Wikipedia editors in India with visual impairments
 * Hindi Wikipedia editors
 * Arabic and French Wikipedia editors in Morocco, Egypt, DR Congo and Mali
 * English Wikipedia editors in Nigeria
 * Indonesian Wikipedia editors
 * Female and Non-Binary Japanese Wikipedia Editors

The groups above have been selected based on 1>Wikimedia Apps/Team/Android/2021Roadmap|research the team conducted earlier this year about how current audiences and potential growth areas in alignment with the WMF Product Strategy.