Wikimedia Apps/iOS Suggested edits project

Background

Suggested edits presents opportunities for small but vital contributions to Wikipedia. The aim is to raise awareness of the various ways people can edit Wikipedia while making quality contributions easier and more accessible.

Suggested edits was initially introduced on Android , and was later incorporated into the Growth experiments for the newcomer homepage . The iOS app was initially created as a reading focus app. Over the last year we focused on adding communication features. This year's annual plan created the perfect opportunity to bring Suggested edits to the iOS app.

Objective

By July 2024 we aim to release a Suggested edits task that:

• Increase unreverted mobile contributions from iOS by 10%
• 2000 articles enhanced using Suggested Edits

Hypothesis 1 alt text suggested edit proof of concept

We believe we can achieve the objective above by releasing a suggested edit task focused on adding alt text to images.

Presently, 50% of images on Wikipedia have captions, 10% of images have alt text, and only 3% have effective alt text. Users of the Wikipedia apps can go into settings when they are in low bandwidth environments and choose to not load images. If an image has alt text, the user in the low bandwidth environment will be able to read the alt text and get an idea of what the image is about. The iOS Wikipedia app was Apple's editor's choice in 2017 as a result of the app's user accessibility features. Accessibility is an important factor to our design and development process on the apps teams, so a task to fill the gap in images with quality alt text is fitting for our team.

Due to this concept being a new suggested edit type, it is important to start with a proof of concept that will allow us to evaluate if the task will be effective.

Fundamental requirements for proof of concept

• Entry point in Settings
• Prominent guidance for writing good alt-text
• Users ability to get context about the image from the article
• Users ability to access relevant metadata (can take user to Web)
• Detection of which images do not have alt-text
• Ability to store the responses we get to evaluate if they are good alt-text
• Users should be able to give feedback about the feature over all

Nice to haves

• Positive Reinforcement
• Exposing to users how many tasks or edits they've completed
• Ability to publish alt-text to Wikipedias
• Limits on number of task can be completed in a given day or session if it is being published to Wikipedia
• Entry point is easily discoverable
• Do not allow users to copy and paste in image caption
• Users should be prompted provide feedback about the feature
• Suggest Alt-text (think Machine Assisted Article Descriptions)
• Can playback what was written in preview

User stories

• As a participant at the GLAM Conference in Uruguay, I want to test out an alt-text Suggested Edits Task, to get a concrete idea of the concept and provide meaningful feedback.
• As the Director of Product, I want concrete proof an alt-text Suggested Edits would increase edits on iOS, without creating a burden for patrollers.
• As a accessibility specialist, I want to evaluate the quality of alt text submitted through Suggested edits tool, so that I can advise if it is a feature that would be helpful or harmful to low vision users

Consultation Strategy

• November 2023 - During the 2023 GLAM conference we will have attendees test the proof of concept
• December 2023 - Partner with accessibility specialists to evaluate submissions through the proof of concept
• January 2024 - Decide whether to build the full featured version of an alt text suggested edit task or pivot to another suggested edit task type

Prototype

The prototype the team built in order to get feedback can be accessed by following the steps below:

1. Visit https://testflight.apple.com/join/ETjXlUyi to download the test build
2. Tap the gear icon in the top right part of the screen
3. Click Login and sign in or create an account
4. Tap Account once logged in
5. Tap Suggested Edits
6. Review onboarding
7. Start Editing!

We collected feedback through this survey. You can read the guidance for adding alt-text via this feature on the Suggested Edits FAQ page.

Decision Matrix

• If less than 45% of edits are scored as a 3 or higher then we will pivot to a different suggested edit. If 46%-70% of edits are scored a 3 or higher we will improve guidance or use AI to better assist users. If 71% or higher of edits are scored a 3 or higher we will scale feature.
• If average edit per unique user is under 3 we will pivot to a different suggested edit. If it is 3-6, we will consider interventions to reduce friction. If it is 7 or higher we can scale
• If user edit through feature a second day we should proceed with improvements and scaling
• If less than 55% or less of users are satisfied with feature we will not scale without making changes
• If we do not have at least 50 people try the feature we will do direct outreach to gain more edits
• If more there are more than 30% of users find the task too difficult we will create an intervention to reduce difficulty before scaling. If 80% or more users find the task too difficult we will consider abandoning depending on supplementary responses
• If the skip rate is 20% higher than image captions on Android we will consider pivoting to a different suggested edit unless evidence points to an intervention that could reduce this rate

Key Qualitative Insights from GLAM Participants

• It would be good to add a bit more context: show information about the specific context of the image in the article (e.g. I had the article 'wheat' with an image of rust on wheat leaves, while I was unsure if I needed to mention the rust stains in the alt text or not). It would also be good to just mention the name of the article itself in the alt text editing screen - a few times I forgot the article and needed to click back.
• Avoid including paintings and other artwork
• Add a field that suggests things via depicts statements
• I'm still unsure whether I provide alt text that is actually useful for the target audiences. "keep it simple" is a very generic tip, maybe a short video with a bit more detailed guidelines (how much do you take the context into account?) would be welcome
• I would add categories that help me place the element in the image in case I don't recognize it. Although it is a random suggestion, it may not separate exactly what is what you are going to describe.

Quantitative Insights from GLAM Participants

• Only 9% of participants found the task difficult
• All participants found the onboarding and guidance somewhat to very helpful
• 54% of participants stated they would try the feature again if no other changes were made to the prototype
• All participants stated they would try the feature again if changes were made to the prototype
• We did not reach our goal of having at least 50 people try the feature, even with follow up with affiliate groups in Latin America.

Next Steps

We believe we did not have at least 50 people try the feature because it required people to download a separate app. Of the people that tried the feature, there were positive indicators that the tool could be successful with improvements. As a result we will run a scaled experiment in the production version of the app.

Hypothesis 2 Add an Image Suggested Edit

We have seen positive results after releasing the Image Recommendations (Add an image) suggested edits task in the Android App , where more than 2000 articles were improved in a 30 day period.

Our hypothesis is that adding Image Recommendations as the first suggested edit available in the iOS app will increase the amount of mobile contributions made in the iOS app by 10%, improving over 2000 articles.

Feature Requirements

• Entry point should reside in the Explore feed (not the first card- under top read) and a permanent home based on navigation work
• There should be a tooltip, notification or some sort of intervention to draw attention to the task for qualifying editors
• Users should have an option to add a caption and alt-text on the screen but it should not be prompted
• There must be a way for users to report issues with the feature, it should take them to the support email (pre-populated)
• Users should have an entry point in the task to the FAQ page (via overflow)
• There should be some sort of way to ensure users aren’t continuously hitting yes in a short (less than 5s) span of time to guard against bad faith edits
• There must be onboarding to the feature
• Users should be able to access full article (via read more) and image metadata
• Do not allow users to skip without providing some feedback (Yes, No, Not Sure)
• Preview the edit and provide an edit summary before publishing
• Selecting No triggers a survey (reasons should have parity with Android)
• Allow users to zoom/pinch on image
• Considerations for images running out or task not being available in certain languages
• Non-intrusive guidance for adding captions and alt-text (can take user out of the app)
• Accessibility in accordance with WCAG
• Interface must be translated in target languages prior to release
• Feature should not be available to users with less than 50 edits in the language they are editing

Nice to Haves

• Users able to filter by topics
• Subtle positive reinforcement

Target Wikis

While we welcome feedback from everyone, we are especially interested in hearing from:

• Spanish, French and Portuguese speakers in the Americas (North, South and Central) and Caribbean
• Chinese speakers in North America

In our user testing, we are committed to fostering a balanced and diverse group of testers.

User Stories

• As a Wikipedia iOS app user with a small screen and inconsistent internet, I would like to evaluate images and determine if they should go into an article, to contribute to Wikipedia articles that are in need of more content.
• When I am using the app, I want to be able to add images with one hand to many articles on my mobile device, so that I can be productive while riding the bus in Bogotà, and listening to music.

How will we know we are successful?

Our leading indicators will be captured after 15 and 30 days:

• Image rejection rate does not exceed 29%
• Edit over-acceptance rate (never skip or reject recommended images) does not exceed 35%
• Task completion rate is not below 30%
• Revert rate does not exceed 18%

Validation

• KR 1.1: 1500 articles have images in a 30 day period
• KR 1.2: Average at least 6 edits per day per unique user
• KR 1.3: 10% of eligible Suggested Editors try image recommendations task
• KR 1.4: 30% of those users complete the task again on a separate day in a 15 day period
• KR 1.5: Accept rate does not deviate from Mobile Web or Android by more than 10 percentage points
• KR 1.6: 10% increase in unreverted edits from iOS in the main namespace

Guardrails

• KR 1.1: Less than 5% of users report NSFW or offensive content
• KR 1.3: All users spend at least 10s evaluating a task before publishing it
• KR 1.4: Bounce rate does not exceed 30%
• Bounce rate defined as users that click Yes then abandon the flow before publishing
• KR 1.5: At least a 50% task completion rate
• Defined as users that click on Add an image as a task, and actually clicks Yes, No or Not sure (interact with the feature)
• KR 1.6: Revert rate does not exceed 5%

Curiosities (nice to have)

• KR 1.1: Do these numbers differ by user tenure?
• KR 1.2: Are we seeing differences in these metrics when focusing on our target audiences as compared to the general population?
• KR 1.3: At what point in workflow are most frequent dropoff events?
• KR 1.4: How often are users adding captions and alt-text (distinguish between the two)?
• KR 1.5: How often are reverted edits captioned?

How to Follow Along

We have created T355270 as our Phabricator Epic to track this work. We invite you to collaborate with us there or on our Talk Page. We will provide periodic updates on this page as we make progress.

We plan to release the Add an Image suggested edit to all Wikipedias by April 2024, and it will be available to users who have more than 50 edits.

Updates

February 2024

• Designs are complete for the Add an Image suggested edit task Image iOS (T355271), and development will begin. The flow is similar to the existing Add an Image suggested edit in the Android App.
• We will now shift to designing an alt-text experiment, which will invite users to add Alt-text to an article’s images after adding an image, or editing that page.

January 2024

• We are planning to pivot how we approach our Suggested Edit hypothesis. Instead of an alt-text task as the first suggested edit, we are going to release Image Recommendations, which will serve as the first suggested edit across all platforms. Then we will run an experiment for alt-text that will help us evaluate that suggested edit type at a larger scale.
• Designs are in review for image recommendations. T355271

• 
  1. Announcement of image recommendation feature
• 
  2. Onboarding screen for image recommendations
• 
  3. Example onboarding tool tip for recommended image
• 
  4. Choosing no requests more information
• 
  5. Choosing yes brings up image details
• 
  6 Preview of image in article
• 
  7. Add edit summary and publish

December 2023

We connected with Editoras Lx and joined their editathon December 16th to gather feedback about an alt-text Suggested Edit task on iOS. In preparation we made our feedback form automatically appear after a first edit to increase the feedback we receive. Attendees were generally positive about the prospect of the feature. Experienced editors thought it could be a good task for onboarding new editors, but also a more casual way of contributing for new editors. Attendees of the edit-a-thon expressed that editing alt-text on the Web is currently difficult because it takes you out of context of the article. There was also an expression of interest in the feature being brought to Android.

November 2023

The team released and presented a working prototype at the GLAM Wiki Conference in Montevideo. There was generally positive feedback with requests for machine suggestions and encouragement to partner with an expert accessibility organization. To test the prototype that attendees saw:

1. Visit https://testflight.apple.com/join/ETjXlUyi to download the test build
2. Once the test build of the app is downloaded, tap the gear icon in the top right part of the screen
3. Click Login and sign in or create an account
4. Tap Account once logged in
5. Tap Suggested Edits
6. Review onboarding
7. Start Editing!

The prototype is currently only available in English, Spanish and Portuguese.

In addition to having attendees test the prototype, the team shared screens of a possible ideal version of the feature.

• 
  Entry point in the Explore feed
• 
  Log in screen for non-logged in users
• 
  Topics selection
• 
  Home for Suggested edits on iOS
• 
  Onboarding to alt text adding
• 
  Tooltips to onboard people 1
• 
  Tooltips to onboard people 2
• 
  Tooltips to onboard people 3
• 
  Main UI to add alt text
• 
  Sheet can be dragged down to get more context
• 
  Article can be explored in its entirety to get more context
• 
  Tapping the image or file name...
• 
  ... leads to the Commons detail page
• 
  Tapping "Add alt text" ...
• 
  ... leads to the "Add alt text" interface
• 
  Tapping "Show alt text suggestions" reveals machine-generated alt text
• 
  Tapping one of the machine-generated suggestions fills the input field
• 
  Audio previews help to evaluate if the text has been written properly for visually impaired users
• 
  Tapping the play button reads the previously typed text out loud and highlights it word by word
• 
  After tapping "Next" on the previous screen, the edit is published.
• 
  Tapping view on the toast message before takes users to the diff view of their edit
• 
  Tapping back from the diff view takes users back to the main task UI

Our next steps are to evaluate the alt-text written by attendees that tried the feature, review the feedback through our surveys (349149) and use our decision matrix to decide our next steps for this feature.

Suggested edits- add alt text

Suggested edits- Participating in the user survey

October 2023- Usability Testing

The unmoderated usability tests have been performed with this prototype on userlytics.com with 5 participants.

• 
• 
• 
• 
• 
• 

Protocol

Background

The apps team discussed the feature's flow and initial design concepts. In addition, a review within the design team has been completed. The next step is to create a clickable prototype with a protocol to get feedback from outside our team.

Introduction

We are excited to announce that the ‘Suggested edits’ feature is debuting in the Wikipedia iOS app. Suggested edits are a new way to edit Wikipedia on iOS. It presents opportunities for small but vital contributions, so-called micro contributions. The first ‘Suggested edits’ task will be adding alt text to images in Wikipedia articles. As we strive to create the best user experience, your feedback will shape the initial version of the feature.

Questionnaire and Testing Steps

• Where would you go to access ‘Suggested edits’?
• Please navigate to ‘Suggested edits’. If you have not done so already, please navigate to ‘Suggested edits’ by tapping ‘Settings’ (cogwheel icon at the top right), then ‘Account’, then ‘Suggested edits’.
• Please narrate your understanding of the ‘How to write alt text for images’ screen and this feature.
• Where would you go to see more information about ‘Suggested edits’?
• If you have explained your understanding of the onboarding screen, tap the blue ‘Continue’ button
• Please narrate your understanding of this screen. (the screen after onboarding)
• Where would you go to see more information about the image?
• Where would you go to see more information about the article?
• Please add alt text to this image.
• If you have not done so already, please tap the blue Suggest alt text for image’ button.
• Please narrate your understanding of this screen.
• Where would you look for guidance on how to write good alt text for images?
• Where would you tap to write alt text for this image? Please go ahead and do so. (The prototype does not have typing capabilities that’s why the input field is filled automatically once users tap it)
• If you have not done so already, please Continue to the next screen.
• Please narrate your understanding of this screen, then go ahead and publish your alt text.
• Please narrate your understanding of this screen, then dismiss ‘Published’ information at the bottom.
• Where would you tap to learn more about Suggested edits on this screen?
• Can you narrate in your own words what the ‘Suggested edits’ feature is and how you think it is used? Thank you for your time!

Participant Demographics

• Age: 21-37
• Countries: Philippines, United Kingdom, Brazil, United States, Mexico
• Devices: iPhone 6s, iPhone 8, iPhone 11, iPhone SE (2nd gen), iPhone 13 Pro Max
• Sex: 1x female, 4x male

Findings

• 1 user read through the onboarding screens but still was surprised that the task was around describing images.
• 1 user did not notice the image information and had a hard time differentiating between article and image information
• 1 user wanted to edit the alt text on the image info page
• 1 user did not initially see the CTA to add alt text due to its position on the page (under the scroll)
• ‘View examples’ is not tappable, but there is a task around this. This wasn't very clear for 1 user.
• 3 of 5 users were confused about the preview screen – they believed they had published after the initial submission on the editing screen
  • 1 user suggested being able to type the text directly on the first screen and then publish vs. going to a secondary screen, seeing a preview, and then having the text be published

September 2023

The team is exploring bring suggested edits to the iOS app. Below are initial designs of a suggested edit task that adds alt text to images.

The designs were based on a comparative review of the Growth team and Android team’s implementation of Suggested Edits:

Our team then iterated on those designs to develop a design prototype that we plan to use for user testing, which will be launched in October through userlytics.