Wikimedia Apps/Team/Wikipedia Year in Review/2025 Year in Review/Year in Review Reading List Experiment
Summary
[edit]This page details an A/B test experiment that the Android app will be running alongside the Year in Review feature. We'll offer users the ability to create a Year in Review-related reading list with their top read articles and save it to their reading lists. This A/B test aims to boost logged-in reader retention and multi-day engagement, contributing to the efforts of the 2025-2026 annual plan where we engage core consumer audiences.
Background
[edit]As part of the annual plan, we are testing improvements to Year in Review, to learn if it results in an increase in overall app retention for logged-in users. We ran user testing with the iOS version of Year in Review, and heard that many users were not interested in/motivated by the custom app icon we offered in the 2025 designs.
We also heard requests in last year's iOS Year in Review for a reading list based on their year in review. We plan to build an experimental version of a Year in Review reading list, to see if users are interested in receiving this type of reward after the Year in Review.
User story
[edit]As a Wikipedia reader, I want a record of the articles I read the longest in 2025, so I can remember this time looking back.
Feature Requirements
[edit]- Run as an A/B test: 50% of users receive reading list, 50% do not
- Available only to logged-in users
- Copy or branding should indicate that this is related to the Year in Review
- Only show to people after they have viewed at least 1 slide of the Year in Review
- Stop showing entry point after March 31, 2026
- Do not show to people who have Year in Review turned off in Settings
- Offer the reading list within "Saved"
- Show an active state on “Saved” navigation tab
- Upon opening “Saved”, users see a Dialog to create the reading list & save it
- Create the list using the articles they read the longest in Minutes from their history
- After the reading list is saved, it should operate like any other reading list
- Include a survey to get feedback on the reading list
How will we measure success
[edit]Validation
[edit]- KR 4.1 - 1% increase in overall app retention among readers who were offered the reading list (B) as compared to those who were not (A).
- KR 4.2 - 10% increase in Reading list engaged retention measured for the experiment group relative to control engaged
Guardrails
[edit]- GR 4.3 - At least 80% of users rate the Year in Review reading list feature as satisfactory or neutral
- GR 4.4 - No more than 10% of survey respondents complain about privacy concerns related to the reading list (survey feedback)
Curiosities
[edit]- CR 4.5 - What did the funnel look like for the reading list? How did it compare to the custom app icon funnel?
Designs
[edit]-
Create your reading list prompt -
Year in Review 2025 reading list
How does this work fit into the Wikimedia Foundation's Annual Plan?
[edit]Under the Wikimedia Foundation's Infrastructure Goal, and within the group of objectives focused on Wiki Experiences, is an objective related to improving the experience of consumers:
Wiki Experiences 3: Consumer experience Objective: Readers from multiple generations engage, and stay engaged, with Wikipedia, leading to measurable increases in retention and donation activity
Several Wikimedia Foundation teams are committed to working on projects under the WE3.3 Key Result: Draft Hypotheses.
Wiki Experiences 3.3 Key Result: Demonstrate a practically significant increase in logged-in reader retention, as measured through A/B testing of one feature per platform, by the end of Q2 FY2025-26.
| Hypothesis | Timeline | Phabricator epic | Status |
|---|---|---|---|
| Wiki Experiences 3.3.9
If we release Year in Review on Android and A/B test offering engaged users a reward to save a custom reading list, we will see a 1% increase in overall app retention rate amongst readers offered the reward compared to those who were not. |
11 October 2025 – 31 December 2025 | T404267 | In progress |
How to follow along
[edit]Our work is being coordinated on this phabricator epic: T407039. Updates will appear on the Updates page each month. At the end of the experiment, we will share results and next steps on this project page.
Feedback? Leave a comment on the project discussion page—your insights shape this experience!