Wikimedia Apps/Team/iOS/Tabbed Browsing (Tabs)/New Tab Experience and Recommendations Experiment

Overview and Version 1

Version 2: New Tab and Recommendations Experiment

Background

Until this year, the iOS app has not supported tabbed browsing. Our V1 of Tabbed browsing released in June 2025 focuses on the core functionality. We believe that if we further improve a core feature like tabbed browsing, readers will be even more likely to spend time in the App day after day, increasing retention.

Annual Plan Alignment 2025 - 2026

WE 3 Objective: Readers from multiple generations engage, and stay engaged, with Wikipedia, leading to measurable increases in retention and donation activity.

WE 3.1 Key Result: By the end of Q2, demonstrate a practically significant increase in logged-out reader retention, as measured through A/B testing of one feature per platform

Our hypothesis is 3.1.1: If we A/B/C test two new versions of the “new tab” experience on iOS, we’ll see a 5% increase in multi-day usage among Tabs users from one of the variants, and decide which "new tab" experience should be default.

Feature requirements

• Set an A/B/C test
• Create new tab experience that has main page tab on first launch (Group B)
• Create new tab experience that has empty state on first launch (Group C)
• Create tab overflow menu
• Allow users to hide and show recommendations
• Compatible with Dynamic Type and Voiceover
• Compatible with iPad
• Create bottom module that shows Did You Know or Because You Read content
• Code cleanup from first attempt at more dynamic tabs
• Tabs overflow menu
• Close all tabs flow
• Update survey
• Highlight active tab

Test structure

We'll test improvements to tabs with 2 new variants. These variants are focused on improving the experience of opening a new tab, and discovering content thorugh the main page and other forms of recommendations.

• A group
  • No changes to tab behavior, users still must have 1 tab open
  • New tab remains the Main Page, search is not active on opening a new tab
• B Group
  • On first launch of tabs users should see a Main Page tab in their overview (which can now be closed).
  • If a user opens a new tab it should take them into a focused search
• C Group
  • There is no "Default" open tab of the main page. Users start with an empty state (0 tabs)
  • When opening a new tab it brings them to main page, search is not active
• BOTH Group B & C
  • Users can have 0 tabs. If someone closes all tabs, they see an empty state
  • Add a module at the bottom of the tabs screen that has Did You Know if they have less than 2 tabs and showing Because You Read content based on their most recent tab should they have 2 or more tabs open.
  • Empty state illustration and text
  • Recommended content pinned to the bottom of the tabs overview screen
  • Overflow menu with the ability to hide recommendations AND close all tabs
  • Highlight the active tab
  • Show the survey

Audience

• Readers with an App primary language of English, Arabic, or German
• Readers with a device country of Germany, or within ESEAP or MENA regions

How we will measure success

Key Results

• KR 1.1 5% increase in multi-day usage of the tabs feature among logged-out Tabs users in Groups B and C compared to Control, Group A.
• KR 1.2 1% increase in overall app retention among logged-out users users in Groups B and C compared to Control, Group A.

Guardrails

• GR 2.1 No more than 5% decrease in “new tab” events from the tabs overview (Compare groups B & C against Group A)
  • New tab sessions = users who open a new tab from the Tabs overview using the "+" button, or from opening a recommendation in "Did you know" or "Because you read" in the tabs overview
• GR 2.2 No more than 15% of unique users turn off the article recommendations using the overflow menu

Curiosities

• CR 3.1 How many people take screenshots of the Tabs overview?
• CR 3.2 What tab experience do users prefer (B or C)? (survey satisfaction for each group, qualitative feedback)
• CR 3.3 What article content experience (Main page, Did You Know, Because you read Recommendations) receives the most engagement broken down by experiment group? (Which one receives the most clicks, and new tabs opened)
  • For group B, which resulted in the most engagement/article views? (Main page, Did you Know, because you read)
  • For group C, which got the most most engagement/article views? (Main page, Did you Know, because you read)
• CR 3.4 Which experience (B or C) leads to the most article engagement? (What is the average # of new tabs opened per unique user for each group?)

How to follow along

For further details and to follow the work, see the related Phabricator Epic Ticket: T396055.

Designs

Test Variant B

• 
  On initial entry, users see the main page as a default tab. A "Did you know" fact shows when users have fewer than 2 tabs open.
• 
  Upon opening a new tab, users land on the main page with Search active
• 
  When a user has more than two tabs open, recommendation surface
• 
  New overflow menu allows users to close all tabs, and hide recommendations

Test Variant C

• 
  On initial entry, users do not have a default tab open. A "Did you know" fact shows when users have fewer than 2 tabs open.
• 
  Upon opening a new tab, users land on the main page
• 
  When a user has more than two tabs open, recommendation surface
• 
  New overflow menu allows users to close all tabs, and hide recommendations

Results

• Our overall hypothesis was: If we A/B test adding recommendations into the Tabs overview on iOS, we’ll see a 5% increase in multi-day usage among tabs users.
  • This was not supported. Displaying recommendations within the tabs overview caused a slight increase in average pageviews per user, and the frequency with which users open new tabs, but it did not move feature retention, or overall retention.

• Key metrics:
• Goal: 5% increase in multi-day usage of the tabs feature among logged-out Tabs users in Groups B and C compared to Control, Group A.
  • Actual: There was not a practical or statistically significant change in feature retention of logged-out users in the test groups.

• Goal: 1% increase in overall app retention among logged-out users in Groups B and C compared to Control, Group A.
  • Actual: There was not a practical or statistically significant change in overall app retention of logged-out users in the test groups.

• Guardrail metrics:

• Goal: No more than 5% decrease in “new tab” sessions from the tabs overview.
  • Actual: We saw an increase in New Tab events for both test groups:
  • Group B: +4.5%
  • Group C: +19.6%

• Goal: No more than 15% of unique users turn off the article recommendations using the overflow menu.
  • Actual: The article recommendation turn-off rate was very low — only 0.3% of test group users turned off recommendations.

• Curiosities:

What tab experience do users prefer (B or C)?

• Qualitative feedback showed:
  • Group C had a slightly higher satisfaction rating (+1.8%).
  • Group C had a lower dissatisfaction rating (-10%).

What was the engagement level with the different recommendation systems (Main page, Did You Know, Because you read)?

• “Because you read” received more engagement than “Did you know”.
• 1.2% of users who opened a new tab, opened at least one new tab from the “Because you read” module.
• 0.4% of users who opened a new tab, opened at least one new tab from the “Did you know” module.

• Main Page impressions:
  • Control: 20% of unique users
  • Group C: 17%
  • Group B: 10%

We weren’t able to measure new tabs or articles opened from the main page.

Which experience (B or C) leads to the most article engagement?

• Both test groups showed higher article engagement than Control:
  • Group C: +8.9% in average daily page views
  • Group B: +7.9%
• In average new tab events per user:
  • Group C: +20.6%
  • Group B: +4.2%

Updates

November 2025

• We launched the A/B/C test to users with an App primary language of English, Arabic, or German, and with a device country of Germany, or within ESEAP or MENA regions., and will complete on November 18. Our work is coordinated on this Epic: T396055
• We continue to see requests for “close all tabs” come in through the app store reviews. After this experiment concludes, we will expedite releasing the parts of this experiment that allow users to close all tabs.
• We measured our experiment’s impact, and the results are shared above. Our next steps are:
  • Scale the new tab experience and “close all tabs” functionality from variant C to all users T411060
  • We will not scale the recommendations “Based on your recent tabs” or “Did you know” modules yet until we’ve completed further investigations.

September 2025

• We added “Did you know” from the Wikipedia Main Page to wikifeeds so that it can be repurposed within this feature T398884

• Designs were finalized and engineers began developing the feature. Designs are available on the EPIC.