Wikipedia Subscription

Wikimedia Foundation projects Wikipedia Subscription A project to test the engagement and interaction of youth audiences with Wikipedia content by providing a daily subscription that suggests and provides Wikipedia content to read through the Telegram platform. Group: Inuka team Start: September, 2023 Team members: Runa Bhattacharjee Stephane Bisson Mary Munyoki Eduardo Medina Huei Tan Sudhanshu Gautam Uzoma Ozurumba Eli Asikin-Garmager Connie Chen Lead: Purity Waigi (product owner)

Wikipedia Subscription is a test project to validate or invalidate a hypothesis that If we disseminated Wikipedia content through chat/messaging platforms, we would be able to directly reach and engage youth audiences with our content.

Background

One of the Wikimedia Foundation product and tech departments identified areas of work in 2023-24 is the Future Audiences objectives and key results. The Future audience bucket will explore ways for the movement to become the essential infrastructure of the ecosystem of free knowledge by making knowledge available to everyone wherever they are on the internet.

The above work aims to reach the global youth who consume information on other platforms with our content to increase their awareness and engagement with our projects (Future Audience KR 2.1).

The Wikipedia Subscription aligns with Future Audience KR 2.1 and will validate or invalidate one of the hypotheses in the Inuka team's 2023-24 annual plan.

• “If we offered a wikipedia subscription service through a chat-based platforms (WhatsApp, Telegram etc), that sends daily reading recommendations in the form of articles/ summaries to younger readers, we would be able to test whether this type of experience with Wikipedia content would increase engagement. Reading recommendations would be sent directly to a group of readers to gauge their preferences by topic, platform, time of delivery, content formats etc. The impact of this test would be measured by how readers interact with the service from the number of opt-ins, recommendation preferences, CTR and subsequent page views.”

Rationale for Wikipedia Subscription

Readers are discovering content/knowledge “serendipitously” either through:

• predictive & personalised recommendations running on socials & 3rd party platforms.
• other reading patterns such as saved/ curated lists in the form of reading lists, playlists, highlights, guides etc.
• creator/publisher-driven approaches that require subscribing to newsletters, blogs, pages or channels.

We also identified some recurring opportunities from previous research projects that indicate a need for personalization or predictive content by readers.

PROJECTS	THEMES
Journey transitions research project: Current experience of readers with Wikipedia is described as Transactional.	Reader Engagement.
Reading Wikipedia research project: The majority of the readers feel that a Feed which allows you to browse various content of interest and Recommended articles based on subject of interest as the most helpful additional features in finding information on Wikipedia. Reading motivation directly influences reading behaviour; readers have multiple / varied motivations and that emerge as different reading patterns/ behaviours.	Content Recommendations. Content Resonance. Content Discoverability.
Wikipedia Preview (WPP) research project: Most readers did not discover the WPP link and as a result did not interact with Wikipedia because they skimmed through the content, had limited interest in the content or focussed on getting an overall understanding about the information given in the article.	Content Discoverability.
Wikimedia Brand Tracking Wave 4 Respondents said they would like to have recommendations of wikipedia articles.	Content Recommendations.

Based on the opportunities identified in the various research above, the hypothesis we will be testing in this experiment is:

If we disseminated Wikipedia content through chat/messaging platforms, we would be able to directly reach and engage youth audiences with our content.

The goal

To test whether a daily reading recommendation service delivered through available chat/messaging platforms is a viable pathway for engaging global youth audiences.

We see opportunities where our content is surfaced to youth audiences where they are by:

• Delivering relevant, interesting and useful encyclopedic content to their inbox/ mailbox at specific moments to read.
• Presenting content in ways that match their existing patterns & experiences.

Experiment approach

Through an Exploratory and Evaluative study i.e. Dairy Studies + Survey, we wanted to answer the following questions:

1. How do readers perceive this type of content delivery?
   • An evaluation of Moments of consumption can reveal if certain reading patterns will emerge from this type of content presentation.
2. How do readers engage with queryless discovery?
   • Content Resonance will evaluate if our recommendation engine recommends articles that closely match their current interests.

To uncover opportunities around queryless discovery for younger audiences, the team will run 3 testing activities:

1. Product discovery activities that will be:
   • unmoderated testing through Userlytics on general user interaction and perceptions on subscription-based content discovery
   • Usability tests on prototype ahead of the Diary Studies
2. Launch a Qualitative survey in English and Spanish to:
   • Uncover current reading experiences with subscriptions, the benefits users perceive, and the potential barriers they face.
3. Diary Studies with 3 participants per language (English & Spanish) across different age groups, exposed to a pilot test for one week where:
   • The participants will subscribe and use the prototype daily on Telegram.
   • The participants will be asked questions during and after testing the prototype.  

The goal is to quickly collect feedback based on first usage and actionable learnings for future product iterations & considerations.

Experiment Results

Survey results

In collaboration with the Wikimedia Foundation Design Research team, the Inuka team conducted a Qualitative survey on user interactions and perceptions on subscription based content discovery and services. 227 people responded to the survey across the English and Spanish readers. The age distribution of participants can be seen in the table below.

Target participants	Number Respondents : 227	Age distribution
		18-25	26-35	36-45	46-55
English Wikipedia	103 English respondents	7	34	38	24
Spanish Wikipedia	124 Spanish respondents	15	19	30	60

Through the survey we collected information to understand what is relevant to users in different aspects, and tabulated our findings:

Themes		Findings
Information rituals		Information rituals for users depends on the triggers and type of response/interaction required and the tool available as a medium
Information expectations: Users need the right triggers to engage. Triggers are classified under two types.	Automatized triggers, like ads or reminders. They are concrete calls to action, with no expected response from the user.	Most of the time, these kinds of messages are left behind pretty quickly. Users can just read and share the information and people visit the site. Important note: Encyclopedic knowledge is part of a personal journey. People want to showcase what they know and are interested in.
	Human triggers, such as a message on a board or group. They  have different iterations which can be in search engines and others.	Two use cases were identified  with the human triggers, they are: Quick & trustworthy messages or responses, and are mainly used to make a point. Rabbithole exploration, that is meant to result in a continuous back and forth between the conversation and the source. Important notes: The Identified barriers in encyclopedic knowledge are still the lack of “ready to be found & shared” features that other types of content such as status, photos or videos have. The above can be in the form of sharing interesting facts or quotes directly from an article or finding something impromptu when you need it. Encyclopedic knowledge here becomes part of the interaction, helping the user build arguments and answers to other conversations.
Information Journey		Understanding that triggers work differently may allow us to define the type of encyclopedic content we want to share, but also, the types of users we might find. Subscription services are not like search engines; the distance between them is still wide. There is a clear growth in the use of Social networks and other platforms for information searching purposes. Still, the English respondents were more cautious about social networks and like their information to be in one trustable place. Important note: Still, there’s a great opportunity for growth if both Social networks and Channels are paired with a trustable sources, such as Wikipedia.
Reading rituals		Engaging by rituals, instead of specific times, could be more effective for encyclopedic knowledge. For some, the morning is the best time to read subscribed content as they are fresh… For others, being able to use it as entertainment while they are waiting or commuting is relevant. A last group thinks closing the day with them is relevant. Important notes: The ability to decide how and in which way subscribers  receive the information is important, still, a second chance comes when saving for later is a possibility. Another important feature for the near future is to resume where you left your last read, that could build a good sense of control in them, but also keep them engaged.
Subscription services: 67 Spanish respondents out of 124 are used to subscription services. 56 English respondents out 103 are used to subscription services.		Subscription services are beneficial, if they have an edge because it’s a commitment. For people to use it regularly, they must understand the differential or direct benefit of the subscription and it must meet their specific expectation of: Accessible content, fit for the user's needs, which are: customization to their everyday rituals. alignment to their interests (they expect services to learn their interests fast). Ability to share what they have learnt from their perspective. Important notes: Subscribers want quicker and accurate content recommendations, but it has to match their own reading pace. There is an opportunity to learn  everything we need at the beginning of the subscription process, so we get a head start on what could be relevant for them.