Wikimedia Product Infrastructure team/Push Notifications Infrastructure

Background
The Wikimedia product teams have a long-standing desire to leverage push notifications to drive user engagement and retention. Product Infrastructure is working on designing and building a push notifications platform for Wikimedia. The goal of the project is to use push technology to provide MediaWiki notifications to Wikimedia web and app users.

I want to contribute to the project
You can find more information on how start contributing to the project in the project repository homepage.

I want to add a new push notification to my app
Our push notifications implementation is heavily based on Echo, so in order to integrate your app with push you need to also interact with Echo.


 * You can register new tokens from your users devices querying the  on MediaWiki action API.
 * For example:
 * Echo sends a  push notification to the registered devices when a new   is created.
 * Here you can find more information on how to register a new Echo notification.
 * When a new  push notification is received on the device, you can query the MediaWiki action API to retrieve more information about the event.
 * For example:

In case there are limitations on the integration and using  is not possible, you might need to create a new push notification type other than   and orchestrate sending notifications manually.

Initial R&D: Q3 FY 2019-2020 (January - March 2020)

 * Initial product use cases and cross-team prioritization [Link to google doc, or move google doc table to wiki?]
 * Technical planning and investigation
 * Technology department consultation
 * TechCom RFC submitted

Phase 1: Native Echo
Initial use cases are mentioned in the TechCom RFC and revolve around forwarding Echo notifications as push notifications for the native apps.

Clients supported:


 * Wikipedia iOS App
 * Wikipedia Android app

Use cases supported:


 * Echo can push existing messages received by a user to the user via native apps for Android and iOS
 * App users can subscribe and unsubscribe to receive Echo notifications
 * App and web users can subscribe and unsubscribe to be pushed specific message types (adding a column to delivery preferences for "Push Notifications"
 * App users can manage their subscriptions via the native apps
 * The system can limit the rate and volume of messages to prevent server overload
 * The platform has a Push Notification service for managing these needs in a shared platform competent
 * Users who can no longer be reached after repeat attempts are removed from the service on a regular cycle
 * Product Infrastructure and SRE can monitor basic service usage via a standard dashboard

April

 * Incorporate RFC feedback
 * Finalize Phase 1 implementation plans
 * Start repository and begin exploratory implementation of the required services

May

 * Initial service development

June-July

 * Service deployed to Beta Cluster
 * Client engineers develop prototypes against initial implementations on staging

August

 * Technology reviews (DBA/Security/Performance) & follow-ups

Sept-Oct

 * Early-mid September: initial service launch date on k8s
 * Target public launch date: Sept 21
 * Post-launch bug fix iteration
 * Beginning app client adoption of production push notifications

Technology approvals required:

 * SRE machine allocation (for launching on k8s)
 * Performance review
 * Security review
 * DBA review

Phase 2: Open Echo
Phase 2 is focused on extending the push system to allow pushes to browsers via the Web Notifications API

Clients supported:


 * Wikipedia iOS App
 * Wikipedia Android App
 * Browsers implementing the Push and Notifications APIs

Use cases supported:


 * As a web user I can receive my Echo notifications as browser notifications
 * As a web user I can be prompted to enable notifications in context where that in meaningful (ie. not just because I land on our domain)
 * As a web feature team I can implement subscription prompts in context where that in meaningful (ie. not just because I land on our domain)
 * App and web users can subscribe and unsubscribe to be pushed specific message types (adding a column to delivery preferences for "Push Notifications")
 * As an Echo developer I can add a new notification type and have it also be delivered via push

Components
There is currently no estimated timeline for Phase 2 and beyond. Please check for future iterations on this page once those plans become firmer. However, we do know the basic components that require development for this phase, and a rough order of operations for their implementation.


 * Initial case defined by a web feature team (eg. Editing or Growth)
 * Define initial client browser target and extend push service to push to the selected platform(s)
 * Web service worker implementation
 * Implement/update sign-up and subscription UIs for web

Phase 3: Other Than Echo
The final envisioned stage of development is the creation of new types of notifications by client teams to serve specialized use cases. Example use cases include:


 * Re-engagement and encouragement of new or "lapsed" editors ("We haven't seen you in a while")
 * New feature and software update announcements
 * Campaigns and fundraising announcements ("Add a reference today to be part of #1Lib1Ref!")
 * Updates and recommendations for topics or portals of interest to a reader
 * Other News or Current events related content suggestions
 * Notifications for anonymous users

The order and time-frame for these will depend on the needs of the teams and their target users.

Architectural Overview
The push notification system consists of two components:


 * Push subscription management and notification request support in the Echo extension; and
 * A standalone service (push-notifications) for batching requests for submission to third party push providers.

In the initial (apps-only) implementation, to mitigate the user privacy and security risks associated with sending potentially sensitive information to users via third-party systems, pushed messages will contain only a short descriptor (e.g., ) to prompt the apps to retrieve message content from Wikimedia servers (e.g., by request to the Action API's notifications module). Further, the push-notifications service will collect push notification requests and send them in periodic batches in order to increase the difficulty of drawing inferences between on-wiki actions and messages traversing the public internet.

Echo extension
The Echo extension has been updated to handle storage of push subscriptions and to provide an API interface for clients to manage them. Push subscriptions are stored by central user ID in the echo_push_subscription table, and can be added or removed by clients using the Action API echopushsubscriptions module.

Additionally, the PushNotifier class has been created to handle sending MediaWiki notifications by push. When an Echo event occurs and the user has chosen to be notified about events of that type via push, an  job is scheduled to retrieve all stored push subscriptions associated with the user's central ID and send a request to the push-notification service that a message be pushed via the relevant service for each subscription.

Push-notifications service
A new Node.js service,, has been created to process push notification requests and submit them to platform-provided push APIs including Firebase Cloud Messaging (FCM) and Apple Push Notification Service (APNS). The push-notifications service is boostrapped from the Wikimedia Node.js service template and written in TypeScript. Interactions with FCM and APNS are supported by Google's official firebase-admin module and the popular community-developed node-apn module.

A primary function of the service is to collect and batch push notification requests before submitting them to platform push APIs. The number of messages to batch before submission and the time to wait between submissions can be defined in the service configuration. Additionally, the time to wait before submitting can be randomized within configurable lower and upper bounds.

Contact the team

 * IRC #product-infrastructure

Links

 * Design decisions
 * TechCom RFC
 * Dependencies Questionnaire
 * WikiTech: Push Notifications
 * MediaWiki development setup
 * Push Service (Node.js): in Gerrit