Beta Features

Purpose/Goals
The primary purpose of Beta Features is to allow for Wikimedia designers and engineers (from the Foundation and community alike) to roll out technical improvements in an environment where a large numbers of users can test, give feedback, and use these features in real-world settings. The secondary purpose of Beta Features is to provide a path so that helpful, well-designed gadgets can be integrated into core after vetting, testing, and review from Wikimedia Design and Engineering.

Current Implementation (in progress)
The code is being worked on as an extension, Extension:BetaFeatures; you can use it on a test wiki here (note: you will need to create an account).

Access
Beta Features' preferences will be available in a Special:Preferences tab, accessible via the personal bar at the top of the page for logged-in users.

Appearance
The layout is purposefully different, to both inspire interest and propose a departure from the standard layout and complexity of the existing user preferences.



Functionality
For our first minimum viable product, users will be able to:
 * opt-in to individual features (manually)
 * enroll in all new features as they are released (automatically)

To use either function, users will select the appropriate checkbox, then click 'Save', which will automatically save their BetaFeatures preferences.

In future releases, as new features are made available to users, other functions will be added, as outlined below. For example, notifications can be sent out via "Echo" when new features are added, as outlined in the Future releases section below.

Features
At this time, we are considering these features for our first release:


 * VisualEditor - New Features
 * Typography Update (Coming Soon)
 * Media Viewer (Coming Soon)

We invite product teams that want to test features through this project to write up detailed descriptions on this New Features page.

Release Plan
Our goal is to release a first barebones version of Beta Features in October, with at least a couple features to start with, but no notifications.

The extension would first be deployed on mw.org and test sites on a Thursday, then the following Thursday on enwiki and a couple wikis, if all goes well (enwiki, frwiki, meta). We recommend testing the extension on these first production sites for a week or so, to get community feedback and fix any bugs. Then we would deploy more widely, in collaboration with our community liaisons.

We will propose specific deployment dates once we have resolved questions related to the Visual Editor, our first big feature for this project. A first deployment date on October 10th is under consideration, subject to team review.

Contents
Here are the proposed headers and descriptions for the first features at launch (this is placeholder copy, to be edited collaboratively):

''Here are some new features we're considering for <$wgSitename>. Please try them out and give us your thoughts, so we can improve them based on your feedback.''

☐ Enable new beta features as they are released


 * VisualEditor - New features | Info | Discussion
 * Get the latest features inside VisualEditor. As we develop tools and make changes for the editor, we make them available for testing ahead of general release. Please remember to always review your changes before saving.


 * Consistent Typography (Coming Soon!) | No info page yet | No discussion page yet
 * Try out this new look that makes text more readable, to benefit readers and editors alike. This beta feature uses serif headings for sections and sans serif for body copy to clearly differentiate them from body paragraphs, for a better visual experience that works consistently across desktop and mobile platforms.


 * Media Viewer (Coming Soon!) | Info | Discussion
 * Improve your multimedia viewing experience with this new tool, which displays images in larger size on pages that have thumbnails. Images are shown in a nicer lightbox overlay, and can also be viewed in full-size. This media viewer aims to reduce confusion by only showing key information, with a link to the file info page for more details.

In coming days, we will edit this placeholder copy and add graphics, as well as links to overview and discussion pages for each feature. To see a preliminary version of this tool, create an account on this prototype site], then log in and click on 'Preferences > Beta features'.

Research
Here is a list of research questions we hope to answer over time for this product, as outlined in this Trello ticket.


 * How many users are opting into beta features?
 * What's the breakdown of adoption by user tenure?
 * What are the most popular features by number of enrolled users?
 * How many users are selecting the auto-enroll option for new features?
 * What proportion of users is disabling beta features after trying them out?

No ad-hoc instrumentation seems needed to measure adoption, as all data required to obtain descriptive usage stats are already available from these sources:
 * the PrefUpdate log
 * the user_properties table (core)
 * the betafeatures_user_counts table (extension)

For this product, we would like to build a set of LIMN dashboards to track basic usage patterns, so that research can inform our next steps. This development may need to be phased over time, based on available resources. Key questions are listed below in order of priority.

Auto-enroll New Features

 * How many users have enabled all new beta features?
 * How many users have disabled new beta features after trying it out?

Individual Features
(repeat for each feature x, z, z)
 * How many unique users enabled feature x?
 * How many unique users disabled feature x after trying it out?

Info & Discussion Pages

 * How many times did users click on the info page icon for feature x?
 * How many times did users click on the discussion page icon for feature x?

Overall Stats

 * How many times did users click on the  'Beta' link in the personal bar?
 * How many times did users view the 'Beta features' preference page? (impressions)
 * What is the clickthrough rate for enabling feature x based on total impressions for the Beta preferences page?

These metrics would be collected with EventLogging and related technologies, and visualized with LIMN dashboards (see example). We would track this data on a daily basis, as well as on a cumulative basis.

Notifications
Beta Features will trigger Echo notifications at different times, as soon as practical. This is to increase visibility and get additional feedback, so the product teamsan react, update and improve features before making a decision to integrate into core, iterate the idea further, or abandon the experiment.

Here's a preliminary list of notifications that seem desirable, subject to feasibility and team discussion:
 * New Feature Available: Sent to all logged-in users who have signed up for Beta features and enabled this notification category, when a new feature is available to test
 * New Feature Enabled: Sent to logged-in users who have enabled automatic feature enrollment  and enabled this notification category
 * Feature Updated: When major changes are made to a feature, a notification may be sent to enrolled users who have signed up for Beta features and enabled this notification category.
 * Feature Graduation (Future): When features have achieved a high level of stability, acceptance and polish, they may be chosen to be integrated by default as an extension on a project-by-project basis. 
 * Back to the Drawing Board (Future): Experiments that don't pass muster may be discontinued, and removed from the Beta Experiments program to be re-evaluated. 

Local Configurations
From time to time, it may be necessary to exclude a beta feature from a particular project, either because it may not work on that project, or at the request of its community. Typically, this use case can be supported by not deploying the particular extension for that beta feature on that project. However, there may be cases where the beta feature is not an extension, but part of core, which would cause it to appear automatically. For such cases, it may be necessary to create a local configuration file or hook that could check for dependencies or community requests for features.