Beta Features

Purpose/Goals
The primary purpose of Beta Features is to allow for Wikimedia designers and engineers (from the Foundation and community alike) to roll out small-scale projects in an environment where a large numbers of user can test, give feedback, and use these features in real-world settings. The secondary purpose of Beta Features is to provide a path so that helpful, well-designed gadgets can be integrated into core after vetting, testing, and review from Wikimedia Design and Engineering.

Current Implementation (in progress)
The code is being worked on as an extension, Extension:BetaFeatures; you can use it on a test wiki here (note: you will need to create an account).

Access
Beta Features' preferences will be available in a Special:Preferences tab, accessible via the personal bar at the top of the page for logged-in users.

Appearance
The layout is purposefully different, to both inspire interest and propose a departure from the standard layout and complexity of the existing user preferences.



Functionality
Users will have the ability to opt-in to individual experiments, and also setting a preference to be automatically enrolled in all new experiments as they are released. When new experiments are made available to users, a notification will be sent out via a site-wide notification ("Echo"). This is to increase visibility and get additional feedback so the design team can react, update and improve experiments before making a decision to integrate into core, iterate the idea further, or abandon the experiment.

Notifications
Beta Experiments will trigger Echo notifications at different times.

Here's a preliminary list of notifications that seem desirable, subject to feasibility and team discussion:
 * New Experiment Available: Sent to all logged-in users who have signed up for Beta features and enabled this notification category, when a new experiment is available to test
 * New Experiment Enabled: Sent to logged-in users who have enabled automatic experiment enrollment  and enabled this notification category
 * Experiment Updated: While experiments could change on a daily or hourly basis based on testing, user feedback, or optimization. When major changes to an experiment are significant, a notification with a change list may be sent to enrolled users who have signed up for Beta features and enabled this notification category.
 * Experiment Graduation (Future): When experiments have achieved a high level of stability, acceptance and polish, they may be chosen to be integrated by default as an extension on a project by project basis. 
 * Back to the Drawing Board (Future): Experiments that don't pass muster may be discontinued, and removed from the Beta Experiments program to be re-evaluated. 

Current Experiments

 * VisualEditor (Coming Soon!)
 * Media Viewer (Coming Soon!)
 * Consistent Typography (Coming Soon!)
 * Better interlanguage Links (Coming Soon!)
 * Compact Personal Bar (Coming Soon!)

Research
For this product, we would like to track basic usage patterns, so that research can inform our next steps.

Here is a preliminary list of research questions we hope to answer, for discussion purposes.
 * How many times did users click on the 'Beta' link in the personal bar?
 * How many times did users view the 'Beta features' preference page? (impressions)

(repeat for each feature x, z, z)
 * How many times did users enable (or disable) features after clicking on the 'Beta' link?
 * How many times did users enable all beta features?
 * How many times did users disable all beta features?
 * How many unique users enabled all features after clicking on the 'Beta' link?
 * How many unique users disabled all features after clicking on the 'Beta' link?
 * How many times did users enable beta feature x?
 * How many times did users disable beta feature x?
 * How many unique users enabled feature x?
 * How many unique users disabled featurex?
 * How many times did users click on the project page icon for feature x?
 * How many times did users click on the discussion page icon for feature x?
 * What is the clickthrough rate for enabling feature x based on total impressions for the Beta preferences page?

These metrics would be collected with EventLogging and related technologies, and visualized with LIMN dashboards (see example). We would track this data on a daily basis, as well as on a cumulative basis.