Beta Features

Purpose/Goals
The primary purpose of Beta Features is to allow for Wikimedia designers and engineers (from the Foundation and community alike) to roll out technical improvements in an environment where a large numbers of users can test, give feedback, and use these features in real-world settings. The secondary purpose of Beta Features is to provide a path so that helpful, well-designed gadgets can be integrated into core after vetting, testing, and review from Wikimedia Design and Engineering.

Current Implementation (in progress)
The code is being worked on as an extension, Extension:BetaFeatures; you can use it on a test wiki here (note: you will need to create an account).

Access
Beta Features' preferences will be available in a Special:Preferences tab, accessible via the personal bar at the top of the page for logged-in users.

Appearance
The layout is purposefully different, to both inspire interest and propose a departure from the standard layout and complexity of the existing user preferences.



Functionality
For our first minimum viable product, users will be able to:
 * opt-in to individual features (manually)
 * enroll in all new features as they are released (automatically)

To use either function, users will select the appropriate checkbox, then click 'Save', which will automatically save their BetaFeatures preferences.

In future releases, as new features are made available to users, other functions will be added, as outlined below. For example, notifications can be sent out via "Echo" when new features are added, as outlined in the Future releases section below.

Tasks
Here are some of the development tasks we are working on for the first release of BetaFeatures.

Browser and Technical Compatibility
As a user, I want to know if a beta feature doesn't work for my browser or technical configuration. (See Mingle ticket #35)

Browser and technology compatibility can vary from one feature to another (VisualEditor in particular isn't available for IE yet). Other features may require Javascript, CSS or other technical configurations as well, that could cause them to malfunction if not enabled.

So each beta feature extension needs to have a list of platforms that it cannot support, stored in Javascript, JSON or other machine-readable format that can be sent to Beta Features.

The Beta Features (or the extension) then needs to check whether the user's configuration matches one of these restricted platforms, so it can disable that feature.

Last but not least, we also need to inform the user about features that are not compatible with their particular browser or configuration. For example, we may want to display a message like:

"This feature is not compatible with this browser: ."

or

"This feature requires ."

This is required so the user understands why they cannot enable that feature (even though it's still checked in their preferences -- so they can use it on other platforms).

That message could be bolded and/or colored in red, to provide a visual cue that this feature will not work. Maybe the entire feature could be grayed out, to make it even more explicit?

Here are the types of notes we may need to display, when a feature is not compatible for the user's configuration:
 * Notes about browser compatibility
 * Text displayed to user "This feature is not compatible with your browser yet"
 * Notes about JS compatibility
 * Text displayed to user "You must enable JavaScript in your browser to use this feature"
 * Notes about CSS compatibility
 * Text displayed to user "Your site skin or custom CSS is not compatible with this feature, to enable it use the default site skin"
 * Notes about other technologies (webm? ogg? svg?)
 * Text displayed to user "This feature is not compatible with your browser yet"

These notes need to be translated into human-readable requirements like "Firefox 22" or "Chrome 232901832" or whatever.

Only one error message should be displayed at any given time, the order of display should be as follows e.g. if there is both a browser and JS compatibility issues the user would only see the JS issue.
 * 1) CSS/Skin compatibility
 * 2) JS compatibility
 * 3) Browser compatibility

VisualEditor Opt-In
As a user, I want the option to enable Visual Editor, if it is not available by default on this site. (See Mingle ticket #43)

Some sites require user opt-in for the VisualEditor (the English and German Wikipedias in particular aren't ready yet to make this feature available by default). So we would like to make the basic version of VisualEditor available as a prominent BetaFeature -- but for those sites only. For example, we could handle this issue by having a site whitelist/blacklist for each beta feature -- or there may also be other technical solutions that work just as effectively to achieve the same results.

Note that on sites where the basic version of VisualEditor is available by default, some users will want the option to opt-out and may need some guidance on how to do this. Because BetaFeatures will also have an option to enable 'Visual Editor - Extra Features', some users may get confused. So we may want to include a sentence at the end of the 'Extra' description to let users know about that basic version, with a link to a page that tells them more, including how to disable it.

Finalize VisualEditor Extra Features
Determine which 'safe' features to include in VE Extras and update extension accordingly. Add feature list on New Features Page (with thumbnail & description).

Popup Guide
As a user, I want to know what the 'Beta' label in my personal menu means. (See Mingle ticket #49)

Show a pop-up guide that explains what Beta Features is the first time you load a page that contains the 'Beta' label in your personal menu.

It would show a short sentence explaining what this 'Beta' label does and point to it (between 'Preferences' and 'Watchlist'). It would say something like:

"Check out new  and let us know what you think. [ OK ]"


 *  would link to the Beta Features page
 * [ OK ] would simply close the popup guide

More tasks may be added here, as needed -- as well as on our Multimedia team's Mingle project management site.

Features
At this time, we are considering these features for our first release:


 * VisualEditor - Basic Opt-in Version (1)
 * VisualEditor - Extra Features
 * Typography Update
 * Media Viewer (if ready)

(1) This option would only be for sites that require opt-in. It would not be available on sites that provide the VE tool on an opt-out basis.

We invite product teams that want to test features through this project to write up detailed descriptions on this New Features page.

Schedule
Our goal is to release a first barebones version of Beta Features in October, with at least a couple features to start with, but no notifications.

Here are our target dates for the first release:
 * Oct. 3 - Install VE on Prototype (for testing/demo purposes)
 * Oct. 10 - Complete outstanding tasks (e.g. Browser Compatibility, VE Opt-in)
 * Oct. 17 - First features ready (VE Opt-in, VE Extras, Typography)
 * Oct. 24 - Release to MediaWiki.org and test.wiki (+ barebones Media Viewer?)
 * Oct 25 - Pre-announce Beta Features on all wikis (with community team)
 * Oct. 28 - Release to sister projects (non-Wikipedia)
 * Oct. 31 - Release to all Wikipedias

The current goal is to try to ride the platform train, if possible -- so most dates above are Thursdays. However, special deploy windows may be needed along the way.

The extension would first be deployed on mw.org and test sites on a Thursday, then the following Thursday on enwiki and a couple wikis, if all goes well (enwiki, frwiki, meta). We recommend testing the extension on these first production sites for a week or so, to get community feedback and fix any bugs. Then we would deploy more widely, in collaboration with our community liaisons.

We may adjust this schedule as needed, based on our progress with deliverables, and subject to team review.

Contents
Here are the proposed headers and descriptions for the first features at launch (this is placeholder copy, to be edited collaboratively):

Top header: Here are some new features we're considering for <$wgSitename>. Please try them out and give us your thoughts, so we can improve them based on your feedback.


 * ☐ Enable new beta features as they are released


 * VisualEditor
 * Try out VisualEditor for a better editing experience. This is the basic version of this new tool. (1)

Links: Info | Discussion

''(1) This VisualEditor - Basic Opt-in Version would only be for sites that require opt-in for VE. It would not be available on sites that provide the VE tool on an opt-out basis. ''


 * VisualEditor - Extra Features
 * Get the latest features inside VisualEditor. As we develop tools and make changes for the editor, we make them available for testing ahead of general release. Please remember to always review your changes before saving.

Links: Info | Discussion


 * Typography Update
 * Try out this new look that makes text more readable, to benefit readers and editors alike. This beta feature uses serif headings for sections and sans serif for body copy to clearly differentiate them from body paragraphs, for a better visual experience that works consistently across desktop and mobile platforms.

Links: No info page yet | No discussion page yet


 * Media Viewer
 * Improve your multimedia viewing experience with this new tool. It displays images in larger size on pages that have thumbnails. Images are shown in a nicer lightbox overlay, and can also be viewed in full-size.

Links: Info | Discussion

''In coming days, we will edit this placeholder copy and add graphics, as well as links to overview and discussion pages for each feature. To see a preliminary version of this tool, create an account on this prototype site], then log in and click on 'Preferences > Beta features'.''

Creating Your Own Beta Features
Beta Features help Wikimedia designers and engineers to test out new features or significant changes to existing features. To add your project to Beta Features if must meet the following basic requirements Once you've met these requirements, follow the next steps for preparing your feature and required asset.
 * Not significantly degrade site performance;
 * Not noticeably degrade perceived performance of the the site, or the user's system;
 * Not crash the user's browser;
 * Not cause data loss, or corruption; and
 * Contribute positively to the user's experience of the site, and be additive in nature. e.g. Beta Features cannot be used to remove site features or functionality.

Research
Here is a list of research questions we hope to answer over time for this product, as outlined in this Trello ticket.


 * How many users are opting into beta features?
 * What's the breakdown of adoption by user tenure?
 * What are the most popular features by number of enrolled users?
 * How many users are selecting the auto-enroll option for new features?
 * What proportion of users is disabling beta features after trying them out?

No ad-hoc instrumentation seems needed to measure adoption, as all data required to obtain descriptive usage stats are already available from these sources:
 * the PrefUpdate log
 * the user_properties table (core)
 * the betafeatures_user_counts table (extension)

For this product, we would like to build a set of LIMN dashboards to track basic usage patterns, so that research can inform our next steps. This development may need to be phased over time, based on available resources. Key questions are listed below in order of priority.

Auto-enroll New Features

 * How many users have enabled all new beta features?
 * How many users have disabled new beta features after trying it out?

Individual Features
(repeat for each feature x, z, z)
 * How many unique users enabled feature x?
 * How many unique users disabled feature x after trying it out?

Info & Discussion Pages

 * How many times did users click on the info page icon for feature x?
 * How many times did users click on the discussion page icon for feature x?

Overall Stats

 * How many times did users click on the  'Beta' link in the personal bar?
 * How many times did users view the 'Beta features' preference page? (impressions)
 * What is the clickthrough rate for enabling feature x based on total impressions for the Beta preferences page?

These metrics would be collected with EventLogging and related technologies, and visualized with LIMN dashboards (see example). We would track this data on a daily basis, as well as on a cumulative basis.

Microsurveys for Beta opt-out
A microsurvey would be created to capture reasons why people decide to opt out of specific beta features, using a similar way to how we collect gender data for account registrations. See Trello ticket for more info about this proposal.

Notifications
Beta Features will trigger Echo notifications at different times, as soon as practical. This is to increase visibility and get additional feedback, so the product teamsan react, update and improve features before making a decision to integrate into core, iterate the idea further, or abandon the experiment.

Here's a preliminary list of notifications that seem desirable, subject to feasibility and team discussion:
 * New Feature Available: Sent to all logged-in users who have signed up for Beta features and enabled this notification category, when a new feature is available to test
 * New Feature Enabled: Sent to logged-in users who have enabled automatic feature enrollment  and enabled this notification category
 * Feature Updated: When major changes are made to a feature, a notification may be sent to enrolled users who have signed up for Beta features and enabled this notification category.
 * Feature Graduation (Future): When features have achieved a high level of stability, acceptance and polish, they may be chosen to be integrated by default as an extension on a project-by-project basis. 
 * Back to the Drawing Board (Future): Experiments that don't pass muster may be discontinued, and removed from the Beta Experiments program to be re-evaluated. 

Local Configurations
From time to time, it may be necessary to exclude a beta feature from a particular project, either because it may not work on that project, or at the request of its community. Typically, this use case can be supported by not deploying the particular extension for that beta feature on that project. However, there may be cases where the beta feature is not an extension, but part of core, which would cause it to appear automatically. For such cases, it may be necessary to create a local configuration file or hook that could check for dependencies or community requests for features.

Sorting Options
As more beta features are added, we may need to provide affordances for sorting the list by importance, by name, or by date. For example, we will want the option to feature an important feature like Visual Editor at the top of the page (because of its significance for the movement's strategic goals), but also give users the option to change that default sorting to list features instead by date (e.g. most recent first) or by name (e.g. alphabetically). It may also become necessary to make the page design more compact, if we are beta testing dozens of different features at the same time.