Article feedback/Public Policy Pilot/Design Phase 2

This page describes the design for phase 2 of the Article Feedback Tool pilot. This document is a companion to the phase 1 design document and follows from there, changes for phase 2 being described here. This document assumes that the reader is familiar with the phase 1 design.

Scope
For phase 2 of the Article Feedback pilot, we want to address the following issues:


 * 1) "Expired" ratings. (if not defining the formula used, at least developing a system whereby that formula can be applied).  This is both a feature modification and a behavior change.
 * 2) Clearing ratings. The ability for users to "clear" a rating of any values they had previously applied.  This is a feature modification.
 * 3) Instant Submit. Modify the feature's widget to allow for instant submission and reveal.  This is part of an A-B test versus the current behavior (that of requiring a "Submit" button).  This is a behavior change.
 * 4) Modified Survey. Different questions, perhaps targeted clearer.
 * 5) Post-rating "calls to action". We want to see if the simple act of rating an article can easily serve as a gateway towards editing or creating an account. This is both a feature modification, a behavior change, and a data tracking experiment.
 * 6) Post-rating Survey Push.  We want to be more aggressive about asking people to take our survey. This is a behavior change.

Stretch Goals

 * 1) Performance increase.  It would be good to know that the tool will not collapse after a certain threshold.

Out of Scope
The following features were deemed out of scope for phase 2, either because of resource availability constraints or design readiness:


 * 1) Rating histograms. At this point in time there is insufficient data to determine which, if any, histograms will be of best value. Accordingly, spending what limited resources we have available on an ill-defined feature component is unadvisable.
 * 2) Component placement.  Without better data as to the opinion of the community regarding the tool's importance or usefulness, making changes to its position and presentation at this time is premature.

A/B/C/D Test Support
One primary intent for phase 2 is the inclusion of a series of A/B tests based on the display and the display's position. To support this, the following changes must happen:


 * 1) One or more columns (as needed) must be added to the database tables.  This column will be used to store which "style" of the tool the user was given and rated. The style will be sent to the server through the API.
 * 2) * This could probably be an integer. There are five total defined "styles" for the tool:
 * 3) *# 0, the "phase one" style (all old entries will be given this value)
 * 4) *# 1, phase 2 style, page bottom location, with submit button.
 * 5) *# 2, phase 2 style, page bottom location, without submit button.
 * 6) *# 3, phase 2 style, sidebar location, with submit button.
 * 7) *# 4, phase 2 style, sidebar location, without submit button.
 * 8) A mechanism to store on the client a cookie which indicates which style they have been bucketed into.
 * 9) A mechanism to generate the style selected for the user.

While it would be ideal to store style values for logged-in users on the server (thus ensuring that they have the same experience during the testing phase, regardless of their client), development timeframe will prevent this.

Determining Applied Style
When a user visits a page that should display the tool, the following happens:


 * 1) The server checks for the value of the article feedback "style" cookie.  If the cookie is present and the value of it is valid, the system produces that style of display.
 * 2) If the value is missing or invalid (a value of 0 or higher than 4), a new style value will be generated and placed on the client.

The value of the style cookie should be able to be overridden via an url parameter to enable testing.

TBD: Storage of style counts for statistical analysis

Fresh/Stale/Expired Calculation
The current "is this rating stale" system must be adjusted to add an "is this rating expired" state as well. The calculations for both should be modified thusly:

A user's ratings are considered "stale" when:
 * The article has achieved 10 revisions or
 * The article has been modified +/- 20% of its size or
 * The article has achieved 5 revisions and +/- 15% of its size

A user's ratings are considered "expired" when:
 * The article has achieved 30 revisions or
 * The article has been modified +/- 35% of its size or
 * The article has achieved 15 revisions and +/- 20% of its size

Ideally, both state calculations could be defined within the server's LocalSettings configuration file to allow for changes to them without redeployment of the extension. Even having the numbers able to be defined there will be useful.