Wikimedia Apps/Team/Release process/Release criteria

From mediawiki.org

Beta releases[edit]

Beta releases[1] are performed as frequently (or infrequently) as required for testing purposes. The base criteria for a feature going in to beta are that:

  • The feature has some basic functionality that delivers some value to the user.
  • The feature does not crash during basic usage.
  • The feature behaves as expected during basic usage.

Production releases[edit]

Production submissions[2] occur on the last Thursday of every month. Features that are currently in the beta are evaluated to see if they should be included in the production release. The base criteria for a feature going in to production are that:

  • The feature has functionality which provides value to the user.
  • The feature is aimed to advancing us towards meeting our quarterly goals.
  • The feature has been QA tested and found to:
    • not crash even under stress.
    • behave as expected in typical use cases.
    • not introduce regressions.
  • The feature has undergone qualitative or quantitative user testing which validated its objectives.
  • The feature has clear criteria for success set which are measurable (quantitatively or qualitatively), such that post-hoc analysis can be performed about its effectiveness.

Footnotes[edit]

  1. On Android, "beta" refers to the Wikipedia Beta app in Google Play, whereas on iOS "beta" refers to our latest build in TestFlight.
  2. As we have a review process on the iOS side due to Apple, which will stretch releases out an unknown amount, this phase is specifically denoted as "submission" and not "release" so that there is congruence between the iOS and Android teams.