Wikimedia Apps/Team/Android/Release process

This page documents the release process for the Mobile Apps Team.

Alpha releases
Alpha apks are built automatically after patches get merged to master on the half hour mark and are available at android-builds.wmflabs.org.

Beta releases
Beta releases get pushed to the store more often and earlier than production releases. Therefore, we bump the version code usually before we build a beta release (but not usually for production).
 * 1) Bump version code
 * 2) +2 in Gerrit
 * 3) Build apk:
 * 4) Test the apk. Install it via  . The   option is useful when testing the upgrade path since it doesn't clear the data.
 * 5) Copy the apk and the mapping.tar.gz files to releases.wikimedia.org:
 * 6) Confirm the copy step worked:
 * 7) Publish the apk on Google Play Publisher.
 * 8) When publishing you want to update the What's New section. You can use this command to compile a list of changes in Git since the last beta release:
 * 9) Push the release tag to gerrit remote:
 * 10) Email mobile-l about the release.
 * 11) Wait ~2 hours and then see the live update in Google Play Store.
 * 1) Email mobile-l about the release.
 * 2) Wait ~2 hours and then see the live update in Google Play Store.

Production releases

 * 1) Make sure you have checked out to the correct commit in Git. Usually that is where the latest beta release is, but there can be exceptions.
 * 2) Build apk:   This will produce two apks:   for uploading to Google Play, and   for uploading to releases.wikimedia.org.
 * 3) Test the apks. Install them via.
 * 4) Copy the apk and the mapping.tar.gz files of releasesprod to releases.wikimedia.org:
 * 5) Confirm the copy step worked:
 * 6) Publish the "r" apk on Google Play Publisher.
 * 7) When publishing you want to update the What's New section. You can use this command to compile a list of changes in Git since the last production release (Note that it's slightly different than for beta):
 * 8) Push the release tag to gerrit remote:
 * 9) Add an entry to the app release history.
 * 10) PM emails mobile-l.
 * 11) Wait ~2 hours and then see the live update in Google Play Store.

Custom releases
Custom releases are done by the partners team. For custom releases there are two options: The process is pretty much the same for both options, except when building the apk.
 * Custom channel
 * Custom channel + custom application ID
 * This acts like a different app, so it can be installed side by side with our production Wikipedia app.
 * 1) Make sure you have checked out to the correct commit in Git. Usually that is where the latest production release is, but there can be exceptions.
 * 2) Build apk:
 * 3) * Custom channel:
 * 4) * Custom channel + custom application ID:
 * 5) ** The application ID will be
 * 6) Test the apk. Install it via.
 * 7) If needed, copy the file to a subfolder in releases.wikimedia.org:
 * 8) Push the release tag to gerrit remote:

iOS
See https://etherpad.wikimedia.org/p/iosappcadencenojoke for notes from the 1-April-2015 meeting and https://etherpad.wikimedia.org/p/qaoniosrocks for notes from the 2-April-2015 meeting.

Cards
Effective mid-sprint 54 on iOS, Phabricator cards on iOS will go through Code Review, QA Signoff, and Design Sign Off before they go to the Ready for Review (Product Manager signoff). We'll tweak the process as necessary. Important points about this:


 * The Description field should contain a link to the design card. The design card should contain mocks, expected user interaction behavior, and any finer points that may escape the attention of engineers (e.g., gradients, font faces, transition durations, etc.). It should address phone and tablet form factor, as well as portrait and landscape mode.
 * The spec should be noted to mobile-tech one week before a sprint starts to allow feedback
 * The Description field should contain sufficient information for QA to know what to test.
 * QA must test on iOS 6, iOS 7, and iOS 8 on a phone form factor, and must test on at least one version of iOS for the tablet form factor. Recommendation: test daily.
 * Alpha builds will be available each weekday morning, so QA should be able to review cards in the QA column at that time. If a crash occurs, QA should start the app back up and submit the crash via HockeyApp.
 * Design to sync with QA twice per week on any cards in its column for sign off. One approach: test alpha Wednesday and Friday of week 1, Tuesday and Thursday of Week 2 for all cards in Design column. If a crash occurs, Design should start the app back up and submit the crash via HockeyApp.
 * The developer working the card should shepherd it through the columns (e.g., prompt QA or Design if necessary). Engineers, QA, Design, Product Manager, and Scrummaster should be online on IRC during normal business hours in case they need to be nudged about a card.
 * Work product may be rejected and moved back to To Do, with a Comment explaining why, if the implementation did not match the specification. Otherwise, new cards (Tasks) should be filed if follow-on work is required, and the work will be prioritized for later implementation (potentially in the current sprint, but potentially in a later sprint); [[Wikimedia_Apps/Team/Bug_reporting bug reporting] guidelines should be used for such follow-on work.
 * If sign off is not required from a particular stakeholder (e.g., Design not needed for data layer testing), the Description card should indicate as much, and the card should be dragged to the appropriate column.
 * The engineer should indicate a risk rating and low level technical testing steps as appropriate to the technical depth of the card.
 * Features that we know can't be used by the end of a spring (e.g., a multi-sprint epic) should be feature flagged early on.
 * Regression tests pertaining to cards in the current sprint identified during work should be added to the Template tab of the iOS Test Cases

Release cards
A Release column (or separate board) should be added for each sprint with the following, and comments added to Phabricator plus the The Releases page updated as updates arrive. Cards can be moved to Done when done.


 * Regression testing
 * App Store submission
 * Comms
 * Released
 * Aborted release

Week 1 of iteration

 * Monday morning: Tech Lead engages Specialists Guild, informing them that Wikipedia Mobile will be released via TestFlight the following day and that their testing results are requested to be returned by Wednesday no later than noon.


 * Monday afternoon: 3 PM San Francisco bugfix code cutoff for prior sprint's work. Ideally, there won't actually be any new code on Monday pertaining to the prior sprint.


 * Tuesday morning, 9:30 AM San Francisco time: whatever's merged into the release branch goes into both Wikipedia Beta for TestFlight *external* testing and Wikipedia Mobile (stable) for TestFlight internal upgrade smoketesting as the release candidate (instructions). The corollary of this is the engineers need to be judicious about not polluting the code; there are many ways to deal with this.


 * Product manager, lead designer, lead QA, and lead iOS engineer do final verification for crashes and serious regressions, flagging blockers. Just in case, because it's possible that cards tested in isolation introduce problems in concert. Team swarms on issues if necessary.


 * Thursday morning: Product Manager gives final go/no-go.


 * If by Thursday afternoon at 3 PM San Francisco time there hasn't been serious badness identified (iOS engineering to report all crashes and user reported feedback), we then submit this release candidate to Apple.

Week 2 of iteration
If we didn't submit for formal App Store review in week 1, then in week 2, we follow the same process as week 1, mandatorily submitting a release for formal App Store review on Thursday week 2 of the iteration.

If we're for some reason still blocked by Thursday week 2 of the iteration, we notify Directors of Engineering, Product, Design, and Platform, plus the Manager of Release Engineering.

If we submitted for App Store review in Week 1 without issue, then 9:30 AM San Francisco time on Tuesday of week 2 the TestFlight for Wikipedia Beta and Wikipedia Mobile (stable) is released for whatever's at the tip of master.

The app should be submitted during early business hours Monday through Thursday in San Francisco, but not Friday, Saturday, or Sunday. Note that if Comms is needed for a big announcement for a given release, they need a heads up well ahead of time, with timely updates as the ship date approaches (the final 5 business days leading up to big announcements are pretty critical with history as a guide). Big splashes don't happen every sprint, and for most sprints, social media only announcements with less time-pegged blog posts (if any blog post) is fine.

Version Bumps
Note that TestFlight version bumps on the x.y (major.minor) part of the x.y.z version require formal Apple review before they can become available to external testers. Therefore, if a major.minor bump is needed, the TestFlight toggle needs to be flipped in iTunes Connect to get that in the queue with sufficient lead time so that Tuesday cutpoints aren't missed. Know what you're doing before you toggle switches. You don't want to end up effectively disabling in-flight testing on release candidates.

Feature cutoff
Feature cutoff for week 2 of a sprint Thursday afternoon 3 PM San Francisco allows for Friday to be a day for experiments and any "last minute" bugfixes.

Unit tests
Broken unit tests on the build server constitute a broken build and must be fixed to make work product shippable.

Open questions

 * Branching strategy to handle rejected cards. Realistically, if rejected cards are treated within the same sprint they're worked, no special branching is required. Daily (QA) and twice weekly (Design) feedback should help to avoid serious problems occurring too late in the sprint.
 * Branching strategy for the week 1 or week 2 swarms when they're required. Presently, the thought is to just branch from the corresponding tag and do all work on that branch, merging such fixes upstream to master as well.
 * The format of what's required in the Description field of cards in Phabricator and what constitutes a sufficient Design card (Trello) isn't fully defined, although most of it is specified above. This said, a checklist could potentially help reduce ambiguity.
 * Is a "Spec" column needed on the main iOS board? It would be before the Next Sprint column to reflect when a design specification is still required. Likewise, is an Estimation and a Released column needed in the main iOS board? This way we have a clear liner progression on the main board from Needs Triage to Released.
 * Is full regression testing (TSG) required in case submission for app store review was pushed to week 2 of the sprint? Probably not, as the entire team should be swarming on the issues that caused the delay.