Wikimedia Apps/Team/Android/Release process

This page documents the release process for the Mobile Apps Team.

Alpha releases
Alpha apks are built automatically after patches get merged to master on the half hour mark and are available at android-builds.wmflabs.org.

Beta+ preflight checklist

 * Don't merge version code bumps until testing is done.
 * Test "don't keep activities".
 * Test Gingerbread, ICS, Lollipop, and Amazon devices.
 * Test a second language.
 * Test an RTL and LTR language.
 * Test new features and churn areas identified when generating release notes.

Beta releases
Beta releases get pushed to the store more often and earlier than production releases. Therefore, we bump the version code usually before we build a beta release (but not usually for production).
 * 1) Bump version code
 * 2) +2 in Gerrit
 * 3) Build apk:   (Consider performing a destructive   before building.)
 * 4) Test the apk. Install it via  . The   option is useful when testing the upgrade path since it doesn't clear the data. It is highly recommended to download the previous release from the Play Store first.
 * 5) Copy the apk and the mapping.tar.gz files to releases.wikimedia.org:
 * 6) Confirm the copy step worked:
 * 7) Publish the apk on Google Play Publisher.
 * 8) When publishing you want to update the What's New section. You can use this command to compile a list of changes in Git since the last beta release:
 * 9) Push the release tag to gerrit remote:
 * 10) Email mobile-l about the release.
 * 11) Wait ~2 hours and then see the live update in Google Play Store.
 * 1) Email mobile-l about the release.
 * 2) Wait ~2 hours and then see the live update in Google Play Store.

Production releases
About a week later:
 * 1) Make sure you have checked out to the correct commit in Git. Usually that is where the latest beta release is.
 * 2) Build apk:   This will produce two apks:   for uploading to Google Play, and   for uploading to releases.wikimedia.org.
 * 3) Test the apks. Install them via  . It is highly recommended to download the previous release from the Play Store first.
 * 4) Copy the apk and the mapping.tar.gz files of releasesprod to releases.wikimedia.org:
 * 5) Confirm the copy step worked:
 * 6) Publish the   apk on Google Play Publisher under the ALPHA TESTING tab.
 * 7) When publishing you want to update the What's New section. You can use this command to compile a list of changes in Git since the last production release (Note that it's slightly different than for beta):
 * 8) Push the release tag to gerrit remote:
 * 9) Copy the   APK and the mapping.tar.gz to the testing folder of the GDrive.
 * 10) Tech lead sends the   APK to the test group.
 * 11) Add an entry to the app release history.
 * 12) Continue with other builds below (Amazon releases + custom channel releases)
 * 1) In Google Play Publisher promote APK from ALPHA TESTING to Production.
 * 2) PM emails mobile-l.
 * 3) Wait ~2 hours and then see the live update in Google Play Store.

Amazon releases
This one is for Amazon's app store. The functionality and release notes are the same as the corresponding Production build. The only difference is the channel name. Ideally you just continue right after building the production APKs. [In the future we could build all other needed APKs with the same command line, with just some smaller modifications to the build script.]
 * 1) Make sure you have checked out to the correct commit in Git. Usually that is where the latest production release was made with.
 * 2) Build apk:
 * 3) Test the apk. Install them via  . It is recommended to download the previous release from the Amazon Store first if possible.
 * 4) Copy the APK and the mapping.tar.gz files to the testing folder of the GDrive.
 * 5) Upload APK to Amazon Developer Console.

Custom channel releases
Custom releases are distributed by the partner team.
 * 1) Make sure you have checked out to the correct commit in Git. Usually that is where the latest production release was made with.
 * 2) Build apk:
 * 3) Test the apk.
 * 4) Copy the APK and the mapping.tar.gz files to the testing folder of the GDrive.

iOS
See the following:


 * https://etherpad.wikimedia.org/p/iosappcadencenojoke (1-April-2015)
 * https://etherpad.wikimedia.org/p/qaoniosrocks (2-April-2015)
 * https://etherpad.wikimedia.org/p/iosreleasing (8-April-2015)

Cards
Effective mid-sprint 54 on iOS, Phabricator cards on iOS will go through Code Review, QA Signoff, and Design Sign Off before they go to the Ready for Review (Product Manager signoff). We'll tweak the process as necessary. Important points about this:


 * The Description field should contain a link to the design card. The design card should contain mocks, expected user interaction behavior, and any finer points that may escape the attention of engineers (e.g., gradients, font faces, transition durations, etc.). It should address phone and tablet form factor, as well as portrait and landscape mode.
 * The spec should be noted to reading-wmf one week before a sprint starts to allow feedback
 * The Product Manager and Designer will ensure the Description field contains sufficient Acceptance Criteria for QA to know what to test (if not already captured on the Design card), and software engineers should update if there are specifics relating to the patch associated with a card.
 * The primary code author must test on iOS 6 and iOS 8 (on 1 form factor at least, both phone and tablet if UI related). Code reviewers must test on at least one device. The QA analyst must test on iOS 6, iOS 7 (simulator, less emphasis), and iOS 8 on a phone form factor, and must test on at least one version of iOS for the tablet form factor (ideally, both iOS 8 and iOS 7, though, for tablet form factor instead of just one iOS version). Recommendation: test daily.
 * Alpha builds will be available each weekday morning, so QA should be able to review cards in the QA column at that time. If a crash occurs, QA should start the app back up and submit the crash via HockeyApp.
 * Design to sync with QA twice per week on any cards in its column for sign off. One approach: test alpha Wednesday and Friday of week 1, Tuesday and Thursday of Week 2 for all cards in Design column. If a crash occurs, Design should start the app back up and submit the crash via HockeyApp.
 * The developer working the card should shepherd it through the columns (e.g., prompt QA or Design if necessary). Engineers, QA, Design, Product Manager, and Scrummaster should be online on IRC during normal business hours in case they need to be nudged about a card.
 * Work product may be rejected and moved back to To Do, with a Comment explaining why, if the implementation did not match the specification. Otherwise, new cards (Tasks) should be filed if follow-on work is required, and the work will be prioritized for later implementation (potentially in the current sprint, but potentially in a later sprint); bug reporting guidelines should be used for such follow-on work as appropriate.
 * If sign off is not required from a particular stakeholder (e.g., Design not needed for data layer testing), the Description card should indicate as much, and the card should be dragged to the appropriate column.
 * The engineer should indicate low level technical testing steps as appropriate to the technical depth of the card.
 * If necessary, note whether the card is high risk.
 * Features that we know can't be used by the end of a sprint should not submitted for TestFlight or App Store.
 * Update the iOS Test Cases for regression testing as necessary.

Philosophy
Release every two weeks. Keep it simple. Give a solid reason if blocking a release. But it's okay if it has to be blocked. This is how we roll.

Feature cutoff is Friday afternoon 3 PM San Francisco on week 2 of the sprint. Cards that don't make it by then get moved to the next sprint or backlogged. Enjoy your weekends!

Release card
There will be a release checklist card added to every sprint board. This should be linked into the roadmap board.

Week 1 of iteration

 * Monday morning: Tech Lead engages Specialists Guild, informing them that Wikipedia Mobile will be released via TestFlight the following day and that their testing results are requested to be returned by Wednesday no later than noon.


 * Monday afternoon: 3 PM San Francisco last minute bugfix code cutoff for prior sprint's work. Ideally, there won't actually be any new code on Monday pertaining to the prior sprint.


 * Tuesday morning, 9:30 AM San Francisco time: whatever's merged into the release branch goes into both Wikipedia Beta for TestFlight external testing and Wikipedia Mobile (stable) for TestFlight internal upgrade smoketesting as the release candidate (instructions). The corollary of this is the engineers need to be judicious about not polluting the code; there are many ways to deal with this.


 * Thursday afternoon at 3 PM San Francisco time there hasn't been serious badness identified (iOS engineering to report all crashes and user reported feedback), we then submit this release candidate to Apple.

Week 2 of iteration
Thursday morning: Product Manager gives final go/no-go for submitting for releasing.

Release
The app should be submitted during early business hours Monday through Thursday in San Francisco, but not Friday, Saturday, or Sunday. Typically this actually should happen the first week of a sprint. Note that if Comms is needed for a big announcement for a given release, they need a heads up well ahead of time, with timely updates as the ship date approaches (the final 5 business days leading up to big announcements are pretty critical with history as a guide). Big splashes don't happen every sprint (more likely, just quarterly), and for most sprints, social media only announcements with less time-pegged blog posts (if any blog post) is fine.

If we hit blockers
Notify Comms, update the release card, etc.

Version Bumps
Note that TestFlight version bumps on the x.y (major.minor) part of the x.y.z version require formal Apple review before they can become available to external testers. Therefore, if a major.minor bump is needed, the TestFlight toggle needs to be flipped in iTunes Connect to get that in the queue with sufficient lead time so that Tuesday cutpoints aren't missed. Know what you're doing before you toggle switches. You don't want to end up effectively disabling in-flight testing on release candidates.

Unit tests
Broken unit tests on the build server constitute a broken build and must be fixed to make work product shippable.

Open questions

 * Branching strategy to handle rejected cards. Realistically, if rejected cards are treated within the same sprint they're worked, no special branching is required. Daily (QA) and twice weekly (Design) feedback should help to avoid serious problems occurring too late in the sprint.
 * Branching strategy for the week 1 or week 2 swarms when they're required. Presently, the thought is to just branch from the corresponding tag and do all work on that branch, merging such fixes upstream to master as well.
 * The format of what's required in the Description field of cards in Phabricator and what constitutes a sufficient Design card (Trello) isn't fully defined, although most of it is specified above. This said, a checklist could potentially help reduce ambiguity.
 * Is a "Spec" column needed on the main iOS board? It would be before the Next Sprint column to reflect when a design specification is still required. Likewise, is an Estimation and a Released column needed in the main iOS board? This way we have a clear liner progression on the main board from Needs Triage to Released.
 * Is full regression testing (TSG) required in case submission for app store review was pushed to week 2 of the sprint? Probably not, as the entire team should be swarming on the issues that caused the delay.