Wikimedia Apps/Team/iOS/Engineering Pre-release Checklist
Note: This checklist is no longer a part of our release process.
Record binary size[edit]
Install the release candidate from TestFlight, then go to General > Storage > record the Wikipedia app size.
| Device | OS | Release | Size | % change |
|---|---|---|---|---|
| ? | ? | 6.6.0 (1742) | 33.1MB | |
| iPad Pro (11-inch) | iOS 14.4 | 6.8.0 (1803) | 35.4MB | ↑07% |
| iPhone 7 | iOS 14.4.2 | 6.8.0 (1803) | 35.4MB | ↑07% |
| iPad Pro (11-inch) | iOS 14.8.1 | 6.8.2 (1867) | 33.5MB | ↓05% |
| iPhone 7 | iOS 14.8.1 | 6.8.2 (1867) | 33.5MB | ↓05% |
| iPad Pro (11-inch) | iOS 15.4.1 | 6.9.0 (1921) | 38.4 MB | ↑15% |
| iPhone 7 | iOS 15.4.1 | 6.9.0 (1921) | 38.3 MB | ↑14% |
| iPad Pro (11-inch) | iOS 16.3 | 7.1.0 (2027) | 36.9 MB | ↓3.67% |
| iPhone 7 | iOS 15.7 | 7.1.0 (2027) | 36.9 MB | ↓3.66% |
| iPad Pro (11-inch) | iOS 16.3 | 7.2.0 (2061) | 37.2 MB | ↑0.81% |
| iPhone 7 | iOS 15.7.3 | 7.2.0 (2061) | 37.3 MB | ↑1.08% |
| iPad Pro (11-inch) | iOS 16.4 | 7.2.1 (2090) | 37 MB↑ | ↓0.53% |
| iPhone 7 | iOS 15.7.3 | 7.2.1 (2090) | ↑(won't load) | |
| iPhone 13 mini | iOS 16.4.1 | 7.2.2 (2182) | 37.7MB | |
| iPad (9th gen) | iOS 16.2 | 7.2.2 (2182) | 37MB | |
| iPad Pro (11-inch) | iOS 16.5 | 7.3.0 (2242) | 40.3 MB | ↑8.91% |
| iPhone 7 | iOS 15.7.6 | 7.3.0 (2242) | 40.3 MB | ↑8.04% |
| iPhone 7 | iOS 15.7.6 | 7.3.1 (2342) | 41.2MB | ↑2.23% |
| iPad Pro (11-inch) | iOS 16.5 | 7.3.1 (2342) | 45.4MB | ↑12.66% |
| iPhone 13 mini | iOS 16.5.1 (c) | 7.4.0 (2459) | 38.6MB | ↑2.4% (comparing same model) |
| iPad (9th gen) | iPadOS 16.2 | 7.4.0 (2459) | 38MB | |
| iPhone 7 | iOS 15.7.6 | 7.4.2 (2736) | 41.8MB | ↑1.45% |
| iPad Pro (11-inch) | iPadOS 16.6 | 7.4.2 (2736) | 41.8MB | ↓7.92% |
| iPhone 7 | iOS 15.7.6 | 7.4.3 (2817) | 43.6MB | ↑4.3 % |
| iPad Pro (11-inch) | iPadOS 16.6 | 7.4.3 (2817) | 43.6MB | ↑4.3 % |
| iPhone 7 | iOS 15.8 | 7.4.6 (2963) | 43.8MB | ↑.045% |
| iPad Pro (11-inch) | iPadOS 16.7.2 | 7.4.6 (2963) | 43.8MB | ↑.045% |
Record launch time[edit]
Do these steps for at least two physical devices and record the results.
- Run the "App Launch" instrument in instruments. Hover over when the purple phase ends (pre-main time), record the miliseconds timestamp in table below. Hover over when the green phase ends (total time), record the miliseconds timestamp in table below. (Note: until 7.2.0, I did not record the green total time, which is why you see some missing values there).
- Do this for a fresh install and for upgrading from the current App Store app with some saved articles.
| Device | OS | Release | Type | pre-main time
total time |
% change |
|---|---|---|---|---|---|
| iPhone 7 | 13.4.1 | 6.6.0 | Fresh install | 318.39ms | |
| iPhone 7 | 13.4.1 | 6.6.0 | Upgrade | 326.69ms | |
| iPad Pro 11-inch | 13.4.1 | 6.6.0 | Fresh install | 328.94ms | |
| iPad Pro 11-inch | 13.4.1 | 6.6.0 | Upgrade | 405.57ms | |
| iPhone 7 | 14.4.2 | 6.8.0 | Fresh install | 261.58ms | ↓18% |
| iPhone 7 | 14.4.2 | 6.8.0 | Upgrade | 229.45ms | ↓30% |
| iPad Pro 11-inch | 14.4 | 6.8.0 | Fresh install | 246.05ms | ↓25% |
| iPad Pro 11-inch | 14.4 | 6.8.0 | Upgrade | 248.27ms | ↓39% |
| iPhone 7 | 14.8.1 | 6.8.2 | Fresh install | 93.73ms | ↓64% |
| iPhone 7 | 14.8.1 | 6.8.2 | Upgrade | 83.00ms | ↓64% |
| iPad Pro 11-inch | 14.8.1 | 6.8.2 | Fresh install | 242.78ms | ↓01% |
| iPad Pro 11-inch | 14.8.1 | 6.8.2 | Upgrade | 254.39ms | ↑02% |
| iPhone 7 | 15.4.1 | 6.9.0 | Fresh install | 231.43ms | ↑147% |
| iPhone 7 | 15.4.1 | 6.9.0 | Upgrade | 221.39ms | ↑168% |
| iPad Pro 11-inch | 15.4.1 | 6.9.0 | Fresh install | 194.73ms | ↓20% |
| iPad Pro 11-inch | 15.4.1 | 6.9.0 | Upgrade | 177.32ms | ↓30% |
| iPhone 7 | 15.7 | 7.1.0 | Fresh install | 188.15ms | ↓18% |
| iPhone 7 | 15.7 | 7.1.0 | Upgrade | 198.5ms | ↓10% |
| iPad Pro 11-inch | 16.3 | 7.1.0 | Fresh install | 134.79ms | ↓31% |
| iPad Pro 11-inch | 16.3 | 7.1.0 | Upgrade | 150.6ms | ↓15% |
| iPhone 7 | 15.7.3 | 7.2.0 | Fresh install | 139.760ms
387.975ms |
↓25.72%
? |
| iPhone 7 | 15.7.3 | 7.2.0 | Upgrade | 141.84ms
298.764ms |
↓28.54%
? |
| iPad Pro 11-inch | 16.3 | 7.2.0 | Fresh install | 158.444ms
334.627ms |
↑17.55%
? |
| iPad Pro 11-inch | 16.3 | 7.2.0 | Upgrade | 165.978ms
324.020ms |
↑10.21%
? |
| iPhone 7 | 15.7.3 | 7.2.1 (2090) | Fresh install | 207.493ms
525.212ms |
↑48.44%
↑35.37% |
| iPhone 7 | 15.7.3 | 7.2.1 (2090) | Upgrade | 133.519ms
311.151ms |
↓05.87%
↑04.14% |
| iPad Pro 11-inch | 16.4 | 7.2.1 (2090) | Fresh install | 171.112ms
480.292ms |
↑08.00%
↑43.53% |
| iPad Pro 11-inch | 16.4 | 7.2.1 (2090) | Upgrade | 147.925ms
303.338ms |
↓06.63%
↓06.38% |
| iPhone 13 mini | 16.4.1 | 7.2.2 (2182) | Fresh Install | 162.174ms
326.862ms |
|
| iPhone 13 mini | 16.4.1 | 7.2.2 (2182) | Upgrade | 133.488ms
246.054ms |
|
| iPad Pro 11-inch | 16.5 | 7.3.0 (2242) | Fresh install | 201.014ms
431.290ms |
↑17.48%
↓10.20% |
| iPad Pro 11-inch | 16.5 | 7.3.0 (2242) | Upgrade | 166.278ms
301.526ms |
↑12.41%
↓0.60% |
| iPhone 7 | 15.7.6 | 7.3.0 (2242) | Fresh install | 188.099ms
414.849ms |
↓9.35%
↓21% |
| iPhone 7 | 15.7.6 | 7.3.0 (2242) | Upgrade | 119.664ms
267.693ms |
↓10.38%
↓13.97% |
| iPhone 7 | 15.7.6 | 7.3.1 (2342) | Fresh install | 144ms
275ms |
↓23%
↓33.71% |
| iPhone 7 | 15.7.6 | 7.3.1 (2342) | Upgrade | 130ms
257ms |
↑8.64%
↓4% |
| iPad Pro 11-inch | 16.5 | 7.3.1 (2342) | Fresh install | 173ms
292ms |
↓14%
↓32% |
| iPad Pro 11-inch | 16.5 | 7.3.1 (2342) | Upgrade | 154ms
278ms |
↓7%
↓8% |
| iPhone 13 mini | 16.5.1 (c) | 7.4.0 (2459) | Fresh install | 144ms
205ms |
|
| iPhone 13 mini | 16.5.1 (c) | 7.4.0 (2459) | Upgrade | 155ms
194ms |
|
| iPad (9th gen) | 16.2 | 7.4.0 (2459) | Fresh install | 163ms
237ms |
|
| iPad (9th gen) | 16.2 | 7.4.0 (2459) | Upgrade | 167ms
253ms |
|
| iPhone 7 | 15.7.6 | 7.4.2 (2736) | Fresh install | 241ms
570ms |
↑67%
↑107% |
| iPhone 7 | 15.7.6 | 7.4.2 (2736) | Upgrade | 124ms
244ms |
↓4.61%
↓5.06% |
| iPad Pro 11-inch | 16.6 | 7.4.2 (2736) | Fresh install | 175ms
416ms |
↑1.15%
↑42.4% |
| iPad Pro 11-inch | 16.6 | 7.4.2 (2736) | Upgrade | 147ms
267ms |
↓4.54%
↓3.95% |
| iPhone 7 | 15.7.6 | 7.4.3 (2817) | Fresh install | 376ms
591ms |
↑56%
↑3.7% |
| iPhone 7 | 15.7.6 | 7.4.3 (2817) | Upgrade | 174ms
319ms |
↑40%
↑31% |
| iPad Pro 11-inch | 16.6 | 7.4.3 (2817) | Fresh install | 167ms
284ms |
↓5%
↓32% |
| iPad Pro 11-inch | 16.6 | 7.4.3 (2817) | Upgrade | 150ms
248ms |
↑2%
↓7% |
| iPhone 7 | 15.8 | 7.4.6 (2963) | Fresh install | 210ms
364ms |
↓44%
↓38% |
| iPhone 7 | 15.8 | 7.4.6 (2963) | Upgrade | 134ms
253ms |
↓22%
↓20% |
| iPad Pro (11-inch) | 16.7.2 | 7.4.6 (2963) | Fresh install | 175ms
297ms |
↑4.7%
↑4.5% |
| iPad Pro (11-inch) | 16.7.2 | 7.4.6 (2963) | Upgrade | 156ms
260ms |
↑4%
↑4.83% |
Run article load performance tests[edit]
Currently we have some performance tests that are not set up to work with our build server / CI process. Until then, it's good to run these manually pre-release as a sanity check. The article loading performance tests are in ArticleManualPerformanceTests.swift. Run these on at least two devices from the Performance Testing scheme. Note that ArticleManualPerformanceTests.swift will stub out the network call with Nocilla, so it is a true native-only performance test.
| Device | OS | Release | Test name | average time | % change |
|---|---|---|---|---|---|
| iPhone 7 | 14.4.2 | 6.6.0 (1742) | testArticleSetupTime | 0.452s | |
| iPhone 7 | 14.4.2 | 6.6.0 (1742) | testContextMenuConfigTime | 0.120s | |
| iPhone 7 | 14.4.2 | 6.6.0 (1742) | testArticlePeekPreviewControllerDisplayTime | 0.085s | |
| iPad Pro 11-inch | 14.4 | 6.6.0 (1742) | testArticleSetupTime | 0.351s | |
| iPad Pro 11-inch | 14.4 | 6.6.0 (1742) | testContextMenuConfigTime | 0.093s | |
| iPad Pro 11-inch | 14.4 | 6.6.0 (1742) | testArticlePeekPreviewControllerDisplayTime | 0.061s | |
| iPhone 7 | 14.4.2 | 6.8.0 | testArticleSetupTime | 0.501s | ↑11% |
| iPhone 7 | 14.4.2 | 6.8.0 | testContextMenuConfigTime | 0.106s | ↓12% |
| iPhone 7 | 14.4.2 | 6.8.0 | testArticlePeekPreviewControllerDisplayTime | 0.051s | ↓40% |
| iPad Pro (11-inch) | 14.4 | 6.8.0 | testArticleSetupTime | 0.309s | ↓12% |
| iPad Pro (11-inch) | 14.4 | 6.8.0 | testContextMenuConfigTime | 0.080s | ↓14% |
| iPad Pro (11-inch) | 14.4 | 6.8.0 | testArticlePeekPreviewControllerDisplayTime | 0.039s | ↓36% |
| iPhone 7 | 14.8.1 | 6.8.2 | testArticleSetupTime | 0.456s | ↓9% |
| iPhone 7 | 14.8.1 | 6.8.2 | testContextMenuConfigTime | 0.150s | ↑41% |
| iPhone 7 | 14.8.1 | 6.8.2 | testArticlePeekPreviewControllerDisplayTime | 0.059 | ↑15% |
| iPad Pro (11-inch) | 14.8.1 | 6.8.2 | testArticleSetupTime | 0.329s | ↑6% |
| iPad Pro (11-inch) | 14.8.1 | 6.8.2 | testContextMenuConfigTime | 0.090s | ↑12.5% |
| iPad Pro (11-inch) | 14.8.1 | 6.8.2 | testArticlePeekPreviewControllerDisplayTime | 0.050s | ↑28% |
| iPhone 7 | 15.4.1 | 6.9.0 | testArticleSetupTime | 0.463s | ↑2% |
| iPhone 7 | 15.4.1 | 6.9.0 | testContextMenuConfigTime | 0.042s | ↓72% |
| iPhone 7 | 15.4.1 | 6.9.0 | testArticlePeekPreviewControllerDisplayTime | 0.036s | ↓39% |
| iPad Pro (11-inch) | 15.4.1 | 6.9.0 | testArticleSetupTime | 0.326s | ↓1% |
| iPad Pro (11-inch) | 15.4.1 | 6.9.0 | testContextMenuConfigTime | 0.064s | ↓29% |
| iPad Pro (11-inch) | 15.4.1 | 6.9.0 | testArticlePeekPreviewControllerDisplayTime | 0.036s | ↓28% |
| iPhone 7 | 15.7 | 7.1.0 | testArticleSetupTime | 0.541s | ↑17% |
| iPhone 7 | 15.7 | 7.1.0 | testContextMenuConfigTime | 0.121s | ↑188% |
| iPhone 7 | 15.7 | 7.1.0 | testArticlePeekPreviewControllerDisplayTime | 0.063s | ↑75% |
| iPad Pro (11-inch) | 16.3 | 7.1.0 | testArticleSetupTime | 0.493s | ↑51% |
| iPad Pro (11-inch) | 16.3 | 7.1.0 | testContextMenuConfigTime | 0.262s | ↑309% |
| iPad Pro (11-inch) | 16.3 | 7.1.0 | testArticlePeekPreviewControllerDisplayTime | 0.235s | ↑552% |
| iPhone 7 | 15.7.3 | 7.2.0 | testArticleSetupTime | 0.539s | ↓0.37% |
| iPhone 7 | 15.7.3 | 7.2.0 | testContextMenuConfigTime | 0.122s | ↑0.83% |
| iPhone 7 | 15.7.3 | 7.2.0 | testArticlePeekPreviewControllerDisplayTime | 0.059s | ↓6.34 |
| iPad Pro (11-inch) | 16.3 | 7.2.0 | testArticleSetupTime | 0.517s | ↑4.87% |
| iPad Pro (11-inch) | 16.3 | 7.2.0 | testContextMenuConfigTime | 0.288s | ↑136% |
| iPad Pro (11-inch) | 16.3 | 7.2.0 | testArticlePeekPreviewControllerDisplayTime | 0.031s | ↓47.46% |
| iPhone 7 | 15.7.3 | 7.2.1 (2090) | testArticleSetupTime | 0.627s | ↑16.33% |
| iPhone 7 | 15.7.3 | 7.2.1 (2090) | testContextMenuConfigTime | 0.162s | ↑32.79% |
| iPhone 7 | 15.7.3 | 7.2.1 (2090) | testArticlePeekPreviewControllerDisplayTime | 0.036s | ↓38.98% |
| iPad Pro (11-inch) | 16.4 | 7.2.1 (2090) | testArticleSetupTime | 0.495s | ↓04.26% |
| iPad Pro (11-inch) | 16.4 | 7.2.1 (2090) | testContextMenuConfigTime | 0.276s | ↓04.17% |
| iPad Pro (11-inch) | 16.4 | 7.2.1 (2090) | testArticlePeekPreviewControllerDisplayTime | 0.027s | ↓12.90% |
| iPhone 13 mini | 16.4.1 | 7.2.2 (2182) | testArticleSetupTime | 0.266s | |
| iPhone 13 mini | 16.4.1 | 7.2.2 (2182) | testContextMenuConfigTime | 0.158s | |
| iPhone 13 mini | 16.4.1 | 7.2.2 (2182) | testArticlePeekPreviewControllerDisplayTime | 0.139s | |
| iPad Pro (11-inch) | 16.5 | 7.3.0 (2242) | testArticleSetupTime | 0.437s | ↓12% |
| iPad Pro (11-inch) | 16.5 | 7.3.0 (2242) | testContextMenuConfigTime | 0.267s | ↓3% |
| iPad Pro (11-inch) | 16.5 | 7.3.0 (2242) | testArticlePeekPreviewControllerDisplayTime | 0.224s | ↑730% |
| iPhone 7 | 15.7.6 | 7.3.0 (2242) | testArticleSetupTime | 0.434s | ↓30.78% |
| iPhone 7 | 15.7.6 | 7.3.0 (2242) | testContextMenuConfigTime | 0.162s | 0% |
| iPhone 7 | 15.7.6 | 7.3.0 (2242) | testArticlePeekPreviewControllerDisplayTime | 0.049s | ↑36% |
| iPad Pro (11-inch) | 16.5 | 7.3.1 (2342) | testArticleSetupTime | 0.252s | ↓42% |
| iPad Pro (11-inch) | 16.5 | 7.3.1 (2342) | testContextMenuConfigTime | 0.117s | ↓56% |
| iPad Pro (11-inch) | 16.5 | 7.3.1 (2342) | testArticlePeekPreviewControllerDisplayTime | 0.031s | ↓86% |
| iPhone 7 | 15.7.6 | 7.3.1 (2342) | testArticleSetupTime | 0.384s | ↓12% |
| iPhone 7 | 15.7.6 | 7.3.1 (2342) | testContextMenuConfigTime | 0.121s | ↓25% |
| iPhone 7 | 15.7.6 | 7.3.1 (2342) | testArticlePeekPreviewControllerDisplayTime | 0.047s | ↓4% |
| iPhone 13 mini | 16.5.1 (c) | 7.4.0 (2459) | testArticleSetupTime | 0.169s | |
| iPhone 13 mini | 16.5.1 (c) | 7.4.0 (2459) | testContextMenuConfigTime | 0.050s | |
| iPhone 13 mini | 16.5.1 (c) | 7.4.0 (2459) | testArticlePeekPreviewControllerDisplayTime | 0.021s | |
| iPad (9th gen) | 16.2 | 7.4.0 (2459) | testArticleSetupTime | 0.393s | |
| iPad (9th gen) | 16.2 | 7.4.0 (2459) | testContextMenuConfigTime | 0.210s | |
| iPad (9th gen) | 16.2 | 7.4.0 (2459) | testArticlePeekPreviewControllerDisplayTime | 0.170s | |
| iPhone 7 | 15.7.6 | 7.4.2 (2736) | testArticleSetupTime | 0.345s | ↓10% |
| iPhone 7 | 15.7.6 | 7.4.2 (2736) | testContextMenuConfigTime | 0.126s | ↑4% |
| iPhone 7 | 15.7.6 | 7.4.2 (2736) | testArticlePeekPreviewControllerDisplayTime | 0.036s | ↓23% |
| iPad Pro (11-inch) | 16.6 | 7.4.2 (2736) | testArticleSetupTime | 0.258s | ↑2.38% |
| iPad Pro (11-inch) | 16.6 | 7.4.2 (2736) | testContextMenuConfigTime | 0.129s | ↑10.26% |
| iPad Pro (11-inch) | 16.6 | 7.4.2 (2736) | testArticlePeekPreviewControllerDisplayTime | 0.205 | ↑561.2% |
| iPhone 7 | 15.7.6 | 7.4.3 (2817) | testArticleSetupTime | 0.45s | ↑30% |
| iPhone 7 | 15.7.6 | 7.4.3 (2817) | testContextMenuConfigTime | 0.141s | ↑12% |
| iPhone 7 | 15.7.6 | 7.4.3 (2817) | testArticlePeekPreviewControllerDisplayTime | 0.058s | ↑61% |
| iPad Pro (11-inch) | 16.6 | 7.4.3 (2817) | testArticleSetupTime | 0.776s | ↑201% |
| iPad Pro (11-inch) | 16.6 | 7.4.3 (2817) | testContextMenuConfigTime | 0.648s | ↑402% |
| iPad Pro (11-inch) | 16.6 | 7.4.3 (2817) | testArticlePeekPreviewControllerDisplayTime | 0.058s | ↓72% |
| iPhone 7 | 15.8 | 7.4.6 (2963) | testArticleSetupTime | 0.408s | ↑806% |
| iPhone 7 | 15.8 | 7.4.6 (2963) | testContextMenuConfigTime | 0.146s | ↑3.54% |
| iPhone 7 | 15.8 | 7.4.6 (2963) | testArticlePeekPreviewControllerDisplayTime | 0.055s | ↓5% |
| iPad Pro (11-inch) | 16.7.2 | 7.4.6 (2963) | testArticleSetupTime | 0.729s | ↓6% |
| iPad Pro (11-inch) | 16.7.2 | 7.4.6 (2963) | testContextMenuConfigTime | 0.620s | ↓4% |
| iPad Pro (11-inch) | 16.7.2 | 7.4.6 (2963) | testArticlePeekPreviewControllerDisplayTime | 0.588s | ↑913% |