Jump to content

Wikimedia Apps/Team/iOS/Engineering Pre-release Checklist

From mediawiki.org
< Wikimedia Apps‎ | Team‎ | iOS

Note: This checklist is no longer a part of our release process.

Record binary size[edit]

Install the release candidate from TestFlight, then go to General > Storage > record the Wikipedia app size.

Device OS Release Size % change
? ? 6.6.0 (1742) 33.1MB
iPad Pro (11-inch) iOS 14.4 6.8.0 (1803) 35.4MB ↑07%
iPhone 7 iOS 14.4.2 6.8.0 (1803) 35.4MB ↑07%
iPad Pro (11-inch) iOS 14.8.1 6.8.2 (1867) 33.5MB ↓05%
iPhone 7 iOS 14.8.1 6.8.2 (1867) 33.5MB ↓05%
iPad Pro (11-inch) iOS 15.4.1 6.9.0 (1921) 38.4 MB ↑15%
iPhone 7 iOS 15.4.1 6.9.0 (1921) 38.3 MB ↑14%
iPad Pro (11-inch) iOS 16.3 7.1.0 (2027) 36.9 MB ↓3.67%
iPhone 7 iOS 15.7 7.1.0 (2027) 36.9 MB ↓3.66%
iPad Pro (11-inch) iOS 16.3 7.2.0 (2061) 37.2 MB ↑0.81%
iPhone 7 iOS 15.7.3 7.2.0 (2061) 37.3 MB ↑1.08%
iPad Pro (11-inch) iOS 16.4 7.2.1 (2090) 37 MB↑ ↓0.53%
iPhone 7 iOS 15.7.3 7.2.1 (2090) ↑(won't load)
iPhone 13 mini iOS 16.4.1 7.2.2 (2182) 37.7MB
iPad (9th gen) iOS 16.2 7.2.2 (2182) 37MB
iPad Pro (11-inch) iOS 16.5 7.3.0 (2242) 40.3 MB ↑8.91%
iPhone 7 iOS 15.7.6 7.3.0 (2242) 40.3 MB ↑8.04%
iPhone 7 iOS 15.7.6 7.3.1 (2342) 41.2MB ↑2.23%
iPad Pro (11-inch) iOS 16.5 7.3.1 (2342) 45.4MB ↑12.66%
iPhone 13 mini iOS 16.5.1 (c) 7.4.0 (2459) 38.6MB ↑2.4% (comparing same model)
iPad (9th gen) iPadOS 16.2 7.4.0 (2459) 38MB
iPhone 7 iOS 15.7.6 7.4.2 (2736) 41.8MB ↑1.45%
iPad Pro (11-inch) iPadOS 16.6 7.4.2 (2736) 41.8MB ↓7.92%
iPhone 7 iOS 15.7.6 7.4.3 (2817) 43.6MB ↑4.3 %
iPad Pro (11-inch) iPadOS 16.6 7.4.3 (2817) 43.6MB ↑4.3 %
iPhone 7 iOS 15.8 7.4.6 (2963) 43.8MB ↑.045%
iPad Pro (11-inch) iPadOS 16.7.2 7.4.6 (2963) 43.8MB ↑.045%
iPhone 13 mini iOS 17.4.1 7.5.0 (3550) 46,8MB ↓4.7%

Record launch time[edit]

Do these steps for at least two physical devices and record the results.

  1. Run the "App Launch" instrument in instruments. Hover over when the purple phase ends (pre-main time), record the miliseconds timestamp in table below. Hover over when the green phase ends (total time), record the miliseconds timestamp in table below. (Note: until 7.2.0, I did not record the green total time, which is why you see some missing values there).
  2. Do this for a fresh install and for upgrading from the current App Store app with some saved articles.
Device OS Release Type pre-main time

total time

% change
iPhone 7 13.4.1 6.6.0 Fresh install 318.39ms
iPhone 7 13.4.1 6.6.0 Upgrade 326.69ms
iPad Pro 11-inch 13.4.1 6.6.0 Fresh install 328.94ms
iPad Pro 11-inch 13.4.1 6.6.0 Upgrade 405.57ms
iPhone 7 14.4.2 6.8.0 Fresh install 261.58ms ↓18%
iPhone 7 14.4.2 6.8.0 Upgrade 229.45ms ↓30%
iPad Pro 11-inch 14.4 6.8.0 Fresh install 246.05ms ↓25%
iPad Pro 11-inch 14.4 6.8.0 Upgrade 248.27ms ↓39%
iPhone 7 14.8.1 6.8.2 Fresh install 93.73ms ↓64%
iPhone 7 14.8.1 6.8.2 Upgrade 83.00ms ↓64%
iPad Pro 11-inch 14.8.1 6.8.2 Fresh install 242.78ms ↓01%
iPad Pro 11-inch 14.8.1 6.8.2 Upgrade 254.39ms ↑02%
iPhone 7 15.4.1 6.9.0 Fresh install 231.43ms ↑147%
iPhone 7 15.4.1 6.9.0 Upgrade 221.39ms ↑168%
iPad Pro 11-inch 15.4.1 6.9.0 Fresh install 194.73ms ↓20%
iPad Pro 11-inch 15.4.1 6.9.0 Upgrade 177.32ms ↓30%
iPhone 7 15.7 7.1.0 Fresh install 188.15ms ↓18%
iPhone 7 15.7 7.1.0 Upgrade 198.5ms ↓10%
iPad Pro 11-inch 16.3 7.1.0 Fresh install 134.79ms ↓31%
iPad Pro 11-inch 16.3 7.1.0 Upgrade 150.6ms ↓15%
iPhone 7 15.7.3 7.2.0 Fresh install 139.760ms

387.975ms

↓25.72%

?

iPhone 7 15.7.3 7.2.0 Upgrade 141.84ms

298.764ms

↓28.54%

?

iPad Pro 11-inch 16.3 7.2.0 Fresh install 158.444ms

334.627ms

↑17.55%

?

iPad Pro 11-inch 16.3 7.2.0 Upgrade 165.978ms

324.020ms

↑10.21%

?

iPhone 7 15.7.3 7.2.1 (2090) Fresh install 207.493ms

525.212ms

↑48.44%

↑35.37%

iPhone 7 15.7.3 7.2.1 (2090) Upgrade 133.519ms

311.151ms

↓05.87%

↑04.14%

iPad Pro 11-inch 16.4 7.2.1 (2090) Fresh install 171.112ms

480.292ms

↑08.00%

↑43.53%

iPad Pro 11-inch 16.4 7.2.1 (2090) Upgrade 147.925ms

303.338ms

↓06.63%

↓06.38%

iPhone 13 mini 16.4.1 7.2.2 (2182) Fresh Install 162.174ms

326.862ms

iPhone 13 mini 16.4.1 7.2.2 (2182) Upgrade 133.488ms

246.054ms

iPad Pro 11-inch 16.5 7.3.0 (2242) Fresh install 201.014ms

431.290ms

↑17.48%

↓10.20%

iPad Pro 11-inch 16.5 7.3.0 (2242) Upgrade 166.278ms

301.526ms

↑12.41%

↓0.60%

iPhone 7 15.7.6 7.3.0 (2242) Fresh install 188.099ms

414.849ms

↓9.35%

↓21%

iPhone 7 15.7.6 7.3.0 (2242) Upgrade 119.664ms

267.693ms

↓10.38%

↓13.97%

iPhone 7 15.7.6 7.3.1 (2342) Fresh install 144ms

275ms

↓23%

↓33.71%

iPhone 7 15.7.6 7.3.1 (2342) Upgrade 130ms

257ms

↑8.64%

↓4%

iPad Pro 11-inch 16.5 7.3.1 (2342) Fresh install 173ms

292ms

↓14%

↓32%

iPad Pro 11-inch 16.5 7.3.1 (2342) Upgrade 154ms

278ms

↓7%

↓8%

iPhone 13 mini 16.5.1 (c) 7.4.0 (2459) Fresh install 144ms

205ms

iPhone 13 mini 16.5.1 (c) 7.4.0 (2459) Upgrade 155ms

194ms

iPad (9th gen) 16.2 7.4.0 (2459) Fresh install 163ms

237ms

iPad (9th gen) 16.2 7.4.0 (2459) Upgrade 167ms

253ms

iPhone 7 15.7.6 7.4.2 (2736) Fresh install 241ms

570ms

↑67%

↑107%

iPhone 7 15.7.6 7.4.2 (2736) Upgrade 124ms

244ms

↓4.61%

↓5.06%

iPad Pro 11-inch 16.6 7.4.2 (2736) Fresh install 175ms

416ms

↑1.15%

↑42.4%

iPad Pro 11-inch 16.6 7.4.2 (2736) Upgrade 147ms

267ms

↓4.54%

↓3.95%

iPhone 7 15.7.6 7.4.3 (2817) Fresh install 376ms

591ms

↑56%

↑3.7%

iPhone 7 15.7.6 7.4.3 (2817) Upgrade 174ms

319ms

↑40%

↑31%

iPad Pro 11-inch 16.6 7.4.3 (2817) Fresh install 167ms

284ms

↓5%

↓32%

iPad Pro 11-inch 16.6 7.4.3 (2817) Upgrade 150ms

248ms

↑2%

↓7%

iPhone 7 15.8 7.4.6 (2963) Fresh install 210ms

364ms

↓44%

↓38%

iPhone 7 15.8 7.4.6 (2963) Upgrade 134ms

253ms

↓22%

↓20%

iPad Pro (11-inch) 16.7.2 7.4.6 (2963) Fresh install 175ms

297ms

↑4.7%

↑4.5%

iPad Pro (11-inch) 16.7.2 7.4.6 (2963) Upgrade 156ms

260ms

↑4%

↑4.83%

Run article load performance tests[edit]

Currently we have some performance tests that are not set up to work with our build server / CI process. Until then, it's good to run these manually pre-release as a sanity check. The article loading performance tests are in ArticleManualPerformanceTests.swift. Run these on at least two devices from the Performance Testing scheme. Note that ArticleManualPerformanceTests.swift will stub out the network call with Nocilla, so it is a true native-only performance test.

Device OS Release Test name average time % change
iPhone 7 14.4.2 6.6.0 (1742) testArticleSetupTime 0.452s
iPhone 7 14.4.2 6.6.0 (1742) testContextMenuConfigTime 0.120s
iPhone 7 14.4.2 6.6.0 (1742) testArticlePeekPreviewControllerDisplayTime 0.085s
iPad Pro 11-inch 14.4 6.6.0 (1742) testArticleSetupTime 0.351s
iPad Pro 11-inch 14.4 6.6.0 (1742) testContextMenuConfigTime 0.093s
iPad Pro 11-inch 14.4 6.6.0 (1742) testArticlePeekPreviewControllerDisplayTime 0.061s
iPhone 7 14.4.2 6.8.0 testArticleSetupTime 0.501s ↑11%
iPhone 7 14.4.2 6.8.0 testContextMenuConfigTime 0.106s ↓12%
iPhone 7 14.4.2 6.8.0 testArticlePeekPreviewControllerDisplayTime 0.051s ↓40%
iPad Pro (11-inch) 14.4 6.8.0 testArticleSetupTime 0.309s ↓12%
iPad Pro (11-inch) 14.4 6.8.0 testContextMenuConfigTime 0.080s ↓14%
iPad Pro (11-inch) 14.4 6.8.0 testArticlePeekPreviewControllerDisplayTime 0.039s ↓36%
iPhone 7 14.8.1 6.8.2 testArticleSetupTime 0.456s ↓9%
iPhone 7 14.8.1 6.8.2 testContextMenuConfigTime 0.150s ↑41%
iPhone 7 14.8.1 6.8.2 testArticlePeekPreviewControllerDisplayTime 0.059 ↑15%
iPad Pro (11-inch) 14.8.1 6.8.2 testArticleSetupTime 0.329s ↑6%
iPad Pro (11-inch) 14.8.1 6.8.2 testContextMenuConfigTime 0.090s ↑12.5%
iPad Pro (11-inch) 14.8.1 6.8.2 testArticlePeekPreviewControllerDisplayTime 0.050s ↑28%
iPhone 7 15.4.1 6.9.0 testArticleSetupTime 0.463s ↑2%
iPhone 7 15.4.1 6.9.0 testContextMenuConfigTime 0.042s ↓72%
iPhone 7 15.4.1 6.9.0 testArticlePeekPreviewControllerDisplayTime 0.036s ↓39%
iPad Pro (11-inch) 15.4.1 6.9.0 testArticleSetupTime 0.326s ↓1%
iPad Pro (11-inch) 15.4.1 6.9.0 testContextMenuConfigTime 0.064s ↓29%
iPad Pro (11-inch) 15.4.1 6.9.0 testArticlePeekPreviewControllerDisplayTime 0.036s ↓28%
iPhone 7 15.7 7.1.0 testArticleSetupTime 0.541s ↑17%
iPhone 7 15.7 7.1.0 testContextMenuConfigTime 0.121s ↑188%
iPhone 7 15.7 7.1.0 testArticlePeekPreviewControllerDisplayTime 0.063s ↑75%
iPad Pro (11-inch) 16.3 7.1.0 testArticleSetupTime 0.493s ↑51%
iPad Pro (11-inch) 16.3 7.1.0 testContextMenuConfigTime 0.262s ↑309%
iPad Pro (11-inch) 16.3 7.1.0 testArticlePeekPreviewControllerDisplayTime 0.235s ↑552%
iPhone 7 15.7.3 7.2.0 testArticleSetupTime 0.539s ↓0.37%
iPhone 7 15.7.3 7.2.0 testContextMenuConfigTime 0.122s ↑0.83%
iPhone 7 15.7.3 7.2.0 testArticlePeekPreviewControllerDisplayTime 0.059s ↓6.34
iPad Pro (11-inch) 16.3 7.2.0 testArticleSetupTime 0.517s ↑4.87%
iPad Pro (11-inch) 16.3 7.2.0 testContextMenuConfigTime 0.288s ↑136%
iPad Pro (11-inch) 16.3 7.2.0 testArticlePeekPreviewControllerDisplayTime 0.031s ↓47.46%
iPhone 7 15.7.3 7.2.1 (2090) testArticleSetupTime 0.627s ↑16.33%
iPhone 7 15.7.3 7.2.1 (2090) testContextMenuConfigTime 0.162s ↑32.79%
iPhone 7 15.7.3 7.2.1 (2090) testArticlePeekPreviewControllerDisplayTime 0.036s ↓38.98%
iPad Pro (11-inch) 16.4 7.2.1 (2090) testArticleSetupTime 0.495s ↓04.26%
iPad Pro (11-inch) 16.4 7.2.1 (2090) testContextMenuConfigTime 0.276s ↓04.17%
iPad Pro (11-inch) 16.4 7.2.1 (2090) testArticlePeekPreviewControllerDisplayTime 0.027s ↓12.90%
iPhone 13 mini 16.4.1 7.2.2 (2182) testArticleSetupTime 0.266s
iPhone 13 mini 16.4.1 7.2.2 (2182) testContextMenuConfigTime 0.158s
iPhone 13 mini 16.4.1 7.2.2 (2182) testArticlePeekPreviewControllerDisplayTime 0.139s
iPad Pro (11-inch) 16.5 7.3.0 (2242) testArticleSetupTime 0.437s ↓12%
iPad Pro (11-inch) 16.5 7.3.0 (2242) testContextMenuConfigTime 0.267s ↓3%
iPad Pro (11-inch) 16.5 7.3.0 (2242) testArticlePeekPreviewControllerDisplayTime 0.224s ↑730%
iPhone 7 15.7.6 7.3.0 (2242) testArticleSetupTime 0.434s ↓30.78%
iPhone 7 15.7.6 7.3.0 (2242) testContextMenuConfigTime 0.162s 0%
iPhone 7 15.7.6 7.3.0 (2242) testArticlePeekPreviewControllerDisplayTime 0.049s ↑36%
iPad Pro (11-inch) 16.5 7.3.1 (2342) testArticleSetupTime 0.252s ↓42%
iPad Pro (11-inch) 16.5 7.3.1 (2342) testContextMenuConfigTime 0.117s ↓56%
iPad Pro (11-inch) 16.5 7.3.1 (2342) testArticlePeekPreviewControllerDisplayTime 0.031s ↓86%
iPhone 7 15.7.6 7.3.1 (2342) testArticleSetupTime 0.384s ↓12%
iPhone 7 15.7.6 7.3.1 (2342) testContextMenuConfigTime 0.121s ↓25%
iPhone 7 15.7.6 7.3.1 (2342) testArticlePeekPreviewControllerDisplayTime 0.047s ↓4%
iPhone 13 mini 16.5.1 (c) 7.4.0 (2459) testArticleSetupTime 0.169s
iPhone 13 mini 16.5.1 (c) 7.4.0 (2459) testContextMenuConfigTime 0.050s
iPhone 13 mini 16.5.1 (c) 7.4.0 (2459) testArticlePeekPreviewControllerDisplayTime 0.021s
iPad (9th gen) 16.2 7.4.0 (2459) testArticleSetupTime 0.393s
iPad (9th gen) 16.2 7.4.0 (2459) testContextMenuConfigTime 0.210s
iPad (9th gen) 16.2 7.4.0 (2459) testArticlePeekPreviewControllerDisplayTime 0.170s
iPhone 7 15.7.6 7.4.2 (2736) testArticleSetupTime 0.345s ↓10%
iPhone 7 15.7.6 7.4.2 (2736) testContextMenuConfigTime 0.126s ↑4%
iPhone 7 15.7.6 7.4.2 (2736) testArticlePeekPreviewControllerDisplayTime 0.036s ↓23%
iPad Pro (11-inch) 16.6 7.4.2 (2736) testArticleSetupTime 0.258s ↑2.38%
iPad Pro (11-inch) 16.6 7.4.2 (2736) testContextMenuConfigTime 0.129s ↑10.26%
iPad Pro (11-inch) 16.6 7.4.2 (2736) testArticlePeekPreviewControllerDisplayTime 0.205 ↑561.2%
iPhone 7 15.7.6 7.4.3 (2817) testArticleSetupTime 0.45s ↑30%
iPhone 7 15.7.6 7.4.3 (2817) testContextMenuConfigTime 0.141s ↑12%
iPhone 7 15.7.6 7.4.3 (2817) testArticlePeekPreviewControllerDisplayTime 0.058s ↑61%
iPad Pro (11-inch) 16.6 7.4.3 (2817) testArticleSetupTime 0.776s ↑201%
iPad Pro (11-inch) 16.6 7.4.3 (2817) testContextMenuConfigTime 0.648s ↑402%
iPad Pro (11-inch) 16.6 7.4.3 (2817) testArticlePeekPreviewControllerDisplayTime 0.058s ↓72%
iPhone 7 15.8 7.4.6 (2963) testArticleSetupTime 0.408s ↑806%
iPhone 7 15.8 7.4.6 (2963) testContextMenuConfigTime 0.146s ↑3.54%
iPhone 7 15.8 7.4.6 (2963) testArticlePeekPreviewControllerDisplayTime 0.055s ↓5%
iPad Pro (11-inch) 16.7.2 7.4.6 (2963) testArticleSetupTime 0.729s ↓6%
iPad Pro (11-inch) 16.7.2 7.4.6 (2963) testContextMenuConfigTime 0.620s ↓4%
iPad Pro (11-inch) 16.7.2 7.4.6 (2963) testArticlePeekPreviewControllerDisplayTime 0.588s ↑913%