Wikimedia Release Engineering Team/Pretrain/Progress reports/2026-06-12

← week ending 2026-06-05

Pretrain progress reports week ending 2026-06-12

week ending 2026-06-19 →

Report on activities in the Pretrain project for the week ending 2026-06-12.

[WE6.7.2] Routing test

Tracked in Phabricator
Task T427659

If we create the Pretrain deployment environment using existing production MediaWiki container images and begin routing testwiki traffic to it, we will develop confidence that our routing design is sufficiently complete to support the full Pretrain MVP (e.g., all testwiki traffic is reliably served by the Pretrain environment and container).

Progress update

• Stage: Engineering/Development
• This week, we continued to refine implementation details and started to onboard an additional engineer to accelerate turn-up. The majority of that work focused on T427668, specifically the limited use case we envision for k8s Ingress in Pretrain and how it must integrate with prior art from Single-Version routing experiments applied to Wikifunctions in 2025 (T384944). Some additional research into the state of non-standard internal API traffic routing for ML serving workloads also made progress (T427666).

Any new metrics related to the hypothesis

• None

Any emerging blockers or risks

• Not yet

Any unresolved dependencies - do you depend on another team that hasn’t already given you what you need? Are you on the hook to give another team something you aren’t able to give right now?

• No

Have there been any new lessons from the hypothesis?

• Not yet

Have there been any changes to the hypothesis scope or timeline?

• No

[WE6.7.3] Automated supervision

Tracked in Phabricator
Task T427968

If we implement an initial set of automated supervision strategies and enable automated deployment to the Pretrain environment for the routing test, we will develop confidence that automated supervision is ready to serve as our primary risk mitigation for the full Pretrain MVP (e.g., false positives are understood, and we have identified a path to mitigate them).

Progress update

• Stage: Design
• Bryan met with Scott to talk in more detail about Scott's requirements and current thoughts on automated supervision. That discussion led to examining and updating Scott's Productionizing Pretrain :: Automated supervision of MediaWiki deployments document.
• Bryan created new backlog tasks based on his current understanding:
  • T428971: Allow configuration of canary and production checks based on deployment target
  • T428972: Configure additional httpbb checks to perform during a deployment
  • T428976: Investigate dynamic threshold model comparisons for canary and production log volume checks
• Ahmon was out of office most of this week.
• Bryan will be on vacation for the next two weeks. Tyler will cover reporting in that period.

Any new metrics related to the hypothesis

• None

Any emerging blockers or risks

• Not yet

Any unresolved dependencies - do you depend on another team that hasn’t already given you what you need? Are you on the hook to give another team something you aren’t able to give right now?

• No

Have there been any new lessons from the hypothesis?

• Not yet

Have there been any changes to the hypothesis scope or timeline?

• No