Data Platform Engineering/Data Platform SRE/Status Update/2026-07-03
Highlights
[edit]- We have 2 new team members! Machine Learning SREs are joining us.
- Ongoing discussion with core-SRE to see how we can improve support for the Turnilo/Druid in the context of anti-DDoS.
- New security model for the Data Platform services is ready for internal review.
- Planning and investigation to migrate all of our servers to Debian Bookworm or later.
- OpenSearch/CirrusSearch upgraded to 2.x on codfw, only eqiad left!
- Completed migration for ML staging k8s, made some progress on prod codfw.
OpenSearch upgrade
[edit]- T425585 Write lightweight OCI-image-based Puppet plans for beta cluster (Resolved)
- T430882 Beta cluster is not working due to error in Translate\TtmServer\ServiceCreationFailure (Resolved)
- T430726 cirrussearch2089.codfw.wmnet fails reimage cookbook (DRAC error)? (Resolved)
Wikidata Platform support
[edit]Misc / Operations
[edit]- T412978 Support for Java 21 and Flink 2 (Resolved)
- T429861 codfw: rack B2 maintenance 2026-07-01 11:00 am CT (Resolved)
- T430219 Degraded RAID on an-worker1231 (Resolved)
- T430485 Add wme_metrics connection to analytics-airflow-product (Resolved)
- T430868 Create depool hiera keys for wdqs hosts (Resolved)
- T403800 Use unified service urls for DPE services (Declined)
- T430484 "IPoid request failed for IP" logs since 2026-06-25 (Resolved)
- T430504 dse-k8s-codfw istiogateway misconfiguration (Resolved)
- T430307 Enable Custom Dimensions plugin in Matomo (Resolved)
- T426196 Investigate an-druid1007 unresponsiveness (Resolved)
- T424112 Presto cluster improvements for concurrency and workload (Resolved)
- T412428 Wikidata full .json.gz dumps not published since 20250625 (Resolved)
- T429888 Delete empty topics from kafka-jumbo (Resolved)