Wikimedia Technology/Annual Plans/FY2019/TEC4: PHP7 Migration/Goals

Program Goals and Status for FY18/19

TEC4 PHP Migration

• Goal Owners: Mark Bergsma and Kate Chapman
• Program Goals for FY18/19: At the conclusion of this program, Zend PHP7 will be the only PHP runtime supported or used in the Wikimedia Foundation production environment.
• Annual Plan: PHP7 Migration
  • Primary Goal is Knowledge as a Service: Evolve our systems and structures
  • Tech Goal: Sustaining

Q1 Goals

Outcome 1 / Output 1.2

Zend PHP7 is the only PHP runtime in use in the WMF environment

All wikis, including wikitech/office/etc, are being run under the Zend PHP7 runtime

Dependencies on: Traffic & Performance teams; Primary team: Service Operations

Goal(s)

Allow MediaWiki requests to be served by PHP7 alongside HHVM

• Install and configure php-fpm alongside HHVM on the application servers In progress
• Refactor Apache configuration to allow selection of PHP engine based on HTTP request In progress
• Stretch: Evaluate performance of PHP 7.0, 7.2 versus HHVM, and pick one. N Stalled
• Stretch: Refactor the puppet module "mediawiki" classes to role/profile structure To do

Status

 Note: July 2018

To do

 Note: August 14, 2018

In progress

 Note: September 11, 2018

In progress This goal is ongoing but slightly behind schedule due to high priority work on the data center switchover, as well as unexpected absence & vacations. It is though expected to complete shortly after EOQ. The stretch goal of evaluating PHP7 performance is currently stalled, as MediaWiki currently still does not pass all tests on PHP7.2

Outcome 1 / Output 1.2

Zend PHP7 is the only PHP runtime in use in the WMF environment

All wikis, including wikitech/office/etc, are being run under the Zend PHP7 runtime

Primary teams: Performance & Core Platform

Goal(s)

• A sampling profiler that works under PHP7 has been identified and is prepared for use in the WMF production environment. If no appropriate profiler can be identified, then a statement of work for contractor effort to build one is prepared.

Status

 Note: July 30, 2018

To do

 Note: August 2018

In progress Somewhat at risk due to Core Platform's other work load, might be at risk for completion this quarter.

 Note: September 11, 2018

In progress Somewhat at risk due to Core Platform's other work load, might be at risk for completion this quarter.

Q2 Goals

Outcome 1 / Output 1.2

Zend PHP7 is the only PHP runtime in use in the WMF environment

All wikis, including wikitech/office/etc, are being run under the Zend PHP7 runtime

Dependencies on: Traffic, Performance, Core Platform teams; Primary team: Service Operations

Goal(s)

Ability to serve a % of production traffic from PHP7

• Separate Traffic layer caches for PHP7/HHVM Partially done
• Allow directing users to PHP7 based on a cookie/header/beta feature  Done
• Evaluate scalability and performance of php7 compared to HHVM (Performance/SRE)  Done
• Package and install PHP 7.2 in production in place of PHP 7.0  Done
• Assist in the production deployment of a custom written PHP profiler (Performance)  Done

Status

 Note: October 18, 2018

MW isn't running under PHP7 yet, so some of the goals are not yet In progress, others should be done by end of October.

 Note: November 14, 2018

PHP 7.2 has been packaged and installed in Beta, production deployment is imminent and is now Partially done

 Note: December 12, 2018

Profiler is tested, final code reviews for wider deploy are In progress and everything else is  Done for this goal.

 Note: January 9, 2019

Discussed that assisting in the production deployment has now been  Done by the Performance team

Outcome 1 / Output 1.2

Zend PHP7 is the only PHP runtime in use in the WMF environment

All wikis, including wikitech/office/etc, are being run under the Zend PHP7 runtime

Primary teams: Performance & Core Platform

Goal(s)

• A sampling profiler that works under PHP7 has been identified and is prepared for use in the WMF production environment. If no appropriate profiler can be identified, then a statement of work for contractor effort to build one is prepared.  Done
• Identify and address code issues and opportunities under PHP 7.2  Done

Status

 Note: July 30, 2018

To do

 Note: August 2018

In progress Somewhat at risk due to Core Platform's other work load, might be at risk for completion this quarter.

 Note: September 11, 2018

In progress Somewhat at risk due to Core Platform's other work load, might be at risk for completion this quarter.

 Note: November 14, 2018

In progress See https://phabricator.wikimedia.org/T205059

 Note: December 12, 2018

Sampling profiler is  Done, deploy is tracked under output 1.1.

Q3 Goals

Outcome 1 / Output 1.2

Zend PHP7 is the only PHP runtime in use in the WMF environment

All wikis, including wikitech/office/etc, are being run under the Zend PHP7 runtime

Dependencies on: Performance, Core Platform, Release Engineering team; Primary team: Service Operations

Goal(s)

Ramp-up serving traffic to PHP 7

• Set up a beta feature to allow users to test PHP 7  Done
• Setup of the sampling/profiling pipeline (Performance, with assistance from SRE) In progress
• Set up curl connection pooling mechanism  Done
• Optimize deployments for PHP7  Done
  • Allow scap to invalidate the opcode cache  Done
  • [stretch] Allow fetching the database configuration from etcd instead of files in order to reduce the cache resets
• Set up A/B testing mechanism for PHP7, ramp up traffic to 20% of users In progress

Status

To do January 2019

Discussed...

 Note: February 13, 2019

• PHP7 has been deployed as a Beta feature, and newly found bugs due to beta testing are being investigated. A curl connection pooling mechanism has been implemented. Scap invalidation of the opcode cache is nearly done. On track and In progress

 Note: March 27, 2019

• Setup of the sampling/profiling pipeline is still In progress and will go into Q4.
• Stretch goal of allowing fetching the database configuration from etcd is still In progress and will continue into Q4
• A/B testing mechanism for PHP7 is at about 1% and will continue to be In progress in Q4

Q4 Goals

Outcome 1 / Output 1.2 (SRE)

Zend PHP7 is the only PHP runtime in use in the WMF environment

All wikis, including wikitech/office/etc, are being run under the Zend PHP7 runtime

Dependancies on: ___________

Goal(s)

• Complete the transition to PHP 7 in production
  • Move all application server & API traffic to PHP 7
  • Move maintenance scripts to PHP 7 (stretch)
  • Move jobrunners to PHP 7 (stretch)
  • Remove HHVM from production (stretch)

Status

 Note: May 8, 2019

• Transition to PHP is at 5% anon, moving traffic over now, but it's taking time to do this smoothly, so In progress
• Stretch goals might not be able to be done this quarter.

 Note: June 13, 2019

• A portion of this is N Blocked on emergent bugs /issues (at 20% right now) and will continue into next week. MOve maint scripts and job runners are In progress but the removing of HHVM won't be done until next FY.

Outcome 1 / Output 1.2 (Performance)

Zend PHP7 is the only PHP runtime in use in the WMF environment

All wikis, including wikitech/office/etc, are being run under the Zend PHP7 runtime

Dependencies on: SRE

Goal(s)

• Support with rollout of PHP7

Status

 Note: May 8, 2019

• This is In progress and going well.

 Note: June 27, 2019

• This is In progress and will finish up in the next quarter.