Wikimedia Technology/Annual Plans/FY2019/TEC4: PHP7 Migration/Goals

From mediawiki.org

Program Goals and Status for FY18/19[edit]

  • Goal Owners: Mark Bergsma and Kate Chapman
  • Program Goals for FY18/19: At the conclusion of this program, Zend PHP7 will be the only PHP runtime supported or used in the Wikimedia Foundation production environment.
  • Annual Plan: PHP7 Migration

[edit]

Outcome 1 / Output 1.2[edit]

Zend PHP7 is the only PHP runtime in use in the WMF environment

All wikis, including wikitech/office/etc, are being run under the Zend PHP7 runtime

Dependencies on: Traffic & Performance teams; Primary team: Service Operations

Goal(s)[edit]

Allow MediaWiki requests to be served by PHP7 alongside HHVM

  • Install and configure php-fpm alongside HHVM on the application servers In progress In progress
  • Refactor Apache configuration to allow selection of PHP engine based on HTTP request In progress In progress
  • Stretch: Evaluate performance of PHP 7.0, 7.2 versus HHVM, and pick one. N Stalled
  • Stretch: Refactor the puppet module "mediawiki" classes to role/profile structure To do To do

Status[edit]

Note Note: July 2018

To do To do

Note Note: August 14, 2018

In progress In progress

Note Note: September 11, 2018

In progress In progress This goal is ongoing but slightly behind schedule due to high priority work on the data center switchover, as well as unexpected absence & vacations. It is though expected to complete shortly after EOQ. The stretch goal of evaluating PHP7 performance is currently stalled, as MediaWiki currently still does not pass all tests on PHP7.2


Outcome 1 / Output 1.2[edit]

Zend PHP7 is the only PHP runtime in use in the WMF environment

All wikis, including wikitech/office/etc, are being run under the Zend PHP7 runtime

Primary teams: Performance & Core Platform

Goal(s)[edit]

  • A sampling profiler that works under PHP7 has been identified and is prepared for use in the WMF production environment. If no appropriate profiler can be identified, then a statement of work for contractor effort to build one is prepared.

Status[edit]

Note Note: July 30, 2018

To do To do

Note Note: August 2018

In progress In progress Somewhat at risk due to Core Platform's other work load, might be at risk for completion this quarter.

Note Note: September 11, 2018

In progress In progress Somewhat at risk due to Core Platform's other work load, might be at risk for completion this quarter.

[edit]

Outcome 1 / Output 1.2[edit]

Zend PHP7 is the only PHP runtime in use in the WMF environment

All wikis, including wikitech/office/etc, are being run under the Zend PHP7 runtime

Dependencies on: Traffic, Performance, Core Platform teams; Primary team: Service Operations

Goal(s)[edit]

Ability to serve a % of production traffic from PHP7

  • Separate Traffic layer caches for PHP7/HHVM Incomplete Partially done
  • Allow directing users to PHP7 based on a cookie/header/beta feature Yes Done
  • Evaluate scalability and performance of php7 compared to HHVM (Performance/SRE) Yes Done
  • Package and install PHP 7.2 in production in place of PHP 7.0 Yes Done
  • Assist in the production deployment of a custom written PHP profiler (Performance) Yes Done

Status[edit]

Note Note: October 18, 2018

MW isn't running under PHP7 yet, so some of the goals are not yet In progress In progress, others should be done by end of October.

Note Note: November 14, 2018

PHP 7.2 has been packaged and installed in Beta, production deployment is imminent and is now Incomplete Partially done

Note Note: December 12, 2018

Profiler is tested, final code reviews for wider deploy are In progress In progress and everything else is Yes Done for this goal.

Note Note: January 9, 2019

Discussed that assisting in the production deployment has now been Yes Done by the Performance team

Outcome 1 / Output 1.2[edit]

Zend PHP7 is the only PHP runtime in use in the WMF environment

All wikis, including wikitech/office/etc, are being run under the Zend PHP7 runtime

Primary teams: Performance & Core Platform

Goal(s)[edit]

  • A sampling profiler that works under PHP7 has been identified and is prepared for use in the WMF production environment. If no appropriate profiler can be identified, then a statement of work for contractor effort to build one is prepared. Yes Done
  • Identify and address code issues and opportunities under PHP 7.2 Yes Done

Status[edit]

Note Note: July 30, 2018

To do To do

Note Note: August 2018

In progress In progress Somewhat at risk due to Core Platform's other work load, might be at risk for completion this quarter.

Note Note: September 11, 2018

In progress In progress Somewhat at risk due to Core Platform's other work load, might be at risk for completion this quarter.

Note Note: November 14, 2018

In progress In progress See https://phabricator.wikimedia.org/T205059

Note Note: December 12, 2018

Sampling profiler is Yes Done, deploy is tracked under output 1.1.

[edit]

Outcome 1 / Output 1.2[edit]

Zend PHP7 is the only PHP runtime in use in the WMF environment

All wikis, including wikitech/office/etc, are being run under the Zend PHP7 runtime

Dependencies on: Performance, Core Platform, Release Engineering team; Primary team: Service Operations

Goal(s)[edit]

Ramp-up serving traffic to PHP 7

  • Set up a beta feature to allow users to test PHP 7 Yes Done
  • Setup of the sampling/profiling pipeline (Performance, with assistance from SRE) In progress In progress
  • Set up curl connection pooling mechanism Yes Done
  • Optimize deployments for PHP7 Yes Done
    • Allow scap to invalidate the opcode cache Yes Done
    • [stretch] Allow fetching the database configuration from etcd instead of files in order to reduce the cache resets
  • Set up A/B testing mechanism for PHP7, ramp up traffic to 20% of users In progress In progress

Status[edit]

To do To do January 2019

Discussed...

Note Note: February 13, 2019

  • PHP7 has been deployed as a Beta feature, and newly found bugs due to beta testing are being investigated. A curl connection pooling mechanism has been implemented. Scap invalidation of the opcode cache is nearly done. On track and In progress In progress

Note Note: March 27, 2019

  • Setup of the sampling/profiling pipeline is still In progress In progress and will go into Q4.
  • Stretch goal of allowing fetching the database configuration from etcd is still In progress In progress and will continue into Q4
  • A/B testing mechanism for PHP7 is at about 1% and will continue to be In progress In progress in Q4

[edit]

Outcome 1 / Output 1.2 (SRE)[edit]

Zend PHP7 is the only PHP runtime in use in the WMF environment

All wikis, including wikitech/office/etc, are being run under the Zend PHP7 runtime

Dependancies on: ___________

Goal(s)[edit]

  • Complete the transition to PHP 7 in production
    • Move all application server & API traffic to PHP 7
    • Move maintenance scripts to PHP 7 (stretch)
    • Move jobrunners to PHP 7 (stretch)
    • Remove HHVM from production (stretch)

Status[edit]

Note Note: May 8, 2019

  • Transition to PHP is at 5% anon, moving traffic over now, but it's taking time to do this smoothly, so In progress In progress
  • Stretch goals might not be able to be done this quarter.

Note Note: June 13, 2019

  • A portion of this is N Blocked on emergent bugs /issues (at 20% right now) and will continue into next week. MOve maint scripts and job runners are In progress In progress but the removing of HHVM won't be done until next FY.


Outcome 1 / Output 1.2 (Performance)[edit]

Zend PHP7 is the only PHP runtime in use in the WMF environment

All wikis, including wikitech/office/etc, are being run under the Zend PHP7 runtime

Dependencies on: SRE

Goal(s)[edit]

  • Support with rollout of PHP7


Status[edit]

Note Note: May 8, 2019

  • This is In progress In progress and going well.

Note Note: June 27, 2019

  • This is In progress In progress and will finish up in the next quarter.