- See HHVM/About for a shorter description, and the December 2014 blog post "How we made editing Wikipedia twice as fast" for more background on HHVM, its benefits, and a look back on the deployment work.
HHVM is a PHP interpreter tool (concretely a virtual machine that compiles PHP bytecode to native instructions at runtime) that was deployed on Wikimedia servers in 2014, decreasing the loading time for: any page you view while logged in, and for saving pages that you've edited whether you are logged in or not.
- Rollout is ongoing. Plan is to be 100% on app servers by Christmas, barring unforeseen issues.
- Tim Starling and Giuseppe Lavagetto isolated and resolved an issue with OAuth: Authorization header was not available to HHVM via apache_request_headers() due to mod_proxy_fcgi bug. Resolved by reinjecting the header via a SetEnvIf directive in the Apache config. See https://wikitech.wikimedia.org/wiki/Incident_documentation/20141126-oauth.
- Tim Starling isolated and fixed API throughput issue (phab:T758). Shelling out to tidy didn’t perform well on HHVM. Solved by making the tidy extension for PHP compatible with HHVM (at least for essential functionality) and using that instead. Needs to be tested and fully deployed.
HHVM is a new way for websites to run MediaWiki (written in PHP). It is sometimes known by its full name HipHop Virtual Machine and has a just-in-time compiler (JIT). HHVM should speed up page viewing for logged-in users and also speed up the saving of edited pages for all users.
This page is about Wikimedia-sponsored work on HipHop support in MediaWiki, and its deployment to Wikimedia production wikis.
Historically, the HipHop compiler was a project by Facebook which involved compiling PHP code into C++ for purposes of speeding up the language. Facebook has since abandoned this project, and now their development efforts are focused on HHVM itself instead
Here is the general plan for deploying HHVM to the production cluster:
- Deployment to Beta Cluster (currently ongoing work, in parallel with other work) Done
- Week of July 21: deployment to a few job runners in production Done
- Deploy to test.wikipedia.org application server Done
- Deploy Varnish module allowing partial deployment to a fraction of application servers Done
- Limited deployment to small number of application servers Done
- Ramp up deployment to more application servers until most servers use HHVM Done
- Deploy to remainder of services Done
It is a well-studied phenomenon that even small delays in response time (e.g. half of a second) can result in sharp declines in web user retention. As a result, popular websites such as Google and Facebook invest heavily in site performance initiatives, and partially as a result, remain popular. Formerly popular sites (such as Friendster) suffered due to lack of attention to these issues. Wikipedia and its sister projects must remain usable and responsive in order for the movement to sustain its mission.
Facebook, as a big user of PHP, has recognized this problem, and invested heavily in a solution: HHVM, a virtual machine that compiles PHP bytecode to native instructions at runtime, the same strategy used by Java and C# to achieve their speed advantages. We're quite confident that this will result in big performance improvements on our sites as well.
What does HipHop do for our end users?
MediaWiki is written in PHP, a language that is interpreted at run-time. The overhead of running this PHP code every time someone views a page necessitates the usage of caching servers, running software such as Varnish, which cache the HTML generated by running this PHP, so that the PHP does not have to run every time a page is viewed. These caches only serve users that are not logged in. Actions which are not affected by the cache, and therefore are affected by the run time of PHP code, include:
- Any page you view while logged in.
- Saving pages that you've edited, whether you are logged in or not.
Therefore, any action we can take to reduce the time it takes for MediaWiki's PHP code will therefore also decrease the loading times of our site for all of our logged in users and anyone who edits anonymously.
HipHop was written to be a faster, more efficient PHP interpreter than our current interpreter (Zend). It is our hope that by implementing HipHop as a replacement for Zend, our users will notice a tangible increase in the performance of our sites.
How does our development work on HipHop affect MediaWiki developers?
In our initial sprint of work, due to be finished at the end of March 2014, we hope to make it so that anyone can elect to use HipHop on Beta Labs instead of Zend. This will be on a totally opt-in basis which can be disabled at any time. This will allow the MediaWiki Core team to gauge the performance of HipHop against that of Zend directly using our current test infrastructure, instead of just estimating theoretical performance increases. It will also create a development environment that will help us see how much work is needed to make HipHop compatible with MediaWiki, and as such let us create an estimate for how long it will take us to get HipHop live on production as a full replacement for Zend.
For other MediaWiki developers, the consequence of HipHop being deployed in this manner is that if they are using the Beta Cluster as a test environment, they will find it trivial to test how their patches perform using HipHop instead of Zend if they wish to. However, to minimise the disruption of our work, the opt-in nature of the infrastructure will allow developers will be able to continue to develop totally agnostic of the future HipHop migration if they wish to do so.
- HHVM/Vagrant: Setting up HHVM in MediaWiki-Vagrant
- Ori presenting at @Scale about migrating to HHVM
- HHVM in production: what that means for Wikimedia developers
References and footnotes
- "Bing and Google Agree: Slow Pages Lose Users" - Brady Forrest - O'Reilly Radar
- Greg Linden's blog: "Marissa Mayer at Web 2.0" - Marissa Mayer pointed out that a change from 0.4 seconds to 0.9 seconds in response time from Google caused a 20% drop in revenue and traffic.
- "Wallflower at the Web Party", New York Times, October 15, 2006. Quote: "Kent Lindstrom, now president of Friendster, said the board failed to address technical issues that caused the company’s overwhelmed Web site to become slower."
- By definition, users that are logged in cannot be served pages from a static cache, as the page served to them must include user-specific HTML such as their username at the top right of the page. This, unfortunately, creates a situation where simply logging in causes a tangible decrease in how well our sites perform for you.