Mobile web/MobileFrontend deployment

From mediawiki.org

Status[edit]

Target Ruby Gateway Mobile Frontend
Deployed Yes Yes
Default for all m. views No Yes
Serving "Mobile View" link No Yes
Squid redirect target No Yes
Default for all non Wikipedia projects No No (Unless you add the mobile tag)
  • Mobile frontend end has been deployed to production for in house testing and is available through both a link in the footer and with the useFormat=mobile url parameter.
  • SQUID acls have been deployed (minus nokia) and were serving 302's to 100% of our web traffic. We ran a blog post earlier in the week to warn people about the change.
  • Varnish cluster is in final prep scheduled to finish Mon/Tues.
  • Opt in launch blog post TBD.

Schedule[edit]

Current Requirements for Trial Deployment[edit]

Backlog[edit]

Task # Description Owner Status
1 WURFL support in MW Memcache interface Patrick Done
2 Squid mobile redirector Patrick Done
3 Trial Rollout Asher Done
4 Mobile Varnish cluster Asher Done
5 Varnish multicast cache purge support Asher Done
6 Varnish udp2log patch Patrick Done
7 Install Mobile frontend in production Patrick Done
8 Proxy frontend on Ruby-based mobile servers Asher Done
9 Create deployment packages for #2, #6 Asher Done
11 Enable ACLs on production Squids Asher Done (note: Nokia ACL pending resolution of 29505
12 Enable Mobile frontend in production Patrick Done

Current Requirements for Prod Deployment[edit]

Task # Description Owner Status
1 Order LVS hardware Asher Done
2 Rack and release hardware Asher Done
3 Open Bugs Patrick Bugs


Detail[edit]

WURFL support in MW Memcache interface[edit]
  • Need to update php wurfl api to use mediawiki's memcache interface -- Blocker
    • Converting from using the php memcache extension to the built-in mediawiki memcache interface for the WURFL php api.
    • Special:Code/MediaWiki/91768
Squid mobile redirector[edit]
  • Squid mobile redirector - the squid config (acl's for mobile device and opt-out cookie detection) is finalized and tested with the redirect logic implemented in perl. The perl needs to be ported to C and ensured to be fast, ram efficient, and leak free. Simple but due to risk involved in impacting nearly all wikipedia requests, may need a full day or more of testing.
Trial Rollout[edit]
  • How do we want a trial version of the new mobile site to be accessed? x% of requests to the current mobile vip, or new vips with test names (i.e. en.newm.wikipedia.org)
  • If we want host names to stay the same and run a cookie based trial (i.e. users transparently get the new site when going to en.m.wikipedia.org if I have the test cookie set) :
    • We need a proxy sitting in front of the ruby mobile servers (such as nginx or haproxy) on port 80
    • Reconfigure ruby apaches to port 81
    • Proxy configured with two backend pools - one is just localhost:81, the other is the individual new-mobile varnish hosts
    • Dispatch based on cookie
    • This isn't complex but if the new mobile varnish cluster is in eqiad, it might add up to 120ms of latency from multiple back and forth hops across colos
  • New vip - what do we name the base? *.nm.wikipedia.org?
    • LVS config for the new vip and varnish hosts behind it (need Ryan's help)
    • DNS for test versions of all wikipedia sites
    • Deploy the new mobile redirect logic to the squids, with the C redirector modified to use the test domain name
    • Add a squid acl to only invoke the redirector if the cookie is set (this should be possible but need to verify)
    • In either case, need to update mobile vcl to rewrite chosen hostnames back to $lang.wikipedia.org to be supported transparently by apache / mediawiki.
Mobile Varnish cluster[edit]
  • Mobile Varnish cluster - waiting to hear from Mark if we can piggyback on and expand the bits cluster
    • If Yes: Need to reconfigure varnish to use all available ram, likely need to put nginx in front, mobile vcl needs to be updated to play nicely with the current version (don't execute at all for bits requests)
    • If No (this might actually be better): we can use the 5 mobile servers ordered in Ashburn for Varnish (may need RobH to complete server builds)
      • Need nginx with consistent uri hashing - that module probably needs to be built from source and packaged into a dpkg for deployment
      • Puppet rules to build mobile cache servers (nginx w/hash module on 80 and varnish on 81 with/mobile vcl)
      • Nagios monitoring for varnish on a non-standard port?
Varnish multicast cache purge support[edit]
Varnish udp2log patch[edit]
  • Logging - we still need Varnish log2udp support via a logging daemon
    • What would be a good TTL setting for multicast? (Right now I defaulted to 1 second which is the same as it is in squid.)
    • Implemented and is in testing by Patrick. Tfinc 19:17, 7 July 2011 (UTC)[reply]
  • TTL is based on hops, not time - might need to be > 1 for running in the remote colo.
  • Date/time format needs to be changed. It's currently in the apache standard, split over two fields. Needs to be like our squids.
    • Example: 2011-08-08T23:51:35.203
    • Hopefully a one line change, but requires making a new patch, building a new varnish package, etc.
Install Mobile frontend in production[edit]
  • Install of mobile extension to production mediawiki cluster - this needs to be done as part of the mediawiki app install process
    • Need help of someone familiar with deploying new mediawiki extensions to prod
Enable Mobile frontend in production[edit]
  • It's installed and testable via a query param but now needs to be enabled based on the presence of the X-Device header.
  • Need a page to set the opt-in cookie. Does this need to go on the RoR mobile site? And an opt-out page via the extension?