Wikimedia Developer Summit/2017/Integrating MediaWiki (and other services) with dynamic configuration

From mediawiki.org

Session Overview[edit]

Title: Integrating MediaWiki (and other services) with dynamic configuration

Day & Time: Tuesday Jan. 10th, 11:00 am – 12:10 pm (19:00 – 20:10 UTC)

Room: Hawthorne

Phabricator Task Link: https://phabricator.wikimedia.org/T149617

Facilitator(s): paravoid, greg-g

Note-Taker(s): Volans, bd808

Remote Moderator:

Advocate: 

Session Summary[edit]

Session recording
Session slides


Purpose[edit]

Find a unified, simple and reliable way to manage the dynamic configuration (live state) of services in WMF.

Agenda[edit]

40 minutes presentation, 30 minutes discussions and conclusions

Style[edit]

Consensus: coming to agreement on one solution

Discussion Topics[edit]

The current way of managing the state of a cluster in WMF is via it's configuration and each change requires a change in the configuration repository (Puppet).

The live state of a service or cluster should be managed separately in a simple and reliable way that allows to change the state quickly, reliably and without running Puppet.

The main two categories of live state are:

  • Service discovery (SOA address book)
  • State management

Two solutions were proposed, based on the different use case:

  • DNS for simple data structures and service discovery through TXT or URI records
  • Confd for more complex data structure or integrating an etcd client in the application layer itself

For the full session notes see the detailed notes.

Action Items

  • Stop relying on a configuration repository to store the live state of clusters, it needs to be treated separately.
  • Use DNS for service location/discovery, and simple data structures, but has some limitations.
  • Use Confd with templates and scripts to manage more complex data structures.
  • For the MediaWiki specific case, there are a bunch of options:
    • generate a JSON file that is read by the application, parsed by MediaWiki and cached into APC.
    • generate a PHP file that will be included directly by the application.
  • Add safe measures to ensure that the configuration is consistent across the cluster and no hosts have a stale one.
Retrieved from "https://www.mediawiki.org/w/index.php?title=Wikimedia_Developer_Summit/2017/Integrating_MediaWiki_(and_other_services)_with_dynamic_configuration&oldid=2379029"