Talk:Requests for comment/Configuration database

(no subject)

Some things out of from my head:

• How to structure configuration options? Relations between them? Related options displayed together?
• Some configuration options have equivalent user preference.

– Nikerabbit 15:35, 25 August 2009 (UTC)

Wikia code

You may want to check out our code that does a similar thing: https://svn.wikia-code.com/wikia/trunk/extensions/wikia/WikiFactory/

--Emil Wikia 16:48, 26 August 2009 (UTC)

Singletons

Singletons are really just globals in disguise. I don't really like them, but they are probably unavoidable without significantly rewriting MediaWiki. -- Bryan 145.94.184.252 13:43, 5 July 2011 (UTC)

Singletons do provide a few significant advantages over a set of globals ($wgFoo, $wgBar) or a global array ($wg['Foo'], $wg['Bar']). Mainly:

• Related functions can be stored and referenced together by being in the same class (rather than being put in random utility classes or global functions)
• Input/Output Read/Write is controlled rather than on the loose. A get/set function (either one-for-all or more specifically) can limit/verify/validate the input (ie. throw exception or correct value when it doesn't validate, instead of having to sanity check every time we access the configuration variable since it could be set to anything...)
  • likewise the read function is able to limit reading values based on the context or moment in execution time (ie. before hook X is fired, or if variable Y is set to Z)
• They are one step closer to an even better system.

I agree though that Singletons, when used as such, are much like globals. A better way would perhaps be a static cache of instances, ie. like Conf::newFromId('enwiki') instead of Conf::singleton() which would either load it or get it from static cache, where Conf::get('foo') would be like Conf::newFromId(Conf::defaultId)->reallyGet using Conf::defaultId which would be set during initialization of the wiki/run. Krinkle 17:15, 5 July 2011 (UTC)

Yes they're globals in disguise, but for something like this you only need a single instance of the config object. Unless you think it's a good idea to have people construct config objects all over the place ;-)

Actually: expanding on the idea of "Contexts" a bit more...we could possibly move in the direction we did with RequestContext. Something like a WikiContext could be easily passed/made available as a member and has access to the configuration. ^demon 17:30, 5 July 2011 (UTC)

I was indeed thinking more along the way of a WikiContext, which should contain the configuration object, among other things like a RequestContext, getDatabase() function, getRepoGroup() function and things like that. -- Bryan (talk|commons) 12:44, 13 July 2011 (UTC)

Well a WikiContext wouldn't necessarily have a RequestContext, but a RequestContext should always have a WikiContext. But otherwise I completely agree. ^demon 18:10, 13 July 2011 (UTC)

Yeah, maintenance script for instance. Krinkle 11:10, 16 July 2011 (UTC)

current plans

See the Platform team's planned activities for the next 12 months:

Configuration database - Revamp our configuration management storage - remove global variables, stop proliferation of new ones, provide extensions means of not adding to technical debt, expand unit-testability

Chad Horohoe is considering " database for normal wikis. Prolly something like CDB for wmf." Timing: "Nothing firm yet. I think we were planning to start it sometime in August/September." Sumana Harihareswara, Wikimedia Foundation Engineering Community Manager (talk) 18:33, 5 July 2012 (UTC)

Notes from Architecture meetings/Wikimania 2013

Below are the raw notes from our discussion at Wikimania 2013:

• consider moving defaults to a serialized format?
  • nasty case -- default is dynamic based on other configuration (especially something like $wgServer, which is initialized rather late during setup)
  • default installs should be workable in the core database only (no FS access), but we'll want CDB files or something for high-scale rollouts like Wikimedia!
  • nasty case -- the default might be a list, to which the actual config wants to add (or replace it). This requires the defaults to actually be applied in LocalSettings.php
• kill the global namespace vars? or keep them for back-compat? or....
• ..... no cookie-licking! let's kick the RFC into shape to where anybody who has the time can implement it (even if outside wmf)
• should we start on new config bits only? or include the existing globals so we have a migration path for them, and then deprecate/kill later?
• we should be opportunistic... maybe dovetail with some project that can use the config db and push it into core starting with that, then move things in?
• startup performance & access performance are REALLY IMPORTANT ON THIS! memory usage may be important as well
  • ^- pluggable storage, not just arrays. Need to hack out some specs for thre quirements on this. <- may need 'hybrid' storage, such as CDB on filesystem for most Wikimedia stuff but a few settings in local database
  • what config can we safely expose via api? (parsoid can use this etc)
  • note that we have maint scripts for getting config info (eg for admin scripts to use), this sort of model should still work
    • avoid reading the config databases directly from other languages (like a python script), read it through MediaWiki so its pluggable storage always works
    • note: avoid turning into a general key-value store. try to keep it configuration-related!
    • consider the config _store_ as a prereq for config _ui_. (though for example wikia kinda cheated, and there's the old Configure extension... but these are scary :D)
  • EventLogging (and several other extensions) use memcached for managing JSON blobs of configuration across wikis.
• avoid access to settings as global state, instead pass individual settings or a task-specifc settings object. Any code accessing a global setting/singleton essentially becomes impossible to unit-test. RobLa-WMF (talk) 04:04, 27 October 2013 (UTC)

Based on my current ideas, I think the answers to your questions are as follows:

• I think we should use something like JSON. Then it's human readable, easily parsed in PHP, and not something scary like serialized PHP.
• I think we'll need a back-compat mode for awhile that allows importing/exporting of globals. We've got over 10 years of extension history that we don't want to gratuitously break.
• Agreed. I think breaking it into concrete steps/phases make it way more likely for this to succeed. For example, here I went ahead and started making configuration more portable.
• I think it should be opt in, as some config is going to be easier to migrate than others. This is easier with back-compat I mentioned above
• Once the initial framework is in place, it'll be very easy for any person with commit access to migrate other settings--99% of them will just be copy+replace usages.
• Agree with this
  • Yeah.
  • I'm paranoid and always figured we'd do whitelisting.
  • especially on backend being pre-req for UI. In theory, once the backend is stable the UI a separate project to start building.
• I think it makes the most sense to be in Contexts, see my gerrit change linked above ^demon^{[omg plz]} 05:36, 30 October 2013 (UTC) 05:36, 30 October 2013 (UTC)

Debugging settings

I was about to say, "In particular, it might be helpful for debugging settings to remain in the config file. Suppose you install an experimental extension that crashes the wiki. To fix it, you have to comment out the require_once line that installs the extension, then go to the config screen to change the debugging settings, then re-enable the extension. This is a three-step process which under the current setup only takes one step." However, it seems to me that on test wikis (as opposed to production wikis), people will typically keep the debugging settings switched on. Leucosticte (talk) 01:26, 5 November 2013 (UTC)

Storing setting on wiki pages

Shameless ContentHandler plug: settings could be stored as structured data (JSON blobs) on wiki pages, using specialized handlers for rendering and editing, similar to how Wikidata does it. Caching would be applied as before.

Config pages would live in a separate namespace, and extensions could have their own pages there (and may perhaps even supply their own UI for complex settings).

Advantages:

• no storage layer code to write.
• versioning/diffing and logging just work.
• revert/undo just work.
• import/export just work.
• permission system (on the per page level) just works.

Disadvanatges:

• can't really hide settings. Read permissions could be set per page, but might still be exposed in dumps, etc.
• accessing config pages from another wiki may not be trivial. Duesentrieb ⇌ 21:20, 6 November 2013 (UTC)