User:Duesentrieb/Fix interwiki

This page describes the evil master plan to get rid of the Interwiki class (or at least render it harmless).

Purpose:

• allow us to use the more flexible Sites system instead of the crusty Interwiki system
• allow us to use Sites and Interwiki side by side, based on the same data
• Preserve the legacy interface for interwiki links (static methods in Interwiki)
• make Sites (at least as) performant as the current multi-cache hodge-podge implementation of Interwiki.
• Maintain interwiki mappings / site definitions in files, not in the database. It's configuration, not state.
• Make WikiMap consistent with Interwiki and SiteLookup

Outline:

• Create a new interface InterwikiLookup with all the public methods from Interwiki
• Create HodgePodgeInterwiki implements InterwikiLookup; implement it using the code currently in Interwiki. "hodge-podge" because is implements several storage backends (sql, cdb, ...). The code should be changed as little as possible, only removing replacing access to global state with injection.
• Make the public static methods in Interwiki delegate to a singleton instance of InterwikiLookup, remove everything else from the class.
• Create an adapter, SiteLookupInterwikiLookup, implementing InterwikiLookup based on a SiteLookup.
• Migrate usages of SiteStore to SiteLookup. Drop all code that modifies site definitions. Site definitions are config, and config is not mutable by the application.
• FileSiteLookup implements a SiteLookup that will simply load site definitions from a list of local files
  • support at least JSON (easy to maintain) and PHP (code, not serialized data; fast with accelerator cache). Go by file extension.
• Add methods for fetching Site objects for a group to SiteLookup
• Make CachingSiteStore (resp CachingSiteLookup) cache individual groups. Use "siblings" group for sister projects of the same language.
• Drop SiteStore (in favor of SiteLookup), deprecate SiteSQLStore (we need it in read-only mode at least, for B/C).
• Make an export script that can generate JSON site definition files:
  • Export from a SiteLookup or InterwikiLookup (needs an adapter that implements a SiteLookup based on an InterwikiLookup)
  • Export all, or a list of groups
  • Export only the ones that differ from the ones defined in a list of given files. This can be used to generate files that contain only the local overrides / additions to a common list of site definitions.

• Make a maintenance script that generates a PHP file with site definitions from a list of JSON (and PHP) files.
• Map WikiMap look up wiki info in a SiteLookup before (after?) checking in $wgConf (optional?).

Configuration:

• InterwikiLookup implementation to use. Default: SiteLookupInterwikiLookup
  • For HodgePodgeInterwiki, use the old interwiki settings controlling CDB usage etc.
• SiteLookup implementation to use. Default: FileSiteLookup
  • per default, read DefaultInterwiki.json (maintained in git) and LocalInterwiki.json (shipped empty).
  • on the Wikimedia cluster, use PHP files for speed. One common file, one per group, and one for local overrides per wiki.
• Caching:
  • which cache (possibly none for PHP files)
  • duration
  • groups to cache separately (all?)