Requests for comment/Shadow namespaces

This is a request for comments regarding implementing shadow namespaces, which refers to the concept where if a local page doesn't exist, it will be transparently fetched from a remote wiki, like how InstantCommons and foreign file repos currently work.

Background
Currently we have in MediaWiki core. The variable name is considered apt by some, although the feature is used without problems on some mid-sized wikis such as MITRE wikis and wiki.wikimedia.it (example).

Other methods for interwiki transclusion on content pages include, for Wikisource, Extension:DoubleWiki and InterWikiTransclusion.js.

MediaWiki also has a shallow concept of shadow namespaces via ForeignFileRepos. Currently if you set up a foreign file repository, pages in the File namespace will pull from local version if it exists. If the local version does not exist, the foreign repo is queried. Foreign repos can be on the same wiki farm and using database connections or on remote wikis using the API.

Proposal
The work here would be to extend and improve the current shadow namespace implementation. Instead of applying only to the File namespace, shadow namespaces could be implemented with the User, Template, Module, and Help namespaces.

The term "global" here means across all MediaWiki instances, so any modern MediaWiki installation should be able to reference any other wiki (possibly requiring installation of extensions/setting up configuration). In addition, wiki-farms would be able to designate one of their wikis to be used as the central repository if they choose not to use the default one.

would be deprecated and/or removed from MediaWiki core.

Invocations (i.e., ) would try the local version first before trying the foreign repo equivalent. Links (i.e., ) would do the same, with appropriate coloring.

First steps
First we would focus on the implementations that utilize "remote-parsing", where the text is parsed on the remote server, and the rendered HTML is displayed by the client wiki.
 * Identify implementation differences between ForeignFileRepo and GlobalUserPage, and begin to reconcile them
 * Switch ForeignFileRepo to use api.php?action=parse instead of index.php?action=render
 * action=parse needs to support absolute URLs
 * Add top icons to to GlobalUserPages
 * GlobalUserPage should use tabs like foreign file repos
 * Start abstracting ForeignFileRepo remote-transcluding code into core to handle other namespaces

Open questions

 * Localization of templates/modules
 * Namespacing - should we have a "Global templates" namesapace or should it be transparent like InstantCommons?
 * GlobalUserPage had a hacky opt-out with / tags, which ended up causing other problems and generally wasn't great. Can we get rid of it?

Applications
Shadow namespaces could be used to solve the following use-cases:


 * Global Scribunto/Lua modules; Module namespace; T41610
 * Global templates; Template namespace; T6547
 * Global user pages; User namespace; T16759
 * Global help pages; Help namespace; T14306
 * would allow sharing of global styles for T90914, T112991, and T105845 in general

Code quality
As previously discussed, reducing duplication is the only proven method to share best practices and increase the quality of all the elements above across the board (also in big wikis), in addition to making usage broader and cheaper (for instance on small wikis which currently lack some features).

Making gadgets global is the other process which is widely recognised to improve code quality, a goal hotly perceived by some.

Compatibility
MediaWiki installations using global scripts will not keep pace with Wikimedia Foundation deployments, so we need a method to continue to support older versions (or maybe just versions that are still supported with code/security updates?).

Licensing
Not a problem if the original page is in cc-by-sa and linked from the transcluding page. Hence not a problem for Wikimedia at all.

Might be a problem for some sysadmin who happened to misconfigure the feature, for instance to transclude from sources which use a license with strict attribution requirements (e.g. GFDL) or a license incompatible with the transcluding wiki (e.g. a NC license on a commercial wiki).

Either way, not an issue MediaWiki can or should deal with.

Search
When content starts living outside of the local wiki, interwiki search suddenly becomes a lot more important.

Recent changes and company
To actually implement T66474, the location where the namespace actually resides should be entirely transparent to the user.

The most important consideration is whether changes on the source wiki are reflected on the local wiki. Lack of such a visibility on the transcluded content is usually considered a dealbreaker for anything content-related at least on bigger Wikimedia wikis, see for instance how Wikidata change propagation was handled. T91192 requests a similar system for Wikimedia Commons.

Usage tracking
For media on Commons, we have GlobalUsage tracking, however that's only Wikimedia-wide, and captures no usage information about non-Wikimedia Foundation installations using InstantCommons. This means that Commons administrators have no idea whether a file is actually used or not when doing destructive operations like renaming (without leaving a redirect behind) or deleting.

Cache invalidation
When a new version of a file is updated on Commons, it will automatically update on InstantCommons sites since images are hotlinked.[citation needed] However if a template is updated, it should cause HTMLCacheUpdate jobs to be queued so all pages are updated.

See also: Mentorship programs/Possible projects (T48525)

Chaining
Should we support chaining to multiple foreign repos? Basically, assuming we follow the zero one infinity rule with foreign repos, are we talking about one or infinity?

For example, imagine a wiki with two foreign repos configured. "Template:Baz" does not exist locally, but does exist on one of the foreign repos. Would there be some kind of fallback or order of precedence?