Jump to content

Requests for comment/Shadow namespaces

From mediawiki.org
Request for comment (RFC)
Shadow namespaces
Component General
Creation date
Author(s) Legoktm, MZMcBride
Document status
See Phabricator.

This is a request for comments regarding implementing shadow namespaces, which refers to the concept where if a local page doesn't exist, it will be transparently fetched from a remote wiki.

For example, if Template:Hi does not exist on wiki A but it exists on the linked wiki B, then if {{Hi }} is added to a page on wiki A, then it will show Template:Hi from wiki B.

This is just like how InstantCommons and foreign file repos currently work (If File:Example.png does not exist on this wiki, but exists on Wikimedia Commons, the Commons wiki image is retrieved and used).



Currently we have $wgEnableScaryTranscluding in MediaWiki core. The variable name is considered apt by some, although the feature is used without problems on some mid-sized wikis such as MITRE wikis and wiki.wikimedia.it (example).

Other methods for interwiki transclusion on content pages include, for Wikisource, Extension:DoubleWiki and InterWikiTransclusion.js.

MediaWiki also has a shallow concept of shadow namespaces via ForeignFileRepos. Currently if you set up a foreign file repository, pages in the File namespace will pull from local version if it exists. If the local version does not exist, the foreign repo is queried. Foreign repos can be on the same wiki farm and using database connections or on remote wikis using the API.



The work here would be to extend and improve the current shadow namespace implementation. Instead of applying only to the File namespace, shadow namespaces could be implemented with the User, Template, Module, and Help namespaces.

The term "global" here means across all MediaWiki instances, so any modern MediaWiki installation should be able to reference any other wiki (possibly requiring installation of extensions/setting up configuration). In addition, wiki-farms would be able to designate one of their wikis to be used as the central repository if they choose not to use the default one.

$wgEnableScaryTranscluding would be deprecated and/or removed from MediaWiki core.

Invocations (i.e., {{foo}}) would try the local version first before trying the foreign repo equivalent. Links (i.e., [[bar]]) would do the same, with appropriate coloring.

First steps


First we would focus on the implementations that utilize "remote-parsing", where the text is parsed on the remote server, and the rendered HTML is displayed by the client wiki.

Open questions

  • Localization of templates and Scribunto/Lua modules
  • Namespacing: should we have a "Global templates" namespace (i.e.gtemplate:) or should it be transparent like InstantCommons?. Or both things?.



Shadow namespaces (global namespaces) are going, and could be, used to solve the following use-cases:

Code quality


As previously discussed, reducing duplication is the only proven method to share best practices and increase the quality of all the elements above across the board (also in big wikis), in addition to making usage broader and cheaper (for instance on small wikis which currently lack some features).

Making gadgets global is the other process which is widely recognised to improve code quality, a goal hotly perceived by some.

Implementation considerations




MediaWiki installations using global scripts will not keep pace with Wikimedia Foundation deployments, so we need a method to continue to support older versions (or maybe just versions that are still supported with code/security updates?).



Not a problem if the original page is in cc-by-sa and linked from the transcluding page. Hence not a problem for Wikimedia at all.

  • GFDL and CC-BY-SA are not recommended for software. Scribunto/Lua modules, JavaScript gadgets, CSS pages, and other content may be considered software.

Licensing might be a problem for some system administrators who misconfigure the shadow namespaces feature, for instance to include content from sources that specify a license with strict attribution requirements (e.g. GFDL) or a license incompatible with the target wiki (e.g. a NC license on a commercial wiki).


When content starts living outside of the local wiki, interwiki search suddenly becomes a lot more important.

Recent changes and company


To actually implement T66474, the location where the namespace actually resides should be entirely transparent to the user.

The most important consideration is whether changes on the source wiki are reflected on the local wiki. Lack of such a visibility on the transcluded content is usually considered a dealbreaker for anything content-related at least on bigger Wikimedia wikis, see for instance how Wikidata change propagation was handled. T91192 requests a similar system for Wikimedia Commons.

Usage tracking


For media on Commons, we have GlobalUsage tracking, however that's only Wikimedia-wide, and captures no usage information about non-Wikimedia Foundation installations using InstantCommons. This means that Commons administrators have no idea whether a file is actually used or not when doing destructive operations like renaming (without leaving a redirect behind) or deleting.

Cache invalidation


When a new version of a file is updated on Commons, it will automatically update on InstantCommons sites since images are hotlinked.[citation needed] However if a template is updated, it should cause HTMLCacheUpdate jobs to be queued so all pages are updated.

See also: Mentorship programs/Possible projects#Build an interwiki notifications framework and implement it for InstantCommons (phab:T48525)



Should we support chaining to multiple foreign repos? Basically, assuming we follow the w:en:zero one infinity rule with foreign repos, are we talking about one or infinity?

For example, imagine a wiki with two foreign repos configured. "Template:Baz" does not exist locally, but does exist on one of the foreign repos. Would there be some kind of fallback or order of precedence?

See also