Global templates/Transition

From mediawiki.org

The implementation of Global templates and modules has two sides:

  1. The creation of the repository and the infrastructure that makes it possible. This is mostly done by software developers, by planning the architecture and then writing PHP, JavaScript, and Lua code in core MediaWiki and extensions.
  2. Once the infrastructure is implemented and deployed, and the repository is working, the work by editors of multiple wiki projects to move the existing templates and modules from local wikis to this repository, and to create new ones.

The “Proposed specification ” document primarily addresses the infrastructure. (Also, as of June 2020, it mostly speaks about it on the level of product requirements and not on the level of internal technical architecture.)

This document here tries to address the other side: How will editors of wiki projects use this infrastructure once it’s available. It may seem that writing such a document so early in the process is too far-fetched because doing any of these things will be possible only when the infrastructure is actually deployed. However, it is useful to try to think about it as early as possible because it can inform the development of the technical infrastructure and the planning of its implementation. It may also help the developers understand which parts of this process can be at least partly automated. Once the infrastructure exists, this page will be converted from a theoretical plan to a practical transition guide.

As with all the other documents about Global templates, this is not the work of one person or a small group. The comments of all editors, developers, and other stakeholders are welcome—otherwise it just won’t work. {{🌎🌍🌏}}

What happens today[edit]

So, let’s say that you want to write an article in the Wikipedia in your language about the Battle of Waterloo, and you want to have in it an infobox like the one in the English Wikipedia article about it. You try to type {{Infobox military conflict}}, but when you publish, you see a red link: Template:Infobox military conflict. So you figure out that you probably have to copy the whole template to your wiki.

If you don’t know much about programming and wikitext (also known as wiki syntax), then you have these options:

  1. Make a simpler template with fewer features.
  2. Make a table that looks kind of like the infobox. (And then make such a table in every article about every battle.)
  3. Just give up and have an article about the Battle of Waterloo without an infobox.

If you do have some time and some knowledge about programming and wikitext, then here’s a simplified list of what you’ll have to do:

  1. Copy the code of this template to your wiki. You realize that it actually uses a module, which has 270 lines of code, and uses several other modules.
  2. You realize that this template also uses a lot of other templates. You learn how to use the Export feature. Export the template, with the Include templates option enabled. As of this writing, this will import 112 templates and modules, and the XML dump will have more than 14,000 lines.
  3. Go through all the wiki syntax code you copied, find all the human-readable user interface strings that have to be translated, and translate them. Be careful not to touch parameters and parser functions.
  4. Go through all the wiki syntax and Lua code that you copied in 112 templates and modules, find all the parameter names and where they are used, and change them to your language. (This step is optional, but useful for editors who don’t know English.)
  5. Copy the documentation pages, delete all the English text from them, and write the same text in your language. Be careful to change the parameter names.
  6. Change the parameter names in TemplateData, but keep the English names as aliases.
  7. Make sure that the TemplateStyles pages were correctly imported. Fix the CSS if needed.
  8. Check that the template actually works and does what you need. If it does not, you may have to search for CSS or JS pages on which it depends.

You have to do all of this for every template that doesn’t exist in your wiki. And it has to be done in every wiki. And now you have a forked version of the template, so importing new features from the English Wikipedia will be far from trivial. These are, basically, the problems that the Global templates proposal is trying to solve.

Working together[edit]

A cross-wiki community of template and module maintainers will have to arise as early as possible. It must be truly global, serving all projects equally: Wikipedia, Wikidata, Commons, and even the less famous projects, such as Wikinews and Wikispecies. All wikis need global templates and there must be no preferential treatment.

TODO: Add some more details about cross-wiki community building...

What to migrate[edit]

It was discussed several times what templates and modules should be migrated, for example on the pages :meta:Which templates should be global? and Multilingual Templates and Modules. All the proposals there make sense.

One approach may be to define which are the most used modules and templates across versions and projects, and then start the migration from there. Attention: the keyword here is used, not needed, since this will help us on two different levels:

  1. Focusing on what is already a core functionality across projects; and
  2. Demonstrating that the whole project works and has positive feedback for involved communities.

Potential candidates for this “phase 1 migration” might be:

And so on. A more precise list of potential candidates can be this one. (It should be noted how many “universally needed” modules and templates are not present in all projects, one more time underlining the necessity of such a project.)

What should be avoided, though, is a potential feeling of imposing something to the local communities. This can arise when locally a module or a template has been modified to the point that it’s not similar to the one of other communities. For this reason, phase 1 should probably limit to “non-controversial modules and templates”, i.e. modules and templates with the most basic functionalities as possible—speedy deletions, boxes, and such.

TODO: Make some queries for statistics about most used templates.

Contributing to the effort without knowing advanced wikitext or Lua programming[edit]

Many of the tasks described on this page require advanced knowledge of wikitext (for templates), the Lua programming language (for modules), or advanced and limited MediaWiki tools, such as protection, deletion, exporting, and importing. However, there are also opportunities for editors to participate in the transition effort even without having to learn these skills or obtaining administrator permissions:

  • Translating templates and modules: Using the Translate interface to translate the strings (messages) of templates and modules after they are migrated to the global repository. This requires only the ability to read in a language in which the template’s strings are already available, to write well in another language, and to write basic wikitext—links ([[Title]]), lists (*, #), etc.
  • Translating documentation: The documentation of templates and modules needs translation, too.
  • Testing: Going over a list of wiki pages that transclude the template, checking that it is displayed correctly, and reporting problems as messages on a talk page.
  • Categorization: There will eventually be thousands of templates global repository. It will be necessary to maintain their categorization in a way that is convenient to template and module developers, to translators, and to casual wiki editors from multiple wikis.
  • TODO: Possibly more things can be added here

Migrating local to global[edit]

TODO: This is only an outline. The section will be rewritten in more detail

Many of the actions are the same for templates and the modules, so they are given together. It’s split where relevant.

Migrating the first version of a template or a module from a local wiki to the global repo[edit]

Every template or module can be a candidate for moving to the global repository. If, for any reason, a template or a module is supposed to remain local, it should be tagged accordingly.

  • Check that it is not already available in the global repo. If it is already there, proceed to the next section.
  • Check whether pages in any wikis transclude a template with the same name:
    • If any pages in any wiki transclude a template with this name, and there is a local copy of that template, this is not a problem because the local versions will be used, and no changes will be caused. Later, the local copies may be deleted if the templates are compatible and the local editing community agrees.
    • If any pages in any wiki transclude a template with this name, and there is no local copy of that template, this will cause these pages to display this template. This may or may not have been the intention of these pages’ editors. Check them one by one. (Something similar to Special:GlobalUsage should probably be enough for that, but this may also be an opportunity to develop a tool that does this more conveniently.)
  • Recursive: If this module or template uses or relies on other modules or templates, migrate the latter first.
  • Export the template or module page and its documentation page from the local wiki and import it to the global repo. Add sitelinks to the imported pages in the corresponding Wikidata item pages.
  • If the template or module is protected in the local wiki, it should probably be protected in the global wiki (as the proposed specification says: “policies about [...] protection [...] of templates and modules will be developed by the editors community”).
  • If the template uses tags (like <section>), parser functions (like {{#ifexist}} or {{#invoke}}), or magic words (like {{DISPLAYTITLE}}), make sure they are written in their generic English name, and not in their localized name.
  • Find which CSS styles does the template or module use.
    • Some of the template’s CSS will be in TemplateStyles. In templates, they appear in <templatestyles> tags, and in modules they are in code that looks like frame:extensionTag{ name = 'templatestyles', args = { src = templatestyles} }. TemplateStyles pages are supposed to be exported and imported automatically in the XML file by the export tool.
    • Some other CSS rules that are used by the template will be in Common.css or other wiki pages. Find them and copy them to a TemplateStyles page in the global repository. (Don’t put them in Common.css. They should not have been there in the first place, because Common.css is for the whole site, and putting styles for particular templates there has always been an unfortunate hack.)
  • For templates, create TemplateData, write at least a short description for the template, add all the parameters, write descriptions for them and specify their types. Not all local templates have TemplateData, but creating the global version is a good opportunity to add it.
  • Make sure that the documentation is up to date.

Making it translatable[edit]

Tracked in Phabricator
Task T238411
Tracked in Phabricator
Task T238417

If this template or module is intended for use in several wikis in the same language, then translatability is not required and this whole section is optional.

  • Find all the human-readable strings that are displayed to users:
    • Modules: Find all the instances of literal strings in quotation marks in Lua code. Skip those that are used internally, and process only those that are actually displayed to users.
    • Templates: Find all the strings that are not pure wiki syntax (tables, links, other template invocations, <noinclude>, etc.). Note that template parameters are treated separately!
  • Move all the strings to a separate storage and replace them with message keys.
    • Modules: Remove the literal strings in quotation marks in Lua code and change them to in message keys. (How will this actually work is to be determined, see task T238417.) Pay attention to string concatenation (e.g. 'Parameter "' .. name .. '" is missing.'), which may require translation placeholders.
    • Templates: Remove the literal strings from the template and replace them with message keys in the wikitext. (How will this actually work is to be determined, see task T238411.)
      • TBD: Another option is to do something like what is done in the {{TNT }} system: add ‎<translate> tags and translate them like a translatable page .
  • Make the strings translatable using Translate:
    • Make sure that this template or module is conveniently discoverable in the project selector.
    • Add message documentation (known as “qqq” in core and extensions) so that translators would understand the context.
    • Apply the usual best practices from the page Localisation .
  • Make the template or module usage documentation translatable as a translatable page . (TODO Consider creating association between the UI strings translation and the documentation translation.)
  • Templates: Make the template’s title and parameter names translatable. (How will this actually work is to be determined, see task T238411.)

Discarding the local wiki page when creating the first version in the global repo[edit]

  • Delete (or tag accordingly) the template or module from the local wiki.
  • Delete (or tag accordingly) the template or module documentation page from the local wiki.
  • Templates: Delete (or tag accordingly) the CSS styles used by the template from TemplateStyles, Common.css, or other CSS wiki pages, unless:
    • These styles are also used by other templates that were not yet migrated to the global repo.
    • These styles need to remain customized in the local wiki, and the customizations cannot be moved to the global repo conveniently.
  • Test that the template or module still works as expected in the local wiki. Fix it if it can be fixed quickly, or restore the local template or module if it’s not trivial. Fix what’s needed, and then delete again, test, etc.

Switching from a local template or module to a version already available in the global repo[edit]

This should be done in every wiki in which a version of a template or a module that is already available in the global repo exists.

Start from comparing the code of the template or module to the code in the global repo.

If the code is the same as the one in the global repo[edit]

  • Translate the template’s or module’s strings into the wiki’s language. (This is optional, but highly desirable.)
  • Go to the “Discarding” section.

If the code is different[edit]

So, you imported a template or module in your wiki some time ago, but...

  1. ...in the meantime it was updated in its original wiki, but left unchanged in your wiki. If the updates are useful for your wiki, and the parameters are overall compatible with the current usage in the local wiki, do the same as in the “If the code is the same” section.
  2. ...was only adapted to your local language. Copy the translated strings from the local wiki to the localizable module in the central repo. If there are no other differences, do the same as in the “If the code is the same” section.
  3. ...you or someone else did some non-trivial functional changes to it. (or: it is a template/module that has similar functionalities, but was developed separately from the first one) Decide in the local wiki on a case by case basis. The main imaginable decisions are (in no particular order):
    • Stop here and keep the local module. Tag it as “don’t move to the global repository”, and explain why.
    • Decide that the local changes are not that important, and do the same as in the “If the code is the same” section.
    • Copy the local changes to the global template or module, making them usable for all wikis. This must be done only after making sure that they don’t break the functionality for other wikis and don’t make the global code harder to maintain.
    • Split the local template or module: Move the local changes to another template or module, and discard the local one.
    • Decide to keep the local template or module only temporarily, in order to prepare a smoother transition to the global one. Once it's done, do the same as in the “If the code is the same” section.

Discarding the local wiki page in favor of a version in the global repo[edit]

  • Templates: If needed, localize the template title and parameters. (How will this actually work is to be determined, see task T238411.)
  • Check the parameters:
    • Modules: Check that the local and the global module have the same parameters. If they are not compatible, replace the current invocations of the module to use parameters that are compatible with the global module. This can often be done with a bot, but sometimes may have to be done manually.
    • Templates: Check that the local and the global template have the same parameters. They may be localized for your language, but the names must be mapped in the global repository. (How will this actually work is to be determined, see task T238411.) If they are not compatible, replace the current transclusions of the template to use parameters that are compatible with the global template. This can often be done with a bot, but sometimes may have to be done manually.
  • Test that the global template or module works as expected in the local wiki. Fix if needed.
  • If the template or module is protected in the local wiki, it should probably be protected in the global wiki. Do this according to the global repository’s protection policy.
  • Delete the local wiki page with the template or module.

Concluding[edit]

  • If you found anything problematic during the migration process, or if you migrate several templates or modules and you find yourself repeatedly doing something that isn’t documented here, update this guide for the benefit of other template developers.

Migrating global to local[edit]

Global templates and modules are usually supposed to be more useful for collaboration and localization, but sometimes a wiki community will want local control over how a template or a module works. Even then, it is better to override only a part of the functionality, but it must be possible to copy a global template or module fully to the local wiki, with all or some of the dependencies, and with translated strings.

Tools that would be useful for transition[edit]

Tracked in Phabricator
Task T245737

Cross-wiki diff[edit]

A tool that shows a diff between revisions of wiki pages on two separate wikis. This would be useful for checking whether the module or template is the same as on another wiki or the global repo. Comparing old revisions will be particularly useful, because it often happens with templates and modules that they are imported and forked.

Dependency lookup[edit]

A tool that gets a list of the templates and the modules that a given template or module uses, and checks which ones are already global.

Module string migrator[edit]

  • Find all the strings in Lua code.
  • Let the user select those that will be shown to users.
  • Copy them to the message storage.
  • Replace the strings in the code with message keys.

(This is just the first step, and the strings will probably have to be tweaked more later.)

Template string migrator[edit]

  • Find all the strings in wikitext: Everything that is not in <noinclude>, not ParserFunctions (including localized ParserFunctions), not template or module calls.
  • Copy them to the message storage.
  • Replace the strings in the code with message keys or put them in ‎<translate> tags.

(This is just the first step, and the strings will probably have to be tweaked more later.)

Template usage analyzer[edit]

To help choose which templates and modules to migrate, it may be useful to analyze:

  • What are the most transcluded and invoked templates and modules in general.
  • What are the most transcluded and invoked templates and modules by other modules and templates.
  • What templates have fewer parameters (and are therefore easier to migrate).

When the dust settles[edit]

TODO: This is only an outline. The section will be rewritten in more detail

  • Some issues to address:
    • Where are the new modules and templates created?
    • How are changes discussed on the repo? (i.e. we set up a freely modifiable sandbox?)
    • How are changes coordinated between different wikis? (i.e. we set up a "version system"?)

Looking even further into the future[edit]

TODO: This is only an outline. The section will be rewritten in more detail

  • Making templates semantic: making them accessible from VE toolbar, more machine readable, etc.
  • Wikilambda

See also[edit]

Retrieved from "https://www.mediawiki.org/w/index.php?title=Global_templates/Transition&oldid=5723854"