Multilingual Templates and Modules

This page explains how to create global, cross-wiki and multilingual modules and templates, and how to keep them synchronized across Wikimedia wikis.

Why is this needed? Because we do not have a single Wikipedia, we have 300+ separate Wikipedias and other wiki projects, and every time someone creates a good new template or Lua module, it gets copied and translated 300+ times. Every translator has to thoroughly understand MediaWiki markup, making copying a very tedious and error-prone process, partially because template authors often assume their templates will be used in just one language. Once copied, the original templates are often improved, and each copy has to be updated while maintaining all existing translations. The pure human expense of copying templates and modules is so high that most of them are either never copied or never updated, especially for the smaller wikis.

Is this the best approach? No, but it is the best approach with the current technology. A significant MediaWiki rewrite is required to make this possible on the system level. Multi-site templates has been requested from the start in 2001, but not much was done simply because this is a hard problem to solve. With this approach it is possible to create multilingual content now, and once MediaWiki supports it, we can easily migrate multilingual content to the new system without much work.

Method

 * A global module is a Lua module designed to be used with exactly the same code in every wiki.
 * To become global, a module must provide ways for wikis to localize everything they need to localize, without having to touch the code of the module itself. This page describes several techniques to accomplish this.
 * Once a module becomes global, the Synchronizer tool can be used to keep it updated across Wikimedia wikis.
 * A global template is a template designed to be used with exactly the same wikitext in every wiki. With the rise of Lua modules, the need for global templates is declining, so this page will focus on modules, but most of the concepts, tools and techniques discussed for global modules apply to global templates too.

Example
Module:Excerpt is currently the best example of a global module. It demonstrates most of the techniques explained below.

Best practices
This section describes some of the current best practices while developing global modules.

Naming
This section can be ignored for modules designed to be called from templates only.

Global modules that are meant to be used by other modules should be named the same in all wikis. This is to avoid dependency breaks between global modules. For example, if a module named A requires a module named B, but in some wiki, module B is named C, then module A will not work in that wiki, unless the source code of module A is changed locally to require C instead of B, which would defeat globalization (of module A).

If a local community doesn't want the global name, or renaming is too much trouble, then one workaround is to create a "redirect module" with the global name that simply requires and returns the module with the localized name (example).

Luckily, the fact that the Module namespace is named differently in each language doesn't break dependencies, because "Module" is an alias for the Module namespace in all languages.

Master module
Global modules should pick one wiki where to do the development. Generally this will be the home wiki of the module, but it may migrate for various reasons, for example a desire to do the development in a more central wiki where chances of finding new developers increase.

Initial comment
Global modules should start with a comment that includes a link to the master module, and some kind of warning asking to contribute to it rather than the local version (example). This practice can not only prevent forking but also help to recruit new developers.

Sandbox
Global modules should have a /sandbox subpage where to test out changes before deploying them on the main module and the other wikis.

Testcases
Global modules should have a /testcases subpage with good unit tests to ensure high quality and stability of the module (example).


 * Testcases should use Module:ScribuntoUnit
 * Testcases should run with both the main module and the sandbox versions, so that we can compare the results (example)
 * Testcases should use require('strict') to avoid accidentally using non-declared variables
 * Testcases should output their results both in /testcases/doc and the main /doc page of the module, to catch errors as early as possible

Documentation
Global modules should have a /doc subpage with:


 * Documentation of all public functions of the module
 * A section listing the summary of the testcase runs for both the primary and the sandbox versions of the module (example)

Localization of template parameters
Global modules should have their parameters localized by the templates that call them. For example, consider the following module that simply outputs the given text (or "Example" if none is given): Then a Spanish template would localize the module like so: Notice that the template not only localizes the name of the "text" parameter ("texto" means "text" in Spanish), but also the default text ("Ejemplo" means "Example" in Spanish).

See Plantilla:Extracto for a real case of a template that localizes a global module (Module:Excerpt) with this technique. Also, see Template:Excerpt for a case where a global module is localized to the English Wikipedia, demonstrating that localization is not always the same as translation.

Synchronization
Once a module is able to be copied unchanged to other wikis, the Synchronizer tool can be used to keep it synced across all Wikimedia wikis.

Using Template:Synchronize, each developer can build their own "dashboard" or "control panel" in their sandbox or in a dedicated page or subpage anywhere they find convenient, and use it to keep the modules under their guard updated (example).

Backwards compatibility
Global modules should keep development backwards-compatible. Changes that are not backwards-compatible may require you to go through all the wikis, templates and modules that use the module, and manually update to the new usage.

Localization of user-readable strings
Many modules need to output user-readable strings, such as error messages and interface elements (like buttons). Hard-coding the text of these strings would force other wikis to modify the code in order to localize them, which would complicate or even prevent future synchronization. To avoid this, developers should provide a way to localize user-readable strings without having to modify the code itself. This section explains several ways to achieve this.

Template parameters
Modules may allow templates to set the user-readable strings when calling the module. This approach is convenient when:


 * The text is likely to vary with each template call.
 * The text is likely to contain a magic word, a template call or some other wiki element.

An example of a module using this approach would be:

This way, every template may modify the text when calling the module, like so:

Notice that in this example, if a template calls the module without specifying the  parameter, then the hard-coded English text 'Example' will be used. This is not necessary. Modules may require template callers to set the  parameter by throwing an error if they don't. However, it's often friendlier to fallback to English.

Config file
Another approach is to have a separate /config subpage where strings are localized. This approach is convenient when:


 * The module is meant to be called by many templates per wiki, thus allowing localizations to be done once and reused again and again.
 * There're many messages to localize, so it's easier to have them all together in their own place.
 * There's already a need for a /config file for other reasons, so we might as well use it for localizations too.

An example of a module using this approach would be:

Then wikis would be able to create /config files like the following:

Translation tables
It's also possible to create a central translation table at Commons and get localizations from there. This approach is convenient when:


 * The strings should vary with the preferred language of the user, rather than the language of the wiki or the page.
 * We want to centralize localization efforts on a single page.

The Module:TNT was created specifically to get strings from translation tables. An example module using TNT could look like this:

See Data:I18n/Template:Graphs.tab for a simple but real example with two messages, each message having a single parameter. It's important to store parameters as parts of the string because in many languages the parameter would have to be placed at a different position in the string according to the norms of the language.

Translation tables should start with the "Data:I18n/..." prefix to separate them from other types of tabular data. If a message has not yet been localized, TNT will fallback to English (or other fallback language as defined by the language's fallback sequence). TNT also supports all standard localization conventions such as NaN undefineds and other parameters.

One downside of this approach is that translation tables cannot be loaded from non-Wikimedia wikis, so relying only on this localization method prevents third-party wikis from using the module as-is.

MediaWiki messages
In some cases, MediaWiki itself (or some extension) may have the messages we need already localized. For example, if we need the string "New page" we may use MediaWiki:Newpage, like so:

See Special:AllMessages for a list of all available messages.

All of the above
Depending on the case, all of the above methods may be combined. For example, MediaWiki messages may be used when available, and when not, a translation table or config file is queried, and if no localization is found there, then a hard-coded English text is used, unless a template parameter overrides it.

Combining several methods can be effective, but the benefits should be weighted against the downsides of the increased complexity, which may cause performance loss and bugs, as well as more difficulty in maintaining the code and attracting new contributors.

Template data
Template parameters are usually stored as a JSON templatedata block inside the template's /doc subpage. This makes it convenient to translate, but when a new parameter is added to a global template, all /doc pages need to be updated in every language. Module:TNT helps with this by automatically generating the templatedata block from a table stored on Commons. Placing the following line into every /doc subpage will use Data:Templatedata/Graph:Lines.tab table to generate all the needed templatedata information in every language. Even if the local community has not translated the full template documentation, they will be able to see all template parameters, centrally updated.