Multilingual Templates and Modules
Introduction[edit]
What is it? This project makes it possible for modules and templates to be used on multiple wikis, without any modifications. All translations are stored in one place, accessible from everywhere.
Why is this needed? Because we do not have a single Wikipedia, we have 300+ separate Wikipedias and other wiki projects, and every time someone creates a good new template or a Lua module, it gets copied and translated 300+ times. Every translator has to thoroughly understand MediaWiki markup, making copying a very tedious and error-prone process, partially because template authors often assume their templates will be used in just one language. Once copied, the original templates are often improved, and each copy has to be updated while maintaining all existing translations. The pure human expense of copying templates and modules is so high that most of them are either never copied or never updated, especially for the smaller wikis.
Is this the best approach? No, but it is the best approach with the current technology. A significant MediaWiki rewrite is required to make this possible on the system level. Multi-site templates has been requested from the start in 2001, but not much was done simply because this is a hard problem to solve. With this approach it is possible to create multilingual content now, and once MediaWiki supports it, we can easily migrate multilingual content to the new system without much work.
Method[edit]
- DiBabel tool allows users to copy templates and modules from origin (mediawiki.org) to all other sites/languages listed in Wikidata for that page, automatically translating dependent template and module names.
- DiBabel only shows pages whose Wikidata item has "instance of" set to Wikimedia multi-site page (Q63090714).
- DiBabel will indicate if the content of the local page exists in the history of the page on origin. E.g. if a user modifies en.wikipedia version of a page, it will not be considered "diverged" unless that exact text ever existed in the history of the page on mediawiki.org. Note that simply reverting a page to an older version will not be enough to mark the page as "diverged". A page has to be different from any of origin's revisions (e.g. add a note why the page was modified).
- Any template names will be automatically localized inside the wiki markup, e.g. {{MW name|...}} text will become {{Localized name|...}}, as long as Template:MW name exists on mediawiki.org, and has a corresponding template on the destination wiki. Shared templates should not use non-shared templates, as that may cause unexpected results.
- Just like templates, modules text
require('Module:MW name')
will be converted torequire('Module:Localized name')
. - NOTE: The original DiBabel bot has been decommissioned in favor of the new user-controlled tool.
How can you help?[edit]
- translate DiBabel interface --most importantly translate these ones: [1], [2], [3], [4] (switch to your language before modifying).
- translate warning messages (shown at the top of all auto-copied templates and modules), and corresponding template data params.
- translate TNT error messages (shown when TNT module finds an error in a template).
- For template and module authors -- create a new localized template or module on mediawiki.org. It might be easiest to start with a simple template, like an info or a warning box, or internationalize a popular existing module. Shared templates and modules should not use any templates or modules that are not already shared, except for the few listed as non-shareable below, like {{Documentation}}.
Module Example[edit]
Module:No globals was migrated to the new system with very few changes. These were the steps taken:
- Ensure proper testing. There could be millions of pages that rely on any given module, so we need to make sure the module is adequately tested before making any changes. For this example, I had to create Module:No_globals/testcases that verified that the module would raise expected errors when global variables were accessed without declaration, and do nothing otherwise. The testcases/doc (detailed) and the main doc (summary) pages were updated to show the results of the test runs.
- Note that the testcases can work with both the main module and the sandbox module, so that we can compare the results.
- Note that the testcases/doc and the main doc pages shows the results of both the sandbox and main modules. The summary test results must always be visible in order to catch errors as early as possible.
- Create translation table from strings. The No globals module contained two English strings:
Tried to read nil global ...
andTried to write global ...
, and both strings were followed by a parametertostring(k)
. We created a new translation table Data:I18n/No_globals.tab on Commons with the two strings, giving each a unique ID:err-read
anderr-write
. The parameter placeholder for both strings had to be included into the string itself as a$1
. It is very important to store parameters as parts of the string because in many languages the parameter would have to be placed at a different position in the string according to the norms of the language. - Replace strings with calls to TNT module. Now we can use Module:TNT format function to convert
err-read
from the translation table into a localized string. Placelocal TNT = require('Module:TNT')
at the top of your module, and callTNT.format('I18n/No globals', 'err-read', tostring(k))
to get the localized text. For this specific example, we do not need to load TNT module unless there is an error, so we can do both calls on the same line to optimize performance. These were the two replacements made:Note that we first modified the Module:No globals/sandbox page to make sure our changes did not break any of the tests (see point #1).... old: error('Tried to read nil global ' .. tostring(k), 2) new: error(require('Module:TNT').format('I18n/No globals', 'err-read', tostring(k)), 2) ... old: error('Tried to write global ' .. tostring(k), 2) new: error(require('Module:TNT').format('I18n/No globals', 'err-write', tostring(k)), 2) ...
- Lastly we need to indicate that this is a shared module by adding a
{{Shared Template Warning|Module:No globals|No globals}}
to the top of the doc page. The first parameter is the name of our template on mediawiki.org, and the second parameter indicates that this module has a translation table called Data:I18n/No_globals.tab on Commons. We also need to add Q63090714 as an additional instance-of for the Module:No globals wikidata entry (Q16748603).
[edit]
- Sandbox: Each module must have a .../sandbox sub-page with a copy of the module code (the code might have a few changes compared to the main module page)
- Testcases: Each module must have a .../testcases page with good unit tests to ensure high quality and stability of the shared module.
- Testcases must use Module:ScribuntoUnit
- Testcases must be able to run with both the module and module/sandbox versions (see TNT or TableTools examples)
- Testcases should have require('Module:no globals') to avoid accidentally using non-declared variables
- Documentation: Each module must have a good quality documentation in the /doc page:
- The /doc page must start with
{{Shared Template Warning|Module:__module_name_on_mw__}}
. - If module has a translation table, add it as a second parameter in the warning, e.g.
Module:TNT
would link to Data:I18n/Module:TNT.tab on Commons. - Add a quality control section at the top, listing the summary of the testcase runs for both the primary and the sandbox versions of the module. See Module:TNT for an example.
- Document all public functions of the module.
- The /doc page must start with
- Translations: module must not have any user-visible strings hard-coded in any human language. If it needs any user-visible strings, it must use the TNT module to convert a message ID with optional parameters to a string for the current language. See below for an example.
[edit]
- Translations: template must not have any user-visible strings in any language. Instead, it should use TNT module to convert a message ID with optional parameters to a string for the current language. See below for an example.
- Documentation: Each template must have a good quality documentation in the /doc page:
- The /doc page must start with
{{Shared Template Warning|Template:__template_name_on_mw__}}
. - If template has a translation table, add it as a second parameter in the warning, e.g.
Template:XX
would link to Data:I18n/Template:XX.tab on Commons. - Template one-line description and parameters must be stored in a template parameter table, see Translating Template Parameters below.
- Insert
{{#invoke:TNT|doc|Template_Name}}
at the end of the /doc page, where Template_Name would mean Data:Templatedata/Template_Name.tab on Commons.
- The /doc page must start with
- If possible, create a /testcases subpage to run unit tests.
Translating user-readable strings[edit]
There is very little value in having non-translated user-readable strings copied as is. The TNT module was created specifically to solve that. Instead of storing text in English or in another human language in a module or a template, TNT allows them to be designed as language-neutral, and store multilingual text in the tabular data pages on Commons. This way your module or template will use those translated strings (messages). If the message has not yet been translated, will fall back to English. When someone updates the translation table, your page will automatically update (might take some time, or you can purge it), but no change in the template or module is needed on any of the wikis. This process is very similar to MediaWiki's localisation, and supports all standard localization conventions such as {{PLURAL|...}} and other parameters.
For a simple example, see Data:I18n/Template:Graphs.tab - a table with two messages, each message having a single parameter. By convention, all translation pages should have '''Data:I18n/...''' prefix to separate them from other types of data.
Global Templates[edit]
Description | Wiki Markup |
---|---|
In a template, this command translates source_table message using Commons' Data:I18n/Template:Graphs.tab translation table. | {{#invoke:TNT | msg | I18n/Template:Graphs.tab | source_table }} |
If your message contains parameters, you can specify them after the message ID. | {{#invoke:TNT | msg | I18n/Template:My Template.tab | message-with-two-params | param1 | param2 }} |
Global Modules[edit]
Just like templates, modules should also use TNT module for localization:
local TNT = require('Module:TNT')
local text = TNT.format('I18n/My_module_messages', 'messageId', 'param1', 'param2', ...)
-- or in a specific language:
local text = TNT.formatInLanguage('fr', 'I18n/My_module_messages', 'messageId', 'param1', 'param2', ...)
Translating Template Parameters[edit]
Template parameters are usually stored as a JSON templatedata block inside the template's /doc subpage. This makes it convenient to translate, but when a new parameter is added to a global template, all /doc pages need to be updated in every language. TNT helps with this by automatically generating the templatedata block from a table stored on Commons. Placing this line into every /doc sub-page will use Data:Templatedata/Graph:Lines.tab table to generate all the needed templatedata information in every language. Even if the local community has not translated the full template documentation, they will be able to see all template parameters, centrally updated.
{{#invoke:TNT | doc | Graph:Lines }}
Volunteers[edit]
- Capankajsmilyo (talk) 18:24, 17 May 2019 (UTC)
- Viztor (talk) 00:13, 5 August 2019 (UTC)
See also[edit]
- Wishlist proposal 2019 (40 votes)
- T122086 - RFC: Sharing templates and modules between wikis - poor man's version (original idea for this bot)
- T121470 - Central Global Repository for Templates, Lua modules, and Gadgets ticket (main ticket for everything cross-site shareable)
- Extension:Scribunto/Central modules
- Global templates/Proposed specification, short version - a proposal to implement a similar idea comprehensively, without the need for the special tools, and with full support in MediaWiki core and extensions
FAQ[edit]
- Q) If templates and modules are centralized, what will be about translated templates and modules?
- A) Each centralized module/template will have Template:Shared Template Warning placed at the top. In this mechanism, code and translation is separated to make the work easier for editors. Code is stored and reused from mediawiki.org and the translations are stored and used from commons Data. For example, code of Template:Shared Template Warning will be copied from Template:Shared Template Warning and the translations are managed at commons:Data:I18n/Shared Template Warning.tab
- Q) We need a template and module which can accept Devanagari Number (१,२....), so this updation will provide sa.wiki this facility?
- A) Only the compatible modules/templates are converted as centralized after aggressive test validations. So specific requirements of each wiki given utmost importance. Conversion of only Module:TNT, Module:No globals, Module:TableTools and Template:Shared Template Warning has been completed so far due to this. Rest are still in the process and will not be impacted till everything is set right.
- Q) If some mistake occurred, who should be informed for correction?
- A) In case of any mistake, you can simply leave a message on talk page or contact on the template/module talk page on mediawiki or can directly contact Yurik or me. The issue will be resolved.
Candidate modules to use this approach[edit]
Done[edit]
- Module:TNT
- Module:No globals (translations) -- implemented , needs to be moved to communities
- Module:TableTools
- Module:Transcluder
- Template:Shared Template Warning
- Template:BCP47 – not very useful on monolingual wikis, although other global templates may use it; auto-update is quite useful on multilingual ones
Pages OK to use without synchronization[edit]
These templates and modules exist on most wikis, but contain too much language-specific customizations to benefit from this system. A shared module/template can rely on these in addition to any other shared module/templates. Any new wiki should copy and modify these templates by hand.
- Template:Documentation -- most templates on most wikis use this template to show a documentation header on the template pages.
- I'm not sure about this. All templates perform the same function, are equally structured and are equally used. All graphic changes can be made through CSS without touching the code itself. You can look at my solution to this: Module:Documentation/sandbox, Template:Documentation/sandbox. Iniquity (talk) 23:45, 28 May 2019 (UTC)
- Module:Yesno -- what each language accepts as "yes" and "no" for template parameters
In progress[edit]
- Module:Arguments (Status: Unit test pending)
Support - while we can adapt it as is, we may also enhance it to allow parameter name translation table to come from Commons (many templates localize their parameters). --Yurik (talk) 19:06, 7 May 2019 (UTC)
Support --Obsuser (talk) 21:09, 11 July 2019 (UTC)
Support, the same version is used in a bunch of wikis. There is a kind of related patch at https://gerrit.wikimedia.org/r/#/c/mediawiki/extensions/Scribunto/+/158323/ . --Amir E. Aharoni {{🌎🌍🌏}} 23:36, 24 January 2020 (UTC)
- Module:Lua banner – ready, but not actually made global because its dependencies are not global yet, which would cause a mess
Proposed[edit]
- Module:Navbar
- Module:Infobox
Neutral - eventually yes, but I suspect infobox has some dependencies we need to solve first. --Yurik (talk) 19:06, 7 May 2019 (UTC)
Neutral - hard CSS dependencies. Missing navbar module can be handled with code. --Zache (talk) 02:01, 8 May 2019 (UTC)
Strong support --Obsuser (talk) 21:09, 11 July 2019 (UTC)
Support, of course. How can there even be any doubt. It's big and difficult, and local communities need to have the ability to override content and design in some places, but it's one of the most visible families of templates, and most of the functionality is the same in all wikis, even if the implementation and the content policies around using infoboxes may be different. For small wikis, the lack of infobox templates is one of the biggest pain points. --Amir E. Aharoni (talk) 10:33, 25 September 2019 (UTC)
Support -- Masumrezarock100 (talk) 11:20, 23 November 2019 (UTC)
- Module:Location map
Very strong support --Obsuser (talk) 21:09, 11 July 2019 (UTC)
Support Thousands of subpages with the name being the only parameter needed to be translated to different languages. --94rain Talk 06:15, 6 August 2019 (UTC)
Oppose location map -template will be eventually deprecated as development is already moving to dynamic maps. --Zache (talk) 06:34, 23 August 2019 (UTC)
- Module:String
- d:Module:Databox
- Module:Navbox
Support --Obsuser (talk) 21:09, 11 July 2019 (UTC)
Support -- Masumrezarock100 (talk) 11:20, 23 November 2019 (UTC)
- Module:Message box
- Module:ScribuntoUnit
- Template:Bot
Support It's one of the most commonly used templates in all fo Wikimedia's projects, perhaps the most used one. --Amir E. Aharoni (talk) 10:33, 25 September 2019 (UTC)
- Template:Convert
Support It even has instructions for adapting to other languages. --Amir E. Aharoni (talk) 06:27, 26 September 2019 (UTC)
- Module:InfoboxImage
SupportCapankajsmilyo (talk) 01:06, 25 March 2020 (UTC)
- Module:DateI18n
Support - This module localizes dates, is the latest incarnation of the code used for over a decade on majority of files on Commons. It is cloned to many other wikis. I recently rewrote it to use c:Data:DateI18n.tab and c:Data:I18n/MonthCases.tab to store data on language specific date formats. --Jarekt (talk) 03:58, 13 April 2020 (UTC)
- Template:User link
- Template:Tl
- comment Suggest to migrate this to be a magic word, instead of copy-pasting everywhere. --Liuxinyu970226 (talk) 14:51, 30 April 2020 (UTC)