Manual:Adding and removing languages

MediaWiki is heavily multilingual and localized. Adding languages to it is a frequent activity. Sometimes, the have to be removed, too. This is done in various contexts, and the procedures can be quite different, both from the technical and the community policies perspectives. This page documents the various language adding procedures.

Core MediaWiki
TODO

jquery.ime
TODO

jquery.uls
jquery.uls is the generic MediaWiki-independent library that provides the core functionality of the Universal Language Selector extension. To add a language to jquery.uls:


 * 1) Add it to the language to language-data.
 * 2) After that, make a GitHub patch in the wikimedia/jquery.uls. In the repo's root folder, run the script.
 * 3) Check the diff. In the commit message, describe the changes, and give a link to the latest language-data commit on GitHub.

Example commit: https://github.com/wikimedia/jquery.uls/pull/422

language-data
language-data is one of the most "liberal" places for adding a language. Its primary purpose is to make the language selectable in Universal Language Selector, although it may also be used in other contexts. Since the generic Universal Language Selector is used for various purposes in MediaWiki, and even outside the MediaWiki and Wikimedia world, it's not as strictly limited to new languages as, for example, the Wikimedia Language proposal policy for new wikis, and it may include ancient and constructed languages or variants that can be generally useful.

To add a language, determine the autonym and make a GitHub patch in the wikimedia/language-data repository. Before doing it for the first time, make sure that you are familiar with that repository's README and with the other documentation to which it links.

After the pull request is merged, update jquery.uls.
 * 1) Edit the file  . Add a line for that language in alphabetical order of language codes. List the ISO 15924 code of the language's script, the continents where the language is spoken, and the autonym.
 * 2) * The ISO 15924 code must appear in one of the  sections towards the end of the file. If the language is written in a script that doesn't yet appear in any of these groups, determine its writing system, make sure that it has a valid four-letter ISO 15924 code, and add it to a one of the groups. Use your judgment to put it in an appropriate group.
 * 3) Run the script.
 * 4) Using , check that the file   was updated.
 * 5) * The  script automatically downloads a file with data about languages from the CLDR server. Note that sometimes it includes updates that are not related to this new language. If this is the case, for the sake of clean Git history, make a separate patch that makes the unrelated changes.
 * 6) Run  . All the tests must succeed.
 * 7) In the commit message, mention the reason for adding the language, and the source for the autonym. (This is similar to adding a language to translatewiki.)
 * 8) Submit a pull request.

Names.php
See Core MediaWiki.

Adding
Adding new languages to translatewiki usually begins as a request on the page Support. The request must include the language's ISO 639-3 language code.

Before adding, read the policy at Translatewiki.net languages and make sure that the language fits it. Ask the requester for clarifications if needed. If the language doesn't fit the policy, politely decline the request.

Determine the autonym. The autonym should be mentioned in the request, but you still have to verify it according to the instructions on this page.

Make a Gerrit patch in the translatewiki repository:

In the commit message:
 * 1) Edit the file  . Add a line for this language in alphabetical order of language codes. Write its autonym in the quotes. In a comment on the same line, add the English name of the language, and write your name and date.
 * 2) Optional: Add default assistant languages. Edit the file , add a line for that language, and list the languages in the array. The language that is the most likely to be helpful to translators should be at the top. Even though the file where it's done is called   and the variable in question is called  , this is not actually a fallback language, but an assistant language that is shown to translators as an aid, when a translation is available. This will make translation easier for people who didn't define assistant language preferences. Only languages that are already available on translatewiki should be added there. English doesn't have to be added there, because it's always shown. Add no more than six languages. Consider also adding the new language as a default assistant languages to other languages. The languages to add there are:
 * 3) * A common foreign or official language in the country where this language is spoken. E.g., if a language is spoken in Indonesia, add Indonesian; if a language is spoken in a Francophone African country, add French; etc.
 * 4) * Languages from the same family. Don't just add everything from the same linguistic family; only add languages that the translators are likely to find helpful.
 * 5) * Other languages spoken in the same country or region.
 * 6) If a language is a variant of another language, and needs only a partial translation, edit the file repoconfig.yaml and add the language code to the relevant always-export-languages sections. (Example: "pap-aw".)


 * 1) Give a reason for adding the language. If it's a request on the Support page, give a direct permanent link to the relevant thread. If the reason is different, explain it clearly.
 * 2) Give the source for the autonym.

See the git log for  example patches.

After the patch is reviewed and merged, deploy the updated configuration to the production translatewiki server. After the deployment, test that the language was added correctly: go to Special:Translate, click the target language selector, and type the new language's code. The new language should appear in the results.

Add the language to language-data and then to Universal Language Selector. The translatewiki.net change can be deployed before the Universal Language Selector change is done, and the language will be usable as a target language for localization, but selecting it can be broken in some cases, so don't forget it and do it as quickly as possible.

Make sure that there is a language portal for the language on translatewiki. The portals have names in the form "Portal:qqf", where qqf is the language code. Some portals for languages that aren't yet configured already exist, but the language may be marked as "disabled". See the instructions at Template:Portal.

Add the language's native and transliterated autonyms to the Wikimedia Portals localization (not to be confused with translatewiki's own language portals).

Removing
Languages are removed from translatewiki in two cases:


 * 1) If the language has crossed the export threshold (see Translating:MediaWiki) and was added to MediaWiki core's , it should be removed from translatewiki.net's own configuration. In this case:
 * 2) Make sure that the language is actually in Names.php, and that the MediaWiki revision that includes it has beed deployed to translatewiki!
 * 3) Remove the language's entry from the file.
 * 4) Check the language's entry in the file  . If the language has no entry, you're all set. If it has an entry, check whether any of the assistant languages has been added to core MediaWiki as the fallback language, remove that language from  . Leave the rest of the languages, as they may still be useful as assistant languages.
 * If, for some reason, the translatewiki administrators decide that MediaWiki and other projects hosted on translatewiki shouldn't be localized into that language. For example, if the language was added not according to policy, if grave mistakes were made while adding the language, or if the localizations are very low quality and it's better to remove them and start the work in that language from scratch, etc.
 * 1) Remove the language's entry from the file   and from.
 * 2) If the language is being removed because it was added by mistake, check whether it appears in any special configurations in , and remove them if needed (to be extra-sure, grep the repository for more appearances).
 * 3) Consider also deleting all the translations from the site.
 * 4) TODO: anything more to document here?

Universal Language Selector
Universal Language Selector is the MediaWiki extension that provides language selection functionality in various contexts.

To add a language to Universal Language Selector, first add it to language-data and then to jquery.uls. After that is merged, make a Gerrit patch for Universal Language Selector:


 * 1) From the repo's root folder, run the script.
 * 2) Check the diff. In the commit message, describe the changes, and give a link to the latest jquery.uls commit on GitHub.

Example commit: https://gerrit.wikimedia.org/r/c/mediawiki/extensions/UniversalLanguageSelector/+/793082

Wikibase
TODO

Wikimedia Portals
Wikimedia Portals are the main language-neutral pages of Wikipedia, Wiktionary, and some other Wikimedia projects. They have www in the beginning of their URL, rather than a language code.

To make sure a language can be properly supported in portals, do the following steps:


 * 1) Make sure that the language is configured in translatewiki.net.
 * 2) Go to Special:Translate on translatewiki.net.
 * 3) Select "Wikimedia Portals" in the message group selector.
 * 4) Select the target language.
 * 5) Click "..." in the toolbar (next to "Translated") and check the "Optional messages" box.
 * 6) In the key portals-language-name, add the autonym.
 * 7) If the autonym is not in the Latin alphabet: In the key portals-language-name-romanized, add a transliteration of the autonym. It may include diacritics and various special character.
 * 8) If the key portals-language-name or portals-language-name-romanized includes any characters outside the basic 26-letter Latin alphabet: In the key portals-language-name-romanized-sorted, add a transliteration of the autonym without any special characters.

Make sure to read the documentation (qqq) for each of the messages above!

The rest of the "Wikimedia Portals" message group should also be translated, but this, of course, should be done by people who know the language.

If there is a project in that language, but its name doesn't appear on that project's portal, or it appears in the wrong section, please create a Phabricator ticket with the tag "#wikimedia-portals". (TODO: Describe the actual fixing.)

Determining the autonym
When people ask to add a language or to change the autonym of an already-configured language, please do your best to verify the autonym. This is sometimes challenging: MediaWiki already supports well over 400 languages, which also happen to be the world's best-documented languages. This means that many of the new languages that are being added are less well documented, so it's generally harder to find information about them. In particular, their autonyms may be hard to find, or different sources may cite different autonyms, and when adding the language you'll have to make a decision without actually knowing the language. Try to use your best judgment and to reach a reasonable compromise between the information in available third-party sources and the information given to you by the requesters.

Autonyms usually appear in lists of languages, so they have to be unique. The users must be able to choose the precise language that they need, and not something with a similar name.

Some sources where you can find autonyms:


 * A good autonym may appear in the Wikipedia article about the language in major languages, such as English, French, German, Portuguese, Spanish, or Russian. You can also look for it at the Wikidata item page for the language. It may be correct, but as with every statement in Wikipedia and Wikidata, do verify it with an external reliable source.
 * The best source for autonyms is a professionally written and published online or printed book about the language: a dictionary, a grammar reference, a standard orthography guide, a history of the language, etc. Academic articles about the language are a good source, too. Glottolog often has names of such sources, although it doesn't always have a link to the online source itself. A good source where actual books or at least parts thereof may be found are the Internet Archive (many such books are available for free time-limited lending in the e-book library). Some providers in The Wikipedia Library, such as L'Harmattan, Cambridge, and others have free access to relevant e-books. Google Books and Amazon.com may have enough in the preview to find the autonym.
 * Other websites and apps in this language can be a very good source, especially if they have a language selector. One particular kind of app to check is Android keyboard apps, such as Gboard: install the app, enable the language, and check how does its name appear when you choose to type in it. Check also the language settings on Windows—recent versions have a very large selection of languages.
 * The UN's Universal Declaration of Human Rights is one of the most translated documents in history, and translations are available online. Unfortunately, the lookup system for languages is not so convenient, and the presentation is non-standardized and quite inconsistent. Nevertheless, it is sometimes a useful source.
 * Ethnologue: the autonym is usually available as part of the freely-shown pages, although it's not always the best option. Try to check in othe sources, too.
 * JW.org: this site is available in more than a thousand languages and has an autonym for each of them, but they are sometimes written differently from other websites or books. Do your best not to rely only on this source.