Extension:UniversalLanguageSelector

The Universal Language Selector is a tool that allows users to select a language and configure its support in an easy way. Where used, it also ships the functionality of both ext-webfonts>Special:MyLanguage/Extension:WebFonts|WebFonts and ext-narayam>Special:MyLanguage/Extension:Narayam|Narayam extensions (both of which have been deprecated in favour of Universal Language Selector). See uls>Special:MyLanguage/Universal Language Selector|Universal Language Selector for background and additional information.

Usage
The primary aim is to allow users to select a language and configure its support in an easy way.

The extension provides the following features:


 * 1) Flexible and easy way to select a language from a large set of languages.
 * 2) Selection can be based on geographical region based browsing, searching.
 * 3) Search can be based on ISO language code, language name written in current user interface language or in its own script(autonym)
 * 4) Cross language search - search language names using any script
 * 5) Autocompletion of language names
 * 6) Auto correction of spelling mistakes in search strings
 * 7) Geo IP based language suggestions
 * 8) Language selection based on users browser/OS language
 * 9) Input methods
 * See  for complete instructions.
 * 1) An easily selectable input method collection, readily available in every editable field
 * 2) Provides a large set of input methods for wide range of languages
 * 3) * An outdated list is available at Extension:Narayam#Supported schemes
 * 4) Per-language input method preferences
 * 5) Webfonts
 * 6) A large collection of fonts to choose for each language to use as embedded font for the page
 * 7) * An outdated list is available at Extension:WebFonts#Supported languages
 * 8) * You can browse [https://phabricator.wikimedia.org/diffusion/EULS/browse/master/data/fontrepo/fonts all the fonts included]
 * 9) Per-language font preferences



Adding fonts
Supporting more languages is only a matter of including the proper fonts in the code. However, please note that we will add support only for freely licensed fonts, for example fonts licensed under GNU GPL, SIL OFL, etc. An example directory of such free fonts is Google Fonts  (not yet fully examined/exploited by the authors of this extension); see also the [http://openfontlibrary.org/ Open Font Library].

First of all, you need to find or produce such a free font (this is the most important part and you have to do it yourself); then, it has to be converted to the required formats, which are eot, ttf, woff; finally, you can [https://phabricator.wikimedia.org/maniphest/task/create/?projects=MediaWiki-extensions-UniversalLanguageSelector file a request] in phabricator>Special:MyLanguage/Phabricator|Phabricator for the font to be added to the extension.

prepare>#Preparing webfonts</>|#Preparing webfonts below explains how to convert the fonts: basic knowledge about GNU/Linux based operating system is required; if you have difficulty in doing this, you can skip this step and ask someone else to do it for you on the same Phabricator request (of course this will slow down the process).

Preparing webfonts
[<tvar|sfntly>https://github.com/googlei18n/sfntly</> Sfntly] is required for creating web fonts (command line only).

Once installed, following commands can be used.

Creating .woff:

 Creating .woff2: 

Use <tvar|url>https://github.com/google/woff2</> to generate woff2 from ttf.

This will produce a compressed woff2 file. Modern browsers support this format.

Create a font.ini file. Here's an example:

An asterisk after a language code means that this font will be the default font for that language. If you just want the option to use this font for that language, don't use the asterisk.

After creating the files, do the following:
 * 1) Create a directory for the font under data/fontrepo/fonts.
 * 2) Put the ttf, woff, eot and font.ini files in that directory and add them to the source repository.
 * 3) Go to data/fontrepo/scripts and run.
 * 4) Commit the changes to the repository  and submit them according to the tldr>Special:MyLanguage/Git/TLDR</>|Git workflow.

Adding support for a new key mapping (input method)
Follow the <tvar|jquery>instructions on the jquery.ime github wiki</>, but file requests in the Wikimedia-extensions-UniversalLanguageSelector phabricator>Special:MyLanguage/Phabricator</>|Phabricator product.

Updating LanguageNameIndex
For performing cross language search, searching autonyms, language data needs to be populated. ULS comes with a pre-populated language name index(data/langnames.ser). In case you want to update it, install ext-cldr>Special:MyLanguage/Extension:CLDR</>|Extension:CLDR and update the data with the following command.

<pre style="overflow:auto;">php UniversalLanguageSelector/data/LanguageNameIndexer.php

and verify that langnames.ser file gets generated in ULS/data/ folder.

Configuration
The following variables are created automatically during initialization and can be used from JavaScript using mw.config.get( NAME ):
 * - an associative array where the keys are language codes and the values are language names in English.
 * - an array of language codes from the user's Accept-Language value. These are the languages selected in the user's browser preferences.

For serving fonts, you might want to add the following mime types to your webserver if not already there. [<tvar|webcoder>http://www.webcoder.de/2010/06/04/how-to-configure-your-webserver-for-webm-video-and-audio-files/</> This guide] might help. <pre style="overflow:auto;"> application/x-font-woff .woff application/vnd.ms-fontobject .eot application/x-font-ttf .ttf The following variables can also be configured:
 * - ULS can use geolocation services to suggest languages based on the country the user is visiting from. Setting this to false will prevent builtin geolocation from being used. You can provide your own geolocation by setting window.Geo to object which has key 'country_code' or 'country'. If set to true, it will query Wikimedia's geoip service. The service should return jsonp that uses the supplied callback parameter. Defaults to http://freegeoip.net/json/ and expects the same format.
 * - Enable language selection, compact language links, input methods and webfonts for everyone, unless the behavior is overridden by the configuration variables below.  Even if false the classes and resource loader modules are registered for the use of other extensions. Language changing via cookie or setlang query parameter is not possible.
 * - Allow anonymous users to change language with cookie and setlang query param. Do not use if you are caching anonymous page views without taking cookies into account. Does not have any effect if either of   or   is set to false.
 * - Disable the input methods feature for all users by default. Can still be enabled manually by the user.
 * - The location and the form of the language selection trigger. The possible values are: : as a link near the username or the log in link in the personal toolbar (default).  : as an icon near the header of the list of interlanguage links in the sidebar.
 * - Array of jQuery selectors of elements on which IME must not be enabled. eg:

Position of ULS trigger

 * - The location and the form of the language selection trigger. The possible values are: : as a link near the username or the log in link in the personal toolbar (default).  : as an icon near the header of the list of interlanguage links in the sidebar.

It is also possible to have ULS trigger anywhere in the screen. An element with  will act as a ULS trigger.

Overriding default fonts
ULS has a large font repository to serve as webfonts. Sometimes there are multiple fonts for a language and there is a default font for each language/script. The order of fonts or default font can be overridden as follows using global scripts (MediaWiki:Common.js) or personal scripts (Special:MyPage/common.js):

Here, languageCode should be a valid language code(eg: en, hi, nl). FontA and FontB are fonts available in font repository. In the above example for languageCode, we set a font available in local computer as default font. ie No default webfont.

Caching configuration
To ensure that the web fonts files are cached on the clients' machines, font file types must be added to the web server configuration. In Apache2 this consists of:
 * Adding font file extensions to the FileTimes regex at FilesMatch for the relevant directory, example:


 * Adding ExpiresByType values to the relevant MIME types, similarly to image MIME types.
 * Note that there's no standard MIME type for TTF. application/x-font-ttf is used for Wikimedia.
 * Adding the MIME types:

<pre style="overflow:auto;"> AddType application/x-woff .woff AddType application/vnd.ms-fontobject .eot AddType application/x-font-ttf .ttf

For a full example see the [<tvar|cluster>https://gerrit.wikimedia.org/r/#patch,unified,657,1,files/apache/apache2.conf.appserver</> caching configuration update done for the Wikimedia cluster].

Page translation
UniversalLanguageSelector is one of the dependencies of the ext-translate>Special:MyLanguage/Extension:Translate</>|Translate extension, which uses it for several language selection features. One of it is the mylang>Special:MyLanguage/MyLanguage</>|MyLanguage system for links, which depends on the interface language of the user, but more can be configured: see config>Special:MyLanguage/Help:Extension:Translate/Configuration#Page translation feature</>|Page translation feature.

Using Webfonts
User can choose webfonts for a language from the Language settings -> Display settings. By default, the first font in that menu will be applied to the wiki. A user can change the font and it will be remembered across the pages. Optionally user can disable the font embedding too by selecting system font.

If the font is available in user's local system, font will not be downloaded from the mediawiki server. It will be taken from the user's computer. Otherwise, font will be downloaded from the server only once. ie when the user selects the font first time. Next time onwards, font will be taken from the local cache.

By specifying font-family
Inside the wiki text YourText, webfonts extension will check whether the font is available with the extension, if so it will download it to the client. So the reader will not face any difficulty in reading the text even if the font specified is not available in their computer.

By specifying language
Inside the wiki text YourText, webfonts extension will check whether any font is available for the given language with the extension, if so it will download it to the client. So the reader will not face any difficulty in reading the text even if the font specified is not available in their computer. If there are multiple fonts for the language, the default font will be used. If default font is not preferred, use the font-family approach to specify the font. If the tag has both lang and font-family definitions, font-family get precedence.

Example:

𒄖𒉈𒅁𒌨𒅎

gives the text rendered in Cuneiform using Akkadian font

𒄖𒉈𒅁𒌨𒅎