VisualEditor/Design/Internationalization requirements

'''This is a work in progress. Even the first chapter is not fully written.'''

This is a draft document of internationalization requirements for the Visual editor.

Requirements for RTL and bidi text are written separately in the Visual editor/Bidirectional text requirements.

General
Existing standards (HTML, CSS, Unicode etc.) already provide pretty good support for internationalization. The same goes for the implementation of the standards in modern versions of common browsers. This is certainly true for Gecko and Webkit (with different bugs in both), and possibly more problematic in Opera and IE.

As much as possible the existing standards and functionalities must be reused and not reimplemented. In particular this regards the following features:
 * Caret appearance and movement: The caret (a.k.a cursor) has a stick shape in English. In bidirectional environments it may have an arrow pointing to the writing direction and in Chinese / Japanese / Korean environments it may have various shapes and behaviors according to the user's chosen input method. It's best to leave this to the OS and the browser.
 * Word boundary calculation: Different languages and scripts have very different understanding of word boundaries. Put simply, it's not necessarily spaces and punctuation marks. Browsers and operating systems are supposed to know this and act accordingly. It must not be reimplemented without need.
 * Keyboard behavior: In some keyboard layouts combinations with Ctrl and Alt keys may produce characters rather than actions. In some operating systems (notably Mac) combinations with arrows affect directionality.

General
It is becoming more and more important to use the HTML lang attribute as much as possible. The WebFonts extension uses it to apply the correct font. Firefox uses it to apply correct punctuation style and starting at version 12 it will (probably) use it to apply the correct spelling checker to a textarea. Mobile devices may use it to show the appropriate on-screen keyboard. Machine translation software may use it for correct translations.

In Wikipedia and in other projects text is often written in a mix of languages. For example, an encyclopedic article may be written in English and include names in foreign spelling, quotations of foreign literary works, titles of foreign-language books, etc.

It can be applied to almost any HTML element - the generic  and  , and also  ,   ,  , , ,   , etc.

Current situation
Currently MediaWiki automatically applies the lang attribute to the element.

To indicate elements in other languages it can be applied manually. This is done haphazardly - either using raw HTML tags or templates such as w:en:Template:Lang and its many derivatives, such as w:en:Template:Lang-sl, w:en:Template:Nihongo, s:en:Template:GHGheb etc. Needless to say, in every project they are implemented separately and differently, if at all.

How it should be
It must be possible to specify that the language of the article's content is different from the language of the wiki. This must somewhere in the page properties. The getPageLanguage can be a starting point, but its actual functionality is very limited. The Page Translation feature in the Translate extension applied language metadata to the pages it creates and this functionality could be moved to the core.

It must also be possible to specify the language of parts of the page using a visual dialog, that can also be used for defining fonts and other properties. Obviously, reusing generic HTML abilities would be a good idea.

The language of the currently selected text must be displayed somewhere; since language is not as visually obvious as font weight, it may be displayed on some kind of a status bar. The fact that the language of a part of a page is different can also be indicated by presenting it with a differently colored font or background.

Naturally, the lang attribute value is supposed to be useful not only for displaying, but for editing, too, for example for choosing the right font while the text is being edited.

Relationship with directionality
The HTML standard specifically says that text direction must not be deduced from the value of the lang attribute, so browsers don't do it. It's unfortunate, but that's how it is. The value of lang can be used to apply correct direction to element server-side.

By default, elements with different directionality should be directionally isolated, either using the tag or the "unicode-bidi: isolate" CSS rule.

See also Visual editor/Bidirectional text requirements.

Current situation
Currently, there are three ways to use fonts in the MediaWiki world:
 * For server-side rendering of SVG images, fonts can be installed on the server. For an example, see Bug 16284 - Support Linux Libertine Font for SVG rendering
 * The WebFonts extension can apply fonts according to lang attribute and CSS font-family declaration. It also allows changing the display font of the rendered page. The Universal Language Selector may change its current functionality.
 * MathJax is an externally developed product. It has its own mechanism to deliver fonts for rendering of mathematical symbols.

How it should be
The Visual editor should of course, support fonts for all languages. It should not, however, go to far and allow the user to choose any font. Wikimedia projects are, after all, rather conservative in their content. Readability and language diversity are essential goals for the projects, but design diversity is not. The Visual editor must, therefore, be flexible enough to allow the user to choose the language of a content element (lang attribute) and the content style (heading, body text, citation, poem, formula, source code example etc.), and to create new styles, if needed, but explicit and direct application of font names to the page is not required and possibly shoud be even prevented.

The functionality of the WebFonts extension should probably be merged into the core. Instead of connecting fonts with languages using this extension, the Language* classes should be aware of the fonts that are relevant for them.

Though math formulas are not exactly a linguistic element, their editing and font usage may be related to how fonts are applied to languages, so this should be remembered.