Language tools

The Language team has the goal of reducing the language barrier that users face when accessing or sharing knowledge. The team has created tools to facilitate the participation of our users regardless of their language. These include the creation of multilingual content and software, facilitate language navigation, and support for easy input and proper display of different scripts. Language tools and their infrastructures need regular maintenance and update efforts. Making our language tools and infrastructure work better helps all the projects that rely on them, and saves time and efforts to the people participating in them, from developers to translators and the final users.

In July 2018, an effort was started to identify key areas to improve in our language tools during the upcoming months. The affected tools, the plan and the status updates are described next.

List of tools

Language tools are listed below:

• Translate extension. The infrastructure used to translate our software and multilingual pages. Communities of translators use it on translatewiki.net and Wikimedia Meta-wiki.
• Universal Language selector. Provides easy access to language settings, and facilitates language selection in different contexts including interlanguage links.
• Project Milkshake. Developer libraries for internationalisation support on the web including input methods, web-fonts, and grammatical rules for languages that can be used on both Wikimedia and external projects.
• MediaWiki Language Extension Bundle. A collection of selected MediaWiki extensions needed by any wiki which desires to be multilingual.
• Content Translation. The tool to facilitate translation of Wikipedia articles across languages. Since it is in active development, the progress of this tool won't be tracked as part of the current maintenance effort, but as part of its own project page for the new version.

Maintenance plan

Since there is a wide catalog of language tools and potential areas to improve we plan to operate in the following way:

1. Language tools audit. Identify areas and their potential impact in order to have clarity on the areas we want to focus our maintenance work regarding language tools.
2. Get feedback. Share our plans and get input from the different people involved, including the users of the tools and people working in other Wikimedia projects.
3. Work on the selected areas. Once areas are identified and the people involved are in the same page, the Language team will work on the areas defined.

Your input is important in this process, so feel free to add your comments to the talk page.

Status updates

May 2020

• Released the most recent version of MLEB.
• Corrected the popup box for subprojects disappearing immediately on Translatewiki.
• Updated translate extensions to not reparse certain files, reducing errors.
• Removed incorrect count columns from Special:Statistics to reduce confusion caused by displaying out of date information.
• Enabled Kiwix Desktop 2.0 for use with Translatewiki.
• Enabled default fonts for translatewiki, correcting a past error.
• Resolved Mediawiki console errors, improving functionality and decreasing spam in error logs.
• Resolved errors that lead to failed Translation statistics reports, improving reporting.
• Corrected errors that caused translated subpages to not move correctly with the parent pages.
• Updated code to now support offline translation exports from the command line.
• Corrected the fatal error of an uncommon code path in the Translation extension due to a missing autoloader entry.
• Removed unnecessary language statistics and validation scripts from MediaWiki core since checks are performed on the translatewiki.net side.
• Translation memory suggestions on Wikimedia projects are no longer duplicated and now suggestions are loaded faster.

April 2020

• Making the list of supported languages more reliable by solving fatal errors.
• Improved QA process by providing access to code changes before deploying them to all users ("canary")
• Better documentation and code quality for managing localization repositories.
• Better access to translation context by fixing the access to instructional documentation.
• Added localization support for Page Content Service.
• Increased the quantity and quality of support apps by adding Bitmark/SPRING support for mobile apps
• Increased translation visibility by adjusting the workflow surrounding translation exporting
• Improve code quality to reduce PHP warnings
• Better offline translation support by fixing issues in the export process
• Increased stability of some translate extensions by fixing errors
• Improved accuracy for the code that updates language name
• Improve code quality to fix errors associated with certain database transactions
• Fixed code errors that prevented adjusting the title and namespace when moving a page in the Chavacano de Zamboanga (cbk-zam) Wikipedia
• Refactored and updated code for Translatewiki.net to increase speed of queries and reduce the chances for outages.

March 2020

• Analyzed the performance issues with Central Notice localization messages, identifying the cost of dealing with a large number of smaller message groups and defining follow-up steps to improve the way those are handled.
• Migrated WikimediaTools projects to the new framework and added plural validation for the ajapaik-app project for more flexible validation of translated messages. Projects previously migrated to the new validator framewok present no issues after the observation period.
• Removed the deprecated checker code to complete the migration to the new validator framework after the observation period.
• Reorganization of insertables (quick suggestions for the user to add when translating) to reduce redundancy of frequent cases defined by each project.
• More reliable translations by applying the new validator framework to check variables in the ajapalk-app project.
• Localization support for QRMedia project added.
• Improved compatibility of Translate with recent PHP versions.
• Added localization support for the COVID-19 poster from the Finnish Institute for Health and welfare.
• Better support for message renames by fixing an error when fetching content of unavailable messages.

February 2020

• Cleaner codebase by removing backward compatibility code to support old Mediawiki versions.
• Better communication of language changes by asking for user confirmation when accessing a link that request a change of language.
• Migrated 9 projects to the new framework for more flexible validation of translated messages: Pywikibot, Citation Hunt, Internet Archive Bot, GapFinder, Wikipedia Library Card Platform, MantisBT, Mazi, Mifos, and FUDForm.
• Migrated Android Mobile and lib.reviews projects to the new framework for more flexible validation of translated messages.

January 2020

• Migration of Open Street Map (website) and FreeCol projects to the new validator framework.
• Improved support for validating plural forms in translation messages to ensure consistency with the source message and the plural rules of each language.
• Updated auto-size libraries and data formats used.
• Improved performance of translation export process to prevent timeouts.
• Cleaner configuration for MLEB releases.

December 2019

• Simplified access to similar messages.
• More reliable import process for translations by ensuring the information is immediately updated at the end of the process.
• Improved saving process by keeping the list of languages immediately up to date.
• Improved performance when selecting a message group to translate.
• Improved validation by allowing to enforce that translation keys use certain values, and limit the validation to specific messages.
• Migration of Nocc to the new validator framework.
• More reliable diffs for translation messages.
• Support for language variants in the title of multilingual pages which was not possible to convert before.
• Technical updates for translatewiki.net related to SSL and searching.
• Improved stability with improvements to performance when loading translators, processing fuzzy translations, and less noisy certificate-related notifications.

November 2019

• Improved import process for translations by better presenting and handling messages that were renamed.
• Better context for translators by providing access to other messages using the proposed machine translation suggestions.
• Better approval process for translators by correcting issue with duplicate approvals.
• Support localization of DiscordWikiBot
• Improved integration of language selection into the personal toolbar by adjusting the visual alignment.
• Better representation of HTML special characters in translation suggestions.
• Fixed issues with extension missing path information.
• Improved control on translations with an option to allow translations only for the defined priority languages.
• More reliable export process by preventing failures in one project to prevent exporting others.
• Improved backup system for translatewiki.net

October 2019

• Improved database access for Translate
• Support for Global CheckUser on translatewiki.net
• Translation support for WhoWroteThat extension.
• Support for changes in AbuseFilter and SpamBlacklist APIs
• Reduce visual noise by removing separator in translatable pages.
• Support for localization of KaiOS Wikipedia app and Femiwiki.
• Compatibility support for language tools with new PHP versions, with older Mediawiki version and with function deprecation of new Mediawiki versions.
• More clearly surface duplicated content errors before users mark pages for translation.

September 2019

• Export repo migration for Encyclopedia of Life project.
• Avoid running Hip Hop Virtual Machine (HHVM) specific tests when such virtual machine is not used.
• Deprecation alerts as part of the migration process to use the new registration system for extensions.
• Improved support for moving translated categories and database load balancing.
• Improvements in the validation framework to check consistency of empty lines, and migration of a new project (Dissemin) to use the new validation framework.
• More reliable support for marking translations as discouraged.
• Migration of a new project (WMCZ-Tracker) to the new validator framework.

August 2019

• Improvements in the sign up process for Translate extension
• Fixed issues that made documentation for translations link to the wrong namespace, and issues from deprecated parsing functions.
• Better alignment with new standards by using the new extension registration system for Translate and for translatewiki.net, avoid using deprecated functions when moving translatable pages, and fixing issues with export functions on new PHP versions and the Hip Hop virtual machine (HHVM).

July 2019

• Improved translation message integrity by supporting custom checks with a new validation framework.
• More fluent account creation process by better communicating the wait to the user and making username available for previously rejected user applications.
• Better support for exporting by preventing issues with empty groups of messages to translate and support for Phabricator translations.

May 2019

• More reliable sharing/bookmarking of messages to translate by ensuring the link considers the current language when it changes.
• Better support for source language by preventing actions that do not apply such as proofreading, or changing the translation status.
• More clarity to communicate priority languages by showing a proper notice when translation to non-priority languages is blocked, avoid showing the message when already translating to a priority language, avoid showing the notice when translating documentation, and including also other languages in the stats.
• Better support for importing translation updates by clarifying the possible merge actions, and the link to import pre-existing translations.
• More reliable language selection by avoiding Simple English to be missing in the interlanguage links when it should be available.

April 2019

• Improved font support for Church Slavonic.

February 2019

• Better handling of searches with a large number of results.

January 2019

• More reliable update for translation statistics.

December 2018

• Support for Microsoft translation restored.
• Message group stats improvements to include review stats and avoid timeout errors.
• More reliably showing the progress of translations.

November 2018

• Improvements on code review infrastructure to prevent false positives in code review.
• Improvements in the list of message groups to provide more clarity by adjusting the colors.
• Better support for moving translatable sub-pages by avoiding moves to affect the parent page.
• Better proofreading support by avoiding translations to be automatically marked as reviewed when they are not.
• More reliable access to message groups by avoiding errors due to the truncation of long IDs, improved query performance, and avoided warnings.

October 2018

• More reliable support for moving and deleting translated pages.

September 2018

• List of proposed interventions documented on wiki for discussion.

July 2018

• Initial discussion on initial potential areas based on the Language goals and whishlist page.