User:APatro (WMF)/T204568 - Extend message checker framework

Objectives
Avoid failing builds due to mistakes by translators [ 1]. To achieve this, project maintainers can add error conditions that will not allow translators to save translations that have errors.

Sub objectives

 * 1) Add support for raising errors in the system during translation.
 * 2) Disallow saving translations that have errors.
 * 3) Allow administrators and fuzzy bot to override the checks.
 * 4) Allow project maintainers to specify errors and warnings in groups.yaml.
 * 5) Maintain backwards compatibility with older YAML file format treating existing checkers as warnings.
 * 6) Merge insertables and checkers since they are pretty same, $name should be insertable, warn if not used, and usually prevent saving.
 * 7) Reduce amount of custom code that has to be written currently to implement Insertables.

The rest of the page details on how these objectives have been achieved.

Validators
A new  framework has been added with the intent of replacing the existing   framework. Validators run on the translated message and based on the configuration, a warning or error message is shown to the translator. Translations with warnings can still be saved, but ones that have error cannot. Only a user with  right can save translations that have errors.

It is possible to add a regex in the configuration when configuring the Validator, and it can also be made an insertable. Hence while declaring a validator, it is possible to make  insertable, warn the translator if it is not used, and prevent saving.

Adding a custom validators is still possible and will be needed for more specialized validations.

Configuration
Following is a summarized validator configuration,

In the example above,


 * 1)   is a bundled validator that can accept a custom regex and run validations.
 * 2)   is a custom validator class.
 * 3)   is another bundled validator.

uses an array format. Lets look at the various parameters being used here in each array item,

Pre-provided / Bundled validators
Following is a list of bundled validators,

BraceBalance
ID:

Ensures that the number of open braces / brackets, matches the number of closed braces / brackets in the translation.

For example, the following translations would pass,



whereas, the following would fail,



As expected, this validator cannot be marked as insertable.

InsertableRegex
ID:

A generic reusable validator that can be used to specify custom validations and insertables.

For example, take the following configuration where the validator is marked as insertable and enforced,

Given the following source message - ''Hello $name. My name is $myName. that is being translated, the translation must have the parameters - $name and $myName''. They will also be displayed as insertables to make it easier for translators to use them in the translation. An absence of these parameters will cause an error to be displayed to the translator.

InsertableRubyVariable
ID:

This is a validator that matches ruby variables in the translations. Internally it extends  and uses the following regex -

MediaWikiMisc
ID:

<>

MediaWikiPlural
ID:

Ensures that if the source / definition contains a, the translation should also have it. It can also be used as an insertable.

UI
The UI has been updated to differentiate between errors and warnings. The icon associated with warning notices has been updated.

During translation, if an error is noticed with the translation, the Save translation button is disabled unless the user who is translating has   permission.

Additionally validation is also done on the server when the user is saving the translation. This is

During development a bug was noticed - T220789 - Invalid "more warnings" label shown during message translation which was fixed.

Custom validators
Custom validators must implement the  interface. Custom validators can also use the trait   that contains some commonly used methods. Below is an example of a custom validator, The format for the  array,

InsertablesSuggesters
The YAML structure for  can now be specified as an array. are used to provide insertables to translators. Many insertables for custom message groups on Translatewiki.net simply use a regular expression to extract parts of text and then use them as Insertables. For this, project maintainers have to create a custom  which involves writing PHP code.

With the array format in place and with the help of the  class it is possible to do this without writing any PHP code.

New configuration format
Consider the suggester and checker for the MathJax project.

The YAML file configuration for the could be updated to, The  can be defined as an array allowing users to specify multiple regular expressions to match insertables in the text. The structure of the  argument is the same as   defined under the   section.