Help:Extension:Translate/API

From mediawiki.org

WORK IN PROGRESS

Introduction[edit]

The Translate extension is a complex extension. Some major functionality groups in it are:

  • creation, modification and deletion of different kinds of messages groups
  • parsing and generating various file formats
  • loading a collection of messages from the database
  • translation aids like translation memory
  • translation editor
  • message mass processing capabilities (import, export, fuzzying)
  • statistics collection and presentation
  • page translation (splitting and generating pages, creating message groups)

The functions are accessed either directly via PHP code (Internal-API) or via the web facing API framework (Web-API). There are also other web facing entry points that do not use the W-API framework (Special-API) and command line scripts (Cli-api). It is in principle possible to modify the database tables or files directly, but that is not recommended nor aim to be supported.

Users access the functionality via dedicated interface on MediaWiki itself, which consists of special pages and added/removed/modified interface elements on other pages, added via various hooks that MediaWiki provides. Some of the interface code calls the W-API to execute its actions, providing ideal separation of concens with the downside that W-API always needs JavaScript support on the browser (but it can also be used from other software without this issue), necessitiating S-API kind of alternative for users without JavaScript. Some of the newest features do not fallback gracefully when JS is not available.

The order of importance to third party developers is assumed to be Special-API, Cli-API, Internal-API and Web-API. I will present in this order what is already available, how flexible they are and how well they are documented. After that I will switch to functional inspection which includes features that are not yet available at all, and on which layers they should be.

Cli-API[edit]

Command line scripts are suitable for executing time consuming operations, like exporting translations into files or bootstrapping translation memory database. They are not suitable for small actions due to (1) difficulty of passing parameters (2) no standard way of doing error reporting and (3) startup overheard gets significant when doing thousand of actions. Cli-API is primarly intented for system administrators, advanced translation administrators and automatic tasks.

Each command line script has --help switch that tells what the command does and what are the parameters. There is general help somewhere how to run command line scripts, but not special help for Translate extension, though it doesn't differ from MediaWiki itself, but finding out that information might be a problem for new users.

There is no detailed overview of the command line scripts in Translate extension. The following list can be used as a staring point.

Maintenance scripts

  • createCheckIndex.php - Runs message checks for each message updating the database state
  • createMessageIndex.php - Message index is needed internally by Translate and it is not always regenerated automatically
  • messadeDust.php - Find unused messages or messages in wrong place
  • populateFuzzy.php - Updates the database fuzzy tag status for each messages
  • ttmserver-export.php - Fills the translation memory with current translations

Message group related

  • export.php
  • fuzzy.php
  • magic-export.php
  • processMessageChanges.php
  • sync-group.php

Sysadmin stuff

  • logfilter.php - for sysadmins
  • migrate-schema2.php - database schema upgrade
  • pagetranslation-check-database.php - database consistency check script
  • list-mwext-i18n-files.php - needed for MediaWiki i18n files for some time yet

Stats

  • languageeditstats.php
  • groupStatistics.php

Tests (should be moved to unit tests)

  • pagetranslation-test-parser.php
  • yaml-tests.php

Other

  • plural-comparison.php
  • cldr-plural-to-yaml.php

A very limited set of functionality is accessible via cli scripts.

Special-API[edit]

Almost all of the user facing functionality is primarly exposed through the special pages with some extensions done as command line scripts, due to restrictions of web environment. Special pages interfaces doesn't really constitute an API that third party users can rely on. Using web pages programmatically is called screencraping and that is highly discouraged. Having good API coverage that is well documented and advertised helps to avoid others to rely on screenscaping, which might cause a compatability problem which refactoring the special pages.

Very high level overview of the special pages:

  • SpecialAggregateGroups.php - Message group management
  • SpecialImportTranslations.php - Message group management
  • SpecialManageGroups.php - Message group management
  • SpecialMagic.php - Very little relevant for 3rd party use
  • SpecialMessageGroupStats.php - Statistics
  • SpecialSupportedLanguages.php - Statistics
  • SpecialTranslations.php - Statistics kind of
  • SpecialLanguageStats.php - Statistics
  • SpecialTranslationStats.php - Statistics
  • SpecialMyLanguage.php - Page translation related
  • SpecialPageTranslation.php - Page translation related
  • SpecialTranslate.php - The main translation interface, including the editor
  • SpecialFirstSteps.php - Translatewiki.net specific signup page

Special pages expose only very little information and functionality, very often not enough to do alternative implementations. Special pages have quite good user documentation scattered around different help pages of Translate extension in mediawiki.org. The calling conventions of them are not documented and wont be documented. One specific case is the, or actually, are the translation editors, which are provided as HTML code by Special:Translate in a way that is highly tied to rest of the Translate extension.

Internal-API[edit]

Internal API offers access basically to all functionality what Translate extension has - and I'm not going to repeat that. But there are certain ways and patterns how the internal api is made available and intented to be used.

Hooks. Hooks are way for programmers to execute their own code in predefined places decided by the Translate extension authors. Hooks provide well-defined injection points - at least in principle. Often the relevant data is not passed to the users code, or the parameters are highly tied to the current implementation and those might break when the implementation is refactored. Translate extension uses many hooks of MediaWiki, but only provides few new hooks itself which are currently only used internally by the extension. Because the extension code itself can access everything it needs, hooks aren't usually added except in obvious cases, or when somebody requests them. The hooks in Translate extension are not documented anywhere. The most important hook that currently exists is the hook which can be used to register new message groups.

Interface and subclasses. Another way to extend the Translate extension code is to write classes that implement some pre-defined interface (in general sense). Translate extensions currently has four interfaces (specific sense):

  • iTTMServer
  • StringMangler (making sure the message keys conform to limitations of wiki titles)
  • MessageGroup
  • FFS (file format support)

The interface defines what methods the user's code must implement, what parameters they get and what they return (not enforceable in PHP). The most useful interface currently is the FFS interface that defines how a file format parser and generator is called. Also MessageGroup interface is common use.

But just providing an interface is not all we can do. Translate extension comes with many classes that implement those interfaces, like SimpleFFS, FileBasedMessageGroup, AggregateMessageGroup and WikiMessageGroup. The user code and extend these classes, just overriding specific parts to implement the behaviour the user wants. This sometimes also works for classes that don't implement any predefined interface (specific sense). It should be noted that the Translate extension must be coded in a way that expects different classes by providing way to register those custom written classes and by providing a way to indicate in some configuration which class to use. The YAML based message group configuration allows registering new classes and defining which class to use for MessageGroup, FFS and StringMangler.

In theory one could also replace one of the core classes with custom class, but this is not recommended nor supported. The target should be that the core classes are extensible enough with hooks, and things like FFS use interfaces and subclassing.

The interfaces and classes are reasonable well documented via code documentation. There could be an overview interfaces and tutorials for them.

Web-API[edit]

Since Translate extension is often provided as service and it acts as the interface (general sense) for the data it contains, being able to call that service over web (in this case http(s)) is of utmost importance.

Here's a list of currently available Web-APIs:

  • ApiAggregateGroups.php - Creation and deletion of aggregate groups and defining their subgroups (write only)
  • ApiGroupReview.php - Changing of message group states (write only)
  • ApiHardMessages.php
  • ApiQueryLanguageStats.php
  • ApiQueryMessageCollection.php - Querying of message of a group, provides filtering and stuff (read only)
  • ApiQueryMessageGroups.php - Information about message groups (read only)
  • ApiQueryMessageGroupStats.php - Statistics of message groups (read only)
  • ApiQueryTranslationAids.php
  • ApiQueryMessageTranslations.php - Special:Translations equivalent (read only)
  • ApiStatsQuery.php
  • ApiTranslateSandbox.php
  • ApiTranslateUser.php
  • ApiTranslationStash.php
  • ApiTranslationReview.php - Reviewing of translations (write only)
  • ApiTTMServer.php - Querying of translation memories (read only)

These are somewhat documented when you access http://translatewiki.net/w/api.php, e.g. [1], though we understand it's hard to get the high level picture of what each module does.

Auto generated documentation[edit]

action=groupreview

(main | groupreview)
  • This module requires read rights.
  • This module requires write rights.
  • This module only accepts POST requests.
  • Source: Translate
  • License: GPL-2.0-or-later

Set message group workflow states.

Specific parameters:
Other general parameters are available.
group

Message group. See action=query&meta=messagegroups.

This parameter is required.
language

Language code.

Default: en
state

The new state for the group.

This parameter is required.
token

A "csrf" token retrieved from action=query&meta=tokens

This parameter is required.
Example:
Mark the state of the German translation for the message group "group-Example" as ready
api.php?action=groupreview&group=page-Example&language=de&state=ready&token=foo [open in sandbox]

action=translationreview

(main | translationreview)
  • This module requires read rights.
  • This module requires write rights.
  • This module only accepts POST requests.
  • Source: Translate
  • License: GPL-2.0-or-later

Mark translations reviewed.

Specific parameters:
Other general parameters are available.
revision

The revision number to review.

This parameter is required.
Type: integer
token

A "csrf" token retrieved from action=query&meta=tokens

This parameter is required.
Example:
Review revision 1
api.php?action=translationreview&revision=1&token=foo [open in sandbox]
Module "translatesandbox" not found.

action=searchtranslations

(main | searchtranslations)

Search translations.

Specific parameters:
Other general parameters are available.
service

Which of the available translation services to use.

One of the following values: codfw, default, eqiad
Default: default
query

The string to search for.

This parameter is required.
sourcelanguage

The language code of the source text.

Default: en
language

The language code to search string for.

Default: (empty)
group

The group ID to search string in. See action=query&meta=messagegroups.

Default: (empty)
filter

Message translation status filter.

One of the following values: Can be empty, or fuzzy, translated, untranslated
Default: (empty)
match

Match any/all search words.

Default: (empty)
case

Case (in)sensitive search.

Default: 0
offset

Offset for the translations.

Type: integer
Default: 0
limit

Size of the result.

Type: integer or max
The value must be between 1 and 50.
Default: 25
Examples:
Show translations for the language.
api.php?action=searchtranslations&language=fr&query=aide [open in sandbox]
Show untranslated messages matching the query in the source language.
api.php?action=searchtranslations&language=fr&query=edit&filter=untranslated [open in sandbox]
Module "translationstash" not found.

action=translationaids

(main | translationaids)

Query all translations aids.

Specific parameters:
Other general parameters are available.
title

Full title of a known message.

This parameter is required.
group

Message group the message belongs to. If empty then primary group is used.

prop

Which translation helpers to include.

Values (separate with | or alternative): definition, definitiondiff, documentation, editsummaries, gettext, groups, inotherlanguages, insertables, mt, support, translation, ttmserver
Default: groups|definition|translation|inotherlanguages|documentation|mt|definitiondiff|ttmserver|support|gettext|insertables|editsummaries
Example:
Show aids for MediaWiki:January/fi
api.php?action=translationaids&title=MediaWiki:January/fi [open in sandbox]

action=ttmserver

(main | ttmserver)

Query suggestions from translation memories.

Specific parameters:
Other general parameters are available.
service

Which of the available translation services to use.

One of the following values:
sourcelanguage

The language code of the source text.

This parameter is required.
targetlanguage

The language code of the suggestion.

This parameter is required.
text

The text to find suggestions for.

This parameter is required.
Example:
Get suggestions for translating "Help" from English to Finnish
api.php?action=ttmserver&sourcelanguage=en&targetlanguage=fi&text=Help [open in sandbox]

meta=messagetranslations (mt)

(main | query | messagetranslations)

Query all translations for a single message.

Specific parameters:
Other general parameters are available.
mttitle

Full title of a known message.

This parameter is required.
mtoffset

When more results are available, use this to continue. More detailed information on how to continue queries can be found on mediawiki.org.

Type: integer
Default: 0
Example:
List of translations in the wiki for "MediaWiki:January"
api.php?action=query&meta=messagetranslations&mttitle=MediaWiki:January [open in sandbox]

list=messagecollection (mc)

(main | query | messagecollection)
  • This module requires read rights.
  • This module can be used as a generator.
  • Source: Translate
  • License: GPL-2.0-or-later

Query MessageCollection about translations.

Specific parameters:
Other general parameters are available.
mcgroup

Message group. See action=query&meta=messagegroups.

This parameter is required.
mclanguage

Language code.

Default: en
mclimit

How many messages to show (after filtering).

Type: integer or max
The value must be between 1 and 5,000.
Default: 500
mcoffset

When more results are available, use this to continue. More detailed information on how to continue queries can be found on mediawiki.org.

Default: (empty)
mcfilter

Message collection filters. Use ! to negate condition. For example !fuzzy means list only all non-fuzzy messages. Filters are applied in the order given.

fuzzy
Messages with fuzzy tag.
optional
Messages which should be translated only if changes are necessary.
ignored
Messages which are never translated.
hastranslation
Messages which have a translation regardless if it is fuzzy or not.
translated
Messages which have a translation which is not fuzzy.
changed
Messages which have been translated or changed since last export.
reviewer:N
Messages where the user number N is among reviewers.
last-translator:N
Messages where the user number N is the last translator.
Separate values with | or alternative.
Maximum number of values is 50 (500 for clients that are allowed higher limits).
Default: !optional|!ignored
mcprop

Which properties to get:

definition
Message definition.
translation
Current translation (without !!FUZZY!! string if any, use the tags to check for outdated or broken translations).
tags
Message tags, like optional, ignored and fuzzy.
properties
Message properties, like status, revision, last-translator. Can vary between messages.
revision
Deprecated. Use mcprop=properties.
Values (separate with | or alternative): definition, properties, tags, translation, revision
Default: definition|translation
Examples:
List of supported languages
api.php?action=query&meta=siteinfo&siprop=languages [open in sandbox]
List of non-optional message definitions for the group "page-Example"
api.php?action=query&list=messagecollection&mcgroup=page-Example [open in sandbox]
List of optional messages in Finnish with tags for the group "page-Example"
api.php?action=query&list=messagecollection&mcgroup=page-Example&mclanguage=fi&mcprop=definition|translation|tags&mcfilter=optional [open in sandbox]
More information about latest translation revisions for the group "page-Example"
api.php?action=query&generator=messagecollection&gmcgroup=page-Example&gmclanguage=nl&prop=revisions [open in sandbox]

meta=languagestats (ls)

(main | query | languagestats)

Query language stats.

Specific parameters:
Other general parameters are available.
lsoffset

When more results are available, use this to continue. More detailed information on how to continue queries can be found on mediawiki.org.

Default: 0
lslanguage

Language code.

This parameter is required.
lsgroup

Message group ID. See action=query&meta=messagegroups.

Examples:
List of translation completion statistics for Finnish
api.php?action=query&meta=languagestats&lslanguage=fi [open in sandbox]
List of translation completion statistics for Finnish for group A
api.php?action=query&meta=languagestats&lslanguage=fi&group=A [open in sandbox]

meta=messagegroups (mg)

(main | query | messagegroups)

Return information about message groups.

Note that the uselang parameter affects the output of language dependent parts.

Specific parameters:
Other general parameters are available.
mgdepth

When using the tree format, limit the depth to this many levels. Value 0 means that no subgroups are shown. If the limit is reached, the output includes a "groupcount" value, which states the number of direct children.

Type: integer
Default: 100
mgfilter

Only return messages with IDs that match one or more of the inputs given (case-insensitive, separated by pipes, * wildcard).

Separate values with | or alternative.
Maximum number of values is 50 (500 for clients that are allowed higher limits).
Default: (empty)
mgformat

In the tree format message groups can exist in multiple places in the tree.

One of the following values: flat, tree
Default: flat
mgiconsize

Preferred size of rasterised group icon.

Type: integer
Default: 64
mgprop

What translation-related information to get:

id
Include ID of the group.
label
Include label of the group.
description
Include description of the group.
class
Include class name of the group.
namespace
Include namespace of the group. Not all groups belong to a single namespace.
exists
Include self-calculated existence property of the group.
icon
Include URLs to icon of the group.
priority
Include priority status like discouraged.
prioritylangs
Include preferred languages. If not set, this returns false.
priorityforce
Include priority status - is the priority languages setting forced.
workflowstates
Include the workflow states for the message group.
sourcelanguage
Include the source language for the message group.
subscription
Include whether the user is subscribed to the group.
Values (separate with | or alternative): class, description, exists, icon, id, label, namespace, priority, priorityforce, prioritylangs, sourcelanguage, subscription, workflowstates
Default: id|label|description|class|exists
mgroot

When using the tree format, instead of starting from top level start from the given message group, which must be an aggregate message group. When using flat format only the specified group is returned.

Default: (empty)
mglanguageFilter

Only return message groups that can be translated to the given language.

Default: (empty)
Example:
Show message groups
api.php?action=query&meta=messagegroups [open in sandbox]

meta=messagegroupstats (mgs)

(main | query | messagegroupstats)

Query message group stats.

Specific parameters:
Other general parameters are available.
mgsoffset

When more results are available, use this to continue. More detailed information on how to continue queries can be found on mediawiki.org.

Default: 0
mgsgroup

Message group ID. See action=query&meta=messagegroups.

This parameter is required.
mgssuppresscomplete

Do not display languages which have completed translation

Type: boolean (details)
mgssuppressempty

Do not display languages which do not have any translations

Type: boolean (details)
Example:
List of translation completion statistics for the group "page-Example"
api.php?action=query&meta=messagegroupstats&mgsgroup=page-Example [open in sandbox]
Retrieved from "https://www.mediawiki.org/w/index.php?title=Help:Extension:Translate/API&oldid=6393700"