Jump to content

MinT

From mediawiki.org

MinT (Machine in Translation) is a machine translation service based on open-source neural machine translation models. The service is hosted in the Wikimedia Foundation infrastructure, and it runs translation models that have been released by other organizations with an open-source license. An open machine translation service can be a key piece of the essential infrastructure of the ecosystem of free knowledge. This page captures the initiatives to scale the service and make this infrastructure more widely available.

You can try MinT as part of projects such as Content Translation and translatewiki.net, or directly in a test instance.

Overview of MinT initiatives[edit]

Machine translation can be useful in different contexts. As more products make use of MinT for different purposes, it is useful to differentiate those different contexts. In this way, when users report a bug it is more clear where it needs to be fixed.

  • MinT Service. The backend service running open-source neural machine translation models.
    • MinT test instance. A basic interface to try the different translation models.
  • MinT for Translators. Initiative to integrate the MinT Service with tools that support other machine translaiton services such as Content Translation and the Translate Extension.
    • MinT Client for Content Translation. Client exposing the MinT Service as one of the machine translation services available in Content Translation.
    • MinT Client for Translate extension. Client exposing the MinT Service as one of the machine translation services available in the Translate extension.
  • MinT for Wiki Readers. Product to enable readers to use machine translation to read contents from other languages on a wiki.

You can read more below about each of the MinT initiatives.

Get involved[edit]

Feel free to share any feedback in the discussion page. Planned improvements are captured in Phabricator (more info), you can report wrong behavior or propose feature enhancements, track the progress of any task, and share your perspective on it. For completed work you can also check the status updates below.

MinT Service[edit]

The MinT Service is designed to provide translations from multiple machine translation models. Currently, it uses the following models:

MinT supports over 200 languages, with more than 70 languages not supported by other services (including 27 languages for which there is no Wikipedia yet). You can read more about the initial release of MinT and check some frequently asked questions in the summary page for the service.

Technical details[edit]

The translation models have been optimized for performance using OpenNMT Ctranslate2 library in order to avoid the need for GPU acceleration. This makes it easier for organizations and individuals to build and run their own instances. For more details you can check the following:

MinT provides a platform to run multiple translation models. In order to support different initiatives, aspects such as sentence segmentation, language detection, pre/post-processing of contents, and rich format support has been developed on top of the plain-text based models.

Test instance[edit]

The MinT test instance is a basic interface to try the different translation models. It allow to translate contents across the selected language pairs and select the preferred translation model when multiple are available. This allows different communities to check how well the models support their language. This instance is intended for testing, so performance and availability may be reduced compared to other MinT-based products. You can check the availability status of the MinT test instance.

MinT for translators[edit]

Mobile translation using MinT

Translation is a common way to contribute in the Wikimedia ecosystem for multilingual users. Machine translation can provide a useful initial translation for users to review and improve. The Language team has developed tools to support translations in their workflows that can integrate different machine translation services to speed up their processes. Once MinT was available, integrating it with these tools was a logical next step to amplify their impact. MinT is available in the following projects:

MinT for wiki readers[edit]

The number of topics and the amount of information a reader can learn about from Wikipedia and other wikis depends on the languages they speak. Machine translation can help people to learn more about their topics of interest when the content is not available in their language.

This initiative explores how to surface the machine translation support from MinT in Wikipedia articles in a way that:

  • Allows readers to learn more about the topics of interest from other languages
  • Clearly differentiates automatically generated content from community-created one.
  • Encourages to access and contribute to community-created content when possible.

At the moment the Language team is working on the initial implementations for this initiative based on the research and the designs. Learnings based on data and community input will determine the next steps for the initiative.

MinT more widely available[edit]

Working on the previous initiatives will help to polish and solidify the system. For now, the MinT API is only available for Wikimedia products. As the system gets ready, we'll consider a wider exposure. Providing a service that can be used by communities in innovative ways can be a very powerful tool. New initiatives to make MinT more widely available will be captured here in the future. Meanwhile, feel free to configure your own MinT instance to experiment with it.

Status updates[edit]

February 2024[edit]

January 2024[edit]

December 2023[edit]

November 2023[edit]

October 2023[edit]

  • Launched the Language Identification service to automatically detect in which language is written a given text. The service supports the detection of 201 languages, and anyone can access the API to use the service or read the model card for more details. Machine Learning team completed the last checks after deploying to LiftWing and evaluating that the service can "easily withstand a high amount of traffic".
  • Basic support for rich text translation by supporting transferring of markup to apply styling such as words in bold from the source text into the equivalent ones in the machine translation (which lacks format since translation models operate with plain-text).
  • Completed the process to enable MinT for languages with no Wikipedia yet . Translation models in MinT support 25 languages for which there is no Wikipedia. These can be tested in MinT's test instance for speakers of those languages to assess quality, and ensures that translation tools are well-equipped once wikis are created for those languages (as it has been the case with the recent graduation of Fon Wikipedia out of incubator).
  • Completed the process to enable MinT for closely-related languages based on Community input . For some languages where machine translation is not available, Wikipedia editors have asked to have access to machine translation in Content Translation using a related language instead of having no support at all. With this enablement translators of Gan (gan) Wikipedia will have machine translation based on the traditional script variant of Chinese as a starting point.
  • Analysis of translation activity on 55 languages for which MinT provides machine translation for the first time shows how (a) translations have increased 2X since MinT is available, and (b) deletion rates have not increased. Activity levels for these 55 wikis changed from ~500 translations/month, to 1K+ translations/month after MinT was enabled. For example, a recent peak of 2.15K translations were published in August 2023 when MinT was available for those languages, which is a significant increase from 225 translations in August 2022 when MinT was not available for them.
  • Better visibility of translation quality by including a tag in translations where unedited machine translation is close to the limits. This will facilitate analysis about translation quality and limits.

September 2023[edit]

August 2023[edit]

July 2023[edit]