Extension talk:Translate

Jump to: navigation, search

About this board

Archive.


Debug machine translation services, translation aids (Yandex mt service)

6
Andreas Plank (talkcontribs)

Hi,

I've installed Translate extension (MLEB 2017.07, MW 1.28.2) and configured the machine translation service for Yandex according to documentation but I get no api activity or network activity whatsoever. If I call the translation aid http://localhost/w/api.php?action=translationaids&format=json&title=Translations:Test-Übersetzung/2/en I get a proper json but the mt array is always empty. When I activate debug logs I get an error message (by Yandex I guess “Unsupported Media Type” see https://tech.yandex.com/market/partner/doc/dg/concepts/error-codes-docpage/):

…
[translationservices] Translation service Yandex problem: Unsupported Media Type
[DBPerformance] Expectation (writes <= 0) by ApiMain::setRequestExpectations not met:
query-m: REPLACE INTO `objectcache` (keyname,value,exptime) VALUES ('X')
#0 /usr/share/mediawiki/includes/libs/rdbms/TransactionProfiler.php(215): TransactionProfiler->reportExpectationViolated('writes', 'query-m: REPLAC...')
#1 /usr/share/mediawiki/includes/libs/rdbms/database/Database.php(959): TransactionProfiler->recordQueryCompletion('query-m: REPLAC...', 1508407232.3874, true, 1)
#2 /usr/share/mediawiki/includes/libs/rdbms/database/Database.php(870): Database->doProfiledQuery('REPLACE INTO `o...', 'REPLACE /* SqlB...', true, 'SqlBagOStuff::s...')
#3 /usr/share/mediawiki/includes/libs/rdbms/database/Database.php(2162): Database->query('REPLACE INTO `o...', 'SqlBagOStuff::s...')
#4 /usr/share/mediawiki/includes/libs/rdbms/database/DatabaseMysqlBase.php(486): Database->nativeReplace('objectcache', Array, 'SqlBagOStuff::s...')
#5 /usr/share/mediawiki/includes/objectcache/SqlBagOStuff.php(367): DatabaseMysqlBase->replace('objectcache', Array, Array, 'SqlBagOStuff::s...')
#6 /usr/share/mediawiki/includes/objectcache/SqlBagOStuff.php(381): SqlBagOStuff->setMulti(Array, 4500)
#7 /var/lib/mediawiki/webroot/extensions/Translate/webservices/TranslationWebService.php(324): SqlBagOStuff->set('my_wiki:transla...', '1|1508407232', 4500)
#8 /var/lib/mediawiki/webroot/extensions/Translate/webservices/TranslationWebService.php(109): TranslationWebService->reportTranslationServiceFailure('Unsupported Med...')
#9 /var/lib/mediawiki/webroot/extensions/Translate/translationaids/MachineTranslationAid.php(71): TranslationWebService->getResultData(Object(TranslationQueryResponse))
#10 /var/lib/mediawiki/webroot/extensions/Translate/translationaids/MachineTranslationAid.php(55): MachineTranslationAid->formatSuggestion(Array)
#11 /var/lib/mediawiki/webroot/extensions/Translate/api/ApiQueryTranslationAids.php(101): MachineTranslationAid->getData()
#12 /usr/share/mediawiki/includes/api/ApiMain.php(1435): ApiTranslationAids->execute()
#13 /usr/share/mediawiki/includes/api/ApiMain.php(510): ApiMain->executeAction()
#14 /usr/share/mediawiki/includes/api/ApiMain.php(481): ApiMain->executeActionWithErrorHandling()
#15 /usr/share/mediawiki/api.php(83): ApiMain->execute()
#16 {main}

Any proposal what I can do to resolve the error ?

Andreas Plank (talkcontribs)

Nobody has an idea? Related to “unsupported media type”, how can I check that the outgoing request to yandex has the proper file type? I have just guessed it could be related to mime-type definition/configuration of my server setup. The FireFox debugger console does not show any request other than localhost.

How can I check that Yandex request in detail when it is not showing up in debugger console?

Nikerabbit (talkcontribs)

I am not using Yandex myself currently. It is possible that they have changed the API and the current code needs to be updated.

Nikerabbit (talkcontribs)

The code in YandexWebService::getQuery() looks okay though.

Andreas Plank (talkcontribs)

Debugging the TranslationQueryResponse object in TranslationWebService::getResultData I figured out that in webservices/YandexWebService.php that return TranslationQuery::factory is somewhat mis-coded. Now I get it working by using queryParamaters() instead of postWithData() like so:

protected function getQuery( $text, $from, $to ) {
// ... clipped code
return TranslationQuery::factory( $this->config['url'] )
  ->timeout( $this->config['timeout'] )
  ->queryParamaters(
    [
      'key' => $this->config['key'],
      'text' => $text,
      'lang' => "$from-$to",
      'format' => 'html',
    ]
  );
}

It needs no 'body' but 'parameters' were not set at all. Do you agree? There is also a misspelled typo: queryParamaters ;-)

Perhaps other webservices have the same code issues.

Andreas Plank (talkcontribs)

I filed a bug https://phabricator.wikimedia.org/T178895

Reply to "Debug machine translation services, translation aids (Yandex mt service)"
ShInKurO (talkcontribs)

Hi all,

I've installed Translate extension (2017.07) on my wiki (mediawiki 1.27). When I try to translate a translable article the translation progressbar of the GUI doesn't go on when I press "Save translation" and I don't obtain any translation of my page after I've finished to translate at all. I've configured $wgJobRunRate to 0 and setted from console configuration of my hosting the cron job php script as written on mediawiki documentation (I cannot access to console).

On Firefox console I've these log:

when I write transation:

When I press save button:

JSON    

error    Object

code    nomessagefortitle

info    Title does not correspond to a translatable message

error    Object

code    unknownerror

info    Unknown error: "tpt-unknown-page"

Any idea about how solve this problem?

Thanks in advance!

Nikerabbit (talkcontribs)

Please see Help:Extension:Translate/Configuration#Saving translation fails with Unknown error: .22tpt-unknown-page.22

I would recommend that you log the jobqueue output from the cron script so that you can see it is working. You can also use the api to check how many jobs are in your jobqueue (Special:ApiSandbox, meta=siteinfo prop=statistics)

ShInKurO (talkcontribs)

Ok after some test my hosting doesn't support the launch of a script in its cron-mode tool, in fact the launch of runJob.php from that cron tool reply: "This script must be run from the command line cron". At this point shall I configure $wgJobRunRate to 1 again? will the translate extension work in these conditions?

Nikerabbit (talkcontribs)

If your wiki has enough traffic, it might just work okay.

ShInKurO (talkcontribs)

Also setting $wgJobRunRate to 1 my errors on click on save button remain:

code    nomessagefortitle

info    Title does not correspond to a translatable message

code unknownerror

info Unknown error: "tpt-unknown-page"

Nikerabbit (talkcontribs)

Yeah, either there hasn't been enough traffic (requests) to process that job (likely), or the job failed to process for some reason (less likely).

ShInKurO (talkcontribs)

Is there a way to log runJob script from browser (modifing some php source file script I suppose, but what php script? Mantainance.php?)?I

Nikerabbit (talkcontribs)

You don't need any specific page, just making requests to the main page URL should start draining the queue.

ShInKurO (talkcontribs)

Sure but what should read from an http request (with firebug I suppose) from runJob.php to give me help to fix this issue?

Nikerabbit (talkcontribs)

There is no output in the requests about the jobs. For that you have to resort to server side logging, which you may have to configure separately to log at the right verbosity.

ShInKurO (talkcontribs)

The problem is I have no access to server side, I can only upload file or execute in a gui kind a periodical job (a call to an url). For this reason I ask if there is a way to have a log written in a file or similar setting a variable or modifyng a script...

Nikerabbit (talkcontribs)

There are many configuration options for logging, and you can also configure the logs to be written in a file. Have look at DefaultSettings.php.

ShInKurO (talkcontribs)

Ok I've enabled some debug mode from LocalSettings.php and I've these errors when I load my homepage:

Notice: Use of undefined constant DB_REPLICA - assumed 'DB_REPLICA' in ../extensions/Translate/tag/TranslatablePage.php on line 874

And when I try to translate a debug row is written in this way:

[error] [221f291bb92735e24646966f] /index.php?title=Speciale:Traduci&group=page-I+5+Numeri&language=en&action=page&filter= ErrorException from line 874 of ../extensions/Translate/tag/TranslatablePage.php: PHP Notice: Use of undefined constant DB_REPLICA - assumed 'DB_REPLICA'

Nikerabbit (talkcontribs)

Until 1.27.4 is released, you need to either use the latest version from 1.27 branch, upgrade to 1.28 or newer, downgrade Translate to previous MLEB, or revert https://phabricator.wikimedia.org/rETRAa97b724a9a653edcbe7ab9c37b954f55e72e76e1

ShInKurO (talkcontribs)

I've actually mediawiki 1.27.1, so to have all work properly shall I upgrade mediawiki to 1.27.3 (latest avaible) or upgrade to 1.28+ mantaining actual version of Translate or downgrade Translate (what version?) ?

Nikerabbit (talkcontribs)

1.28+ should work. 1.27.3 won't. Do note that in next MLEB release we are likely going to drop support for 1.27.

ShInKurO (talkcontribs)

Ok I've update mediawiki to 1.29 and Translate to the latest version and now it works, thanks :)

Last question: if I want to remove a translation shall I remove all translations? I've tried to remove a translation (I've selected french instead of english and I've put english translation into french page) without result until I've not removed my page from Translate control.

Nikerabbit (talkcontribs)

Use the Delete action on page "Page/fr", that should remove the translations properly.

Reply to "Save button doesn't work"
67.244.49.134 (talkcontribs)

(Moved from support desk and modified) I have translate installed on my wiki, but when I try to add a translation for a page I get the following error. Saving the translation failed: Unknown error: "tpt-unknown-page" FuzzyBot also isn't running when I mark a page for translation. Running runJobs.php did nothing. Running showJobs.php gave the result 0. Any help would be great. Thanks.

Tuxxic (talkcontribs)

Hi,

Did you run runJobs.php from your actual PHP CLI instead of using an extension like MaintenanceShell ?

I had the same problem but running from CLI solved it.

In fact, in my case I'd like to runJobs eachtime I mark a page for translation, since it isn't done automatically ... :)

This comment was hidden by 24.39.1.139 (history)
24.39.1.139 (talkcontribs)

I ran it using SSH.

2002:43F4:3186:1234:35AE:218:4B7:62E5 (talkcontribs)

Please help me.

Nikerabbit (talkcontribs)

So if runJobs.php doesn't work, does running extensions/Translate/scripts/createMessageIndex.php work?

24.39.1.139 (talkcontribs)

That worked

24.39.1.139 (talkcontribs)

FuzzyBot still isn't creating the /en subpages though.

Nikerabbit (talkcontribs)

I don't have much info, so my best guess is that either the JobQueue is seriously broken and not running jobs, or there is some kind of error that prevents Translate from adding jobs to the queue. Enabling PHP error reporting as well as logging exceptions could help to identify the cause. For example:

error_reporting( E_ALL | E_STRICT );

$wgShowExceptionDetails = true;

$wgDebugLogGroups['exception'] = ini_get( 'error_log' ); 

$wgDebugLogGroups['error'] = ini_get( 'error_log' );

$wgDebugLogGroups['fatal'] = ini_get( 'error_log' );

Tuxxic (talkcontribs)

Edit : So here is what I get in my PHP error log after :

  • Adding a new chunk of text
  • Marking therefore the page for translation
  • then trying to translate it afterwards

I don't experience the problem if I update an already existing message group, if it can help you.

Thanks for advice on error logs :)

2017-05-03 12:01:19 SERVERNAME database-name: [3f588d59648bf5725b6fb31e] /index.php?title=Sp%C3%A9cial:PageTranslation   DBUnexpectedError from line 2866 of C:\mediawiki-path\includes\libs\rdbms\database\Database.php: MWCallableUpdate::doUpdate: Cannot flush snapshot because writes are pending (JobQueueDB::doBatchPush).

#0 C:\mediawiki-path\includes\libs\rdbms\loadbalancer\LoadBalancer.php(1092): Database->flushSnapshot(string)

#1 [internal function]: LoadBalancer->{closure}(DatabaseMysqli)

#2 C:\mediawiki-path\includes\libs\rdbms\loadbalancer\LoadBalancer.php(1390): call_user_func_array(Closure, array)

#3 C:\mediawiki-path\includes\libs\rdbms\loadbalancer\LoadBalancer.php(1100): LoadBalancer->forEachOpenMasterConnection(Closure)

#4 [internal function]: LoadBalancer->beginMasterChanges(string)

#5 C:\mediawiki-path\includes\libs\rdbms\lbfactory\LBFactory.php(176): call_user_func_array(array, array)

#6 [internal function]: LBFactory->{closure}(LoadBalancer, string, array)

#7 C:\mediawiki-path\includes\libs\rdbms\lbfactory\LBFactorySimple.php(134): call_user_func_array(Closure, array)

#8 C:\mediawiki-path\includes\libs\rdbms\lbfactory\LBFactory.php(179): LBFactorySimple->forEachLB(Closure, array)

#9 C:\mediawiki-path\includes\libs\rdbms\lbfactory\LBFactory.php(200): LBFactory->forEachLBCallMethod(string, array)

#10 C:\mediawiki-path\includes\deferred\DeferredUpdates.php(262): LBFactory->beginMasterChanges(string)

#11 C:\mediawiki-path\includes\deferred\DeferredUpdates.php(225): DeferredUpdates::runUpdate(MWCallableUpdate, LBFactorySimple, integer)

#12 C:\mediawiki-path\includes\deferred\DeferredUpdates.php(133): DeferredUpdates::execute(array, string, integer)

#13 C:\mediawiki-path\includes\MediaWiki.php(891): DeferredUpdates::doUpdates(string)

#14 C:\mediawiki-path\includes\MediaWiki.php(720): MediaWiki->restInPeace(string)

#15 C:\mediawiki-path\includes\MediaWiki.php(739): MediaWiki->{closure}()

#16 C:\mediawiki-path\includes\MediaWiki.php(543): MediaWiki->doPostOutputShutdown(string)

#17 C:\mediawiki-path\index.php(43): MediaWiki->run()

#18 {main}

Nikerabbit (talkcontribs)

Interesting. There are some related reports for this error message: https://phabricator.wikimedia.org/search/query/dmAqtYvLSuam/ I'll test this myself to see if I can reproduce. Which version of MediaWiki are you running?

Tuxxic (talkcontribs)

In my case, 1.28.2, but I experienced it also on 1.28.0.

113.172.20.5 (talkcontribs)

Hi Nikerabbit,

I got same problem Saving the translation failed: Unknown error: "tpt-unknown-page"

Wiki version 1.28.1

Use extensions/Translate/scripts/createMessageIndex.php is OK, but translate new page doesn't work

Nikerabbit (talkcontribs)

I don't know if this is specific to 1.28, but I just tried with MediaWiki vagrant on a clean install, using default MLEB configuration, and I did not have this problem.

Tuxxic (talkcontribs)

Thanks for testing.

Would you be interested in the list of my extensions installed, in order to see which one might be the culprit then ?

Nikerabbit (talkcontribs)

There is some info at https://phabricator.wikimedia.org/T100085 about different but possible related error message. It says it would be about exceptions.

I also can reproduce "Cannot flush snapshot" errors on my wiki now, so let's see if I can debug further.

171.233.234.227 (talkcontribs)

Thanks for debug Nikerabbit

ShInKurO (talkcontribs)

Hi all,

I've installed Translate extension (2017.07) on my wiki (mediawiki 1.27). When I try to translate a translable article the translation progressbar of the GUI doesn't go on when I press "Save translation" and I don't obtain any translation of my page after I've finished to translate at all. I've configured $wgJobRunRate to 0 and setted from console configuration of my hosting the cron job php script as written on mediawiki documentation (I cannot access to console).

Any idea about how solve this problem?

Thanks in advance!

Nikerabbit (talkcontribs)

Please open a new thread and check why the saving of the translations is failing, for example using PHP logs or your browser's developer tool and network tab.

ShInKurO (talkcontribs)

new thread opened! :)

Reply to "Translate"

Translation in the source language, English!

2
Donxello (talkcontribs)

Hi, we have an English Wiki but a German article to be translated. By default the extension marks the article as English and consequently we get

Translations to this language in this group have been disabled. Reason:

English is the source language of this page.

Any ideas how to mark this article as German and how to allow translations in the source language?

Thanks

Tristan

Nikerabbit (talkcontribs)

Hopefully you have recent enough MediaWiki to enable Manual:$wgPageLanguageUseDB in LocalSettings.php. Then you can use this feature to change the page content language to German. But you should do that before marking the page for translation.

Reply to "Translation in the source language, English!"

Namespace changed from the definition

4
Summary by Kghbln

Issue reported with task T172382

Kghbln (talkcontribs)

No matter what I add to the translation of the page title. The fuzzy bot keeps on fuzzying the message and emitting the above mentioned error notice. I have no idea how to resolve this and what may be the cause of it. The message conveys no meaning to me. I have several similar use cased on the wiki - working without issues. Here is the page. Any hint is appreciated.

Kghbln (talkcontribs)

Ok, I figured it out. It is possible to translate the namespace names for namespace category but not for namespace help. I guess a consistent behaviour will be nice.

Nikerabbit (talkcontribs)

Could you file a task in Phabricator? This is about MediaWikiMessageChecker::pagenameMessagesCheck incorrectly being applied also to translatable pages.

Kghbln (talkcontribs)

Sure. Done with task T172382.

MyISAM dependency blocking adoption of Azure Database for MySql

4
Lsilverman (talkcontribs)

I just attempted to move my mediawiki DB into Azure Database for MySql (their new hosted MySql offering). Got blocked on the use of MyISAM fulltext search on translate_tmf table.

Can this be migrated to InnoDB soon? Latest MySQLs have support for fulltext in InnoDB.

Nikerabbit (talkcontribs)

There are no such plans currently. ElasticSearch is recommended as the backend for translation memory and it also provides translation search. If you use ElasticSearch those database tables are unused.

Lsilverman (talkcontribs)

Nevertheless, the extension would fail to install due to the attempt to create the table using MYISAM.

Nikerabbit (talkcontribs)

Well, you can just comment out those files for the installation to remove the blocker for you. The oldest MySQL version that MediaWiki supports doesn't have fulltext support. You can file a task in Phabricator to make sure it is not forgotten when the minimum version requirement is updated.

Reply to "MyISAM dependency blocking adoption of Azure Database for MySql"
Lsilverman (talkcontribs)

I'm running MW 1.27 with CirrusSearch+Elastica, which requires ElasticSearch 1.X. Cirrus and Elastica are pinned to the REL1_27 branches.

My wiki emailed me the following error upon attempting to translate my first page:

PHP Fatal error:  Call to undefined method Elastica\Query\BoolQuery::addFilter() in /var/www/devmediawiki-1.27/w/extensions/Translate/ttmserver/ElasticSearchTTMServer.php on line 282

LocalSettings.php portion looks like this:

$wgTranslateTranslationServices['TTMServer'] = array(
        'type' => 'ttmserver',
        'class' => 'ElasticSearchTTMServer',
        'cutoff' => 0.75,
        /*
         * See http://elastica.io/getting-started/installation.html
         * See https://github.com/ruflin/Elastica/blob/master/lib/Elastica/Client.php
        'config' => This will be passed to \Elastica\Client
         */
);
Lsilverman (talkcontribs)

Tracing the problem further, I see that REL1_27 of Elastica includes via composer ruflin/elastica 2.3.1.

elastica 2.3.1's version of Elastica/Query/BoolQuery.php does not contain the function addFilter().

Nikerabbit (talkcontribs)

Unfortunately this incompatibility has not been caught in our testing. Given 1.29 is soon going to be released, which means we are going to drop support for 1.27 from MLEB and Translate, I would encourage you to start planning for an upgrade and use the database backend for translation memory for the time being.

Lsilverman (talkcontribs)

We use Active Directory and LDAP. The LDAP auth plugins have not been updated in some time, and as a result we are stuck on 1.27.

Reply to "Error using ElasticSearch in 1.27"

Conflict with Extension:LDAP_Authentication

1
217.6.145.253 (talkcontribs)

Topic:Tfu65b5pncef5p6s

Reply to "Conflict with Extension:LDAP_Authentication"

Cannot mark for translation

3
Summary by Nikerabbit

A conflict between ConfirmEdit and Translate.

217.6.145.253 (talkcontribs)

When i click the link 'Diese Seite zur Übersetzung freigeben' (Wiki language is german) I get redirected to

index.php?title=Spezial:Übersetzung_von_Seiten&target=TargetPage&do=mark.

But the special page does not exist! What could i do?

Nikerabbit (talkcontribs)

It sounds some kind of configuration error to me. I have not seen such an issue before.

217.6.145.253 (talkcontribs)

It seems it was a conflict between Extension:ConfirmEdit and Translate. Since we didn't really need that one we just removed it.

Are there Active steps planned for the internationalization of category names?

2
Atef81 (talkcontribs)

I am starting a multilingual environment that will have mainly 4 languages. The problem is that the language that will have the most content is Arabic, and my aim is to get as much content translated to the other languages.

When I tried the Translate extension, it worked, but having the categories names in Arabic then "/en" and "/fr" is not practical as non Arabic speakers will find it impossible to understand.

I checked several forums and read that there are steps in the pipeline to rectify this. However, I can't find enough information about this on the Extension official page.

I am trying to start from a page like this and reach an end result like this, where page title and category names are translated to the article language.

Is this likely to be an option soon, or I need to setup a family of wikis that share the same resources like this Mediawiki manual?

@Nikerabbit

Nikerabbit (talkcontribs)

Thanks for asking!

Due to possible naming conflicts, it's not a good idea to translate the actual page titles, unless you disambiguate them with a language code anyway.

But what you can do is to use the feature to translate the (category) page title. This does not affect the URL of the page, nor any links to that page. However, there are feature requests for MediaWiki core to use this translated title in links, including the category list at the bottom of the pages. It is not clear, though, when these will happen. I believe the features are not controversial, but some performance concerns must be taken into account while implementing them.

Some links:

Reply to "Are there Active steps planned for the internationalization of category names?"