Translation of app string resources
Current localization documentation is available
 | This page is obsolete. It is being retained for archival purposes. It may document extensions or features that are obsolete and/or no longer supported. Do not rely on any information on this page. |
Translations of string resources are done at translatewiki.net (aka TWN).
Repo Configuration[edit]
See Wikipedia iOS and Wikipedia Android groups in translatewiki.net Gerrit project.
Requesting Access[edit]
All of the following steps must be performed on translatewiki.net, and therefore require that you have access to the machines. To request access:
- File a Phabricator ticket to request access and add it to the translatewiki.net project (see example)
- Be sure to include the public key of the RSA key you want to use to access the machine
- Ensure your local SSH config uses the correct identity file for the translatewiki.net host
- The i18n team will then review your request for access, and if granted, update the translatewiki.net manifest to add your key
- File another Phabricator ticket to request access to review changes (see example)
- Create a user on translatewiki.net
- Mention user name in Phab ticket
SSH Config[edit]
In step 2 above, your SSH config should look something like this:
Host translatewiki.net User bgerstle IdentityFile ~/.ssh/translate_wiki_rsa
TWN sync for Android[edit]
About once a week we sync string resources between our code and TWN. The terms import and export are used based on the TWN perspective. We first import new strings (English + qqq) from our code bases to TWN then export translated strings out of TWN to our code bases. The following section document how this is done at a low level.
Before starting the TWN sync make sure you have joined the #mediawiki-i18n IRC channel.
Import from Android code to TWN[edit]
ssh translatewiki.net
cd ~/wikipedia-android
git fetch origin
git checkout origin/master
repoupdate wikipedia-android
repo update wikipedia-android
php /www/translatewiki.net/docroot/w/extensions/Translate/scripts/processMessageChanges.php --group=out-wikimedia-mobile-wikipedia-android-\* --name=$USER
If the last command say "No changes" then continue with the export steps.
If it says something like "Process changes at https://translatewiki.net/wiki/Special:ManageMessageGroups" then go to that URL in your browser, review the to be imported strings. See also #Notes about flags.
Run job queue if there were changes to expedite the process (otherwise you have to wait a few minutes to make sure the import, which runs in the background, is complete:
sudo -u betawiki php /www/translatewiki.net/docroot/w/maintenance/runJobs.php
Export from TWN to Android code[edit]
This will push a new commit to Gerrit in the respective app repo(s):
repoexport wikipedia-android
repocommit wikipedia-android
exit
Go to Gerrit and run the translation tests (time ./gradlew connectedAlphaReleaseAndroidTest -Pandroid.testInstrumentationRunnerArguments.class=org.wikipedia.test.TranslationTests), then merge.
Semi-automation[edit]
#!/usr/bin/env bash
set -eou pipefail
cd ~/wikipedia-android
git fetch origin
git checkout origin/master
repoupdate wikipedia-android
repo update wikipedia-android
php /www/translatewiki.net/docroot/w/extensions/Translate/scripts/processMessageChanges.php --group=out-wikimedia-mobile-wikipedia-android-\* --name=$USER
read -p '<enter> to continue, <ctrl-c> to abort: '
sudo -u betawiki php /www/translatewiki.net/docroot/w/maintenance/runJobs.php
repoexport wikipedia-android
repocommit wikipedia-android
TWN sync for iOS[edit]
Import from iOS code to TWN[edit]
ssh translatewiki.net
cd $HOME/wikipedia-ios
git fetch --prune origin
git checkout develop
git reset --hard remotes/origin/develop
# don't ask, just do them both
repoupdate wikipedia-ios
repo update wikipedia-ios
php /www/translatewiki.net/docroot/w/extensions/Translate/scripts/processMessageChanges.php --group=out-wikimedia-mobile-wikipedia-ios --name=$USER
If the last command say "No changes" the continue with the export steps.
If it says something like "Process changes at https://translatewiki.net/wiki/Special:ManageMessageGroups" then go to that URL in your browser, review the to be imported strings.
Run job queue if there were changes to expedite the process (otherwise you have to wait a few minutes to make sure the import, which runs in the background, is complete:
sudo -u betawiki php /www/translatewiki.net/docroot/w/maintenance/runJobs.php
Export from TWN to iOS code[edit]
The steps are as follows:
# Check out new, timestamped branch to apply changes to (which will be pushed to remote at the end)
git checkout -b twn/$(date +%m-%d-%Y)
# Update QQQ with any missing strings
scripts/update-qqq.php
# Export changes from TWN and apply to current repo
# First argument is the project name, second is the repo where the project was cloned
repoexport wikipedia-ios /home/$USER
# Run git status here to make sure you have a bunch of unstaged changes to localized files
git status
# Double check no merge conflicts slipped in.
find -iname \*.strings|xargs -rd\\n grep -E '[<>]{7}'
# Fix any reversed dollar signs from RTL translations.
# TODO: this doesn't appear to work for RTL languages. Find an alternative.
find -iname \*.strings |xargs -rd\\n sed -ri 's%(\d+)\$%$\1%g'
# Check for HTML comments.
find -iname \*.strings|xargs -rd\\n grep '<!--'
# Add any new untracked localization files
git add --all
# Use script to generate standard TWN commit message
# Arguments passed in the same manner as repoexport above
repocommit wikipedia-ios /home/$USER
# Push the current branch to a remote ref w/ the same name
# This is the step where you enter your GitHub username and password, using a personal access token if necessary (see Pushing TWN Changes to GitHub below)
git push origin HEAD:twn/$(date +%m-%d-%Y)
Pushing TWN changes to GitHub[edit]
If your GitHub account uses two-factor authentication (2FA) you'll need to create a personal access token, which will be used as your GitHub password in the git push phase. If it doesn't use 2FA, it should!
Integrating new localizations[edit]
Sometimes new languages will be added for an existing localized resource. Manually verify that all localized assets are integrated into the Xcode project by checking for added files and ensuring their corresponding language is checked in the Xcode file inspector panel. TODO: automatically integrate "lproj" assets into Xcode project via xcodeproj Ruby gem.
Semi-automation[edit]
#!/usr/bin/env bash
set -eou pipefail
cd $HOME/wikipedia-ios
git fetch --prune origin
git checkout develop
git reset --hard remotes/origin/develop
# don't ask, just do them both
repoupdate wikipedia-ios
repo update wikipedia-ios
php /www/translatewiki.net/docroot/w/extensions/Translate/scripts/processMessageChanges.php --group=out-wikimedia-mobile-wikipedia-ios --name=$USER
read -p '<enter> to continue, <ctrl-c> to abort: '
sudo -u betawiki php /www/translatewiki.net/docroot/w/maintenance/runJobs.php
declare br="twn/$(date +%m-%d-%Y)"
git checkout -b "$br"
repoexport wikipedia-ios /home/$USER || :
scripts/update-qqq.php
! find -iname \*.strings|xargs -rd\\n grep -E '[<>]{7}'
! find -iname \*.strings|xargs -rd\\n grep '<!--'
find -iname \*.strings|xargs -rd\\n sed -ri 's%([0-9]+)\$%$\1%g; s/%([0-9]+)/$\1/g; s/\$d/%d/g; s/%\$([0-9]+)/$\1/g'
git add -A .
repocommit wikipedia-ios /home/$USER
echo git push origin "HEAD:$br"
Notes about flags[edit]
Note about the ignore flag: We tend to ignore translated string resource changes, unless there is a good reason for them to be done in our code and it should be reflected in TWN. (Usually translated strings come from TWN, and not the other way around.)
Note about the fuzzy flag: If the meaning of the English string has changed then mark it as fuzzy so it shows up as Outdated in the TWN interface. Then translators know they should translate the string again.
One time TWN sync setup[edit]
Set GERRIT_USER to the proper value. This is currently written for both Android and iOS but will need to be split up in the future.
GERRIT_USER=
ssh translatewiki.net
cd /resources/$GERRIT_USER/
cp ~/REPOCONF .
repo create wikipedia-android
repo create wikipedia-ios
cd wikipedia-android
git remote add gerrit ssh://$GERRIT_USER@gerrit.wikimedia.org:29418/apps/android/wikipedia.git
git remote add origin ssh://$GERRIT_USER@gerrit.wikimedia.org:29418/apps/android/wikipedia.git
cd ../wikipedia-ios/
git remote add gerrit ssh://$GERRIT_USER@gerrit.wikimedia.org:29418/apps/ios/wikipedia.git
git remote add origin ssh://$GERRIT_USER@gerrit.wikimedia.org:29418/apps/ios/wikipedia.git
Follow steps in Gerrit settings to set up a new set of ssh keys so you don't need to do ssh agent forwarding.
Fixing translations on TWN site[edit]
To get to the string in TWN fast you can:
- go to the Message group statistics for Android or iOS,
- click on the language you want,
- switch from
UntranslatedtoTranslatedorAll, - then enter the translated string in the
Filter. - From there (in the "List" tab) you can click on
Edit.
Change translations[edit]
You can change the string here but that should be left to the translators. What you can do though is to add the string !!FUZZY!! in front of the current text to mark it fuzzy/outdated. You can also look at the translation history of that particular string by clicking on the little down arrow.
Example URLs[edit]
Spanish[edit]
Talk: https://translatewiki.net/wiki/Wikimedia_talk:Wikipedia-android-strings-app_name_prod/es
Simplified Chinese: For zh/strings.xml use zh-hans[edit]
Traditional Chinese: For zh-rTW/strings.xml use zh-hant[edit]
More about Message group statistics[edit]
A given language is only exported from TWN if it has reached or exceeded a certain translation completeness threshold. Currently the threshold is set to 35% [1].
Advanced[edit]
Translatewiki.net on Gerrit[edit]
https://gerrit.wikimedia.org/r/#/admin/projects/translatewiki
For Android TWN sync see groups/Wikimedia/WikimediaMobile-android.yaml, esp. the codeMap section.