Uploading new .po file, merging translations

Asked by Honore on 2009-12-09

This questions has probably been answered across a few threads before, but it seems others may have a different setup, so I'll ask again just in case.

I work on the WiserEarth project, and we're using Launchpad primarily just to translate our site.

We're looking to upload a newer version of the .po file to account for changes we've made to our code. However, we're concerned - will we lose the translations that our users have already worked on in the old .po file?

What's the best process to "merge" the translations of the old .po and incorporate it into the new .po file?

Thanks!

Some notes about our setup:
We use a symfony framework running on PHP. Symfony doesn't have very strong native support for the .po file - it only generates a fairly basic file unfortunately.

Question information

Language:
English Edit question
Status:
Answered
For:
Launchpad itself Edit question
Assignee:
No assignee Edit question
Last query:
2009-12-14
Last reply:
2012-05-25
Adi Roiban (adiroiban) said : #1

Hi Honore,

Are you going to upload POT , PO or both type of files?

POT is the "template" , the one containing only translations in english and that is generated by the tool extracting translation strings from your files.

The merge should be done in a transparent way.
For backup you can alway request an export and wait for it to be generated and the link sent via email.

I'm also using Launchpad for my project and in the past I had no problems with importing POT files. All unchanged messages retain their translations.

Henning Eggers (henninge) said : #2

When updating the template by uploading an new POT file (just the English strings with no translations), all changed strings will become untranslated, all unchanged strings will remain translated as they were. Changed strings are the same as new strings, Launchpad does not use fuzzy matching or anything like that.

Honore (honore) said : #3

Hey Adi,

We're uploading new .po files. The way it's happening is as follows:

1. We first uploaded a .po file containing general text strings used across the site

2. We're looking to upload .po files containg text strings pulled from other parts of our site, such as different database tables (error messages, validation confirmations, drop-down menu values, etc).

Our developer has uploaded a new set of files here: https://translations.launchpad.net/wiserearth/+imports

I'm wondering what happens once they're approved. When a user goes into the Translations and clicks on a language, will he see multiple .po files he can translate per language? Or will all the text strings be merged into one big file on Launchpad's end?

David Planella (dpm) said : #4

El dl 14 de 12 de 2009 a les 19:17 +0000, en/na Honore va escriure:
> Question #93405 on Launchpad Translations changed:
> https://answers.launchpad.net/rosetta/+question/93405
>
> Status: Answered => Open
>
> Honore is still having a problem:
> Hey Adi,
>
> We're uploading new .po files. The way it's happening is as follows:
>
> 1. We first uploaded a .po file containing general text strings used
> across the site
>
> 2. We're looking to upload .po files containg text strings pulled from
> other parts of our site, such as different database tables (error
> messages, validation confirmations, drop-down menu values, etc).
>
> Our developer has uploaded a new set of files here:
> https://translations.launchpad.net/wiserearth/+imports
>
> I'm wondering what happens once they're approved. When a user goes into
> the Translations and clicks on a language, will he see multiple .po
> files he can translate per language? Or will all the text strings be
> merged into one big file on Launchpad's end?
>

Looking at the imports queue, I think I see what the problem is.

There are two types of files involved in translations:

* POT files: that's what we also call templates, and it is what the
project maintainers need to generate and upload to Launchpad to expose
them to translators. These are the files which have a .pot extension and
contain pairs of msgid and (empty) msgstr entries.

* PO files: this files contain the translations, and are usually
uploaded by translators and not the maintainers (unless they are
importing existing translations done somewhere else). Translators
translating online do not usually have to deal with these kind of files,
although they can download/upload them if they prefer working offline.

The problem is that the maintainer uploaded PO files (translations) as
if they were POT files (templates).

The steps I'd suggest to maintainers to get the templates approved and
exposed in Launchpad are:

      * Regenerate (or just rename the .po extensions to .pot) the .pot
        templates
      * Upload them again to Launchpad in the usual way (either
        individually or in a tarball)
      * Wait for them to be approved the first time (you can comment
        again on this thread for someone to have a look at them)
      * Afterwards, all these templates will be exposed to translators,
        in a similar way to
        https://translations.launchpad.net/enlightenment/trunk/+lang/en_GB (to pick up a project with multiple templates).

I hope this clarifies things.

Regards,
David.

Honore (honore) said : #5

Thanks, David, will confer with Sang, our developer, who should also be a member on Launchpad, and will work with him to get these files corrected.

Honore (honore) said : #6

Hey David, I believe Sang just modified and re-uploaded the files per your instructions above. Hopefully you or someone on the LP team can take a look and review it quickly! Thanks again as always for your very thorough explanations!

David Planella (dpm) said : #7

Hi Honore,

I've had a look at the imported POT files and I've noticed a couple of things:

* The project has several different translation templates (POT files) in the same folder (pot) from the source tree. This layout will not generally work with translations, since it is recommended that the po files are named after the language code (e.g. de.po, pt_BR.po) and stored under the same directory as the POT file. Having several POT files in the same folder would bring you conflict when trying to store translations for different templates but with the same file name.

I'd suggest your developer to consult the following documentation for more information on this:

https://help.launchpad.net/Translations/YourProject/ImportPolicy#Sample%20directory%20layout
https://help.launchpad.net/Translations/YourProject/ImportingTemplates

Basically, the suggestion would be to create a separate folder for each template, for example:

po-areas/areas.pot
po-notifications/notifications.pot
po-s_activity/s_activity.pot
...

* POT file header. This is a minor thing, but I thought I would mention it for the sake of completeness. Looking at the POT file from one of the imported files, I see that the "POT-Creation-Date" field is missing. What tool are you using to generate the POT templates? In what language/CMS is WiserEarth developed?

"Project-Id-Version: 1.0"
"PO-Revision-Date: 2009-12-10 09:55-0700"
"Last-Translator: WiserEarth Translation Team <email address hidden>"
"Language-Team: English"
"MIME-Version: 1.0"
"Content-Type: text/plain; charset=UTF-8"
"Content-Transfer-Encoding: 8bit"

As an example, here is a header generated from another project using gettext:

"Project-Id-Version: PACKAGE VERSION\n"
"POT-Creation-Date: 2009-10-22 17:49+0200\n"
"PO-Revision-Date: YEAR-MO-DA HO:MI+ZONE\n"
"Last-Translator: FULL NAME <EMAIL@ADDRESS>\n"
"Language-Team: LANGUAGE <email address hidden>\n"
"MIME-Version: 1.0\n"
"Content-Type: text/plain; charset=UTF-8\n"
"Content-Transfer-Encoding: 8bit\n"

Henning Eggers (henninge) said : #8

Let me re-inforce what David said: It is important that you follow the suggested directory layout and also put each template in its own directory. I can approve your templates and change the path at the same, as a service to good customers. ;-) Do you agree with the layout that David suggested?

Honore (honore) said : #9

Hey David, Henning,

Sure - the layout suggested seems great. I'll ask my developer Sang to modify the files and folders as recommended above. Thanks again for the clarifications - it's not always easy to ferret these details out in the documentation!

-Honore

swong (sang-wiserearth) said : #10

David and Henning,

Thanks for your helpful input. Our translation texts are pulled from our database. The POT (template) files are generated by our internal script (not gettext). Some of the text is html (containing multiple lines, quotes, etc). Do you have any suggestions as to how to handle this in the POT file?

# ===== Example: multiple lines and quotes is msgid string =====
msgid "[definition:
Acid "rain" is any form of precipitation (rain, snow, hail, or fog)
that contains harmful amounts of nitrogen and sulfur oxides. The
major human sources are from industry, transportation, and power
plants. These industrial pollutants from the air increase the
acidity of the precipitation and harm plants, buildings, and
humans.] Browse the <a href="http://www.wiserearth.org/user/active">people pages</a>"
msgstr ""

Thanks,

Sang

swong (sang-wiserearth) said : #11

David + Hennings

The modified files and uploaded to the queue. Really appreciated if you can take a look and approve as we are on a tight deadline. Thanks.

Sang

David Planella (dpm) said : #12

swong,

I've now approved the templates, and are available for translation at:

  https://translations.launchpad.net/wiserearth

Further uploads of these templates will be imported automatically and will not require review (this is a process for new templates only).

Regarding your question on how to best extract the translations to be put in the POT file, you say you are using Symfony and PHP. Unfortunately, I'm not familiar with their i18n approach, but you can perhaps look at PHP-gettext https://launchpad.net/php-gettext and the standard gettext tools (http://www.gnu.org/software/gettext/manual/gettext.html#xgettext-Invocation).

A couple of things I noticed:

* You've set the translation permissions to Open (https://help.launchpad.net/Translations/YourProject/PermissionPolicies) and assigned it to the Launchpad Translators group. While this is fine if it is deliberate, I'd recommend you to choose Structured or Restricted if the quality of translations is a concern for you (I'm just mentioning it because we had a discussion some time ago about this).

* I'd recommend you to announce that WiserEarth is now open for translations through your usual channels and the Launchpad Translators mailing list (https://launchpad.net/~launchpad-translators)

I hope this helps.

BISAG (mirenkaramta87) said : #13

i cannot find the web interface to upload the .pot file. Can any one help me?

Can you help with this problem?

Provide an answer of your own, or ask Honore for more information if necessary.

To post a message you must log in.