Migrate translation to www.transifex.com

[ggeldenhuis]

Hi
Would you support an effort to migrate translations of JOSM to the transifex.com website? I have been contributing to translations for iD using the website and have found it very useful and easy to use. More importantly it is very easy for non-technical people to contribute translations. There is thus much less barriers for entry. The translation effort for josm looks a bit daunting at first glance.

[stoecker]

From my point of view transifex actually is less useful. For launchpad all you need is a login.

So the answer is No.

[simon04]

I would also favour the move to transifex since it provides a more appealing user interface and the slowness of launchpad is really annoying. Some/many other FOSS projects have moved from launchpad to transifex (google for "launchpad to transifex"). Plus, other OSM projects utilize transifex as well: ​https://www.transifex.com/tag/openstreetmap/

[Don-vip]

+1 I'm tired of Launchpad and it slowness. In fact I nearly stopped any translation effort, I think there are other people so annoyed by the failures they stopped as well. Plus having all OSM translation at a single place would be really interesting for consistency between website and editors :)

[Don-vip]

​https://bugs.launchpad.net/launchpad/+bug/736005 No progress in almost two years, it's a shame

[stoecker]

I still don't like Transifex much, but there is NO progress for Launchpad. Maybe it really is time to move. Will you do an initial setup for tests and later the necessary automatisms for data up- and download?

[skyper]

+1, I am really tired reloading Launchpad

[simon04]

Set setup a transifex project: ​https://www.transifex.com/projects/p/josm/
Upload of the translations is in progress …

[simon04]

I added a transifex client config file in [o30039].

Most languages should be present already; some require some manual interaction due to syntax errors in the .po files (transifex checks uploaded files using msgfmt -c).

[ggeldenhuis]

Thanks! I have started work on the Afrikaans translation.

[Don-vip]

Is it possible to have 2 resources ? "core" and "plugins" ?

[Don-vip]

Also, is there a way to preserve translation history in the migration ?

[simon04]

Replying to gerhardus.geldenhuis@…:

Thanks! I have started work on the Afrikaans translation.

Please note that we are currently testing/evaluating transifex …

Replying to Don-vip:

Is it possible to have 2 resources ? "core" and "plugins" ?

+1. This requires to modify some scripts in the i18n/ directory. We should also avoid to have some strings listed under core as well as plugins. Dirk, what do you think of this separation? Maybe split presets and imagery strings to a third package ("data"?)?

Replying to Don-vip:

Also, is there a way to preserve translation history in the migration ?

I don't see how to. The launchpad only allows to export translations in the PO/MO format (cf. ​https://translations.launchpad.net/josm/trunk/+export), which doesn't name the translator of each string. Is there any other way?

[stoecker]

Note, that launchpad will get faster again as well, when we split josm into smaller subpackages. It has advantages and disadvantages. I personally like one big package more, as strings are shared between the packages.

[Don-vip]

We could try to build distinct packages and count the number of shared strings, then decide what's best :)

[simon04]

Replying to Don-vip:

We could try to build distinct packages and count the number of shared strings, then decide what's best :)

msgcomm po/core.pot po/plugin.pot | grep -c 'msgid ' gives 256

Btw: there's a msgcat command, but we'd need something like msgsubtract – still investigating ;-)

[Don-vip]

Can someone do i18n so we can release new tested ? :)

Another question: is it possible with transifex to have automatic translations ? (an update of languages during nightly build ?)

[simon04]

Replying to Don-vip:

Can someone do i18n so we can release new tested ? :)

i18n is on the way ... To perform this update, check out the complete JOSM repository and execute ant updatecore in i18n/. Then commit the contents of core/data.

Another question: is it possible with transifex to have automatic translations ? (an update of languages during nightly build ?)

There's a transifex client to push translation templates (pot files) and pull translated languages (po files). This can be performed automatically as it is currently for launchpad.

[skyper]

Does anyone know how transifex handels outdated translations e.g. are all translations lost once a string is changed or even if only *_context is removed or are these translations preserved ?

[simon04]

Transifex displays translation suggestions, cf. ​http://support.transifex.com/customer/portal/articles/1156770 (e.g., ​https://www.transifex.com/projects/p/osmand/translate/#de/stringsxml/14831283?translated=no). Concerning *_context, I don't know.

[aceman]

Replying to skyper:

Does anyone know how transifex handels outdated translations e.g. are all translations lost once a string is changed or even if only *_context is removed or are these translations preserved ?

This would interest me too. Other projects use a "fuzzy" feature, i.e. if the original English string changes, the translation is not lost, only marked "fuzzy". The translation will see which strings are fuzzy and can review/update the translated string again. In JOSM currently the string is lost and we need to translate it whole anew (or retrieve from backup). The "fuzzy" feature is not specific to .po files, e.g. the Weblate and Pootle translation systems have it too.
And I hope transifex supports export/import from .po files.

[aceman]

Should we work on the old system for now and how will we get notified when to switch to the new system? I hope the data will get merged at that time.

[simon04]

The transifex is currently used only for testing, so use launchpad for "productive" translations. Sure, when we decide to use transifex, we move the latest data from launchpad (i.e., the translations, not the translator itself, since it's unclear how to export as well as import that information).

[simon04]

Split translations into core, data, plugins and uploaded templates to transifex. See ​https://www.transifex.com/projects/p/josm/language/de/ for an example.

data and plugins do not contain strings from core – see i18n.build.xml.v2.patch​ for generation details.

[stoecker]

No progress here. Would it be possible to add me as admin in the project? If not, can we create an admin user, where all JOSM admins know login data? I still don't understand the permission system of Transifex. It seems awful complicated to give permissions to someone.

What's necessary to proceed?

What about automatic upload of translations from server? Is there an ant task ready? A script?

[simon04]

Replying to stoecker:

No progress here. Would it be possible to add me as admin in the project?

Done.

What's necessary to proceed?

What about automatic upload of translations from server? Is there an ant task ready? A script?

There's the Transifex client available which uses a configuration started in [o30039] and extended in attachment:i18n.build.xml.v2.patch​. The client allows to tx push POT files to Transifex and tx pull the current PO files.

The next steps would be to split the PO files set in the config file via file_filter to obtain separate files per component (core/data/plugins) – after the current PO files from Launchpad are updated on Transifex… Then one would have to integrate the push/pull commands into the Ant build file. Finally a migration strategy is needed.

[stoecker]

Ok, I now added an auto-upload to the project. Next step is to make the build process ready for downloading, so .lang files can be generated. The possibility to separate the translations already exists. So after a "tx pull" the only missing task is to merge the translations into one big file for each language again.

[akks]

By the way, on Russian forum people start asking me why many validator warings are in English (they are using newly relaesd tested version, on launchpad there are no significat untranslated strings).

[simon04]

Replying to akks:

By the way, on Russian forum people start asking me why many validator warings are in English (they are using newly relaesd tested version, on launchpad there are no significat untranslated strings).

I created a separate ticket #9626 since this issue is unrelated to the Transifex migration. Thanks for noticing! :-)

[simon04]

Replying to stoecker:

Ok, I now added an auto-upload to the project. Next step is to make the build process ready for downloading, so .lang files can be generated. The possibility to separate the translations already exists. So after a "tx pull" the only missing task is to merge the translations into one big file for each language again.

Patch attachment:8645_build.patch​​ attached. This should do the trick.

(Today, I translated ≈30 outstanding German strings of JOSM by a sequence of downloads/uploads of po file/s from/to launchpad/transifex. The UI is awesome. Looking forward to the completed migration to transifex.) :-)

[stoecker]

Hmm, the patch drops existing structure. I wanted to run the launchpad and transifex stuff for some time in parallel before switching:

• download from launchpad
• build lang files
• upload the downloaded files to transifex
• download from transifex
• build lang files
• compare the files (should be equal)

[stoecker]

Also I don't like to drop a working infrastructure. Who knows if we may switch back one day?

[simon04]

What about locally duplicating the i18n directory into i18n.lp and i18n.tx, applying the patch on i18n.tx? Then steps 1–3 can be performed in i18n.lp and the subsequent steps 4–6 can be performed in i18n.tx. Then one only needs to take care to obtain the XY.lang files from both processes.

[simon04]

Dirk, what is the status of the migration? Is any help needed? I'd really love to see the migration to transifex.com competed. :-)

[stoecker]

I'm still not perfectly happy with you last patch, but actually the major problem is, that I did not look into this. I updated the Trac spamfilter plugin lately to support statistics and that took all my time.

A patch which does not break the current i18n-build would nevertheless be appreciated. Simply clone the build targets in the build.xml instead of replacing them :-)

[stoecker]

BTW: If someone wants to test Transifex already - Josm webpages also benefit from translation of ​Spamfilter :-)

[Don-vip]

I really can't stand buggy Launchpad anymore, it's incredibly difficult to fix erroneous strings to achieve i18n. Can we switch to transifex next month?

[stoecker]

No problem. The only problem I have with above patch is that it drops launchpad support. If it would simply add transifex without dropping launchpad I'd be happy. Should be easy to change it accordingly. But for some time now I do very little with my PC in free-time which also affects JOSM contribution.

[stoecker]

P.S. Launchpad says they will update DB servers probably reducing this issues a lot :-)

[aceman]

Yeah, ​https://bugs.launchpad.net/launchpad/+bug/736005 .

[Klumbumbus]

Any news to this? The Launchpad webinterface is still not usable. I always get the timeout error.

[Don-vip]

Launchpad is still buggy and the "couple of weeks" promised delay is, not surprisingly, not respected at all. I have some free time to look at this subject this week-end if neither of you can finish the patch. See #10257: people don't even try to fix translation issues, and we can't really blame them.

[malenki]

@ Klubumbus: for big projects you should download the translation file, enhance it locally and upload it again. Problem: from clicking "download" until the download is ready launchpad needed 30 min at my last (and only) try (which Don-vip refers to).
I'd welcome the move from to transifex very much.

[Klumbumbus]

Replying to malenki:

@ Klubumbus: for big projects you should download the translation file, enhance it locally and upload it again. Problem: from clicking "download" until the download is ready launchpad needed 30 min at my last (and only) try (which Don-vip refers to).

I know. I also fixed the "Überführungsbauwerke" together with another one. I got the link to the .po download file within a few minutes, not 30. So I was a bit faster then you, and my translation got merged before yours. Therefore you got the conflict error :P

"Überführungsbauwerke" is fixed now in launchpad, though not in josm latest, because there hasn't been a manually pulling from launchpad since fixing it.

I'd welcome the move from to transifex very much.

+1

[Mkyral]

Replying to Klumbumbus:

Replying to malenki:

I'd welcome the move from to transifex very much.

+1

Me too. There are some strings that I want to correct. But I don't want to waste my time with launchpad. I see, the files on Transifex are updated, but I don't know if I can start with translation (means, that the translated files from Transifex are merged back to Launchpad and josm). I still waiting for the green light ;-)

BTW: how can I add myself to this issue to monitor it? I can't see any option there.

[stoecker]

Status: See comment 38 what needs to be done.

BTW: how can I add myself to this issue to monitor it? I can't see any option there.

There is a CC field. But if I remember configuration right, then the fact that you added a text should be enough already as well.

[Don-vip]

Ticket #10352 has been marked as a duplicate of this ticket.

[Don-vip]

Simon, Dirk, any chance to see some progress to finish the transition? I tried to look at it the other time but I wasn't sure to understand everything in the i18n scripts...

[stoecker]

Replying to Don-vip:

Simon, Dirk, any chance to see some progress to finish the transition? I tried to look at it the other time but I wasn't sure to understand everything in the i18n scripts...

I have holidays ATM and I planned to do it in next two weeks, but currently I'm pretty much abstinent from any computer and busy reading Piers Anthony's Xanth series. Still approx 20cm (i.e. 9 books) left :-) So no promises.

[Don-vip]

This is not a minor problem as I'm pretty sure we're loosing translators because of this. We still have complains, see #10442.

[daganzdaanda]

Yep, it's not very smooth to work with...
What I did is to dl the PO file and find the ID of the string I was interested in there. Then I could set the &start parameter in the URL to point to that ID. Then I changed it and hit save, and it seems to be OK.

[Don-vip]

Replying to stoecker:

I have holidays ATM and I planned to do it in next two weeks, but currently I'm pretty much abstinent from any computer and busy reading Piers Anthony's Xanth series. Still approx 20cm (i.e. 9 books) left :-)

How's the reading going? ;)

[stoecker]

Reading finished on time. And I also did do some stuff and then work catched me again. I hate this ticket...

[Don-vip]

What I hate more is retrying thirteen times to reach an erroneous translated string in Launchpad :(

[anonymous]

Replying to stoecker:

Reading finished on time. And I also did do some stuff and then work catched me again. I hate this ticket...

Could we help somehow?

[stoecker]

Replying to Don-vip:

What I hate more is retrying thirteen times to reach an erroneous translated string in Launchpad :(

I agree. Still I always find more interesting stuff than to care for that ticket.

Could we help somehow?

Change the build.xml in a way, that it works smooth with old Launchpad and new Transifex, so that Transifex is merely a toogle switch. And proper testing upload of current translations from Launchpad to new Transifex and download of these Transifex translations to build language files. That's all.

[Don-vip]

Assigning to milestone. It would be nice to move at the same time we change the logo :)

[stoecker]

I used transifex for other tools in the last weeks and the more I use it, the more I dislike it. Major flaws I see after short time:

• It will drop all related translations if only a single time the pot file upload is broken!
• Administration is overly complicated

I thought about making my own service after beeing confronted with Launchpad for the first time, but found it too much work. Then I though about making a translation service based on Trac and still found it too complicated and too restricted. Now I did a lot more Trac development (spamfilter, translatedpages macro, ...) and I'm convinced that the best solution would be a Trac plugin for project translation. Advantages I see:

• OpenSource
• No translation service, but based on the local users we already have and also in the wiki we already have everything else
• Easy to setup and manageable
• Project specific

Disadvantages:

• No project overlapping (but that didn't bring much for launchpad in the past anyway)
• Will take some time until stable.

Interface would be with permission handling based on Trac and Translation handling similar to Transifex and/or Launchpad with Focus on the stability against errors (i.e. language uploads to wrong language, broken pot uploads, ...) - all these issues which caused a lot trouble in the past for Launchpad and transifex. Also proper fuzzy handling to ease retranslations (which Laucnhpad is missing compared to transifex).

With SpamFilter I reached version 1.0 now (i.e. feature complete for now), so I'm anyway searching for a new target and this would involve JavaScripting a bit more :-) Could be my winter holidays task.

What do you think? In case I fail it will delay Transifex migration (which I anyway delayed because I'm missing motivation). In case it works we have an easy installable OpenSource translation service for many projects (which use Trac).

The current Trac SpamFilter is a sole result of JOSM Trac (as I fixed and reworked the original one to keep this Trac instance working) and now I18NForTrac could be a similar success :-)

[Don-vip]

Whatever working solution instead of this shitty, unmaintained Launchpad! I didn't tried Transifex a lot yet so I cannot judge.
A solution based on Trac looks nice as it would be simpler both for us and our translators :) If you're motivated to do this, go ahead!

[Mkyral]

What about simple downloading of .po file and let user to choose a favorite po editor? And back upload via Trac ticket, later could be automatized.

[Don-vip]

I think that would be a burden for the majority of translators who would prefer a web interface.

[jezevec]

Maybe check this ​https://translatewiki.net/

[Mkyral]

Very nice.

[bastiK]

Replying to stoecker:

I used transifex for other tools in the last weeks and the more I use it, the more I dislike it. Major flaws I see after short time:

• It will drop all related translations if only a single time the pot file upload is broken!

Is this fixed by simply reuploading the file?

• Administration is overly complicated

Very general statement, is this a real a problem?

I thought about making my own service after beeing confronted with Launchpad for the first time, but found it too much work. Then I though about making a translation service based on Trac and still found it too complicated and too restricted. Now I did a lot more Trac development (spamfilter, translatedpages macro, ...) and I'm convinced that the best solution would be a Trac plugin for project translation.

There are thousands of open source projects and probably hundreds with a similar or larger number of strings to translate. This problem is not unique to JOSM. I think having to create an extra account is acceptable for translators, as they will spend a lot of time with this anyway.

From the perspective of someone who is not very interested in the i18n stuff, this sounds like an huge investment of time and energy to get something perfect that we could have 98 % perfect almost for free.

On the other hand, if you think it's fun, then sure, why not...

[stoecker]

Replying to bastiK:

Replying to stoecker:

I used transifex for other tools in the last weeks and the more I use it, the more I dislike it. Major flaws I see after short time:

• It will drop all related translations if only a single time the pot file upload is broken!

Is this fixed by simply reuploading the file?

It seems not.

• Administration is overly complicated

Very general statement, is this a real a problem?

Yes.

From the perspective of someone who is not very interested in the i18n stuff, this sounds like an huge investment of time and energy to get something perfect that we could have 98 % perfect almost for free.

On the other hand, if you think it's fun, then sure, why not...

I'm searching for interesting tasks. This seems to be one.

[Don-vip]

Concerning i18n, I gave a look to our translations status on Launchpad:

Our lowest translation rates concern Estonian and Bulgarian, with 27% and 28% ot the total number of strings (core+plugins).

Three languages have better scores, but are not yet included in the software:

• 30%: Khmer (km)
• 83%: Catalan (Valencia) (ca@valencia)
• 97%: Asturian (ast)

Asturian has one of the very best translation scores so I think we must include it right now.

Catalan de Valencia has a very good score, I'm just not sure about its code: is really ca@valencia the code we must use? It looks strange.

Khmer has a lower score but it's better than 2 other languages we offer, and it has been updated only three hours ago, so including it into JOSM core seems reasonable to me.

What do you think?

[stoecker]

Rule is stated here:

Translations: New languages will be added to JOSM when there are at least 2000 translated strings.

Catalan de Valencia has a very good score, I'm just not sure about its code: is really ca@valencia the code we must use? It looks strange.

Hmm, does Java know about it? Any Java code assigned to it?

The language "ast" (and maybe "km") sounds like "must be included".

To do:

• fix I18n.java to support it (proper multi-string encoding setting).
• fix launchpad.pl to support it (add to supported hash)
• create StartupPage and VersionHistory pages (and links in the history pages).
• Update TracLanguages when not done (Ast is included)
• build files.
• Anything I forgot. It's a long time since last change.

Should I or do you want to do it?

[Don-vip]

I'll give it a try, to learn :) Thanks!

[Don-vip]

Replying to stoecker:

Hmm, does Java know about it? Any Java code assigned to it?

Valencian: Not yet. There are ​some big i18n enhancements in Java 8, including the adoption of ​CLDR. But Valencian has only been ​added in CLDR v25 while ​Java 8 only includes CLDR v21. So it seems we can't support this locale until Java 9 is released.

Asturian: No JRE support but it has been ​added to CLDR v22, so same as Valencian :(

Khmer: No JRE support but it is included in CLDR for a long time, so I should be able to make it work somewhat for those who run Java 8

[stoecker]

I think we can add languages also without full Java support. Only thing not working should be the auto-language detection I think. Selecting language by hand (in prefs or commandline) should be possible. But we never tested it.

[Don-vip]

I have added experimental support of Khmer with Java 8 in r7894 r7895 but it does not seem to work well:

I'm not sure if the .lang file is corrupted, or if it's a Java/Windows problem? I do not have any Linux machine right now to test, can you please give it a look?

[Don-vip]

Problems fixed in r7896 by choosing one of the fonts installed by default on Windows 7. Fonts list probably needs to be adapted to include other possible fonts such as:

• ​https://www.archlinux.org/packages/extra/any/ttf-khmer/
• ​http://rpm.pbone.net/index.php3/stat/4/idpl/26588939/dir/scientific_linux_other/com/google-noto-sans-khmer-fonts-20130807-2.el7.noarch.rpm.html
• ​http://scripts.sil.org/cms/scripts/page.php?site_id=nrsi&id=Mondulkiri
• ​https://tracker.debian.org/pkg/fonts-khmeros

[stoecker]

In 7897/josm:

see #8645 - also enable new language on older systems

[stoecker]

With my java 1.7.0_55 choosing Khmer works by hand, but not automatic, so I removed the Java 8 check. I have the khmeros fonts installed and texts look fine.

I think you can add "ast" as well without waiting for newer Java.

[Don-vip]

Ok thanks :)

[jbcorbeille-josm@…]

Hello,
Any news here ?
I just cannot modify or add any translation.
For the fun, « note » in the "new" notes fonctionnality is translated in French to « paiement », meaning « payment », probably like in banknote. But for now, it will remain that way, I could get nowhere with launchpad, getting oopsies after oopsies.
JB.

[stoecker]

Replying to jbcorbeille-josm@…:

Any news here ?

Not much, but I'm working on it.

Sorry for the long time, but the more I have to do at work the lower is the assigned amount of computer-time in my freetime :-)

I just cannot modify or add any translation.
For the fun, « note » in the "new" notes fonctionnality is translated in French to « paiement », meaning « payment », probably like in banknote. But for now, it will remain that way, I could get nowhere with launchpad, getting oopsies after oopsies.

Suggested workaround is to download the .po file, use a proper editor (e.g. kbabel, poedit, gted or a simple text editor) and reupload the changed file when done. That's BTW also the way I already worked in the past before these Launchpad delays came up. A proper editor is much better in any case.

[Don-vip]

In 7907/josm:

see #8645 - i18n - add payment text context to "Note" in default preset to avoid translation conflict with OSM Notes

[anonymous]

Wasn't the "Note" string already fixed elsewhere?

[Klumbumbus]

Replying to anonymous:

Wasn't the "Note" string already fixed elsewhere?

Yes, in r7785.

r7907 is wrong, because it is not payment, but the tag note=.... I will remove this, as there is no reason for note=* in the vending machine preset (it is in no other preset).

[Klumbumbus]

see r7908

[skyper]

Replying to Klumbumbus:

....
I will remove this, as there is no reason for note=* in the vending machine preset (it is in no other preset).

This preset is really outdated, see #6268.

[Klumbumbus]

Replying to jbcorbeille-josm@…:

getting oopsies after oopsies.

Sometimes(!) it helps to reduce the number of displayed untranslated strings. For this change the URL from ...batch=10... to e.g. ...batch=2...
If this does not help, then only the download and reupolad of the po file and the hope that nobody else edits at the same time is the way to go.

[Don-vip]

Replying to Klumbumbus:

r7907 is wrong, because it is not payment, but the tag note=....

Indeed I was misleaded by the French translation which was horribly wrong in fact :D Thanks.

[Don-vip]

move tickets that have not been treated this month to next milestone

[Don-vip]

after 4 years (!), Launchpad guys have finally decided to fix this problem:
​https://bugs.launchpad.net/launchpad/+bug/736005/comments/36

The search is finally working as expected, so I guess there's no need anymore to switch to a different platform.

[bastiK]

Wow, didn't see that coming.

[Don-vip]

We should spread the word that everyone can now use the web interface to translate JOSM, without becoming crazy :)

[Klumbumbus]

Replying to Don-vip:

We should spread the word that everyone can now use the web interface to translate JOSM, without becoming crazy :)

Yes, should be on StartupPageSource :)

[stoecker]

A note: I'll continue development of a Trac based alternative, which allows to directly integrates the JOSM workflow (input and output) as plugin, but gladly I have more time now to do so and less pressure. Maybe when I have holidays in summer. I wanted to do this for years and now I have a concept ready which only needs to be programmed.

Launchpad is really fast again, only the translation download mail is slow as always :-)

[simon04]

Should we:

• revert [o30039]?
• revert [o30227]?
• remove ​https://www.transifex.com/projects/p/josm/?

[Klumbumbus]

Atleast we should find a way that people do not longer translate strings at transifex since this is wased time/effort.

[stoecker]

No revert of the infrastructure parts I think. I'd simply clear the project at transifex and leave it empty.

[stoecker]

Cleared the project.

[anonymous]

A great shame, transifix makes it really easy to do translations with. The issue was never raised because of bugs with Launchpad but because of it being cumbersome to use.

[Don-vip]

It's happening again! Looks like we only had a few months of peace and we're going to face again this real pain:

(Error ID: OOPS-8290302df82f2cb4a8e0ec50e0781f86)
(Error ID: OOPS-9d95aea497076c6419a8a80e7dd75f0b)

Time to move on for good?

[Don-vip]

As ​Dirk points out, the situation has not improved. A volunteer to help to resume the transition to Transifex?

[openstreetmap.org-user-d1g]

As translator, I find transifex.com is more reliable than Launchpad. It is faster to translate at transifex because of UI, short-cuts and powerful filters.

Automatic diff/history of translations makes translation unbelievably easy compared to other tools.

It is impossible to discuss/leave remarks at Launchpad AFAIK.

Unlike Launchpad, transifex is ready for serious translation efforts (​translation memory, ​semi-automatic translation assistance), glossaries.

Many OSM projects are hosted at transifex.com. It is very easy for translators to switch between ​QGIS/​OSM/JOSM/<s>OsmAnd</s> translations if they all hosted at one platform (currently this platform is transifex.com IMO).

Also, I don't like to use tools that doesn't work :)

[openstreetmap.org-user-d1g]

If we switch, we should ask for ​“Open Source” program later on, as JOSM may qualify as "What is considered an Open Source project?" ​http://docs.transifex.com/faq/

[aceman]

Replying to openstreetmap.org-user-d1g:

Many OSM projects are hosted at transifex.com. It is very easy for translators to switch between QGIS/OSM/JOSM/OsmAnd translations if they all hosted at one platform (currently this platform is transifex.com).

Not sure what you mean by OSM, but OsmAnd is on Weblate.org.

[Don-vip]

I just can't fix translations errors right now. Tried about 20 times to bypass OOPS errors to search a string in Italian language and gave up. Is there someone willing to step up to finish the migration please?

[Don-vip]

Another problem with Launchpad, the translations are not synchronized for three days now: ​https://bugs.launchpad.net/launchpad/+bug/1676975/comments/2

[floscher]

Hello,
let's continue the discussion from the mailing list ​https://lists.openstreetmap.org/pipermail/josm-dev/2018-January/007954.html.

I prepared a patch for the i18n/build.xml, which would exclude JOSM plugins with a file .tx/config from pushing translations to Launchpad. It would also exclude plugins without a data/ directory from importing translations from Launchpad.
This would affect the plugins Mapillary and geojson.

The other option would be to remove the svn:external for both plugins and exclude them from the Ant build altogether.

One question that came to my mind in this context: What's determining if a JOSM plugin that's hosted on GitHub is shown as "from an external source"? Is it that the project is under https://github.com/JOSM/ or is it that it's part of the Ant build?

---

Another topic I thought about was in which organisation on Transifex the translations should be maintained in. Two options: ​JOSM and ​OpenStreetMap.
I'm currently leaning towards the JOSM organisation, mainly because it could violate the GPL license of the translations to be reused in other projects in the OpenStreetMap organisation on Transifex. And I don't want to create even more places where strings for JOSM can be translated. One location on Launchpad and one location on Transifex should be enough, and the JOSM organisation is probably a location more developers could agree on.

In the case that we'd conclude in the future to join translation forces with ​https://www.transifex.com/openstreetmap/, we could still migrate the translations over.

Would you find it sensible to put all new translations made on Transifex under a more liberal license that would allow easier mixing and matching with other projects?

[floscher]

Patch to exclude plugins with a .tx/config file from being translated on Launchpad and only write the translations from Launchpad to plugins containing a data/ directory

[floscher]

Improved version of the previous patch, the <if></if> was surrounding too much of the plugintrans target.

[stoecker]

Replying to floscher:

The other option would be to remove the svn:external for both plugins and exclude them from the Ant build altogether.

I'd prefer this. The external is only there for translations, so removing it is the right thing. The patch would cause useless resources, as svn checkout fetches ressources which aren't needed at all.

One question that came to my mind in this context: What's determining if a JOSM plugin that's hosted on GitHub is shown as "from an external source"? Is it that the project is under https://github.com/JOSM/ or is it that it's part of the Ant build?

A project where JOSM admins have access and can do fixes and i18n updates (i.e. part of JOSM on GitHub).

Would you find it sensible to put all new translations made on Transifex under a more liberal license that would allow easier mixing and matching with other projects?

Individual short translations can't have a license. Only a set of translations gets licensable. Same situation as for the OSM database. So using individual strings is not a problem. Only copying bigger parts (or longer individual texts).

[holgermappt]

Replying to stoecker:

Replying to floscher:

Would you find it sensible to put all new translations made on Transifex under a more liberal license that would allow easier mixing and matching with other projects?

Individual short translations can't have a license. Only a set of translations gets licensable. Same situation as for the OSM database. So using individual strings is not a problem. Only copying bigger parts (or longer individual texts).

What is the license of the translations made on Transifex? I don't see anything regarding a license. That would mean no license, right? And no license means no right for anyone, right? Isn't that a bad idea? So yes, any license more liberal than "no right for anyone" would be highly appreciated. And no, copying strings or words one-by-one because "individual short translations can't have a license" is not an option.

[floscher]

Replying to holgermappt:

Replying to stoecker:

Replying to floscher:

Would you find it sensible to put all new translations made on Transifex under a more liberal license that would allow easier mixing and matching with other projects?

Individual short translations can't have a license. Only a set of translations gets licensable. Same situation as for the OSM database. So using individual strings is not a problem. Only copying bigger parts (or longer individual texts).

What is the license of the translations made on Transifex? I don't see anything regarding a license. That would mean no license, right? And no license means no right for anyone, right? Isn't that a bad idea? So yes, any license more liberal than "no right for anyone" would be highly appreciated. And no, copying strings or words one-by-one because "individual short translations can't have a license" is not an option.

Nobody suggested to not put them under a license. Until decided otherwise I think the translations (should) have the GPLv2 or later license like the JOSM source code. Most of the translations for the Mapillary and geojson plugins are imported from Launchpad, so these have to be under GPL. My question was, if the translations should be put under a license that's less strict than GPL about where the strings can be reused (e.g. via Transifex's translation memory). But if the translation license really only matters when someone wants to copy the whole set of translations, reusing single strings in projects with other licenses shouldn't be a problem.

Where would I add the license that the translations are under? I couldn't find an option in the web interface and if I add it to the source files, the translators probably wouldn't see it.

[Don-vip]

Patch applied in [o34023].

[stoecker]

Related to the gradle builds? See #15809.

[stoecker]

Replying to Don-vip:

Patch applied in [o34023].

I18n build is broken.

[Don-vip]

@floscher: can you please take a look?

[floscher]

Replying to stoecker:

Replying to Don-vip:

Patch applied in [o34023].

I18n build is broken.

Is it directly related to the patch, though? The i18n build completed successfully two times since the patch was applied.
I will investigate further what caused the build to break. The build log says somehow the ​`msgcomm` command to create the `plugins.pot` failed.

[stoecker]

Didn't see that error message. Probably got confused by all these "filterpluginsource:" (can we get rid of these?).

Now it works. Does somebody know what changed?

[floscher]

Replying to stoecker:

Didn't see that error message. Probably got confused by all these "filterpluginsource:" (can we get rid of these?).

Probably the ​NoBannerLogger would get rid of these messages. If I understand it correctly, it would suppress logging of target names for targets without logging output.

Now it works. Does somebody know what changed?

I have no clue. On my machine I couldn't reproduce the issue and I couldn't find a possible cause in the build log.

[simon04]

• Prepared the splitting of core resources in [o34686:34689].
• Translation template core.pot has been uploaded to Transifex: ​https://www.transifex.com/josm/josm/core/
• Translations have been uploaded from a Launchpad export. Probably this step should be done again using a dedicated user like ​https://www.transifex.com/user/profile/JOSM_Translation_auto_updater/ or JOSM_Translation_migrator?

Transifex expects different plural forms for some languages, for instance cs:

# Launchpad
"Plural-Forms: nplurals=3; plural=(n==1) ? 0 : (n>=2 && n<=4) ? 1 : 2;\n"
# Transifex
"Plural-Forms: nplurals=4; plural=(n == 1 && n % 1 == 0) ? 0 : (n >= 2 && n <= 4 && n % 1 == 0) ? 1: (n % 1 != 0 ) ? 2 : 3;\n"

The plural form on Transifex seems to match ​http://www.unicode.org/cldr/charts/latest/supplemental/language_plural_rules.html Just fixing the Plural-Forms and uploading the translations did not work, see ​https://www.transifex.com/josm/josm/translate/#cs/core/156001581 for an example which was translated on Launchpad, but is now missing. Does anyone know a solution for ​plural message migration? I will contact Transifex and ask them as well. Some tickets in other FOSS projects I found are ​https://github.com/mate-desktop/mate-desktop/issues/257, ​https://github.com/mumble-voip/mumble/issues/2854, ​https://code.djangoproject.com/ticket/26123.

[floscher]

For the plugins that I migrated to Transifex, I simply deleted pluralized strings in the Launchpad export for these languages (it weren't many). The only solution I can think of that would keep existing translations, would be to add an additional plural to the *.po files with a placeholder string (e.g. english plural (translate this on transifex)).

Should we make the core resources on transifex read-only, so nobody translates strings now, which would get overriden by the dedicated user importing from Launchpad again?

Is the current plan to leave the plugins in the same project together with core? Or should they move to the new project 'JOSM-plugins'? I categorized the resources so you can filter the list of resources by core, plugin and website.
It seems moving a resource from one project to another inside transifex is not trivial and would throw away translation history and translation dates: ​https://docs.transifex.com/projects/moving-content
If the plugins should be a separate project, it would be better to rename the current josm project to josm-plugins and put the core files in a new project.

[holgermappt]

There are some translations that are in Launchpad for some time but not in the Transifex import. Was there some filter applied? It would be some extra work to copy the missing string manually. E.g. the untranslated German strings from the pluginlist are in Launchpad (at least the strings I looked at).

I also think it would be a good idea to set one of the instances to read only to avoid work that will be discarded later.

[Don-vip]

Replying to floscher:

Should we make the core resources on transifex read-only, so nobody translates strings now, which would get overriden by the dedicated user importing from Launchpad again?

I guess so. Now that we have some strings we should try to build a local JOSM using transifex data, and check the about dialog (translator credits).

Is the current plan to leave the plugins in the same project together with core? Or should they move to the new project 'JOSM-plugins'? I categorized the resources so you can filter the list of resources by core, plugin and website.

I think having two projects is the best approach.

It seems moving a resource from one project to another inside transifex is not trivial and would throw away translation history and translation dates: ​https://docs.transifex.com/projects/moving-content

Good to know. What happens if we merge different resources in the same project? Is it easy? Do we loose something?

If the plugins should be a separate project, it would be better to rename the current josm project to josm-plugins and put the core files in a new project.

Good idea!

[simon04]

Replying to floscher:

For the plugins that I migrated to Transifex, I simply deleted pluralized strings in the Launchpad export for these languages (it weren't many). The only solution I can think of that would keep existing translations, would be to add an additional plural to the *.po files with a placeholder string (e.g. english plural (translate this on transifex)).

Is is probably not an option since there are 178 trn strings.

Should we make the core resources on transifex read-only, so nobody translates strings now, which would get overriden by the dedicated user importing from Launchpad again?

Done by disabling "Your translators can translate resource strings".

Is the current plan to leave the plugins in the same project together with core? Or should they move to the new project 'JOSM-plugins'? I categorized the resources so you can filter the list of resources by core, plugin and website.

Ah, good to know that. Then we probably do not have to split.

After importing also the non-shipped languages to Transifex, here's the result:

• plural problems in ar, be, cs, fa, he, lt, pl, ro, ru, sk, uk
• missing language code on Transifex: awa, hil, skr, wae (all <1% done)

The changed plural forms are tricky; we need to:

• somehow bring the plural strings to Transifex
• update the I18n.PluralMode enums accordingly, update plugin main version
• perform I18n update for JOSM and all plugins

Damn. As if the migration would not have been complicated enough. What a pity that Transifex does not permit custom/non-default plural rules.

[floscher]

What happens if we merge different resources in the same project? Is it easy? Do we loose something?

I'll test that. My guess is that inside one project the strings can be moved around between resources without losing anything (same with merging).
Maybe it would be best to split not between core and plugins, but between core with officially supported plugins and not officially supported plugins, so translations can be shared easier at least with the official plugins.

Is is probably not an option since there are 178 trn strings.

For some of these languages, actually not all of these strings are translated (e.g. ar 18, fa 15).
My current understanding is that Transifex doesn't accept translations that only translate a subset of all plural forms. So if we want to import to transifex, we need to provide something as fourth plural form (e.g. english plural, one of the other plurals of that language, generic placeholder.

[anonymous]

What is the plural forms problem for 'sk' ? What rule does Transifex have?

Also the rule for 'cs' seems kinda bogus. Most other projects use 3 forms for cs and sk.

Transifex claims to go by ​http://www.unicode.org/cldr/charts/latest/supplemental/language_plural_rules.html . If the 4th form it uses is the fractional form ("many") then that form is not used in JOSM. And then there should be easy mapping from JOSM forms to the Transifex form.

Please do not delete any forms from JOSM sources until this is resolved and the mapping is done (i.e for cs you map JOSM=>Transifex: 0 => 0; 1 => 1 and 2 => 3).

[Klumbumbus]

There are about 1850 untranslated strings in the german transifex. In Launchpad only 36. Anything went wrong there? attachment:josmtransifex.PNG​

[anonymous]

Replying to anonymous:

What is the plural forms problem for 'sk' ? What rule does Transifex have?

# Launchpad
"Plural-Forms: nplurals=3; plural=(n==1) ? 1 : (n>=2 && n<=4) ? 2 : 0;\n"
# Transifex
"Plural-Forms: nplurals=4; plural=(n % 1 == 0 && n == 1 ? 0 : n % 1 == 0 && n >= 2 && n <= 4 ? 1 : n % 1 != 0 ? 2: 3);\n"

Transifex claims to go by ​http://www.unicode.org/cldr/charts/latest/supplemental/language_plural_rules.html . If the 4th form it uses is the fractional form ("many") then that form is not used in JOSM. And then there should be easy mapping from JOSM forms to the Transifex form.

Please do not delete any forms from JOSM sources until this is resolved and the mapping is done (i.e for cs you map JOSM=>Transifex: 0 => 0; 1 => 1 and 2 => 3).

According to floscher, we either provide an initial value for the added plural form (e.g. 3 → 3+4) or we cannot import the trn strings. Is this sensible for cs, for sk?

Replying to Klumbumbus:

There are about 1850 untranslated strings in the german transifex. In Launchpad only 36. Anything went wrong there? attachment:josmtransifex.PNG​

For now, I only imported core, but not yet maplist/pluginlist/presets. The process takes some time and first I'd like to have the plural issue clarified.

[aceman]

It seems 'n % 1 != 0' means fractional (non-cardinal) number.
Yes, if you copy form "other" of JOSM into form "many" and "other" of Transifex that is fine for cs and sk. I think JOSM never displays fractional numbers in the strings so it is not very important what form "many" of Transifex will contain. But if ever the form "many" displays, having the value of JOSM form "other" is bearable for the language reader.

But I messed the numbering up. The mapping is like this:
JOSM form 0 => Transifex form 3 ("other")
JOSM form 1 => Transifex form 0 ("one")
JOSM form 2 => Transifex form 1 ("few")
JOSM form 0 => Transifex form 2 ("many")

[floscher]

When looking into ar language, it seems Launchpad and Transifex support the same six plural forms:

• 0
• 1
• 2
• n % 100 >= 3 && n % 100 <= 10
• n % 100 >= 11 && n % 100 <= 99
• everything else

The problem seems to be, that some strings are translated on Launchpad to only some of these six forms (18 strings are translated, 8 of those have all six forms).

Excerpts from josm-ar.po:

msgid "Combine {0} way"
msgid_plural "Combine {0} ways"
msgstr[0] "ضم {0} خط"

msgid "object"
msgid_plural "objects"
msgstr[0] "غرض"
msgstr[1] "غرضين"
msgstr[2] "أغراض"

msgid "node"
msgid_plural "nodes"
msgstr[0] "عقدة"
msgstr[1] "عقدتين"
msgstr[2] "عقدات"
msgstr[3] "عقد"
msgstr[4] "عقد"
msgstr[5] "عقد"

[floscher]

Replying to Don-vip:

It seems moving a resource from one project to another inside transifex is not trivial and would throw away translation history and translation dates: ​https://docs.transifex.com/projects/moving-content

Good to know. What happens if we merge different resources in the same project? Is it easy? Do we loose something?

I played around a bit with transifex, this is how it seems to work:

• if you upload new strings to a resource, transifex searches all resources from that project and uses exactly matching translations for the new strings
• if two resources A and B have the same string and you change the translation in resource A, the translation of the same string in resource B only changes if it was previously untranslated in resource B

So merging in the same project shouldn't be a problem. Once a string is translated inside one project, it can be moved around to other resources and will stay translated.

[Don-vip]

OK so maybe a single project for both core resources and plugins is a better option. We want to be able to merge plugins in core easily and without loosing anything.

[Don-vip]

@simon04 it seems there's a problem with the latest i18n changes, see jenkins/job/JOSM-i18n/1182/parsed_console/ we have 8117 warnings now.

Links for reference: [o34686:34689], [o34692:34696]

[simon04]

The culprit was [o34694]. I reverted this one in [o34697].

[simon04]

I implemented a small too for the plural message translation, ​https://github.com/simon04/plural-message-migration. For the moment, it supports the currently shipped languages with changed plural forms, namely be, cs, lt, pl, ru, sk, uk. I applied the migration to tonight's Launchpad export and updated all shipped Languages on Transifex. I implemented the following plural message migrations (Launchpad index → Transifex index):

• be: 0→0, 1→1, 2→2, 2→3
• cs: 0→0, 1→1, 2→2, 2→3
• lt: 0→0, 1→1, 2→2, 2→3
• sk: 1→0, 2→1, 0→2, 0→3 (see comment:129)
• pl: 0→0, 1→1, 2→2, 2→3
• ru: 0→0, 1→1, 2→2, 2→3
• uk: 0→0, 1→1, 2→2, 2→3

Please review the translations on Transifex. In a nutshell and excluding sk, the new plural forms include a newly "frational"/"other" form at index 3 (or 2 for lt), and the "many" strings have been copied to the "fractional" strings.

[aceman]

Should we already see the migrated strings on Transifex? The 'core' resource only contains imagery names, which don't have any plurals so we can't check the migration.

[simon04]

The core resource is this ​https://www.transifex.com/josm/josm/core/. This one already been updated and should contain all strings of JOSM core except imagery, plugin descriptions, presets.

[Don-vip]

Replying to simon04:

The culprit was [o34694]. I reverted this one in [o34697].

Thanks. I confirm this is fixed. Jenkins is happy again :)

[aceman]

OK, I can sometimes see all the core strings, somehow the "sub-resources" of core (presets, maplist) are shown instead of the pure 'core' one. It is quite a mess in the Transifex Web UI. Best bet seems to be to display all at once, but that is what I do not want to do (but only translate some resources). I also only see about 5 plugin resources and can't access any others.

So, the 'core' plurals seem OK, but e.g. "Plugin > Mapillary" resource has plurals wrongly ordered. You probably didn't apply the transformation from comment 135 to that one yet (it is dated 21 Oct).

[floscher]

Replying to aceman:

OK, I can sometimes see all the core strings, somehow the "sub-resources" of core (presets, maplist) are shown instead of the pure 'core' one. It is quite a mess in the Transifex Web UI. Best bet seems to be to display all at once, but that is what I do not want to do (but only translate some resources). I also only see about 5 plugin resources and can't access any others.

So, the 'core' plurals seem OK, but e.g. "Plugin > Mapillary" resource has plurals wrongly ordered. You probably didn't apply the transformation from comment 135 to that one yet (it is dated 21 Oct).

The resources starting with plugin › were imported to Transifex separately before the current import, so that's unrelated to the migrations simon04 is doing.

Which languages have the plurals in the wrong order? What would be the correct one?

[aceman]

I have only checked 'sk' (but if that comes out correct, then I think you need the same order also for 'cs').
The 'core' is in correct order, but plugins are not. So those will need re-importing according to the rules in comment 135.

[Don-vip]

IIRC someone told me about Weblate: ​https://weblate.org/en/hosting/free/

It's free software and used by OpenSUSE & OsmAnd. Maybe we could setup a test instance and compare with Transifex to see which is the better choice for us?

[simon04]

This document lists even more tools to evaluate: ​https://docs.google.com/spreadsheets/d/1t6Zo9OmlRCH3N9dOumnWn0w8QrgI9Is2zQXSJB7K-QI/edit

From a translator's perspective, I like the frontend of Transifex. It is responsive and allows searching for already translated similar phrases. With respect to uploading templates and downloading translations, I think we resolved most issues/problems. I'm not so keen to evaluate an alternative solution …

[Don-vip]

OK it was mainly a suggestion in case we face blocking problems with Transifex :)

[stoecker]

Replying to Don-vip:

IIRC someone told me about Weblate: ​https://weblate.org/en/hosting/free/

That one looks good. We could setup our own instance and it is actively developed. It misses a display of similar strings as far as I see, but has a Glossary function OTOH (after a first glance).

This document lists even more tools to evaluate

Nothing better. Either already evaluated or proprietary and remotely hosted. From these services transifex was the best, but I always searched for a software we can setup ourselves. A result could be, that e.g. StartupPage translation could be integrated instead of using its own system. Also status display to users would be more immediate.

[Don-vip]

Replying to stoecker:

IIRC someone told me about Weblate: ​https://weblate.org/en/hosting/free/

That one looks good. We could setup our own instance and it is actively developed. It misses a display of similar strings as far as I see, but has a Glossary function OTOH (after a first glance).

We could, but Simon has spent a lot of time to resolve transifex issues, so I'm also in favor to finish the work he started, unless we find an absolutely blocking issue. But this is good to know we have an hosted alternative in case we get big problems with Transifex in the future.

[floscher]

Replying to aceman:

I have only checked 'sk' (but if that comes out correct, then I think you need the same order also for 'cs').
The 'core' is in correct order, but plugins are not. So those will need re-importing according to the rules in comment 135.

The translations of plugin strings for language sk should now be in the correct order. There's one string from the wikipedia plugin, that had the same translation for all four plural forms, for which I created a ​Transifex issue.

The cs translations should already be in the correct order, because in Launchpad the sk order is: other, one, few (between 2 and 4). The cs order is: one, few (between 2 and 4), other .

# Launchpad cs
plural=(n==1) ? 0 : (n>=2 && n<=4) ? 1 : 2
# Launchpad sk
plural=(n==1) ? 1 : (n>=2 && n<=4) ? 2 : 0

[floscher]

What is the status of the migration to transifex? What still needs to be done to complete it?

[aceman]

Thanks, also the "plugins > Mapillary" is now correctly migrated for 'sk'.

The ​https://www.transifex.com/josm/josm/translate/#sk/$/142419252?q=issue%3Aopen is not a big issue, it is hard to create plural forms for the word 'ID' :) But yes, I can change it to 'identifiers' and then it works.

Where should we make translations currently? At launchpad or transifex? Which is the primary source now for JOSM releases?

[floscher]

Replying to aceman:

Where should we make translations currently? At launchpad or transifex? Which is the primary source now for JOSM releases?

The resources that are on Transifex and that are editable (currently 7 plugins) are translated ​on transifex.com.

Everything else is currently translated ​on launchpad.net, that's why the resources for JOSM core are set to read-only in Transifex.

It seemed to me that the migration was almost done, but now there was no further activity for some weeks. So probably the strings from Launchpad would have to be exported again and imported to Transifex, so the translations made in the meantime don't get lost.

[holgermappt]

Any news? Lately it takes 5 to 10 reloads at Launchpad to go to the untranslated strings due to the infamous Timeout error.

[Klumbumbus]

Ticket #19085 has been marked as a duplicate of this ticket.

[anonymous]

Replying to stoecker:

Replying to Don-vip:

IIRC someone told me about Weblate: ​https://weblate.org/en/hosting/free/

That one looks good. We could setup our own instance and it is actively developed. It misses a display of similar strings as far as I see, but has a Glossary function OTOH (after a first glance).

Weblate now has string grouping.

This document lists even more tools to evaluate

Nothing better. Either already evaluated or proprietary and remotely hosted. From these services transifex was the best, but I always searched for a software we can setup ourselves. A result could be, that e.g. StartupPage translation could be integrated instead of using its own system. Also status display to users would be more immediate.

Having used all options extensively, Weblate is the best option. I thought Hosted Weblate would be the best bet.
Transifex is a bucket of problematic eels.

[simon04]

See also the same discussion for iD: Replace Transifex · Issue 7508 · openstreetmap/iD · ​https://github.com/openstreetmap/iD/issues/7508

[stoecker]

Maybe we should reevaluate weblate?

A second option to relieve translators would be to split launchpad translation into the 3 sections we already have. That should reduce the timeouts a lot, as they seems to depend on the total translation size.

[Don-vip]

It seems the original timeout issue of Launchpad is gone. I had no problem to perform searches this morning. First time in years. So maybe we can keep Launchpad after all, if the issue doesn't come back.

[stoecker]

Replying to Don-vip:

It seems the original timeout issue of Launchpad is gone. I had no problem to perform searches this morning. First time in years. So maybe we can keep Launchpad after all, if the issue doesn't come back.

It always depended a bit on the time when you use it, but they did improvements recently. :-) Splitting it in 3 sections may nevertheless be a good idea even if not needed because of the timeout issue? Opinions?

[Don-vip]

Sounds a good idea indeed. Does it have any downside?

[stoecker]

Replying to Don-vip:

Sounds a good idea indeed. Does it have any downside?

My thoughts regarding that:

• Pro: Makes it easier to know what's plugin, core, other, ...
• Contra: Makes it easier to ignore plugins, other, ...
• I'm not sure what happens when strings move between the groups. Are they kept or at least easy available?
• We should be careful to not loose translations in the process even for the many many partial languages when we change that
• Upload/Download scripts must be checked
• I think lots/most of work on the scripting side was already done

[stoecker]

Idea for Migration:

• Add the 3 additional POT files (or first only 1 :-)
• Check everything and re-import translations into the new files when necessary.
• Last step drop the combined POT file

[stoecker]

Seems Transifex migration makes no more sense. Improvements to launchpad file handling should be handled in an own ticket when someone wants to do the work.

[Don-vip]

Launchpad timeout errors are back... :'(

[Klumbumbus]

Yes it's not possible to search a string on launchpad atm.

[Hb---]

I'd like to present the offline translation solution in use here. The base is the OmegaT programm added by WinMerge as diff viewer and a crappy batch file for some automatics.

OmegaT provides a fast search with regex et al. This makes it easy to maintain and uphold the consistency of a translation.

@echo off
echo This batch provides an offline translation cycle for JOSM. Steps are:
echo 1. Download the compiled po-file
echo 2. Automatically normalize the po-file
echo 3. Check and Translate with a diff tool and OmegaT
echo 4. Upload to Launchpad

rem Ensure the working directory is the batch files directory
set wrkdir=%~p0
cd %wrkdir%

rem STEP 1
rem Without Bazaar downloads can be requested via email link or done manually:
echo Please download the file "de.po" from the appearing web page
echo to the target directory %USERPROFILE%\Downloads\
echo Continue with any key to open the web browser.
pause >nul
echo Opening the web browser ...
echo [InternetShortcut] >%TEMP%\start.url
echo url=​https://bazaar.launchpad.net/~openstreetmap/josm/josm_trans/files/head:/josm >>%tmp%\start.url
%tmp%\start.url
echo .
echo Download finished? Continue with any key.
pause >nul
rem Move the downloaded file into the source directory of the
rem OmegaT normalisation project. It is important to rename the file
rem according to the email-based download. Otherwise OmegaT would see
rem two different source files and mismatch its translation memory.
if exist "%USERPROFILE%\Downloads\de.po" move %USERPROFILE%\Downloads\de.po .\omde2norma\source\josm_josm-de.po

rem STEP 2
rem Normalizing the incoming po file is needed to get acceptable diff
rem views. Launchpad delivers files with line breaks in msgstr strings.

rem Delete all translation memories to empty OmegaT project.
del .\omde2norma\omde2norma-*.tmx >nul
del .\omde2norma\omegat\project_save.* >nul

rem Run the project quietly
java -jar "C:\Program Files (x86)\OmegaT\OmegaT.jar" .\omde2norma\ --mode=console-translate --quiet
rem Delete memory files again
del .\omde2norma\omde2norma-*.tmx >nul
del .\omde2norma\omegat\project_save.* >nul
echo .

rem STEP 3
echo Now OmegaT and a file difference viewer are started in parallel.
echo The diff viewer compares the downloaded translation version with the
echo most recent local version.
echo .
echo Initially this is the version you uploaded the last time. So you see
echo which strings are new and what others have translated in the meantime.
echo .
echo After generating a translation in OmegaT (Ctrl+D) and rediffing you will
echo see your current translation work against the downloaded version.

start javaw -Duser.language=en -jar "C:\Program Files (x86)\OmegaT\OmegaT.jar" omde\

rem Wait soemseconds while OmegaT is starting up
ping localhost >nul
"C:\Program Files (x86)\WinMerge\WinMergeU.exe" omde\source\josm_josm-de.po omde\target\

rem STEP 4
echo Invoke the Launchpad Upload page ...
echo [InternetShortcut] >%TEMP%\start.url
echo url=​https://translations.launchpad.net/josm/trunk/+pots/josm/de/+upload >>%TEMP%\start.url
%TEMP%\start.url

rem Housekeeping
del %TEMP%\start.url
exit

rem NEEDED SETTINGS FOR OMEGAT
rem ==========================
rem Project|Properties for the normalisation and the translation projects
rem + Enable Sentence-level Segmenting = OFF (Default=On)
rem + Auto-propagation of Translations = OFF (Default=On)
rem
rem Project|Properties only for the normalisation project
rem + Translated Files Folder = path/to/source/folder/of/translation/project
rem
rem
rem Resommendations for OmegaT
rem View|
rem X Display source segments
rem
rem Options|Preferences
rem + File Filters
rem + PO File Format Options:
rem + Allow blank translations in target files = OFF (default=off)
rem + Skip PO Header = ON (default=off) ??
rem + View
rem Display all source segments in bold = OFF (default=on)
rem + Display active source segment in bold = ON
rem + Include first non unique segment ??
rem + Updates
rem + Automatically check for updates = OFF (default=on)

[stoecker]

There are pretty good offline tools for handling po-files:

I used KBabel in the past. It's now replaced by Lokalize. They are much better when doing larger works, but first small updates the webinterface is still the best solution.

[Hb---]

Replying to stoecker:
Lokalize is KDE based and offers binaries for Windows (Localize is non free). I'll try.

They are much better when doing larger works, but for small updates the webinterface is still the best solution.

Starting with a lame translation that has 20 % correct strings is a large work. Even maintaining a translation is an endless task. All the small bits done in web interfaces tend to disintegrate the consistency over time. Current example is #20353.

If the turn-around time for

1. Loading
2. Checking for new strings
3. Translating while being able to compare with similar strings already translated and
4. Uploading

is favorable, then I would always prefer an offline solution.

[Don-vip]

Search works again, Launchpad guy fixed the problem after Dirk complained :)

[stoecker]

Replying to Don-vip:

Search works again, Launchpad guy fixed the problem after Dirk complained :)

So a shell commands with 100 failures was enough to prove the issue. I had about 300 fails as it took some time to have a proper command. Was worth the work ;-)