Modify

Opened 11 years ago

Closed 3 years ago

Last modified 15 months ago

#8645 closed enhancement (wontfix)

Migrate translation to www.transifex.com

Reported by: ggeldenhuis Owned by: team
Priority: normal Milestone:
Component: Core Version:
Keywords: i18n hack-weekend-2018-10 transifex Cc: jezevec, openstreetmap.org-user-d1g, naoliv

Description

Hi
Would you support an effort to migrate translations of JOSM to the transifex.com website? I have been contributing to translations for iD using the website and have found it very useful and easy to use. More importantly it is very easy for non-technical people to contribute translations. There is thus much less barriers for entry. The translation effort for josm looks a bit daunting at first glance.

Attachments (8)

i18n.build.xml.patch (1.6 KB ) - added by simon04 10 years ago.
i18n.build.xml.v2.patch (2.7 KB ) - added by simon04 10 years ago.
8645_build.patch (6.1 KB ) - added by simon04 10 years ago.
khmer.png (33.1 KB ) - added by Don-vip 9 years ago.
exludeTransifexPlugins.patch (6.1 KB ) - added by floscher 6 years ago.
Patch to exclude plugins with a .tx/config file from being translated on Launchpad and only write the translations from Launchpad to plugins containing a data/ directory
excludeTransifexPlugins2.patch (4.6 KB ) - added by floscher 6 years ago.
Improved version of the previous patch, the <if></if> was surrounding too much of the plugintrans target.
hack-weekend-2018-10-i18n.png (85.3 KB ) - added by simon04 5 years ago.
josmtransifex.PNG (25.2 KB ) - added by Klumbumbus 5 years ago.

Download all attachments as: .zip

Change History (176)

comment:1 by stoecker, 11 years ago

Resolution: wontfix
Status: newclosed

From my point of view transifex actually is less useful. For launchpad all you need is a login.

So the answer is No.

comment:2 by simon04, 10 years ago

Resolution: wontfix
Status: closedreopened

I would also favour the move to transifex since it provides a more appealing user interface and the slowness of launchpad is really annoying. Some/many other FOSS projects have moved from launchpad to transifex (google for "launchpad to transifex"). Plus, other OSM projects utilize transifex as well: https://www.transifex.com/tag/openstreetmap/

comment:3 by Don-vip, 10 years ago

+1 I'm tired of Launchpad and it slowness. In fact I nearly stopped any translation effort, I think there are other people so annoyed by the failures they stopped as well. Plus having all OSM translation at a single place would be really interesting for consistency between website and editors :)

comment:4 by Don-vip, 10 years ago

https://bugs.launchpad.net/launchpad/+bug/736005 No progress in almost two years, it's a shame

comment:5 by stoecker, 10 years ago

I still don't like Transifex much, but there is NO progress for Launchpad. Maybe it really is time to move. Will you do an initial setup for tests and later the necessary automatisms for data up- and download?

comment:6 by skyper, 10 years ago

+1, I am really tired reloading Launchpad

comment:7 by simon04, 10 years ago

Set setup a transifex project: https://www.transifex.com/projects/p/josm/
Upload of the translations is in progress …

comment:8 by simon04, 10 years ago

I added a transifex client config file in [o30039].

Most languages should be present already; some require some manual interaction due to syntax errors in the .po files (transifex checks uploaded files using msgfmt -c).

comment:9 by ggeldenhuis, 10 years ago

Thanks! I have started work on the Afrikaans translation.

comment:10 by Don-vip, 10 years ago

Is it possible to have 2 resources ? "core" and "plugins" ?

comment:11 by Don-vip, 10 years ago

Also, is there a way to preserve translation history in the migration ?

in reply to:  11 comment:12 by simon04, 10 years ago

Replying to gerhardus.geldenhuis@…:

Thanks! I have started work on the Afrikaans translation.

Please note that we are currently testing/evaluating transifex …

Replying to Don-vip:

Is it possible to have 2 resources ? "core" and "plugins" ?

+1. This requires to modify some scripts in the i18n/ directory. We should also avoid to have some strings listed under core as well as plugins. Dirk, what do you think of this separation? Maybe split presets and imagery strings to a third package ("data"?)?

Replying to Don-vip:

Also, is there a way to preserve translation history in the migration ?

I don't see how to. The launchpad only allows to export translations in the PO/MO format (cf. https://translations.launchpad.net/josm/trunk/+export), which doesn't name the translator of each string. Is there any other way?

comment:13 by stoecker, 10 years ago

Note, that launchpad will get faster again as well, when we split josm into smaller subpackages. It has advantages and disadvantages. I personally like one big package more, as strings are shared between the packages.

comment:14 by Don-vip, 10 years ago

We could try to build distinct packages and count the number of shared strings, then decide what's best :)

in reply to:  14 comment:15 by simon04, 10 years ago

Replying to Don-vip:

We could try to build distinct packages and count the number of shared strings, then decide what's best :)

msgcomm po/core.pot po/plugin.pot | grep -c 'msgid ' gives 256

Btw: there's a msgcat command, but we'd need something like msgsubtract – still investigating ;-)

Last edited 10 years ago by simon04 (previous) (diff)

by simon04, 10 years ago

Attachment: i18n.build.xml.patch added

comment:16 by Don-vip, 10 years ago

Can someone do i18n so we can release new tested ? :)

Another question: is it possible with transifex to have automatic translations ? (an update of languages during nightly build ?)

in reply to:  16 comment:17 by simon04, 10 years ago

Replying to Don-vip:

Can someone do i18n so we can release new tested ? :)

i18n is on the way ... To perform this update, check out the complete JOSM repository and execute ant updatecore in i18n/. Then commit the contents of core/data.

Another question: is it possible with transifex to have automatic translations ? (an update of languages during nightly build ?)

There's a transifex client to push translation templates (pot files) and pull translated languages (po files). This can be performed automatically as it is currently for launchpad.

comment:18 by skyper, 10 years ago

Does anyone know how transifex handels outdated translations e.g. are all translations lost once a string is changed or even if only *_context is removed or are these translations preserved ?

comment:20 by simon04, 10 years ago

Keywords: i18n added; translate removed

in reply to:  18 comment:21 by aceman, 10 years ago

Replying to skyper:

Does anyone know how transifex handels outdated translations e.g. are all translations lost once a string is changed or even if only *_context is removed or are these translations preserved ?

This would interest me too. Other projects use a "fuzzy" feature, i.e. if the original English string changes, the translation is not lost, only marked "fuzzy". The translation will see which strings are fuzzy and can review/update the translated string again. In JOSM currently the string is lost and we need to translate it whole anew (or retrieve from backup). The "fuzzy" feature is not specific to .po files, e.g. the Weblate and Pootle translation systems have it too.
And I hope transifex supports export/import from .po files.

comment:22 by aceman, 10 years ago

Should we work on the old system for now and how will we get notified when to switch to the new system? I hope the data will get merged at that time.

comment:23 by simon04, 10 years ago

The transifex is currently used only for testing, so use launchpad for "productive" translations. Sure, when we decide to use transifex, we move the latest data from launchpad (i.e., the translations, not the translator itself, since it's unclear how to export as well as import that information).

comment:24 by simon04, 10 years ago

Split translations into core, data, plugins and uploaded templates to transifex. See https://www.transifex.com/projects/p/josm/language/de/ for an example.

data and plugins do not contain strings from core – see i18n.build.xml.v2.patch for generation details.

by simon04, 10 years ago

Attachment: i18n.build.xml.v2.patch added

comment:25 by stoecker, 10 years ago

No progress here. Would it be possible to add me as admin in the project? If not, can we create an admin user, where all JOSM admins know login data? I still don't understand the permission system of Transifex. It seems awful complicated to give permissions to someone.

What's necessary to proceed?

What about automatic upload of translations from server? Is there an ant task ready? A script?

Last edited 10 years ago by stoecker (previous) (diff)

in reply to:  25 comment:26 by simon04, 10 years ago

Replying to stoecker:

No progress here. Would it be possible to add me as admin in the project?

Done.

What's necessary to proceed?

What about automatic upload of translations from server? Is there an ant task ready? A script?

There's the Transifex client available which uses a configuration started in [o30039] and extended in attachment:i18n.build.xml.v2.patch. The client allows to tx push POT files to Transifex and tx pull the current PO files.

The next steps would be to split the PO files set in the config file via file_filter to obtain separate files per component (core/data/plugins) – after the current PO files from Launchpad are updated on Transifex… Then one would have to integrate the push/pull commands into the Ant build file. Finally a migration strategy is needed.

comment:27 by stoecker, 10 years ago

Ok, I now added an auto-upload to the project. Next step is to make the build process ready for downloading, so .lang files can be generated. The possibility to separate the translations already exists. So after a "tx pull" the only missing task is to merge the translations into one big file for each language again.

comment:28 by akks, 10 years ago

By the way, on Russian forum people start asking me why many validator warings are in English (they are using newly relaesd tested version, on launchpad there are no significat untranslated strings).

in reply to:  28 comment:29 by simon04, 10 years ago

Replying to akks:

By the way, on Russian forum people start asking me why many validator warings are in English (they are using newly relaesd tested version, on launchpad there are no significat untranslated strings).

I created a separate ticket #9626 since this issue is unrelated to the Transifex migration. Thanks for noticing! :-)

by simon04, 10 years ago

Attachment: 8645_build.patch added

in reply to:  27 comment:30 by simon04, 10 years ago

Replying to stoecker:

Ok, I now added an auto-upload to the project. Next step is to make the build process ready for downloading, so .lang files can be generated. The possibility to separate the translations already exists. So after a "tx pull" the only missing task is to merge the translations into one big file for each language again.

Patch attachment:8645_build.patch​ attached. This should do the trick.

(Today, I translated ≈30 outstanding German strings of JOSM by a sequence of downloads/uploads of po file/s from/to launchpad/transifex. The UI is awesome. Looking forward to the completed migration to transifex.) :-)

comment:31 by stoecker, 10 years ago

Hmm, the patch drops existing structure. I wanted to run the launchpad and transifex stuff for some time in parallel before switching:

  • download from launchpad
  • build lang files
  • upload the downloaded files to transifex
  • download from transifex
  • build lang files
  • compare the files (should be equal)

comment:32 by stoecker, 10 years ago

Also I don't like to drop a working infrastructure. Who knows if we may switch back one day?

comment:33 by simon04, 10 years ago

What about locally duplicating the i18n directory into i18n.lp and i18n.tx, applying the patch on i18n.tx? Then steps 1–3 can be performed in i18n.lp and the subsequent steps 4–6 can be performed in i18n.tx. Then one only needs to take care to obtain the XY.lang files from both processes.

comment:34 by simon04, 10 years ago

Dirk, what is the status of the migration? Is any help needed? I'd really love to see the migration to transifex.com competed. :-)

comment:35 by stoecker, 10 years ago

I'm still not perfectly happy with you last patch, but actually the major problem is, that I did not look into this. I updated the Trac spamfilter plugin lately to support statistics and that took all my time.

A patch which does not break the current i18n-build would nevertheless be appreciated. Simply clone the build targets in the build.xml instead of replacing them :-)

comment:36 by stoecker, 10 years ago

BTW: If someone wants to test Transifex already - Josm webpages also benefit from translation of Spamfilter :-)

comment:37 by Don-vip, 10 years ago

I really can't stand buggy Launchpad anymore, it's incredibly difficult to fix erroneous strings to achieve i18n. Can we switch to transifex next month?

comment:38 by stoecker, 10 years ago

No problem. The only problem I have with above patch is that it drops launchpad support. If it would simply add transifex without dropping launchpad I'd be happy. Should be easy to change it accordingly. But for some time now I do very little with my PC in free-time which also affects JOSM contribution.

comment:39 by stoecker, 10 years ago

P.S. Launchpad says they will update DB servers probably reducing this issues a lot :-)

comment:41 by Klumbumbus, 10 years ago

Any news to this? The Launchpad webinterface is still not usable. I always get the timeout error.

comment:42 by Don-vip, 10 years ago

Launchpad is still buggy and the "couple of weeks" promised delay is, not surprisingly, not respected at all. I have some free time to look at this subject this week-end if neither of you can finish the patch. See #10257: people don't even try to fix translation issues, and we can't really blame them.

comment:43 by malenki, 10 years ago

@ Klubumbus: for big projects you should download the translation file, enhance it locally and upload it again. Problem: from clicking "download" until the download is ready launchpad needed 30 min at my last (and only) try (which Don-vip refers to).
I'd welcome the move from to transifex very much.

in reply to:  43 ; comment:44 by Klumbumbus, 10 years ago

Replying to malenki:

@ Klubumbus: for big projects you should download the translation file, enhance it locally and upload it again. Problem: from clicking "download" until the download is ready launchpad needed 30 min at my last (and only) try (which Don-vip refers to).

I know. I also fixed the "Überführungsbauwerke" together with another one. I got the link to the .po download file within a few minutes, not 30. So I was a bit faster then you, and my translation got merged before yours. Therefore you got the conflict error :P

"Überführungsbauwerke" is fixed now in launchpad, though not in josm latest, because there hasn't been a manually pulling from launchpad since fixing it.

I'd welcome the move from to transifex very much.

+1

in reply to:  44 ; comment:45 by Mkyral, 10 years ago

Replying to Klumbumbus:

Replying to malenki:

I'd welcome the move from to transifex very much.

+1

Me too. There are some strings that I want to correct. But I don't want to waste my time with launchpad. I see, the files on Transifex are updated, but I don't know if I can start with translation (means, that the translated files from Transifex are merged back to Launchpad and josm). I still waiting for the green light ;-)

BTW: how can I add myself to this issue to monitor it? I can't see any option there.

in reply to:  45 comment:46 by stoecker, 10 years ago

Status: See comment 38 what needs to be done.

BTW: how can I add myself to this issue to monitor it? I can't see any option there.

There is a CC field. But if I remember configuration right, then the fact that you added a text should be enough already as well.

comment:47 by Don-vip, 10 years ago

Ticket #10352 has been marked as a duplicate of this ticket.

comment:48 by Don-vip, 10 years ago

Cc: jezevec added

Simon, Dirk, any chance to see some progress to finish the transition? I tried to look at it the other time but I wasn't sure to understand everything in the i18n scripts...

in reply to:  48 ; comment:49 by stoecker, 10 years ago

Replying to Don-vip:

Simon, Dirk, any chance to see some progress to finish the transition? I tried to look at it the other time but I wasn't sure to understand everything in the i18n scripts...

I have holidays ATM and I planned to do it in next two weeks, but currently I'm pretty much abstinent from any computer and busy reading Piers Anthony's Xanth series. Still approx 20cm (i.e. 9 books) left :-) So no promises.

comment:50 by Don-vip, 10 years ago

Priority: minornormal

This is not a minor problem as I'm pretty sure we're loosing translators because of this. We still have complains, see #10442.

comment:51 by daganzdaanda, 10 years ago

Yep, it's not very smooth to work with...
What I did is to dl the PO file and find the ID of the string I was interested in there. Then I could set the &start parameter in the URL to point to that ID. Then I changed it and hit save, and it seems to be OK.

in reply to:  49 comment:52 by Don-vip, 9 years ago

Replying to stoecker:

I have holidays ATM and I planned to do it in next two weeks, but currently I'm pretty much abstinent from any computer and busy reading Piers Anthony's Xanth series. Still approx 20cm (i.e. 9 books) left :-)

How's the reading going? ;)

comment:53 by stoecker, 9 years ago

Reading finished on time. And I also did do some stuff and then work catched me again. I hate this ticket...

comment:54 by Don-vip, 9 years ago

What I hate more is retrying thirteen times to reach an erroneous translated string in Launchpad :(

in reply to:  53 comment:55 by anonymous, 9 years ago

Replying to stoecker:

Reading finished on time. And I also did do some stuff and then work catched me again. I hate this ticket...

Could we help somehow?

in reply to:  54 comment:56 by stoecker, 9 years ago

Replying to Don-vip:

What I hate more is retrying thirteen times to reach an erroneous translated string in Launchpad :(

I agree. Still I always find more interesting stuff than to care for that ticket.

Could we help somehow?

Change the build.xml in a way, that it works smooth with old Launchpad and new Transifex, so that Transifex is merely a toogle switch. And proper testing upload of current translations from Launchpad to new Transifex and download of these Transifex translations to build language files. That's all.

comment:57 by Don-vip, 9 years ago

Milestone: 14.10

Assigning to milestone. It would be nice to move at the same time we change the logo :)

comment:58 by Don-vip, 9 years ago

Milestone: 14.1014.11

comment:59 by stoecker, 9 years ago

I used transifex for other tools in the last weeks and the more I use it, the more I dislike it. Major flaws I see after short time:

  • It will drop all related translations if only a single time the pot file upload is broken!
  • Administration is overly complicated

I thought about making my own service after beeing confronted with Launchpad for the first time, but found it too much work. Then I though about making a translation service based on Trac and still found it too complicated and too restricted. Now I did a lot more Trac development (spamfilter, translatedpages macro, ...) and I'm convinced that the best solution would be a Trac plugin for project translation. Advantages I see:

  • OpenSource
  • No translation service, but based on the local users we already have and also in the wiki we already have everything else
  • Easy to setup and manageable
  • Project specific

Disadvantages:

  • No project overlapping (but that didn't bring much for launchpad in the past anyway)
  • Will take some time until stable.

Interface would be with permission handling based on Trac and Translation handling similar to Transifex and/or Launchpad with Focus on the stability against errors (i.e. language uploads to wrong language, broken pot uploads, ...) - all these issues which caused a lot trouble in the past for Launchpad and transifex. Also proper fuzzy handling to ease retranslations (which Laucnhpad is missing compared to transifex).

With SpamFilter I reached version 1.0 now (i.e. feature complete for now), so I'm anyway searching for a new target and this would involve JavaScripting a bit more :-) Could be my winter holidays task.

What do you think? In case I fail it will delay Transifex migration (which I anyway delayed because I'm missing motivation). In case it works we have an easy installable OpenSource translation service for many projects (which use Trac).

The current Trac SpamFilter is a sole result of JOSM Trac (as I fixed and reworked the original one to keep this Trac instance working) and now I18NForTrac could be a similar success :-)

comment:60 by Don-vip, 9 years ago

Whatever working solution instead of this shitty, unmaintained Launchpad! I didn't tried Transifex a lot yet so I cannot judge.
A solution based on Trac looks nice as it would be simpler both for us and our translators :) If you're motivated to do this, go ahead!

comment:61 by Don-vip, 9 years ago

Milestone: 14.1114.12

comment:62 by Mkyral, 9 years ago

What about simple downloading of .po file and let user to choose a favorite po editor? And back upload via Trac ticket, later could be automatized.

comment:63 by Don-vip, 9 years ago

I think that would be a burden for the majority of translators who would prefer a web interface.

comment:65 by Mkyral, 9 years ago

Very nice.

in reply to:  59 ; comment:66 by bastiK, 9 years ago

Replying to stoecker:

I used transifex for other tools in the last weeks and the more I use it, the more I dislike it. Major flaws I see after short time:

  • It will drop all related translations if only a single time the pot file upload is broken!

Is this fixed by simply reuploading the file?

  • Administration is overly complicated

Very general statement, is this a real a problem?

I thought about making my own service after beeing confronted with Launchpad for the first time, but found it too much work. Then I though about making a translation service based on Trac and still found it too complicated and too restricted. Now I did a lot more Trac development (spamfilter, translatedpages macro, ...) and I'm convinced that the best solution would be a Trac plugin for project translation.

There are thousands of open source projects and probably hundreds with a similar or larger number of strings to translate. This problem is not unique to JOSM. I think having to create an extra account is acceptable for translators, as they will spend a lot of time with this anyway.

From the perspective of someone who is not very interested in the i18n stuff, this sounds like an huge investment of time and energy to get something perfect that we could have 98 % perfect almost for free.

On the other hand, if you think it's fun, then sure, why not...

in reply to:  66 comment:67 by stoecker, 9 years ago

Replying to bastiK:

Replying to stoecker:

I used transifex for other tools in the last weeks and the more I use it, the more I dislike it. Major flaws I see after short time:

  • It will drop all related translations if only a single time the pot file upload is broken!

Is this fixed by simply reuploading the file?

It seems not.

  • Administration is overly complicated

Very general statement, is this a real a problem?

Yes.

From the perspective of someone who is not very interested in the i18n stuff, this sounds like an huge investment of time and energy to get something perfect that we could have 98 % perfect almost for free.

On the other hand, if you think it's fun, then sure, why not...

I'm searching for interesting tasks. This seems to be one.

comment:68 by Don-vip, 9 years ago

Milestone: 14.1215.01

comment:69 by Don-vip, 9 years ago

Concerning i18n, I gave a look to our translations status on Launchpad:

Our lowest translation rates concern Estonian and Bulgarian, with 27% and 28% ot the total number of strings (core+plugins).

Three languages have better scores, but are not yet included in the software:

  • 30%: Khmer (km)
  • 83%: Catalan (Valencia) (ca@valencia)
  • 97%: Asturian (ast)

Asturian has one of the very best translation scores so I think we must include it right now.

Catalan de Valencia has a very good score, I'm just not sure about its code: is really ca@valencia the code we must use? It looks strange.

Khmer has a lower score but it's better than 2 other languages we offer, and it has been updated only three hours ago, so including it into JOSM core seems reasonable to me.

What do you think?

comment:70 by stoecker, 9 years ago

Rule is stated here:

Translations: New languages will be added to JOSM when there are at least 2000 translated strings.

Catalan de Valencia has a very good score, I'm just not sure about its code: is really ca@valencia the code we must use? It looks strange.

Hmm, does Java know about it? Any Java code assigned to it?

The language "ast" (and maybe "km") sounds like "must be included".

To do:

  • fix I18n.java to support it (proper multi-string encoding setting).
  • fix launchpad.pl to support it (add to supported hash)
  • create StartupPage and VersionHistory pages (and links in the history pages).
  • Update TracLanguages when not done (Ast is included)
  • build files.
  • Anything I forgot. It's a long time since last change.

Should I or do you want to do it?

Last edited 9 years ago by stoecker (previous) (diff)

comment:71 by Don-vip, 9 years ago

I'll give it a try, to learn :) Thanks!

in reply to:  70 comment:72 by Don-vip, 9 years ago

Replying to stoecker:

Hmm, does Java know about it? Any Java code assigned to it?

Valencian: Not yet. There are some big i18n enhancements in Java 8, including the adoption of CLDR. But Valencian has only been added in CLDR v25 while Java 8 only includes CLDR v21. So it seems we can't support this locale until Java 9 is released.

Asturian: No JRE support but it has been added to CLDR v22, so same as Valencian :(

Khmer: No JRE support but it is included in CLDR for a long time, so I should be able to make it work somewhat for those who run Java 8

comment:73 by stoecker, 9 years ago

I think we can add languages also without full Java support. Only thing not working should be the auto-language detection I think. Selecting language by hand (in prefs or commandline) should be possible. But we never tested it.

comment:74 by Don-vip, 9 years ago

I have added experimental support of Khmer with Java 8 in r7894 r7895 but it does not seem to work well:


I'm not sure if the .lang file is corrupted, or if it's a Java/Windows problem? I do not have any Linux machine right now to test, can you please give it a look?

Last edited 9 years ago by Don-vip (previous) (diff)

by Don-vip, 9 years ago

Attachment: khmer.png added

comment:76 by stoecker, 9 years ago

In 7897/josm:

see #8645 - also enable new language on older systems

comment:77 by stoecker, 9 years ago

With my java 1.7.0_55 choosing Khmer works by hand, but not automatic, so I removed the Java 8 check. I have the khmeros fonts installed and texts look fine.

I think you can add "ast" as well without waiting for newer Java.

comment:78 by Don-vip, 9 years ago

Ok thanks :)

comment:79 by jbcorbeille-josm@…, 9 years ago

Hello,
Any news here ?
I just cannot modify or add any translation.
For the fun, « note » in the "new" notes fonctionnality is translated in French to « paiement », meaning « payment », probably like in banknote. But for now, it will remain that way, I could get nowhere with launchpad, getting oopsies after oopsies.
JB.

in reply to:  79 comment:80 by stoecker, 9 years ago

Replying to jbcorbeille-josm@…:

Any news here ?

Not much, but I'm working on it.

Sorry for the long time, but the more I have to do at work the lower is the assigned amount of computer-time in my freetime :-)

I just cannot modify or add any translation.
For the fun, « note » in the "new" notes fonctionnality is translated in French to « paiement », meaning « payment », probably like in banknote. But for now, it will remain that way, I could get nowhere with launchpad, getting oopsies after oopsies.

Suggested workaround is to download the .po file, use a proper editor (e.g. kbabel, poedit, gted or a simple text editor) and reupload the changed file when done. That's BTW also the way I already worked in the past before these Launchpad delays came up. A proper editor is much better in any case.

comment:81 by Don-vip, 9 years ago

In 7907/josm:

see #8645 - i18n - add payment text context to "Note" in default preset to avoid translation conflict with OSM Notes

comment:82 by anonymous, 9 years ago

Wasn't the "Note" string already fixed elsewhere?

in reply to:  82 ; comment:83 by Klumbumbus, 9 years ago

Replying to anonymous:

Wasn't the "Note" string already fixed elsewhere?

Yes, in r7785.

r7907 is wrong, because it is not payment, but the tag note=.... I will remove this, as there is no reason for note=* in the vending machine preset (it is in no other preset).

comment:84 by Klumbumbus, 9 years ago

see r7908

in reply to:  83 comment:85 by skyper, 9 years ago

Replying to Klumbumbus:

....
I will remove this, as there is no reason for note=* in the vending machine preset (it is in no other preset).

This preset is really outdated, see #6268.

in reply to:  79 comment:86 by Klumbumbus, 9 years ago

Replying to jbcorbeille-josm@…:

getting oopsies after oopsies.

Sometimes(!) it helps to reduce the number of displayed untranslated strings. For this change the URL from ...batch=10... to e.g. ...batch=2...
If this does not help, then only the download and reupolad of the po file and the hope that nobody else edits at the same time is the way to go.

in reply to:  83 comment:87 by Don-vip, 9 years ago

Replying to Klumbumbus:

r7907 is wrong, because it is not payment, but the tag note=....

Indeed I was misleaded by the French translation which was horribly wrong in fact :D Thanks.

comment:88 by Don-vip, 9 years ago

Milestone: 15.0115.02

move tickets that have not been treated this month to next milestone

comment:89 by Don-vip, 9 years ago

Milestone: 15.02
Resolution: wontfix
Status: reopenedclosed

after 4 years (!), Launchpad guys have finally decided to fix this problem:
https://bugs.launchpad.net/launchpad/+bug/736005/comments/36

The search is finally working as expected, so I guess there's no need anymore to switch to a different platform.

comment:90 by bastiK, 9 years ago

Wow, didn't see that coming.

comment:91 by Don-vip, 9 years ago

We should spread the word that everyone can now use the web interface to translate JOSM, without becoming crazy :)

in reply to:  91 comment:92 by Klumbumbus, 9 years ago

Replying to Don-vip:

We should spread the word that everyone can now use the web interface to translate JOSM, without becoming crazy :)

Yes, should be on StartupPageSource :)

comment:93 by stoecker, 9 years ago

A note: I'll continue development of a Trac based alternative, which allows to directly integrates the JOSM workflow (input and output) as plugin, but gladly I have more time now to do so and less pressure. Maybe when I have holidays in summer. I wanted to do this for years and now I have a concept ready which only needs to be programmed.

Launchpad is really fast again, only the translation download mail is slow as always :-)

comment:94 by simon04, 9 years ago

Resolution: wontfix
Status: closedreopened

Should we:

comment:95 by Klumbumbus, 9 years ago

Atleast we should find a way that people do not longer translate strings at transifex since this is wased time/effort.

comment:96 by stoecker, 9 years ago

No revert of the infrastructure parts I think. I'd simply clear the project at transifex and leave it empty.

comment:97 by stoecker, 9 years ago

Resolution: fixed
Status: reopenedclosed

Cleared the project.

comment:98 by anonymous, 9 years ago

A great shame, transifix makes it really easy to do translations with. The issue was never raised because of bugs with Launchpad but because of it being cumbersome to use.

comment:99 by Don-vip, 9 years ago

Resolution: fixed
Status: closedreopened

It's happening again! Looks like we only had a few months of peace and we're going to face again this real pain:

(Error ID: OOPS-8290302df82f2cb4a8e0ec50e0781f86)
(Error ID: OOPS-9d95aea497076c6419a8a80e7dd75f0b)

Time to move on for good?

comment:100 by Don-vip, 8 years ago

As Dirk points out, the situation has not improved. A volunteer to help to resume the transition to Transifex?

comment:101 by openstreetmap.org-user-d1g, 8 years ago

Cc: openstreetmap.org-user-d1g added

comment:102 by openstreetmap.org-user-d1g, 8 years ago

As translator, I find transifex.com is more reliable than Launchpad. It is faster to translate at transifex because of UI, short-cuts and powerful filters.

Automatic diff/history of translations makes translation unbelievably easy compared to other tools.

It is impossible to discuss/leave remarks at Launchpad AFAIK.

Unlike Launchpad, transifex is ready for serious translation efforts (translation memory, semi-automatic translation assistance), glossaries.

Many OSM projects are hosted at transifex.com. It is very easy for translators to switch between QGIS/OSM/JOSM/<s>OsmAnd</s> translations if they all hosted at one platform (currently this platform is transifex.com IMO).

Also, I don't like to use tools that doesn't work :)

Last edited 8 years ago by openstreetmap.org-user-d1g (previous) (diff)

comment:103 by openstreetmap.org-user-d1g, 8 years ago

If we switch, we should ask for “Open Source” program later on, as JOSM may qualify as "What is considered an Open Source project?" http://docs.transifex.com/faq/

in reply to:  102 comment:104 by aceman, 8 years ago

Replying to openstreetmap.org-user-d1g:

Many OSM projects are hosted at transifex.com. It is very easy for translators to switch between QGIS/OSM/JOSM/OsmAnd translations if they all hosted at one platform (currently this platform is transifex.com).

Not sure what you mean by OSM, but OsmAnd is on Weblate.org.

comment:105 by naoliv, 7 years ago

Cc: naoliv added

comment:106 by Don-vip, 7 years ago

I just can't fix translations errors right now. Tried about 20 times to bypass OOPS errors to search a string in Italian language and gave up. Is there someone willing to step up to finish the migration please?

Last edited 7 years ago by Don-vip (previous) (diff)

comment:107 by Don-vip, 7 years ago

Another problem with Launchpad, the translations are not synchronized for three days now: https://bugs.launchpad.net/launchpad/+bug/1676975/comments/2

comment:108 by floscher, 6 years ago

Hello,
let's continue the discussion from the mailing list https://lists.openstreetmap.org/pipermail/josm-dev/2018-January/007954.html.

I prepared a patch for the i18n/build.xml, which would exclude JOSM plugins with a file .tx/config from pushing translations to Launchpad. It would also exclude plugins without a data/ directory from importing translations from Launchpad.
This would affect the plugins Mapillary and geojson.

The other option would be to remove the svn:external for both plugins and exclude them from the Ant build altogether.

One question that came to my mind in this context: What's determining if a JOSM plugin that's hosted on GitHub is shown as "from an external source"? Is it that the project is under https://github.com/JOSM/ or is it that it's part of the Ant build?

---

Another topic I thought about was in which organisation on Transifex the translations should be maintained in. Two options: JOSM and OpenStreetMap.
I'm currently leaning towards the JOSM organisation, mainly because it could violate the GPL license of the translations to be reused in other projects in the OpenStreetMap organisation on Transifex. And I don't want to create even more places where strings for JOSM can be translated. One location on Launchpad and one location on Transifex should be enough, and the JOSM organisation is probably a location more developers could agree on.

In the case that we'd conclude in the future to join translation forces with https://www.transifex.com/openstreetmap/, we could still migrate the translations over.

Would you find it sensible to put all new translations made on Transifex under a more liberal license that would allow easier mixing and matching with other projects?

by floscher, 6 years ago

Patch to exclude plugins with a .tx/config file from being translated on Launchpad and only write the translations from Launchpad to plugins containing a data/ directory

by floscher, 6 years ago

Improved version of the previous patch, the <if></if> was surrounding too much of the plugintrans target.

in reply to:  108 ; comment:109 by stoecker, 6 years ago

Replying to floscher:

The other option would be to remove the svn:external for both plugins and exclude them from the Ant build altogether.

I'd prefer this. The external is only there for translations, so removing it is the right thing. The patch would cause useless resources, as svn checkout fetches ressources which aren't needed at all.

One question that came to my mind in this context: What's determining if a JOSM plugin that's hosted on GitHub is shown as "from an external source"? Is it that the project is under https://github.com/JOSM/ or is it that it's part of the Ant build?

A project where JOSM admins have access and can do fixes and i18n updates (i.e. part of JOSM on GitHub).

Would you find it sensible to put all new translations made on Transifex under a more liberal license that would allow easier mixing and matching with other projects?

Individual short translations can't have a license. Only a set of translations gets licensable. Same situation as for the OSM database. So using individual strings is not a problem. Only copying bigger parts (or longer individual texts).

in reply to:  109 ; comment:110 by holgermappt, 6 years ago

Replying to stoecker:

Replying to floscher:

Would you find it sensible to put all new translations made on Transifex under a more liberal license that would allow easier mixing and matching with other projects?

Individual short translations can't have a license. Only a set of translations gets licensable. Same situation as for the OSM database. So using individual strings is not a problem. Only copying bigger parts (or longer individual texts).

What is the license of the translations made on Transifex? I don't see anything regarding a license. That would mean no license, right? And no license means no right for anyone, right? Isn't that a bad idea? So yes, any license more liberal than "no right for anyone" would be highly appreciated. And no, copying strings or words one-by-one because "individual short translations can't have a license" is not an option.

in reply to:  110 comment:111 by floscher, 6 years ago

Replying to holgermappt:

Replying to stoecker:

Replying to floscher:

Would you find it sensible to put all new translations made on Transifex under a more liberal license that would allow easier mixing and matching with other projects?

Individual short translations can't have a license. Only a set of translations gets licensable. Same situation as for the OSM database. So using individual strings is not a problem. Only copying bigger parts (or longer individual texts).

What is the license of the translations made on Transifex? I don't see anything regarding a license. That would mean no license, right? And no license means no right for anyone, right? Isn't that a bad idea? So yes, any license more liberal than "no right for anyone" would be highly appreciated. And no, copying strings or words one-by-one because "individual short translations can't have a license" is not an option.

Nobody suggested to not put them under a license. Until decided otherwise I think the translations (should) have the GPLv2 or later license like the JOSM source code. Most of the translations for the Mapillary and geojson plugins are imported from Launchpad, so these have to be under GPL. My question was, if the translations should be put under a license that's less strict than GPL about where the strings can be reused (e.g. via Transifex's translation memory). But if the translation license really only matters when someone wants to copy the whole set of translations, reusing single strings in projects with other licenses shouldn't be a problem.

Where would I add the license that the translations are under? I couldn't find an option in the web interface and if I add it to the source files, the translators probably wouldn't see it.

comment:112 by Don-vip, 6 years ago

Patch applied in [o34023].

comment:113 by stoecker, 6 years ago

Related to the gradle builds? See #15809.

in reply to:  112 ; comment:114 by stoecker, 6 years ago

Replying to Don-vip:

Patch applied in [o34023].

I18n build is broken.

comment:115 by Don-vip, 6 years ago

@floscher: can you please take a look?

in reply to:  114 comment:116 by floscher, 6 years ago

Replying to stoecker:

Replying to Don-vip:

Patch applied in [o34023].

I18n build is broken.

Is it directly related to the patch, though? The i18n build completed successfully two times since the patch was applied.
I will investigate further what caused the build to break. The build log says somehow the `msgcomm` command to create the `plugins.pot` failed.

comment:117 by stoecker, 6 years ago

Didn't see that error message. Probably got confused by all these "filterpluginsource:" (can we get rid of these?).

Now it works. Does somebody know what changed?

in reply to:  117 comment:118 by floscher, 6 years ago

Replying to stoecker:

Didn't see that error message. Probably got confused by all these "filterpluginsource:" (can we get rid of these?).

Probably the NoBannerLogger would get rid of these messages. If I understand it correctly, it would suppress logging of target names for targets without logging output.

Now it works. Does somebody know what changed?

I have no clue. On my machine I couldn't reproduce the issue and I couldn't find a possible cause in the build log.

comment:119 by simon04, 5 years ago

Keywords: hack-weekend-2018-10 added

by simon04, 5 years ago

comment:120 by simon04, 5 years ago

Transifex expects different plural forms for some languages, for instance cs:

# Launchpad
"Plural-Forms: nplurals=3; plural=(n==1) ? 0 : (n>=2 && n<=4) ? 1 : 2;\n"
# Transifex
"Plural-Forms: nplurals=4; plural=(n == 1 && n % 1 == 0) ? 0 : (n >= 2 && n <= 4 && n % 1 == 0) ? 1: (n % 1 != 0 ) ? 2 : 3;\n"

The plural form on Transifex seems to match http://www.unicode.org/cldr/charts/latest/supplemental/language_plural_rules.html Just fixing the Plural-Forms and uploading the translations did not work, see https://www.transifex.com/josm/josm/translate/#cs/core/156001581 for an example which was translated on Launchpad, but is now missing. Does anyone know a solution for plural message migration? I will contact Transifex and ask them as well. Some tickets in other FOSS projects I found are https://github.com/mate-desktop/mate-desktop/issues/257, https://github.com/mumble-voip/mumble/issues/2854, https://code.djangoproject.com/ticket/26123.

Last edited 5 years ago by simon04 (previous) (diff)

comment:121 by floscher, 5 years ago

For the plugins that I migrated to Transifex, I simply deleted pluralized strings in the Launchpad export for these languages (it weren't many). The only solution I can think of that would keep existing translations, would be to add an additional plural to the *.po files with a placeholder string (e.g. english plural (translate this on transifex)).

Should we make the core resources on transifex read-only, so nobody translates strings now, which would get overriden by the dedicated user importing from Launchpad again?

Is the current plan to leave the plugins in the same project together with core? Or should they move to the new project 'JOSM-plugins'? I categorized the resources so you can filter the list of resources by core, plugin and website.
It seems moving a resource from one project to another inside transifex is not trivial and would throw away translation history and translation dates: https://docs.transifex.com/projects/moving-content
If the plugins should be a separate project, it would be better to rename the current josm project to josm-plugins and put the core files in a new project.

comment:122 by holgermappt, 5 years ago

There are some translations that are in Launchpad for some time but not in the Transifex import. Was there some filter applied? It would be some extra work to copy the missing string manually. E.g. the untranslated German strings from the pluginlist are in Launchpad (at least the strings I looked at).

I also think it would be a good idea to set one of the instances to read only to avoid work that will be discarded later.

Last edited 5 years ago by holgermappt (previous) (diff)

in reply to:  121 ; comment:123 by Don-vip, 5 years ago

Replying to floscher:

Should we make the core resources on transifex read-only, so nobody translates strings now, which would get overriden by the dedicated user importing from Launchpad again?

I guess so. Now that we have some strings we should try to build a local JOSM using transifex data, and check the about dialog (translator credits).

Is the current plan to leave the plugins in the same project together with core? Or should they move to the new project 'JOSM-plugins'? I categorized the resources so you can filter the list of resources by core, plugin and website.

I think having two projects is the best approach.

It seems moving a resource from one project to another inside transifex is not trivial and would throw away translation history and translation dates: https://docs.transifex.com/projects/moving-content

Good to know. What happens if we merge different resources in the same project? Is it easy? Do we loose something?

If the plugins should be a separate project, it would be better to rename the current josm project to josm-plugins and put the core files in a new project.

Good idea!

in reply to:  121 comment:124 by simon04, 5 years ago

Replying to floscher:

For the plugins that I migrated to Transifex, I simply deleted pluralized strings in the Launchpad export for these languages (it weren't many). The only solution I can think of that would keep existing translations, would be to add an additional plural to the *.po files with a placeholder string (e.g. english plural (translate this on transifex)).

Is is probably not an option since there are 178 trn strings.

Should we make the core resources on transifex read-only, so nobody translates strings now, which would get overriden by the dedicated user importing from Launchpad again?

Done by disabling "Your translators can translate resource strings".

Is the current plan to leave the plugins in the same project together with core? Or should they move to the new project 'JOSM-plugins'? I categorized the resources so you can filter the list of resources by core, plugin and website.

Ah, good to know that. Then we probably do not have to split.

After importing also the non-shipped languages to Transifex, here's the result:

  • plural problems in ar, be, cs, fa, he, lt, pl, ro, ru, sk, uk
  • missing language code on Transifex: awa, hil, skr, wae (all <1% done)

The changed plural forms are tricky; we need to:

  • somehow bring the plural strings to Transifex
  • update the I18n.PluralMode enums accordingly, update plugin main version
  • perform I18n update for JOSM and all plugins

Damn. As if the migration would not have been complicated enough. What a pity that Transifex does not permit custom/non-default plural rules.

comment:125 by floscher, 5 years ago

What happens if we merge different resources in the same project? Is it easy? Do we loose something?

I'll test that. My guess is that inside one project the strings can be moved around between resources without losing anything (same with merging).
Maybe it would be best to split not between core and plugins, but between core with officially supported plugins and not officially supported plugins, so translations can be shared easier at least with the official plugins.

Is is probably not an option since there are 178 trn strings.

For some of these languages, actually not all of these strings are translated (e.g. ar 18, fa 15).
My current understanding is that Transifex doesn't accept translations that only translate a subset of all plural forms. So if we want to import to transifex, we need to provide something as fourth plural form (e.g. english plural, one of the other plurals of that language, generic placeholder.

comment:126 by anonymous, 5 years ago

What is the plural forms problem for 'sk' ? What rule does Transifex have?

Also the rule for 'cs' seems kinda bogus. Most other projects use 3 forms for cs and sk.

Transifex claims to go by http://www.unicode.org/cldr/charts/latest/supplemental/language_plural_rules.html . If the 4th form it uses is the fractional form ("many") then that form is not used in JOSM. And then there should be easy mapping from JOSM forms to the Transifex form.

Please do not delete any forms from JOSM sources until this is resolved and the mapping is done (i.e for cs you map JOSM=>Transifex: 0 => 0; 1 => 1 and 2 => 3).

comment:127 by Klumbumbus, 5 years ago

There are about 1850 untranslated strings in the german transifex. In Launchpad only 36. Anything went wrong there? attachment:josmtransifex.PNG

Last edited 5 years ago by Klumbumbus (previous) (diff)

by Klumbumbus, 5 years ago

Attachment: josmtransifex.PNG added

in reply to:  127 comment:128 by anonymous, 5 years ago

Replying to anonymous:

What is the plural forms problem for 'sk' ? What rule does Transifex have?

# Launchpad
"Plural-Forms: nplurals=3; plural=(n==1) ? 1 : (n>=2 && n<=4) ? 2 : 0;\n"
# Transifex
"Plural-Forms: nplurals=4; plural=(n % 1 == 0 && n == 1 ? 0 : n % 1 == 0 && n >= 2 && n <= 4 ? 1 : n % 1 != 0 ? 2: 3);\n"

Transifex claims to go by http://www.unicode.org/cldr/charts/latest/supplemental/language_plural_rules.html . If the 4th form it uses is the fractional form ("many") then that form is not used in JOSM. And then there should be easy mapping from JOSM forms to the Transifex form.

Please do not delete any forms from JOSM sources until this is resolved and the mapping is done (i.e for cs you map JOSM=>Transifex: 0 => 0; 1 => 1 and 2 => 3).

According to floscher, we either provide an initial value for the added plural form (e.g. 3 → 3+4) or we cannot import the trn strings. Is this sensible for cs, for sk?

Replying to Klumbumbus:

There are about 1850 untranslated strings in the german transifex. In Launchpad only 36. Anything went wrong there? attachment:josmtransifex.PNG

For now, I only imported core, but not yet maplist/pluginlist/presets. The process takes some time and first I'd like to have the plural issue clarified.

PS: This was me, simon04.

Last edited 5 years ago by simon04 (previous) (diff)

comment:129 by aceman, 5 years ago

It seems 'n % 1 != 0' means fractional (non-cardinal) number.
Yes, if you copy form "other" of JOSM into form "many" and "other" of Transifex that is fine for cs and sk. I think JOSM never displays fractional numbers in the strings so it is not very important what form "many" of Transifex will contain. But if ever the form "many" displays, having the value of JOSM form "other" is bearable for the language reader.

But I messed the numbering up. The mapping is like this:
JOSM form 0 => Transifex form 3 ("other")
JOSM form 1 => Transifex form 0 ("one")
JOSM form 2 => Transifex form 1 ("few")
JOSM form 0 => Transifex form 2 ("many")

comment:130 by floscher, 5 years ago

When looking into ar language, it seems Launchpad and Transifex support the same six plural forms:

  • 0
  • 1
  • 2
  • n % 100 >= 3 && n % 100 <= 10
  • n % 100 >= 11 && n % 100 <= 99
  • everything else

The problem seems to be, that some strings are translated on Launchpad to only some of these six forms (18 strings are translated, 8 of those have all six forms).

Excerpts from josm-ar.po:

msgid "Combine {0} way"
msgid_plural "Combine {0} ways"
msgstr[0] "ضم {0} خط"

msgid "object"
msgid_plural "objects"
msgstr[0] "غرض"
msgstr[1] "غرضين"
msgstr[2] "أغراض"

msgid "node"
msgid_plural "nodes"
msgstr[0] "عقدة"
msgstr[1] "عقدتين"
msgstr[2] "عقدات"
msgstr[3] "عقد"
msgstr[4] "عقد"
msgstr[5] "عقد"

in reply to:  123 comment:131 by floscher, 5 years ago

Replying to Don-vip:

It seems moving a resource from one project to another inside transifex is not trivial and would throw away translation history and translation dates: https://docs.transifex.com/projects/moving-content

Good to know. What happens if we merge different resources in the same project? Is it easy? Do we loose something?

I played around a bit with transifex, this is how it seems to work:

  • if you upload new strings to a resource, transifex searches all resources from that project and uses exactly matching translations for the new strings
  • if two resources A and B have the same string and you change the translation in resource A, the translation of the same string in resource B only changes if it was previously untranslated in resource B

So merging in the same project shouldn't be a problem. Once a string is translated inside one project, it can be moved around to other resources and will stay translated.

comment:132 by Don-vip, 5 years ago

OK so maybe a single project for both core resources and plugins is a better option. We want to be able to merge plugins in core easily and without loosing anything.

comment:133 by Don-vip, 5 years ago

@simon04 it seems there's a problem with the latest i18n changes, see jenkins/job/JOSM-i18n/1182/parsed_console/ we have 8117 warnings now.

Links for reference: [o34686:34689], [o34692:34696]

Last edited 5 years ago by Don-vip (previous) (diff)

comment:134 by simon04, 5 years ago

The culprit was [o34694]. I reverted this one in [o34697].

comment:135 by simon04, 5 years ago

I implemented a small too for the plural message translation, https://github.com/simon04/plural-message-migration. For the moment, it supports the currently shipped languages with changed plural forms, namely be, cs, lt, pl, ru, sk, uk. I applied the migration to tonight's Launchpad export and updated all shipped Languages on Transifex. I implemented the following plural message migrations (Launchpad index → Transifex index):

  • be: 0→0, 1→1, 2→2, 2→3
  • cs: 0→0, 1→1, 2→2, 2→3
  • lt: 0→0, 1→1, 2→2, 2→3
  • sk: 1→0, 2→1, 0→2, 0→3 (see comment:129)
  • pl: 0→0, 1→1, 2→2, 2→3
  • ru: 0→0, 1→1, 2→2, 2→3
  • uk: 0→0, 1→1, 2→2, 2→3

Please review the translations on Transifex. In a nutshell and excluding sk, the new plural forms include a newly "frational"/"other" form at index 3 (or 2 for lt), and the "many" strings have been copied to the "fractional" strings.

comment:136 by aceman, 5 years ago

Should we already see the migrated strings on Transifex? The 'core' resource only contains imagery names, which don't have any plurals so we can't check the migration.

comment:137 by simon04, 5 years ago

The core resource is this https://www.transifex.com/josm/josm/core/. This one already been updated and should contain all strings of JOSM core except imagery, plugin descriptions, presets.

in reply to:  134 comment:138 by Don-vip, 5 years ago

Replying to simon04:

The culprit was [o34694]. I reverted this one in [o34697].

Thanks. I confirm this is fixed. Jenkins is happy again :)

comment:139 by aceman, 5 years ago

OK, I can sometimes see all the core strings, somehow the "sub-resources" of core (presets, maplist) are shown instead of the pure 'core' one. It is quite a mess in the Transifex Web UI. Best bet seems to be to display all at once, but that is what I do not want to do (but only translate some resources). I also only see about 5 plugin resources and can't access any others.

So, the 'core' plurals seem OK, but e.g. "Plugin > Mapillary" resource has plurals wrongly ordered. You probably didn't apply the transformation from comment 135 to that one yet (it is dated 21 Oct).

in reply to:  139 comment:140 by floscher, 5 years ago

Replying to aceman:

OK, I can sometimes see all the core strings, somehow the "sub-resources" of core (presets, maplist) are shown instead of the pure 'core' one. It is quite a mess in the Transifex Web UI. Best bet seems to be to display all at once, but that is what I do not want to do (but only translate some resources). I also only see about 5 plugin resources and can't access any others.

So, the 'core' plurals seem OK, but e.g. "Plugin > Mapillary" resource has plurals wrongly ordered. You probably didn't apply the transformation from comment 135 to that one yet (it is dated 21 Oct).

The resources starting with plugin › were imported to Transifex separately before the current import, so that's unrelated to the migrations simon04 is doing.

Which languages have the plurals in the wrong order? What would be the correct one?

comment:141 by aceman, 5 years ago

I have only checked 'sk' (but if that comes out correct, then I think you need the same order also for 'cs').
The 'core' is in correct order, but plugins are not. So those will need re-importing according to the rules in comment 135.

comment:142 by Don-vip, 5 years ago

Keywords: transifex added

IIRC someone told me about Weblate: https://weblate.org/en/hosting/free/

It's free software and used by OpenSUSE & OsmAnd. Maybe we could setup a test instance and compare with Transifex to see which is the better choice for us?

comment:143 by simon04, 5 years ago

This document lists even more tools to evaluate: https://docs.google.com/spreadsheets/d/1t6Zo9OmlRCH3N9dOumnWn0w8QrgI9Is2zQXSJB7K-QI/edit

From a translator's perspective, I like the frontend of Transifex. It is responsive and allows searching for already translated similar phrases. With respect to uploading templates and downloading translations, I think we resolved most issues/problems. I'm not so keen to evaluate an alternative solution …

comment:144 by Don-vip, 5 years ago

OK it was mainly a suggestion in case we face blocking problems with Transifex :)

in reply to:  142 ; comment:145 by stoecker, 5 years ago

Replying to Don-vip:

IIRC someone told me about Weblate: https://weblate.org/en/hosting/free/

That one looks good. We could setup our own instance and it is actively developed. It misses a display of similar strings as far as I see, but has a Glossary function OTOH (after a first glance).

This document lists even more tools to evaluate

Nothing better. Either already evaluated or proprietary and remotely hosted. From these services transifex was the best, but I always searched for a software we can setup ourselves. A result could be, that e.g. StartupPage translation could be integrated instead of using its own system. Also status display to users would be more immediate.

in reply to:  145 comment:146 by Don-vip, 5 years ago

Replying to stoecker:

IIRC someone told me about Weblate: https://weblate.org/en/hosting/free/

That one looks good. We could setup our own instance and it is actively developed. It misses a display of similar strings as far as I see, but has a Glossary function OTOH (after a first glance).

We could, but Simon has spent a lot of time to resolve transifex issues, so I'm also in favor to finish the work he started, unless we find an absolutely blocking issue. But this is good to know we have an hosted alternative in case we get big problems with Transifex in the future.

in reply to:  141 comment:147 by floscher, 5 years ago

Replying to aceman:

I have only checked 'sk' (but if that comes out correct, then I think you need the same order also for 'cs').
The 'core' is in correct order, but plugins are not. So those will need re-importing according to the rules in comment 135.

The translations of plugin strings for language sk should now be in the correct order. There's one string from the wikipedia plugin, that had the same translation for all four plural forms, for which I created a Transifex issue.

The cs translations should already be in the correct order, because in Launchpad the sk order is: other, one, few (between 2 and 4). The cs order is: one, few (between 2 and 4), other .

# Launchpad cs
plural=(n==1) ? 0 : (n>=2 && n<=4) ? 1 : 2
# Launchpad sk
plural=(n==1) ? 1 : (n>=2 && n<=4) ? 2 : 0

comment:148 by floscher, 5 years ago

What is the status of the migration to transifex? What still needs to be done to complete it?

comment:149 by aceman, 5 years ago

Thanks, also the "plugins > Mapillary" is now correctly migrated for 'sk'.

The https://www.transifex.com/josm/josm/translate/#sk/$/142419252?q=issue%3Aopen is not a big issue, it is hard to create plural forms for the word 'ID' :) But yes, I can change it to 'identifiers' and then it works.

Where should we make translations currently? At launchpad or transifex? Which is the primary source now for JOSM releases?

in reply to:  149 comment:150 by floscher, 5 years ago

Replying to aceman:

Where should we make translations currently? At launchpad or transifex? Which is the primary source now for JOSM releases?

The resources that are on Transifex and that are editable (currently 7 plugins) are translated on transifex.com.

Everything else is currently translated on launchpad.net, that's why the resources for JOSM core are set to read-only in Transifex.

It seemed to me that the migration was almost done, but now there was no further activity for some weeks. So probably the strings from Launchpad would have to be exported again and imported to Transifex, so the translations made in the meantime don't get lost.

comment:151 by holgermappt, 4 years ago

Any news? Lately it takes 5 to 10 reloads at Launchpad to go to the untranslated strings due to the infamous Timeout error.

comment:152 by Klumbumbus, 4 years ago

Ticket #19085 has been marked as a duplicate of this ticket.

in reply to:  145 comment:153 by anonymous, 4 years ago

Replying to stoecker:

Replying to Don-vip:

IIRC someone told me about Weblate: https://weblate.org/en/hosting/free/

That one looks good. We could setup our own instance and it is actively developed. It misses a display of similar strings as far as I see, but has a Glossary function OTOH (after a first glance).

Weblate now has string grouping.

This document lists even more tools to evaluate

Nothing better. Either already evaluated or proprietary and remotely hosted. From these services transifex was the best, but I always searched for a software we can setup ourselves. A result could be, that e.g. StartupPage translation could be integrated instead of using its own system. Also status display to users would be more immediate.

Having used all options extensively, Weblate is the best option. I thought Hosted Weblate would be the best bet.
Transifex is a bucket of problematic eels.

comment:154 by simon04, 4 years ago

See also the same discussion for iD: Replace Transifex · Issue 7508 · openstreetmap/iD · https://github.com/openstreetmap/iD/issues/7508

comment:155 by stoecker, 4 years ago

Maybe we should reevaluate weblate?

A second option to relieve translators would be to split launchpad translation into the 3 sections we already have. That should reduce the timeouts a lot, as they seems to depend on the total translation size.

comment:156 by Don-vip, 4 years ago

It seems the original timeout issue of Launchpad is gone. I had no problem to perform searches this morning. First time in years. So maybe we can keep Launchpad after all, if the issue doesn't come back.

in reply to:  156 comment:157 by stoecker, 4 years ago

Replying to Don-vip:

It seems the original timeout issue of Launchpad is gone. I had no problem to perform searches this morning. First time in years. So maybe we can keep Launchpad after all, if the issue doesn't come back.

It always depended a bit on the time when you use it, but they did improvements recently. :-) Splitting it in 3 sections may nevertheless be a good idea even if not needed because of the timeout issue? Opinions?

comment:158 by Don-vip, 4 years ago

Sounds a good idea indeed. Does it have any downside?

in reply to:  158 comment:159 by stoecker, 4 years ago

Replying to Don-vip:

Sounds a good idea indeed. Does it have any downside?

My thoughts regarding that:

  • Pro: Makes it easier to know what's plugin, core, other, ...
  • Contra: Makes it easier to ignore plugins, other, ...
  • I'm not sure what happens when strings move between the groups. Are they kept or at least easy available?
  • We should be careful to not loose translations in the process even for the many many partial languages when we change that
  • Upload/Download scripts must be checked
  • I think lots/most of work on the scripting side was already done

comment:160 by stoecker, 4 years ago

Idea for Migration:

  • Add the 3 additional POT files (or first only 1 :-)
  • Check everything and re-import translations into the new files when necessary.
  • Last step drop the combined POT file

comment:161 by stoecker, 3 years ago

Resolution: wontfix
Status: reopenedclosed

Seems Transifex migration makes no more sense. Improvements to launchpad file handling should be handled in an own ticket when someone wants to do the work.

comment:162 by Don-vip, 3 years ago

Launchpad timeout errors are back... :'(

comment:163 by Klumbumbus, 3 years ago

Yes it's not possible to search a string on launchpad atm.

comment:164 by Hb---, 3 years ago

I'd like to present the offline translation solution in use here. The base is the OmegaT programm added by WinMerge as diff viewer and a crappy batch file for some automatics.

OmegaT provides a fast search with regex et al. This makes it easy to maintain and uphold the consistency of a translation.

@echo off
echo This batch provides an offline translation cycle for JOSM. Steps are:
echo 1. Download the compiled po-file
echo 2. Automatically normalize the po-file
echo 3. Check and Translate with a diff tool and OmegaT
echo 4. Upload to Launchpad

rem Ensure the working directory is the batch files directory
set wrkdir=%~p0
cd %wrkdir%

rem STEP 1
rem Without Bazaar downloads can be requested via email link or done manually:
echo Please download the file "de.po" from the appearing web page
echo to the target directory %USERPROFILE%\Downloads\
echo Continue with any key to open the web browser.
pause >nul
echo Opening the web browser ...
echo [InternetShortcut] >%TEMP%\start.url
echo url=https://bazaar.launchpad.net/~openstreetmap/josm/josm_trans/files/head:/josm >>%tmp%\start.url
%tmp%\start.url
echo .
echo Download finished? Continue with any key.
pause >nul
rem Move the downloaded file into the source directory of the
rem OmegaT normalisation project. It is important to rename the file
rem according to the email-based download. Otherwise OmegaT would see
rem two different source files and mismatch its translation memory.
if exist "%USERPROFILE%\Downloads\de.po" move %USERPROFILE%\Downloads\de.po .\omde2norma\source\josm_josm-de.po

rem STEP 2
rem Normalizing the incoming po file is needed to get acceptable diff
rem views. Launchpad delivers files with line breaks in msgstr strings.

rem Delete all translation memories to empty OmegaT project.
del .\omde2norma\omde2norma-*.tmx >nul
del .\omde2norma\omegat\project_save.* >nul

rem Run the project quietly
java -jar "C:\Program Files (x86)\OmegaT\OmegaT.jar" .\omde2norma\ --mode=console-translate --quiet
rem Delete memory files again
del .\omde2norma\omde2norma-*.tmx >nul
del .\omde2norma\omegat\project_save.* >nul
echo .

rem STEP 3
echo Now OmegaT and a file difference viewer are started in parallel.
echo The diff viewer compares the downloaded translation version with the
echo most recent local version.
echo .
echo Initially this is the version you uploaded the last time. So you see
echo which strings are new and what others have translated in the meantime.
echo .
echo After generating a translation in OmegaT (Ctrl+D) and rediffing you will
echo see your current translation work against the downloaded version.

start javaw -Duser.language=en -jar "C:\Program Files (x86)\OmegaT\OmegaT.jar" omde\

rem Wait soemseconds while OmegaT is starting up
ping localhost >nul
"C:\Program Files (x86)\WinMerge\WinMergeU.exe" omde\source\josm_josm-de.po omde\target\

rem STEP 4
echo Invoke the Launchpad Upload page ...
echo [InternetShortcut] >%TEMP%\start.url
echo url=https://translations.launchpad.net/josm/trunk/+pots/josm/de/+upload >>%TEMP%\start.url
%TEMP%\start.url

rem Housekeeping
del %TEMP%\start.url
exit

rem NEEDED SETTINGS FOR OMEGAT
rem ==========================
rem Project|Properties for the normalisation and the translation projects
rem + Enable Sentence-level Segmenting = OFF (Default=On)
rem + Auto-propagation of Translations = OFF (Default=On)
rem
rem Project|Properties only for the normalisation project
rem + Translated Files Folder = path/to/source/folder/of/translation/project
rem
rem
rem Resommendations for OmegaT
rem View|
rem X Display source segments
rem
rem Options|Preferences
rem + File Filters
rem + PO File Format Options:
rem + Allow blank translations in target files = OFF (default=off)
rem + Skip PO Header = ON (default=off) ??
rem + View
rem Display all source segments in bold = OFF (default=on)
rem + Display active source segment in bold = ON
rem + Include first non unique segment ??
rem + Updates
rem + Automatically check for updates = OFF (default=on)

comment:165 by stoecker, 3 years ago

There are pretty good offline tools for handling po-files:

I used KBabel in the past. It's now replaced by Lokalize. They are much better when doing larger works, but first small updates the webinterface is still the best solution.

Last edited 3 years ago by Hb--- (previous) (diff)

in reply to:  165 comment:166 by Hb---, 3 years ago

Replying to stoecker:
Lokalize is KDE based and offers binaries for Windows (Localize is non free). I'll try.

They are much better when doing larger works, but for small updates the webinterface is still the best solution.

Starting with a lame translation that has 20 % correct strings is a large work. Even maintaining a translation is an endless task. All the small bits done in web interfaces tend to disintegrate the consistency over time. Current example is #20353.

If the turn-around time for

  1. Loading
  2. Checking for new strings
  3. Translating while being able to compare with similar strings already translated and
  4. Uploading

is favorable, then I would always prefer an offline solution.

Last edited 3 years ago by Hb--- (previous) (diff)

comment:167 by Don-vip, 3 years ago

Search works again, Launchpad guy fixed the problem after Dirk complained :)

in reply to:  167 comment:168 by stoecker, 3 years ago

Replying to Don-vip:

Search works again, Launchpad guy fixed the problem after Dirk complained :)

So a shell commands with 100 failures was enough to prove the issue. I had about 300 fails as it took some time to have a proper command. Was worth the work ;-)

Modify Ticket

Change Properties
Set your email in Preferences
Action
as closed The owner will remain team.
as The resolution will be set.
The resolution will be deleted. Next status will be 'reopened'.

Add Comment


E-mail address and name can be saved in the Preferences .
 
Note: See TracTickets for help on using tickets.