Opened 6 years ago
Last modified 6 years ago
#16360 new defect
I18n script creates data for transifex directories
| Reported by: | stoecker | Owned by: | team |
|---|---|---|---|
| Priority: | normal | Milestone: | |
| Component: | unspecified | Version: | |
| Keywords: | i18n transifex | Cc: | floscher, michael2402 |
Description
The I18n script only skips transifex on po file generation, not at data file step.
Attachments (1)
Change History (21)
comment:1 by , 6 years ago
| Keywords: | i18n transifex added |
|---|
comment:2 by , 6 years ago
| Cc: | added |
|---|
comment:3 by , 6 years ago
What is the "data file step" that should be skipped? Which i18n script, one of the perl scripts or one of the Ant targets? I don't really understand what exactly goes wrong…
comment:4 by , 6 years ago
The buildplugins target is calling plugintrans for all plugins. It should skip the transifex ones.
Maybe: The target singpleplugintrans could issue a warning for transifex targets.
comment:5 by , 6 years ago
Hi, maybe this patch could be a solution. It wraps the contents of target plugintrans in an <if><available><then></then><else></else></if> construct.
If a Transifex configuration is present, the target only echoes a warning and does nothing else. Otherwise the target is executed like before.
This should fix both buildplugins and singleplugintrans.
follow-up: 9 comment:8 by , 6 years ago
I saw that there were issues with the last JOSM-i18n run of Jenkins. Only had a brief look, will examine that later today.
follow-up: 10 comment:9 by , 6 years ago
Replying to floscher:
I saw that there were issues with the last JOSM-i18n run of Jenkins.
The i18n extraction/upload to Launchpad is done by private scripts on JOSM server every time a new "latest" version is released (automatically each night, or manually on demand). The Jenkins job is trigerred every morning after Launchpad updates the bazaar branch with new translations from contributors. The goal is to detect Java translations issues (simple quotes).
comment:10 by , 6 years ago
Replying to Don-vip:
Replying to floscher:
I saw that there were issues with the last JOSM-i18n run of Jenkins.
The i18n extraction/upload to Launchpad is done by private scripts on JOSM server every time a new "latest" version is released (automatically each night, or manually on demand). The Jenkins job is trigerred every morning after Launchpad updates the bazaar branch with new translations from contributors. The goal is to detect Java translations issues (simple quotes).
I'm mainly concerned by https://josm.openstreetmap.de/jenkins/job/JOSM-i18n/1051/consoleFull#ant-target-314 :
plugintrans:
[mkdir] Created dir: /var/lib/jenkins/jobs/JOSM-i18n/workspace/i18n/build/plugin_Mapillary.2.OLD
[foreach] Exception in thread "Thread-155" The following error occurred while executing this line:
[foreach] /var/lib/jenkins/jobs/JOSM-i18n/workspace/i18n/build.xml:173: exec returned: 2
[foreach] at org.apache.tools.ant.ProjectHelper.addLocationToBuildException(ProjectHelper.java:568)
[foreach] at org.apache.tools.ant.taskdefs.Ant.execute(Ant.java:443)
[foreach] at org.apache.tools.ant.taskdefs.CallTarget.execute(CallTarget.java:105)
[foreach] at net.sf.antcontrib.logic.ForEach$1.run(ForEach.java:125)
[foreach] at net.sf.antcontrib.util.ThreadPoolThread.run(ThreadPoolThread.java:53)
[foreach] Caused by: /var/lib/jenkins/jobs/JOSM-i18n/workspace/i18n/build.xml:173: exec returned: 2
[foreach] at org.apache.tools.ant.taskdefs.ExecTask.runExecute(ExecTask.java:644)
[foreach] at org.apache.tools.ant.taskdefs.ExecTask.runExec(ExecTask.java:670)
[foreach] at org.apache.tools.ant.taskdefs.ExecTask.execute(ExecTask.java:496)
[foreach] at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:293)
[foreach] at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
[foreach] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[foreach] at java.lang.reflect.Method.invoke(Method.java:498)
[foreach] at org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:106)
[foreach] at org.apache.tools.ant.Task.perform(Task.java:348)
[foreach] at org.apache.tools.ant.taskdefs.Sequential.execute(Sequential.java:68)
[foreach] at net.sf.antcontrib.logic.IfTask.execute(IfTask.java:217)
[foreach] at sun.reflect.GeneratedMethodAccessor37.invoke(Unknown Source)
[foreach] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[foreach] at java.lang.reflect.Method.invoke(Method.java:498)
[foreach] at org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:106)
[foreach] at org.apache.tools.ant.TaskAdapter.execute(TaskAdapter.java:155)
[foreach] at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:293)
[foreach] at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
[foreach] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[foreach] at java.lang.reflect.Method.invoke(Method.java:498)
[foreach] at org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:106)
[foreach] at org.apache.tools.ant.Task.perform(Task.java:348)
[foreach] at org.apache.tools.ant.Target.execute(Target.java:435)
[foreach] at org.apache.tools.ant.Target.performTasks(Target.java:456)
[foreach] at org.apache.tools.ant.Project.executeSortedTargets(Project.java:1405)
[foreach] at org.apache.tools.ant.helper.SingleCheckExecutor.executeTargets(SingleCheckExecutor.java:38)
[foreach] at org.apache.tools.ant.Project.executeTargets(Project.java:1260)
[foreach] at org.apache.tools.ant.taskdefs.Ant.execute(Ant.java:441)
[foreach] ... 3 more
Where does Mapillary.2.OLD come from?? The plugintrans task should be skipped for Mapillary
And https://josm.openstreetmap.de/jenkins/job/JOSM-i18n/1051/consoleFull#ant-target-313:
plugintrans:
[mkdir] Created dir: /var/lib/jenkins/jobs/JOSM-i18n/workspace/i18n/build/plugin_Mapillary
[gettext-extract] ../plugins/Mapillary
[gettext-extract] build/plugin_Mapillary
[gettext-extract] build
[gettext-extract] /tmp/srcfiles8641058713880215098.tmp
[gettext-extract] Executing: xgettext -c --from-code=utf-8 --output=/var/lib/jenkins/jobs/JOSM-i18n/workspace/i18n/build/plugin_Mapillary/josm.pot --language=Java -k -ktrc:1c,2 -kmarktrc:1c,2 -ktr -kmarktr -ktrn:1,2 -ktrnc:1c,2,3 --files-from=/tmp/srcfiles8641058713880215098.tmp
[exec] Reading file po/sk.po (lang sk)
[exec] Reading file po/sv.po (lang sv)
[exec] Reading file po/uk.po (lang uk)
[exec] Reading file po/vi.po (lang vi)
[exec] Reading file po/zh_CN.po (lang zh_CN)
[exec] Reading file po/zh_TW.po (lang zh_TW)
[exec] Created file ../plugins/Create_grid_of_ways/data/ast.lang : Added 7 strings out of 7 (100.0%).
[exec] Created file ../plugins/Create_grid_of_ways/data/be.lang : Added 7 strings out of 7 (100.0%).
[exec] Created file ../plugins/Create_grid_of_ways/data/bg.lang : Added 7 strings out of 7 (100.0%).
[exec] Created file ../plugins/Create_grid_of_ways/data/ca.lang : Added 7 strings out of 7 (100.0%).
[exec] Created file ../plugins/Create_grid_of_ways/data/ca-valencia.lang: Added 7 strings out of 7 (100.0%).
[exec] Created file ../plugins/Create_grid_of_ways/data/cs.lang : Added 7 strings out of 7 (100.0%).
[exec] Created file ../plugins/Create_grid_of_ways/data/da.lang : Added 7 strings out of 7 (100.0%).
[exec] Created file ../plugins/Create_grid_of_ways/data/de.lang : Added 7 strings out of 7 (100.0%).
[exec] Created file ../plugins/Create_grid_of_ways/data/el.lang : Added 2 strings out of 7 ( 28.5%).
[exec] Created file ../plugins/Create_grid_of_ways/data/en_AU.lang : Added 7 strings out of 7 (100.0%).
[exec] Created file ../plugins/Create_grid_of_ways/data/en_GB.lang : Added 7 strings out of 7 (100.0%).
[exec] Created file ../plugins/Create_grid_of_ways/data/es.lang : Added 7 strings out of 7 (100.0%).
[exec] Created file ../plugins/Create_grid_of_ways/data/et.lang : Added 2 strings out of 7 ( 28.5%).
[exec] Created file ../plugins/Create_grid_of_ways/data/fi.lang : Added 1 strings out of 7 ( 14.2%).
[exec] Created file ../plugins/Create_grid_of_ways/data/fr.lang : Added 7 strings out of 7 (100.0%).
[exec] Created file ../plugins/Create_grid_of_ways/data/gl.lang : Added 3 strings out of 7 ( 42.8%).
[exec] Created file ../plugins/Create_grid_of_ways/data/hu.lang : Added 7 strings out of 7 (100.0%).
[exec] Created file ../plugins/Create_grid_of_ways/data/id.lang : Added 7 strings out of 7 (100.0%).
[exec] Created file ../plugins/Create_grid_of_ways/data/it.lang : Added 7 strings out of 7 (100.0%).
[exec] Created file ../plugins/Create_grid_of_ways/data/ja.lang : Added 7 strings out of 7 (100.0%).
[exec] Created file ../plugins/Create_grid_of_ways/data/km.lang : Added 2 strings out of 7 ( 28.5%).
[exec] Created file ../plugins/Create_grid_of_ways/data/ko.lang : Added 1 strings out of 7 ( 14.2%).
[exec] Created file ../plugins/Create_grid_of_ways/data/lt.lang : Added 3 strings out of 7 ( 42.8%).
[exec] Created file ../plugins/Create_grid_of_ways/data/nb.lang : Added 7 strings out of 7 (100.0%).
[exec] Created file ../plugins/Create_grid_of_ways/data/nl.lang : Added 7 strings out of 7 (100.0%).
[exec] Created file ../plugins/Create_grid_of_ways/data/pl.lang : Added 4 strings out of 7 ( 57.1%).
[exec] Created file ../plugins/Create_grid_of_ways/data/pt.lang : Added 7 strings out of 7 (100.0%).
[exec] Created file ../plugins/Create_grid_of_ways/data/pt_BR.lang : Added 7 strings out of 7 (100.0%).
[exec] Created file ../plugins/Create_grid_of_ways/data/ru.lang : Added 7 strings out of 7 (100.0%).
[exec] Created file ../plugins/Create_grid_of_ways/data/sk.lang : Added 7 strings out of 7 (100.0%).
[exec] Created file ../plugins/Create_grid_of_ways/data/sv.lang : Added 6 strings out of 7 ( 85.7%).
[exec] Created file ../plugins/Create_grid_of_ways/data/uk.lang : Added 7 strings out of 7 (100.0%).
[exec] Created file ../plugins/Create_grid_of_ways/data/vi.lang : Added 7 strings out of 7 (100.0%).
[exec] Created file ../plugins/Create_grid_of_ways/data/zh_CN.lang : Added 7 strings out of 7 (100.0%).
[exec] Created file ../plugins/Create_grid_of_ways/data/zh_TW.lang : Added 7 strings out of 7 (100.0%).
[exec] Created file ../plugins/Create_grid_of_ways/data/en.lang : Added 7 strings out of 7 (100.0%).
[delete] Deleting directory /var/lib/jenkins/jobs/JOSM-i18n/workspace/i18n/build/plugin_Create_grid_of_ways
Here also, plugintrans should be skipped but isn't. Also why does Mapillary change to Create_grid_of_ways in the middle of the task?
comment:11 by , 6 years ago
Sometimes Jenkins and SVN are confused with the svn:externals => Github links and produce stale data. I'll clean the workspace and restart the job.
comment:12 by , 6 years ago
Can you wait a sec, I'm currently in the process of making the pt_assistant plugin an svn:external ;-) …
comment:14 by , 6 years ago
P.S. Thanks for the fix. I had a look at it before making the ticket, but I consider build.xml syntax brain damaging and hoped you'd be faster.
comment:16 by , 6 years ago
Seems to be at least a bit better now (the "OLD" directories are gone), but I'm still seriously confused to why inside a single plugintrans target multiple plugin names occur:
plugintrans:
[mkdir] Created dir: /var/lib/jenkins/jobs/JOSM-i18n/workspace/i18n/build/plugin_Mapillary
[exec] Created file ../plugins/DirectUpload/data/pt_BR.lang : Added 28 strings out of 29 ( 96.5%).
[exec] Created file ../plugins/DirectUpload/data/ru.lang : Added 29 strings out of 29 (100.0%).
[exec] Created file ../plugins/DirectUpload/data/sk.lang : Added 29 strings out of 29 (100.0%).
[exec] Created file ../plugins/DirectUpload/data/sv.lang : Added 20 strings out of 29 ( 68.9%).
[exec] Created file ../plugins/DirectUpload/data/uk.lang : Added 29 strings out of 29 (100.0%).
[exec] Created file ../plugins/DirectUpload/data/vi.lang : Added 28 strings out of 29 ( 96.5%).
[exec] Created file ../plugins/DirectUpload/data/zh_CN.lang : Added 28 strings out of 29 ( 96.5%).
[exec] Created file ../plugins/DirectUpload/data/zh_TW.lang : Added 28 strings out of 29 ( 96.5%).
[exec] Created file ../plugins/DirectUpload/data/en.lang : Added 29 strings out of 29 (100.0%).
[delete] Deleting directory /var/lib/jenkins/jobs/JOSM-i18n/workspace/i18n/build/plugin_DirectUpload
[gettext-extract] ../plugins/Mapillary
At least for the Mapillary and wikipedia plugins artifacts are no longer produced, but the ones for geojson are still there. So it appears to be not fully solved yet.
comment:19 by , 6 years ago
I addressed the issue in [o34262].
The problem was that the $path variable in plugintrans is potentially an absolute path and when I concatenated it with another path, I got something like /var/lib/jenkins/jobs/JOSM-i18n/workspace/plugins/var/lib/jenkins/jobs/JOSM-i18n/workspace/plugins/ElevationProfile/.tx/config and that file obviously always doesn't exist.
I would attribute the other issue with changing plugin names to asynchronous execution, which I guess mixed up the logging output. The separate plugintrans targets seem to execute in parallel, so the log output is not properly grouped together for each target, but the outputs of the targets get mixed together.
So hopefully this only looked like an issue, but isn't actually one.
I'm probably spoiled by Gradle, which neatly groups together the logging outputs of each task in the console or when writing logging files, even if the tasks ran in parallel.
comment:20 by , 6 years ago
| Cc: | added |
|---|
For reference: [o34023] (#8645)