[Patch] Modifying multiple objects overwrites distinct tag values

[riiga_92@…]

What steps will reproduce the problem?

1. Draw (or use two already existing) closed ways
2. Add the tag "ele=100" to one of them
3. Select both and apply the natural=water preset

What is the expected result?

Two objects:

• natural=water, ele=100
• natural=water

What happens instead?

Two objects:

• natural=water, ele=<different
• natural=water, ele=<different

Please provide any additional information below. Attach a screenshot if possible.

This seems like it could be related to ticket 20952. It seems to not happen with every tag either, trying it with building:levels and tagging a building works fine.

URL:https://josm.openstreetmap.de/svn/trunk
Repository:UUID: 0c6e7542-c601-0410-84e7-c038aed88b3b
Last:Changed Date: 2021-09-03 03:12:33 +0200 (Fri, 03 Sep 2021)
Build-Date:2021-09-03 01:31:19
Revision:18193
Relative:URL: ^/trunk

Identification: JOSM/1.5 (18193 en) Linux Manjaro Linux
Memory Usage: 1367 MB / 3557 MB (1038 MB allocated, but free)
Java version: 1.8.0_292-b10, Oracle Corporation, OpenJDK 64-Bit Server VM
Look and Feel: javax.swing.plaf.metal.MetalLookAndFeel
Screen: :0.0 1920×1200 (scaling 1.00×1.00)
Maximum Screen Size: 1920×1200
Best cursor sizes: 16×16→16×16, 32×32→32×32
Environment variable LANG: sv_SE.UTF-8
System property file.encoding: UTF-8
System property sun.jnu.encoding: UTF-8
Locale info: en_SE
Numbers with default locale: 1234567890 -> 1234567890
Desktop environment: KDE
VM arguments: [-Djosm.restart=true, -Dawt.useSystemAAFontSettings=gasp]
Dataset consistency test: No problems found

Plugins:
+ AddrInterpolation (35640)
+ FastDraw (35640)
+ ImportImagePlugin (35567)
+ OpeningHoursEditor (35640)
+ PicLayer (1.0.1)
+ apache-commons (35524)
+ apache-http (35589)
+ areaselector (368)
+ austriaaddresshelper (1597341117)
+ buildings_tools (35821)
+ ejml (35458)
+ geotools (35458)
+ imagery_offset_db (35640)
+ jaxb (35543)
+ jna (35662)
+ jogl (1.2.3)
+ jts (35458)
+ kendzi3d (1.0.205)
+ kendzi3d-resources (0.0.2)
+ log4j (35458)
+ measurement (35640)
+ opendata (35803)
+ pt_assistant (1ff2e15)
+ reverter (35732)
+ tageditor (35640)
+ terracer (35640)
+ turnlanes-tagging (288)
+ turnrestrictions (35640)
+ utilsplugin2 (35792)

Tagging presets:
+ https://josm.openstreetmap.de/josmfile?page=Presets/PublicTransportOneClick&zip=1
+ ${HOME}/Dokument/QuickSets.zip
+ https://josm.openstreetmap.de/josmfile?page=Presets/ParkingLanes&zip=1
+ https://github.com/kendzi/Simple3dBuildingsPreset/releases/download/0.9_2018-05-08/s3db-preset.zip

Map paint styles:
- https://josm.openstreetmap.de/josmfile?page=Styles/Fixme&zip=1
- https://josm.openstreetmap.de/josmfile?page=Styles/LayerChecker&zip=1
- https://josm.openstreetmap.de/josmfile?page=Styles/Coloured_Streets&zip=1
- https://josm.openstreetmap.de/josmfile?page=Styles/Maxspeed&zip=1
- https://josm.openstreetmap.de/josmfile?page=Styles/Surface-DataEntry&zip=1
- https://josm.openstreetmap.de/josmfile?page=Styles/Coloured_buildings_sv&zip=1
- https://josm.openstreetmap.de/josmfile?page=Styles/Enhanced_Lane_and_Road_Attributes&zip=1
- https://josm.openstreetmap.de/josmfile?page=Styles/Lane_and_Road_Attributes&zip=1
- https://josm.openstreetmap.de/josmfile?page=Styles/ParkingLanes&zip=1
- https://josm.openstreetmap.de/josmfile?page=Styles/PublicTransport&zip=1
- https://josm.openstreetmap.de/josmfile?page=Styles/PriorityRoad&zip=1
- ${HOME}/sidewalks-mod.mapcss
- https://josm.openstreetmap.de/josmfile?page=Styles/SimpleBuildingTags&zip=1
- https://josm.openstreetmap.de/josmfile?page=Styles/SimpleRoofTags&zip=1
- https://raw.githubusercontent.com/yopaseopor/indoormap/master/indoormap-style.mapcss
- https://github.com/GlassOceanos/indoor-JOSM-style/archive/master.zip
- https://raw.githubusercontent.com/species/josm-preset-traffic_sign_direction/master/direction.mapcss
- https://josm.openstreetmap.de/josmfile?page=Styles/SidewalksAndFootways&zip=1

Last errors/warnings:
- 00001.456 W: Update plugins - You updated your JOSM software. To prevent problems the plugins should be updated as well.  Update plugins now?
- 00008.262 E: Failed to locate image 'bus.png'

[skyper]

I cannot reproduce this with r18225. In my case, ele=* is removed from both objects, which is even worse.

[riiga_92@…]

It was present in the previous stable version too (r18118) and it was there I discovered it. I then upgraded to the newest release and found the bug still present and filled this bug report.

I discovered the error thanks to the JOSM validator before upload, it complained about the value of the ele tag. :-)

[skyper]

Yes, with r16709 up to r18218 I am able to reproduce with e.g. wilderness_hut but not with r16703. I think fixing the typo in r16707 made it visible and the bug is tied to the optional length=* for input values as height=* in Communication Tower preset shows the same problem.

Between r18218 and r18225 the behavior changed however to the worst outcome, removing the tags completely.

[marcello@…]

The max length shortens <different> to <different so it isn't the magic word anymore. Since this must work even with a max length of 1 the magic word approach is unfeasible.

[skyper]

Or <different> should never be cut off by max length.

[marcello@…]

Quick and dirty patch: exempt <different> from max length restriction

[skyper]

Do not think it is that dirty, as the shorten magic word was confusing.

Mmh, at least height=* and its ​~6000 obvious cases could profit from a revert/fix. Is there a tool/script which helps in finding the proper value in history?

[skyper]

Looking at e.g. this ​chronology, I think this bug was present for around a decade, now. ​w197175790 is another example with three compromised values.

[Don-vip]

In 18236/josm:

fix #21375 - exempt <different> from max length restriction (patch by marcello)

[gaben]

Is it possible to create a unit test for this?

[gaben]

Replying to skyper:

Is there a tool/script which helps in finding the proper value in history?

Probably there is, although I don't know a ready solution only code libraries. Writing a small script shouldn't be a problem. I will probably do it in a small part of Europe, if you are interested, open a new ticket.

[skyper]

I thought about fixing at least the obvious ones but I`ll wait a few more weeks until more user use the fix JOSM version. Still it will be quite some work as finding the proper value for each object is time consuming.

@Gabe
I will come back to your offer, I guess.

[skyper]

Replying to skyper:

Or <different> should never be cut off by max length.

Well, I remembered correctly, that I had reported this a while ago, #19479.

[Don-vip]

Milestone renamed

[skyper]

Replying to skyper:

I thought about fixing at least the obvious ones but I`ll wait a few more weeks until more user use the fix JOSM version. Still it will be quite some work as finding the proper value for each object is time consuming.

@Gabe
I will come back to your offer, I guess.

Oh, sorry, I have started but did not spread the info here but only on the ​German forum. Was definitely the wrong order.

Now, I could need some help. Is there an easy way to get all translations of <different> which are included in JOSM?

I need to look for < plus at least three symbols (not numbers). Only letters does not work with e.g. Japanese. Additionally, most of the time the expression is at the beginning of the value but not all the time.

[gaben]

I already did a (Python) script, so it's ready, including all translations :) Even uploaded it to GitLab as a snippet, but there are a few edge cases that I want to fix before making it public.

Edit: my approach was to use GeoFabric extracts to get reproducible results, plus once downloaded, it works offline.

Fix method:

• search for issues
• look up the last known non <diff..> value (including translations) in the history file
• use this value instead of <diff..>
• create an XML as final output in JOSM loadable format
• manual upload to OSM

Btw I don't know if overpass can handle a complex global query with all the translations on keys and values.

[skyper]

Replying to gaben:

I already did a (Python) script, so it's ready, including all translations :) Even uploaded it to GitLab as a snippet, but there are a few edge cases that I want to fix before making it public.

Edit: my approach was to use GeoFabric extracts to get reproducible results, plus once downloaded, it works offline.

Fix method:

• search for issues
• look up the last known non <diff..> value (including translations) in the history file
• use this value instead of <diff..>
• create an XML as final output in JOSM loadable format
• manual upload to OSM

Btw I don't know if overpass can handle a complex global query with all the translations on keys and values.

It already takes 700+ seconds for whole planet with nrw[~"."~"^<dif"].
I tried nwr[~"."~".*<[\x21-\x7E]{3}.*"] but this does only work on small areas and still does not find Japanese or Cyrillic letters.

Regarding replacing the values it is not always that easy. It is fine if you find a value in previous versions but if not it depends on the key and your effort to fix it properly, or to ask for help.

Example:

• I was able to always fix addr:city and most of addr:suburb but almost none of addr:postcode
• Especially, source has been changed meanwhile, so I had to adjust the values. Some of them additionally had duplicates within the value.

[skyper]

Replying to gaben:

I already did a (Python) script, so it's ready, including all translations :)

Is it possible to share the translations as list or text file. Finding all objects seems at least for me, the tough task, then actually fixing the problems. Maybe, I`ll find someone who can extract all objects from a dump.

[GerdP]

Its probably enough to concentrate on these few besides <dif:
<různé>, <различные>,<unterschiedlich>

[skyper]

It is hard without knowing all affected keys but mmd and myself found:

• <差異あり> - ​changeset/15881801, ​changeset/15870833
• <verschillend>
• <diverso>
• <olika>
• <різні>

• <many> ??? - ​changeset/34101771
• <Null> ???

[gaben]

Replying to skyper:

Is it possible to share the translations as list or text file. Finding all objects seems at least for me, the tough task, then actually fixing the problems. Maybe, I`ll find someone who can extract all objects from a dump.

Sorry, forgot this. I'll attach the whole script which I used for searching and fixing the issues. I have a feeling that I may overdone a bit :)

[gaben]

Python fixer script for josm#21375

[gaben]

If you are interested, I can publish the JOSM translation extractor as well.

[skyper]

No problem, forgot about this ticket, too.
Thanks for the script, though, I will have no time to work on this in the near future.