Modify

Opened 3 years ago

Last modified 6 months ago

#18074 new enhancement

Small issue with precision

Reported by: naoliv Owned by: team
Priority: normal Milestone:
Component: Core validator Version:
Keywords: coordinate precision rounding Cc:

Description

Open the attached file, where we have 2 layers: Uploaded (already in OSM) and Before uploading

Validate both: on Before uploading we see only an informational message Building with an almost square angle while at Uploaded (already in OSM) we see a Building duplicated node error, a Crossing buildings warning and a new Overlapping Areas at other.

Together with a friend we were trying to understand why this was happening and we found that the precision of the nodes was changing.

For example, a node that locally was at -23.51108285, -46.489264256 is now at -23.5110829, -46.4892643 in OSM.

Shouldn't the validation tests (and maybe other parts of JOSM?) round the coordinates to avoid inserting data with errors like this?

JOSM:

Relative:URL: ^/trunk
Repository:UUID: 0c6e7542-c601-0410-84e7-c038aed88b3b
Last:Changed Date: 2019-08-23 22:53:24 +0200 (Fri, 23 Aug 2019)
Revision:15315
Build-Date:2019-08-24 01:31:00
URL:https://josm.openstreetmap.de/svn/trunk

Identification: JOSM/1.5 (15315 en) Linux Debian GNU/Linux bullseye/sid
Memory Usage: 321 MB / 2048 MB (123 MB allocated, but free)
Java version: 13-ea+30-Debian-1, Debian, OpenJDK 64-Bit Server VM
Screen: :0.0 1920x1080
Maximum Screen Size: 1920x1080
Java ATK Wrapper package: libatk-wrapper-java:all-0.35.0-3
libcommons-compress-java: libcommons-compress-java:all-1.18-2
libcommons-logging-java: libcommons-logging-java:all-1.2-2
VM arguments: [-Dawt.useSystemAAFontSettings=gasp]
Program arguments: [--language=en]
Dataset consistency test: No problems found

Plugins:
+ EasyPresets (1537621333)
+ FastDraw (34977)
+ OpeningHoursEditor (34977)
+ SimplifyArea (34977)
+ apache-commons (34908)
+ buildings_tools (34982)
+ ejml (35049)
+ geojson (124)
+ geotools (34908)
+ http2 (35062)
+ jaxb (35014)
+ jogl (1.2.3)
+ jts (35064)
+ log4j (34908)
+ measurement (35051)
+ opendata (35102)
+ reverter (35084)
+ tageditor (34977)
+ tagging-preset-tester (34908)
+ todo (30306)
+ turnlanes-tagging (281)
+ turnrestrictions (34977)
+ undelete (34977)
+ utilsplugin2 (35098)
+ wikipedia (v1.1.1)

Attachments (1)

example.joz (4.5 KB) - added by naoliv 3 years ago.

Download all attachments as: .zip

Change History (13)

Changed 3 years ago by naoliv

Attachment: example.joz added

comment:1 Changed 3 years ago by naoliv

#14053 is partially related with this

comment:2 Changed 14 months ago by simon04

Keywords: coordinate precision rounding added
Milestone: 21.03
Owner: changed from team to simon04
Status: newassigned

The rounding in JOSM and on the OSM server yields slightly different results (-23.5110829 vs. -23.5110828):

Input  -23.51108285,  -46.489264256
OSM DB -23.5110829,   -46.4892643
JOSM   -23.5110828,   -46.4892643    via org.openstreetmap.josm.data.coor.LatLon#getRoundedToOsmPrecision
JOSM*  -23.511083,    -46.489264     new via org.openstreetmap.josm.data.validation.tests.DuplicateNode.NodeHash#roundCoord

Input  -23.511082861, -46.489264251
OSM DB -23.5110829,   -46.4892643
JOSM   -23.5110829,   -46.4892643    via org.openstreetmap.josm.data.coor.LatLon#getRoundedToOsmPrecision
JOSM*  -23.511083,    -46.489264     new via org.openstreetmap.josm.data.validation.tests.DuplicateNode.NodeHash#roundCoord

Let's reduce the precision used by the duplicate node test by one decimal digit.

The advanced preference value validator.duplicatenodes.precision now defaults to 1e-6 (previously 1e-7).

comment:3 Changed 14 months ago by simon04

Resolution: fixed
Status: assignedclosed

In 17612/josm:

fix #18074 - Duplicate Node: detect errors caused by different rounding in JOSM and on OSM server

Reduce the precision used by the duplicate node test by one decimal digit

The advanced preference value validator.duplicatenodes.precision now defaults to 1e-6 (previously 1e-7).

comment:5 Changed 14 months ago by GerdP

Hmm, the default for validator.duplicatenodes.precision was 0.0, not 1e-7
I don't see how this will fix a problem with crossing lines or overlapping areas. It just relaxes the duplicate node test, right?

comment:6 in reply to:  5 Changed 14 months ago by simon04

Replying to GerdP:

Hmm, the default for validator.duplicatenodes.precision was 0.0, not 1e-7

The default comes from https://github.com/openstreetmap/josm/blob/094d5522ac6f9ce00fece7fe64ac8a6bc5c84d52/src/org/openstreetmap/josm/data/validation/tests/DuplicateNode.java#L70

I don't see how this will fix a problem with crossing lines or overlapping areas. It just relaxes the duplicate node test, right?

Yeah.

comment:7 Changed 14 months ago by simon04

In 17624/josm:

see #18074 - Javadoc warnings

comment:8 Changed 14 months ago by GerdP

Resolution: fixed
Status: closedreopened

I think this should be reverted.
I get lots of false positives from this test now. Try eg ways w43266057 w88667177. Distance between the nodes is 4cm, not much, but JOSM clearly shows a gap.

comment:9 Changed 14 months ago by GerdP

BTW: Using the value 0.0 for validator.duplicatenodes.precision results in precision being 1/0.0 = Double.infinite and completely weird results.

comment:10 Changed 14 months ago by simon04

Milestone: 21.03
Owner: changed from simon04 to team
Status: reopenednew

Thanks for your extensive testing. I'll revert.

comment:11 Changed 14 months ago by simon04

In 17665/josm:

see #18074 - Revert "Duplicate Node: detect errors caused by different rounding in JOSM and on OSM server"

This reverts commit r17624.

comment:12 Changed 6 months ago by smootheFiets

I ran into what looks like the same issue yesterday. I've uploaded self-intersecting building geometries. I did run the JOSM validator before uploading, it didn't flag the issue. After deleting the data layer and re-downloading an overlapping area, the JOSM validator duly warned "self crossing ways." Expected behaviour: JOSM validator should've given that warning before uploading.

Two examples: https://www.openstreetmap.org/way/1003961672 (revert to v1 to see the self-intersection) and https://www.openstreetmap.org/way/587543281 (v2 shows the self intersection)

I encounter the same issue with other geometry errors, e.g. building duplicate nodes or building overlaps: no warning before uploading, warning after a fresh download. I can provide examples on request.

I suspect, like naoliv, that the issue is with the finite numerical precision of floats (float32 vs float64?). I don't see what else can go wrong between uploading and re-downloading the same area. In the case of the self-intersections I give above, it's enough to move one node by << 1mm to make the issue go away. So, we're really talking edge cases. But still, I get them on a regular basis. I import building outlines from the Dutch cadaster, a couple thousand in an average week. I get ~a handful of false negatives per week that slip through the cracks of the JOSM validator (which is really good at finding most issues in the data, truth be told).

I don't know to which precision the OSM servers store floats. But surely that can be found out. Would it be possible to round all coordinates in JOSM to that same precision before executing any validator checks? That should make validator results more reproducible, I guess.

Modify Ticket

Change Properties
Set your email in Preferences
Action
as new The owner will remain team.
as The resolution will be set.
to The owner will be changed from team to the specified user.
The owner will change to naoliv
as duplicate The resolution will be set to duplicate.The specified ticket will be cross-referenced with this ticket
The owner will be changed from team to anonymous.

Add Comment


E-mail address and name can be saved in the Preferences.

 
Note: See TracTickets for help on using tickets.