GPX layer glitch

[bielebog]

What steps will reproduce the problem?

1. Load GPX data in download dialog

What is the expected result?

GPX traces from OSM database should be shown correctly.

What happens instead?

GPX data is shown, but somehow the latitudinal connection of GPX trace points is messed up.

Please provide any additional information below. Attach a screenshot if possible.

URL:https://josm.openstreetmap.de/svn/trunk
Repository:UUID: 0c6e7542-c601-0410-84e7-c038aed88b3b
Last:Changed Date: 2018-11-05 01:16:59 +0100 (Mon, 05 Nov 2018)
Build-Date:2018-11-05 02:33:06
Revision:14412
Relative:URL: ^/trunk

Identification: JOSM/1.5 (14412 de) Linux Ubuntu 14.04.5 LTS
Memory Usage: 1248 MB / 1751 MB (714 MB allocated, but free)
Java version: 1.8.0_171-8u171-b11-2~14.04-b11, Oracle Corporation, OpenJDK 64-Bit Server VM
Screen: :0.0 1280x800, :0.1 1680x1050
Maximum Screen Size: 1680x1050
Java package: openjdk-8-jre:amd64-8u171-b11-2~14.04
Java ATK Wrapper package: libatk-wrapper-java:all-0.30.4-4
VM arguments: [-Djosm.restart=true, -Djosm.dir.name=JOSM-latest, -Djava.net.useSystemProxies=true]
Program arguments: [${HOME}/Tracks/2018/11/2018-11-06.joz]
Dataset consistency test: No problems found

Plugins:
+ CustomizePublicTransportStop (34501)
+ DirectUpload (34502)
+ EasyPresets (1537621333)
+ ImportImagePlugin (34576)
+ InfoMode (34523)
+ Mapillary (v1.5.17)
+ PicLayer (34544)
+ RoadSigns (34553)
+ Tracer2 (34564)
+ apache-commons (34506)
+ apache-http (34632)
+ changeset-viewer (1537565805)
+ download_along (34503)
+ ejml (34389)
+ eventbus (34636)
+ geojson (87)
+ geotools (34513)
+ gpsblam (34515)
+ gpxfilter (34506)
+ graphview (34576)
+ imagery_offset_db (34641)
+ jaxb (34506)
+ jna (34633)
+ jts (34524)
+ livegps (34526)
+ log4j (34527)
+ measurement (34529)
+ pbf (34576)
+ photo_geotagging (34576)
+ photoadjust (34684)
+ pt_assistant (v2.1.9)
+ public_transport (34548)
+ public_transport_layer (34549)
+ rasterfilters (34550)
+ reltoolbox (34614)
+ reverter (34552)
+ rex (49)
+ tageditor (34560)
+ tagging-preset-tester (34561)
+ terracer (34584)
+ todo (30306)
+ turnlanes-tagging (272)
+ undelete (34568)
+ utilsplugin2 (34506)
+ wikipedia (v1.1.1)

Tagging presets:
+ https://josm.openstreetmap.de/josmfile?page=Presets/Trees&zip=1
+ https://josm.openstreetmap.de/josmfile?page=Presets/Historical_Objects&zip=1
+ ${HOME}/Downloads/Presets_Historic_Stone.zip
+ https://josm.openstreetmap.de/josmfile?page=Presets/Playground_Equipment&zip=1
+ https://josm.openstreetmap.de/josmfile?page=Presets/CommonKeyboardShortcuts&zip=1
+ https://josm.openstreetmap.de/josmfile?page=Presets/Communication_Towers&zip=1
+ https://josm.openstreetmap.de/josmfile?page=Presets/Heritage&zip=1
+ https://josm.openstreetmap.de/josmfile?page=Presets/BuildingPreset&zip=1
+ https://raw.githubusercontent.com/species/josm-preset-wheelchair/master/sidewalks_kerbs.xml
+ https://josm.openstreetmap.de/josmfile?page=Presets/Community_Centre&zip=1
+ https://josm.openstreetmap.de/josmfile?page=Presets/Healthcare&zip=1
+ https://josm.openstreetmap.de/josmfile?page=Presets/Historic_Stone&zip=1
+ https://josm.openstreetmap.de/josmfile?page=Presets/Crafts&zip=1
+ https://josm.openstreetmap.de/josmfile?page=Presets/NewTags&zip=1
+ https://josm.openstreetmap.de/josmfile?page=Presets/hiking_routes_with_trail_marking&zip=1
+ https://raw.github.com/Flacus/Windrad/master/windrad.xml
+ https://josm.openstreetmap.de/josmfile?page=Presets/public_bookcase&zip=1
+ https://josm.openstreetmap.de/josmfile?page=Presets/Surveillance&zip=1
+ https://josm.openstreetmap.de/josmfile?page=Presets/Industrial&zip=1
+ https://raw.githubusercontent.com/yopaseopor/traffic_signs_preset_JOSM/master/NL.zip
+ https://josm.openstreetmap.de/josmfile?page=Presets/caravan_site&zip=1
+ ${HOME}/.josm/surveillance_preset

Map paint styles:
- https://josm.openstreetmap.de/josmfile?page=Styles/Lane_features&zip=1
- https://josm.openstreetmap.de/josmfile?page=Styles/Lane_and_Road_Attributes&zip=1
- https://josm.openstreetmap.de/josmfile?page=Styles/Cycleways&zip=1
- https://josm.openstreetmap.de/josmfile?page=Styles/Enhanced_Lane_and_Road_Attributes&zip=1
- https://josm.openstreetmap.de/josmfile?page=Styles/Lane_features_ryg&zip=1
- https://josm.openstreetmap.de/josmfile?page=Styles/PublicTransport&zip=1

Last errors/warnings:
- W: java.io.IOException: Attribution is not loaded yet
- W: java.io.IOException: Attribution is not loaded yet
- W: java.io.IOException: Attribution is not loaded yet
- W: java.io.IOException: Attribution is not loaded yet
- W: java.io.IOException: Attribution is not loaded yet
- W: java.io.IOException: Attribution is not loaded yet
- W: java.io.IOException: Attribution is not loaded yet
- W: java.io.IOException: Attribution is not loaded yet
- W: java.io.IOException: Attribution is not loaded yet
- W: java.io.IOException: Attribution is not loaded yet

[aseerel4c26]

Really a strange look on the screenshot. That may be a result of the change done in response to ​https://github.com/openstreetmap/openstreetmap-website/issues/2046 (see there). If it is, then JOSM needs to stop drawing lines between points of "private" and "public" traces.

[aseerel4c26]

By the way, I confirm - happens for me too and corresponds to the API code change.

[Klumbumbus]

Well, while I can understand the reason for that API change/fix (privacy...), it kinda totally breaks the display of downloaded gpx data in all JOSM versions.

[geow]

Confirmed, a hot fix would be great!

Thanks!

JOSM/1.5 (14382 en) Mac OS X 10.12.6
OS Build number: Mac OS X 10.12.6 (16G1618)
Memory Usage: 864 MB / 2731 MB (197 MB allocated, but free)
Java version: 1.8.0_191-b12, Oracle Corporation, Java HotSpot(TM) 64-Bit Server VM

[bielebog]

Replying to Klumbumbus:

Well, while I can understand the reason for that API change/fix (privacy...), it kinda totally breaks the display of downloaded gpx data in all JOSM versions.

If I understand this right, the underlying API issue ​https://github.com/openstreetmap/openstreetmap-website/issues/2046 probably won't be reverted until fixed in JOSM. So it might be wise to inform JOSM users via JOSM MOTD about this known issue.

[stoecker]

Further reduce "draw.rawgps.max-line-length" to nearly 0 or set "draw.rawgps.lines" to false? If they really send unordered data we can't draw lines anymore for server data.

NOTE that affects only server data. For GPX files different settings are used (extension ".local").

[stoecker]

Ahh, we need to handle the complete open tracks separate. They weren't there when that was initially implemented.

[Klumbumbus]

Replying to stoecker:

If they really send unordered data

Only for tracks of type Public and Private, not for tracks of type Trackable and Identifyable as far as I understood.

[aseerel4c26]

Replying to Klumbumbus:

Replying to stoecker:

If they really send unordered data

Only for tracks of type Public and Private, not for tracks of type Trackable and Identifyable as far as I understood.

Correct. Each single Public and Private trace ("unordered") is lumped together with all other "unordered" points and returned in latitude-longitude-timestamp order (that is: sorted by coordinates) which currently creates that interesting zigzag pattern (see screenshot on top).

At least as the API state is now a possiblity for editors would be to draw lines for points which contain a timestamp, but only ~2x2 pixel dots/heatmap/whatever for points without a timestamp.

[Don-vip]

Workaround is to hide problematic tracks:

[Don-vip]

@team: I won't have time to dig into this before 19/11. If someone's willing to fix it, go ahead.

[stoecker]

Ticket #16970 has been marked as a duplicate of this ticket.

[Don-vip]

Ticket #16983 has been marked as a duplicate of this ticket.

[Klumbumbus]

Ticket #16986 has been marked as a duplicate of this ticket.

[Pontiac_CZ]

I confirm this issue with version 14382.

BTW I have figured out that if I split the "Downloaded GPX data" layer using split tracks into new layers I get a couple of named GPX layers along with a special one called "GPX split result". That's the problematic one. If I disable it (and the original GPX layer of course), I get rid off those horizontal lines all over the screen (until next GPX download).

[Klumbumbus]

Ticket #16996 has been marked as a duplicate of this ticket.

[Don-vip]

Ticket #17023 has been marked as a duplicate of this ticket.

[Don-vip]

In 14451/josm:

fix #16963 - do not draw lines for GPX unordered tracks

[aseerel4c26]

Great, thanks for adjusting, Don-vip! :-) Tested with version 14454.

Now it is quite easy to produce something usable with the option "big GPS dots" ("Große GPS-Punkte") and, if wanted, heatmap style. A follow-up visualisation idea in #17032.

[bielebog]

Yup, thanks for fixing this, Don-vip! Works in latest 14454. I second aseerel4c26's visualization ideas.

[Don-vip]

Ticket #17560 has been marked as a duplicate of this ticket.

[Bjoeni]

There's one issue with this fix: Lines for local tracks without timestamps aren't drawn either. The ​Gpx standard clearly states

A Track Segment holds a list of Track Points which are logically connected in order. To represent a single GPS track where GPS reception was lost, or the GPS receiver was turned off, start a new Track Segment for each continuous span of track data.

so the data returned by the server is completely invalid, but pretty much all the other files on this planet still follow the standard. And there are plenty of reasons for tracks without timestamps, including privacy and artificially created ones (to display a route).

I see two options:

• The OSM website guys actually care about the GPX standard and create an extension that allows them to add unordered points to a GPX file (I don't really see this happening, though I haven't contacted them). They could also create a new segment for each point but that would probably result in too big files.
• We only define segments as unordered when the file is coming from an OSM server (so essentially a hack... because our own system breaks the standard).

Any chance of at least the latter one happening?

[Bjoeni]

And also, when being converted to a data layer the server data still creates a big mess. But that's probably ok as is, because at least I don't really see any reason for converting public GPX tracks to data layers.

[Bjoeni]

see #17596