#20174 closed enhancement (fixed)
Exclude administrative borders, especially national administrative borders from "very long segment"
| Reported by: | mkoniecz | Owned by: | team |
|---|---|---|---|
| Priority: | normal | Milestone: | 20.12 |
| Component: | Core validator | Version: | |
| Keywords: | template_report long segment boundary | Cc: |
Description
What steps will reproduce the problem?
- Try to debug subtly broken geometries (why Belarus is partially in Poland, see final section for more details)
- Try using JOSM validator
- Download Poland border for validation
What is the expected result?
No spurious validator warnings.
What happens instead?
Very long segment of 18 kilometres (1)
https://www.openstreetmap.org/way/715181351
Very long segment of 20 kilometres (1)
https://www.openstreetmap.org/way/9419404
Very long segment of 27 kilometres (1)
https://www.openstreetmap.org/way/9419404
But nowadays I think that national borders are fairly accurate and very long segments there actually happen. This reports seem spurious and unwanted.
Please provide any additional information below. Attach a screenshot if possible.
Right now I am trying to debug why Belarus is in Poland using JOSM validator and this errors seem spurious
Broken (land area of Poland): http://overpass-turbo.eu/s/10CN https://www.openstreetmap.org/relation/936128
Working (Poland): http://overpass-turbo.eu/s/10CO https://www.openstreetmap.org/relation/49715
Relative:URL: ^/trunk Repository:UUID: 0c6e7542-c601-0410-84e7-c038aed88b3b Last:Changed Date: 2020-11-25 13:40:57 +0100 (Wed, 25 Nov 2020) Revision:17360 Build-Date:2020-11-26 02:30:53 URL:https://josm.openstreetmap.de/svn/trunk Identification: JOSM/1.5 (17360 en_GB) Linux Ubuntu 20.04.1 LTS Memory Usage: 473 MB / 976 MB (107 MB allocated, but free) Java version: 11.0.9.1+1-Ubuntu-0ubuntu1.20.04, Ubuntu, OpenJDK 64-Bit Server VM Look and Feel: javax.swing.plaf.metal.MetalLookAndFeel Screen: :0.0 1920×1080 (scaling 1.00×1.00) Maximum Screen Size: 1920×1080 Best cursor sizes: 16×16→16×16, 32×32→32×32 Desktop environment: LXQt Java package: openjdk-11-jre:amd64-11.0.9.1+1-0ubuntu1~20.04 Java ATK Wrapper package: libatk-wrapper-java:all-0.37.1-1 Environment variable LANG: en_GB.UTF-8 libcommons-logging-java: libcommons-logging-java:all-1.2-2 fonts-noto: fonts-noto:- Dataset consistency test: No problems found Plugins: + buildings_tools (35640) + measurement (35640) + reverter (35640) + todo (30306) Last errors/warnings: - 00166.625 W: java.net.NoRouteToHostException: No route to host (Host unreachable) - 00166.626 E: java.net.NoRouteToHostException: No route to host (Host unreachable) - 00166.655 E: java.util.concurrent.ExecutionException: org.openstreetmap.josm.io.OsmTransferException: Could not connect to the OSM server. Please check your internet connection.. Cause: org.openstreetmap.josm.io.OsmTransferException: Could not connect to the OSM server. Please check your internet connection.. Cause: java.net.NoRouteToHostException: No route to host (Host unreachable) - 00166.663 E: org.openstreetmap.josm.io.OsmTransferException: Could not connect to the OSM server. Please check your internet connection.. Cause: java.net.NoRouteToHostException: No route to host (Host unreachable) - 00166.667 W: java.net.NoRouteToHostException: No route to host (Host unreachable) - 00166.673 E: java.net.NoRouteToHostException: No route to host (Host unreachable) - 00166.745 E: Network exception - <html>Failed to open a connection to the remote server<br>'https://api.openstreetmap.org/api/0.6/nodes?nodes=3826719559,7429244288,7738176106,615635937,3826719558,7429244289,7429244290,1741658269,7738176104,615635939,7429244291,7738176105,615635938,7429244292,615635941,952747933,7429244293,7738176111,7429244294,1741658265,615635943,7429244295,615635942,7429244296,7738176098,7429244297,1741658262,7738176099,7429244298,7738176096,615635947,7429244299,7738176097,615635946,7429244300,7738176102,7429244301,1741658258,7738176103,615635948,7429244302,7738176100,615635951,7429244303,7738176101,615635950,7429244336,1741658287,7738176090,615635921,7429244337,7738176091,7429244338,1741658285,7738176088,615635923,7429244339,7738176089,615635922,7429244340,1741658283,7738176094,615635925,7429244341,7738176095,7429244342,7738176092,615635927,7429244343,7738176093,615635926,7429244344,1741658279,615635929,7429244345,7429244346,7429244347,615635930,7429244348,1741658275,7738176086,615635933,7429244349,7738176087,615635932,7429244350,615635935,7429244351,1741658272,7738176085,615635934,7429244320,7429244321,1741658302,615635904,7429244322,615635907,7429244323,615635906,7429244324,7429244325,1741658298,615635908,7429244326,615635911,7429244327,1741658296,615635910,7429244328,615635913,7429244329,1741658294,7429244330,615635915,7429244331,1741658292,615635914,7429244332,7429244333,615635916,7429244334,1741658289,615635919,7429244335,615635918,7429244240,615635761,7429244241,615635760,7429244242,615635763,7429244243,615635762,7429244244,7429244245,615635764,7429244246,615635767,7429244247,2001832150,615635766,7429244248,615635769,7429244249,615635768,7429244250,615635771,7429244251,615635770,7429244252,615635773,7429244253,615635772,7429244254,7429244255,615635774,7429244224,615635745,7429244225,615635744,7429244226,615635747,7429244227,7429244228,7429244229,615635748,7429244230,615635751,7429244231,615635750'.<br>Please check your internet connection.</html> - 02691.478 W: java.net.SocketTimeoutException: Read timed out. Cause: java.net.SocketTimeoutException: Read timed out
Attachments (0)
Change History (24)
comment:2 by , 3 years ago
| Component: | Core → Core validator |
|---|---|
| Keywords: | long segment boundary added |
comment:3 by , 3 years ago
Have to take a deeper look. Is maritime=yes the keyword for this special situation or is it more general.
I have to admit that your examples and closed ways are probably cases we do not want to warn about.
Does splitting the way harm in general?
I know it is not needed but similar too large MPs changing these long ways will result in a bigger CS area.
At least for linear ways, an info warning does not harm, in my eyes.
follow-up: 6 comment:4 by , 3 years ago
Is maritime=yes the keyword for this special situation or is it more general.
maritime=yes is for maritime boundaries https://wiki.openstreetmap.org/wiki/Tag:maritime%3Dyes
Also some use in a different meaning, also documented on that page.
But there is plenty of land borders forming straight lines, see https://www.openstreetmap.org/#map=7/24.946/23.522
Does splitting the way harm in general?
Edits done solely to pacify validator without any benefit whatsoever seems to be a bad idea to me. What is the benefit of splitting Egypt border in small chunks?
Maybe making easier to download it (what happens when you have node in bb) is a sufficient justification?
At least for linear ways, an info warning does not harm, in my eyes.
Downgrading to info would be fine for me. Currently it is at warning level and mixes with reports about actual problems.
comment:5 by , 3 years ago
| Summary: | Exlude administrative borders, especially national administrative borders from "very long segment" → Exclude administrative borders, especially national administrative borders from "very long segment" |
|---|
comment:6 by , 3 years ago
Replying to mkoniecz:
Is maritime=yes the keyword for this special situation or is it more general.
maritime=yesis for maritime boundaries https://wiki.openstreetmap.org/wiki/Tag:maritime%3Dyes
Also some use in a different meaning, also documented on that page.
But there is plenty of land borders forming straight lines, see https://www.openstreetmap.org/#map=7/24.946/23.522
Thanks, that is what I feared about areas with low population and no lower level of boundaries. Can we define the land areas to use boundary.xml?
Does splitting the way harm in general?
Edits done solely to pacify validator without any benefit whatsoever seems to be a bad idea to me. What is the benefit of splitting Egypt border in small chunks?
I was thinking about missing lower level boundaries and that these ways will have to be split in (near) future, anyway, but it is a bad idea.
Maybe making easier to download it (what happens when you have node in bb) is a sufficient justification?
At least for linear ways, an info warning does not harm, in my eyes.
Downgrading to info would be fine for me. Currently it is at warning level and mixes with reports about actual problems.
If it is only a problem with admin_level < 5 (6)?, these ways could be excluded to smaller the list of false positive.
comment:7 by , 3 years ago
If it is only a problem with admin_level < 5 (6)?, these ways could be excluded to smaller the list of false positive.
less important ones also can be straight, see https://www.openstreetmap.org/?mlat=34.3125&mlon=-103.0430#map=14/34.3125/-103.0430
And going by Poland case many borders have section of straight lines, even ones looking to be more wiggly
Thanks, that is what I feared about areas with low population and no lower level of boundaries. Can we define the land areas to use boundary.xml?
Not sure what kind of effect it would have.
I was thinking about missing lower level boundaries and that these ways will have to be split in (near) future, anyway, but it is a bad idea.
None of cases in Poland would qualify for that, internal divisions are not reaching into sea and end near shore. https://www.openstreetmap.org/#map=16/54.1889/15.5448
comment:8 by , 3 years ago
Damn I mixed up waylength and length of segment. My previous comments are quite off the mark.
No objection about ignoring boundaries.
comment:9 by , 3 years ago
So, exclude if either the way has tag boundary=administrative or if is member of a boundary relation with this tag?
Edit: Or can we simply use the maritime=yes tag?
comment:10 by , 3 years ago
No maritime=yes is not a good choice. Maybe even all boundary=* are problematic as normal warning.
comment:13 by , 3 years ago
| Milestone: | → 20.12 |
|---|
comment:14 by , 3 years ago
Ah, I forgot about the case when the tags are all on the relations and the way does not have the tags from the ignore list. Was this already handled?
comment:15 by , 3 years ago
No. Do you think we can simply check if a parent relation has the boundary tag? Or should there be another check that the way itself doesn't have tags like e.g. highway=*?
comment:16 by , 3 years ago
I would say the latter, because if there is a highway with a very long segment which is "randomly" in a boundary relation it is still suspicious.
follow-up: 19 comment:17 by , 3 years ago
Okay, waterway=* and barrier=* would be my next candidates. Maybe also railway=*?
comment:18 by , 3 years ago
Linear waterways have long segments when they go through lakes. (e.g. https://www.openstreetmap.org/way/514055688#map=11/47.6004/9.4112)
comment:19 by , 3 years ago
Replying to GerdP:
Okay,
waterway=*andbarrier=*would be my next candidates. Maybe alsorailway=*?
Did you mean to remove the warning for them?
comment:20 by , 3 years ago
Well, I wouldn't mind to remove the whole test ;)
I thought about two lists, one as "always check" with highway=*and the other tags from comment:17
and - if not matched - the exclude list plus a check if a parent relation is a type=boundary:
private boolean ignoreWay(Way w) { if (w.hasKey("highway", "railway", "waterway", "barrier")) return false; return w.hasKey("boundary") || w.hasTag("route", "ferry") || w.hasTag("bay", "fjord") || w.hasTag("natural", "strait") || visitedWays.contains(w) || w.referrers(Relation.class).anyMatch(Relation::isBoundary); }
comment:22 by , 3 years ago
Like this:
private boolean ignoreWay(Way w) { if (visitedWays.contains(w)) return true; if (w.hasKey("highway", "railway", "waterway", "barrier")) return false; return w.hasKey("boundary") || w.hasTag("route", "ferry") || w.hasTag("bay", "fjord") || w.hasTag("natural", "strait") || w.referrers(Relation.class).anyMatch(Relation::isBoundary); }
Another possible change would be to limit this check on ways with modified or new nodes if this is about unintended changes.
comment:23 by , 3 years ago
I'm not such a big fan of checks only for new/changed objects, because
- if it is an error then it is always an error, no matter if the object is old or new
- if the original uploader "shameless" ignores the error it is then hidden for other more careful mappers, which manually run the validator on a dataset
I would not mind to add this to the exclude list. We already have these excludes: