Modify

Opened 3 months ago

Closed 3 months ago

Last modified 3 months ago

#20174 closed enhancement (fixed)

Exclude administrative borders, especially national administrative borders from "very long segment"

Reported by: mkoniecz Owned by: team
Priority: normal Milestone: 20.12
Component: Core validator Version:
Keywords: template_report long segment boundary Cc:

Description

What steps will reproduce the problem?

  1. Try to debug subtly broken geometries (why Belarus is partially in Poland, see final section for more details)
  2. Try using JOSM validator
  3. Download Poland border for validation

What is the expected result?

No spurious validator warnings.

What happens instead?

Very long segment of 18 kilometres (1)
https://www.openstreetmap.org/way/715181351

Very long segment of 20 kilometres (1)
https://www.openstreetmap.org/way/9419404

Very long segment of 27 kilometres (1)
https://www.openstreetmap.org/way/9419404

But nowadays I think that national borders are fairly accurate and very long segments there actually happen. This reports seem spurious and unwanted.

Please provide any additional information below. Attach a screenshot if possible.

Right now I am trying to debug why Belarus is in Poland using JOSM validator and this errors seem spurious

Broken (land area of Poland): http://overpass-turbo.eu/s/10CN https://www.openstreetmap.org/relation/936128

Working (Poland): http://overpass-turbo.eu/s/10CO https://www.openstreetmap.org/relation/49715

Relative:URL: ^/trunk
Repository:UUID: 0c6e7542-c601-0410-84e7-c038aed88b3b
Last:Changed Date: 2020-11-25 13:40:57 +0100 (Wed, 25 Nov 2020)
Revision:17360
Build-Date:2020-11-26 02:30:53
URL:https://josm.openstreetmap.de/svn/trunk

Identification: JOSM/1.5 (17360 en_GB) Linux Ubuntu 20.04.1 LTS
Memory Usage: 473 MB / 976 MB (107 MB allocated, but free)
Java version: 11.0.9.1+1-Ubuntu-0ubuntu1.20.04, Ubuntu, OpenJDK 64-Bit Server VM
Look and Feel: javax.swing.plaf.metal.MetalLookAndFeel
Screen: :0.0 1920×1080 (scaling 1.00×1.00)
Maximum Screen Size: 1920×1080
Best cursor sizes: 16×16→16×16, 32×32→32×32
Desktop environment: LXQt
Java package: openjdk-11-jre:amd64-11.0.9.1+1-0ubuntu1~20.04
Java ATK Wrapper package: libatk-wrapper-java:all-0.37.1-1
Environment variable LANG: en_GB.UTF-8
libcommons-logging-java: libcommons-logging-java:all-1.2-2
fonts-noto: fonts-noto:-
Dataset consistency test: No problems found

Plugins:
+ buildings_tools (35640)
+ measurement (35640)
+ reverter (35640)
+ todo (30306)

Last errors/warnings:
- 00166.625 W: java.net.NoRouteToHostException: No route to host (Host unreachable)
- 00166.626 E: java.net.NoRouteToHostException: No route to host (Host unreachable)
- 00166.655 E: java.util.concurrent.ExecutionException: org.openstreetmap.josm.io.OsmTransferException: Could not connect to the OSM server. Please check your internet connection.. Cause: org.openstreetmap.josm.io.OsmTransferException: Could not connect to the OSM server. Please check your internet connection.. Cause: java.net.NoRouteToHostException: No route to host (Host unreachable)
- 00166.663 E: org.openstreetmap.josm.io.OsmTransferException: Could not connect to the OSM server. Please check your internet connection.. Cause: java.net.NoRouteToHostException: No route to host (Host unreachable)
- 00166.667 W: java.net.NoRouteToHostException: No route to host (Host unreachable)
- 00166.673 E: java.net.NoRouteToHostException: No route to host (Host unreachable)
- 00166.745 E: Network exception - <html>Failed to open a connection to the remote server<br>'https://api.openstreetmap.org/api/0.6/nodes?nodes=3826719559,7429244288,7738176106,615635937,3826719558,7429244289,7429244290,1741658269,7738176104,615635939,7429244291,7738176105,615635938,7429244292,615635941,952747933,7429244293,7738176111,7429244294,1741658265,615635943,7429244295,615635942,7429244296,7738176098,7429244297,1741658262,7738176099,7429244298,7738176096,615635947,7429244299,7738176097,615635946,7429244300,7738176102,7429244301,1741658258,7738176103,615635948,7429244302,7738176100,615635951,7429244303,7738176101,615635950,7429244336,1741658287,7738176090,615635921,7429244337,7738176091,7429244338,1741658285,7738176088,615635923,7429244339,7738176089,615635922,7429244340,1741658283,7738176094,615635925,7429244341,7738176095,7429244342,7738176092,615635927,7429244343,7738176093,615635926,7429244344,1741658279,615635929,7429244345,7429244346,7429244347,615635930,7429244348,1741658275,7738176086,615635933,7429244349,7738176087,615635932,7429244350,615635935,7429244351,1741658272,7738176085,615635934,7429244320,7429244321,1741658302,615635904,7429244322,615635907,7429244323,615635906,7429244324,7429244325,1741658298,615635908,7429244326,615635911,7429244327,1741658296,615635910,7429244328,615635913,7429244329,1741658294,7429244330,615635915,7429244331,1741658292,615635914,7429244332,7429244333,615635916,7429244334,1741658289,615635919,7429244335,615635918,7429244240,615635761,7429244241,615635760,7429244242,615635763,7429244243,615635762,7429244244,7429244245,615635764,7429244246,615635767,7429244247,2001832150,615635766,7429244248,615635769,7429244249,615635768,7429244250,615635771,7429244251,615635770,7429244252,615635773,7429244253,615635772,7429244254,7429244255,615635774,7429244224,615635745,7429244225,615635744,7429244226,615635747,7429244227,7429244228,7429244229,615635748,7429244230,615635751,7429244231,615635750'.<br>Please check your internet connection.</html>
- 02691.478 W: java.net.SocketTimeoutException: Read timed out. Cause: java.net.SocketTimeoutException: Read timed out

Attachments (0)

Change History (24)

comment:1 Changed 3 months ago by GerdP

I would not mind to add this to the exclude list. We already have these excludes:

w.hasTag("route", "ferry") || w.hasTag("bay", "fjord") || w.hasTag("natural", "strait");
Last edited 3 months ago by GerdP (previous) (diff)

comment:2 Changed 3 months ago by skyper

Component: CoreCore validator
Keywords: long segment boundary added

comment:3 Changed 3 months ago by skyper

Have to take a deeper look. Is maritime=yes the keyword for this special situation or is it more general.
I have to admit that your examples and closed ways are probably cases we do not want to warn about.

Does splitting the way harm in general?
I know it is not needed but similar too large MPs changing these long ways will result in a bigger CS area.
At least for linear ways, an info warning does not harm, in my eyes.

comment:4 Changed 3 months ago by mkoniecz

Is maritime=yes the keyword for this special situation or is it more general.

maritime=yes is for maritime boundaries https://wiki.openstreetmap.org/wiki/Tag:maritime%3Dyes

Also some use in a different meaning, also documented on that page.

But there is plenty of land borders forming straight lines, see https://www.openstreetmap.org/#map=7/24.946/23.522

Does splitting the way harm in general?

I consider edits done solely to pacify validator without any benefit whatsoever seems to be a bad idea to me. What is the benefit of splitting Egypt border in small chunks?

Maybe making easier to download it (what happens when you have node in bb) is a sufficient justification?

At least for linear ways, an info warning does not harm, in my eyes.

Downgrading to info would be fine for me. Currently it is at warning level and mixes with reports about actual problems.

Version 0, edited 3 months ago by mkoniecz (next)

comment:5 Changed 3 months ago by mkoniecz

Summary: Exlude administrative borders, especially national administrative borders from "very long segment"Exclude administrative borders, especially national administrative borders from "very long segment"

comment:6 in reply to:  4 Changed 3 months ago by skyper

Replying to mkoniecz:

Is maritime=yes the keyword for this special situation or is it more general.

maritime=yes is for maritime boundaries https://wiki.openstreetmap.org/wiki/Tag:maritime%3Dyes

Also some use in a different meaning, also documented on that page.

But there is plenty of land borders forming straight lines, see https://www.openstreetmap.org/#map=7/24.946/23.522

Thanks, that is what I feared about areas with low population and no lower level of boundaries. Can we define the land areas to use boundary.xml?

Does splitting the way harm in general?

Edits done solely to pacify validator without any benefit whatsoever seems to be a bad idea to me. What is the benefit of splitting Egypt border in small chunks?

I was thinking about missing lower level boundaries and that these ways will have to be split in (near) future, anyway, but it is a bad idea.

Maybe making easier to download it (what happens when you have node in bb) is a sufficient justification?

At least for linear ways, an info warning does not harm, in my eyes.

Downgrading to info would be fine for me. Currently it is at warning level and mixes with reports about actual problems.

If it is only a problem with admin_level < 5 (6)?, these ways could be excluded to smaller the list of false positive.

comment:7 Changed 3 months ago by mkoniecz

If it is only a problem with admin_level < 5 (6)?, these ways could be excluded to smaller the list of false positive.

less important ones also can be straight, see https://www.openstreetmap.org/?mlat=34.3125&mlon=-103.0430#map=14/34.3125/-103.0430

And going by Poland case many borders have section of straight lines, even ones looking to be more wiggly

Thanks, that is what I feared about areas with low population and no lower level of boundaries. Can we define the land areas to use boundary.xml?

Not sure what kind of effect it would have.

I was thinking about missing lower level boundaries and that these ways will have to be split in (near) future, anyway, but it is a bad idea.

None of cases in Poland would qualify for that, internal divisions are not reaching into sea and end near shore. https://www.openstreetmap.org/#map=16/54.1889/15.5448

comment:8 Changed 3 months ago by skyper

Damn I mixed up waylength and length of segment. My previous comments are quite off the mark.

No objection about ignoring boundaries.

comment:9 Changed 3 months ago by GerdP

So, exclude if either the way has tag boundary=administrative or if is member of a boundary relation with this tag?
Edit: Or can we simply use the maritime=yes tag?

Last edited 3 months ago by GerdP (previous) (diff)

comment:10 Changed 3 months ago by skyper

No maritime=yes is not a good choice. Maybe even all boundary=* are problematic as normal warning.

comment:11 Changed 3 months ago by Klumbumbus

Yes, simply boundary=* should added to the exclusion list.

comment:12 Changed 3 months ago by GerdP

Resolution: fixed
Status: newclosed

In 17382/josm:

fix #20174: Exclude administrative borders, especially national administrative borders from "very long segment"

  • exclude ways with boundary=* from test LongSegment

comment:13 Changed 3 months ago by GerdP

Milestone: 20.12

comment:14 Changed 3 months ago by Klumbumbus

Ah, I forgot about the case when the tags are all on the relations and the way does not have the tags from the ignore list. Was this already handled?

comment:15 Changed 3 months ago by GerdP

No. Do you think we can simply check if a parent relation has the boundary tag? Or should there be another check that the way itself doesn't have tags like e.g. highway=*?

comment:16 Changed 3 months ago by Klumbumbus

I would say the latter, because if there is a highway with a very long segment which is "randomly" in a boundary relation it is still suspicious.

comment:17 Changed 3 months ago by GerdP

Okay, waterway=* and barrier=* would be my next candidates. Maybe also railway=*?

comment:18 Changed 3 months ago by Klumbumbus

Linear waterways have long segments when they go through lakes. (e.g. https://www.openstreetmap.org/way/514055688#map=11/47.6004/9.4112)

comment:19 in reply to:  17 Changed 3 months ago by Klumbumbus

Replying to GerdP:

Okay, waterway=* and barrier=* would be my next candidates. Maybe also railway=*?

Did you mean to remove the warning for them?

comment:20 Changed 3 months ago by GerdP

Well, I wouldn't mind to remove the whole test ;)
I thought about two lists, one as "always check" with highway=*and the other tags from comment:17
and - if not matched - the exclude list plus a check if a parent relation is a type=boundary:

    private boolean ignoreWay(Way w) {
        if (w.hasKey("highway", "railway", "waterway", "barrier"))
            return false;
        return w.hasKey("boundary") || w.hasTag("route", "ferry") || w.hasTag("bay", "fjord")
                || w.hasTag("natural", "strait") || visitedWays.contains(w)
                || w.referrers(Relation.class).anyMatch(Relation::isBoundary);
    }

comment:21 Changed 3 months ago by GerdP

Have to move the clause visitedWays.contains(w)

comment:22 Changed 3 months ago by GerdP

Like this:

    private boolean ignoreWay(Way w) {
        if (visitedWays.contains(w))
            return true;
        if (w.hasKey("highway", "railway", "waterway", "barrier"))
            return false;
        return w.hasKey("boundary") || w.hasTag("route", "ferry") || w.hasTag("bay", "fjord")
                || w.hasTag("natural", "strait")
                || w.referrers(Relation.class).anyMatch(Relation::isBoundary);
    }

Another possible change would be to limit this check on ways with modified or new nodes if this is about unintended changes.

comment:23 Changed 3 months ago by Klumbumbus

I'm not such a big fan of checks only for new/changed objects, because

  • if it is an error then it is always an error, no matter if the object is old or new
  • if the original uploader "shameless" ignores the error it is then hidden for other more careful mappers, which manually run the validator on a dataset

comment:24 Changed 3 months ago by GerdP

OK, I agree. Do you think the enhanced check will work well?

Modify Ticket

Change Properties
Set your email in Preferences
Action
as closed The owner will remain team.
as The resolution will be set.
The resolution will be deleted.

Add Comment


E-mail address and name can be saved in the Preferences.

 
Note: See TracTickets for help on using tickets.