Modify

Opened 9 months ago

Closed 2 months ago

#17453 closed defect (fixed)

[PATCH] GeoJSON export error

Reported by: BrackoNe Owned by: team
Priority: normal Milestone: 19.10
Component: Core geojson Version:
Keywords: polygon export Cc:

Description (last modified by Don-vip)

Hi all,

We have a problem in the process of exporting GeoJSON files using geojson (version 116).

If you try to export any polygon/area (closed way) to geojson it saves to LineString instead to Polygon. If I open GeoJSON file which I have exported from QGIS and it has following part:

{"type":"Feature","geometry":{"type":"Polygon","coordinates":[]...}

it works well.
However, if you try to re-save the same file from JOSM, it will automatically save to LineString.

I'm trying to fix some lakes and forests, but as soon as I export geometry, it messes everything because this conversion.

Can you fix this issue or tell me how to bypass this problem?

Thanks in advance,
Nemanja

Please provide any additional information below. Attach a screenshot if possible.

URL:https://josm.openstreetmap.de/svn/trunk
Repository:UUID: 0c6e7542-c601-0410-84e7-c038aed88b3b
Last:Changed Date: 2018-12-31 15:09:58 +0100 (Mon, 31 Dec 2018)
Build-Date:2018-12-31 14:24:10
Revision:14620
Relative:URL: ^/trunk

Identification: JOSM/1.5 (14620 en) Windows 10 64-Bit
OS Build number: Windows 10 Pro 1803 (17134)
Memory Usage: 1581 MB / 1820 MB (654 MB allocated, but free)
Java version: 1.8.0_144-b01, Oracle Corporation, Java HotSpot(TM) 64-Bit Server VM
Screen: \Display0 2560x1440, \Display1 2560x1440
Maximum Screen Size: 2560x1440
Dataset consistency test: No problems found

Plugins:
+ FastDraw (34867)
+ Mapillary (1.5.18)
+ RoadSigns (34867)
+ apache-commons (34506)
+ apache-http (34632)
+ buildings_tools (34904)
+ ejml (34389)
+ geojson (116)
+ geotools (34513)
+ imagery_offset_db (34867)
+ jaxb (34678)
+ jna (34867)
+ jogl (1.2.2)
+ jts (34524)
+ kendzi3d (1.0.205)
+ kendzi3d-resources (0.0.2)
+ log4j (34527)
+ mbtiles (a3e491d)
+ measurement (34867)
+ microsoft (1535663476)
+ opendata (34911)
+ splinex (34901)
+ tageditor (34867)
+ turnlanes (34678)
+ turnlanes-tagging (280)
+ turnrestrictions (34867)
+ utilsplugin2 (34915)

Tagging presets:
+ https://raw.githubusercontent.com/yopaseopor/traffic_signs_preset_JOSM/master/RS.zip

Map paint styles:
+ https://josm.openstreetmap.de/josmfile?page=Styles/Lane_and_Road_Attributes&zip=1
- https://josm.openstreetmap.de/josmfile?page=Styles/ParkingLanes&zip=1
- https://josm.openstreetmap.de/josmfile?page=Styles/Coloured_Streets&zip=1
+ https://pasharm.github.io/New_basic_style_for_JOSM/New_basic_style.mapcss
- %UserProfile%\Desktop\TrafficSigns_edited2.zip
+ %UserProfile%\Desktop\beta_style_josm-master.zip
+ https://raw.githubusercontent.com/yopaseopor/traffic_signs_style_JOSM/master/Styles_Traffic_signs_EUR_OR.zip
+ %UserProfile%\Desktop\Microsoft-TurnRestriction.mapcss

Last errors/warnings:
- E: Error while reading json file!
- E: java.lang.ClassCastException: Cannot cast org.glassfish.json.JsonNumberImpl$JsonBigDecimalNumber to javax.json.JsonArray
- E: Error while reading json file!
- E: java.lang.ClassCastException: Cannot cast org.glassfish.json.JsonNumberImpl$JsonBigDecimalNumber to javax.json.JsonArray
- E: Error while reading json file!
- E: java.lang.ClassCastException: Cannot cast org.glassfish.json.JsonNumberImpl$JsonBigDecimalNumber to javax.json.JsonArray
- E: Error while reading json file!
- E: java.lang.ClassCastException: Cannot cast org.glassfish.json.JsonNumberImpl$JsonBigDecimalNumber to javax.json.JsonArray
- E: Error while reading json file!
- E: java.lang.ClassCastException: Cannot cast org.glassfish.json.JsonNumberImpl$JsonBigDecimalNumber to javax.json.JsonArray

Attachments (5)

Test.geojson (269 bytes) - added by BrackoNe 9 months ago.
Example of geojson file with Polygon entity.
json1.osm (1.4 KB) - added by GerdP 6 months ago.
simple multipolygon for export/import test
17453.patch (5.5 KB) - added by GerdP 6 months ago.
simple solution, not trying to produce output similar to osmtogeojson
17453-v2.patch (5.5 KB) - added by GerdP 6 months ago.
areas_fixes.osm (57.1 KB) - added by Zverikk 3 months ago.
Example file that gets saved as a series of linestrings

Download all attachments as: .zip

Change History (31)

Changed 9 months ago by BrackoNe

Attachment: Test.geojson added

Example of geojson file with Polygon entity.

comment:1 Changed 9 months ago by BrackoNe

Hi,

Can you provide any update on this?

Thanks,
Nemanja

comment:2 Changed 6 months ago by GerdP

The code in JOSM checks if the way is closed AND has an area style attribute:

                if (w.isClosed() && ElemStyles.hasAreaElemStyle(w, false)) {
                    final JsonArrayBuilder container = Json.createArrayBuilder().add(array);
                    geomObj.add("type", "Polygon");
                    geomObj.add("coordinates", container);
                } else {
                    geomObj.add("type", "LineString");
                    geomObj.add("coordinates", array);
                }

An area-style attribute would be a tag like natural=water or landuse=forest etc.
No idea if that restriction makes sense?

Last edited 6 months ago by GerdP (previous) (diff)

Changed 6 months ago by GerdP

Attachment: json1.osm added

simple multipolygon for export/import test

comment:3 Changed 6 months ago by GerdP

I think the output for multipolygons is also wrong. At least when I save attached josm1.osm as json and load it again the
result is very different. I guess the format we use doesn't even allow to get a similar result.
In #7307 osmtogeojson is mentioned and overpass also uses it.
I wonder why we don't produce an output similar to osmtogeojson? Or maybe we can use it directly?

comment:4 Changed 6 months ago by Don-vip

Use it directly: no. Oracle has deprecated nashorn and seem in a hurry to remove in favour of their "graal" vm. Not a good time to add another JavaScript library as we will probably need to recode the ones we currently use in plain Java...
Update our code to match its behaviour: yes

comment:5 Changed 6 months ago by BrackoNe

Can I help you somehow or do you need any additional information or files?

This is something that makes serious issues to us.

Thanks,
Nemanja

comment:6 Changed 6 months ago by Don-vip

The issue is clear enough, thank you :)

comment:7 Changed 6 months ago by anonymous

I'm here if you need anything else. Thanks a lot for picking this up.

Best Regards,
Nemanja

Changed 6 months ago by GerdP

Attachment: 17453.patch added

simple solution, not trying to produce output similar to osmtogeojson

comment:8 Changed 6 months ago by GerdP

I fear I still don't understand enough about the purpose of the geojson format, so this patch probably creates new problems.
My thoughts:
If I got that right the unpatched code tries to avoid the creation of a Polygon for a e.g. a closed way with highway=service, and it would be right to do so.
A closed way without any tags is probably better written as Polygon unless it is also a member of a multipolygon.
Problems starts with the multipolygon test file \josm\core\test\data\multipolygon.osm in which all ways have a ref tag.
The patch suppresses the output of the ways as line strings, but that is wrong because the information about the ref tag is lost.
The website http://tyrasd.github.io/osmtogeojson/ writes the polygons as Linestring objects, and it also does that with a closed way without any tags, so it seems that the current code in JOSM is not that wrong.

@Nemanja: Maybe you can change the data so that polygons have a tag which is recognized as an area? An alternative would be to change the importer in the plugin so that it adds a special tag to the ways to indicate the original type
and recognize that tag in the exporter.

comment:9 Changed 6 months ago by anonymous

I want to share one of many processes why do we need GeoJsons.
As you may know, Microsoft have published 128 mil. buildings in US.
We had many estimations before we have published. Our developers exporting these data in GeoJson or WKT.
WKT is not supported at all by JOSM. Even WKT is better for us and it would save us a lot of time, GeoJson is easy enough to manipulate with.
The problem is that we are not allowed to add any tags. So if first and last coordinates are the same within some entity, it is polygon - otherwise this is a way. If there is just one coordinate - it's node.

comment:10 Changed 6 months ago by GerdP

You write that you are "not allowed to add any tags". Does that mean that also the proposed alternative solution is not suitable?

comment:11 Changed 6 months ago by BrackoNe

Yes, you are right. Proposed alternative solution is not suitable because I can't add any tags because of 2 reasons:

  • It's not allowed by our developers;
  • Imagine that you have one city with just 100k of buildings. I would spend more time to add these tags instead of fixing exact issue.

I am not sure why is it big issue if first_coordinate == last_coordinate it is polygon? If first_coordinate != last_coordinate then it is line.

comment:12 in reply to:  11 Changed 6 months ago by GerdP

Replying to brackone@…:

Yes, you are right. Proposed alternative solution is not suitable because I can't add any tags because of 2 reasons:

  • It's not allowed by our developers;
  • Imagine that you have one city with just 100k of buildings. I would spend more time to add these tags instead of fixing exact issue.

I talked about the alternative that the program adds this tag when loading the data and - of course - removes it when storing the data. Anyway, I don't know if this is even possible.

I am not sure why is it big issue if first_coordinate == last_coordinate it is polygon? If first_coordinate != last_coordinate then it is line.

In OSM, not all closed ways are polygons. Think of a junction=roundabout.
In your special case (way without any tags) I think it would be OK to assume that a closed way is a polygon.

comment:13 Changed 6 months ago by BrackoNe

OSM concept of keeping everything in one layer (not separate points, lines and polygons) are cosing this problem what you said - junction is closed way, but not an polygon. I can see the resolution only as an option to check if you want to clear all attributes and to save the data as they are (if closed then it's polygon) or if it is not checked, then to try to resove a nature of the data.

What do you say about that?

comment:14 Changed 6 months ago by GerdP

Well, that would be a very special solution. I still don't understand why you use JOSM when you don't want to upload the data to OSM and also don't want to change it? In what situation do you actually save the data in geojson format?

comment:15 Changed 6 months ago by BrackoNe

When you have a lot of editors and they are used to use JOSM, and there are building tool for example which makes life easier, it is very good idea to use JOSM for such "light" editing instead of using e.g. QGIS.

Hope it helps to understand our process better.

Changed 6 months ago by GerdP

Attachment: 17453-v2.patch added

comment:16 Changed 6 months ago by GerdP

Ok, so your editors modify the data that is in geojson format and the modified data is later processed by another program?

Anyway, the attached 2nd version of the patch is less destructive as it requires a new preference setting
geojson.export.untagged-closed-is-polygon=true
So, if you have a closed untagged way and the setting is found in preferences the way is exported as Polygon instead of LineString. Besides that the patch suppresses the output of untagged ways which are written as members of multipolygons.

comment:17 in reply to:  16 Changed 6 months ago by BrackoNe

Replying to GerdP:

Ok, so your editors modify the data that is in geojson format and the modified data is later processed by another program?

In most cases yes, but not always a case. For example it is possible to get building predictions, and then to sighty improve (if necessary) before upload.

Anyway, the attached 2nd version of the patch is less destructive as it requires a new preference setting
geojson.export.untagged-closed-is-polygon=true
So, if you have a closed untagged way and the setting is found in preferences the way is exported as Polygon instead of LineString. Besides that the patch suppresses the output of untagged ways which are written as members of multipolygons.

I think this will work for us as well.

Thanks!

PS. Can I try this patch in some night build or should wait few more days?

comment:18 Changed 6 months ago by GerdP

Well, if you cannot compile JOSM you may have to wait until someone else commits this patch. I'll not have time for programming for some weeks soon and I don't want to mess up something in JOSM now in a field that I don't fully understand.
Compiling JOSM is quite easy, see https://josm.openstreetmap.de/wiki/Source%20code#CompilingusingAnt

comment:19 Changed 6 months ago by anonymous

Thanks for clarification!

comment:20 Changed 3 months ago by Zverikk

I have stumbled upon this bug recently. When I open a GeoJSON in JOSM, edit it and save it back, all polygons get converted into linestrings. This is very frustrating. I had to write a script to fix the output.

Could somebody remove the requirement for a area tags, and just save untagged polygons as polygons? See the attached osm file for an example.

Changed 3 months ago by Zverikk

Attachment: areas_fixes.osm added

Example file that gets saved as a series of linestrings

comment:21 Changed 2 months ago by Don-vip

Description: modified (diff)
Version: tested

comment:22 Changed 2 months ago by Don-vip

Component: Plugin geojsonCore
Keywords: geojson added; issue removed
Owner: changed from Larry0ua to team
Summary: GeoJSON export error[PATCH] GeoJSON export error

comment:23 Changed 2 months ago by Don-vip

Milestone: 19.10

@Gerd: the patch looks fine, can you please commit it?

comment:24 Changed 2 months ago by Don-vip

Component: CoreCore geojson

comment:25 Changed 2 months ago by Don-vip

Keywords: geojson removed

comment:26 Changed 2 months ago by GerdP

Resolution: fixed
Status: newclosed

In 15429/josm:

fix #17453: GeoJSON export error (17453-v2.patch)

Modify Ticket

Change Properties
Set your email in Preferences
Action
as closed The owner will remain team.
as The resolution will be set.
The resolution will be deleted.

Add Comment


E-mail address and name can be saved in the Preferences.

 
Note: See TracTickets for help on using tickets.