Opened 3 years ago
Last modified 12 months ago
#4299 new enhancement
Group related objects when in chunked mode
| Reported by: | Ldp | Owned by: | team |
|---|---|---|---|
| Priority: | normal | Component: | Core |
| Version: | latest | Keywords: | |
| Cc: | toby.murray@… |
Description (last modified by ToeBee)
When doing large uploads in chunked mode, every chunk is constrain-checked and committed to OSM as it's finished. It still sticks to all nodes before ways before relations. This introduced some things:
- Nodes are visible in OSM long before the chunk with the accompanying ways is committed. You run the risk that someone else deletes/modifies a node before the way appears in the db.
- I believe it was mentioned that server-side constraint checking is a bit faster if a way and its nodes are in the same osmChange.
Prevention of the first issue is, in my eyes, more important than sticking to the exact chunk size, if that would simplify calculations.
Attachments (0)
Change History (3)
comment:1 Changed 3 years ago by Ldp
comment:2 Changed 14 months ago by ToeBee
- Cc toby.murray@… added
- Description modified (diff)
As someone who has cleaned up messes left behind by JOSM's chunked upload used on huge imports... +1000 on this.
I have deleted a couple hundred thousand nodes from failed imports. Not all of them were from JOSM (one big group was from ArcMap) but an appreciable number were.
Perhpas the logic from things like smarter-sort could be reused?
http://svn.openstreetmap.org/applications/utils/import/bulkupload/smarter-sort.py
comment:3 Changed 12 months ago by bastiK
Ticket #5414 has been marked as a duplicate of this ticket.



This should also work for non-chunked mode. I've done 50k+ uploads, where the first changeset had 50k nodes, and the second changeset had the remaining nodes and all the ways and relations.