Opened 16 years ago
Last modified 13 years ago
#4299 new enhancement
Group related objects when in chunked mode — at Version 2
| Reported by: | Ldp | Owned by: | team |
|---|---|---|---|
| Priority: | normal | Milestone: | |
| Component: | Core | Version: | latest |
| Keywords: | Cc: | ToeBee |
Description (last modified by )
When doing large uploads in chunked mode, every chunk is constrain-checked and committed to OSM as it's finished. It still sticks to all nodes before ways before relations. This introduced some things:
- Nodes are visible in OSM long before the chunk with the accompanying ways is committed. You run the risk that someone else deletes/modifies a node before the way appears in the db.
- I believe it was mentioned that server-side constraint checking is a bit faster if a way and its nodes are in the same osmChange.
Prevention of the first issue is, in my eyes, more important than sticking to the exact chunk size, if that would simplify calculations.
Change History (2)
comment:1 by , 15 years ago
comment:2 by , 13 years ago
| Cc: | added |
|---|---|
| Description: | modified (diff) |
As someone who has cleaned up messes left behind by JOSM's chunked upload used on huge imports... +1000 on this.
I have deleted a couple hundred thousand nodes from failed imports. Not all of them were from JOSM (one big group was from ArcMap) but an appreciable number were.
Perhpas the logic from things like smarter-sort could be reused?
http://svn.openstreetmap.org/applications/utils/import/bulkupload/smarter-sort.py
This should also work for non-chunked mode. I've done 50k+ uploads, where the first changeset had 50k nodes, and the second changeset had the remaining nodes and all the ways and relations.