Modify

Opened 3 years ago

Last modified 12 months ago

#4299 new enhancement

Group related objects when in chunked mode

Reported by: Ldp Owned by: team
Priority: normal Component: Core
Version: latest Keywords:
Cc: toby.murray@…

Description (last modified by ToeBee)

When doing large uploads in chunked mode, every chunk is constrain-checked and committed to OSM as it's finished. It still sticks to all nodes before ways before relations. This introduced some things:

  • Nodes are visible in OSM long before the chunk with the accompanying ways is committed. You run the risk that someone else deletes/modifies a node before the way appears in the db.
  • I believe it was mentioned that server-side constraint checking is a bit faster if a way and its nodes are in the same osmChange.

Prevention of the first issue is, in my eyes, more important than sticking to the exact chunk size, if that would simplify calculations.

Attachments (0)

Change History (3)

comment:1 Changed 3 years ago by Ldp

This should also work for non-chunked mode. I've done 50k+ uploads, where the first changeset had 50k nodes, and the second changeset had the remaining nodes and all the ways and relations.

comment:2 Changed 14 months ago by ToeBee

  • Cc toby.murray@… added
  • Description modified (diff)

As someone who has cleaned up messes left behind by JOSM's chunked upload used on huge imports... +1000 on this.

I have deleted a couple hundred thousand nodes from failed imports. Not all of them were from JOSM (one big group was from ArcMap) but an appreciable number were.

Perhpas the logic from things like smarter-sort could be reused?

http://svn.openstreetmap.org/applications/utils/import/bulkupload/smarter-sort.py

comment:3 Changed 12 months ago by bastiK

Ticket #5414 has been marked as a duplicate of this ticket.

Add Comment

Modify Ticket

Change Properties
<Author field>
Action
as new .
as The resolution will be set. Next status will be 'closed'.
to The owner will be changed from team. Next status will be 'new'.
Next status will be 'needinfo'.The owner will change to Ldp
as duplicate The resolution will be set to duplicate. Next status will be 'closed'.The specified ticket will be cross-referenced with this ticket
The owner will be changed from team to anonymous. Next status will be 'assigned'.
Author


E-mail address and user name can be saved in the Preferences.

 
Note: See TracTickets for help on using tickets.