Modify

Opened 15 years ago

Last modified 12 years ago

#4299 new enhancement

Group related objects when in chunked mode

Reported by: Ldp Owned by: team
Priority: normal Milestone:
Component: Core Version: latest
Keywords: Cc: ToeBee

Description (last modified by ToeBee)

When doing large uploads in chunked mode, every chunk is constrain-checked and committed to OSM as it's finished. It still sticks to all nodes before ways before relations. This introduced some things:

  • Nodes are visible in OSM long before the chunk with the accompanying ways is committed. You run the risk that someone else deletes/modifies a node before the way appears in the db.
  • I believe it was mentioned that server-side constraint checking is a bit faster if a way and its nodes are in the same osmChange.

Prevention of the first issue is, in my eyes, more important than sticking to the exact chunk size, if that would simplify calculations.

Attachments (0)

Change History (3)

comment:1 by Ldp, 14 years ago

This should also work for non-chunked mode. I've done 50k+ uploads, where the first changeset had 50k nodes, and the second changeset had the remaining nodes and all the ways and relations.

comment:2 by ToeBee, 12 years ago

Cc: ToeBee added
Description: modified (diff)

As someone who has cleaned up messes left behind by JOSM's chunked upload used on huge imports... +1000 on this.

I have deleted a couple hundred thousand nodes from failed imports. Not all of them were from JOSM (one big group was from ArcMap) but an appreciable number were.

Perhpas the logic from things like smarter-sort could be reused?

http://svn.openstreetmap.org/applications/utils/import/bulkupload/smarter-sort.py

comment:3 by bastiK, 12 years ago

Ticket #5414 has been marked as a duplicate of this ticket.

Modify Ticket

Change Properties
Set your email in Preferences
Action
as new The owner will remain team.
as The resolution will be set. Next status will be 'closed'.
to The owner will be changed from team to the specified user.
Next status will be 'needinfo'. The owner will be changed from team to Ldp.
as duplicate The resolution will be set to duplicate. Next status will be 'closed'. The specified ticket will be cross-referenced with this ticket.
The owner will be changed from team to anonymous. Next status will be 'assigned'.

Add Comment


E-mail address and name can be saved in the Preferences .
 
Note: See TracTickets for help on using tickets.