Modify

Opened 10 years ago

Closed 10 years ago

#4554 closed defect (fixed)

Fixing duplicate nodes takes increasingly longer when more nodes are being fixed simultaneously

Reported by: fsteggink Owned by: team
Priority: normal Milestone:
Component: Core validator Version: latest
Keywords: duplicate nodes fix Cc:

Description

After running the validator, the larger the number of duplicate nodes one wants to fix simultaneously, the more time it takes. The time is even more for each individual node. I guess that the performance degradation is at least quadratic, possibly more. I find that odd, since the detection of duplicate nodes itself is much quicker, and there are much more nodes to test.

So, I'm currently fixing duplicate nodes on smaller sets, by selecting only a part of the duplicate nodes. (Run validator without selection, select duplicate nodes, unselect portion of it, run validator again, fix.) This is faster, but quite annoying to do, especially when there are thousands of nodes to be fixed.

JOSM version: 2971

Attachments (0)

Change History (5)

comment:1 Changed 10 years ago by Ldp

Fixing stays relatively fast when the nodes to be fixed are not selected. When they *are* selected, the fixing slows down the more nodes there are to fix, and this slowdown happens quite soon in the fixing stage.

comment:2 Changed 10 years ago by fsteggink

To be clear, when importing some data for Canvec, it was necessary to merge tens of thousands of duplicate nodes, since all feature types are in separate files. When only a few thousand nodes need to be deduplicated, the performance is OK.

comment:3 Changed 10 years ago by bastiK

(In [3142]) see #4554 - Fixing duplicate nodes takes increasingly longer

comment:4 Changed 10 years ago by bastiK

Please check if it is now fast enough. If not, I can make it even faster, but I'd rather not do it, unless it's really necessary.

There is the related problem, that mass-fixing duplicate nodes consumes a lot of memory. When it hits the limit, it gets slower and slower. So please check you have enough memory when doing your tests.

comment:5 Changed 10 years ago by bastiK

Resolution: fixed
Status: newclosed

You need to update both JOSM and Validator plugin.

(See [o20546], [3143])

Modify Ticket

Change Properties
Set your email in Preferences
Action
as closed The owner will remain team.
as The resolution will be set.
The resolution will be deleted.

Add Comment


E-mail address and name can be saved in the Preferences.

 
Note: See TracTickets for help on using tickets.