Modify

Opened 13 years ago

Closed 8 years ago

#6536 closed defect (fixed)

CrossingWays causes OutOfMemory (was: josm runs out of memory after clicking "upload", upload is impossible)

Reported by: Cobra Owned by: team
Priority: critical Milestone:
Component: Core validator Version: latest
Keywords: out of memory crossing_way Cc:

Description (last modified by skyper)

I can't reproduce this reliably yet, but it happened several times now.

Immediately after clicking the upload button, josm consumes as much cpu and ram as possible until a "out of memory" message pops up. The upload dialog doesn't open but josm stays usable. I have to save the osm layer and restart josm to upload it.

I'll try to investigate this further but this may take some time.

Repository Root: http://josm.openstreetmap.de/svn
Build-Date: 2011-07-04 01:31:38
Last Changed Author: stoecker
Revision: 4201
Repository UUID: 0c6e7542-c601-0410-84e7-c038aed88b3b
URL: http://josm.openstreetmap.de/svn/trunk
Last Changed Date: 2011-07-04 00:07:05 +0200 (Mon, 04 Jul 2011)
Last Changed Rev: 4201

Identification: JOSM/1.5 (4201 en)
Memory Usage: 647 MB / 910 MB (564 MB allocated, but free)
Java version: 1.6.0_24, Sun Microsystems Inc., Java HotSpot(TM) 64-Bit Server VM
Operating system: Linux
Dataset consistency test: No problems found

Plugin: ImproveWayAccuracy (26174)
Plugin: PicLayer (26174)
Plugin: alignways (26174)
Plugin: contourmerge (25626)
Plugin: measurement (26174)
Plugin: openstreetbugs (26174)
Plugin: openvisible (26174)
Plugin: proj4j (26174)
Plugin: reltoolbox (26174)
Plugin: reverter (26174)
Plugin: terracer (26174)
Plugin: undelete (26242)
Plugin: utilsplugin2 (26174)

Attachments (1)

CrossingWays_StressGrid.osm (90.1 KB ) - added by mrwojo 12 years ago.

Download all attachments as: .zip

Change History (13)

comment:1 by jttt, 13 years ago

Please run java with -XX:+HeapDumpOnOutOfMemoryError parameter. Next time OutOfMemory happens, java will generate heap dump that we can use to analyze what happened. You should use OAuth instead of plain text password, otherwise it's possible to get your password from the dump.

comment:2 by Cobra, 13 years ago

I finally managed to reproduce it again.

Now I've got a 1.3GB large heap dump file. I also have the file I loaded before trying to upload.

Context: I was using josm for some time, saved my changes to a file, deleted all layers and loaded an older file. When I tried to upload changes in this file, josm went unresponsive with 100% CPU load, used lots of RAM and finally displayed a dialog labeled "error" but blank like josm's main window.

comment:3 by jttt, 13 years ago

Can you upload the file somewhere? It shouldn't be too big zipped.

comment:5 by jttt, 13 years ago

OutOfMemory is caused by ineffective algorithm used for CrossingWays validator test. I'll replace it either with some proper line segment intersection algorithm like Bentley Ottman or the naive algorithm speeded up by QuadBuckets.

comment:6 by skyper, 12 years ago

Component: CoreCore validator
Description: modified (diff)
Keywords: out of memory added

comment:7 by simon04, 12 years ago

Summary: josm runs out of memory after clicking "upload", upload is impossibleCrossingWays causes OutOfMemory (was: josm runs out of memory after clicking "upload", upload is impossible)

@jttt: Did you start with the implementation? Just asking to avoid potential duplicate work on the same issue.

comment:8 by jttt, 12 years ago

Nope, sorry, I've completely forgot about this. If you want, please go ahead and fix it. As far as I remember, problem was that it parcels checked area into cells with predefined size, so if area is big then it takes huge amount of memory. Also used structures were unnecessarily memory hungry, like caching thinks that can be easily calculated on demand.

If you want to use Bentley Ottmann, here is sample library. But it probably shouldn't be used as is because it's really academical (for example it's using arbitrary precision numbers which will probably really slow it down and also eat lots of memory)

by mrwojo, 12 years ago

Attachment: CrossingWays_StressGrid.osm added

comment:9 by skyper, 10 years ago

Keywords: crossing_way added

comment:10 by simon04, 10 years ago

In 6580/josm:

see #6536 - Refactor crossing way test in order to decrease memory footprint

Drop data structure ExtendedSegment and obtain needed values on demand
from the way tags.

comment:11 by GerdP, 8 years ago

I think this ticket should be closed, the memory footprint for the test is (now) rather small.

comment:12 by Don-vip, 8 years ago

Resolution: fixed
Status: newclosed

Modify Ticket

Change Properties
Set your email in Preferences
Action
as closed The owner will remain team.
as The resolution will be set.
The resolution will be deleted. Next status will be 'reopened'.

Add Comment


E-mail address and name can be saved in the Preferences .
 
Note: See TracTickets for help on using tickets.