Modify

Opened 4 years ago

Closed 4 years ago

Last modified 4 years ago

#13907 closed defect (fixed)

[Patch] Memory Error opening big OSM file

Reported by: marcoahp@… Owned by: team
Priority: normal Milestone: 16.10
Component: Core Version: latest
Keywords: memory error performance regression Cc:

Description

What steps will reproduce the problem?

  1. Open a big OSM file, for example spain-latest.osm from geofabrik.de

What is the expected result?

Open the data

What happens instead?

Memory error.

Please provide any additional information below. Attach a screenshot if possible.

I've tested with -Xms2048m parameter but with the same result

URL:http://josm.openstreetmap.de/svn/trunk
Repository:UUID: 0c6e7542-c601-0410-84e7-c038aed88b3b
Last:Changed Date: 2016-09-06 00:16:07 +0200 (Tue, 06 Sep 2016)
Build-Date:2016-09-05 22:21:00
Revision:10966
Relative:URL: ^/trunk

Identification: JOSM/1.5 (10966 es) Windows 10 64-Bit
Memory Usage: 247 MB / 247 MB (124 MB allocated, but free)
Java version: 1.8.0_111-b14, Oracle Corporation, Java HotSpot(TM) Client VM
Screen: \Display0 1920x1080
Maximum Screen Size: 1920x1080
VM arguments: [-Djava.security.manager, -Djava.security.policy=file:<java.home>\lib\security\javaws.policy, -DtrustProxy=true, -Djnlpx.home=<java.home>\bin, -Djnlpx.origFilenameArg=%UserProfile%\AppData\Local\Microsoft\Windows\INetCache\IE\IE1IG3IQ\josm.jnlp, -Djnlpx.remove=true, -Djava.util.Arrays.useLegacyMergeSort=true, -Djnlpx.splashport=60863, -Djnlpx.jvm=<java.home>\bin\javaw.exe, -Djnlpx.vmargs=LURqYXZhLnV0aWwuQXJyYXlzLnVzZUxlZ2FjeU1lcmdlU29ydD10cnVlAA==]

Attachments (1)

StyleCache_intern.patch (562 bytes) - added by GerdP 4 years ago.

Download all attachments as: .zip

Change History (11)

comment:1 Changed 4 years ago by GerdP

You'll probably need far more memory (and a lot of patience) for such a large file. My experience is that JOSM can handle an osm.pbf file with 15 MB, but many actions might be very slow.

comment:2 Changed 4 years ago by bastiK

Resolution: invalid
Status: newclosed

We try to optimize the JOSM so that big extracts can be opened and edited. There has to be a limit somewhere and the Spain extract is much too big for 2 GB of memory. Please give more details if you think JOSM is wasting memory or you think it is a regression.

Changed 4 years ago by GerdP

Attachment: StyleCache_intern.patch added

comment:3 Changed 4 years ago by GerdP

Please review attached small patch.
I think either this should be used or remove the call of intern().

comment:4 Changed 4 years ago by GerdP

Some numbers for a file with ~ 1.000.000 nodes:
The unpatched version allocates ~ 772 MiB (~995.000 StyleCache instances),
the patched version allocates ~ 335 MiB (351 StyleCache instances)

I think the code for StyleCache doesn't work since r9284.

comment:5 Changed 4 years ago by GerdP

Resolution: invalid
Status: closedreopened
Summary: Memory Error opening big OSM file[Patch] Memory Error opening big OSM file

comment:6 Changed 4 years ago by Don-vip

Keywords: performance regression added
Milestone: 16.10

comment:7 Changed 4 years ago by Don-vip

Resolution: fixed
Status: reopenedclosed

In 11212/josm:

fix #13907 - StyleCache broken (regression from r9284, patch by Gerd Petermann)

comment:8 Changed 4 years ago by Don-vip

In 11214/josm:

see #13907 - fix unit test

comment:9 Changed 4 years ago by bastiK

In 11215/josm:

see #13907 - add unit test for problem fixed in [11212]

comment:10 in reply to:  4 Changed 4 years ago by bastiK

Replying to Gerd Petermann <gpetermann_muenchen@…>:

Some numbers for a file with ~ 1.000.000 nodes:
The unpatched version allocates ~ 772 MiB (~995.000 StyleCache instances),
the patched version allocates ~ 335 MiB (351 StyleCache instances)

I think the code for StyleCache doesn't work since r9284.

Great find! A bit embarrassing not to notice this before ...

Modify Ticket

Change Properties
Set your email in Preferences
Action
as closed The owner will remain team.
as The resolution will be set.
The resolution will be deleted.

Add Comment


E-mail address and name can be saved in the Preferences.

 
Note: See TracTickets for help on using tickets.