Modify

Opened 12 years ago

Closed 12 years ago

#4101 closed defect (fixed)

thumbnails for georeferenced pictures: not really needed, slow, no caching

Reported by: malenki Owned by: team
Priority: normal Milestone:
Component: Core Version: latest
Keywords: Cc: malenki

Description

While builting the AgPifoJ-plugin into Josm a thumbnail-generator was also coded. Sure it was meant good, but it has some flaws:
*It slows down the system quite a lot. After loading ~800 pictures the generating of the thumbnails one core of my C2D E6600 goes full blast with the java-engine for seven minutes. Will be really bad when I'd try to use this on my Atom-Netbook I bet. (I won't try and will stuck there with some old josm for some time)
*The thumbnails aren't cached so one has the joy to see the thumbs being generated every time he loads images into josm.
*The thumbnails are really small. Maybe somebody likes to use such tiny images for the fun of it, but I don't need them really (and don't want bigger ones).

Desired behaviour: being able to switch off thumbnails
Additionally desired: caching of thumbnails since otherwise people using thumbs would have to generate them anew every time the load the same images

Attachments (0)

Change History (11)

comment:1 Changed 12 years ago by malenki

Cc: malenki added
Version: latest

I just saw: the active thumbnail (the one which is right now shown) isn't marked by hilighting or such but the up to now standardly used image-icon is displayed in red in front of the thumbnail. Minor bug, fits here.

Josm version 2574

comment:2 Changed 12 years ago by bastiK

It was added recently and it's work in progress. I agree with you on each point. I would like to have thumbs off by default in the end. (But put a fairly visible option to the loading dialog.)

Could you please check the memory usage over time when loading all these thumbs? It should be relatively constant, on a low level.

comment:3 Changed 12 years ago by malenki

In short: Memory usage grows constantly all the time while generating thumbs and remains at the reached high level

In long:
Opened 909 georeferenced images
on Netbook Acer Aspire One ZG5 with 8GB flashdrive, Atom N270@1,6GHz 1,5GB RAM, pictures being loaded from NAS:
Josm without thumbnails:
virgin state:
3950 59.8 3.5 1.218.540 54420 pts/1 Sl+ 16:
20 0:13 java -jar -Xmx1024M -Dsun.java2d.opengl=true /home/nadar/bin/josm-archive/josm-2485.jar

909 pictures loaded:
3950 17.4 4.0 1.219.672 61924 pts/1 Sl+ 16:
20 0:25 java -jar -Xmx1024M -Dsun.java2d.opengl=true /home/nadar/bin/josm-archive/josm-2485.jar

Josm with thumbnails:
virgin state:
3710 21.1 4.4 1.223.980 68660 pts/1 Sl 15:
44 0:22 java -jar -Xmx1024M -Dsun.java2d.opengl=true /home/nadar/bin/josm-archive/josm-2578.jar

909 pictures loaded and thumbnails generated:
3710 76.2 8.6 1.227.364 133.120 pts/1 Sl 15:
44 20:05 java -jar -Xmx1024M -Dsun.java2d.opengl=true /home/nadar/bin/josm-archive/josm-2578.jar

on my big box with Core2Duo E6600 2,4GHz, 4GB RAM:
Josm without thumbnails:
virgin state:
8438 63.2 3.7 1.286.272 136.080 pts/4 Sl+ 16:
40 0:09 java -jar -Xmx1024M -Dsun.java2d.opengl=true /home/nadar/bin/josm-archive/josm-2485.jar

909 pictures loaded:
8438 31.1 4.4 1.286.432 160.764 pts/4 Sl+ 16:
40 0:16 java -jar -Xmx1024M -Dsun.java2d.opengl=true /home/nadar/bin/josm-archive/josm-2485.jar

Josm with thumbnails:
virgin state
8647 13.7 3.7 1.288.956 135.720 pts/4 Sl+ 16:
45 0:08 java -jar -Xmx1024M -Dsun.java2d.opengl=true /home/nadar/bin/josm-archive/josm-2578.jar

loaded a small gpx:
8647 13.0 4.1 1.289.280 149.544 pts/4 Sl+ 16:
45 0:11 java -jar -Xmx1024M -Dsun.java2d.opengl=true /home/nadar/bin/josm-archive/josm-2578.jar

909 pictures loaded and thumbnails generated:
8647 77.9 30.5 1.306.336 1.110.176 pts/4 Sl+ 16:
45 6:57 java -jar -Xmx1024M -Dsun.java2d.opengl=true /home/nadar/bin/josm-archive/josm-2578.jar

comment:4 Changed 12 years ago by bastiK

Owner: changed from team to jttt

Thanks, how large are your images?

Unfortunately Java seems to be quite greedy and keeps 1 GB of memory allocated even when garbage collection would shrink this to about 30 MB. Can you run again with -Xmx100m and just see if you get OutOfMemory error? If so, please try -Xmx200m.

(I already did on my machine and it works, but I like to have a test on another system.)

@jttt:
To me it seems crude to have 1 GB of memory with trash and not do something about it. Do you know any solution? I tried System.gc(). Most of the times it works, but occasionally it is just ignored.

comment:5 Changed 12 years ago by jttt

System.gc() is just a hint. If Java think that it has enough memory and calling gc will hurt performance, it might just ignore the request. Also I didn't check how geoimage code works now, but it used to cache images (using SoftReference), so the memory wasn't just trash but preloaded images.

@malenki: I'm not sure if I'm interpretting the results correctly but it seems to me that on netbook is memory usage alright, problem is only on big box. That might be because C2D with 4GB ram and linux is considered as server by Java. In server mode, JOSM runs faster but also Java doesn't care that much about memory usage and JOSM startup might be slower. You can try parameter -client.

Also if you want to play with this more, you can use jconsole tool (comes with jdk). It will show you heap size and allow to do garbage collection.

comment:6 in reply to:  4 Changed 12 years ago by malenki

Replying to bastiK:

Thanks, how large are your images?

6MP, max 1.5 MB

Unfortunately Java seems to be quite greedy and keeps 1 GB of memory allocated even when garbage collection would shrink this to about 30 MB. Can you run again with -Xmx100m and just see if you get OutOfMemory error? If so, please try -Xmx200m.

(I already did on my machine and it works, but I like to have a test on another system.)

Strangely Josm isn't in the mood to generate thumbnails at all atm. 2578 and the latest 2591 both won't to...

comment:7 Changed 12 years ago by malenki

Ermm. A 0-Byte-jpg stopped Josm from generating thumbnails. (note-to-self: nice workaround)

with 100 MB:
9022 21.2 5.7 342.940 207.432 pts/8 Sl+ 17:51 0:38 java -jar -Xmx100M -Dsun.java2d.opengl=true /home/nadar/bin/josm-archive/josm-2578.jar

9022 73.2 5.7 342.940 207.888 pts/8 Sl+ 17:51 7:46 java -jar -Xmx100M -Dsun.java2d.opengl=true /home/nadar/bin/josm-archive/josm-2578.jar

With -client memory usage varies:
thats the highest value I found:
9462 99.6 19.7 2.330.472 718.060 pts/8 Sl+ 18:02 4:19 java -jar -Xmx2048M -client -Dsun.java2d.opengl=true /home/nadar/bin/josm-archive/josm-2578.jar

a moment later it went down again to a moderate value:
9462 99.3 4.1 2.330.472 149.456 pts/8 Sl+ 18:02 4:19 java -jar -Xmx2048M -client -Dsun.java2d.opengl=true /home/nadar/bin/josm-archive/josm-2578.jar

After finishing it looked like this:
9462 91.1 12.3 2.330.472 447.596 pts/8 Sl+ 18:02 6:48 java -jar -Xmx2048M -client -Dsun.java2d.opengl=true /home/nadar/bin/josm-archive/josm-2578.jar

I don't know much about heap size, garbage collection and such. I'd like to map instead of filing bugs and making benchmarks. Yesterday I didn't do much more then drag around streets a little...

comment:8 Changed 12 years ago by bastiK

(In [2592]) geoimage: make thumbnails optional + cosmetics (see #4101)

comment:9 in reply to:  7 Changed 12 years ago by bastiK

Owner: changed from jttt to team

Replying to malenki:

Ermm. A 0-Byte-jpg stopped Josm from generating thumbnails. (note-to-self: nice workaround)

ok, new error


I don't know much about heap size, garbage collection and such. I'd like to map instead of filing bugs and making benchmarks. Yesterday I didn't do much more then drag around streets a little...

Please open tickets if you find bugs, I won't bother you with this any more. (I have all I wanted to know :) )
Happy mapping!

comment:10 Changed 12 years ago by bastiK

(In [2606]) geoimage: add thumbnail caching (fixes #4116, see #4101)

comment:11 Changed 12 years ago by bastiK

Resolution: fixed
Status: newclosed

(In [2617]) geoimage: improved thumbnails (closes #4101)

Modify Ticket

Change Properties
Set your email in Preferences
Action
as closed The owner will remain team.
as The resolution will be set.
The resolution will be deleted.

Add Comment


E-mail address and name can be saved in the Preferences.

 
Note: See TracTickets for help on using tickets.