ImageryCache locks up JOSM when Bing attribution is loaded

[rickmastfan67]

Just happened to notice this happening last night and tonight to me.

But when the Bing attribution is loaded for the first time on your system for the day, JOSM gets locked up for about 15-20 seconds. I can't seem to duplicate this on command. It just happens once every 24h or so.

Just thought I would throw this out there just in case anybody else notices this happening to them.

Repository Root: http://josm.openstreetmap.de/svn
Build-Date: 2013-04-11 01:36:56
Last Changed Author: akks
Revision: 5838
Repository UUID: 0c6e7542-c601-0410-84e7-c038aed88b3b
URL: http://josm.openstreetmap.de/svn/trunk
Last Changed Date: 2013-04-10 15:27:58 +0200 (Wed, 10 Apr 2013)
Last Changed Rev: 5838

Identification: JOSM/1.5 (5838 en) Windows 7
Memory Usage: 154 MB / 2730 MB (78 MB allocated, but free)
Java version: 1.7.0_15, Oracle Corporation, Java HotSpot(TM) 64-Bit Server VM
Operating system: Windows 7
VM arguments: [-Xmx3072M]

Plugin: ImageryCache (29484)
Plugin: OpeningHoursEditor (29435)
Plugin: buildings_tools (29435)
Plugin: mapdust (29435)
Plugin: measurement (29435)
Plugin: mirrored_download (29435)
Plugin: openstreetbugs (29435)
Plugin: osmarender (29435)
Plugin: reverter (29435)
Plugin: turnrestrictions (29435)
Plugin: undelete (29474)
Plugin: utilsplugin2 (29435)

[akks]

Does it only when ImageryCache is enabled?
When exactly it was locked - when opening Download from OSM? Was the whole gui locked or only imagery was not loading?

[rickmastfan67]

1. Yes, only when ImageryCache is enabled. Never had any problem like this before the plugin was installed. In fact, this has only happened since the recent update to the plugin.
2. Happened when Bing imagery was loading in the background after I had downloaded a section of OSM data to edit. And it was the entire GUI of JOSM that was "frozen" for the 15-20 seconds.

[rickmastfan67]

Well, today it didn't lock up, so I don't know what to say. Maybe deleting my Bing database files fixed this, who knows. I'll post an update when/if this happens again.

[akks]

I forgot to mention - after plugin update, the database format was changed due to MapDB update. Manual cleaning was essential, but I do not know how it can block the main thread... Please infrom me if the locking continues.

[Don-vip]

I fixed a bug in Bing attribution loading yesterday, not sure if it's related (​http://josm.openstreetmap.de/changeset/5839/josm/#file1)

[rickmastfan67]

Replying to akks:

I forgot to mention - after plugin update, the database format was changed due to MapDB update. Manual cleaning was essential, but I do not know how it can block the main thread... Please infrom me if the locking continues.

Just had it happen again. Lasted about 15 seconds.

Here's what I had done:

1. Started JOSM.
2. Opened a local file.
3. File > Update Data
4. Zoomed into the section I wanted to start working on.
5. Hit the Bing imagery button and have a few tiles load.
6. Clicked on a node to start twinning a highway.
7. After the first newly created node, JOSM hung.

I'm assuming this is when JOSM was loading the Bing attribution file and the JOSM froze.

Repository Root: http://josm.openstreetmap.de/svn
Build-Date: 2013-04-14 01:34:25
Last Changed Author: Don-vip
Revision: 5851
Repository UUID: 0c6e7542-c601-0410-84e7-c038aed88b3b
URL: http://josm.openstreetmap.de/svn/trunk
Last Changed Date: 2013-04-14 00:49:25 +0200 (Sun, 14 Apr 2013)
Last Changed Rev: 5851

Identification: JOSM/1.5 (5851 en) Windows 7 64-Bit
Memory Usage: 199 MB / 2730 MB (112 MB allocated, but free)
Java version: 1.7.0_15, Oracle Corporation, Java HotSpot(TM) 64-Bit Server VM
Operating system: Windows 7 64-Bit
VM arguments: [-Xmx3072M]
Dataset consistency test: No problems found

Plugin: ImageryCache (29484)
Plugin: OpeningHoursEditor (29435)
Plugin: buildings_tools (29435)
Plugin: mapdust (29435)
Plugin: measurement (29435)
Plugin: mirrored_download (29435)
Plugin: openstreetbugs (29435)
Plugin: osmarender (29435)
Plugin: reverter (29435)
Plugin: turnrestrictions (29435)
Plugin: undelete (29474)
Plugin: utilsplugin2 (29435)

[rickmastfan67]

Just wanted to mention that this is still happening. :( Even with r6015 and the latest copy of the ImageryCache plugin.

[akks]

Sorry, I hoped it was gone :)
Is is always happening when Bing imagery is loaded? Try to turn on debug output (search imagery in advanced preferences) and run with
java -jar josm-latest.jar

Can you post last messages in console when it hangs?
Also try to turn off antivirus for a while (to know if it is involved).

[rickmastfan67]

Replying to akks:

Is is always happening when Bing imagery is loaded? Try to turn on debug output (search imagery in advanced preferences) and run with
java -jar josm-latest.jar

Yes and no. It's only with Bing does this lockup ever happen. It seems to happens most of the time on the first startup of a new build of JOSM & whenever it's time for the Bing attribution to be updated. So, the next time a new "latest" file is released might be the best time to test this. However, it does sometimes lock up after ~24 hours when a new Bing attribution file is downloaded, but not always.

When you say turn on the "Debug output", I searched "imagery" in the advance preferences area and couldn't find anything there to turn the debug output on with. Are you sure that's the correct area for it?

Also, I already run JOSM the way you suggested unless you want me to get rid of my custom memory setting.
"C:\Program Files\Java\jre7\bin\java.exe" -Xmx3072M -jar josm-tested.jar

Replying to akks:

Can you post last messages in console when it hangs?
Also try to turn off antivirus for a while (to know if it is involved).

My antivirus shouldn't be effecting it as it hasn't change since this has started happening. As I said back up in comment:2, this locking only started after the last update to this plugin, [o29484]. Before that, I never had any lockups what so ever with Bing and this plugin in JOSM. I'd rather try the first part you suggest before attempt to turn off Comodo (my antivirus/firewall program) to see if anything shows up in the debug log.

[anonymous]

Comodo is often blocking UI when progran needs some "exotic" permissions, it is my own experience.
I am almost sure it is unvolved ( MapDB needs some exotic IO functions)

Try searching advanced prefs for "debug". I'll have a look and make ultra-verbose version for you to test.

[rickmastfan67]

The only "debug" that I find in the advance preferences is the following:
debug.edit-checker.enable=false

And with Comodo, nothing has changed with it before the [o29484] release of ImageryCache. So, I highly doubt that could be the case, but who knows. I've never have seen anything popup in Defense+ pointing to any problems. BTW, I'm still using the 5 branch of Comodo. Still don't fell like upgrading to Comodo 6 because of the messed up GUI it has.

[akks]

"imagerycache.debug" is the parameter.

If the blocking hapens only once after JOSM update, it is surely Comodo. In last version MapDB source was updated and some low-level operations were possibly introduced.

Try turning off defence+ after latest release, if it will not hang- set manual mode - it will surely ask for something after loading the first new tile.

[rickmastfan67]

I think you might be right that it is Comodo. I just started up JOSM with the same release (r6015) as last night and it did hang with the Bing imagery. I had turned on the imagerycache debug setting before loading the Bing imagery and nothing seems to stand out indicating a bug there. Here's what showed up:

GET http://api.openstreetmap.org/api/0.6/user/details
Opened database file successfully: C:\Users\***\OSM-Tiles\tiles_Bing_Aerial_Maps
Successfully loaded Bing attribution data.
Tile 74853697558: Loading from OSM, Tile 17/35557/51222@Bing Aerial Maps
Tile 74857891860: Loading from OSM, Tile 17/35559/51220@Bing Aerial Maps
Tile 74851600403: Loading from OSM, Tile 17/35556/51219@Bing Aerial Maps
Tile 74849503252: Loading from OSM, Tile 17/35555/51220@Bing Aerial Maps
Tile 74855794709: Loading from OSM, Tile 17/35558/51221@Bing Aerial Maps
Tile 74857891861: Loading from OSM, Tile 17/35559/51221@Bing Aerial Maps
Tile 74849503254: Loading from OSM, Tile 17/35555/51222@Bing Aerial Maps
Tile 74851600404: Loading from OSM, Tile 17/35556/51220@Bing Aerial Maps
Tile 74853697555: Loading from OSM, Tile 17/35557/51219@Bing Aerial Maps
Tile 74855794710: Loading from OSM, Tile 17/35558/51222@Bing Aerial Maps
Tile 74853697557: Loading from OSM, Tile 17/35557/51221@Bing Aerial Maps
Tile 74853697556: Loading from OSM, Tile 17/35557/51220@Bing Aerial Maps
Tile 74849503253: Loading from OSM, Tile 17/35555/51221@Bing Aerial Maps
Tile 74855794707: Loading from OSM, Tile 17/35558/51219@Bing Aerial Maps
Tile 74849503251: Loading from OSM, Tile 17/35555/51219@Bing Aerial Maps
Tile 74857891859: Loading from OSM, Tile 17/35559/51219@Bing Aerial Maps
Tile 74851600406: Loading from OSM, Tile 17/35556/51222@Bing Aerial Maps
Tile 74851600405: Loading from OSM, Tile 17/35556/51221@Bing Aerial Maps
Tile 74857891862: Loading from OSM, Tile 17/35559/51222@Bing Aerial Maps
Tile 74855794708: Loading from OSM, Tile 17/35558/51220@Bing Aerial Maps
Tile 74851600403: Updating tile in base
Tile 74849503252: Updating tile in base
Tile 74857891860: Updating tile in base
Tile 74853697558: Updating tile in base
Tile 74855794709: Updating tile in base
Tile 74857891861: Updating tile in base
Tile 74849503254: Updating tile in base
Tile 74851600404: Updating tile in base
Tile 74853697555: Updating tile in base
Tile 74855794710: Updating tile in base
Tile 74853697557: Updating tile in base
Tile 74853697556: Updating tile in base
Tile 74849503253: Updating tile in base
Tile 74855794707: Updating tile in base
Tile 74849503251: Updating tile in base
Tile 74857891859: Updating tile in base
Tile 74851600406: Updating tile in base
Tile 74851600405: Updating tile in base
Tile 74857891862: Updating tile in base
Tile 74855794708: Updating tile in base
Tile 74855794706: Loading from OSM, Tile 17/35558/51218@Bing Aerial Maps
Tile 74851600402: Loading from OSM, Tile 17/35556/51218@Bing Aerial Maps
Tile 74855794706: Updating tile in base
Tile 74853697554: Loading from OSM, Tile 17/35557/51218@Bing Aerial Maps
Tile 74851600402: Updating tile in base
Tile 74857891858: Loading from OSM, Tile 17/35559/51218@Bing Aerial Maps
Tile 74853697554: Updating tile in base
Tile 74857891858: Updating tile in base
Tile 74853697553: Loading from OSM, Tile 17/35557/51217@Bing Aerial Maps
Tile 74853697553: Updating tile in base
Tile 74849503250: Loading from OSM, Tile 17/35555/51218@Bing Aerial Maps
Tile 74851600401: Loading from OSM, Tile 17/35556/51217@Bing Aerial Maps
Tile 74855794705: Loading from OSM, Tile 17/35558/51217@Bing Aerial Maps
Tile 74849503249: Loading from OSM, Tile 17/35555/51217@Bing Aerial Maps
Tile 74857891857: Loading from OSM, Tile 17/35559/51217@Bing Aerial Maps
Tile 74857891857: Updating tile in base
Tile 74849503249: Updating tile in base
Tile 74855794705: Updating tile in base
Tile 74851600401: Updating tile in base
Tile 74849503250: Updating tile in base
Tile 74847406099: Loading from OSM, Tile 17/35554/51219@Bing Aerial Maps
Tile 74847406098: Loading from OSM, Tile 17/35554/51218@Bing Aerial Maps
Tile 74847406097: Loading from OSM, Tile 17/35554/51217@Bing Aerial Maps
Tile 74853697552: Loading from OSM, Tile 17/35557/51216@Bing Aerial Maps
Tile 74849503248: Loading from OSM, Tile 17/35555/51216@Bing Aerial Maps
Tile 74851600399: Loading from OSM, Tile 17/35556/51215@Bing Aerial Maps
Tile 74849503247: Loading from OSM, Tile 17/35555/51215@Bing Aerial Maps
Tile 74851600400: Loading from OSM, Tile 17/35556/51216@Bing Aerial Maps
Tile 74855794703: Loading from OSM, Tile 17/35558/51215@Bing Aerial Maps
Tile 74847406096: Loading from OSM, Tile 17/35554/51216@Bing Aerial Maps
Tile 74857891856: Loading from OSM, Tile 17/35559/51216@Bing Aerial Maps
Tile 74855794704: Loading from OSM, Tile 17/35558/51216@Bing Aerial Maps
Tile 74845308945: Loading from OSM, Tile 17/35553/51217@Bing Aerial Maps
Tile 74845308943: Loading from OSM, Tile 17/35553/51215@Bing Aerial Maps
Tile 74853697551: Loading from OSM, Tile 17/35557/51215@Bing Aerial Maps
Tile 74847406095: Loading from OSM, Tile 17/35554/51215@Bing Aerial Maps
Tile 74847406100: Loading from OSM, Tile 17/35554/51220@Bing Aerial Maps
Tile 74845308946: Loading from OSM, Tile 17/35553/51218@Bing Aerial Maps
Tile 74845308944: Loading from OSM, Tile 17/35553/51216@Bing Aerial Maps
Tile 74847406099: Updating tile in base
Tile 74847406098: Updating tile in base
Tile 74847406097: Updating tile in base
Tile 74853697552: Updating tile in base
Tile 74849503248: Updating tile in base
Tile 74851600399: Updating tile in base
Tile 74849503247: Updating tile in base
Tile 74855794704: Updating tile in base
Tile 74851600400: Updating tile in base
Tile 74857891856: Updating tile in base
Tile 74847406096: Updating tile in base
Tile 74855794703: Updating tile in base
Tile 74845308945: Updating tile in base
Tile 74845308943: Updating tile in base
Tile 74853697551: Updating tile in base
Tile 74847406095: Updating tile in base
Tile 74847406100: Updating tile in base
Tile 74845308946: Updating tile in base
Tile 74845308944: Updating tile in base

I'll have to wait another 24 hours before I can try with Defense+ off. Main reason I say that is because I've tried to duplicate this in the past by deleting the Bing attribution file and then restart JOSM and it doesn't happen.

If this also helps, my Bing cache file is only ~92MB at this time.

[akks]

You can try custom built jar ​https://dl.dropboxusercontent.com/u/63393258/josm-custom.jar or just go to Comodo known applications list (somewhere in "advanced") and delete josm.jar from the list.

I have also updated ImageryCache plugin, introducing imagerycache.randomAccessFile parameter. Try turn it to "true" - it can solve the problem (but may work little slower).

[rickmastfan67]

Replying to akks:

just go to Comodo known applications list (somewhere in "advanced") and delete josm.jar from the list.

I just took a look at all the "Trusted" files in the Defense+ of Comodo. No "josm*.jar" named files were in there. It seems that JOSM's jar files just piggyback off of Java being trusted.

I have also updated ImageryCache plugin, introducing imagerycache.randomAccessFile parameter. Try turn it to "true" - it can solve the problem (but may work little slower).

Downloaded the new version of the plugin. I also did download that custom josm file above. Will test both later tonight.

[rickmastfan67]

Can't use that "josm-custom" file. Kept getting a NPE when trying to download some data so I could test Bing Imagery in the background. :(

Build-Date: 2013-06-21 20:23:11
Revision: 5977
Is-Local-Build: true

Identification: JOSM/1.5 (5977 SVN en) Windows 7 64-Bit
Memory Usage: 122 MB / 2730 MB (79 MB allocated, but free)
Java version: 1.7.0_25, Oracle Corporation, Java HotSpot(TM) 64-Bit Server VM
VM arguments: [-Xmx3072M]

Plugin: ImageryCache (29690)
Plugin: OpeningHoursEditor (29435)
Plugin: buildings_tools (29596)
Plugin: mapdust (29525)
Plugin: measurement (29625)
Plugin: mirrored_download (29643)
Plugin: openstreetbugs (29435)
Plugin: osmarender (29639)
Plugin: reverter (29663)
Plugin: turnrestrictions (29435)
Plugin: undelete (29555)
Plugin: utilsplugin2 (29672)

java.lang.NullPointerException
	at org.openstreetmap.josm.gui.progress.ProgressMonitorExecutor.submit(ProgressMonitorExecutor.java:40)
	at org.openstreetmap.josm.gui.io.RecentlyOpenedFilesMenu$2.actionPerformed(RecentlyOpenedFilesMenu.java:68)
	at javax.swing.AbstractButton.fireActionPerformed(Unknown Source)
	at javax.swing.AbstractButton$Handler.actionPerformed(Unknown Source)
	at javax.swing.DefaultButtonModel.fireActionPerformed(Unknown Source)
	at javax.swing.DefaultButtonModel.setPressed(Unknown Source)
	at javax.swing.AbstractButton.doClick(Unknown Source)
	at javax.swing.plaf.basic.BasicMenuItemUI.doClick(Unknown Source)
	at javax.swing.plaf.basic.BasicMenuItemUI$Handler.mouseReleased(Unknown Source)
	at java.awt.Component.processMouseEvent(Unknown Source)
	at javax.swing.JComponent.processMouseEvent(Unknown Source)
	at java.awt.Component.processEvent(Unknown Source)
	at java.awt.Container.processEvent(Unknown Source)
	at java.awt.Component.dispatchEventImpl(Unknown Source)
	at java.awt.Container.dispatchEventImpl(Unknown Source)
	at java.awt.Component.dispatchEvent(Unknown Source)
	at java.awt.LightweightDispatcher.retargetMouseEvent(Unknown Source)
	at java.awt.LightweightDispatcher.processMouseEvent(Unknown Source)
	at java.awt.LightweightDispatcher.dispatchEvent(Unknown Source)
	at java.awt.Container.dispatchEventImpl(Unknown Source)
	at java.awt.Window.dispatchEventImpl(Unknown Source)
	at java.awt.Component.dispatchEvent(Unknown Source)
	at java.awt.EventQueue.dispatchEventImpl(Unknown Source)
	at java.awt.EventQueue.access$200(Unknown Source)
	at java.awt.EventQueue$3.run(Unknown Source)
	at java.awt.EventQueue$3.run(Unknown Source)
	at java.security.AccessController.doPrivileged(Native Method)
	at java.security.ProtectionDomain$1.doIntersectionPrivilege(Unknown Source)
	at java.security.ProtectionDomain$1.doIntersectionPrivilege(Unknown Source)
	at java.awt.EventQueue$4.run(Unknown Source)
	at java.awt.EventQueue$4.run(Unknown Source)
	at java.security.AccessController.doPrivileged(Native Method)
	at java.security.ProtectionDomain$1.doIntersectionPrivilege(Unknown Source)
	at java.awt.EventQueue.dispatchEvent(Unknown Source)
	at java.awt.EventDispatchThread.pumpOneEventForFilters(Unknown Source)
	at java.awt.EventDispatchThread.pumpEventsForFilter(Unknown Source)
	at java.awt.EventDispatchThread.pumpEventsForHierarchy(Unknown Source)
	at java.awt.EventDispatchThread.pumpEvents(Unknown Source)
	at java.awt.EventDispatchThread.pumpEvents(Unknown Source)
	at java.awt.EventDispatchThread.run(Unknown Source)

[rickmastfan67]

Anyways, I think changed my file to the new [6016] that was just released today. Didn't see any lockup of JOSM when using the Bing Imagery. But I will not pass judgement of the fix to the plugin till tomorrow (after 24h) to make sure today wasn't a fluke.

[rickmastfan67]

No lockup tonight. It seems what you did the other day akks to the plugin was the trick.

[akks]

Sorry for broken josm-custom.jar .

To be honest, I did nothing to the plugin except adding option: ​changes

If you did not set imagerycache.randomAccessFile = true, then the behavior is exactly the same. If it is set, alternative method of accessing disk is used that is possible less "suspicious" to Comodo.

[rickmastfan67]

Replying to akks:

Sorry for broken josm-custom.jar .

To be honest, I did nothing to the plugin except adding option: ​changes

If you did not set imagerycache.randomAccessFile = true, then the behavior is exactly the same. If it is set, alternative method of accessing disk is used that is possible less "suspicious" to Comodo.

I guess so with Comodo. I did set imagerycache.randomAccessFile=true, so, that fix to add that did help me out. I'll give it a few more days of testing (since the Bing attribution is normally only loaded once every 24h) and then I'll mark this as closed if I have no more problems with that setting in my config file for JOSM.

[rickmastfan67]

I've had no lockups in the last few days. So, I'll now mark this as fixed. :)