Exceeding Bing's maximum zoom level results in terrible "white no camera" image

[bdiscoe]

What steps will reproduce the problem?

1. Add the Bing imagery layer.
2. Go to any part of the world where Bing does not have imagery at all zoom levels. For example, here: ​https://www.openstreetmap.org/#map=18/24.34152/93.71703
3. Note that at zoom level 18, there is imagery.
4. Try to zoom in slightly. Note that the imagery vanishes, replaced with a blank white image showing a camera with a line through it.

What is the expected result?

When you exceed the maximum zoom, JOSM should show the highest zoom level available, magnified.

What happens instead?

JOSM hides all the imagery with the "blank no camera" image. This is EXTREMELY frustrating, forcing used to zoom back to the previous level making it very hard to trace features such as buildings.

If this is fundamentally a limitation of the Bing server (rather than JOSM), then JOSM must either:

1. Detect when the zoom level imagery is missing, so it can show the highest zoom level available instead; if necessary, it can show the "no imagery available at this zoom level" message as an overlay, rather than obscuring everything.
2. Or, provide some way to work around the Bing server's limitation, e.g. by allowing the user to set a maximum zoom level which is respected. E.g. if Bing only has imagery to zoom level 18, and JOSM is not able to detect this, then allow the user to specify "18" as the maximum level, so that zooming in further will show 18 magnified, rather than the "blank no camera" image.

Please provide any additional information below. Attach a screenshot if possible.

If the user is lucky, you CAN zoom in and see the imagery correctly magnified ... for a while. As you start to pan, JOSM gets around to requesting the higher-zoom images and starts to show the dreaded "white no camera". Trying to exploit this workaround leads to a dance of careful zooms without pans, so that buildings can be traced quickly, without panning, then zoom out again before it start to blank out.

URL:http://josm.openstreetmap.de/svn/trunk
Repository:UUID: 0c6e7542-c601-0410-84e7-c038aed88b3b
Last:Changed Date: 2015-11-24 00:04:12 +0100 (Tue, 24 Nov 2015)
Build-Date:2015-11-23 23:14:21
Revision:9060
Relative:URL: ^/trunk

Identification: JOSM/1.5 (9060 en) Windows 7 64-Bit
Memory Usage: 1277 MB / 7237 MB (463 MB allocated, but free)
Java version: 1.8.0_66, Oracle Corporation, Java HotSpot(TM) 64-Bit Server VM
VM arguments: [-Djava.security.manager, -Djava.security.policy=file:C:\Program Files\Java\jre1.8.0_66\lib\security\javaws.policy, -DtrustProxy=true, -Djnlpx.home=<java.home>\bin, -Djnlpx.origFilenameArg=C:\Users\Ben\AppData\LocalLow\Sun\Java\Deployment\cache\6.0\56\1ee8cfb8-4546c991, -Djnlpx.remove=false, -Djava.util.Arrays.useLegacyMergeSort=true, -Djnlpx.splashport=61261, -Djnlp.application.href=https://josm.openstreetmap.de/download/josm.jnlp, -Djnlpx.jvm=<java.home>\bin\javaw.exe, -Djnlpx.vmargs=LURqYXZhLnV0aWwuQXJyYXlzLnVzZUxlZ2FjeU1lcmdlU29ydD10cnVlAC1Eam5scC5hcHBsaWNhdGlvbi5ocmVmPWh0dHBzOi8vam9zbS5vcGVuc3RyZWV0bWFwLmRlL2Rvd25sb2FkL2pvc20uam5scAA=]
Dataset consistency test: No problems found

Plugins:
- FastDraw (31772)
- SimplifyArea (31772)
- buildings_tools (31772)
- download_along (31772)
- missingRoads (42)
- scripting (30722)
- tofix (171)
- utilsplugin2 (31772)

Last errors/warnings:
- W: org.openstreetmap.josm.io.OsmTransferException: Could not connect to the OSM server. Please check your internet connection.. Cause: java.net.NoRouteToHostException: No route to host: connect
- E: java.net.NoRouteToHostException: No route to host: connect
- W: org.openstreetmap.josm.io.OsmTransferException: Could not connect to the OSM server. Please check your internet connection.. Cause: java.net.NoRouteToHostException: No route to host: connect
- E: java.net.NoRouteToHostException: No route to host: connect
- W: org.openstreetmap.josm.io.OsmTransferException: Could not connect to the OSM server. Please check your internet connection.. Cause: java.net.NoRouteToHostException: No route to host: connect

[bdiscoe]

missing imagery that JOSM fails to notice

[malenki]

I cannot confirm JOSM's behaviour you describe.

As a temporary workaround you may zoom to the level where imagery is still available and then disable in the context menu
[ ] Auto Zoom
This will cause JOSM to just load the tiles of this particular layer.

Additionally, in JOSM's preferences you already have the possibility to set the range of zoom levels for TMS in
Imagery Preferences –> Settings

I use JOSM latest and Java 1.8

[malenki]

JOSM with Imagery and error message for missing tiles at ZL

[wiktorn]

I saw similar behaviour and it was due to responses from server not containing the header indicating, that there is no imagery. But I couldn't make it reproducible. I suspect, that there might be some race condition within JOSM and we are dropping the headers or some Bing servers are misconfigured. I'd start with first assumption.

[bdiscoe]

Thanks for trying to reproduce, malenki. I am sad to hear it does not happen everywhere, so it is much harder to fix. But, the workaround of disabling "Auto Zoom" is great, that will save me a lot of frustration! Hopefully a repeatable case (other than on my machine) can be found.

FWIW, I did try changing the JOSM TMS settings in all sorts of ways, both the global settings and for the Bing layer, and the problem still happens.

[wiktorn]

For this area - I am able to reproduce this in this area when running from JNLP (Java Web Start) just by flushing the tile cache.

I'm not able do reproduce this, when running the Java directly.

Clearing Java Web Start cache didn't help either.

[wiktorn]

In 9210/josm:

Temporary logging of response headers at debug level.

See: #12235

[wiktorn]

Now I see - not all headers are cached within Java Web Start cache. So the problem occurs, when the file is downloaded from JWS cache, and is not existing in JOSM cache.

I recommend:

• run javacpl.exe (Configure Java)
• On General tab, under "Temporary Internet Files" choose Settings
• Switch off "keep temporary internet files on my computer"

This should workaround this problem. But it's worth going to Oracle with this bug, I'll do it later.

[wiktorn]

In 9228/josm:

Do not use cache when downloading.

This avoids getting objects from Java Web Start Cache, though it will
also skip any tile cache in between, which may result in bigger load of
tile servers.

See #12235

[wiktorn]

Let's see if this helps, before reporting to Oracle

[bastiK]

Replying to wiktorn:

Now I see - not all headers are cached within Java Web Start cache. So the problem occurs, when the file is downloaded from JWS cache, and is not existing in JOSM cache.

I cannot reproduce with java web start either (Ubuntu). Here it only caches the files it downloads itself, i.e. .jar files, certificates and the like. How does it even intercept the https connection? Very strange.

[Don-vip]

Java web start is completely different between oracle jdk and Debian/ubuntu openjdk, because oracle did not open source it. Have you tried with oracle jdk?

[wiktorn]

I was able to reproduce the problem on JOSM [9227] on Oracle JDK 1.8.0_65

[bastiK]

r9229 is new tested, any easy way to test older version with webstart?

[wiktorn]

Use attached jnlp for version 9227

[wiktorn]

Submitted bug to Oracle:
​https://bugs.openjdk.java.net/browse/JI-9028123

Link should be accessible after in few days.

[Don-vip]

can you please also update JavaBugs? :)

[bastiK]

Replying to wiktorn:

I was able to reproduce the problem on JOSM [9227] on Oracle JDK 1.8.0_65

Confirmed with same versions on Linux. I can also see the cached images in the Java cache, along with data files containing crippled htttp-header.

[wiktorn]

Replying to Don-vip:

can you please also update JavaBugs? :)

done

[Don-vip]

Good news: bug is public: ​javabug:8146450

Bad news: bug is marked "resolved" and lacks josm-found label. It someone from Oracle contacts you, can you please ask him to add this label?

[wiktorn]

I got a message from Oracle today, I'll ask for josm-found label and also explain how to reproduce.

[wiktorn]

Replying to Don-vip:

Good news: bug is public: ​javabug:8146450

Bad news: bug is marked "resolved" and lacks josm-found label. It someone from Oracle contacts you, can you please ask him to add this label?

The bug went to status open and now it's once again closed with some commits in Java Web Start behind. Though bug description indicates, that's fixed only in Java 9. I'll take a look at latest Early Access version to check, though I'm not sure, if the fix is already released.

[wiktorn]

Looks like the fix is available in Java 9 Early Access build 105 at:
​https://jdk9.java.net/download/

I hope, that the fix will get included in Java 8 too, so we can rollback [9228] and be more intermediate cache (proxy, CDN) friendly.

[wiktorn]

In 9807/josm:

Make map returned by getHeaderFields() case insensitive.

RFC 2616, section 4.2, specifies that header names are case insensitive. This is properly handled by getHeaderField(...) in HttpUrlConnection, but in case of getHeaderFields() Map returned by Oracle JDK is case sensitive. By using case insentive map we avoid bigger refactoring of all usages, to use getHeaderField(...) where appropiate.

See #12235: Java Web Start cache returns all headers as lowercase, when real server return mixed case.

[Don-vip]

​javabug:8146450 is marked as fixed in Java 9, so please update to this version.

[simon04]

In 15740/josm:

fix #18588, see #12235 - Comply with OSM Tile Usage Policy

Do not send no-cache headers unless hitting JDK-8146450