LVM WMTS fails: "stream is closed"

[richlv]

Attempting to use a recently published WMTS service by LVM (Latvian State Forestry company), ​https://lvmgeoserver.lvm.lv/geoserver/gwc/service/wmts .

Adding it in JOSM and marking the "Set default layer?" checkbox enables the "Get layers" button.
Clicking it errors out with "Error getting layers: stream is closed" (the error message cannot be copied).

Not quite sure whether it's some support missing in JOSM, or the published service not fully following standards.

URL:https://josm.openstreetmap.de/svn/trunk
Repository:UUID: 0c6e7542-c601-0410-84e7-c038aed88b3b
Last:Changed Date: 2023-03-30 16:51:36 +0200 (Thu, 30 Mar 2023)
Build-Date:2023-03-31 01:30:56
Revision:18700
Relative:URL: ^/trunk

Identification: JOSM/1.5 (18700 en_GB) Mac OS X 10.15.7
OS Build number: Mac OS X 10.15.7 (19H2026)
Memory Usage: 2428 MB / 3641 MB (1112 MB allocated, but free)
Java version: 1.8.0_361-b09, Oracle Corporation, Java HotSpot(TM) 64-Bit Server VM
Look and Feel: com.apple.laf.AquaLookAndFeel
Screen: Display 1127230987 1920×1080 (scaling 1.00×1.00) Display 69733382 1680×1050 (scaling 1.00×1.00)
Maximum Screen Size: 1920×1080
Best cursor sizes: 16×16→16×16, 32×32→32×32
System property file.encoding: UTF-8
System property sun.jnu.encoding: UTF-8
Locale info: en_GB
Numbers with default locale: 1234567890 -> 1234567890
VM arguments: [-Djnlp.application.href=https://josm.openstreetmap.de/download/josm.jnlp, -Djava.util.Arrays.useLegacyMergeSort=true, -Djnlp.tk=awt, -Djnlpx.jvm=<java.home>/bin/java, -Djnlpx.splashport=-1, -Djnlpx.home=<java.home>/bin, -Djnlpx.remove=false, -Djnlpx.offline=false, -Djnlpx.relaunch=true, -Djnlpx.session.data=/var/folders/nl/flqxqsmj5q963r7tcnfrdt3c0000gn/T/session5588668227892844362, -Djnlpx.heapsize=NULL,NULL, -Djava.security.policy=file:<java.home>/lib/security/javaws.policy, -DtrustProxy=true, -Djnlpx.origFilenameArg=/Users/richlv/Library/Application Support/Oracle/Java/Deployment/cache/6.0/56/1ee8cfb8-72e8e992, -Dsun.awt.warmup=true, -Djava.security.manager]

Plugins:
+ HouseNumberTaggingTool (35951)
+ InfoMode (35978)
+ Mapillary (2.1.2)
+ PicLayer (1.0.2)
+ apache-commons (36034)
+ apache-http (35924)
+ buildings_tools (36011)
+ dataimport (35932)
+ ejml (35924)
+ geotools (36028)
+ imagery_offset_db (35978)
+ jackson (36034)
+ jaxb (35952)
+ jna (36005)
+ jts (36004)
+ measurement (35978)
+ opendata (36025)
+ pbf (36034)
+ photo_geotagging (35933)
+ reverter (36043)
+ undelete (36011)
+ utilsplugin2 (36011)

Map paint styles:
- https://josm.openstreetmap.de/josmfile?page=Styles/Coloured_Streets&zip=1

Last errors/warnings:
- 00027.624 W: java.net.SocketTimeoutException: connect timed out
- 00027.625 E: java.net.SocketTimeoutException: connect timed out
- 00027.626 E: org.openstreetmap.josm.io.OsmTransferException: Could not connect to the OSM server. Please check your internet connection.. Cause: java.net.SocketTimeoutException: connect timed out
- 00054.935 W: java.net.SocketTimeoutException: connect timed out
- 00055.370 W: Already here java.net.SocketTimeoutException: connect timed out
- 00055.371 E: java.net.SocketTimeoutException: connect timed out
- 00055.373 W: org.openstreetmap.josm.io.OsmTransferException: Could not connect to the OSM server. Please check your internet connection.. Cause: java.net.SocketTimeoutException: connect timed out

[taylor.smock]

Try add ?REQUEST=GetCapabilities to the end of the URL.

With that said, it looks like there is some issue with the source:

2023-04-17 07:21:12.203 WARNING: Can't use layer Ortofoto IS 7/8. cikls because no supported formats were found. Layer is available in formats: image/vnd.jpeg-png8
2023-04-17 07:21:12.438 java[17429:515097] 2023-04-17 07:21:12.203 WARNING: Can't use layer Ortofoto IS 7/8. cikls because no supported formats were found. Layer is available in formats: image/vnd.jpeg-png8
2023-04-17 07:21:12.439 WARNING: Can't use layer Veģetācijas virsmas modelis, 1.cikls because no supported formats were found. Layer is available in formats: image/vnd.jpeg-png8
2023-04-17 07:21:12.439 java[17429:515097] 2023-04-17 07:21:12.439 WARNING: Can't use layer Veģetācijas virsmas modelis, 1.cikls because no supported formats were found. Layer is available in formats: image/vnd.jpeg-png8
2023-04-17 07:21:12.440 WARNING: Can't use layer Ortofoto NGR 7.cikls because no supported formats were found. Layer is available in formats: image/vnd.jpeg-png8
2023-04-17 07:21:12.440 java[17429:515097] 2023-04-17 07:21:12.440 WARNING: Can't use layer Ortofoto NGR 7.cikls because no supported formats were found. Layer is available in formats: image/vnd.jpeg-png8
2023-04-17 07:21:12.441 WARNING: Can't use layer Reljefa slīpuma modelis because no supported formats were found. Layer is available in formats: image/vnd.jpeg-png8
2023-04-17 07:21:12.441 java[17429:515097] 2023-04-17 07:21:12.441 WARNING: Can't use layer Reljefa slīpuma modelis because no supported formats were found. Layer is available in formats: image/vnd.jpeg-png8
2023-04-17 07:21:12.441 WARNING: Can't use layer Izlīdzināts veģetācijas virsmas modelis, 1.cikls because no supported formats were found. Layer is available in formats: image/vnd.jpeg-png8
2023-04-17 07:21:12.442 java[17429:515097] 2023-04-17 07:21:12.441 WARNING: Can't use layer Izlīdzināts veģetācijas virsmas modelis, 1.cikls because no supported formats were found. Layer is available in formats: image/vnd.jpeg-png8
2023-04-17 07:21:12.442 WARNING: Can't use layer Ortofoto 7/8. cikls because no supported formats were found. Layer is available in formats: image/vnd.jpeg-png8
2023-04-17 07:21:12.443 java[17429:515097] 2023-04-17 07:21:12.442 WARNING: Can't use layer Ortofoto 7/8. cikls because no supported formats were found. Layer is available in formats: image/vnd.jpeg-png8
2023-04-17 07:21:12.444 WARNING: Can't use layer Topokarte 1:10 000 ar reljefa ēnojumu because no supported formats were found. Layer is available in formats: image/vnd.jpeg-png8
2023-04-17 07:21:12.444 java[17429:515097] 2023-04-17 07:21:12.444 WARNING: Can't use layer Topokarte 1:10 000 ar reljefa ēnojumu because no supported formats were found. Layer is available in formats: image/vnd.jpeg-png8
2023-04-17 07:21:12.444 WARNING: Can't use layer Ortofoto IS 7.cikls because no supported formats were found. Layer is available in formats: image/vnd.jpeg-png8
2023-04-17 07:21:12.445 java[17429:515097] 2023-04-17 07:21:12.444 WARNING: Can't use layer Ortofoto IS 7.cikls because no supported formats were found. Layer is available in formats: image/vnd.jpeg-png8
2023-04-17 07:21:12.445 WARNING: Can't use layer Horizontāles ar augstumu vērtībām because no supported formats were found. Layer is available in formats: image/vnd.jpeg-png8
2023-04-17 07:21:12.446 java[17429:515097] 2023-04-17 07:21:12.445 WARNING: Can't use layer Horizontāles ar augstumu vērtībām because no supported formats were found. Layer is available in formats: image/vnd.jpeg-png8
2023-04-17 07:21:12.447 WARNING: Can't use layer Digitāls virsmas modelis because no supported formats were found. Layer is available in formats: image/vnd.jpeg-png8
2023-04-17 07:21:12.447 java[17429:515097] 2023-04-17 07:21:12.447 WARNING: Can't use layer Digitāls virsmas modelis because no supported formats were found. Layer is available in formats: image/vnd.jpeg-png8
2023-04-17 07:21:12.448 WARNING: Can't use layer Kvartāra nogulumu karte because no supported formats were found. Layer is available in formats: image/vnd.jpeg-png8
2023-04-17 07:21:12.448 java[17429:515097] 2023-04-17 07:21:12.448 WARNING: Can't use layer Kvartāra nogulumu karte because no supported formats were found. Layer is available in formats: image/vnd.jpeg-png8
2023-04-17 07:21:12.448 WARNING: Can't use layer Digitāls reljefa modelis because no supported formats were found. Layer is available in formats: image/vnd.jpeg-png8
2023-04-17 07:21:12.449 java[17429:515097] 2023-04-17 07:21:12.448 WARNING: Can't use layer Digitāls reljefa modelis because no supported formats were found. Layer is available in formats: image/vnd.jpeg-png8
2023-04-17 07:21:12.449 WARNING: Can't use layer Reljefa modelis ar slīpumu un horizontālēm because no supported formats were found. Layer is available in formats: image/vnd.jpeg-png8
2023-04-17 07:21:12.450 java[17429:515097] 2023-04-17 07:21:12.449 WARNING: Can't use layer Reljefa modelis ar slīpumu un horizontālēm because no supported formats were found. Layer is available in formats: image/vnd.jpeg-png8

Looking at the ​documentation, it looks like it is a vendor specific image format which can return either a jpeg or a png.

We do not currently support that vendor format.

[richlv]

Thank you so much for looking into this.
I guess there are several issues here:

1. Would it be useful for JOSM to automatically add "?REQUEST=GetCapabilities"?
2. Is there an existing ticket to seek support for image/vnd.jpeg-png8? Searched but couldn't find one.
3. Would it be possible to improve error messages in such cases?

[taylor.smock]

1. Probably. We do that for WMS already.
2. Not as far as I know, but there are a lot of open tickets. So it could have been missed.
3. Maybe. It kind of depends -- geoserver appears to return an XML that could be parsed, but I don't know if other imagery servers would have the same behavior.

[stoecker]

Looking at the ​documentation, it looks like it is a vendor specific image format which can return either a jpeg or a png.

Shouldn't that be easy to support? We don't really care if it is png or jpeg, do we?

[taylor.smock]

Replying to stoecker:

Shouldn't that be easy to support? We don't really care if it is png or jpeg, do we?

I think so (as in, I think we just pass the bytes to ImageIO), I just haven't tested it yet. I'll try to get around to that today.

[taylor.smock]

@richlv: Can you give me some coordinates where there is imagery (and the layer with the imagery)? I'm not seeing anything. I'm debugging that, but it would be helpful to have a known-good location. For reference, I've been looking at way 45105364.

EDIT: Nevermind. It turns out that the layer I was using didn't have data.

[taylor.smock]

In 18703/josm:

See #22869: Add geoserver custom mimetypes

• image/vnd.jpeg-png
• image/vnd.jpeg-png8

Both are functionally the same, in so far as geoserver attempts to dynamically
find the "best" size between JPEG and PNG/PNG8 formats, and send that to the client.

For further details on the formats, see
​https://docs.geoserver.org/latest/en/user/services/wms/outputformats.html

[richlv]

Yay, it seems to get the layer list, thank you so much :)
What about automatically trying to add "?REQUEST=GetCapabilities", would it make sense to do (and perhaps handle in this ticket)?

[taylor.smock]

In order to "close" this ticket, we need to code something like attemptGetCapabilities​ for WMTS.

@skyper: I didn't have an actual ticket for the image mime-types. The initial problem was that @richlv couldn't get any layers because there was a missing REQUEST=GetCapabilities query parameter, but while I was investigating that, I found the mime-type issue.

[mnalis]

It would be very nice if WMTS would automatically detect that it needs to append that; as WMTS providers will usually only say something like "use ​https://geoportal.dgu.hr/services/auth/orthophoto_lidar_2022_2023/wmts?authKey=1234 but replace 1234 with your key"

While users of JOSM are often somewhat more advanced than average iD user, it is not reasonable to expect them to have arcane knowledge of internals of WMTS protocol, and so to know that they have to add "&REQUEST=GetCapabilities" (or "?REQUEST=GetCapabilities" if there are not HTTP query params existing) to the end of base URL in order to resolve "Error getting layers: stream is closed" error.

so I would suggest:

• autodetect that such parameter is missing, and add it (e.g. if the error is returned, automatically retry with added param)
• as an interim solution, at least update the error message to say "Error getting layers: stream is closed. Perhaps you need to add "&REQUEST=GetCapabilities" or "?REQUEST=GetCapabilities" to the end of your base WMTS URL?"