#14361 closed defect (fixed)
Support for image/webp for WMTS satellite imagery?
Reported by: | SanderH | Owned by: | team |
---|---|---|---|
Priority: | normal | Milestone: | 17.02 |
Component: | Core imagery | Version: | |
Keywords: | wmts webp | Cc: |
Description
I'm trying to get a satellite imagery working that's available in the netherlands since a few days but don't get the WMTS version working.
Configured the WMTS imagery for this url:
https://geodata.nationaalgeoregister.nl/luchtfoto/wmts?&request=GetCapabilities&service=WMS
The provider prefers consumers to use WMTS over WMS (which is working fine) to reduce load on their servers.
When selecting the WMTS from the Imagery menu, I get a nice box to choose from the 3 projections for the 2 layers.
Whatever you choose there it won't work. JOSM keeps complaining "Could not load image file from server".
The tile properties show for example this url with http status code 200:
http://geodata.nationaalgeoregister.nl/luchtfoto/wmts?SERVICE=WMTS&REQUEST=GetTile&VERSION=1.0.0&LAYER=Actueel_ortho25&STYLE=default&FORMAT=image/webp&tileMatrixSet=EPSG:28992&tileMatrix=8&tileRow=134&tileCol=131
I noticed the default format is image/webp, which I'm unable to change to one of the known working image formats (png, png8, jpg) which is also supported for this WMTS.
Since everything else seems working OK (url does work in the browser, status code 200) the only thing I can think of is that the image format itself could be the problem.
Do you know if JOSM supports image/webp?
Can you have a look and see what's going wrong and maybe fix it accordingly if it's something JOSM can do about it?
Thanks in advance.
Attachments (0)
Change History (28)
comment:1 by , 8 years ago
Keywords: | wmts webp added |
---|
comment:2 by , 8 years ago
comment:4 by , 4 years ago
Resolution: | fixed |
---|---|
Status: | closed → reopened |
Would https://github.com/opsmatters/webp-imageio-core be an option?
comment:5 by , 4 years ago
Milestone: | → 17.02 |
---|---|
Resolution: | → fixed |
Status: | reopened → closed |
In source program, coders need to put native lib files like .so/.dll/.dylib into the folder of java.library.path.
No go, sorry.
comment:7 by , 2 years ago
Resolution: | fixed |
---|---|
Status: | closed → reopened |
Reopening to start a discussion..
comment:8 by , 2 years ago
I'll be honest, this kind of sounds like it could be done in a plugin (and probably should be done in a plugin). I don't know if JOSM really wants to bundle a bunch of jar files it doesn't directly depend upon. The few jars I checked from TwelveMonkeys seemed to be reasonably sized (<100kb).
follow-up: 10 comment:9 by , 2 years ago
I made an attempt at https://github.com/ramSeraph/josm-webp-plugin , did not work. I am not very familiar with java. it almost feels like the imageio available to the imagery code was already initialised by the time the plugins load.
comment:10 by , 2 years ago
Replying to Sreeram K <kandimalla.sreeram@…>:
I made an attempt at https://github.com/ramSeraph/josm-webp-plugin , did not work. I am not very familiar with java. it almost feels like the imageio available to the imagery code was already initialised by the time the plugins load.
Notes:
- I probably would have called it
josm-imageio-extensions
or something like that, and then downloaded common extensions from the maven repository (example:avif
/heif
in the future).- Use the plugin manifest
Class-Path
instead of packing the imageio extensions into the jar file if you do this. - This is more complicated. You probably don't want to do that right now.
- Use the plugin manifest
- To get the current plugin to build, I would do this:
git mv lib/build.gradle.kts .; git mv lib/src .
in the root directory of the repository. - Apply the following patch to get the WebP ImageIO extension registered on plugin load
-
src/main/java/org/openstreetmap/josm/plugins/webp/WebpPlugin.java
diff --git a/src/main/java/org/openstreetmap/josm/plugins/webp/WebpPlugin.java b/src/main/java/org/openstreetmap/josm/plugins/webp/WebpPlugin.java index 8cc90aa..b890418 100644
a b 1 1 package org.openstreetmap.josm.plugins.webp; 2 2 3 import javax.imageio.ImageIO; 3 4 import org.openstreetmap.josm.plugins.Plugin; 4 5 import org.openstreetmap.josm.plugins.PluginInformation; 5 6 … … public class WebpPlugin extends Plugin { 11 12 */ 12 13 public WebpPlugin(PluginInformation info) { 13 14 super(info); 15 ImageIO.scanForPlugins(); 14 16 } 15 17 }
-
follow-up: 12 comment:11 by , 2 years ago
The plugin in the current form builds.. though I would definitely want to get rid of that extra sub directory structure.
And apparently the problem was not with the plugin, but with the webp layer I was testing.. I checked with another webp layer I found on the internet.. it seems to be working fine. even without the changes you suggested. I would like to debug further as to what is going wrong with the layer I am trying.
About the suggestion regarding the Class-Path and the name, would this be a prerequisite for adding it into the main plugin repository?
comment:12 by , 2 years ago
Replying to Sreeram K <kandimalla.sreeram@…>:
The plugin in the current form builds.. though I would definitely want to get rid of that extra sub directory structure.
You probably have to run gradle
inside the lib
directory instead of ./gradlew
for your current directory structure. I generally avoid gradle
just due to versioning issues.
And apparently the problem was not with the plugin, but with the webp layer I was testing.. I checked with another webp layer I found on the internet.. it seems to be working fine. even without the changes you suggested. I would like to debug further as to what is going wrong with the layer I am trying.
I don't know then -- I wasn't seeing WebP when I was stepping through the code without the scanForPlugins
bit.
About the suggestion regarding the Class-Path and the name, would this be a prerequisite for adding it into the main plugin repository?
No. Using ant
, however, is (just due to how the JOSM CI works). If you want to do dual ant/gradle build scripts, see the mapillary or MapWithAI plugins for examples.
If you want a very basic release CI for GitHub actions, I created one for Lanes that you can use. I should look into making it a discrete repository so that changes can be more easily propagated. Right now, the only change you should have to make is to change the default branch name from master
to main
in the workflow (at the time I wrote the script, I was unable to use a variable for that).
Just keep in mind that someday you may want to add additional formats (specifically AVIF, and maybe JPEG XL), and then you would either have to refactor the plugin, add another plugin, or start fiddling with different build specifications.
comment:13 by , 2 years ago
I will look into the options you provided and get ping back here once I am done. For now I have another problem/query.. the imagery layer I had problems with, which also happens to be the reason i attempted this plugin is hosted on Google Cloud Storage by me and looks like JOSM blacklisted all "*.googleapis.*".. this seems a bit extreme. Do you know anyway for me to override this?
comment:14 by , 2 years ago
The *.googleapis.*
is actually blacklisted by the OpenStreetMap API.
See https://api.openstreetmap.org/api/0.6/capabilities (<blacklist regex=".*\.google(apis)?\..*/.*"/>
).
The only suggestion I have is to set up a domain that is not google related, and use that instead. We are not going to work around the OpenStreetMap blacklist.
With that being said, feel free to open a ticket on the openstreetmap-website repository, if you feel that the blacklist is overly broad. But I don't think it will go anywhere (just due to not wanting to chance copyright issues with Google Maps).
comment:16 by , 2 years ago
Published the plugin to the plugin wiki page. Shows up as an external plugin in the plugin list. Has been tested on windows, linux and mac. Works nicely. In the end I had to put in the scanForPlugins
line. Doesn't work consistently otherwise. Thanks for all the help @taylor.smock
comment:17 by , 2 years ago
Resolution: | → fixed |
---|---|
Status: | reopened → closed |
I'm going to go ahead and close this again since Sreeram released a plugin for webp.
comment:18 by , 19 months ago
The Sreeram's webp plugin is does not work on Apple M1 or non-x86 architectures (the docs are wrong).
https://github.com/haraldk/TwelveMonkeys as suggested earlier is a brilliant drop-in replacement for ImageIO with better compatibility and WebP support. It is also faster.
comment:19 by , 19 months ago
When I last looked at it, TwelveMonkeys didn't have support full read support for WEBP, That's why I dropped the idea( I was the one who suggested it BTW :) ). Also, about the existing plugin not working on non-x86 Apple M1, I did a spot check on my laptop.. it worked. I am guessing my JVM might have been running in x86 compatibility mode. Looks like proper WEBP support is now available in TwelveMonkeys. Maybe you can submit a pull request to my repo or start a new plugin pulling in all of the image formats supported by TwelveMonkeys?
comment:20 by , 19 months ago
Yeah, JOSM runs in x86 compatibility mode on M1 Macs, because automatic builds using github actions are complicated (https://github.com/github/roadmap/issues/528) in the way we're currently building things.
On my homemade M1 build, the webp plugin installs, but adding https://stereo.dev.openstreetmap.org/testglobe.webp as a test tile layer produces an exception:
STACK TRACE
Thread: TMS-downloader-9 (116) java.lang.NoClassDefFoundError: Could not initialize class com.luciad.imageio.webp.WebPDecoderOptions at com.luciad.imageio.webp.WebPReadParam.<init>(WebPReadParam.java:24) at com.luciad.imageio.webp.WebPReader.getDefaultReadParam(WebPReader.java:147) at java.desktop/javax.imageio.ImageIO.read(Unknown Source) at java.desktop/javax.imageio.ImageIO.read(Unknown Source) at org.openstreetmap.gui.jmapviewer.Tile.loadImage(Tile.java:247) at org.openstreetmap.josm.data.imagery.TMSCachedTileLoaderJob.tryLoadTileImage(TMSCachedTileLoaderJob.java:328) at org.openstreetmap.josm.data.imagery.TMSCachedTileLoaderJob.loadingFinished(TMSCachedTileLoaderJob.java:210) at org.openstreetmap.josm.data.cache.JCSCachedTileLoaderJob.finishLoading(JCSCachedTileLoaderJob.java:265) at org.openstreetmap.josm.data.cache.JCSCachedTileLoaderJob.run(JCSCachedTileLoaderJob.java:234) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: java.lang.ExceptionInInitializerError: Exception java.lang.UnsatisfiedLinkError: Can't load library: /var/folders/1y/hwgwq23x3jbb7cyj0j998yrh0000gp/T/6605156055280924213libwebp-imageio.dylib [in thread "JCS Downloading: https://stereo.dev.openstreetmap.org/testglobe.webp"] at java.base/java.lang.ClassLoader.loadLibrary(Unknown Source) at java.base/java.lang.Runtime.load0(Unknown Source) at java.base/java.lang.System.load(Unknown Source) at com.luciad.imageio.webp.NativeLibraryUtils.loadFromJar(WebP.java:131) at com.luciad.imageio.webp.WebP.loadNativeLibrary(WebP.java:31) at com.luciad.imageio.webp.WebP.<clinit>(WebP.java:38) at com.luciad.imageio.webp.WebPDecoderOptions.<clinit>(WebPDecoderOptions.java:20) ... 12 more
comment:21 by , 17 months ago
Resolution: | fixed |
---|---|
Status: | closed → reopened |
comment:23 by , 17 months ago
Resolution: | → fixed |
---|---|
Status: | reopened → closed |
Instead of opening an old ticket here, open a ticket for the plugin at github where it can be fixed.
comment:25 by , 16 months ago
@stoecker: I'm not going to make a binary for the plugin I just pushed to svn yet -- I want to give you a chance to make certain that the download code isn't going to kill our servers (it downloads the ImageIO dependencies off of our nexus).
I don't think it will, but I'd rather not kill our servers accidentally.
comment:27 by , 16 months ago
Replying to stoecker:
Static downloads aren't an issue for the server. ;-)
Good to know. I didn't know if the downloads were static or generated on demand (I've seen some interesting implementations for stuff before).
WebP is not supported by the JDK, see javabug:8116096. We must make sure to use only supported formats.