[Patch] Support URLs with other protocol than `http` or `https` for plugin sites

[floscher]

The problem I'm trying to solve

I tried to point JOSM to a custom plugin site using a file:// URL, but it didn't work. I was able to add the site, but when clicking download list, nothing happened. JOSM didn't complain, but also didn't download anything.

Why the problem exists

PluginDownloadTask and ReadRemotePluginInformationTask use HttpClient for downloading, which does seem to support file:// URLs.

How I try to solve the problem

For URLs with protocol http or https everything stays the same (i.e. HttpClient is used), but for other URLs the download happens via url.openConnection().getInputStream().

With the changes applied I can now "download" a plugin list (like the one at https://josm.openstreetmap.de/plugin) that is located on my local filesystem. And I can even "download" a plugin *.jar file from my local filesystem, which the plugin list from my local filesystem references.

My proposed changes

src/org/openstreetmap/josm/plugins/PluginDownloadTask.java

@@ -121 +121 @@
                 throw new PluginDownloadException(msg);
             }
             URL url = new URL(pi.downloadlink);
-            synchronized (this) {
-                downloadConnection = HttpClient.create(url).setAccept(PLUGIN_MIME_TYPES);
-                downloadConnection.connect();
-            }
-            try (InputStream in = downloadConnection.getResponse().getContent()) {
-                Files.copy(in, file.toPath(), StandardCopyOption.REPLACE_EXISTING);
+            Logging.debug("Download plugin {0} from {1}...", pi.name, url);
+            if ("https".equals(url.getProtocol()) || "http".equals(url.getProtocol())) {
+                synchronized (this) {
+                    downloadConnection = HttpClient.create(url).setAccept(PLUGIN_MIME_TYPES);
+                    downloadConnection.connect();
+                }
+                try (InputStream in = downloadConnection.getResponse().getContent()) {
+                    Files.copy(in, file.toPath(), StandardCopyOption.REPLACE_EXISTING);
+                }
+            } else {
+                // this is an alternative for e.g. file:// URLs where HttpClient doesn't work
+                try (InputStream in = url.openConnection().getInputStream()) {
+                    Files.copy(in, file.toPath(), StandardCopyOption.REPLACE_EXISTING);
+                }
             }
         } catch (MalformedURLException e) {
             String msg = tr("Cannot download plugin ''{0}''. Its download link ''{1}'' is not a valid URL. Skipping download.",

src/org/openstreetmap/josm/plugins/ReadRemotePluginInformationTask.java

@@ -10 +10 @@
 import java.io.FilenameFilter;
 import java.io.IOException;
 import java.io.InputStream;
+import java.io.InputStreamReader;
 import java.io.PrintWriter;
 import java.net.MalformedURLException;
 import java.net.URL;
@@ -154 +155 @@
             monitor.beginTask("");
             monitor.indeterminateSubTask(tr("Downloading plugin list from ''{0}''", printsite));
 
-            URL url = new URL(site);
-            connection = HttpClient.create(url).useCache(false);
-            final HttpClient.Response response = connection.connect();
-            content = response.fetchContent();
-            if (response.getResponseCode() != 200) {
-                throw new IOException(tr("Unsuccessful HTTP request"));
+            final URL url = new URL(site);
+            if ("https".equals(url.getProtocol()) || "http".equals(url.getProtocol())) {
+                connection = HttpClient.create(url).useCache(false);
+                final HttpClient.Response response = connection.connect();
+                content = response.fetchContent();
+                if (response.getResponseCode() != 200) {
+                    throw new IOException(tr("Unsuccessful HTTP request"));
+                }
+                return content;
+            } else {
+                // e.g. when downloading from a file:// URL, we can't use HttpClient
+                try (InputStreamReader in = new InputStreamReader(url.openConnection().getInputStream(), StandardCharsets.UTF_8)) {
+                    final StringBuilder sb = new StringBuilder();
+                    final char[] buffer = new char[8192];
+                    int numChars;
+                    while ((numChars = in.read(buffer)) >= 0) {
+                        sb.append(buffer, 0, numChars);
+                        if (canceled) {
+                            return null;
+                        }
+                    }
+                    return sb.toString();
+                }
             }
-            return content;
+
         } catch (MalformedURLException e) {
             if (canceled) return null;
             Logging.error(e);

[Don-vip]

In 12794/josm:

fix #15274 - Support URLs with other protocol than http or https for plugin sites (patch by floscher)

[Don-vip]

thank you!

[floscher]

Thank you, too!

I hope I can already put this to use in the next release of the ​gradle-josm-plugin, which should allow developers to point their JOSM to the build directory of a plugin and then "download" the current development state of their plugin for testing.

[stoecker]

Why not simply copy the plugin into the JOSM directory like done in the ant builds?

[floscher]

Then you'd need to detect, where to copy the file to. Different operating systems or even JOSM versions (josm-latest uses different JOSM_HOME directory IIRC).
Also, JOSM will then take care of getting the required plugins and activating all of them in the settings.
I think this would be a more robust solution especially for custom setups, than detecting JOSM_HOME and copying the *.jar over.

This will only be the alternative solution anyway. The gradle-josm-plugin already has a runJosm task, which sets up a separate JOSM_HOME directory inside the build directory and runs a JOSM instance independent of any installed JOSM. In that case Gradle does it like you said and copies everything necessary over and sets it up. That's the recommended way to test a JOSM plugin built with gradle-josm-plugin.

I only want to implement this for people who don't want to test in an isolated environment, but in their normal JOSM setup.

[floscher]

The second sentence of my last comment should have continued with:

…use different JOSM_HOME directories, some people have custom and/or multiple JOSM_HOME directories.