Modify

Opened 13 days ago

Closed 13 days ago

Last modified 13 days ago

#15274 closed enhancement (fixed)

[Patch] Support URLs with other protocol than `http` or `https` for plugin sites

Reported by: floscher Owned by: team
Priority: normal Milestone: 17.09
Component: Core Version:
Keywords: Cc:

Description

The problem I'm trying to solve

I tried to point JOSM to a custom plugin site using a file:// URL, but it didn't work. I was able to add the site, but when clicking download list, nothing happened. JOSM didn't complain, but also didn't download anything.

Why the problem exists

PluginDownloadTask and ReadRemotePluginInformationTask use HttpClient for downloading, which does seem to support file:// URLs.

How I try to solve the problem

For URLs with protocol http or https everything stays the same (i.e. HttpClient is used), but for other URLs the download happens via url.openConnection().getInputStream().

With the changes applied I can now "download" a plugin list (like the one at https://josm.openstreetmap.de/plugin) that is located on my local filesystem. And I can even "download" a plugin *.jar file from my local filesystem, which the plugin list from my local filesystem references.

My proposed changes

  • src/org/openstreetmap/josm/plugins/PluginDownloadTask.java

    diff --git a/src/org/openstreetmap/josm/plugins/PluginDownloadTask.java b/src/org/openstreetmap/josm/plugins/PluginDownloadTask.java
    index 40231cb4c5..fcb5858867 100644
    a b public class PluginDownloadTask extends PleaseWaitRunnable { 
    121121                throw new PluginDownloadException(msg);
    122122            }
    123123            URL url = new URL(pi.downloadlink);
    124             synchronized (this) {
    125                 downloadConnection = HttpClient.create(url).setAccept(PLUGIN_MIME_TYPES);
    126                 downloadConnection.connect();
    127             }
    128             try (InputStream in = downloadConnection.getResponse().getContent()) {
    129                 Files.copy(in, file.toPath(), StandardCopyOption.REPLACE_EXISTING);
     124            Logging.debug("Download plugin {0} from {1}...", pi.name, url);
     125            if ("https".equals(url.getProtocol()) || "http".equals(url.getProtocol())) {
     126                synchronized (this) {
     127                    downloadConnection = HttpClient.create(url).setAccept(PLUGIN_MIME_TYPES);
     128                    downloadConnection.connect();
     129                }
     130                try (InputStream in = downloadConnection.getResponse().getContent()) {
     131                    Files.copy(in, file.toPath(), StandardCopyOption.REPLACE_EXISTING);
     132                }
     133            } else {
     134                // this is an alternative for e.g. file:// URLs where HttpClient doesn't work
     135                try (InputStream in = url.openConnection().getInputStream()) {
     136                    Files.copy(in, file.toPath(), StandardCopyOption.REPLACE_EXISTING);
     137                }
    130138            }
    131139        } catch (MalformedURLException e) {
    132140            String msg = tr("Cannot download plugin ''{0}''. Its download link ''{1}'' is not a valid URL. Skipping download.",
  • src/org/openstreetmap/josm/plugins/ReadRemotePluginInformationTask.java

    diff --git a/src/org/openstreetmap/josm/plugins/ReadRemotePluginInformationTask.java b/src/org/openstreetmap/josm/plugins/ReadRemotePluginInformationTask.java
    index 324a33121a..aa3012b083 100644
    a b import java.io.File; 
    1010import java.io.FilenameFilter;
    1111import java.io.IOException;
    1212import java.io.InputStream;
     13import java.io.InputStreamReader;
    1314import java.io.PrintWriter;
    1415import java.net.MalformedURLException;
    1516import java.net.URL;
    public class ReadRemotePluginInformationTask extends PleaseWaitRunnable { 
    154155            monitor.beginTask("");
    155156            monitor.indeterminateSubTask(tr("Downloading plugin list from ''{0}''", printsite));
    156157
    157             URL url = new URL(site);
    158             connection = HttpClient.create(url).useCache(false);
    159             final HttpClient.Response response = connection.connect();
    160             content = response.fetchContent();
    161             if (response.getResponseCode() != 200) {
    162                 throw new IOException(tr("Unsuccessful HTTP request"));
     158            final URL url = new URL(site);
     159            if ("https".equals(url.getProtocol()) || "http".equals(url.getProtocol())) {
     160                connection = HttpClient.create(url).useCache(false);
     161                final HttpClient.Response response = connection.connect();
     162                content = response.fetchContent();
     163                if (response.getResponseCode() != 200) {
     164                    throw new IOException(tr("Unsuccessful HTTP request"));
     165                }
     166                return content;
     167            } else {
     168                // e.g. when downloading from a file:// URL, we can't use HttpClient
     169                try (InputStreamReader in = new InputStreamReader(url.openConnection().getInputStream(), StandardCharsets.UTF_8)) {
     170                    final StringBuilder sb = new StringBuilder();
     171                    final char[] buffer = new char[8192];
     172                    int numChars;
     173                    while ((numChars = in.read(buffer)) >= 0) {
     174                        sb.append(buffer, 0, numChars);
     175                        if (canceled) {
     176                            return null;
     177                        }
     178                    }
     179                    return sb.toString();
     180                }
    163181            }
    164             return content;
     182
    165183        } catch (MalformedURLException e) {
    166184            if (canceled) return null;
    167185            Logging.error(e);

Attachments (1)

file-URLs.patch (4.4 KB) - added by floscher 13 days ago.

Download all attachments as: .zip

Change History (8)

Changed 13 days ago by floscher

Attachment: file-URLs.patch added

comment:1 Changed 13 days ago by Don-vip

Milestone: 17.09

comment:2 Changed 13 days ago by Don-vip

Resolution: fixed
Status: newclosed

In 12794/josm:

fix #15274 - Support URLs with other protocol than http or https for plugin sites (patch by floscher)

comment:3 Changed 13 days ago by Don-vip

thank you!

comment:4 Changed 13 days ago by floscher

Thank you, too!

I hope I can already put this to use in the next release of the gradle-josm-plugin, which should allow developers to point their JOSM to the build directory of a plugin and then "download" the current development state of their plugin for testing.

comment:5 Changed 13 days ago by stoecker

Why not simply copy the plugin into the JOSM directory like done in the ant builds?

comment:6 Changed 13 days ago by floscher

Then you'd need to detect, where to copy the file to. Different operating systems or even JOSM versions (josm-latest uses different JOSM_HOME directory IIRC).
Also, JOSM will then take care of getting the required plugins and activating all of them in the settings.
I think this would be a more robust solution especially for custom setups, than detecting JOSM_HOME and copying the *.jar over.

This will only be the alternative solution anyway. The gradle-josm-plugin already has a runJosm task, which sets up a separate JOSM_HOME directory inside the build directory and runs a JOSM instance independent of any installed JOSM. In that case Gradle does it like you said and copies everything necessary over and sets it up. That's the recommended way to test a JOSM plugin built with gradle-josm-plugin.

I only want to implement this for people who don't want to test in an isolated environment, but in their normal JOSM setup.

comment:7 Changed 13 days ago by floscher

The second sentence of my last comment should have continued with:

…use different JOSM_HOME directories, some people have custom and/or multiple JOSM_HOME directories.

Modify Ticket

Change Properties
Set your email in Preferences
Action
as closed The owner will remain team.
as The resolution will be set.
The resolution will be deleted.

Add Comment


E-mail address and name can be saved in the Preferences.

 
Note: See TracTickets for help on using tickets.