Modify

Opened 9 months ago

Closed 9 months ago

#16400 closed defect (fixed)

continuosDownload want JOSM version 13.927.

Reported by: anonymous Owned by: team
Priority: normal Milestone:
Component: Plugin continuosDownload Version: tested
Keywords: template_report github git Cc: stoecker, floscher, michael2402, Gubaer

Description

What steps will reproduce the problem?

  1. Update plugins i tested version 13878.

What is the expected result?

Ok with tested JOSM version.

What happens instead?

Want new JOSM vesion.

Please provide any additional information below. Attach a screenshot if possible.


URL:https://josm.openstreetmap.de/svn/trunk
Repository:UUID: 0c6e7542-c601-0410-84e7-c038aed88b3b
Last:Changed Date: 2018-05-31 00:51:20 +0200 (Thu, 31 May 2018)
Build-Date:2018-05-30 22:58:01
Revision:13878
Relative:URL: ^/trunk

Identification: JOSM/1.5 (13878 da) Linux Ubuntu 17.10
Memory Usage: 447 MB / 843 MB (161 MB allocated, but free)
Java version: 1.8.0_171-8u171-b11-0ubuntu0.17.10.1-b11, Oracle Corporation, OpenJDK 64-Bit Server VM
Screen: :0.0 1280x1024
Maximum Screen Size: 1280x1024
Java package: openjdk-8-jre:amd64-8u171-b11-0ubuntu0.17.10.1
Java ATK Wrapper package: libatk-wrapper-java:all-0.33.3-13
VM arguments: [-Djosm.restart=true, -Djava.net.useSystemProxies=true]

Plugins:
+ Mapillary (v1.5.14+post13733)
+ PicLayer (34170)
+ apache-commons (34109)
+ apache-http (34109)
+ continuosDownload (68)
+ reverter (34109)
+ scripting (30785)
+ turnrestrictions (34129)
+ utilsplugin2 (34221)

Map paint styles:
+ https://josm.openstreetmap.de/josmfile?page=Styles/Coloured_Streets&zip=1

Last errors/warnings:
- W: No configuration settings found.  Using hardcoded default values for all pools.
- E: org.openstreetmap.josm.plugins.PluginDownloadException: Download sprunget over

Attachments (2)

screendump.png (272.9 KB) - added by anonymous 9 months ago.
screendump.2.png (120.3 KB) - added by anonymous 9 months ago.

Download all attachments as: .zip

Change History (21)

Changed 9 months ago by anonymous

Attachment: screendump.png added

Changed 9 months ago by anonymous

Attachment: screendump.2.png added

comment:1 Changed 9 months ago by anonymous


comment:2 Changed 9 months ago by Don-vip

Arf, at some point the expected format has not been respected:

<xxx>_Plugin-Url entries, the format is <josm_version>_Plugin-Url: <plugin_version>;<url>. Here, <josm_version> denotes the lowest JOSM version, the plugin <plugin_version> will work with and it can be downloaded using <url>.

And the error propagated to many Github plugins. We have a lot of plugins now using this format:

	11609_Plugin-Url: https://github.com/JOSM/conflation/releases/download/v0.5.5/conflation.jar
	12859_Plugin-Url: https://github.com/JOSM/conflation/releases/download/v0.5.6/conflation.jar
	13561_Plugin-Url: https://github.com/JOSM/conflation/releases/download/v0.5.7/conflation.jar

	7588_Plugin-Url: https://github.com/Gnonthgol/JOSM-continuos-download/releases/download/v1.1.01/continuosDownload.jar
	7636_Plugin-Url: https://github.com/Gnonthgol/JOSM-continuos-download/releases/download/v1.1.02/continuosDownload.jar
	8942_Plugin-Url: https://github.com/JOSM/JOSM-continuos-download/releases/download/v1.2.2/continuosDownload.jar
	10282_Plugin-Url: https://github.com/JOSM/JOSM-continuos-download/releases/download/v1.2.3/continuosDownload.jar
	10580_Plugin-Url: https://github.com/JOSM/JOSM-continuos-download/releases/download/v1.2.4/continuosDownload.jar
	12643_Plugin-Url: https://github.com/JOSM/JOSM-continuos-download/releases/download/v1.2.5/continuosDownload.jar
	12840_Plugin-Url: https://github.com/JOSM/JOSM-continuos-download/releases/download/v1.2.6/continuosDownload.jar

	10580_Plugin-Url: https://github.com/JOSM/todo/releases/download/v30300/conflation.jar

	13170_Plugin-Url: https://github.com/maripo/JOSM_quicklabel/releases/download/v1.6.1/QuickLabel.jar
	12545_Plugin-Url: https://github.com/maripo/JOSM_quicklabel/releases/download/v1.0.1/QuickLabel.jar

comment:3 Changed 9 months ago by Don-vip

Cc: stoecker floscher added
Keywords: github git added

In fact with more and more plugin developers using Git/Github the update mechanism is broken for them, as they often don't use a numeric version, this is what we have right now:

        Plugin-Version: 0.0.1
        Plugin-Version: 0.5.9
        Plugin-Version: 1.0.203
        Plugin-Version: 1.1.0
        Plugin-Version: 56d872a
        Plugin-Version: UNKNOWN
        Plugin-Version: v0.4.2
        Plugin-Version: v0.4.3
        Plugin-Version: v0.9.2
        Plugin-Version: v1.0.1
        Plugin-Version: v1.5.14+post13733
        Plugin-Version: v2.4.0

What should we do?

comment:4 in reply to:  3 ; Changed 9 months ago by stoecker

Replying to Don-vip:

What should we do?

Change the server parser and reject wrong format and version (i.e. drop invalid plugins from list). That forces the authors to fix their code.

Last edited 9 months ago by stoecker (previous) (diff)

comment:5 in reply to:  4 ; Changed 9 months ago by floscher

Cc: michael2402 added

Replying to stoecker:

Replying to Don-vip:

What should we do?

Change the server parser and reject wrong format and version (i.e. drop invalid plugins from list). That forces the authors to fix their code.

Yes, regarding rejecting the wrong format I'm with you. Every plugin should respect the format <minJosmVersion>_Plugin-Url: <pluginVersion>;<pluginUrl>

As a sidenote: The plugins built with Gradle all use the correct manifest format, since there is a dedicated method for formatting these manifest entries: (see documentation).

About the version number format: When I started using semantic version numbers for plugins, I researched how JOSM handles the plugin version numbers and found that it more or less only cares if the local version is different than the remote one. I didn't find the server code, is it publicly available somewhere? Does the server code do anything like comparing different version numbers, where the script needs to interpret the version number?

That the plugins using git also use semantic versions is motivated by the fundamental differences between git and SVN. Just counting commits since "the beginning" as version number is considered a code smell in the git-world. The server script should be made more robust against semantic versions instead of forbidding all non-numeric version numbers.

I'll fix the 56d872a version (pt_assistant) tomorrow, the UNKNOWN version is from MicrosoftStreetside.

comment:6 Changed 9 months ago by Don-vip

In PluginInformation, this code doesn't work I think:

        if (oldcheck && mainversion > Version.getInstance().getVersion()) {
            int myv = Version.getInstance().getVersion();
            for (Map.Entry<Object, Object> entry : attr.entrySet()) {
                try {
                    String key = ((Attributes.Name) entry.getKey()).toString();
                    if (key.endsWith("_Plugin-Url")) {
                        int mv = Integer.parseInt(key.substring(0, key.length()-11));
                        if (mv <= myv && (mv > mainversion || mainversion > myv)) {
                            String v = (String) entry.getValue();
                            int i = v.indexOf(';');
                            if (i > 0) {
                                downloadlink = v.substring(i+1);
                                mainversion = mv;
                                version = v.substring(0, i);
                                oldmode = true;
                            }
                        }
                    }
                } catch (NumberFormatException e) {
                    Logging.error(e);
                }
            }
        }
Last edited 9 months ago by Don-vip (previous) (diff)

comment:7 Changed 9 months ago by floscher

Cc: Gubaer added

The contourmerge plugin should be also fixed soon, there will be a release because of another issue. The plugin switched to Gradle, but hasn't made a release since.

comment:8 Changed 9 months ago by floscher

Ok, current progress:

  • invalid manifest entries
  • version numbers that are neither numeric nor semantic versions:
    • MicrosoftStreetside: published *.jar seems to be fixed, don't know where and how
    • pt_assistant: fixed in 26cec6a, will be in effect with next release

I think as soon as these are all fixed, this issue can be closed. We should discuss any fixes the server code needs to handle unusual input in a separate ticket. The issue that was reported here should be solved as soon as all plugins use the correct format for *_Plugin-Url.

@DonVip: I think the code snippet you posted should work correctly, it ignores any *_Plugin-Url entries that don't have a semicolon in the value. The issue reported in this ticket occured because JOSM handled the invalid manifest entries correctly by ignoring them.

Last edited 9 months ago by floscher (previous) (diff)

comment:9 in reply to:  6 ; Changed 9 months ago by stoecker

Replying to Don-vip:

In PluginInformation, this code doesn't work I think:

Why do you think so?

comment:10 in reply to:  5 Changed 9 months ago by stoecker

About the version number format: When I started using semantic version numbers for plugins, I researched how JOSM handles the plugin version numbers and found that it more or less only cares if the local version is different than the remote one.

Yes. That's the only requirement. Because it never worked to establish a common versioning method.

Anyway I personally don't like the "v" in some of the plugin versions as well as git hashes. That's ugly.

I didn't find the server code, is it publicly available somewhere?

No. That's private. There is no reason to make it public as only the admins work with it and I don't want to make it too easy to find possible security holes.

Does the server code do anything like comparing different version numbers, where the script needs to interpret the version number?

The server code only extracts the data. It does not care much about it except, that it expects that version numbers are actually CHANGING :-)

Last edited 9 months ago by stoecker (previous) (diff)

comment:11 in reply to:  9 Changed 9 months ago by Don-vip

Replying to stoecker:

Replying to Don-vip:

In PluginInformation, this code doesn't work I think:

Why do you think so?

Ah I read too fast the part Integer.parseInt(key.substring(0, key.length()-11)); I thought it was extracting the first field of the value, and not the first part of the key.

comment:12 Changed 9 months ago by stoecker

@Vincent:

Add Manifest validity checks to plugin integration test?

comment:13 Changed 9 months ago by Don-vip

Good idea!

comment:14 Changed 9 months ago by floscher

The *_Plugin-Url manifest entries and the version numbers that are neither integers nor semantic versions are now all fixed.

comment:15 Changed 9 months ago by Don-vip

Ticket #16435 has been marked as a duplicate of this ticket.

comment:16 Changed 9 months ago by Don-vip

Ticket #16446 has been marked as a duplicate of this ticket.

comment:17 Changed 9 months ago by floscher

@Don-Vip can you release a new version of the continuousDownload plugin? The fix for the manifest entries is not in the current latest released version.

comment:18 Changed 9 months ago by Don-vip

In 13993/josm:

see #16400 - update integration test to detect invalid manifest entries

comment:19 Changed 9 months ago by Don-vip

Resolution: fixed
Status: newclosed

Modify Ticket

Change Properties
Set your email in Preferences
Action
as closed The owner will remain team.
as The resolution will be set.
The resolution will be deleted.

Add Comment


E-mail address and name can be saved in the Preferences.

 
Note: See TracTickets for help on using tickets.