Modify

Opened 11 years ago

Closed 2 years ago

#8442 closed defect (invalid)

OSM objects parameter via stdin are not XML since 2 XML documents

Reported by: jjaf.de Owned by: Hind
Priority: major Milestone:
Component: Plugin commandline Version: tested
Keywords: XML header footer parsing error Cc:

Description

Problem

While looking for a automatisation for WIWOSM-project i came to CommandLine.
When selecting an OSM object to be passed as parameter via stdin to a script you get two XML documents instead of one, e.g.:

<?xml version='1.0' encoding='UTF-8'?>
<osm version='0.6' generator='JOSM'>
</osm>
<?xml version='1.0' encoding='UTF-8'?>
<osm version='0.6' generator='JOSM'>
  <relation id='253676' timestamp='2009-09-18T01:18:49Z' uid='120146' user='TIGERcnl' visible='true' version='1' changeset='2518319'>
[...]
</osm>

This of course does not parse.

Cause

http://svn.openstreetmap.org/applications/editors/josm/plugins/CommandLine/src/CommandLine/CommandLine.java Line 520 has unconditional

osmWriter.footer();

for ending referenced objects before conditional

osmWriter.header();
[...]
osmWriter.footer();

for parameters.

Resolution

Leave out the conditional statements and put unconditional footer at the end of parameters processing.

Attachments (2)

8442.patch (40.9 KB ) - added by Don-vip 11 years ago.
8442v2.patch (51.8 KB ) - added by Don-vip 11 years ago.
new patch

Download all attachments as: .zip

Change History (18)

by Don-vip, 11 years ago

Attachment: 8442.patch added

comment:1 by Don-vip, 11 years ago

Summary: OSM objects parameter via stdin are not XML since 2 XML documents[PATCH] OSM objects parameter via stdin are not XML since 2 XML documents

This should do the trick (needs testing however before committing)

comment:2 by Don-vip, 11 years ago

Mmm patch is quite unreadable because of automatic indentation. Here's the new piece of code:

                osmWriter.header();
                DataSet ds = new DataSet();
                for (OsmPrimitive primitive : refObjects) {
                    ds.addPrimitive(primitive);
                    if (bbox == null)
                        bbox = new BBox(primitive.getBBox());
                    else
                        bbox.addPrimitive(primitive, 0.0);
                }
                for (Parameter parameter : parameters) {
                    if (!parameter.isOsm())
                        continue;
                    pObjects = parameter.getParameterObjects();
                    for (OsmPrimitive primitive : pObjects) {
                        ds.addPrimitive(primitive);
                        if (bbox == null)
                            bbox = new BBox(primitive.getBBox());
                        else
                            bbox.addPrimitive(primitive, 0.0);
                    }
                }
                osmWriter.writeContent(ds);
                osmWriter.footer();
                osmWriter.flush();

comment:3 by Don-vip, 11 years ago

Can you see if it works ? (by replacing the plugin jar). Thanks :)

comment:4 by jjaf.de, 11 years ago

Sorry when selecting one or more OSM objects to be passed as parameter it throws:

Repository Root: http://josm.openstreetmap.de/svn
Build-Date: 2013-02-06 02:31:34
Last Changed Author: Don-vip
Revision: 5697
Repository UUID: 0c6e7542-c601-0410-84e7-c038aed88b3b
URL: http://josm.openstreetmap.de/svn/trunk
Last Changed Date: 2013-02-06 01:35:48 +0100 (Wed, 06 Feb 2013)
Last Changed Rev: 5697

Identification: JOSM/1.5 (5697 de)
Memory Usage: 82 MB / 247 MB (30 MB allocated, but free)
Java version: 1.7.0_15, Oracle Corporation, Java HotSpot(TM) Client VM
Operating system: Windows XP
Dataset consistency test: No problems found

Plugin: CommandLine (29210)
Plugin: FixAddresses (29210)
Plugin: PicLayer (29210)
Plugin: ext_tools (29210)
Plugin: jts (28945)
Plugin: mirrored_download (29210)
Plugin: namemanager (29210)
Plugin: openstreetbugs (29210)
Plugin: pdfimport (29210)
Plugin: restart (29210)
Plugin: reverter (29216)
Plugin: tag2link (29210)
Plugin: terracer (29210)
Plugin: turnrestrictions (29210)
Plugin: waydownloader (29210)
Plugin: wayselector (29210)
Plugin: wikipedia (29218)

org.openstreetmap.josm.data.osm.DataIntegrityProblemException: Primitive cannot be included in more than one Dataset
	at org.openstreetmap.josm.data.osm.OsmPrimitive.setDataset(OsmPrimitive.java:309)
	at org.openstreetmap.josm.data.osm.Node.setDataset(Node.java:189)
	at org.openstreetmap.josm.data.osm.DataSet.addPrimitive(DataSet.java:359)
	at CommandLine.CommandLine$3.run(CommandLine.java:518)
	at java.lang.Thread.run(Unknown Source)

comment:5 by Don-vip, 11 years ago

In 5737/josm:

see #8442 - Add write methods to OsmWriter to not rely only on DataSet

by Don-vip, 11 years ago

Attachment: 8442v2.patch added

new patch

comment:6 by Don-vip, 11 years ago

new piece of code:

                Collection<OsmPrimitive> contents = new ArrayList<OsmPrimitive>();
                for (OsmPrimitive primitive : refObjects) {
                    contents.add(primitive);
                    if (bbox == null)
                        bbox = new BBox(primitive.getBBox());
                    else
                        bbox.addPrimitive(primitive, 0.0);
                }
                for (Parameter parameter : parameters) {
                    if (!parameter.isOsm())
                        continue;
                    pObjects = parameter.getParameterObjects();
                    for (OsmPrimitive primitive : pObjects) {
                        contents.add(primitive);
                        if (bbox == null)
                            bbox = new BBox(primitive.getBBox());
                        else
                            bbox.addPrimitive(primitive, 0.0);
                    }
                }
                osmWriter.writeNodes(new SubclassFilteredCollection<OsmPrimitive, Node>(contents, OsmPrimitive.nodePredicate));
                osmWriter.writeWays(new SubclassFilteredCollection<OsmPrimitive, Way>(contents, OsmPrimitive.wayPredicate));
                osmWriter.writeRelations(new SubclassFilteredCollection<OsmPrimitive, Relation>(contents, OsmPrimitive.relationPredicate));
                osmWriter.footer();
                osmWriter.flush();

comment:7 by Don-vip, 11 years ago

And new jar. Can you test it with tomorrow's latest ? (r5737+). Thanks.

comment:8 by jjaf.de, 11 years ago

Test on 5737/josm with above patch positive!
Tested one and several relations without and and the same with referenced objects.

comment:9 by Don-vip, 11 years ago

Resolution: fixed
Status: newclosed
Summary: [PATCH] OSM objects parameter via stdin are not XML since 2 XML documentsOSM objects parameter via stdin are not XML since 2 XML documents

Fixed in [o29273]

comment:10 by jjaf.de, 10 years ago

Resolution: fixed
Status: closedreopened

comment:11 by jjaf.de, 10 years ago

I believe some later changes made this symptom reappear.
For this case i did not delete the workaround JOSMticket8442() in http://osm.jjaf.de/wiwosm/bot/?source

comment:12 by Don-vip, 9 years ago

Indeed my change has been reverted in r29817/osm.

@Hind, can you please see with with jjaf if he's using the plugin as you would expect? I though my fix was the way to handle this issue, it looks like not :)

Last edited 2 years ago by stoecker (previous) (diff)

in reply to:  12 ; comment:13 by Hind, 9 years ago

Hi all :3
First XML document contains dependences for parameters which type is OSM object.

In total, CommandLine passes 1 + N XMLs to external tool, where N is the number of such parameters.

comment:14 by taylor.smock, 2 years ago

Stupid question: is this still a problem? Or can we close this issue?

comment:15 by jjaf.de, 2 years ago

Thank you taylor.smock for trying to close this ticket. Sadly there seems to be an issue here still.

CommandLine test command

I created a test command for CommandLine that dumps the input to a file for debugging. Put these two files in the CommandLine-plugin directory (e.g. ~/.local/share/JOSM/plugins/CommandLine/) and modify output filename /tmp/stdin2file.out if required:

stdin2file.xml

<?xml version="1.0" encoding="UTF-8"?>
<command version="1" name="stdin2file" run="python stdin2file.py">
        <parameter required="true" type="any" maxinstances="0">
                <name>Objects</name>
                <description>Objects to write to file</description>
        </parameter>
</command>

stdin2file.py

#!/usr/bin/env python

import sys
import fileinput

file = open("/tmp/stdin2file.out", "w") 

for line in fileinput.input():
        file.write(line)

file.close()

Test

Tests where conducted with JOSM 18427 and CommandLine 35951.

node

Creating a new node and sending it to the command results in defect xml having to xml documents concatenated the first one empty:

<?xml version='1.0' encoding='UTF-8'?>
<osm version='0.6' generator='JOSM'>
</osm>
<?xml version='1.0' encoding='UTF-8'?>
<osm version='0.6' generator='JOSM'>
  <node id='-101762' visible='true' lat='52.36710389541' lon='9.73775417476' />
</osm>

way

Creating a new way shows that the first xml document is not always empty having node separated from way.

<?xml version='1.0' encoding='UTF-8'?>
<osm version='0.6' generator='JOSM'>
  <node id='-101761' visible='true' lat='52.3671080428' lon='9.7377493544' />
  <node id='-101762' visible='true' lat='52.36710389541' lon='9.73775417476' />
</osm>
<?xml version='1.0' encoding='UTF-8'?>
<osm version='0.6' generator='JOSM'>
  <way id='-101783' visible='true'>
    <nd ref='-101761' />
    <nd ref='-101762' />
  </way>
</osm>

relation

Also with relations the first document is not empty but unlike a way it contains the whole way (node and way) but separated from relation:

<?xml version='1.0' encoding='UTF-8'?>
<osm version='0.6' generator='JOSM'>
  <node id='-101761' visible='true' lat='52.3671080428' lon='9.7377493544' />
  <node id='-101762' visible='true' lat='52.36710389541' lon='9.73775417476' />
  <way id='-101783' visible='true'>
    <nd ref='-101761' />
    <nd ref='-101762' />
  </way>
</osm>
<?xml version='1.0' encoding='UTF-8'?>
<osm version='0.6' generator='JOSM'>
  <relation id='-99762' visible='true'>
    <member type='way' ref='-101783' role='' />
    <tag k='type' v='test' />
  </relation>
</osm>

in reply to:  13 comment:16 by jjaf.de, 2 years ago

Resolution: invalid
Status: reopenedclosed

Replying to Hind:

First XML document contains dependences for parameters which type is OSM object.

In total, CommandLine passes 1 + N XMLs to external tool, where N is the number of such parameters.

Seems to me CommandLine does behave like given in this description but it is not described that explicitly in the linked wiki article. If this is really the correct description then one could close this bug and have to handle two XML documents coming on the same stream in the receiving application.

Modify Ticket

Change Properties
Set your email in Preferences
Action
as closed The owner will remain Hind.
as The resolution will be set.
The resolution will be deleted. Next status will be 'reopened'.

Add Comment


E-mail address and name can be saved in the Preferences .
 
Note: See TracTickets for help on using tickets.