osmChange file format no longer supported for import

[rickmastfan67]

Alright, I have this older (from February) *.osm file that a friend made for me out of the 2010 TIGER data so I could use it to replace/fix/add roads to OSM where necessary locally.

Well, I had been able to open the file with no problems in JOSM up until recently. It last opened in r4264. Every release after that, it refuses to open. When I try to open it in r4272 or r4274, the following message shows up:

Could not read file 'TIGER2010_Allegheny_PA.osm'.
Error is:
Line 1 column 60: Unsupported version: 0.3 (at line 1, column 60)

Back in r4264 and previous versions, I did notice this when I opened the file in the CL:

Undefined element: 'osmChange' found in input stream. Skipping.
Undefined element: 'modify' found in input stream. Skipping.

But I didn't pay attention to it since it was "skipped".

Is this something that can be fixed in JOSM to have the file work again, or is there something I should fix in the osm file so that it can be loaded in JOSM versions later than r4264?

[bastiK]

Can you attach the file?

[rickmastfan67]

Alright, I'm attaching the Butler County 2010 TIGER file my buddy provided me. I would attach the Allegheny County one, but it's 115MB un-compressed, and only compresses down to 6.26MB (with 7-zip). However, the Butler County one compresses down to below 2MB, so I can attach it. It does the same thing as the Allegheny one; works in r4264; doesn't work in r4272 & r4274 and gives the same messages in the CL and inside of JOSM.

[rickmastfan67]

2010 TIGER Data - Butler County, PA

[rickmastfan67]

Also, I think [4268] might be the reason this is happening, since I do see "osmChange" mentioned in it.

[bastiK]

Replying to rickmastfan67:

Also, I think [4268] might be the reason this is happening, since I do see "osmChange" mentioned in it.

Right, I think we never really supported import of osc files, it is more a coincidence that this works (or used to work).

Open your file in an editor and change the version from 0.3 to 0.6. JOSM is right to reject this outdated version, this is probably a problem of the generating software. If you open ​http://api.openstreetmap.org/api/0.6/changeset/98051/download, you'll get a osc file with correct version.

[rickmastfan67]

Replying to bastiK:

Open your file in an editor and change the version from 0.3 to 0.6. JOSM is right to reject this outdated version, this is probably a problem of the generating software. If you open ​http://api.openstreetmap.org/api/0.6/changeset/98051/download, you'll get a osc file with correct version.

That did the trick. The file(s) are now loading once again in JOSM r4277.

I've also directed my friend who originally created the files for me to this ticket. He might comment on it later, but I'll mark this as "othersoftware" right now.

[rickmastfan67]

I hate to have to reopen this, but the files are now no longer opening in JOSM. When I go to open said file, JOSM says "No data found in file (file location)".

What do you think this time could have broken the file? I'll try out several older JOSM copies I have to try to nail down when the files stopped working again. It does have the " osmChange version="0.6" " line in it, so it can't be that problem.

Will update this once I find the last version the files worked in.

[rickmastfan67]

Alright, found the place where the file stopped loading in JOSM and giving the message above.

[4412] works
[4413] doesn't work

What do you think might be the problem?

[Don-vip]

It's because of the unknown tag <modify>. Delete it, and it works again.

[bastiK]

The ​osc format is different from the ​(j)osm file format. JOSM's osm parser used to be very lenient, and opened osc files by ignoring the create, modify and delete tags. The result is incorrect, but could be still useful.

We had reasons, to make the osm parser more strict. As a side affect, it won't accept the osmChange files any longer.

So we need a proper implementation of a osc importer.

[rickmastfan67]

Replying to Don-vip:

It's because of the unknown tag <modify>. Delete it, and it works again.

:facepalm: That's for letting me know that. I deleted it and it's working again.

[Don-vip]

In [4530/josm]:

see #6653 and #6960 - Allow JOSM to download osmChange files again

[bastiK]

Thanks. Why not import deleted objects as deleted primitives? I think doNothing() could cause hiccup, when there are inner elements. Use jumpToEnd(false) instead?

[Don-vip]

Replying to bastiK:

Thanks. Why not import deleted objects as deleted primitives? I think doNothing() could cause hiccup, when there are inner elements. Use jumpToEnd(false) instead?

Cause I just didn't have this bright idea :)

[Don-vip]

In [4532/josm]:

see #6653 - Import <delete> primitives as deleted primitives

[bastiK]

In [4533/josm]:

add importer for osmChange files (see #6653)

[bastiK]

close ticket?

[Don-vip]

EDIT: yes :)

[rickmastfan67]

Hate to reopen this once again, but now the files will not open at all in [4537]. I had to go back to [4529] to open the files once again. Any suggestions on making a fix to the file to get them to open in [4537] again, or will another change in JOSM be needed to fix this?

When I do open the file in [4533] and later, I get this message from JOSM:

No data found in file <file location>.

Also, the following is showing up once again in the CL:

Undefined element: 'osmChange' found in input stream. Skipping.

Meanwhile, when I load the same file in [4529], that message doesn't show up in the CL and the file opens with no problems.

I was testing this with the same "TIGER2010_Butler_PA.osm" that's attached to this ticket with the file edited with the fixes mentioned in comment:4 and comment:9 being applied to the file.

[bastiK]

change file extension to .osc

[MikeN]

For the example of the attached file, the file extension cannot be simply changed to .osc. Apparently there is a mix of utilities and one of them used the 'wrong xml markup' per current JOSM.

Instead the file extension must be left as .osm and must be edited as follows:

1. Change the first line from

<osmChange version="0.6" generator="TIGER abbrev expander">

to the following:

<?xml version='1.0' encoding='UTF-8'?>
<osm version='0.6' generator='TIGER abbrev expander'>

2. Change the last line from

</osmChange>

to

</osm>

[bastiK]

Replying to MikeN:

For the example of the attached file, the file extension cannot be simply changed to .osc. Apparently there is a mix of utilities and one of them used the 'wrong xml markup' per current JOSM.

Instead the file extension must be left as .osm and must be edited as follows:

1. Change the first line from

<osmChange version="0.6" generator="TIGER abbrev expander">

to the following:

<?xml version='1.0' encoding='UTF-8'?>
<osm version='0.6' generator='TIGER abbrev expander'>

2. Change the last line from

</osmChange>

to

</osm>

and remove <modify> / </modify>.

The attached file is broken as a OsmChange file, because it creates new objects (negative id) in a <modify> section. If <modify> is changed to <create>, this could result in a valid .osc file (untested).