missing latlon causes JOSM crashes

[A_Pirard]

In January '13, a Doug Brown reported vandalism on Tagging.
His work, "shrimp pond dikes" had been erased.
He was advised to do a Revert but didn't succeed having Revert work on his JOSM.
So, I made it for him, and that produced shrimp_pond_dike.osm.
Strangely, a number of its nodes are swiped into the upper left corner.
As Doug was hesitating, I wanted to improve the file by removing the way beam.
But as I tried to remove those ways or nearby ones, JOSM was repeatedly crashing.
JOSM was staying alive, though, and that produced shrimp_pond_dike_r.osm.
Then, curious me wanted to know more, peeked into the file and I understood.
The corner nodes had no lat= lon=.
So, I added fake latlon values, the beam moved to there and JOSM stopped crashing.
Then, separating parts of the beam produced shrimp_pond_dike_OK.osm.
(unless I invert the last two files)
Doug was delighted with this file and so were the shrimps a few weeks later.

You may want to play with the first file and see while JOSM crashes if it still does.
You may want to investigate while there were no latlon.
More information on the mailing list and in the OSM history.
But, unless I forgot something, don't ask me more.
"That's all we know" ;-)
Best regards.

[Don-vip]

You're reporting a problem you've seen 8 months ago without attaching any file.
Please, we are not archaeologists. If there's a problem right now with latest JOSM tell us what with concrete material.
A lot of work has been done on core, reverter and undelete plugins in the past months to handle nodes without coordinates as they are now returned by the API after deletion.

[A_Pirard]

I report an 8 month old problem that is not mine because nobody did it, because someone said it should be done but didn't do it himself. I did not attach the files because JOSM limits attachment to 2 MB and I have had no time to make the files available yet because I have been very busy helping other people who thanked me.

If that is your kind of welcome, you can close this case.

[A_Pirard]

I uploaded compressed versions of the files.
Latest JOSM crashes when you try to move the ways beaming from the top left corner.
And it won't exit after that.

[Don-vip]

Thanks, we'll look into it. I may have been a bit harsh but you have to understand we are a small team of volunteers like you, and we do not have enough time/resources/manpower to investigate 8-month bugs without detailed information and a minimal amount of work on reporter's side :)

[skyper]

@A_Pirard:

• When were these files generated ? Eight month ago with buggy JOSM + buggy reverter or the last weeks with up-to-date versions of core and plugin ?
  • Some major bugs concerning these strange nodes were fixed over the last month.
• Is it possible to add the changeset IDs you tried to revert ?

[A_Pirard]

That's all-right, be sure that I appreciate what you do, that I love and advertise JOSM and that I'm doing my best to be helpful.

The files were generated at the time I helped Doug Brown, that's what I sent him.
​The search for "shrimp pond dikes tagging mailing list" produces this result.
The first message from Dough mentions the changeset: 11807195.
You have the full story and it's indeed unknown if the cause of the missing latlons is in OSM or JOSM.
The goal was to help Doug and his shrimps, and nobody analyzed the problem beyond what I did.
Feel free to ask any unanswered question.

Cheers.

[skyper]

Did you try to revert the changeset with up-to-date core and plugin ?

I am not sure if it is worth the effort to support such broken files.

[Don-vip]

Moving a way with a faulty node results in:

java.lang.AssertionError: null coordinates detected in node list
	at org.openstreetmap.josm.command.MoveCommand.executeCommand(MoveCommand.java:163)
	at org.openstreetmap.josm.data.UndoRedoHandler.addNoRedraw(UndoRedoHandler.java:36)
	at org.openstreetmap.josm.data.UndoRedoHandler.add(UndoRedoHandler.java:58)
	at org.openstreetmap.josm.actions.mapmode.SelectAction.updateCommandWhileDragging(SelectAction.java:696)
	at org.openstreetmap.josm.actions.mapmode.SelectAction.mouseDragged(SelectAction.java:552)

[Don-vip]

Then:

java.lang.NullPointerException
	at org.openstreetmap.josm.command.MoveCommand.updateCoordinates(MoveCommand.java:153)
	at org.openstreetmap.josm.command.MoveCommand.applyVectorTo(MoveCommand.java:121)
	at org.openstreetmap.josm.actions.mapmode.SelectAction.updateCommandWhileDragging(SelectAction.java:694)
	at org.openstreetmap.josm.actions.mapmode.SelectAction.mouseDragged(SelectAction.java:552)

And:

java.lang.NullPointerException
	at org.openstreetmap.josm.actions.mapmode.SelectAction.updateCommandWhileDragging(SelectAction.java:700)
	at org.openstreetmap.josm.actions.mapmode.SelectAction.mouseDragged(SelectAction.java:552)

[Don-vip]

In 6215/josm:

fix #9042 - Robustness to invalid .osm files containing null coordinates

[A_Pirard]

My intention in reporting this was that you didn't miss a rare case and your fix is just that.
Now (because of your interest and as I wanted to watch the news ;-)), I launched Revert.
It stopped doing anything after 12/14. No CPU (0% flat), no I/O, no network.
I Canceled, I thought it wouldn't, but it finally did.
I restarted Revert anew in a fresh layer and, after enough time, it crashed as below.
Of course, the conditions are different when everything being reverted was restored already.
Sorry for the archeology. I didn't choose the word Bugzilla ;-)

Repository Root: ​http://josm.openstreetmap.de/svn
Build-Date: 2013-07-09 01:35:09
Last Changed Author: stoecker
Revision: 6060
Repository UUID: 0c6e7542-c601-0410-84e7-c038aed88b3b
URL: ​http://josm.openstreetmap.de/svn/trunk
Last Changed Date: 2013-07-08 22:13:24 +0200 (Mon, 08 Jul 2013)
Last Changed Rev: 6060

Identification: JOSM/1.5 (6060 en) Linux Mint 13 Maya
Memory Usage: 74 MB / 1267 MB (14 MB allocated, but free)
Java version: 1.7.0_25, Oracle Corporation, OpenJDK 64-Bit Server VM
VM arguments: [-Djava.net.useSystemProxies=true]
Dataset consistency test: No problems found

Plugin: reltoolbox (29535)
Plugin: reverter (29663)
Plugin: turnrestrictions (29435)

java.lang.IllegalStateException: Missing merge target of type node with id 1,576,994,434

at reverter.DataSetCommandMerger.getMergeTarget(DataSetCommandMerger.java:53)
at reverter.DataSetCommandMerger.mergeNode(DataSetCommandMerger.java:73)
at reverter.DataSetCommandMerger.merge(DataSetCommandMerger.java:149)
at reverter.DataSetCommandMerger.<init>(DataSetCommandMerger.java:41)
at reverter.ChangesetReverter.getCommands(ChangesetReverter.java:335)
at reverter.RevertChangesetTask.realRun(RevertChangesetTask.java:88)
at org.openstreetmap.josm.gui.PleaseWaitRunnable.doRealRun(PleaseWaitRunnable.java:82)
at org.openstreetmap.josm.gui.PleaseWaitRunnable.run(PleaseWaitRunnable.java:145)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:724)