DataIntegrityProblemException: Primitive must be part of the dataset

[anonymous]

After the checkin for ticket #2801, I am getting DataIntegrityProblemExceptions in some cases.

The stack trace is as follows:

org.openstreetmap.josm.data.osm.DataIntegrityProblemException: Primitive must be part of the dataset: {Node id=-641 version=0 V lat=49.60200676334942,lon=6.554085275435658}

at org.openstreetmap.josm.data.osm.OsmPrimitive.checkDataset(OsmPrimitive.java:169)
at org.openstreetmap.josm.data.osm.OsmPrimitive.getReferrers(OsmPrimitive.java:946)
at org.openstreetmap.josm.data.osm.Node.isJunctionNode(Node.java:200)
at org.openstreetmap.josm.gui.mappaint.SimpleNodeElemStyle.paintPrimitive(SimpleNodeElemStyle.java:28)
at org.openstreetmap.josm.data.osm.visitor.paint.MapPaintVisitor.drawNode(MapPaintVisitor.java:132)
at org.openstreetmap.josm.data.osm.visitor.paint.MapPaintVisitor.visitAll(MapPaintVisitor.java:748)
at org.openstreetmap.josm.gui.layer.OsmDataLayer.paint(OsmDataLayer.java:268)
at org.openstreetmap.josm.gui.MapView.paint(MapView.java:517)

This bug seems to be reproducible in the following way:

Take a way. Then insert a node in the middle of the way by dragging one of the '+' of the way. When you then undo the operation with ctrl-z, the problem occurs. However, it does not work for all ways, or all '+' in a way, but for those where it does, it is repeatable. One way for which it does work is Way 24545038. (Although if you only load that way and nothing else, it doesn't crash. So what else is loaded also seems to matter)

[anonymous]

I haven't managed to track it down fully yet, but here are some observations that might help find this bug

The problem seems to be in DataSet.reindexNode(). In some cases the number of nodes increases by one during the reindex. This then means that in the undo operation, only one of the instances is removed from the quadBucket, leaving one still in there which then causes the DataIntegrityProblem.

in reindexNode, the removeNode fails and as the return value isn't checked, the node is added a second time even though the removed didn't succeed.

I haven't figured out yet why the removeNode fails though

[bastiK]

Do you have another example, because way 24545038 is kind of strange. It has node list A,B,A,C. (Nevertheless it shouldn't cause an exception...)

[bastiK]

Never mind, I fixed it in a text editor and I still get the exception.

[bastiK]

[3154] broke quadbuckets: A node is not removed from the tree, because it looks in the bucket for the new coordinates but instead the node is still in the bucket with the old (unknown) coordinates.

Then it tries to paint the zombie node which is no longer in the data set.

[jttt]

QuadBuckets is fine, but Dataset.reindexNode is broken (and always been). It should first remove node, update position, add node.

[bastiK]

(In [3163]) fixed #4815 (after [3154] you cannot remove a node from quadbuckets if the coordinates have changed -> first remove the node from quadbuckets then change the coordinates then add it back)

[anonymous]

Ticket #4817 has been marked as a duplicate of this ticket.