DataIntegrityProblemException: Deleted node referenced while using conflation

[bagage]

What steps will reproduce the problem?

1. Configure conflation.
2. Conflate (first tab) some items
3. Reconfigure conflation by clicking the configure button and click again on "Generate matches" (no need to change anything)
4. Reconflate the same items from step 2 (which now have score=1, distance=0 as expected)

What is the expected result?

Items are conflated, nothing special happens.

What happens instead?

Exception is thrown. Actually it is worse than that because even if you ignore the exception and continue your work, once you will try to upload your change to OSM servers it will fail with "Invalid way reference -30100 cannot be found" or similar (didn't kept that).

Please provide any additional information below. Attach a screenshot if possible.

Build-Date:2017-04-04 16:27:53
Revision:11837
Is-Local-Build:true

Identification: JOSM/1.5 (11837 SVN en) Linux Debian GNU/Linux 9.0 (stretch)
Memory Usage: 710 MB / 1760 MB (494 MB allocated, but free)
Java version: 1.8.0_121-8u121-b13-4-b13, Oracle Corporation, OpenJDK 64-Bit Server VM
Screen: :0.0 1920x1080
Maximum Screen Size: 1920x1080
Java package: openjdk-8-jre:amd64-8u121-b13-4
Java ATK Wrapper package: libatk-wrapper-java:all-0.33.3-13
Program arguments: [--language=en, ${HOME}/majcadastre/CL081-CHATEAUDOUBLE/CL081-CHATEAUDOUBLE-houses-prediction_segmente.osm]
Dataset consistency test: No problems found

Plugins:
+ PicLayer (33148)
+ buildings_tools (33004)
+ conflation (0.5.3)
+ jts (32699)
+ rex (26)
+ todo (30000)
+ utilsplugin2 (33201)

Last errors/warnings:
- W: Cannot start IPv6 remotecontrol server on port 8111: Adresse déjà utilisée (Bind failed)
- W: Cannot start IPv4 remotecontrol https server on port 8112: Adresse déjà utilisée (Bind failed)
- W: Cannot start IPv6 remotecontrol https server on port 8112: Adresse déjà utilisée (Bind failed)
- W: Old style SideButton usage for action org.openstreetmap.josm.plugins.todo.TodoDialog$SelectAction@6fa6650
- W: Old style SideButton usage for action org.openstreetmap.josm.plugins.todo.TodoDialog$AddAction@4ee3c85
- W: Old style SideButton usage for action org.openstreetmap.josm.plugins.todo.TodoDialog$PassAction@1275528b
- W: Old style SideButton usage for action org.openstreetmap.josm.plugins.todo.TodoDialog$MarkAction@45fead2a
- W: Old style SideButton usage for action org.openstreetmap.josm.plugins.todo.TodoDialog$MarkSelectedAction@3cfa1eec
- W: Unable to delete archived backup file <josm.userdata>/autosave/deleted_layers/Data Layer 1_20170406_120240443.osm
- E: Handled by bug report queue: org.openstreetmap.josm.data.osm.DataIntegrityProblemException: Deleted node referenced: {Way id=-102966 version=0 MVT nodes=[{Node id=-98156 version=0 MV lat=44.8994393,lon=5.0922982}, {Node id=-98170 version=0 MVD lat=44.8994377,lon=5.0922763}, {Node id=-98172 version=0 MVD lat=44.8994314,lon=5.0922766}, {Node id=-98154 version=0 MVD lat=44.8994319,lon=5.0922985}, {Node id=-98156 version=0 MV lat=44.8994393,lon=5.0922982}]}


=== REPORTED CRASH DATA ===
BugReportExceptionHandler#handleException:
No data collected.

Warning issued by: BugReportExceptionHandler#handleException

=== STACK TRACE ===
Thread: AWT-EventQueue-0 (17) of main
org.openstreetmap.josm.data.osm.DataIntegrityProblemException: Deleted node referenced: {Way id=-102966 version=0 MVT nodes=[{Node id=-98156 version=0 MV lat=44.8994393,lon=5.0922982}, {Node id=-98170 version=0 MVD lat=44.8994377,lon=5.0922763}, {Node id=-98172 version=0 MVD lat=44.8994314,lon=5.0922766}, {Node id=-98154 version=0 MVD lat=44.8994319,lon=5.0922985}, {Node id=-98156 version=0 MV lat=44.8994393,lon=5.0922982}]}
	at org.openstreetmap.josm.data.osm.Way.checkNodes(Way.java:601)
	at org.openstreetmap.josm.data.osm.Way.fireNodesChanged(Way.java:617)
	at org.openstreetmap.josm.data.osm.Way.setDeleted(Way.java:488)
	at org.openstreetmap.josm.data.osm.OsmPrimitive.load(OsmPrimitive.java:1202)
	at org.openstreetmap.josm.data.osm.Way.load(Way.java:283)
	at org.openstreetmap.josm.command.AddPrimitivesCommand.undoCommand(AddPrimitivesCommand.java:141)
	at org.openstreetmap.josm.plugins.conflation.command.ConflateMatchCommand.buildCopyAndReplaceGeometryCommand(ConflateMatchCommand.java:167)
	at org.openstreetmap.josm.plugins.conflation.command.ConflateMatchCommand.buildCommand(ConflateMatchCommand.java:130)
	at org.openstreetmap.josm.plugins.conflation.command.ConflateMatchCommand.executeCommand(ConflateMatchCommand.java:80)
	at org.openstreetmap.josm.plugins.conflation.command.StopOnErrorSequenceCommand.executeCommand(StopOnErrorSequenceCommand.java:63)
	at org.openstreetmap.josm.data.UndoRedoHandler.addNoRedraw(UndoRedoHandler.java:53)
	at org.openstreetmap.josm.data.UndoRedoHandler.add(UndoRedoHandler.java:81)
	at org.openstreetmap.josm.plugins.conflation.ConflationToggleDialog$ConflateAction.conflateMatchActionPerformed(ConflationToggleDialog.java:815)
	at org.openstreetmap.josm.plugins.conflation.ConflationToggleDialog$ConflateAction.actualActionPerformed(ConflationToggleDialog.java:793)
	at org.openstreetmap.josm.plugins.conflation.ConflationToggleDialog$BatchAction.actionPerformed(ConflationToggleDialog.java:644)
	at javax.swing.AbstractButton.fireActionPerformed(AbstractButton.java:2022)
	at javax.swing.AbstractButton$Handler.actionPerformed(AbstractButton.java:2348)
	at javax.swing.DefaultButtonModel.fireActionPerformed(DefaultButtonModel.java:402)
	at javax.swing.DefaultButtonModel.setPressed(DefaultButtonModel.java:259)
	at javax.swing.plaf.basic.BasicButtonListener.mouseReleased(BasicButtonListener.java:252)
	at java.awt.AWTEventMulticaster.mouseReleased(AWTEventMulticaster.java:289)
	at java.awt.Component.processMouseEvent(Component.java:6533)
	at javax.swing.JComponent.processMouseEvent(JComponent.java:3324)
	at java.awt.Component.processEvent(Component.java:6298)
	at java.awt.Container.processEvent(Container.java:2236)
	at java.awt.Component.dispatchEventImpl(Component.java:4889)
	at java.awt.Container.dispatchEventImpl(Container.java:2294)
	at java.awt.Component.dispatchEvent(Component.java:4711)
	at java.awt.LightweightDispatcher.retargetMouseEvent(Container.java:4888)
	at java.awt.LightweightDispatcher.processMouseEvent(Container.java:4525)
	at java.awt.LightweightDispatcher.dispatchEvent(Container.java:4466)
	at java.awt.Container.dispatchEventImpl(Container.java:2280)
	at java.awt.Window.dispatchEventImpl(Window.java:2746)
	at java.awt.Component.dispatchEvent(Component.java:4711)
	at java.awt.EventQueue.dispatchEventImpl(EventQueue.java:758)
	at java.awt.EventQueue.access$500(EventQueue.java:97)
	at java.awt.EventQueue$3.run(EventQueue.java:709)
	at java.awt.EventQueue$3.run(EventQueue.java:703)
	at java.security.AccessController.doPrivileged(Native Method)
	at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:80)
	at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:90)
	at java.awt.EventQueue$4.run(EventQueue.java:731)
	at java.awt.EventQueue$4.run(EventQueue.java:729)
	at java.security.AccessController.doPrivileged(Native Method)
	at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:80)
	at java.awt.EventQueue.dispatchEvent(EventQueue.java:728)
	at java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:201)
	at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:116)
	at java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:105)
	at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:101)
	at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:93)
	at java.awt.EventDispatchThread.run(EventDispatchThread.java:82)

[Tyndare]

Thanks for this bug report. I'll investigate.

[Tyndare]

AddPrimitivesCommand undo ordering pb patch

[Tyndare]

I think the problem is related to a modification I proposed to the AddPrimitivesCommand in JOSM core.
Are you able to apply the attached patch to JOSM core sources file and check that is solve the problem for you ?

[bagage]

Yes it seems so. At least the exception is gone, thanks!

[bagage]

Hey Tyndare, could you merge the branch upstream and close the bug? Thanks!

[GerdP]

Seems there was a missunderstanding here. The patch is for core, and it was never applied.

[GerdP]

@Tyndare: I try to create a unit test that shows the problem. Failed so far.
I think one problem in the unpatched code is that it only compares unique ids when looking for existing data. Maybe that fails when you have a way and node with the same id?

[GerdP]

sample session to reproduce the problem

[GerdP]

In 17240/josm:

fix #14620: DataIntegrityProblemException: Deleted node referenced while using conflation
The plugin shows an error in the core class AddPrimitivesCommand.

• AddPrimitivesCommand undo crashes with existing data with status "deleted" Patch by Tyndare, modified
• add stability for primitives with the same unique id but different types, like a way and a node with the same id.
• add unit tests for both

[GerdP]

was a regression from r17240 r11609

[GerdP]

Ticket #16962 has been marked as a duplicate of this ticket.

[skyper]

Replying to GerdP:

was a regression from r17240

Typo?

[GerdP]

Thanks. Too many numbers ;)

[Don-vip]

Milestone renamed