#19438 closed defect (fixed)
DataIntegrityProblemException: Deleted node referenced during conflict solving
| Reported by: | Owned by: | GerdP | |
|---|---|---|---|
| Priority: | normal | Milestone: | 20.11 |
| Component: | Core | Version: | tested |
| Keywords: | template_report DataIntegrityProblemException Deleted node referenced conflict | Cc: |
Description
What steps will reproduce the problem?
- Trying to solve several conflicts with the JOSM-conflict solving tool (JOSM compares my current version with the server version)
- Accepting my version
- Appling my version is the correct one > JOSM crashes
What is the expected result?
What happens instead?
Please provide any additional information below. Attach a screenshot if possible.
URL:https://josm.openstreetmap.de/svn/trunk
Repository:UUID: 0c6e7542-c601-0410-84e7-c038aed88b3b
Last:Changed Date: 2020-06-02 08:27:29 +0200 (Tue, 02 Jun 2020)
Build-Date:2020-06-02 11:11:35
Revision:16538
Relative:URL: ^/trunk
Identification: JOSM/1.5 (16538 de) Mac OS X 10.15.5
OS Build number: Mac OS X 10.15.5 (19F101)
Memory Usage: 1093 MB / 1820 MB (558 MB allocated, but free)
Java version: 1.8.0_251-b08, Oracle Corporation, Java HotSpot(TM) 64-Bit Server VM
Look and Feel: com.apple.laf.AquaLookAndFeel
Screen: Display 441084945 2560x1080
Maximum Screen Size: 2560x1080
VM arguments: [-Djava.library.path=/private/var/folders/mq/zwtfhdb533d13ylzfqpbjrfc0000gn/T/AppTranslocation/38FCBD25-246E-4750-BCDC-B55EE2CD434E/d/JOSM.app/Contents/MacOS, -DLibraryDirectory=${HOME}/Library, -DDocumentsDirectory=${HOME}/Documents, -DApplicationSupportDirectory=${HOME}/Library/Application Support, -DCachesDirectory=${HOME}/Library/Caches, -DApplicationDirectory=${HOME}/Applications, -DAutosavedInformationDirectory=${HOME}/Library/Autosave Information, -DDesktopDirectory=${HOME}/Desktop, -DDownloadsDirectory=${HOME}/Downloads, -DMoviesDirectory=${HOME}/Movies, -DMusicDirectory=${HOME}/Music, -DPicturesDirectory=${HOME}/Pictures, -DSharedPublicDirectory=${HOME}/Public, -DSystemLibraryDirectory=/Library, -DSystemApplicationSupportDirectory=/Library/Application Support, -DSystemCachesDirectory=/Library/Caches, -DSystemApplicationDirectory=/Applications, -DSystemUserDirectory=/Users, -DUserHome=${HOME}, -DSandboxEnabled=true, -DLaunchModifierFlags=0, -DLaunchModifierFlagCapsLock=false, -DLaunchModifierFlagShift=false, -DLaunchModifierFlagControl=false, -DLaunchModifierFlagOption=false, -DLaunchModifierFlagCommand=false, -DLaunchModifierFlagNumericPad=false, -DLaunchModifierFlagHelp=false, -DLaunchModifierFlagFunction=false, -Dapple.laf.useScreenMenuBar=true, -Dcom.apple.macos.use-file-dialog-packages=true, -Dcom.apple.macos.useScreenMenuBar=true, -Dcom.apple.mrj.application.apple.menu.about.name=JOSM, -Dcom.apple.smallTabs=true]
Dataset consistency test:
[DELETED REFERENCED] {Way id=726526507 version=2 MVT nodes=[{Node id=6809960493 version=1 V lat=47.0164802,lon=16.1641422}, {Node id=6718736816 version=1 V lat=47.0167444,lon=16.1639893}, {Node id=6718736818 version=2 MVD lat=47.0168788,lon=16.1640765}]} refers to deleted primitive {Node id=6718736818 version=2 MVD lat=47.0168788,lon=16.1640765}
Plugins:
+ FastDraw (35405)
+ RoadSigns (35424)
+ austriaaddresshelper (57)
+ buildings_tools (35474)
+ continuosDownload (91)
+ contourmerge (v0.1.6)
+ reverter (35474)
+ utilsplugin2 (35476)
Map paint styles:
+ ${HOME}/Downloads/browser/New_basic_style.mapcss
Last errors/warnings:
- W: Konflikte gefunden - <html>1 Konflikt wurde entdeckt.</html>
- W: Unable to convert property color to type class java.awt.Color: found # of type class java.lang.String!
- W: Unable to convert property color to type class java.awt.Color: found # of type class java.lang.String!
- W: Unable to convert property color to type class java.awt.Color: found # of type class java.lang.String!
- W: Konflikte gefunden - <html>8 Konflikte wurden entdeckt.</html>
- W: Unable to convert property color to type class java.awt.Color: found # of type class java.lang.String!
- W: Konflikte gefunden - <html>9 Konflikte wurden entdeckt.</html>
- W: Konflikte gefunden - <html>8 Konflikte wurden entdeckt.</html>
- E: Handled by bug report queue: org.openstreetmap.josm.data.osm.DataIntegrityProblemException: Deleted node referenced: {Way id=726526507 version=2 MVT nodes=[{Node id=6809960493 version=1 V lat=47.0164802,lon=16.1641422}, {Node id=6718736815 version=1 V lat=47.0166987,lon=16.1640108}, {Node id=6718736816 version=1 V lat=47.0167444,lon=16.1639893}, {Node id=6718736818 version=2 MVD lat=47.0168788,lon=16.1640765}]}
- E: Handled by bug report queue: org.openstreetmap.josm.data.osm.DataIntegrityProblemException: Deleted node referenced: {Way id=726526507 version=2 MVT nodes=[{Node id=6809960493 version=1 V lat=47.0164802,lon=16.1641422}, {Node id=6718736816 version=1 V lat=47.0167444,lon=16.1639893}, {Node id=6718736818 version=2 MVD lat=47.0168788,lon=16.1640765}]}
=== REPORTED CRASH DATA ===
BugReportExceptionHandler#handleException:
No data collected.
Warning issued by: BugReportExceptionHandler#handleException
=== STACK TRACE ===
Thread: AWT-EventQueue-0 (18) of main
org.openstreetmap.josm.data.osm.DataIntegrityProblemException: Deleted node referenced: {Way id=726526507 version=2 MVT nodes=[{Node id=6809960493 version=1 V lat=47.0164802,lon=16.1641422}, {Node id=6718736816 version=1 V lat=47.0167444,lon=16.1639893}, {Node id=6718736818 version=2 MVD lat=47.0168788,lon=16.1640765}]}
at org.openstreetmap.josm.data.osm.Way.checkNodes(Way.java:537)
at org.openstreetmap.josm.data.osm.Way.fireNodesChanged(Way.java:553)
at org.openstreetmap.josm.data.osm.Way.setNodes(Way.java:66)
at org.openstreetmap.josm.data.osm.DataSet.lambda$unlinkNodeFromWays$14(DataSet.java:799)
at org.openstreetmap.josm.data.osm.DataSet.update(DataSet.java:989)
at org.openstreetmap.josm.data.osm.DataSet.unlinkNodeFromWays(DataSet.java:791)
at org.openstreetmap.josm.data.osm.DataSet.lambda$unlinkReferencesToPrimitive$17(DataSet.java:843)
at org.openstreetmap.josm.data.osm.DataSet.update(DataSet.java:989)
at org.openstreetmap.josm.data.osm.DataSet.unlinkReferencesToPrimitive(DataSet.java:840)
at org.openstreetmap.josm.command.conflict.DeletedStateConflictResolveCommand.deleteMy(DeletedStateConflictResolveCommand.java:77)
at org.openstreetmap.josm.command.conflict.DeletedStateConflictResolveCommand.executeCommand(DeletedStateConflictResolveCommand.java:60)
at org.openstreetmap.josm.command.SequenceCommand.executeCommand(SequenceCommand.java:79)
at org.openstreetmap.josm.data.UndoRedoHandler.addNoRedraw(UndoRedoHandler.java:299)
at org.openstreetmap.josm.data.UndoRedoHandler.add(UndoRedoHandler.java:353)
at org.openstreetmap.josm.gui.dialogs.ConflictResolutionDialog$ApplyResolutionAction.actionPerformed(ConflictResolutionDialog.java:206)
at javax.swing.AbstractButton.fireActionPerformed(AbstractButton.java:2022)
at javax.swing.AbstractButton$Handler.actionPerformed(AbstractButton.java:2348)
at javax.swing.DefaultButtonModel.fireActionPerformed(DefaultButtonModel.java:402)
at javax.swing.DefaultButtonModel.setPressed(DefaultButtonModel.java:259)
at javax.swing.plaf.basic.BasicButtonListener.mouseReleased(BasicButtonListener.java:252)
at java.awt.AWTEventMulticaster.mouseReleased(AWTEventMulticaster.java:289)
at java.awt.Component.processMouseEvent(Component.java:6539)
at javax.swing.JComponent.processMouseEvent(JComponent.java:3324)
at java.awt.Component.processEvent(Component.java:6304)
at java.awt.Container.processEvent(Container.java:2239)
at java.awt.Component.dispatchEventImpl(Component.java:4889)
at java.awt.Container.dispatchEventImpl(Container.java:2297)
at java.awt.Component.dispatchEvent(Component.java:4711)
at java.awt.LightweightDispatcher.retargetMouseEvent(Container.java:4904)
at java.awt.LightweightDispatcher.processMouseEvent(Container.java:4535)
at java.awt.LightweightDispatcher.dispatchEvent(Container.java:4476)
at java.awt.Container.dispatchEventImpl(Container.java:2283)
at java.awt.Window.dispatchEventImpl(Window.java:2746)
at java.awt.Component.dispatchEvent(Component.java:4711)
at java.awt.EventQueue.dispatchEventImpl(EventQueue.java:760)
at java.awt.EventQueue.access$500(EventQueue.java:97)
at java.awt.EventQueue$3.run(EventQueue.java:709)
at java.awt.EventQueue$3.run(EventQueue.java:703)
at java.security.AccessController.doPrivileged(Native Method)
at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:74)
at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:84)
at java.awt.EventQueue$4.run(EventQueue.java:733)
at java.awt.EventQueue$4.run(EventQueue.java:731)
at java.security.AccessController.doPrivileged(Native Method)
at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:74)
at java.awt.EventQueue.dispatchEvent(EventQueue.java:730)
at java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:205)
at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:116)
at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:109)
at java.awt.WaitDispatchSupport$2.run(WaitDispatchSupport.java:190)
at java.awt.WaitDispatchSupport$4.run(WaitDispatchSupport.java:235)
at java.awt.WaitDispatchSupport$4.run(WaitDispatchSupport.java:233)
at java.security.AccessController.doPrivileged(Native Method)
at java.awt.WaitDispatchSupport.enter(WaitDispatchSupport.java:233)
at java.awt.Dialog.show(Dialog.java:1084)
at java.awt.Component.show(Component.java:1671)
at java.awt.Component.setVisible(Component.java:1623)
at java.awt.Window.setVisible(Window.java:1014)
at java.awt.Dialog.setVisible(Dialog.java:1005)
at org.openstreetmap.josm.gui.ExtendedDialog.setVisible(ExtendedDialog.java:459)
at org.openstreetmap.josm.gui.ExtendedDialog.showDialog(ExtendedDialog.java:255)
at org.openstreetmap.josm.gui.dialogs.ConflictDialog.resolve(ConflictDialog.java:209)
at org.openstreetmap.josm.gui.dialogs.ConflictDialog.access$200(ConflictDialog.java:79)
at org.openstreetmap.josm.gui.dialogs.ConflictDialog$ResolveAction.actionPerformed(ConflictDialog.java:466)
at javax.swing.AbstractButton.fireActionPerformed(AbstractButton.java:2022)
at javax.swing.AbstractButton$Handler.actionPerformed(AbstractButton.java:2348)
at javax.swing.DefaultButtonModel.fireActionPerformed(DefaultButtonModel.java:402)
at javax.swing.DefaultButtonModel.setPressed(DefaultButtonModel.java:259)
at javax.swing.plaf.basic.BasicButtonListener.mouseReleased(BasicButtonListener.java:252)
at java.awt.AWTEventMulticaster.mouseReleased(AWTEventMulticaster.java:289)
at java.awt.Component.processMouseEvent(Component.java:6539)
at javax.swing.JComponent.processMouseEvent(JComponent.java:3324)
at java.awt.Component.processEvent(Component.java:6304)
at java.awt.Container.processEvent(Container.java:2239)
at java.awt.Component.dispatchEventImpl(Component.java:4889)
at java.awt.Container.dispatchEventImpl(Container.java:2297)
at java.awt.Component.dispatchEvent(Component.java:4711)
at java.awt.LightweightDispatcher.retargetMouseEvent(Container.java:4904)
at java.awt.LightweightDispatcher.processMouseEvent(Container.java:4535)
at java.awt.LightweightDispatcher.dispatchEvent(Container.java:4476)
at java.awt.Container.dispatchEventImpl(Container.java:2283)
at java.awt.Window.dispatchEventImpl(Window.java:2746)
at java.awt.Component.dispatchEvent(Component.java:4711)
at java.awt.EventQueue.dispatchEventImpl(EventQueue.java:760)
at java.awt.EventQueue.access$500(EventQueue.java:97)
at java.awt.EventQueue$3.run(EventQueue.java:709)
at java.awt.EventQueue$3.run(EventQueue.java:703)
at java.security.AccessController.doPrivileged(Native Method)
at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:74)
at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:84)
at java.awt.EventQueue$4.run(EventQueue.java:733)
at java.awt.EventQueue$4.run(EventQueue.java:731)
at java.security.AccessController.doPrivileged(Native Method)
at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:74)
at java.awt.EventQueue.dispatchEvent(EventQueue.java:730)
at java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:205)
at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:116)
at java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:105)
at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:101)
at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:93)
at java.awt.EventDispatchThread.run(EventDispatchThread.java:82)
Attachments (1)
Change History (57)
comment:1 by , 4 years ago
| Keywords: | DataIntegrityProblemException Deleted node referenced conflict added |
|---|---|
| Owner: | changed from to |
| Status: | new → needinfo |
| Summary: | JOSM crashes during conflict solving → DataIntegrityProblemException: Deleted node referenced during conflict solving |
| Version: | → tested |
comment:2 by , 4 years ago
Conflicts occurs due to different versions between server and my computer/JOSM-data.
OSM data source: always the same that JOSM uses. I do not know in detail, which data source JOSM uses. Sorry.
Last actions: deleting node duplicates of lines.
comment:3 by , 4 years ago
| Owner: | changed from to |
|---|---|
| Status: | needinfo → new |
comment:4 by , 4 years ago
Just an additional information on the duplicate nodes:
I use OSMOSE quality insurance tool to find mistakes in mapping. Duplicate nodes are shown there. This is how I find them in detail: searching for a marker on OSMOSE > opening in JOSM > deleting the node.
comment:5 by , 4 years ago
How do you download the data? With "normal" download data or do you only download the nodes using "Download object" or even a remote control link?
How do you make sure, you did download all parent ways of the nodes you want to delete?
Is it reproducible? If possible, plaese, add a step by step description of your actions.
comment:6 by , 4 years ago
Again I've got a crash of JOSM.
In detail (a sample), how to reproduce: Open via osmose http://osmose.openstreetmap.fr/de/josm_proxy?load_object?objects=w253518798 in JOSM. There are duplicates of landuses. Delete this object.
Download data in this area from OSM > several conflicts occur.
Try to solve the conflicts > during this process JOSM crashes.
comment:7 by , 4 years ago
Is it possible for you to create a example file including two data layers, one with some of your edits and the other with the download OSM data. When merging these two layers you should get the conflicts and hopefully the same exceptions.
How is the result of a data consistency check before you download data in the area from OSM? Is it already inconsistent or is it fine?
comment:8 by , 4 years ago
| Milestone: | → 20.10 |
|---|---|
| Owner: | changed from to |
| Status: | new → assigned |
I can reproduce this problem and have a fix :)
- load attached session
- remove node in Data Layer 4, ignore the warning and click Yes
- right click on Data Layer 3 and click merge, select Data Layer 4
- Click OK in the popup
- Right click on the conflict and chose "Resolve the conflict to my versions"
- Right click on Data Layer 4 and chose "Dataset consistency test"
[DELETED REFERENCED] {Way id=542904077 version=1 MVT nodes=[{Node id=5248536615 version=1 V lat=52.8876891,lon=8.5304175}, {Node id=5248536611 version=1 V lat=52.8876753,lon=8.5304264}, {Node id=5248536610 version=1 V lat=52.887688,lon=8.5304804}, {Node id=5248536609 version=1 V lat=52.8876209,lon=8.530524}, {Node id=5248536612 version=1 V lat=52.8876081,lon=8.5304701}, {Node id=5248536614 version=1 V lat=52.8875343,lon=8.5305181}, {Node id=5248536613 version=1 V lat=52.8874963,lon=8.5303574}, {Node id=5248536616 version=1 MVD lat=52.8876511,lon=8.5302568}]} refers to deleted primitive {Node id=5248536616 version=1 MVD lat=52.8876511,lon=8.5302568}
Reason: the deleted node is the closing node in in the building and thus appears twice. The code in Dataset.unlinkNodeFromWays() only removes the first occurence.
follow-up: 11 comment:10 by , 4 years ago
@team: There are some other tickets with the text org.openstreetmap.josm.data.osm.DataSet.unlinkNodeFromWays in the traceback. How do I close all of them as duplicate of this ticket?
comment:11 by , 4 years ago
Replying to GerdP:
@team: There are some other tickets with the text
org.openstreetmap.josm.data.osm.DataSet.unlinkNodeFromWaysin the traceback. How do I close all of them as duplicate of this ticket?
- https://josm.openstreetmap.de/report and modifying the ticket search which will bring you here
- check the checkbox for all
- use batch modify below
comment:24 by , 4 years ago
This ticket is not linked in all the other tickets like when you close a single ticket as dulplicate (like in e.g. ticket:19927#comment:2)
comment:25 by , 4 years ago
Ah, I see. Will do it manually tomorrow if not yet done by someone else ;)
follow-up: 36 comment:35 by , 4 years ago
In fact I cannot be sure that all these tickets were caused by the bug that was fixed with r17190, but I think it is very likely. Following that logic I might close even more tickets:
https://josm.openstreetmap.de/query?status=assigned&status=needinfo&status=new&status=reopened&description=~DataIntegrityProblemException%3A+Deleted+node+referenced&col=id&col=summary&col=status&col=type&col=priority&col=milestone&col=component&order=priority
follow-up: 45 comment:36 by , 4 years ago
Replying to GerdP:
In fact I cannot be sure that all these tickets were caused by the bug that was fixed with r17190, but I think it is very likely. Following that logic I might close even more tickets:
https://josm.openstreetmap.de/query?status=assigned&status=needinfo&status=new&status=reopened&description=~DataIntegrityProblemException%3A+Deleted+node+referenced&col=id&col=summary&col=status&col=type&col=priority&col=milestone&col=component&order=priority
Do that. Keeping them open does not help much without finding the reason even if they aren't a duplicate.
Regarding back-reference. The action for that back-reference has a warning that it wont work in certain situations. Now we know one of them :-)
follow-up: 46 comment:45 by , 4 years ago
Replying to stoecker:
Replying to GerdP:
In fact I cannot be sure that all these tickets were caused by the bug that was fixed with r17190, but I think it is very likely. Following that logic I might close even more tickets:
https://josm.openstreetmap.de/query?status=assigned&status=needinfo&status=new&status=reopened&description=~DataIntegrityProblemException%3A+Deleted+node+referenced&col=id&col=summary&col=status&col=type&col=priority&col=milestone&col=component&order=priority
Do that. Keeping them open does not help much without finding the reason even if they aren't a duplicate.
I've closed those tickets which mention conflicts or show corresponding problems in the "Last errors/warnings:" section. The remaining problems may well be caused by another error. I'll look at them again during the next days.
Regarding back-reference. The action for that back-reference has a warning that it wont work in certain situations. Now we know one of them :-)
Yeah, causes a lot of noise :(
comment:46 by , 4 years ago
Yeah, causes a lot of noise :(
If the noise is because you found and fixed one long-standing issue I have not the slightest problem with that!
comment:47 by , 4 years ago
BTW: Not sure which one it is but there is/was a QA tool that warns when you use java.util.List.remove(Object o).
follow-up: 52 comment:48 by , 4 years ago
I wonder if my solution introduced a new problem. For my example in comment:8 the building=yes way is no longer closed after the given steps.
The code in Way.calculateRemoveNodes() takes care of that. I think JOSM should use it instead of wayNodes.removeIf(node::equals)?
comment:49 by , 4 years ago
If it is a closed way in both layers and only one child node is deleted it should be a closed way after conflict resolution in all cases. Not sure how to difference between a closed way on one side and an open way on the other but I guess that one node is twice in the way's node list is the indicator.
comment:50 by , 4 years ago
The way can be P-shaped as well or it can be an incorrect self-intersecting geometry.
comment:51 by , 4 years ago
Ok, a closed way has the first and the last ref on the same node. If this node is deleted the next node needs to be referenced twice, e.g. it needs to be added as ref at the bottom.
comment:52 by , 4 years ago
| Resolution: | fixed |
|---|---|
| Status: | closed → reopened |
Replying to GerdP:
I wonder if my solution introduced a new problem. For my example in comment:8 the building=yes way is no longer closed after the given steps.
The code in Way.calculateRemoveNodes() takes care of that. I think JOSM should use it instead ofwayNodes.removeIf(node::equals)?
Think there should be another conflict on the way, when merging the layers as deleting the node does change the parent ways. Merge layers should report this.
Anyway, right now the change of the way is not correct, as it should be closed.
comment:53 by , 4 years ago
I agree that the result looks wrong. OTOH we cannot use method Way.calculateRemoveNodes() here as it only works for complete ways.
comment:55 by , 4 years ago
@skyper: Don't know what to think about other conflicts. I think the one that is created contains all info that is needed. Feel free to create a new ticket for that.
How did you get the conflicts?
What OSM data sources did you use?
What were your last actions before the conflicts?
How did you get the inconsistent dataset?