Modify

Opened 12 years ago

Closed 12 years ago

#3909 closed defect (fixed)

NPE and broken data during upload attempt. (QuadBuckets)

Reported by: bilbo Owned by: hansendc
Priority: major Milestone:
Component: Core Version: latest
Keywords: QuadBuckets NPE Cc:

Description

I edited some data (using JOSM 2437), then I wanted to use validator and do upload and I got just a messagebox saying "nullpointer exception" and on console I found this stacktrace:

Exception occurred during event dispatching:
java.lang.NullPointerException
        at org.openstreetmap.josm.data.osm.QuadBuckets$QBLevel.nextContentNode QuadBuckets.java:485)
        at org.openstreetmap.josm.data.osm.QuadBuckets$QuadBucketIterator.next_content_node(QuadBuckets.java:969)
        at org.openstreetmap.josm.data.osm.QuadBuckets$QuadBucketIterator.peek(QuadBuckets.java:1015)
        at org.openstreetmap.josm.data.osm.QuadBuckets$QuadBucketIterator.hasNext(QuadBuckets.java:993)
        at org.openstreetmap.josm.data.osm.QuadBuckets.toArrayList(QuadBuckets.java:943)
        at org.openstreetmap.josm.data.osm.QuadBuckets.toArray(QuadBuckets.java:953)
        at java.util.Collections$UnmodifiableCollection.toArray(Collections.java:1001)
        at java.util.ArrayList.<init>(ArrayList.java:131)
        at org.openstreetmap.josm.data.osm.visitor.MapPaintVisitor.selectedLast(MapPaintVisitor.java:1429)
        at org.openstreetmap.josm.data.osm.visitor.MapPaintVisitor.visitAll(MapPaintVisitor.java:1523)
        at org.openstreetmap.josm.gui.layer.OsmDataLayer.paint(OsmDataLayer.java:247)
        at org.openstreetmap.josm.gui.MapView.paint(MapView.java:388)
        at javax.swing.JComponent.paintChildren(JComponent.java:864)
        at javax.swing.JSplitPane.paintChildren(JSplitPane.java:1030)
        at javax.swing.JComponent.paint(JComponent.java:1038)
        at javax.swing.JComponent.paintChildren(JComponent.java:864)
        at javax.swing.JComponent.paint(JComponent.java:1038)
        at javax.swing.JComponent.paintChildren(JComponent.java:864)
        at javax.swing.JComponent.paint(JComponent.java:1038)
        at javax.swing.JComponent.paintChildren(JComponent.java:864)
        at javax.swing.JComponent.paint(JComponent.java:1038)
        at javax.swing.JComponent.paintChildren(JComponent.java:864)
        at javax.swing.JComponent.paint(JComponent.java:1038)
        at javax.swing.JLayeredPane.paint(JLayeredPane.java:567)
        at javax.swing.JComponent.paintChildren(JComponent.java:864)
        at javax.swing.JComponent.paintToOffscreen(JComponent.java:5131)
        at javax.swing.RepaintManager$PaintManager.paintDoubleBuffered(RepaintManager.java:1475)
        at javax.swing.RepaintManager$PaintManager.paint(RepaintManager.java:1406)
        at javax.swing.BufferStrategyPaintManager.paint(BufferStrategyPaintManager.java:294)
        at javax.swing.RepaintManager.paint(RepaintManager.java:1220)
        at javax.swing.JComponent.paint(JComponent.java:1015)
        at java.awt.GraphicsCallback$PaintCallback.run(GraphicsCallback.java:21)
        at sun.awt.SunGraphicsCallback.runOneComponent(SunGraphicsCallback.java:60)
        at sun.awt.SunGraphicsCallback.runComponents(SunGraphicsCallback.java:97)
        at java.awt.Container.paint(Container.java:1780)
        at javax.swing.RepaintManager.paintDirtyRegions(RepaintManager.java:814)
        at javax.swing.RepaintManager.paintDirtyRegions(RepaintManager.java:714)
        at javax.swing.RepaintManager.seqPaintDirtyRegions(RepaintManager.java:694)
        at javax.swing.SystemEventQueueUtilities$ComponentWorkRequest.run(SystemEventQueueUtilities.java:128)
        at java.awt.event.InvocationEvent.dispatch(InvocationEvent.java:209)
        at java.awt.EventQueue.dispatchEvent(EventQueue.java:597)
        at java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:269)
        at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:184)
        at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:178)
        at java.awt.Dialog$1.run(Dialog.java:1045)
        at java.awt.Dialog$3.run(Dialog.java:1097)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.awt.Dialog.show(Dialog.java:1095)
        at java.awt.Component.show(Component.java:1563)
        at java.awt.Component.setVisible(Component.java:1515)
        at java.awt.Window.setVisible(Window.java:841)
        at java.awt.Dialog.setVisible(Dialog.java:985)
        at org.openstreetmap.josm.gui.progress.PleaseWaitProgressMonitor$3.run(PleaseWaitProgressMonitor.java:79)
        at java.awt.event.InvocationEvent.dispatch(InvocationEvent.java:209)
        at java.awt.EventQueue.dispatchEvent(EventQueue.java:597)
        at java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:269)
        at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:184)
        at java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:174)
        at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:169)
        at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:161)
        at java.awt.EventDispatchThread.run(EventDispatchThread.java:122)
java.lang.NullPointerException
        at org.openstreetmap.josm.data.osm.QuadBuckets$QBLevel.split(QuadBuckets.java:289)
        at org.openstreetmap.josm.data.osm.QuadBuckets$QBLevel.add_to_leaf(QuadBuckets.java:328)
        at org.openstreetmap.josm.data.osm.QuadBuckets$QBLevel.add(QuadBuckets.java:564)
        at org.openstreetmap.josm.data.osm.QuadBuckets.add(QuadBuckets.java:819)
        at org.openstreetmap.josm.data.osm.QuadBuckets.addAll(QuadBuckets.java:873)
        at org.openstreetmap.josm.data.osm.DataSet.reindexAll(DataSet.java:767)
        at org.openstreetmap.josm.plugins.validator.tests.UnconnectedWays.startTest(UnconnectedWays.java:77)
        at org.openstreetmap.josm.plugins.validator.ValidateAction$ValidationTask.realRun(ValidateAction.java:164)
        at org.openstreetmap.josm.gui.PleaseWaitRunnable.doRealRun(PleaseWaitRunnable.java:63)
        at org.openstreetmap.josm.gui.PleaseWaitRunnable.run(PleaseWaitRunnable.java:105)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
        at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
        at java.util.concurrent.FutureTask.run(FutureTask.java:138)
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
        at java.lang.Thread.run(Thread.java:619)
java.lang.NullPointerException
        at org.openstreetmap.josm.data.osm.QuadBuckets$QBLevel.split(QuadBuckets.java:289)
        at org.openstreetmap.josm.data.osm.QuadBuckets$QBLevel.add_to_leaf(QuadBuckets.java:328)
        at org.openstreetmap.josm.data.osm.QuadBuckets$QBLevel.add(QuadBuckets.java:564)
        at org.openstreetmap.josm.data.osm.QuadBuckets.add(QuadBuckets.java:819)
        at org.openstreetmap.josm.data.osm.QuadBuckets.addAll(QuadBuckets.java:873)
        at org.openstreetmap.josm.data.osm.DataSet.reindexAll(DataSet.java:767)
        at org.openstreetmap.josm.plugins.validator.tests.UnconnectedWays.startTest(UnconnectedWays.java:77)
        at org.openstreetmap.josm.plugins.validator.ValidateAction$ValidationTask.realRun(ValidateAction.java:164)
        at org.openstreetmap.josm.gui.PleaseWaitRunnable.doRealRun(PleaseWaitRunnable.java:63)
        at org.openstreetmap.josm.gui.PleaseWaitRunnable.run(PleaseWaitRunnable.java:105)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
        at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
        at java.util.concurrent.FutureTask.run(FutureTask.java:138)
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
        at java.lang.Thread.run(Thread.java:619)
p1 elapesd: 0
p2 elapesd: 0
p99 elapesd: 0

I then tried to upload data to server to save my work and I found that nodes were uploaded fine, but ways were not uploaded at all (the changeset contained no ways, even though I've modified some of them)

Also, all ways disappeared from my view - from the dataset I've seen in JOSM.

Attachments (0)

Change History (8)

comment:1 Changed 12 years ago by bilbo

Now happened second time - when I pressed "Validate" after making some changes and uploading them to server, then I got the nullpointer exception messagebox (ordinary messagebox, not that one with "Report bug" button, etc ...), when I dismiss it, all ways disappear and only nodes remain visible in the dataset.

comment:2 Changed 12 years ago by anonymous

Keywords: QuadBuckets NPE added
Summary: NPE and broken data during upload attempt.NPE and broken data during upload attempt. (QuadBuckets)

comment:3 Changed 12 years ago by hansendc

Owner: changed from team to hansendc

Urg. I think this is a race.

The add() code is splitting a node. It allocates children[], but not any child QBLevels. The iteration code concurrently comes along and notices the empty children[] array, considers it a dead end, and nulls out children[]. The split() code now goes and tries to access children[].

I need to think about the locking a bit.

comment:4 Changed 12 years ago by hansendc

Resolution: fixed
Status: newclosed

I've added some locking to QuadBuckets to hopefully ensure that this won't happen any more.

comment:5 Changed 12 years ago by jttt

Resolution: fixed
Status: closedreopened

I've just got similar exception in r2440

java.lang.AssertionError: branch node had no children
	at org.openstreetmap.josm.data.osm.QuadBuckets.abort(QuadBuckets.java:29)
	at org.openstreetmap.josm.data.osm.QuadBuckets$QBLevel.nextContentNode(QuadBuckets.java:493)
	at org.openstreetmap.josm.data.osm.QuadBuckets$QuadBucketIterator.next_content_node(QuadBuckets.java:986)
	at org.openstreetmap.josm.data.osm.QuadBuckets$QuadBucketIterator.peek(QuadBuckets.java:1033)
	at org.openstreetmap.josm.data.osm.QuadBuckets$QuadBucketIterator.hasNext(QuadBuckets.java:1011)
	at java.util.Collections$UnmodifiableCollection$1.hasNext(Collections.java:1009)
	at org.openstreetmap.josm.gui.layer.OsmDataLayer.undeletedSize(OsmDataLayer.java:491)
	at org.openstreetmap.josm.gui.layer.OsmDataLayer.getToolTipText(OsmDataLayer.java:254)
	at org.openstreetmap.josm.gui.io.SaveLayerInfoCellRenderer.prepareLayerNameRenderer(SaveLayerInfoCellRenderer.java:37)
	at org.openstreetmap.josm.gui.io.SaveLayerInfoCellRenderer.getTableCellRendererComponent(SaveLayerInfoCellRenderer.java:138)
	... uninteresting rest of stacktrace ...

comment:6 Changed 12 years ago by jttt

This happen much more often if MAX_OBJECTS_PER_LEVEL is set to small number. I had it set to 2 when I was debugging QuadBuckets (see r2441) and exception happend for almost any bigger dataset.

comment:7 Changed 12 years ago by hansendc

I've tracked this down to the DataSetMerger:

myWay.getNodes().size(): 0 myNodes.size(): 13
set primitive modified: 5331187

at org.openstreetmap.josm.data.osm.OsmPrimitive.setModified(OsmPrimitive.java:268)
at org.openstreetmap.josm.data.osm.DataSetMerger.mergeNodeList(DataSetMerger.java:199)
at org.openstreetmap.josm.data.osm.DataSetMerger.fixReferences(DataSetMerger.java:166)
at org.openstreetmap.josm.data.osm.DataSetMerger.merge(DataSetMerger.java:368)

It looks like it's having difficulty finding existing ways, so it creates a new empty way. Merging the new data with that empty way sets it modified in mergeNodeList().

I'm trying to figure out why it is having problems finding the existing way during the merge.

comment:8 Changed 12 years ago by hansendc

Resolution: fixed
Status: reopenedclosed

(In [2442]) This should fix #3909

We created empty branches when we attempted to split a QB node
with ways in it and none of the ways could fit into a child.
The children were allocated before we knew that we had something
to put in them, and we ended up with an empty array of children.
That triggered this check.

So, defer child array allocation until we really need it.

Modify Ticket

Change Properties
Set your email in Preferences
Action
as closed The owner will remain hansendc.
as The resolution will be set.
The resolution will be deleted.

Add Comment


E-mail address and name can be saved in the Preferences.

 
Note: See TracTickets for help on using tickets.