[patch] doubles parsing

[shinigami]

Double.parseDouble makes more than 15% of garbage in my test, when parsing osm file.

This patch is experimental replacement. It can parse simply writen doubles only, no NaNs, no doubles with E..., just numbers like 123.456789 and so. But if I understand well, it is enough for parsing osm files. It generates no garbage and seems to be ~3x faster than generic Double.parseDouble.

[shinigami]

patch

[stoecker]

Primitives is not a good name, as this is used for node, way, relation in JOSM. Didn't check the general idea of the patch.

[bastiK]

I feel that we should be able to parse scientific notation (E...) as I expect this to be in the output of third party tools than create .osm files. (Import scripts and the like.)

Also we should have unit test to prove that the output of your optimized code is bit-identical to Double.parseDouble().

Could you explain why it is a problem to have "garbage", as it gets collected and the memory is re-used right away? What is the overall gain in runtime when loading an osm file?

Btw., your username makes me a little uneasy... ;)

[Don-vip]

Replying to shinigami:

seems to be ~3x faster than generic Double.parseDouble.

from which JDK? There was a big ​performance issue with JDK7 but it seems to have been fixed in JDK8.

[shinigami]

Don-vip: jdk8u45, it still delegates parsing to some internal classes, creates buffers for it and so.

bastiK: aww, I have looked to osm file description briefly and understood it as it can contain this simple format only:/.

Problem with garbage is it must be collected;) I started with this patches because of #11492. I can't open that file at my hw (have 16GB only), so I always sample first minute of parsing. Half of time it spends collecting garbage.

[bastiK]

Replying to shinigami:

bastiK: aww, I have looked to osm file description briefly and understood it as it can contain this simple format only:/.

You could first attempt to parse it in a simple way. Then, if that failed, try again with Double.parseDouble(). This should be both fast and reliable, right?

Problem with garbage is it must be collected;) I started with this patches because of #11492. I can't open that file at my hw (have 16GB only), so I always sample first minute of parsing. Half of time it spends collecting garbage.

So basically it does not affect any memory constraints, but speeds up the process? By how much? What tools do you use to investigate?

Btw., your effort to optimize the loading of large .osm files is highly appreciated. :)

[simon04]

​https://github.com/wrandelshofer/FastDoubleParser (MIT License; 28 KB) claims to offer a 4x speed-up over Double.parseDouble

parsing numbers in file data/canada.txt
read 111126 lines
=== warmup 1000 times =====
=== number of trials 32 =====
FastDoubleParser               MB/s avg: 341.740221, min: 303.93, max: 366.63
FastDoubleParserFromByteArray  MB/s avg: 460.556911, min: 405.39, max: 481.79
Double                         MB/s avg: 80.854956, min: 68.74, max: 85.77
Speedup FastDoubleParser              vs Double: 4.226583
Speedup FastDoubleParserFromByteArray vs Double: 5.696088

---

org.openstreetmap.josm.io.OsmReaderPerformanceTest#testPlain on nodist/data/neubrandenburg.osm.bz2

-<measurement><name>load .osm-file 4 times(ms)</name><value>2851.0</value></measurement>
+<measurement><name>load .osm-file 4 times(ms)</name><value>2664.0</value></measurement>

org.openstreetmap.josm.io.OsmReaderPerformanceTest#testCompressed on montenegro-latest.osm.bz2 (44M)

-<measurement><name>load compressed (.osm.bz2) 4 times(ms)</name><value>69327.0</value></measurement>
+<measurement><name>load compressed (.osm.bz2) 4 times(ms)</name><value>69156.0</value></measurement>

My conclusion: rather negligible effects for parsing OSM XML data.

---

ivy.xml

@@ -31 +31 @@
         <dependency conf="api->default" org="ch.poole" name="OpeningHoursParser" rev="0.23.2"/>
         <dependency conf="api->default" org="oauth.signpost" name="signpost-core" rev="2.0.0"/>
         <dependency conf="api->default" org="org.webjars.npm" name="tag2link" rev="2021.3.21"/>
+        <dependency conf="api->default" org="ch.randelshofer" name="fastdoubleparser" rev="0.3.0"/>
         <!-- sources->sources -->
         <dependency conf="sources->sources" org="org.openstreetmap.jmapviewer" name="jmapviewer" rev="2.15"/>
         <dependency conf="sources->sources" org="javax.json" name="javax.json-api" rev="1.1.4"/>

src/org/openstreetmap/josm/io/AbstractParser.java

@@ -5 +5 @@
 
 import java.util.Date;
 
+import ch.randelshofer.fastdoubleparser.FastDoubleParser;
 import org.openstreetmap.josm.data.coor.LatLon;
 import org.openstreetmap.josm.data.osm.OsmPrimitiveType;
 import org.openstreetmap.josm.data.osm.RelationMemberData;
@@ -80 +81 @@
         }
         double d = 0.0;
         try {
-            d = Double.parseDouble(v);
+            d = FastDoubleParser.parseDouble(v);
         } catch (NumberFormatException e) {
             throwException(tr("Illegal value for attribute ''{0}'' of type double. Got ''{1}''.", name, v), e);
         }

[Don-vip]

I would prefer to wait for a JDK enhancement rather than adding a new lib just for that.

[GerdP]

And it seems to require Java 11?