Opened 5 years ago

Last modified 5 years ago

#20633 closed enhancement

Download object: Too slow when downloading a relation with members with OSM api — at Initial Version

Reported by: GerdP Owned by: team
Priority: normal Milestone: 21.03
Component: Core Version: tested
Keywords: template_report performance Cc: Don-vip, simon04

Description

What steps will reproduce the problem?

  1. Download a complex relation like r9488835 with members to a new layer (not using overpass)
  2. Watch how it takes some seconds to actually download all the data and much more time to prcoess it before a the rendered data is shown.
  3. Save to *.osm file
  4. Load from *.osm file

What is the expected result?

It should take more or less the same time to process the downloaded data as it takes to open the file.

What happens instead?

The downloaded data takes much longer.

Please provide any additional information below. Attach a screenshot if possible.

I used VisualVM to monitor the program and found that a lot of time is spent in DatasetMerger.fixReferences(). It turned out that the data that is retrieved with the 1st call

2021-03-21 08:44:43.115 INFO: GET https://api.openstreetmap.org/api/0.6/relations?relations=9488835 -> HTTP/1.1 200 (322 ms)
2021-03-21 08:44:53.634 INFO: GET https://api.openstreetmap.org/api/0.6/relation/9488835/full -> HTTP/1.1 200 (10.0 s)

is causing the trouble. The result of the call is the incomplete relation. This incomplete data is stored in field MultiFetchServerObjectReader.outputDataSet and later, all the detailed data is merged with this incomplete relation. This forces lots of rather slow QuadBucketPrimitiveStore.reindexRelation()
calls. This merging isn't really needed, the only reason for the initial download is to find out if the relation exists.

URL:https://josm.openstreetmap.de/svn/trunk
Repository:UUID: 0c6e7542-c601-0410-84e7-c038aed88b3b
Last:Changed Date: 2021-03-17 18:58:40 +0100 (Wed, 17 Mar 2021)
Build-Date:2021-03-17 17:59:56
Revision:17580
Relative:URL: ^/trunk

Identification: JOSM/1.5 (17580 en) Windows 10 64-Bit
OS Build number: Windows 10 Home 2004 (19041)
Memory Usage: 412 MB / 3641 MB (221 MB allocated, but free)
Java version: 1.8.0_221-b11, Oracle Corporation, Java HotSpot(TM) 64-Bit Server VM
Look and Feel: com.sun.java.swing.plaf.windows.WindowsLookAndFeel
Screen: \Display0 1920×1080 (scaling 1.00×1.00)
Maximum Screen Size: 1920×1080
Best cursor sizes: 16×16→32×32, 32×32→32×32
VM arguments: [-XX:StartFlightRecording=name=MyRecording2,settings=d:\dbg\gerd.jfc, -XX:FlightRecorderOptions=defaultrecording=true,dumponexit=true,dumponexitpath=e:\ld\perf_20210321_082504.jfr]

Change History (2)

by GerdP, 5 years ago

Attachment: analysis.PNG added

screenshot VisualVM

by GerdP, 5 years ago

Attachment: 20633.patch added
Note: See TracTickets for help on using tickets.