Modify

Opened 9 years ago

Closed 7 years ago

Last modified 7 years ago

#10638 closed enhancement (fixed)

[Patch] "Way connected to Area" taking too long

Reported by: naoliv Owned by: team
Priority: normal Milestone: 17.02
Component: Core validator Version:
Keywords: performance Cc:

Description

Validate the attached file and see how long it takes on "Way connected to Area" (here on an i7-2670QM it took 14 minutes)

Is it right? Could it be optimized somehow?

JOSM:

Repository Root: http://josm.openstreetmap.de/svn
Build-Date: 2014-10-14 01:35:20
Last Changed Author: Don-vip
Revision: 7620
Repository UUID: 0c6e7542-c601-0410-84e7-c038aed88b3b
Relative URL: ^/trunk
URL: http://josm.openstreetmap.de/svn/trunk
Last Changed Date: 2014-10-14 01:27:10 +0200 (Tue, 14 Oct 2014)
Last Changed Rev: 7620

Identification: JOSM/1.5 (7620 en) Linux Debian GNU/Linux unstable (sid)
Memory Usage: 298 MB / 4029 MB (87 MB allocated, but free)
Java version: 1.7.0_65, Oracle Corporation, OpenJDK 64-Bit Server VM
Java package: openjdk-7-jre:amd64-7u65-2.5.2-4.1
VM arguments: [-Dawt.useSystemAAFontSettings=on]
Program arguments: [--debug]
Dataset consistency test: No problems found

Plugins:
- AddrInterpolation (30416)
- Create_grid_of_ways (30416)
- FixAddresses (30416)
- ImageryCache (30416)
- OpeningHoursEditor (30609)
- PicLayer (30436)
- SimplifyArea (30624)
- buildings_tools (30485)
- editgpx (30634)
- geotools (30569)
- graphview (30416)
- jts (30416)
- measurement (30641)
- merge-overlap (30714)
- notes (v0.9.4)
- opendata (30607)
- pdfimport (30416)
- poly (30495)
- reverter (30521)
- tagging-preset-tester (30592)
- todo (29154)
- turnrestrictions (30651)
- undelete (30416)
- utilsplugin2 (30460)

Attachments (7)

long.osm.bz2 (975.2 KB ) - added by naoliv 9 years ago.
10638.patch (938 bytes ) - added by GerdP 8 years ago.
OsmDataLayer.patch (1.4 KB ) - added by GerdP 8 years ago.
10638-v2.patch (5.1 KB ) - added by GerdP 8 years ago.
10638-v3.patch (5.1 KB ) - added by GerdP 7 years ago.
based on r11228, no further changes
10638-v4.patch (881 bytes ) - added by GerdP 7 years ago.
Solve remaining zoom in/out problem for given test data
10638-v5.patch (1.4 KB ) - added by GerdP 7 years ago.
Solve remaining zoom in/out problem for given test data (extract from v3)

Download all attachments as: .zip

Change History (25)

by naoliv, 9 years ago

Attachment: long.osm.bz2 added

comment:1 by Don-vip, 9 years ago

Keywords: performance added

by GerdP, 8 years ago

Attachment: 10638.patch added

comment:2 by GerdP, 8 years ago

The problem is caused by a rather huge amount of different "bounds" elements: 1665.
These 1665 areas are combined in DataSource.getDataSourceArea(Collection<DataSource> dataSources) each time when
Node.isOutsideDownloadArea() is called, which is done very often in this test.
I've attached a patch which makes the calculation much faster (test is finished within a few seconds now),
but I think it should be possible to store the calculated value somewhere and update it for each added datasource.

by GerdP, 8 years ago

Attachment: OsmDataLayer.patch added

comment:3 by GerdP, 8 years ago

Summary: "Way connected to Area" taking too long[Pacth] "Way connected to Area" taking too long

Similar performance problem is in other sources were multiple Areas are subtracted from or added to one area.
In most cases it is faster to append all areas to a Path2D.Double and add or substract once. Exception:
If it is likely that the area from which other areas are subtracted is emptied quickly, this seems to be the case in
the loops in UpdateDataAction.actionPerformed(ActionEvent e).
The patch OsmDataLayer.patch significantly improves zooming out (e.g. Key 1 for "Zoomto data" in the osm file attached here.

by GerdP, 8 years ago

Attachment: 10638-v2.patch added

comment:4 by GerdP, 8 years ago

10638-v2.patch combines the previous patches (with checkstyle corrections) and adds further improvements to avoid
frequent calls of the getDataSourceArea() method. It would require a lot more changes to store the calculated area in DataSource because the field dataSources is public.

comment:5 by Don-vip, 8 years ago

Milestone: 16.09
Summary: [Pacth] "Way connected to Area" taking too long[Patch] "Way connected to Area" taking too long

comment:6 by Don-vip, 7 years ago

Milestone: 16.0916.10

comment:7 by simon04, 7 years ago

Milestone: 16.1016.11

Milestone renamed

by GerdP, 7 years ago

Attachment: 10638-v3.patch added

based on r11228, no further changes

comment:8 by Don-vip, 7 years ago

Milestone: 16.1116.12

Milestone renamed

comment:9 by Don-vip, 7 years ago

Milestone: 16.1217.01

comment:10 by Don-vip, 7 years ago

Milestone: 17.0117.02

comment:11 by Don-vip, 7 years ago

Milestone: 17.0217.03

comment:12 by GerdP, 7 years ago

This problem was probably solved with r11627, see also comment 2.

by GerdP, 7 years ago

Attachment: 10638-v4.patch added

Solve remaining zoom in/out problem for given test data

comment:13 by GerdP, 7 years ago

Tried it with 11632, validator problem is solved for the attached file.
Good work!
Problem with zooming in/out on this data still exists because OSMDataLayer.paint() uses it own routine to calculate the area.
I didn't find a good reason for this.
Attached patch v4 solves this.

comment:14 by GerdP, 7 years ago

Oops, sorry, please ignore patch v4, it doesn't work. I probably tested v3.

by GerdP, 7 years ago

Attachment: 10638-v5.patch added

Solve remaining zoom in/out problem for given test data (extract from v3)

comment:15 by GerdP, 7 years ago

OK, v5 is an extract of v3 to solve the remaining zoom problem. The wrong code in v4 ignored the different projection.
It would probably still be faster to apply the correct projection to the cached area returned by Dataset.getDataSourceArea().

comment:16 by Don-vip, 7 years ago

Milestone: 17.0317.02

comment:17 by Don-vip, 7 years ago

Resolution: fixed
Status: newclosed

In 11633/josm:

fix #10638 - fix zoom problem when validating "Way connected to Area" with a lot of data sources (patch by GerdP)

comment:18 by Don-vip, 7 years ago

In 11634/josm:

see #10638 - apply also optimization for initial DataSource area computation (patch by GerdP)

Modify Ticket

Change Properties
Set your email in Preferences
Action
as closed The owner will remain team.
as The resolution will be set.
The resolution will be deleted. Next status will be 'reopened'.

Add Comment


E-mail address and name can be saved in the Preferences .
 
Note: See TracTickets for help on using tickets.