Modify

Opened 2 years ago

Closed 2 years ago

Last modified 2 years ago

#22104 closed defect (fixed)

[PATCH] Significantly reduce allocations in DataSetMerger when merging nodes

Reported by: taylor.smock Owned by: team
Priority: normal Milestone: 22.06
Component: Core Version:
Keywords: performance Cc:

Description

When I was profiling the MapWithAI plugin, I noticed that DataSetMerger#merge used ~400 MiB worth of allocations. With the attached patch, it uses ~19 MiB worth of allocations.

Steps to reproduce:

  1. osm-download.bounds=39.0635486;-108.5716152;39.0708789;-108.5593414
  2. Enable City of Grand Junction, CO, Esri Community Trees, Esri Roads Sample (Preview), MapWithAI, Statewide Aggregate Addresses in Colorado 2019 (Public), and United States Addresses.
  3. Start profiling and download data + mapwithai data.

Attachments (1)

22104.patch (6.3 KB ) - added by taylor.smock 2 years ago.

Download all attachments as: .zip

Change History (12)

by taylor.smock, 2 years ago

Attachment: 22104.patch added

comment:1 by taylor.smock, 2 years ago

Keywords: performance added

comment:2 by taylor.smock, 2 years ago

Related ticket: #7159

comment:3 by taylor.smock, 2 years ago

Resolution: fixed
Status: newclosed

In 18464/josm:

Extract equalsEpsilon from LatLon into ILatLon

This significantly reduces allocations from DataSetMerger#merge.
In testing, it went from ~400 MiB allocations to ~19 MiB allocations.

The allocations largely came from checking if two nodes had equal coordinates.
The code used LatLon#getCoor followed by equalsEpsilon, when all the
required information was already present in Node, but LatLon had the method.

As an additional enhancement, equalsEpsilon was somewhat generified, in that
there is an additional method where we can specify a different precision.

This fixes #22104, see #7159.

comment:4 by Don-vip, 2 years ago

In 18489/josm:

see #22104 - fix deprecation warnings

in reply to:  4 ; comment:5 by taylor.smock, 2 years ago

Replying to Don-vip:

In 18489/josm:

see #22104 - fix deprecation warnings

Fair enough. I hadn't done that since it would have automatically been fixed when/if we dropped the code in LatLon. I suppose I also could have cast the variables to ILatLon as well.

comment:6 by Don-vip, 2 years ago

In 35975/osm:

see #22104 - fix deprecation warnings

in reply to:  5 ; comment:7 by Don-vip, 2 years ago

Replying to taylor.smock:

Fair enough. I hadn't done that since it would have automatically been fixed when/if we dropped the code in LatLon. I suppose I also could have cast the variables to ILatLon as well.

I hesitated as well to remove the method immediately. But the code being there for a long time, it's safer to keep it deprecated for plugins we don't control.

in reply to:  7 comment:8 by taylor.smock, 2 years ago

Replying to Don-vip:

I hesitated as well to remove the method immediately. But the code being there for a long time, it's safer to keep it deprecated for plugins we don't control.

That is one of the reasons why I marked it as deprecated, instead of removing it. The hard part is that it shadows/overrides the ILatLon method without casting the second argument to an ILatLon (I don't believe the Java compiler is smart enough to understand LatLon#equalsEpsilon is deprecated, so it should use ILatLon#equalsEpsilon instead). So at some point, when we do decide to remove the method, we're going to have breakage.

With that being said, I do have a GitLab CI job I can use to find plugins where it will cause breakage.

in reply to:  6 comment:9 by taylor.smock, 2 years ago

Replying to Don-vip:

In 35975/osm:

see #22104 - fix deprecation warnings

Sorry, I missed this.

I was going to update the plugins after the next tested release. Mostly because I was going to try and do the changes for #22115 at the same time.

Anyway, I'll go ahead and do the changes that I intended to do for this ticket now, so I don't miss them (specifically, Node#getCoor#equalsEpsilon -> Node#equalsEpsilon).

comment:10 by taylor.smock, 2 years ago

In 35976/osm:

see #22104: Remove usages of Node#getCoor where possible

This also accounts for cases where Node has the methods used later,
so a new LatLon is unnecessary.

comment:11 by taylor.smock, 2 years ago

In 35977/osm:

see #22104 (dist): Remove usages of Node#getCoor where possible

This also accounts for cases where Node has the methods used later,
so a new LatLon is unnecessary.

Modify Ticket

Change Properties
Set your email in Preferences
Action
as closed The owner will remain team.
as The resolution will be set.
The resolution will be deleted. Next status will be 'reopened'.

Add Comment


E-mail address and name can be saved in the Preferences .
 
Note: See TracTickets for help on using tickets.