Opened 4 years ago

Last modified 4 years ago

#22032 closed defect

[PATCH][RFC] Various memory enhancements for MVT tiles — at Version 1

Reported by: taylor.smock Owned by: team
Priority: normal Milestone: 22.06
Component: Core Version:
Keywords: performance Cc:

Description (last modified by taylor.smock)

JOSM start arugment: --download=39.0637818,-108.5670233,39.0660809,-108.5620022
Plugins: Mapillary (shift+, to download MVT tiles)

Memory allocations over 50s for r18430, limited to threads with protobuf related methods:

  • Run 1: 8,979,920,352 bytes
  • Run 2: 8,773,894,032 bytes

Memory allocations over 50s for r18431, limited to threads with protobuf related methods

  • Run 1: 7,653,697,192 bytes
  • Run 2: 7,133,589,400 bytes
  • Run 3: 7,532,364,336 bytes

I'll go ahead and merge this patch on Monday, absent objections.

Memory allocations over 50s for r18431 + patch, limited to threads with protobuf related methods:

  • Run 1: 823,974,256 bytes
  • Run 2: 850,025,712 bytes
  • Run 3: 756,248,936 bytes

Most of the performance enhancements come from:

  1. Avoiding various ArrayList#grow calls
    • A lot is from passing around a `ByteArrayOutputStream
  2. Caching bboxes in VectorNodes (I'm kind of ambivalent on this -- I don't want to keep the bboxes around, but a significant number of allocations came from QuadBuckets#search, of which a significant number of allocations came from VectorNode#getBBox, ~1,400,000,000 bytes, or ~20%)
  3. Caching nodes in a Map<ILatLon, VectorNode> map to avoid calling QuadBuckets#search when possible

The patch additionally fixes a bug with VectorRelation#setMembers and adds a default method putAll for Tagged.

Change History (3)

by taylor.smock, 4 years ago

Attachment: 22032.patch added

MVT performance improvements

comment:1 by taylor.smock, 4 years ago

Description: modified (diff)

by taylor.smock, 4 years ago

Attachment: flamegraph_22032.png added

Flamegraph (green == removed allocations, red == added allocations)

Note: See TracTickets for help on using tickets.