Modify

Opened 2 months ago

Closed 5 weeks ago

Last modified 5 days ago

#17291 closed enhancement (fixed)

[Patch] improve handling of return code 404 in multifetch api

Reported by: GerdP Owned by: team
Priority: normal Milestone: 19.03
Component: Core Version:
Keywords: performance Cc:

Description

Please review:
With the current code the MultiFetchServerObjectReader switches to single gets when a list of ids returns a HTTP return code 404. It should instead divide the list into two halves and try again with both halves
unless the lists are very small.

The attached patch implements that approach.
There is also a constant

    /**
     * the max. number of primitives retrieved in one step. Assuming IDs with 7 digits,
     * this leads to a max. request URL of ~ 1600 Bytes ((7 digits +  1 Separator) * 200),
     * which should be safe according to the
     * <a href="http://www.boutell.com/newfaq/misc/urllength.html">WWW FAQ</a>.
     */
    private static final int MAX_IDS_PER_REQUEST = 200;

that looks suspicious as most node ids have 9 or 10 digits now. So maybe the limit is outaged?

Attachments (2)

17291.patch (2.0 KB) - added by GerdP 2 months ago.
17291-v2.patch (2.7 KB) - added by GerdP 5 weeks ago.

Download all attachments as: .zip

Change History (5)

Changed 2 months ago by GerdP

Attachment: 17291.patch added

Changed 5 weeks ago by GerdP

Attachment: 17291-v2.patch added

comment:1 Changed 5 weeks ago by GerdP

I've played with the MAX_IDS_PER_REQUEST constant. It seems that the server has no problems when I set it to 500, but with 1000 I see problems. Maybe it also depends on the server load. With the current value of 200 we often produce URLs with ~ 2200 bytes, so I decided to reduce the number to 170 in v2 of the patch.
If I hear no complains I'll commit this next monday.

comment:2 Changed 5 weeks ago by GerdP

Resolution: fixed
Status: newclosed

In 14898/josm:

fix #17291: improve handling of return code 404 in multifetch api

  • improve performance when server returns error code 404 (redacted): split workload into two halves instead of requesting each single id
  • reduce MAX_IDS_PER_REQUEST from 200 to 170 (this might decrease performace for large amounts of objects but should be safer)

comment:3 Changed 5 days ago by Don-vip

Milestone: 19.03

Modify Ticket

Change Properties
Set your email in Preferences
Action
as closed The owner will remain team.
as The resolution will be set.
The resolution will be deleted.

Add Comment


E-mail address and name can be saved in the Preferences.

 
Note: See TracTickets for help on using tickets.