Update Data should warn if one of the bounds has grown beyond 50k items

[mk408]

If a previous downloaded area (<bounds>) has, on the OSM server, grown to contain more than 50000 items, trying to update will fail due to the server returning an error that the requested download is too large.

A partial fix would be to continue with further requests, if there are any, and communicating the error to the user.

A complete fix would involve automatically splitting the area and retrying. Currently, I perform this manually by editing the .osm file.

[xeen]

What is the actual use case for this? I believe it’s very rare that an area grows beyond 50k items if it’s been downloaded recently. Of course, the error should be reported properly, but auto-splitting seems like a way to circumvent the server’s limits -- I doubt the 50k items are really a problem, it’s more likely it was implemented to prevent users from downloading way-too-large areas.

[stoecker]

We had multiple requests for "splitting download" in the past. The idea of the server limits is to keep users from selecting way too large areas and thus stressing the server. JOSM already has some ways to prevent this (e.g. the update data function), but implementing a tiled download as you request would finally break the idea of download restrictions.

Until we have a really important use-case where also OSM server admins would agree this is a WONTFIX.

[xeen]

While I agree with you, I doubt this ticket is fixed. If update data fails for that reason, we should announce that to the users and educate them. If we don’t, we’ll likely get further “implement splitting” tickets. However, as message like “Sorry, but your data set is too large. The server’s little mind can only handle that much without affecting other OSM users, so it kindly asks you to make smaller requests «insert donate link here»” might actually make a user change behavior instead of assuming it’s a bug.

I consider working on the bug; however I have a hard time to think of how to test this… it doesn’t sound to appealing to setup your own server and I don’t really want to send a lot of 50k+ requests to the server so it rejects me…

[stoecker]

Simply use test API, see ​http://wiki.openstreetmap.org/wiki/API_v0.6#URL_.2B_Authentication.

[xeen]

thanks for the tip, but there doesn’t seem to be data anywhere in the Dev-API. Let’s hope that the “50k exceeded” and “bbox too large” have the same HTTP code…

[xeen]

Well, there actually is a proper error message for that. I suggest rewording it then, but this is open for discussion. The current wording is:

+ tr("The area you tried to download is too big or your request was too large."
+ "<br>Either request a smaller area or use an export file provided by the OSM community.")

My suggestion (<br>s have been added so that the lines are roughly equal in length, but we probably want to make that a real, auto-breaking paragraph…):

+ tr("Your download is either too large or contains too many items.<br>"
+ "The server’s little mind only handles that much, before being<br>"
+ "slow for other OSM users. It kindly asks you to make smaller<br>"
+ "requests. You can also <a href=\"http://donate.openstreetmap.org/\">donate to keep it running</a>.");

[stoecker]

No sloppy messages please.

[xeen]

Different suggestion then:

Your download is either too large or contains too many items. The server can handle only that much before being slow for other OSM users. That’s why you may only make smaller requests. You can also donate to keep it running.

[Don-vip]

Does the problem now happens with 10k objects? (since the API lowered the maximum changeset elements from 50k to 10k)

[stoecker]

Replying to Don-vip:

Does the problem now happens with 10k objects? (since the API lowered the maximum changeset elements from 50k to 10k)

Also for downloads?

[Don-vip]

I don't know if the download limit is the same as upload limit.

[stoecker]

org.openstreetmap.josm.io.OsmApiException: ResponseCode=400, Error Header=<You requested too many nodes (limit is 50000). Either request a smaller area, or use planet.osm>

Still 50k.