Opened 16 years ago
Closed 15 years ago
#2075 closed defect (fixed)
API 0.6: don't upload more than 50K edits at once
| Reported by: | ivansanchez | Owned by: | Gubaer |
|---|---|---|---|
| Priority: | major | Milestone: | |
| Component: | Core | Version: | latest |
| Keywords: | 0.6 | Cc: |
Description
The current rails API will fail if JOSM tries to upload more than 50000 edits (in order to prevent massive changesets from locking the main DB for too long) in the same changeset. More than 50K edits at once is a very rare but possible scenario.
JOSM should split the edits in batches of 50K, or just commit the first 50K edits (and let the user hit "upload" again for the next 50K edits).
Attachments (0)
Change History (14)
comment:1 by , 16 years ago
comment:2 by , 16 years ago
DEFINITELY, a way of splitting up the upload would be massively useful. I have a changeset of ~ 2300 updates I'm trying to push to the server at this point in time and the server is crashing and not providing any feedback as to the error. To be able to break the update down into smaller chunks so I could actually work out which edit is causing the problem would go a long way to solving the issue.
comment:4 by , 16 years ago
| Owner: | changed from to |
|---|
follow-up: 6 comment:5 by , 15 years ago
Yes, I am also having this problem. I am often offline and when I get to upload there often a lots of conflicts to solve and then restart uploading from the begining. Even with only 400 changes it takes around one hour and if someone is online-editing the same time you have to restart uploading mutiple times.
Please split the upload in smallest pieces as possible. The changeset can stay open and you can upload piece after piece.
Thanks a lot !!
Skyper
follow-up: 8 comment:6 by , 15 years ago
Please split the upload in smallest pieces as possible. The changeset can stay open and you can upload piece after piece.
Thanks a lot !!
Try to disable the checkbox "Upload all changes in one request" in the Upload Dialog. JOSM then uploads your changes with the smallest possible granularity: one request per changed object. Uploading 400 changes with individual requests may take longer but you know better where you are in the upload process (better progress indication).
... and if someone is online-editing the same time you have to restart uploading mutiple times.
You can't avoid that with either upload approach. If data you've modified has been modified on the server too, you have to resolve the conflicts and you have to restart multiple times.
comment:7 by , 15 years ago
You could try using file > update data, which will download all the areas that you have previously downloaded. This will throw up all the conflicts that there are, thus make it far more likely to be able to upload in one go.
follow-up: 9 comment:8 by , 15 years ago
Replying to Gubaer:
... and if someone is online-editing the same time you have to restart uploading mutiple times.
You can't avoid that with either upload approach. If data you've modified has been modified on the server too, you have to resolve the conflicts and you have to restart multiple times.
But why do I have to restart all over again. Is is not possible to upload as much as possible then sync, resolve conflicts and upload the rest ?
follow-up: 10 comment:9 by , 15 years ago
Replying to skyper:
But why do I have to restart all over again. Is is not possible to upload as much as possible then sync, resolve conflicts and upload the rest ?
As stated above: Try to disable the checkbox "Upload all changes in one request" in the Upload Dialog.
Then it will upload one by one rather than everything at once.
follow-up: 11 comment:10 by , 15 years ago
Replying to anonymous:
As stated above: Try to disable the checkbox "Upload all changes in one request" in the Upload Dialog.
Then it will upload one by one rather than everything at once.
Do I have to close the changeset ?
comment:11 by , 15 years ago
Replying to skyper:
Do I have to close the changeset ?
No. The changeset will automatically close either 1 hour after the last change or 24 hours after opening, whichever comes first.
comment:13 by , 15 years ago
| Owner: | changed from to |
|---|
I'm going to work on this after the next release (assigning to myself). I feel major changes will be necessary and I'd like to combine it with #3684.
comment:14 by , 15 years ago
| Resolution: | → fixed |
|---|---|
| Status: | new → closed |
(In [2599]) fixed #4130: Chunked upload mode counter is wrong
fixed #4118: Upload dialog too complicated
fixed #4129: Hide the new "Upload data in one request/chunks/individually" behind an expanding "Upload method" box
fixed #2075: API 0.6: don't upload more than 50K edits at once
fixed #4044: Huge uploads never end [should be solved with chunked upload mode]
fixed #4110: Upload dialog spacing wrong
fixed #3386: Upload dialog has empty areas when the changeset doesn't include all of add/modify/delete operations
see #3369: bulk import helper [JOSM now supports multi changesets uploads]
See online help for more details.
Completes r2598
More specifically JOSM should be checking the capabilities of the server to see how many edits should be going into a changeset, before uploading, and splitting the upload into smaller chunks of multiple changesets.