Modify ↓
Opened 3 years ago
Last modified 7 months ago
#21563 new defect
Opening history of relation with high version number leads to exceeding OSM bandwidth limit
| Reported by: | skyper | Owned by: | team |
|---|---|---|---|
| Priority: | normal | Milestone: | |
| Component: | Core | Version: | |
| Keywords: | template_report history error 509 bandwidth limit download | Cc: |
Description (last modified by )
What steps will reproduce the problem?
- Have a relation with quite some versions, in my case 1281 (relation/2769137)
- Open history without noticing the high version number
- Cancel download after some seconds as it takes too long
- Notice the high version number and open history
- Sucessfully, look at the last 20 version
- Download a member
What is the expected result?
The member is downloaded and other downloads work as expected
What happens instead?
SEVERE: Bandwidth Limit Exceeded - <html>Communication with the OSM server 'api.openstreetmap.org'failed. The server replied the following error code and the following error message: <strong>Error code:<strong> 509 <strong>Error message (untranslated)</strong>: You have downloaded too much data. Please try again later.</html>
and the user is temporarly blocked
Please provide any additional information below. Attach a screenshot if possible.
I am not sure what to do here. Maybe, only loading the last 20/50/100 version could help or downloading not all versions from main server.
Anyway, this is annoying as I got blocked without even really noticing that I may exceed the limit and I only wanted to take a look at the versions of the last couple month and not at all versions of the last decade.
Relative:URL: ^/trunk Repository:UUID: 0c6e7542-c601-0410-84e7-c038aed88b3b Last:Changed Date: 2021-11-12 11:23:59 +0100 (Fri, 12 Nov 2021) Revision:18323 Build-Date:2021-11-13 02:31:03 URL:https://josm.openstreetmap.de/svn/trunk Last errors/warnings: - 12417.014 E: java.util.concurrent.ExecutionException: org.openstreetmap.josm.io.OsmApiException: ResponseCode=509, Error Header=<You have downloaded too much data. Please try again later.>. Cause: org.openstreetmap.josm.io.OsmApiException: ResponseCode=509, Error Header=<You have downloaded too much data. Please try again later.> - 12417.014 E: org.openstreetmap.josm.io.OsmApiException: ResponseCode=509, Error Header=<You have downloaded too much data. Please try again later.> - 12417.014 E: org.openstreetmap.josm.io.OsmApiException: ResponseCode=509, Error Header=<You have downloaded too much data. Please try again later.> - 12417.020 E: Bandwidth Limit Exceeded - <html>Communication with the OSM server 'api.openstreetmap.org'failed. The server replied<br>the following error code and the following error message:<br><strong>Error code:<strong> 509<br><strong>Error message (untranslated)</strong>: You have downloaded too much data. Please try again later.</html> - 12445.113 E: org.openstreetmap.josm.io.OsmApiException: ResponseCode=509, Error Header=<You have downloaded too much data. Please try again later.> - 12445.113 E: org.openstreetmap.josm.io.OsmApiException: ResponseCode=509, Error Header=<You have downloaded too much data. Please try again later.> - 12445.125 E: Bandwidth Limit Exceeded - <html>Communication with the OSM server 'api.openstreetmap.org'failed. The server replied<br>the following error code and the following error message:<br><strong>Error code:<strong> 509<br><strong>Error message (untranslated)</strong>: You have downloaded too much data. Please try again later.</html> - 12527.828 E: org.openstreetmap.josm.io.OsmApiException: ResponseCode=509, Error Header=<You have downloaded too much data. Please try again later.> - 12527.829 E: org.openstreetmap.josm.io.OsmApiException: ResponseCode=509, Error Header=<You have downloaded too much data. Please try again later.> - 12527.837 E: Bandwidth Limit Exceeded - <html>Communication with the OSM server 'api.openstreetmap.org'failed. The server replied<br>the following error code and the following error message:<br><strong>Error code:<strong> 509<br><strong>Error message (untranslated)</strong>: You have downloaded too much data. Please try again later.</html>
Attachments (0)
Change History (6)
comment:1 by , 3 years ago
comment:2 by , 3 years ago
| Description: | modified (diff) |
|---|
comment:3 by , 16 months ago
Looking at the debug output,
GET https://api.openstreetmap.org/api/0.6/relation/2769137/history -> HTTP_2 200 (4.6 s)
RESPONSE HEADERS: {:status=[200], cache-control=[private, max-age=0, must-revalidate], content-encoding=[gzip], content-type=[application/xml; charset=utf-8], date=[Thu, 27 Apr 2023 20:16:57 GMT], server=[Apache/2.4.54 (Ubuntu)], strict-transport-security=[max-age=31536000; includeSubDomains; preload]}
SAX parsing completed in 27.1 s
GET https://api.openstreetmap.org/api/0.6/changesets?changesets=15235306,15237358,17113347,15876341,124125036,28633534,31203799,46088883,70132770,87565620,27756983,60185478,47051481,87657772,88411475,89204038,50529051,75930782,99761642,69637183,88687952,88470879,94469563,37962339,27763075,88472931,52132663,88808805,111122094,88175989,88227190,48372439,100150735,32147922,72971363,100505031,50871092,28631434,44155618,88167687,16629946,24863036,68963419,87647601,36334179,88331528,68627545,81585298,88814878,16138401,89351428,87725410,88630545,25219547,33573467,22608132,88292637,53187403,88368421,57977623,41327122,55030563,124100353,29413799,38064685,70542431,74071046,88636708,20490581,87567702,16793967,23404820,89255200,94121453,17930594,73681936,27759058,23130391,69389403,88515900,52368228,28819912,24064493,88477124,37827268,27844896,83078266,87657908,20179362,15231098,89079261,49629798,37020321,71218344,100363629,48097866,30165332,58608614,74077425,88083931 -> HTTP_2 200 (351 ms; 10.0 kB)
RESPONSE HEADERS: {:status=[200], cache-control=[max-age=0, private, must-revalidate], content-encoding=[gzip], content-length=[10248], content-security-policy=[default-src 'self'; child-src 'self'; connect-src 'self' matomo.openstreetmap.org; font-src 'none'; form-action 'self'; frame-ancestors 'self'; frame-src 'self'; img-src 'self' data: www.gravatar.com *.wp.com tile.openstreetmap.org *.tile.openstreetmap.org *.tile.thunderforest.com tileserver.memomaps.de *.openstreetmap.fr matomo.openstreetmap.org https://openstreetmap-user-avatars.s3.dualstack.eu-west-1.amazonaws.com https://openstreetmap-gps-images.s3.dualstack.eu-west-1.amazonaws.com; manifest-src 'self'; media-src 'none'; object-src 'self'; script-src 'self' matomo.openstreetmap.org; style-src 'self'; worker-src 'none'], content-type=[application/xml; charset=utf-8], date=[Thu, 27 Apr 2023 20:17:29 GMT], etag=[W/"b399ad8462160d9b2881c53c8ca36b1c-gzip"], referrer-policy=[strict-origin-when-cross-origin], server=[Apache/2.4.54 (Ubuntu)], status=[200 OK], strict-transport-security=[max-age=31536000; includeSubDomains; preload], vary=[Origin,Accept-Encoding], x-content-type-options=[nosniff], x-download-options=[noopen], x-frame-options=[sameorigin], x-permitted-cross-domain-policies=[none], x-powered-by=[Phusion Passenger(R) 6.0.17], x-request-id=[ZErYWRo4dMYNK0nyfjfVjgAAqi0], x-runtime=[0.197043], x-xss-protection=[1; mode=block]}
SAX parsing completed in 5 ms
GET https://api.openstreetmap.org/api/0.6/changesets?changesets=87655846,84473256,24988124,83299412,17205664,27767041,88962538,35111615,20695442,88458732,86140302,48185968,26626343,72635623,88811005,43534895,89179623,46772855,58254278,19087772,46406148,87612922,88040951,68619486,88626562,55221126,52919285,101451487,66421536,88528275,94166346,20666860,31336713,34593492,36569842,71634075,79596640,21770669,88511911,16367643,16087056,39127728,65157897,29014358,20169182,50508783,28957014,67261687,61158226,21289406,65698582,44133281,22646872,88151106,33227004,70599986,33540343,50397706,88243273,80903644,19998757,21608541,24512611,85013508,17225745,95351990,100972316,17412112,55186003,88255579,17162264,58305124,46240735,15223287,88351835,20690970,79820263,56380027,72578420,15235521,17352737,72973680,89248888,84964411,18491447,88173676,31236332,38996835,50553394,82496987,96480497,22474850,17205299,58053196,35334944,93330605,77596063,20041840,21248005,27912416 -> HTTP_2 200 (376 ms; 10.3 kB)
RESPONSE HEADERS: {:status=[200], cache-control=[max-age=0, private, must-revalidate], content-encoding=[gzip], content-length=[10552], content-security-policy=[default-src 'self'; child-src 'self'; connect-src 'self' matomo.openstreetmap.org; font-src 'none'; form-action 'self'; frame-ancestors 'self'; frame-src 'self'; img-src 'self' data: www.gravatar.com *.wp.com tile.openstreetmap.org *.tile.openstreetmap.org *.tile.thunderforest.com tileserver.memomaps.de *.openstreetmap.fr matomo.openstreetmap.org https://openstreetmap-user-avatars.s3.dualstack.eu-west-1.amazonaws.com https://openstreetmap-gps-images.s3.dualstack.eu-west-1.amazonaws.com; manifest-src 'self'; media-src 'none'; object-src 'self'; script-src 'self' matomo.openstreetmap.org; style-src 'self'; worker-src 'none'], content-type=[application/xml; charset=utf-8], date=[Thu, 27 Apr 2023 20:17:29 GMT], etag=[W/"81a0e0f3be92f68586a2bf182fc71f32-gzip"], referrer-policy=[strict-origin-when-cross-origin], server=[Apache/2.4.54 (Ubuntu)], status=[200 OK], strict-transport-security=[max-age=31536000; includeSubDomains; preload], vary=[Origin,Accept-Encoding], x-content-type-options=[nosniff], x-download-options=[noopen], x-frame-options=[sameorigin], x-permitted-cross-domain-policies=[none], x-powered-by=[Phusion Passenger(R) 6.0.17], x-request-id=[ZErYWRo4dMYNK0nyfjfVkAAAnC0], x-runtime=[0.229301], x-xss-protection=[1; mode=block]}
SAX parsing completed in 6 ms
GET https://api.openstreetmap.org/api/0.6/changesets?changesets=89312281,101386054,44137454,35578703,15452600,15755687,89072664,94467321,30310551,57973290,72992007,61565684,20420695,88067167,33995623,17688687,101709675,36113216,88982570,99675287,55716411,23191563,36342599,15235463,89351228,28819652,93734117,16679306,18221154,26947823,52150894,88142906,15995279,70206804,87362633,41286410,71073212,85734556,88511686,88282306,17440898,20168891,31985763,37682096,87643304,75640080,88960222,88450257,89070811,28123195,89345221,31785085,88042663,79871323,87725217,88349912,85734530,49959770,29229084,30150766,16171349,88806639,87772318,83807572,89165053,24432859,26040377,15083859,63404670,52114080,18075823,24891588,95677555,85458132,23347360,45677437,88253573,88409231,34147266,91747516,46187277,20177148,79975701,52040398,77892977,88177816,20656359,99542066,36709358,39787393,47569675,85626106,87655643,88528036,100904928,88974505,17854711,78099776,95632468,32956447 -> HTTP_2 200 (366 ms; 10.2 kB)
RESPONSE HEADERS: {:status=[200], cache-control=[max-age=0, private, must-revalidate], content-encoding=[gzip], content-length=[10465], content-security-policy=[default-src 'self'; child-src 'self'; connect-src 'self' matomo.openstreetmap.org; font-src 'none'; form-action 'self'; frame-ancestors 'self'; frame-src 'self'; img-src 'self' data: www.gravatar.com *.wp.com tile.openstreetmap.org *.tile.openstreetmap.org *.tile.thunderforest.com tileserver.memomaps.de *.openstreetmap.fr matomo.openstreetmap.org https://openstreetmap-user-avatars.s3.dualstack.eu-west-1.amazonaws.com https://openstreetmap-gps-images.s3.dualstack.eu-west-1.amazonaws.com; manifest-src 'self'; media-src 'none'; object-src 'self'; script-src 'self' matomo.openstreetmap.org; style-src 'self'; worker-src 'none'], content-type=[application/xml; charset=utf-8], date=[Thu, 27 Apr 2023 20:17:29 GMT], etag=[W/"964a54573583caafb86ed7bd3876bed5-gzip"], referrer-policy=[strict-origin-when-cross-origin], server=[Apache/2.4.54 (Ubuntu)], status=[200 OK], strict-transport-security=[max-age=31536000; includeSubDomains; preload], vary=[Origin,Accept-Encoding], x-content-type-options=[nosniff], x-download-options=[noopen], x-frame-options=[sameorigin], x-permitted-cross-domain-policies=[none], x-powered-by=[Phusion Passenger(R) 6.0.17], x-request-id=[ZErYWRo4dMYNK0nyfjfVkgAAqC0], x-runtime=[0.215565], x-xss-protection=[1; mode=block]}
SAX parsing completed in 3 ms
GET https://api.openstreetmap.org/api/0.6/changesets?changesets=63044136,79820076,88452267,16556307,17176809,42287988,55454399,56142506,25059469,51086072,20979901,114847527,63173059,15829749,87658300,100147197,40042596,88604480,34129920,47038660,89100102,62153151,21764931,50397454,89077568,91801453,88528720,67153431,88661843,15635184,64297419,24732505,53754132,42727594,15338184,60047795,61530509,50374956,99948507,24597317,88326001,46749948,88098679,88614783,100992061,88743795,88475516,58020168,27448290,89083663,38203407,87656304,28298213,85958485,88336136,20593524,86636391,46096624,56185098,31302536,61864441,89286404,74913327,52511096,69639804,64967044,88571674,19088253,88667926,16177833,88278812,73442817,41646104,18678648,89308981,41941013,88096558,24099584,98361267,88495927,69639765,27075563,78415578,16603773,29530947,70526641,53623221,67122823,66277755,46350410,89094086,16593527,72658655,16384623,22389696,49856621,69639857,72693452,89233355,33894558 -> HTTP_2 200 (375 ms; 10.5 kB)
RESPONSE HEADERS: {:status=[200], cache-control=[max-age=0, private, must-revalidate], content-encoding=[gzip], content-length=[10742], content-security-policy=[default-src 'self'; child-src 'self'; connect-src 'self' matomo.openstreetmap.org; font-src 'none'; form-action 'self'; frame-ancestors 'self'; frame-src 'self'; img-src 'self' data: www.gravatar.com *.wp.com tile.openstreetmap.org *.tile.openstreetmap.org *.tile.thunderforest.com tileserver.memomaps.de *.openstreetmap.fr matomo.openstreetmap.org https://openstreetmap-user-avatars.s3.dualstack.eu-west-1.amazonaws.com https://openstreetmap-gps-images.s3.dualstack.eu-west-1.amazonaws.com; manifest-src 'self'; media-src 'none'; object-src 'self'; script-src 'self' matomo.openstreetmap.org; style-src 'self'; worker-src 'none'], content-type=[application/xml; charset=utf-8], date=[Thu, 27 Apr 2023 20:17:30 GMT], etag=[W/"c6c5b19c552c222841eb3df8e7cac943-gzip"], referrer-policy=[strict-origin-when-cross-origin], server=[Apache/2.4.54 (Ubuntu)], status=[200 OK], strict-transport-security=[max-age=31536000; includeSubDomains; preload], vary=[Origin,Accept-Encoding], x-content-type-options=[nosniff], x-download-options=[noopen], x-frame-options=[sameorigin], x-permitted-cross-domain-policies=[none], x-powered-by=[Phusion Passenger(R) 6.0.17], x-request-id=[ZErYWho4dMYNK0nyfjfVlQAAki0], x-runtime=[0.214004], x-xss-protection=[1; mode=block]}
SAX parsing completed in 3 ms
GET https://api.openstreetmap.org/api/0.6/changesets?changesets=46321758,53785000,16622176,17148824,31648639,19125181,23964659,49631338,23952333,124128711,19063684,88778733,43681841,111272970,15350340,56306165,63445350,88414187,40042707,64004450,29424500,88494064,30131059,60725543,29406074,74219223,46186621,35641566,50397632,16663100,114492433,73905828,103015653,22104986,88500109,17386457,51235293,60506447,21179286,88121237,38582427,83121709,88960916,89288587,62624097,39250058,61524343,16952284,50102306,52132293,87414741,88813481,36493514,50397673,74862238,17798118,27743049,75091594,19442652,124128659,61292893,68843233,23071645,124093848,18428441,51015691,51134472,95446713,17205772,44634531,69308205,30132934,88135246,96454317,83324900,72068957,88817239,29760230,95352462,45992348,36317448,88086114,48865738,72966002,16694227,15778782,72380286,70295301,88168052,88528499,89253484,55414899,45220237,84573749,99761843,25008703,74180392,31265429,39178527,88684037 -> HTTP_2 200 (599 ms; 10.4 kB)
RESPONSE HEADERS: {:status=[200], cache-control=[max-age=0, private, must-revalidate], content-encoding=[gzip], content-length=[10680], content-security-policy=[default-src 'self'; child-src 'self'; connect-src 'self' matomo.openstreetmap.org; font-src 'none'; form-action 'self'; frame-ancestors 'self'; frame-src 'self'; img-src 'self' data: www.gravatar.com *.wp.com tile.openstreetmap.org *.tile.openstreetmap.org *.tile.thunderforest.com tileserver.memomaps.de *.openstreetmap.fr matomo.openstreetmap.org https://openstreetmap-user-avatars.s3.dualstack.eu-west-1.amazonaws.com https://openstreetmap-gps-images.s3.dualstack.eu-west-1.amazonaws.com; manifest-src 'self'; media-src 'none'; object-src 'self'; script-src 'self' matomo.openstreetmap.org; style-src 'self'; worker-src 'none'], content-type=[application/xml; charset=utf-8], date=[Thu, 27 Apr 2023 20:17:30 GMT], etag=[W/"ecbbc212779aa157bbd3e5ad3132d8ee-gzip"], referrer-policy=[strict-origin-when-cross-origin], server=[Apache/2.4.54 (Ubuntu)], status=[200 OK], strict-transport-security=[max-age=31536000; includeSubDomains; preload], vary=[Origin,Accept-Encoding], x-content-type-options=[nosniff], x-download-options=[noopen], x-frame-options=[sameorigin], x-permitted-cross-domain-policies=[none], x-powered-by=[Phusion Passenger(R) 6.0.17], x-request-id=[ZErYWho4dMYNK0nyfjfVlgAAhS0], x-runtime=[0.318229], x-xss-protection=[1; mode=block]}
SAX parsing completed in 2 ms
GET https://api.openstreetmap.org/api/0.6/changesets?changesets=16358324,88503819,30016158,34068819,18080325,69932925,28631785,93824713,88051294,17461869,88686112,67628906,37785902,74647324,87658070,86867545,39526702,74143517,40042772,19503697,71305016,61831381,75325191,87656003,50632824,74051348,114879909,52549726,89339603,88821448,88039099,100365951,79404862,87830201,22608593,50374794,68843408,25635339,50397325,35307914,64964687,69449654,88143576,124128510,25002721,88133340,88639189,48996683,74815445,69361539,58118355,88364770,20495007,33053270,83322712,87885452,88157936,80740221,22149862,102573477,15223633,74385366,71340026,22459117,85350061,89247466,85341866,83087181,38511092,16393010,25426506,20683504,66938934,48996646,87920364,15223610,58312875,88405654,88497810,22155911,74233785,69640176,43532623,50374872,74172338,15717173,48859445,88159901,25584219,55242900,88467099,48775479,26153581,19575497,72677298,34572773,70627271,73029545,24865422,52424931 -> HTTP_2 200 (613 ms; 9.91 kB)
RESPONSE HEADERS: {:status=[200], cache-control=[max-age=0, private, must-revalidate], content-encoding=[gzip], content-length=[10150], content-security-policy=[default-src 'self'; child-src 'self'; connect-src 'self' matomo.openstreetmap.org; font-src 'none'; form-action 'self'; frame-ancestors 'self'; frame-src 'self'; img-src 'self' data: www.gravatar.com *.wp.com tile.openstreetmap.org *.tile.openstreetmap.org *.tile.thunderforest.com tileserver.memomaps.de *.openstreetmap.fr matomo.openstreetmap.org https://openstreetmap-user-avatars.s3.dualstack.eu-west-1.amazonaws.com https://openstreetmap-gps-images.s3.dualstack.eu-west-1.amazonaws.com; manifest-src 'self'; media-src 'none'; object-src 'self'; script-src 'self' matomo.openstreetmap.org; style-src 'self'; worker-src 'none'], content-type=[application/xml; charset=utf-8], date=[Thu, 27 Apr 2023 20:17:31 GMT], etag=[W/"706d01c182124281da2a33ccaaede46d-gzip"], referrer-policy=[strict-origin-when-cross-origin], server=[Apache/2.4.54 (Ubuntu)], status=[200 OK], strict-transport-security=[max-age=31536000; includeSubDomains; preload], vary=[Origin,Accept-Encoding], x-content-type-options=[nosniff], x-download-options=[noopen], x-frame-options=[sameorigin], x-permitted-cross-domain-policies=[none], x-powered-by=[Phusion Passenger(R) 6.0.17], x-request-id=[ZErYWxo4dMYNK0nyfjfVlwAAny0], x-runtime=[0.273742], x-xss-protection=[1; mode=block]}
SAX parsing completed in 2 ms
GET https://api.openstreetmap.org/api/0.6/changesets?changesets=42742130,17253637,102949409,25693580,66088952,18607378,20122936,74188903,15078635,64966610,28984756,15496444,88672601,89319746,38162004,26731907,71039023,18609409,102949432,87724321,51484428,32966121,112992931,23637322,74164297,55408488,33469912,44269189,89082216,88258922,61554561,52369202,27637130,24511814,29328783,56379241,67520574,17466675,88410495,97325559,32110034,24519990,88680712,98363806,30342541,39818780,55334671,29306363,88422657,65159083,57513767,42426057,51519300,68567104,69658736,88498463,50374491,26721739,84906323,18081102,30438798,19828047,38362664,88783144,32058766,88168745,28569050,26861043,43796214,31982979,39814708,34371168,71333933,38415931,74074136,38948385,86002004,81471661,88473918,88090938,25292282,38950441,68839521,28630473,16641149,31235420,36228776,88197569,16903552,124099556,89201115,34961048,88822214,23309804,41090813,26737925,31114568,53884834,124130290,88095189 -> HTTP_2 200 (613 ms; 10.3 kB)
RESPONSE HEADERS: {:status=[200], cache-control=[max-age=0, private, must-revalidate], content-encoding=[gzip], content-length=[10545], content-security-policy=[default-src 'self'; child-src 'self'; connect-src 'self' matomo.openstreetmap.org; font-src 'none'; form-action 'self'; frame-ancestors 'self'; frame-src 'self'; img-src 'self' data: www.gravatar.com *.wp.com tile.openstreetmap.org *.tile.openstreetmap.org *.tile.thunderforest.com tileserver.memomaps.de *.openstreetmap.fr matomo.openstreetmap.org https://openstreetmap-user-avatars.s3.dualstack.eu-west-1.amazonaws.com https://openstreetmap-gps-images.s3.dualstack.eu-west-1.amazonaws.com; manifest-src 'self'; media-src 'none'; object-src 'self'; script-src 'self' matomo.openstreetmap.org; style-src 'self'; worker-src 'none'], content-type=[application/xml; charset=utf-8], date=[Thu, 27 Apr 2023 20:17:31 GMT], etag=[W/"f087e57d21e4cc332e59791dbc03863a-gzip"], referrer-policy=[strict-origin-when-cross-origin], server=[Apache/2.4.54 (Ubuntu)], status=[200 OK], strict-transport-security=[max-age=31536000; includeSubDomains; preload], vary=[Origin,Accept-Encoding], x-content-type-options=[nosniff], x-download-options=[noopen], x-frame-options=[sameorigin], x-permitted-cross-domain-policies=[none], x-powered-by=[Phusion Passenger(R) 6.0.17], x-request-id=[ZErYWxo4dMYNK0nyfjfVmQAAoS0], x-runtime=[0.257034], x-xss-protection=[1; mode=block]}
SAX parsing completed in 2 ms
GET https://api.openstreetmap.org/api/0.6/changesets?changesets=101880455,61527869,81078338,26852608,57972717,100991647,86340025,89188812,88074654,89205232,17237413,93442355,88502753,89094631,31960398,45057541,59514663,88805856,30901627,57182149,37557906,88816121,30866787,44043797,88959480,45635086,64970599,88459773,31294823,54030210,61568785,71100550,36329105,66934595,41381576,88379783,88775055,35907300,51486678,91836849,23248296,34911962,106545814,20663787,74055866,88142228,110252614,46210582,15078462,20583905,18451909,88693141,89166221,114108944,66740004,71153884,28845402,41307797,66508565,29547814,15078400,29230422,73377973,74119296,88447407,98521397,25589098,70230211,88529320,93364606,15813633,19989953,50399216,68208865,15305754,54181833,33918705,51824610,17751547,74217626,63375167,17323508,88811959,48373279,76298353,87724485,39833439,100376825,18586655,71813443,96010431,56198737,28933303,72735067,66795243,27444407,20599852,89344068,15957481,36306738 -> HTTP_2 200 (611 ms; 10.2 kB)
RESPONSE HEADERS: {:status=[200], cache-control=[max-age=0, private, must-revalidate], content-encoding=[gzip], content-length=[10468], content-security-policy=[default-src 'self'; child-src 'self'; connect-src 'self' matomo.openstreetmap.org; font-src 'none'; form-action 'self'; frame-ancestors 'self'; frame-src 'self'; img-src 'self' data: www.gravatar.com *.wp.com tile.openstreetmap.org *.tile.openstreetmap.org *.tile.thunderforest.com tileserver.memomaps.de *.openstreetmap.fr matomo.openstreetmap.org https://openstreetmap-user-avatars.s3.dualstack.eu-west-1.amazonaws.com https://openstreetmap-gps-images.s3.dualstack.eu-west-1.amazonaws.com; manifest-src 'self'; media-src 'none'; object-src 'self'; script-src 'self' matomo.openstreetmap.org; style-src 'self'; worker-src 'none'], content-type=[application/xml; charset=utf-8], date=[Thu, 27 Apr 2023 20:17:32 GMT], etag=[W/"803fbd03fa41e93701ef4b4d6157533f-gzip"], referrer-policy=[strict-origin-when-cross-origin], server=[Apache/2.4.54 (Ubuntu)], status=[200 OK], strict-transport-security=[max-age=31536000; includeSubDomains; preload], vary=[Origin,Accept-Encoding], x-content-type-options=[nosniff], x-download-options=[noopen], x-frame-options=[sameorigin], x-permitted-cross-domain-policies=[none], x-powered-by=[Phusion Passenger(R) 6.0.17], x-request-id=[ZErYXBo4dMYNK0nyfjfVmgAAky0], x-runtime=[0.251775], x-xss-protection=[1; mode=block]}
SAX parsing completed in 3 ms
GET https://api.openstreetmap.org/api/0.6/changesets?changesets=106545986,114940869,52020743,16938016,33375455,88780910,54035997,88815725,34961202,53528087,51402297,69171510,72692092,88813690,74006876,50398777,87928839,17151035,40515370,85778527,89313297,28067048,26733778,63702665,69640544,87648383,83167676,17529921,100918089,21406733,18017373,70785399,61574904,23247927,56645173,88416285,88616981,43964353,73033989,88528934,96620708,100858725,88807458,18099326,88545321,30143674,70744390,59182837,23956505,55858720,33816440,87658562,71493950,88748086,49629066,23360742,69431713,88299713,88547522,86040742,89092296,87656624,87050426,18099353,24491256,33804172,102568850,88199376,100575338,124130035,64548431,100128878,20602017,34545556,60689925,17904780,18273462,74101192,23209158,18101436,88799458,21595360,87689367,88746212,90788039,16854194,99762241,15238483,61849106,88584441,18261206,52121307,19451113,88256640,17259713,75121074,17466566,88635527,61388408,33806290 -> HTTP_2 200 (623 ms; 10.4 kB)
RESPONSE HEADERS: {:status=[200], cache-control=[max-age=0, private, must-revalidate], content-encoding=[gzip], content-length=[10613], content-security-policy=[default-src 'self'; child-src 'self'; connect-src 'self' matomo.openstreetmap.org; font-src 'none'; form-action 'self'; frame-ancestors 'self'; frame-src 'self'; img-src 'self' data: www.gravatar.com *.wp.com tile.openstreetmap.org *.tile.openstreetmap.org *.tile.thunderforest.com tileserver.memomaps.de *.openstreetmap.fr matomo.openstreetmap.org https://openstreetmap-user-avatars.s3.dualstack.eu-west-1.amazonaws.com https://openstreetmap-gps-images.s3.dualstack.eu-west-1.amazonaws.com; manifest-src 'self'; media-src 'none'; object-src 'self'; script-src 'self' matomo.openstreetmap.org; style-src 'self'; worker-src 'none'], content-type=[application/xml; charset=utf-8], date=[Thu, 27 Apr 2023 20:17:32 GMT], etag=[W/"3cbdb8241a84fd246da79f88b8fe3573-gzip"], referrer-policy=[strict-origin-when-cross-origin], server=[Apache/2.4.54 (Ubuntu)], status=[200 OK], strict-transport-security=[max-age=31536000; includeSubDomains; preload], vary=[Origin,Accept-Encoding], x-content-type-options=[nosniff], x-download-options=[noopen], x-frame-options=[sameorigin], x-permitted-cross-domain-policies=[none], x-powered-by=[Phusion Passenger(R) 6.0.17], x-request-id=[ZErYXBo4dMYNK0nyfjfVnAAApS0], x-runtime=[0.244171], x-xss-protection=[1; mode=block]}
SAX parsing completed in 1 ms
GET https://api.openstreetmap.org/api/0.6/changesets?changesets=17413341,33824720,87562476,89110682,50374358,41424801,18359490,89071764,58482337,89342093,30067754,56585918,86022337,69663173,86782153,16866535,39159731,28568660,39815093,74011010,22469819,29969446,69659081,88486057,44791643,61566553,88156338,72071615,88152245,19088598,58261135,64243235,15078684,37720005,15236371,33976315,56184155,85797662,100363263,79172286,78121647,19769120,88562506,88210255,18224921,88578903,41804911,70785581,100500456,88288085,18224897,88613727,17205023,88494943,63108574,27174784,38513748,70140431,89101166,18988802,20739871,35316798,48176377,61599105,47035617,87657237,88013586,21247851,54556017,88787832,17327921,88472447,88619892,88818550,74078804,37506051,24581943,30285710,18224983,89293584,46220420,86051686,85902166,24872757,32855995,88820485,89598744,88195861,28645344,93615041,15609529,15750823,88222485,88021862,20600672,94163910,32729000,70642281,83288746,15926957 -> HTTP_2 200 (412 ms; 10.1 kB)
RESPONSE HEADERS: {:status=[200], cache-control=[max-age=0, private, must-revalidate], content-encoding=[gzip], content-length=[10341], content-security-policy=[default-src 'self'; child-src 'self'; connect-src 'self' matomo.openstreetmap.org; font-src 'none'; form-action 'self'; frame-ancestors 'self'; frame-src 'self'; img-src 'self' data: www.gravatar.com *.wp.com tile.openstreetmap.org *.tile.openstreetmap.org *.tile.thunderforest.com tileserver.memomaps.de *.openstreetmap.fr matomo.openstreetmap.org https://openstreetmap-user-avatars.s3.dualstack.eu-west-1.amazonaws.com https://openstreetmap-gps-images.s3.dualstack.eu-west-1.amazonaws.com; manifest-src 'self'; media-src 'none'; object-src 'self'; script-src 'self' matomo.openstreetmap.org; style-src 'self'; worker-src 'none'], content-type=[application/xml; charset=utf-8], date=[Thu, 27 Apr 2023 20:17:33 GMT], etag=[W/"b6b2da38b5631d85fc4b1ba7fcdcacde-gzip"], referrer-policy=[strict-origin-when-cross-origin], server=[Apache/2.4.54 (Ubuntu)], status=[200 OK], strict-transport-security=[max-age=31536000; includeSubDomains; preload], vary=[Origin,Accept-Encoding], x-content-type-options=[nosniff], x-download-options=[noopen], x-frame-options=[sameorigin], x-permitted-cross-domain-policies=[none], x-powered-by=[Phusion Passenger(R) 6.0.17], x-request-id=[ZErYXRo4dMYNK0nyfjfVnQAAgS0], x-runtime=[0.231637], x-xss-protection=[1; mode=block]}
SAX parsing completed in 2 ms
GET https://api.openstreetmap.org/api/0.6/changesets?changesets=89185038,30150547,34280532,89203470,88808235,36228171,88040284,69323332,51901809,17205089,87655249,50400618,36305989,85341050,35490925,19769180,99634052,15550103,88240952,89199405,88304445,15609455,89197520,61994293,85687199,88677325,88034234,42621964,49189992,70480564,96437048,24850419,88097736,18220954,89336799,88656839,85894031,52096393,32147324,58254830,84161426,88507359,52368770,42288155,74199794,46840917,16600669,107516104,21145571,53491089,48063615,27111230,88361952,111122491,44620800,36318336,36062349,89240573,86072204,17921983,18220985,15337050,21135345,87657354,49699918,17205183,43056169,67560124,58049992,82371112,15240747,50400706,76751440,88499073,89303960,28061536,64249112,89471897,67109587,124101046,21522332,46670878,54277611,73681587,88812444,21288861,19087352,27766652,58238361,111339587,49007624,52960715,85806078,21766056,30394153,36334785,80808506,100074268,49189891,26820471 -> HTTP_2 200 (435 ms; 10.3 kB)
RESPONSE HEADERS: {:status=[200], cache-control=[max-age=0, private, must-revalidate], content-encoding=[gzip], content-length=[10590], content-security-policy=[default-src 'self'; child-src 'self'; connect-src 'self' matomo.openstreetmap.org; font-src 'none'; form-action 'self'; frame-ancestors 'self'; frame-src 'self'; img-src 'self' data: www.gravatar.com *.wp.com tile.openstreetmap.org *.tile.openstreetmap.org *.tile.thunderforest.com tileserver.memomaps.de *.openstreetmap.fr matomo.openstreetmap.org https://openstreetmap-user-avatars.s3.dualstack.eu-west-1.amazonaws.com https://openstreetmap-gps-images.s3.dualstack.eu-west-1.amazonaws.com; manifest-src 'self'; media-src 'none'; object-src 'self'; script-src 'self' matomo.openstreetmap.org; style-src 'self'; worker-src 'none'], content-type=[application/xml; charset=utf-8], date=[Thu, 27 Apr 2023 20:17:34 GMT], etag=[W/"f6b29c127c961af5a60ecd614f60c031-gzip"], referrer-policy=[strict-origin-when-cross-origin], server=[Apache/2.4.54 (Ubuntu)], status=[200 OK], strict-transport-security=[max-age=31536000; includeSubDomains; preload], vary=[Origin,Accept-Encoding], x-content-type-options=[nosniff], x-download-options=[noopen], x-frame-options=[sameorigin], x-permitted-cross-domain-policies=[none], x-powered-by=[Phusion Passenger(R) 6.0.17], x-request-id=[ZErYXho4dMYNK0nyfjfVngAAkC0], x-runtime=[0.255654], x-xss-protection=[1; mode=block]}
SAX parsing completed in 2 ms
GET https://api.openstreetmap.org/api/0.6/changesets?changesets=50974203,29781811,17205232,79481418,19466192,63845684,15824396,18157547,88157119,89186896,88044092,100962567,88365632,60046482,112896444,38405453,26392218,124100717,43355550,21210701,68594438,100649709,20043298,65703161,85383685,74277754,86526518,65737972,97546959,31733447,83082192,59716779,53130255,22134393,15724486,99760860,87656982,86155794,89930348,100145865,89080445,17937957,64220389,88463987,15241168,86059544,70286090,68969255,88175228,50562109,20739708,30434960,94540514,15790001,87657080,73448226,17888851,70923129,30893714,75799490,48426405,87614079,75523016,21693954,27743981,74777406,34626908,35515725,87202400,56396815,86090359,37997851,42552784,101798219,17673814,17475156,33539745,16424869,101855602,88402471,28630738,17649252,78853082,15239043,99762841,16230299,15241094,20020831,93428479,18327139,79881143,44193242,50400377,87687746,21253695,88523462,37354938,88453830,70063017,34170252 -> HTTP_2 200 (425 ms; 11.5 kB)
RESPONSE HEADERS: {:status=[200], cache-control=[max-age=0, private, must-revalidate], content-encoding=[gzip], content-length=[11768], content-security-policy=[default-src 'self'; child-src 'self'; connect-src 'self' matomo.openstreetmap.org; font-src 'none'; form-action 'self'; frame-ancestors 'self'; frame-src 'self'; img-src 'self' data: www.gravatar.com *.wp.com tile.openstreetmap.org *.tile.openstreetmap.org *.tile.thunderforest.com tileserver.memomaps.de *.openstreetmap.fr matomo.openstreetmap.org https://openstreetmap-user-avatars.s3.dualstack.eu-west-1.amazonaws.com https://openstreetmap-gps-images.s3.dualstack.eu-west-1.amazonaws.com; manifest-src 'self'; media-src 'none'; object-src 'self'; script-src 'self' matomo.openstreetmap.org; style-src 'self'; worker-src 'none'], content-type=[application/xml; charset=utf-8], date=[Thu, 27 Apr 2023 20:17:34 GMT], etag=[W/"94a108223cca4807a23fd51c0b7269fd-gzip"], referrer-policy=[strict-origin-when-cross-origin], server=[Apache/2.4.54 (Ubuntu)], status=[200 OK], strict-transport-security=[max-age=31536000; includeSubDomains; preload], vary=[Origin,Accept-Encoding], x-content-type-options=[nosniff], x-download-options=[noopen], x-frame-options=[sameorigin], x-permitted-cross-domain-policies=[none], x-powered-by=[Phusion Passenger(R) 6.0.17], x-request-id=[ZErYXho4dMYNK0nyfjfVoAAAnC0], x-runtime=[0.249489], x-xss-protection=[1; mode=block]}
SAX parsing completed in 3 ms
GET https://api.openstreetmap.org/api/0.6/changesets?changesets=88492739,19087013,28630578,29273656,88961733,19451554,17292937,49263969,42708251,85430918,15798115,21104336,67627925,76559108,52368522,15241072,16989849,61791284,74142709,79416101,75246556,15241034,53511315,63120485,89930486,53529756,93825594,50398383,34594237,95640071,64965738,88765180,88941307,96598649,15241041,33402439,19246750,72029175,17344182,89215727,67550150,39648664,88679053,19418862,25566784,76596055,22564498,95853181,33727951,48997668,75641683,107784638,31294985,55207070,15241021,70824942,16617258,36300286,88136346,15241011,74122166,87722724,70599635,17379054,88244907,21452460,49038592,18220795,86002383,15240986,33154572,17694453,88545983,30131761,41805187,54277314,18607844,47842597,78789453 -> HTTP_2 200 (379 ms; 8.56 kB)
RESPONSE HEADERS: {:status=[200], cache-control=[max-age=0, private, must-revalidate], content-encoding=[gzip], content-length=[8762], content-security-policy=[default-src 'self'; child-src 'self'; connect-src 'self' matomo.openstreetmap.org; font-src 'none'; form-action 'self'; frame-ancestors 'self'; frame-src 'self'; img-src 'self' data: www.gravatar.com *.wp.com tile.openstreetmap.org *.tile.openstreetmap.org *.tile.thunderforest.com tileserver.memomaps.de *.openstreetmap.fr matomo.openstreetmap.org https://openstreetmap-user-avatars.s3.dualstack.eu-west-1.amazonaws.com https://openstreetmap-gps-images.s3.dualstack.eu-west-1.amazonaws.com; manifest-src 'self'; media-src 'none'; object-src 'self'; script-src 'self' matomo.openstreetmap.org; style-src 'self'; worker-src 'none'], content-type=[application/xml; charset=utf-8], date=[Thu, 27 Apr 2023 20:17:34 GMT], etag=[W/"2174c29e6e810d78af0103a8bcbae134-gzip"], referrer-policy=[strict-origin-when-cross-origin], server=[Apache/2.4.54 (Ubuntu)], status=[200 OK], strict-transport-security=[max-age=31536000; includeSubDomains; preload], vary=[Origin,Accept-Encoding], x-content-type-options=[nosniff], x-download-options=[noopen], x-frame-options=[sameorigin], x-permitted-cross-domain-policies=[none], x-powered-by=[Phusion Passenger(R) 6.0.17], x-request-id=[ZErYXho4dMYNK0nyfjfVpAAAsC0], x-runtime=[0.199664], x-xss-protection=[1; mode=block]}
SAX parsing completed in 2 ms
I think the problem lies with the first call, https://api.openstreetmap.org/api/0.6/relation/2769137/history. Unfortunately, it doesn't look like we have the size to print to console there, so I had to wait to download it again.
The history for relation 2769137 is 190M uncompressed. When compressed with gzip, it is 22M (22761298 bytes). Which is over the CGIMap default limits (204800, no clue what unit that is in though).
I believe the changesets API calls proceed to work since they are hitting the ruby endpoints instead of the CGIMap endpoints. But the CGIMap endpoints controls the stuff most editors care about.
In order to fix this bug, changes need to be made on the OSM website api. Specifically, history calls must be paginated.
comment:4 by , 16 months ago
For the time being maybe use the multi fetch instead and provide exact version numbers you want to download: https://wiki.openstreetmap.org/wiki/API_v0.6#Multi_fetch:_GET_/api/0.6/%5Bnodes%7Cways%7Crelations%5D?%23parameters
comment:5 by , 16 months ago
Pagination would have to be added to Rails first by the way, before any changes could be done to CGImap.
comment:6 by , 7 months ago
If I got that right we could replace our api call
https://api.openstreetmap.org/api/0.6/relation/2769137/history by a bunch of calls like
https://api.openstreetmap.org/api/0.6/relations?relations=2769137v1,2769137v2,2769137v3...
https://api.openstreetmap.org/api/0.6/relations?relations=2769137v128,2769137v129,2769137v130...
...
https://api.openstreetmap.org/api/0.6/relations?relations=2769137v1699,2769137v1700
where each call would list maybe a 127 ids or so.
What would be the difference on the server side? Would that really improve something?
Note:
See TracTickets
for help on using tickets.
+1 for selective download of a few versions only, if that is possible. Maybe tricky since #20879
See also #20405.