Modify

Opened 8 years ago

Closed 8 years ago

Last modified 7 years ago

#8904 closed defect (fixed)

map download gets outdated cache content

Reported by: maarten Owned by: team
Priority: normal Milestone:
Component: Core Version: latest
Keywords: close changeser Cc:

Description

As discussed on the talk mailinglist from http://lists.openstreetmap.org/pipermail/talk/2013-July/067693.html

It appears that JOSM 6060 does not always close a changeset at the end of the session leaving it to think data is unchanged. Data in OSM is in fact changed but when you download it in JOSM again, you get the old data. When looking at the history of an object (ctrl-H) you will see the proper history with changes.

I have no idea how to reproduce it. It only happens occasionally.

Attachments (0)

Change History (23)

comment:1 Changed 8 years ago by bastiK

Resolution: othersoftware
Status: newclosed

I'm not completely certain what happened here, but there doesn't seem to be a problem with JOSM but with the server you are downloading from. If it's a mirror, this would quite normal, if it's the main API then it's a little odd.

comment:2 Changed 8 years ago by skyper

We have #5783 still open, where JOSM does not get important information from the server.

@mdeen:
Do you remember if the changeset was closed and JOSM did not get this information or was the changeset still open ?

comment:3 Changed 8 years ago by maarten

@bastiK:
I am using the default OSM server URL (checkbox next to that field) and that is set tot the main api.

@skyper:
The changeset in OSM says it was closed: <http://www.openstreetmap.org/browse/changeset/17057508>
But in JOSM when I did "Close open changesets" (some 2 hours later than when OSM says it was closed) it informed me that there was one changeset open. So JOSM thought it was open. I did not get an exception or closed JOSM abnormally.

comment:4 Changed 8 years ago by bastiK

Even if the changeset is still open, you can always download the new version of the objects immediately after upload. If this does not work and you know the upload was successful, then this must be a server problem.

comment:5 Changed 8 years ago by skyper

So we need to fill a ticket at https://trac.openstreetmap.org ?

comment:6 Changed 8 years ago by maarten

@Batik:
When I downloaded from an area, I got the old version. When I downloaded an individual item after that using download object I got a conflict.
So there was a difference in handling downloads by area and downloads by id.

comment:7 in reply to:  6 Changed 8 years ago by bastiK

Replying to mdeen@…:

@Batik:
When I downloaded from an area, I got the old version. When I downloaded an individual item after that using download object I got a conflict.
So there was a difference in handling downloads by area and downloads by id.

The osm server alway delivers the most recent version of any object, no matter what download method you choose. If this was broken at a certain time (very strange) then there is still nothing we can do about it in JOSM.

comment:8 Changed 8 years ago by maarten

I don't know where it was broken, but JOSM still thought the changeset was open. That seems to be a JOSM problem.

comment:9 Changed 8 years ago by bastiK

If JOSM get the open/closed state of a changeset wrong, this is a minor issue and cannot have the consequences you describe.

But you are right, this is probably a JOSM bug and if would be helpful if you can reproduce the problem (or try to remember the steps next time it happens).

comment:10 Changed 8 years ago by skyper

Keywords: close changeser added
Resolution: othersoftware
Status: closedreopened

comment:11 Changed 8 years ago by anonymous

I have no idea where the bug is, but I have the same issue again, only JOSM now does not see a closed changeset.
I have uploaded changeset http://www.openstreetmap.org/browse/changeset/17269345 and the changes are also visible on the map. But when I download the same area in JOSM, I see the old version of items. When I change something and upload it, JOSM says there is a validation issue (which is good).

I have closed JOSM between these editing session.

I don't know if this might be related, but I also have had occasions that I uploaded all my changes and when I want to exit JOSM, it ways I need to upload my changes. When I do so, the upload box shows no changes and JOSM closes.

Could this maybe be a persistent cache somewhere between me and the OSM server? In JOSM I have "no proxy" checked.

comment:12 Changed 8 years ago by maarten

It did download the correct versions after I downloaded and uploaded items in a different area. Which could point to a possible bug: what if this only happens when you do not change the download area in the download box? Now I downloaded a different area, next time I encounter this phenomenon I'll just change the download area.

comment:13 Changed 8 years ago by bastiK

For every download, JOSM prints something like this

GET http://api.openstreetmap.org/api/0.6/map?bbox=6.4623213,50.8064229,6.4663124,50.8094331

on the command line.

Next time the bug occurs, please download this URL manually with a browser (or wget/curl), save as map.osm and open in JOSM. If you still get the old version, then JOSM can be excluded as the source of error.

comment:14 Changed 8 years ago by maarten

It does not do that in the webstart version on Windows.

comment:15 Changed 8 years ago by Don-vip

comment:16 Changed 8 years ago by maarten

Perfect. And it did happen again. I'm starting to think this is reproducable. In the java console I see this:

GET http://api.openstreetmap.org/api/0.6/map?bbox=5.3000021,51.4044531,5.3081346,51.4077593
CacheEntryhttp://api.openstreetmap.org/api/0.6/map?bbox=5.3000021,51.4044531,5.3081346,51.4077593: updateAvailable=false,lastModified=Thu Jan 01 01:00:00 CET 1970,length=-1

Note the "CacheEntry". Does JOSM cache things? When I get the area with wget I get the correct data. The "CacheEntry" line is not there when I change the download area and then I get the correct data also.

comment:17 Changed 8 years ago by bastiK

Thanks, this brings us much closer to the real problem. The next step for diagnosis would be to run wireshark when JOSM fetches the outdated data. Basically what I'd like to see is, if there is any connection to the OSM server at all or if it uses the cache file from the hard-drive without further validation. Could you add a status report, please?

See http://stackoverflow.com/questions/17181129/java-1-7-applet-cacheentry-preventing-dynamic-updates

The webstart version of Java seems to cache static resources like a browser. In principle, it should recognize dynamic content by looking at the http header, but this seems to go wrong here.

The osm server sends the following header:

wget -S 'http://api.openstreetmap.org/api/0.6/map?bbox=5.3000021,51.4044531,5.3081346,51.4077593'
HTTP request sent, awaiting response... 
  HTTP/1.1 200 OK
  Date: Fri, 09 Aug 2013 06:38:38 GMT
  Server: Apache/2.2.22 (Ubuntu)
  Content-Disposition: attachment; filename="map.osm"
  Content-Encoding: identity
  Cache-Control: private, max-age=0, must-revalidate
  Vary: Accept-Encoding
  Content-Type: text/xml; charset=utf-8
  Keep-Alive: timeout=5, max=100
  Connection: Keep-Alive
  Transfer-Encoding: chunked
Length: unspecified [text/xml]

Before the ticket is closed as "othersoftware", we should find out, how many users are affected by this. If it's a common problem, we can think about a workaround (e.g. dummy parameter with random value).

comment:18 Changed 8 years ago by bastiK

Summary: JOSM 6060 does not always close changesetsmap download gets outdated cache content

comment:19 Changed 8 years ago by Don-vip

We could try this:
http://docs.oracle.com/javase/7/docs/technotes/guides/javaws/enhancements6.html

The "cache-control: no-cache" HTTP header is now supported. "no-cache" means cached contents cannot be used without re-validation. When the no-cache directive is used , an update check is always made to make sure the cached content is the same as the server copy pointed to, by the resource URL. If so, cached copy can be used. If not, server copy will be downloaded. Expiration field is ignored, which means that even if the cached resource is not expired yet, update check will still be performed.

comment:20 Changed 8 years ago by Don-vip

Yep, this is a change in JDK7u2 and later:
http://docs.oracle.com/javase/7/docs/technotes/guides/jweb/enhancements-7.html#javase7u2

Caching enabled by default: Caching of network content for application code running in Web Start mode is now enabled by default. This allows application improved performance and consistency with applet execution mode. To ensure the latest copy of content is used, the application can use URLConnection.setUseCaches(false) or request header Cache-Control values no-cache/no-store.

comment:21 in reply to:  20 Changed 8 years ago by bastiK

Replying to Don-vip:

Yep, this is a change in JDK7u2 and later:
http://docs.oracle.com/javase/7/docs/technotes/guides/jweb/enhancements-7.html#javase7u2

Caching enabled by default: Caching of network content for application code running in Web Start mode is now enabled by default. This allows application improved performance and consistency with applet execution mode. To ensure the latest copy of content is used, the application can use URLConnection.setUseCaches(false) or request header Cache-Control values no-cache/no-store.

Alright, URLConnection.setUseCaches(false) sounds promising...

(Could not reproduce on my system:

icedtea-web 1.3.2 (1.3.2-1ubuntu0.12.10.2)
java version "1.7.0_25"
OpenJDK Runtime Environment (IcedTea 2.3.10) (7u25-2.3.10-1ubuntu0.12.10.2)
OpenJDK 64-Bit Server VM (build 23.7-b01, mixed mode)

)

comment:22 Changed 8 years ago by Don-vip

Resolution: fixed
Status: reopenedclosed

In 6123/josm:

fix #8904 - Disable Java HTTP caching, enabled by default with Java Web Start in 7u2 and later

comment:23 Changed 7 years ago by ScM

Same issue with changeset 19815334 in josm version 6502 with java 1.7.0_45.
I uploaded the changeset and the changes are visible on
http://www.openstreetmap.org/changeset/19815334#map=15/10.7609/123.0549&layers=H
and already (partially) rendered on
http://www.openstreetmap.org/#map=16/10.7608/123.0500
and are shown in iD, but if I try to redownload the area in JOSM the outdated pre-changeset version appears. I tried to restart josm, to no avail. I also tried to change the download area, without solving the issue.

Modify Ticket

Change Properties
Set your email in Preferences
Action
as closed The owner will remain team.
as The resolution will be set.
The resolution will be deleted.

Add Comment


E-mail address and name can be saved in the Preferences.

 
Note: See TracTickets for help on using tickets.