relative path in .joz session file

[StephaneP]

Session files can contain absolute path to images, gps trace.

But relative path would be better to share the session and it's files.

[Don-vip]

Simon, can you please apply your patches?

[simon04]

In 14630/josm:

fix #16497 - Support relative paths in .joz session file

[StephaneP]

Hi!

I just tested with Josm 14715 :

• gpx track are stored with a relative path
• images are not stored with a relative path

[simon04]

In 14718/josm:

see #16497 - Relative paths for geoimages in session file

[simon04]

There are a bunch of problems with relative paths:

1. #17228
2. Reading relative paths has always been broken when the jos file lies on a UNC path (the second preceding \ is missing in \my-server\my-share\)

java.nio.file.NoSuchFileException: \my-server\my-share\Downloads\gpx\first.gpx
        at sun.nio.fs.WindowsException.translateToIOException(Unknown Source)
        at sun.nio.fs.WindowsException.rethrowAsIOException(Unknown Source)
        at sun.nio.fs.WindowsException.rethrowAsIOException(Unknown Source)
        at sun.nio.fs.WindowsFileSystemProvider.newByteChannel(Unknown Source)
        at java.nio.file.Files.newByteChannel(Unknown Source)
        at java.nio.file.Files.newByteChannel(Unknown Source)
        at java.nio.file.spi.FileSystemProvider.newInputStream(Unknown Source)
        at java.nio.file.Files.newInputStream(Unknown Source)
        at org.openstreetmap.josm.io.Compression.getUncompressedFileInputStream(Compression.java:178)
        at org.openstreetmap.josm.io.session.SessionReader$ImportSupport.getInputStream(SessionReader.java:305)
        at org.openstreetmap.josm.io.session.GpxTracksSessionImporter.load(GpxTracksSessionImporter.java:45)
        at org.openstreetmap.josm.io.session.SessionReader.parseJos(SessionReader.java:563)
        at org.openstreetmap.josm.io.session.SessionReader.loadSession(SessionReader.java:749)
        at org.openstreetmap.josm.io.session.SessionReader.loadSession(SessionReader.java:709)
        at org.openstreetmap.josm.actions.SessionLoadAction$Loader.realRun(SessionLoadAction.java:180)
        at org.openstreetmap.josm.gui.PleaseWaitRunnable.doRealRun(PleaseWaitRunnable.java:95)
        at org.openstreetmap.josm.gui.PleaseWaitRunnable.run(PleaseWaitRunnable.java:143)
        at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
        at java.util.concurrent.FutureTask.run(Unknown Source)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
        at java.lang.Thread.run(Unknown Source)

3. Reading non-embedded relative files for joz sessions is broken

    <layers active="1">
        <layer index="1" name="osmdata.osm" type="osm-data" version="0.1" visible="true">
            <file>layers/01/data.osm</file> <!-- embedded in joz, okay -->
        </layer>
        <layer index="2" name="first.gpx" type="tracks" version="0.1" visible="true">
            <file>gpx/first.gpx</file> <!-- relative to joz, NOK -->
        </layer>
    </layers>

java.nio.file.NoSuchFileException: gpx/first.gpx
        at com.sun.nio.zipfs.ZipFileSystem.newInputStream(ZipFileSystem.java:544)
        at com.sun.nio.zipfs.ZipPath.newInputStream(ZipPath.java:645)
        at com.sun.nio.zipfs.ZipFileSystemProvider.newInputStream(ZipFileSystemProvider.java:278)
        at java.nio.file.Files.newInputStream(Unknown Source)
        at org.openstreetmap.josm.io.session.SessionReader$ImportSupport.getInputStream(SessionReader.java:309)
        at org.openstreetmap.josm.io.session.GpxTracksSessionImporter.load(GpxTracksSessionImporter.java:45)
        at org.openstreetmap.josm.io.session.SessionReader.parseJos(SessionReader.java:559)
        at org.openstreetmap.josm.io.session.SessionReader.loadSession(SessionReader.java:733)
        at org.openstreetmap.josm.io.session.SessionReader.loadSession(SessionReader.java:702)
        at org.openstreetmap.josm.actions.SessionLoadAction$Loader.realRun(SessionLoadAction.java:180)
        at org.openstreetmap.josm.gui.PleaseWaitRunnable.doRealRun(PleaseWaitRunnable.java:95)
        at org.openstreetmap.josm.gui.PleaseWaitRunnable.run(PleaseWaitRunnable.java:143)
        at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
        at java.util.concurrent.FutureTask.run(Unknown Source)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
        at java.lang.Thread.run(Unknown Source)

4. Hopefully no more!

Let's see whether those issues can be fixed …

[simon04]

In 14723/josm:

see #16497 fix #17228 - Avoid backslashes in session files

[simon04]

In 14724/josm:

see #16497 - Use NIO.2 in SessionReader

[simon04]

In 14725/josm:

see #16497 - SessionWriter: use ../ to escape ZIP file

[simon04]

Concerning comment:8, r14723 should address issue 1, r14724 should address issue 2, r14725 should address issue 3.

[aseerel4c26]

Apparently relative paths for external gpx files still do not work correctly:

What steps will reproduce the problem?

0. have linux 4.20.3
1. have a file /tmp/foobar.gpx
2. open this gpx file in josm
3. save session into /home/yourusername/JOSM/testsession.jos
4. close josm
5. open josm and load /home/yourusername/JOSM/testsession.jos

What is the expected result?

loads

What happens instead?

error message, cannot find the file /home/yourusername/JOSM/tmp/foobar.gpx

Please provide any additional information below. Attach a screenshot if possible.

2019-01-26 23:51:21.553 SCHWERWIEGEND: java.nio.file.NoSuchFileException: /home/yourusername/JOSM/tmp/foobar.gpx
java.nio.file.NoSuchFileException: /home/yourusername/JOSM/tmp/foobar.gpx
        at java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:92)
        at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111)
        at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:116)
        at java.base/sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:215)
        at java.base/java.nio.file.Files.newByteChannel(Files.java:370)
        at java.base/java.nio.file.Files.newByteChannel(Files.java:421)
        at java.base/java.nio.file.spi.FileSystemProvider.newInputStream(FileSystemProvider.java:420)
        at java.base/java.nio.file.Files.newInputStream(Files.java:155)
        at org.openstreetmap.josm.io.Compression.getUncompressedFileInputStream(Compression.java:178)
        at org.openstreetmap.josm.io.session.SessionReader$ImportSupport.getInputStream(SessionReader.java:304)
        at org.openstreetmap.josm.io.session.GpxTracksSessionImporter.load(GpxTracksSessionImporter.java:45)
        at org.openstreetmap.josm.io.session.SessionReader.parseJos(SessionReader.java:559)
        at org.openstreetmap.josm.io.session.SessionReader.loadSession(SessionReader.java:733)
        at org.openstreetmap.josm.io.session.SessionReader.loadSession(SessionReader.java:702)
        at org.openstreetmap.josm.actions.SessionLoadAction$Loader.realRun(SessionLoadAction.java:180)
        at org.openstreetmap.josm.gui.PleaseWaitRunnable.doRealRun(PleaseWaitRunnable.java:95)
        at org.openstreetmap.josm.gui.PleaseWaitRunnable.run(PleaseWaitRunnable.java:143)
        at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base/java.lang.Thread.run(Thread.java:834)

Relative:URL: ^/trunk
Repository:UUID: 0c6e7542-c601-0410-84e7-c038aed88b3b
Last:Changed Date: 2019-01-26 00:17:26 +0100 (Sat, 26 Jan 2019)
Revision:14733
Build-Date:2019-01-26 02:33:28
URL:https://josm.openstreetmap.de/svn/trunk

Identification: JOSM/1.5 (14733 de) Linux
Memory Usage: 236 MB / 2048 MB (92 MB allocated, but free)
Java version: 11.0.2+7, Oracle Corporation, OpenJDK 64-Bit Server VM
VM arguments: [-Dawt.useSystemAAFontSettings=on, -Dswing.aatext=true]

Last errors/warnings:
- W: No configuration settings found.  Using hardcoded default values for all pools.
- E: java.nio.file.NoSuchFileException: ${HOME}/JOSM/tmp/foobar.gpx

[StephaneP]

Perhaps relative path should be an option in the save session dialog box?

And maybe a NoSuchFileException should let the user select the correct folder?

[aseerel4c26]

In this case the saved relative path is just wrong (some ../ seem to be missing), I guess. Relative paths are a good idea - but in my case the best option simply would be an absolute path (otherwise it would need to be ../../tmp/foobar.gpx ... not nice).

I guess that relative paths should only be defaulted to if the referenced files are in the same or a sub-directory of the location where the session file is saved. Yes, a radio button for absolute/relative paths would be good, then the user could override the default.

I was curious how do other programs handle this? E.g. ​Inkscape (for svg images): I have tested it with a bitmap image linked into a svg image. If a svg image is saved to somewhere when I drag the bitmap into the svg, then Inkscape saves the absolute AND a relative path. If I intentionally break one of the links by moving one or both of the files to another location inkscape uses the working link of both and fixes the other one on save. By the way, when inkscape cannot find the linked Bitmap it silently fails (just a placeholder is shown) - not that good (in case the linked image does not have a big display size). In the svg files the relative path is the svg object "image" attribute "xlink:href" and the absolute path is the inkscape custom (non-standard) attribute "sodipodi:absref". Is this a good strategy? I am not sure - it at least adds quite some complexity.

A dialogue box for opening the right file would be nice, but not required.

[simon04]

In 14741/josm:

see #16497 - revert r14725/r14724/r14723/r14718/r14630

Support for relative paths in .joz session file needs more testing.

[simon04]

Surprise: mixing various concepts, such as NIO.2 Path, URI, NIO File, and ZipFile gives rise to various bugs on various platforms. Let's start from the beginning at a later point in tine.

[RicardoRodriguezFernandez]

Please, is there any update on this issue? Is anybody working on it? Sorry, I'm not able to understand the current state of the enhancement request.

For us, not to be able to share sessions between platforms (Windows, macOS, and Windows) is a big problem.

Any insight will be welcome! Thanks!

[Bjoeni]

You should be able to share *.joz sessions (archives) containing all relevant files between platforms. And since #21923 the workflow should be improved, meaning you can easily save/load/reopen a single session file. If you encounter issues with that, I'd suggest opening a new ticket.

This ticket is about files not contained in the session archive, meaning mainly *.jos files (and also e.g. geoimage layers as the GPS locations are saved in the archive, not however the corresponding image files).
I agree that it makes sense to support relative paths for those files at some point in the future too, but for now the easiest solution should be to share *.joz files between platforms. That will also work when using the "File" > "Save Session" command.

[anonymous]

Thanks for your thoughtful answer. It was clear to me that I must make a difference between files stored in a session file and those not included. But I realize now that I don't understand how JOSM kept information. You said, "files not contained in the session archive," and gave some examples. But, please, although it is a bit out of the topic of the ticket, is there anywhere a comprehensive list and explanation of what kind of files will be or can be stored in the *.joz file and which can not? With my shallow level of skills, my main concern is how to move sessions, including *.osm layers, between platforms and share them with other users.
Thanks!

[skyper]

Meanwhile, there should be a checkbox to save each local files within a session, see Help/Action/SessionSaveAs. If a certain file type is missing, please, create a new ticket and attach a (small) example file, thanks.

[RicardoRodriguezFernandez]

Replying to skyper:

Meanwhile, there should be a checkbox to save each local files within a session, see Help/Action/SessionSaveAs. If a certain file type is missing, please, create a new ticket and attach a (small) example file, thanks.

Thanks! I think what I'm seeing here is a mixture between my lack of skills and some misfunctions in JOSM running on macOS. I'll open a new ticket accordingly.

Please, is there a reference for good practices in this community? For instance, if I detect something I consider I'm able to improve in a help page, should I modify it directly? Or should I propose a change somewhere? I would be grateful if you could point me to the best place to ask such questions. I'm sure that inserting them in this thread is not the best idea! Thanks!

[skyper]

It is a wiki, e.g. if you do not plan to completely rework a page, please, go ahead. Otherwise it is better to discuss the changes in advance in a new ticket (component=wiki content).