Modify

Opened 5 months ago

Closed 5 months ago

Last modified 4 months ago

#14422 closed enhancement (fixed)

[PATCH] Dynamic NTV2 grids

Reported by: Don-vip Owned by: Don-vip
Priority: normal Milestone: 17.03
Component: Core Version:
Keywords: ntv2 grid jar linux debian ubuntu projection Cc: sebastic

Description

From #5399 comments where we plan to add a 3MB NTV2 grid for Swiss:

It would be worth to implement a NTV2 file management as official JOSM extension. If the default JOSM jar does not contain the required NTV2 files, the first time a user chooses a projection that requires ones, the grid is downloaded from JOSM server and stored in JOSM cache directory.

We could also still build a bigger jar file containing all grids for users who need JOSM to remain fully operational without Internet access.

The idea is to remove the NTV2 grids from default jar for majority of users, but still provide a "full" jar in parallel for advanced/concerned users and/or Linux distributions.

Attachments (2)

14422.patch (15.8 KB) - added by Don-vip 5 months ago.
josm_proj-data_dependency.patch (1.2 KB) - added by sebastic 4 months ago.
Add proj-data to dependencies for grid shift files.

Download all attachments as: .zip

Change History (27)

comment:1 in reply to:  description ; Changed 5 months ago by bastiK

Replying to Don-vip:

We could also still build a bigger jar file containing all grids for users who need JOSM to remain fully operational without Internet access.

The idea is to remove the NTV2 grids from default jar for majority of users, but still provide a "full" jar in parallel for advanced/concerned users and/or Linux distributions.

Traditional projections that require grid files are a niche use case. Not being able or willing to use the auto-downloader (for any reason) is a niche of a niche. So it may be overkill to build bigger jar files in parallel for each release.

We can simply offer a zip file download for all the grid files. The user can extract those grid files to a directory and specify the directory in the preferences or advanced options. This would also work for Linux: The Ubuntu package proj-data already contains the ntf_r93.gsb and the chenyx06etrs.gsb file. The distributors can simply add a dependency to that package and point JOSM to the system directory where they are stored (/usr/share/proj).

comment:2 Changed 5 months ago by stoecker

We can use URL rewriting, so that we have a nice URL like josm.../grid/..., but the data itself may be in josm svn data_nodist/grid/...

comment:3 in reply to:  1 Changed 5 months ago by Don-vip

Replying to bastiK:

The Ubuntu package proj-data already contains the ntf_r93.gsb and the chenyx06etrs.gsb file. The distributors can simply add a dependency to that package and point JOSM to the system directory where they are stored (/usr/share/proj).

I didn't know that this is great! The swiss grid inclusion is discussed here.

comment:4 Changed 5 months ago by Don-vip

Cc: sebastic added
Keywords: linux debian ubuntu projection added

Adding sebastic to this ticket. When implemented the josm package will need a dependency on proj-data package.

comment:5 Changed 5 months ago by anonymous

I update the proj package to update the Swiss NTv2 data grind occasionally because it's one of the few non-problematically licensed grids (unlike most national grid shift files which limit modification of the correction values like EPSG making it incompatible with GPL licensed projects like PostGIS & JOSM). Several versions are now included since PROJ 4.9.x (see datumgrids-ch.list, prior versions only had the old grid which may not be the one expected by JOSM (e.g. PROJ.4 4.8.0 in Debian jessie & Ubuntu trusty).

comment:6 in reply to:  2 Changed 5 months ago by Don-vip

Replying to stoecker:

We can use URL rewriting, so that we have a nice URL like josm.../grid/..., but the data itself may be in josm svn data_nodist/grid/...

Good idea, can you please enable the rule?

comment:7 Changed 5 months ago by Don-vip

Owner: changed from team to Don-vip
Status: newassigned
Summary: Dynamic NTV2 grids[PATCH] Dynamic NTV2 grids

Changed 5 months ago by Don-vip

Attachment: 14422.patch added

comment:8 Changed 5 months ago by Don-vip

This should do the trick :)

comment:9 Changed 5 months ago by anonymous

You may want to support the PROJ_LIB environment variable in addition to the hardcoded paths, it is used in PROJ.4 to set a custom path to the data.

comment:10 Changed 5 months ago by Don-vip

Already done :)

comment:11 Changed 5 months ago by stoecker

Test: https://josm.openstreetmap.de/proj/ntf_r93_b.gsb

Links to data/projection files ATM. Tell me when it should be switch to data_nodist.

comment:12 Changed 5 months ago by stoecker

Note I used proj instead of grid, as the directory is also named projection and not grid :-)

comment:13 in reply to:  11 Changed 5 months ago by Don-vip

Replying to stoecker:

Tell me when it should be switch to data_nodist.

I'll switch it, thanks :)

comment:14 Changed 5 months ago by Don-vip

Resolution: fixed
Status: assignedclosed

In 11642/josm:

fix #14422 - Dynamic NTV2 grids

comment:15 Changed 5 months ago by bastiK

In [11644]:
switch to little-endian version of the french ntv2 grid file

both files have equivalent data; reason to switch is that
ntf_r93.gsb is in the debian package proj-data, but not ntf_r93_b.gsb;
this avoids unnecessary downloads and makes it easier for distributors

comment:16 Changed 5 months ago by Don-vip

Why r11643 was necessary? It seems to be a Netbeans bug.

comment:17 Changed 5 months ago by bastiK

Yes, seems to be: reported here.

comment:18 Changed 5 months ago by Don-vip

In 11648/josm:

see #14422 - update unit test

comment:19 in reply to:  4 Changed 4 months ago by sebastic

Replying to Don-vip:

Adding sebastic to this ticket. When implemented the josm package will need a dependency on proj-data package.

The PPA packaging (tested, latest) wasn't updated to include the dependency, is this intentional or an oversight?

comment:20 Changed 4 months ago by Don-vip

An oversight which needs to be fixed. Thanks for making us notice it :)

Changed 4 months ago by sebastic

Add proj-data to dependencies for grid shift files.

comment:21 Changed 4 months ago by Klumbumbus

Will this require an hotfix release?

comment:22 Changed 4 months ago by bastiK

No, it is a very minor thing: The users of our .deb repository will (for this release) not have the added bonus that grid files are already present on the file system, but need to use the new on-demand auto-downloader like everyone else.

comment:23 Changed 4 months ago by sebastic

Users that require the grid shift files can install proj-data themselves (or use the auto-downloader). But most systems will have it installed already as part of the dependency chain of other geospatial packages (it is by far the most popular geospatial package because pretty much everything depends on proj & gdal).

comment:24 Changed 4 months ago by Klumbumbus

OK thanks. I'm going to announce the new stable on facebook/google+ now.

comment:25 Changed 4 months ago by bastiK

In 11833/josm:

see #14422 - Debian package: add dependency for proj-data (patch by sebastic)

Modify Ticket

Change Properties
Set your email in Preferences
Action
as closed The owner will remain Don-vip.
as The resolution will be set. Next status will be 'closed'.
The resolution will be deleted. Next status will be 'reopened'.

Add Comment


E-mail address and name can be saved in the Preferences.

 
Note: See TracTickets for help on using tickets.