Modify

Opened 7 years ago

Closed 7 years ago

Last modified 6 years ago

#14422 closed enhancement (fixed)

[PATCH] Dynamic NTV2 grids

Reported by: Don-vip Owned by: Don-vip
Priority: normal Milestone: 17.03
Component: Core Version:
Keywords: ntv2 grid jar linux debian ubuntu projection Cc: sebastic

Description

From #5399 comments where we plan to add a 3MB NTV2 grid for Swiss:

It would be worth to implement a NTV2 file management as official JOSM extension. If the default JOSM jar does not contain the required NTV2 files, the first time a user chooses a projection that requires ones, the grid is downloaded from JOSM server and stored in JOSM cache directory.

We could also still build a bigger jar file containing all grids for users who need JOSM to remain fully operational without Internet access.

The idea is to remove the NTV2 grids from default jar for majority of users, but still provide a "full" jar in parallel for advanced/concerned users and/or Linux distributions.

Attachments (2)

14422.patch (15.8 KB ) - added by Don-vip 7 years ago.
josm_proj-data_dependency.patch (1.2 KB ) - added by sebastic 7 years ago.
Add proj-data to dependencies for grid shift files.

Download all attachments as: .zip

Change History (28)

in reply to:  description ; comment:1 by bastiK, 7 years ago

Replying to Don-vip:

We could also still build a bigger jar file containing all grids for users who need JOSM to remain fully operational without Internet access.

The idea is to remove the NTV2 grids from default jar for majority of users, but still provide a "full" jar in parallel for advanced/concerned users and/or Linux distributions.

Traditional projections that require grid files are a niche use case. Not being able or willing to use the auto-downloader (for any reason) is a niche of a niche. So it may be overkill to build bigger jar files in parallel for each release.

We can simply offer a zip file download for all the grid files. The user can extract those grid files to a directory and specify the directory in the preferences or advanced options. This would also work for Linux: The Ubuntu package proj-data already contains the ntf_r93.gsb and the chenyx06etrs.gsb file. The distributors can simply add a dependency to that package and point JOSM to the system directory where they are stored (/usr/share/proj).

comment:2 by stoecker, 7 years ago

We can use URL rewriting, so that we have a nice URL like josm.../grid/..., but the data itself may be in josm svn data_nodist/grid/...

in reply to:  1 comment:3 by Don-vip, 7 years ago

Replying to bastiK:

The Ubuntu package proj-data already contains the ntf_r93.gsb and the chenyx06etrs.gsb file. The distributors can simply add a dependency to that package and point JOSM to the system directory where they are stored (/usr/share/proj).

I didn't know that this is great! The swiss grid inclusion is discussed here.

comment:4 by Don-vip, 7 years ago

Cc: sebastic added
Keywords: linux debian ubuntu projection added

Adding sebastic to this ticket. When implemented the josm package will need a dependency on proj-data package.

comment:5 by anonymous, 7 years ago

I update the proj package to update the Swiss NTv2 data grind occasionally because it's one of the few non-problematically licensed grids (unlike most national grid shift files which limit modification of the correction values like EPSG making it incompatible with GPL licensed projects like PostGIS & JOSM). Several versions are now included since PROJ 4.9.x (see datumgrids-ch.list, prior versions only had the old grid which may not be the one expected by JOSM (e.g. PROJ.4 4.8.0 in Debian jessie & Ubuntu trusty).

in reply to:  2 comment:6 by Don-vip, 7 years ago

Replying to stoecker:

We can use URL rewriting, so that we have a nice URL like josm.../grid/..., but the data itself may be in josm svn data_nodist/grid/...

Good idea, can you please enable the rule?

comment:7 by Don-vip, 7 years ago

Owner: changed from team to Don-vip
Status: newassigned
Summary: Dynamic NTV2 grids[PATCH] Dynamic NTV2 grids

by Don-vip, 7 years ago

Attachment: 14422.patch added

comment:8 by Don-vip, 7 years ago

This should do the trick :)

comment:9 by anonymous, 7 years ago

You may want to support the PROJ_LIB environment variable in addition to the hardcoded paths, it is used in PROJ.4 to set a custom path to the data.

comment:10 by Don-vip, 7 years ago

Already done :)

comment:11 by stoecker, 7 years ago

Test: https://josm.openstreetmap.de/proj/ntf_r93_b.gsb

Links to data/projection files ATM. Tell me when it should be switch to data_nodist.

comment:12 by stoecker, 7 years ago

Note I used proj instead of grid, as the directory is also named projection and not grid :-)

in reply to:  11 comment:13 by Don-vip, 7 years ago

Replying to stoecker:

Tell me when it should be switch to data_nodist.

I'll switch it, thanks :)

comment:14 by Don-vip, 7 years ago

Resolution: fixed
Status: assignedclosed

In 11642/josm:

fix #14422 - Dynamic NTV2 grids

comment:15 by bastiK, 7 years ago

In [11644]:
switch to little-endian version of the french ntv2 grid file

both files have equivalent data; reason to switch is that
ntf_r93.gsb is in the debian package proj-data, but not ntf_r93_b.gsb;
this avoids unnecessary downloads and makes it easier for distributors

comment:16 by Don-vip, 7 years ago

Why r11643 was necessary? It seems to be a Netbeans bug.

comment:17 by bastiK, 7 years ago

Yes, seems to be: reported here.

comment:18 by Don-vip, 7 years ago

In 11648/josm:

see #14422 - update unit test

in reply to:  4 comment:19 by sebastic, 7 years ago

Replying to Don-vip:

Adding sebastic to this ticket. When implemented the josm package will need a dependency on proj-data package.

The PPA packaging (tested, latest) wasn't updated to include the dependency, is this intentional or an oversight?

comment:20 by Don-vip, 7 years ago

An oversight which needs to be fixed. Thanks for making us notice it :)

by sebastic, 7 years ago

Add proj-data to dependencies for grid shift files.

comment:21 by Klumbumbus, 7 years ago

Will this require an hotfix release?

comment:22 by bastiK, 7 years ago

No, it is a very minor thing: The users of our .deb repository will (for this release) not have the added bonus that grid files are already present on the file system, but need to use the new on-demand auto-downloader like everyone else.

comment:23 by sebastic, 7 years ago

Users that require the grid shift files can install proj-data themselves (or use the auto-downloader). But most systems will have it installed already as part of the dependency chain of other geospatial packages (it is by far the most popular geospatial package because pretty much everything depends on proj & gdal).

comment:24 by Klumbumbus, 7 years ago

OK thanks. I'm going to announce the new stable on facebook/google+ now.

comment:25 by bastiK, 7 years ago

In 11833/josm:

see #14422 - Debian package: add dependency for proj-data (patch by sebastic)

comment:26 by bastiK, 6 years ago

The BETA2007.gsb grid file has finally been added by the proj.4 team: https://github.com/OSGeo/proj.4/pull/383

As soon as this change propagates to proj-data, users of the JOSM .deb package will have all grid files locally available.

Modify Ticket

Change Properties
Set your email in Preferences
Action
as closed The owner will remain Don-vip.
as The resolution will be set.
The resolution will be deleted. Next status will be 'reopened'.

Add Comment


E-mail address and name can be saved in the Preferences .
 
Note: See TracTickets for help on using tickets.