Opened 3 years ago

Last modified 3 years ago

#11682 new enhancement

Improve the usability of importing a CSV file

Reported by: flaimo Owned by: Don-vip
Priority: normal Milestone:
Component: Plugin opendata Version:
Keywords: template_report Cc:

Description (last modified by flaimo)

I tried importing data from a simple CSV file and I think that, at the moment, this use case is very unintuitive and not flexible enough.

a) First of all I think that – regardless of other file formats – the CSV import functionality should be implemented in the core, not just as a plugin. Nevertheless, if you go to the plugins tab in the settings and search for CSV you won't find the plugin (opendata) where it is currently implemented. That is bad. So either extract the CSV functionality and put it in the core or in a separate import plugin which has nothing to do with opendata portals (why are those two things mangled together anyway) and give it a fitting description.

b) There is basically no interaction with the user during the import procedure. If the data doesn't fit, you just get an error message and have to tingle with the CSV file yourself. The usability of this use case could be vastly improved if it was implemented like the import wizard of Microsoft Excel, which could look something like this:

1) User clicks File→Open and selects a CSV file.
2) The first dialog pops up. The user selects the file encoding and the column separator character and if the data is in quotation marks or not. The user can also define if the first line are the column headers or not. The wizard tries to preparse the file and suggest the right values in this dialog.
3) The User clicks "Next" and the next dialog shows up. Here the user can define, if there is one column that holds the list of coordinates that describes a closed way, or the two columns that represent latitude and longitude. Also the projection can be selected. Again the wizard tries to preparse the file and suggest the right values in this dialog.
4) The User clicks "Next" and the next dialog shows up. Here the user can map the remaining individual columns to OSM keys using text input fields. Again the wizard tries to prefill the text fields with the most fitting OSM key based on the column header (if available).
5) The User clicks "Next". If there is no data layer yet, a new one is created an the data is imported there. If there is a data layer open already, the wizard asks if the data should be added to the current layer or imported into a new layer.

Other thinks to consider:
c) How large can the CSV file be. Can JOSM handle the import of 100000 lines and above or does a maximum number of lines limit need to be implemented to ensure the stability of the running program?
d) In case the CSV file contains a column with an external ID there could be an addon functionality which tries to synchronise the objects of an open data layer with the ones of the CSV file based on that ID. The user could mark the ID column during the import using a simple checkbox and afterwards JOSM updates all existing objects with those IDs it can find in the data layer. Of course there would be the need to handle conflicts on a case per case basis but also in batches (aka “Skip all existing objects”, “Only add new tags”, “Replace data of all existing objects”, and so on.). The same sould be implemented if the CSV file contains a column with the OSM database ID of an object as long as the object type (node or way) matches the existing one.

Real life example: My home town provides a list of all managed trees as a CSV file (german "Baumkataster") which gets updated regularly. This way updating the data without having to have database knowledge would be a breeze.

Please provide any additional information below. Attach a screenshot if possible.

Revision: 8491
Repository Root:
Relative URL: ^/trunk
Last Changed Author: Don-vip
Last Changed Date: 2015-06-16 23:27:08 +0200 (Tue, 16 Jun 2015)
Build-Date: 2015-06-16 21:45:58
Repository UUID: 0c6e7542-c601-0410-84e7-c038aed88b3b
Last Changed Rev: 8491

Identification: JOSM/1.5 (8491 en) Mac OS X 10.10.4
Memory Usage: 409 MB / 910 MB (125 MB allocated, but free)
Java version: 1.8.0_45, Oracle Corporation, Java HotSpot(TM) 64-Bit Server VM
VM arguments: [-Djava.library.path=/Applications/, -DLibraryDirectory=/Users/flaimo/Library, -DDocumentsDirectory=/Users/flaimo/Documents, -DApplicationSupportDirectory=/Users/flaimo/Library/Application Support, -DCachesDirectory=/Users/flaimo/Library/Caches, -DSandboxEnabled=false, -Dapple.laf.useScreenMenuBar=true,,,,]

- ColorPlugin (1414145445)
- HouseNumberTaggingTool (31241)
- OpeningHoursEditor (31241)
- PicLayer (31241)
- RoadSigns (31241)
- ShapeTools (1000)
- buildings_tools (31241)
- contourmerge (1013)
- geotools (31126)
- jts (31126)
- kendzi3d-jogl (37)
- kendzi3d_Improved_by_Andrei (1.0.179-SNAPSHOT)
- log4j (31231)
- measurement (31241)
- opendata (31241)
- reverter (31241)
- terracer (31241)
- turnlanes (31241)
- turnrestrictions (31241)
- undelete (31241)
- utilsplugin2 (31241)
- wikipedia (31241)

Last errors/warnings:
- E: Failed to locate image ''
- E: Failed to locate image ''
- E: Failed to locate image ''
- W: No valid coordinates have been found.
- E: java.lang.IllegalArgumentException: No valid coordinates have been found.

Attachments (0)

Change History (2)

comment:1 Changed 3 years ago by flaimo

Description: modified (diff)

comment:2 Changed 3 years ago by simon04

Component: CorePlugin opendata
Owner: changed from team to Don-vip

I think it's related to the opendata plugin.

Modify Ticket

Change Properties
Set your email in Preferences
as new The owner will remain Don-vip.
as The resolution will be set.
to The owner will be changed from Don-vip to the specified user.
The owner will change to flaimo
as duplicate The resolution will be set to duplicate.The specified ticket will be cross-referenced with this ticket
The owner will be changed from Don-vip to anonymous.

Add Comment

E-mail address and name can be saved in the Preferences.

Note: See TracTickets for help on using tickets.