Modify

Opened 12 months ago

Last modified 6 months ago

#16258 new enhancement

GDPR related API changes

Reported by: Don-vip Owned by: team
Priority: major Milestone:
Component: Core Version:
Keywords: GDPR api anonymous Cc:

Description (last modified by Don-vip)

Seen on dev mailing list:

The LWG has made some recommendations about what we need to change on the web site and API to comply with future European data protection rules. On the whole this boils down to "logged-in users get the same stuff they get today, but guests who are not logged in will not see details about users".

The detailed LWG recommendations are here.

List of what Frederik believes needs to be changed on the API and web site, here.

For JOSM it means we have to check the various anonymous API calls don't provoke any crash when data is missing. I doubt it :)

GDPR enters into force on 25th May. So we must release 18.05 before that.

The API is not going to change before GDPR enters in to force, as per Frederik's e-mail:

the general feeling is that it's going to be something like a 6 month project for the whole package, with some measures certainly being taken early on to demonstrate our will to comply. Regarding the API changes specifically, once we're clear about the scope we'll need to determine how much work it is to make the changes, and then find someone to make them... it's certainly on a scale of months not weeks.

Attachments (0)

Change History (13)

comment:1 Changed 12 months ago by Don-vip

Description: modified (diff)
Milestone: 18.0518.07

comment:2 Changed 12 months ago by Don-vip

Description: modified (diff)

comment:3 Changed 12 months ago by stoecker

Would be nice when we have some time between these changes in DEV API and the real one. Also for JOSM this probably will be a "few months" topic.

To ease changes we may keep internally the structure and simply replace them with a "pseudo" user on import. Then only import/export and the external calls (history, userinfo, webpage calls) must handle these cases. Not all the dialogs.

comment:4 Changed 12 months ago by mmd

Most JOSM users should already have entered some credentials, and as long as you provide those credentials to the API, you're still getting exactly the same data as before. Only for unauthenticated API calls some data will get dropped, like mentioned on the GDPR Wiki page.

JOSM currently offers both OAuth and Basic auth. Unfortunately, for those calls handled by cgimap (most importantly that's /map), Basic auth is not yet implemented. If you think that's an important use case for JOSM users, please raise an issue over at https://github.com/zerebubuth/openstreetmap-cgimap/issues and suggest to have Basic Auth also implemented for cgimap.

comment:5 Changed 12 months ago by mmd

I'm just testing a bit on a custom /map implementation that hides changeset, uid and user for unauthenticated users. It's meant as a very first impression where potential issues might be, not much more.

Object history

Missing changeset attribute

org.openstreetmap.josm.io.OsmTransferException: org.xml.sax.SAXException: (3,83)Notwendiges Attribut "changeset" fehlt.
	at org.openstreetmap.josm.io.OsmServerHistoryReader.parseHistory(OsmServerHistoryReader.java:77)
	at org.openstreetmap.josm.gui.history.HistoryLoadTask.loadHistory(HistoryLoadTask.java:197)
	at org.openstreetmap.josm.gui.history.HistoryLoadTask.loadHistory(HistoryLoadTask.java:187)
	at org.openstreetmap.josm.gui.history.HistoryLoadTask.realRun(HistoryLoadTask.java:172)
	at org.openstreetmap.josm.gui.PleaseWaitRunnable.doRealRun(PleaseWaitRunnable.java:95)
	at org.openstreetmap.josm.gui.PleaseWaitRunnable.run(PleaseWaitRunnable.java:143)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)
Caused by: org.xml.sax.SAXException: (3,83)Notwendiges Attribut "changeset" fehlt.
	at org.openstreetmap.josm.io.OsmHistoryReader$Parser.throwException(OsmHistoryReader.java:48)
	at org.openstreetmap.josm.io.AbstractParser.getMandatoryAttributeLong(AbstractParser.java:44)
	at org.openstreetmap.josm.io.AbstractParser.createPrimitive(AbstractParser.java:110)
	at org.openstreetmap.josm.io.AbstractParser.startWay(AbstractParser.java:149)
	at org.openstreetmap.josm.io.AbstractParser.doStartElement(AbstractParser.java:187)
	at org.openstreetmap.josm.io.OsmHistoryReader$Parser.startElement(OsmHistoryReader.java:58)
	at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.startElement(AbstractSAXParser.java:509)
	at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.scanStartElement(XMLNSDocumentScannerImpl.java:380)
	at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl$FragmentContentDriver.next(XMLDocumentFragmentScannerImpl.java:2787)
	at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(XMLDocumentScannerImpl.java:606)
	at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.next(XMLNSDocumentScannerImpl.java:118)
	at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:510)
	at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:848)
	at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:777)
	at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:141)
	at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1213)
	at com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$JAXPSAXParser.parse(SAXParserImpl.java:643)
	at com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl.parse(SAXParserImpl.java:327)
	at org.openstreetmap.josm.tools.Utils.parseSafeSAX(Utils.java:1418)
	at org.openstreetmap.josm.io.OsmHistoryReader.parse(OsmHistoryReader.java:94)
	at org.openstreetmap.josm.io.OsmServerHistoryReader.parseHistory(OsmServerHistoryReader.java:70)
	... 10 more

Map result:

<?xml version='1.0' encoding='UTF-8'?>
<osm version='0.6' generator='custom'>
  <bounds minlat='39.9105262' minlon='116.3581486' maxlat='39.9140166' maxlon='116.363392' origin='CGImap 0.6.0 (3940 ubuntu)' />
  <node id='-39436' action='modify' lat='39.91321943402' lon='116.35923758974' />
  <node id='5001648038' timestamp='2018-05-06T21:05:31Z' version='1' lat='39.9129772' lon='116.3657367' />
  <node id='5001648039' timestamp='2018-05-06T21:05:31Z' version='1' lat='39.9129375' lon='116.3632287' />
  <node id='5001648040' timestamp='2018-05-06T21:05:31Z' version='1' lat='39.9136042' lon='116.3654369' />
  <node id='5001648041' timestamp='2018-05-06T21:05:31Z' version='1' lat='39.9118641' lon='116.3657018' />
...
Last edited 12 months ago by Don-vip (previous) (diff)

comment:6 in reply to:  5 Changed 12 months ago by Don-vip

Replying to mmd:

I'm just testing a bit on a custom /map implementation that hides changeset, uid and user for unauthenticated users. It's meant as a very first impression where potential issues might be, not much more.

Thank you :) Is it available somewhere so that we could test ourselves?

comment:7 Changed 12 months ago by Don-vip

In 13762/josm:

see #16258 - changeset attribute can now be missing

comment:8 Changed 12 months ago by mmd

Great, thanks a lot for the fix!

Is it available somewhere so that we could test ourselves?

At this time, I'm testing a local prototype, along with a local DB and Rails port. Hopefully, we'll have something more accessible soon... see https://github.com/zerebubuth/openstreetmap-cgimap/issues/144

You could also try to create an osm.xml file locally where those attributes are missing, and load it into JOSM. In any case, a proper API implementation would be more helpful, though.

comment:9 Changed 11 months ago by Don-vip

OK, let us know once the prototype is available.

comment:10 Changed 9 months ago by Don-vip

Milestone: 18.0718.08

comment:11 Changed 9 months ago by Don-vip

See #16499 for GDPR-compliant extracts

comment:12 Changed 8 months ago by Don-vip

Milestone: 18.08

Modify Ticket

Change Properties
Set your email in Preferences
Action
as new The owner will remain team.
as The resolution will be set.
to The owner will be changed from team to the specified user.
The owner will change to Don-vip
as duplicate The resolution will be set to duplicate.The specified ticket will be cross-referenced with this ticket
The owner will be changed from team to anonymous.

Add Comment


E-mail address and name can be saved in the Preferences.

 
Note: See TracTickets for help on using tickets.