Detect captive portals

[richlv]

tried to download and got redirected to a crappy hotel login page

Repository Root: http://josm.openstreetmap.de/svn
Build-Date: 2012-11-10 02:31:13
Last Changed Author: bastiK
Revision: 5576
Repository UUID: 0c6e7542-c601-0410-84e7-c038aed88b3b
URL: http://josm.openstreetmap.de/svn/trunk
Last Changed Date: 2012-11-09 21:16:38 +0100 (Fri, 09 Nov 2012)
Last Changed Rev: 5576

Identification: JOSM/1.5 (5576 en)
Memory Usage: 142 MB / 682 MB (20 MB allocated, but free)
Java version: 1.6.0_24, Sun Microsystems Inc., OpenJDK 64-Bit Server VM
Operating system: Linux
Dataset consistency test: No problems found

Plugin: PicLayer (28807)
Plugin: terracer (28807)

org.openstreetmap.josm.io.OsmTransferException: org.xml.sax.SAXParseException; lineNumber: 2; columnNumber: 6; The processing instruction target matching "[xX][mM][lL]" is not allowed.
	at org.openstreetmap.josm.io.BoundingBoxDownloader.parseRawGps(BoundingBoxDownloader.java:91)
	at org.openstreetmap.josm.actions.downloadtasks.DownloadGpsTask$DownloadTask.realRun(DownloadGpsTask.java:105)
	at org.openstreetmap.josm.gui.PleaseWaitRunnable.doRealRun(PleaseWaitRunnable.java:82)
	at org.openstreetmap.josm.gui.PleaseWaitRunnable.run(PleaseWaitRunnable.java:145)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
	at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
	at java.util.concurrent.FutureTask.run(FutureTask.java:166)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
	at java.lang.Thread.run(Thread.java:679)
Caused by: org.xml.sax.SAXParseException; lineNumber: 2; columnNumber: 6; The processing instruction target matching "[xX][mM][lL]" is not allowed.
	at com.sun.org.apache.xerces.internal.util.ErrorHandlerWrapper.createSAXParseException(ErrorHandlerWrapper.java:198)
	at com.sun.org.apache.xerces.internal.util.ErrorHandlerWrapper.fatalError(ErrorHandlerWrapper.java:177)
	at com.sun.org.apache.xerces.internal.impl.XMLErrorReporter.reportError(XMLErrorReporter.java:391)
	at com.sun.org.apache.xerces.internal.impl.XMLScanner.reportFatalError(XMLScanner.java:1404)
	at com.sun.org.apache.xerces.internal.impl.XMLScanner.scanPIData(XMLScanner.java:691)
	at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanPIData(XMLDocumentFragmentScannerImpl.java:997)
	at com.sun.org.apache.xerces.internal.impl.XMLScanner.scanPI(XMLScanner.java:659)
	at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl$PrologDriver.next(XMLDocumentScannerImpl.java:931)
	at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(XMLDocumentScannerImpl.java:625)
	at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:488)
	at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:819)
	at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:748)
	at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:123)
	at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1208)
	at com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$JAXPSAXParser.parse(SAXParserImpl.java:525)
	at javax.xml.parsers.SAXParser.parse(SAXParser.java:392)
	at org.openstreetmap.josm.io.GpxReader.parse(GpxReader.java:420)
	at org.openstreetmap.josm.io.BoundingBoxDownloader.downloadRawGps(BoundingBoxDownloader.java:45)
	at org.openstreetmap.josm.io.BoundingBoxDownloader.parseRawGps(BoundingBoxDownloader.java:79)
	... 9 more

[bastiK]

Could you please describe the redirect process in as much detail as possible? I'm asking, because (as far as I know) there is no code that would cause JOSM to open a browser window at that point.

Is it reproducible? In that case a wireshark log would be very helpful.

Please save the JOSM file that caused this and note the URL you got redirected to.

[richlv]

tcpdump -w output when downloading

[richlv]

tcpdump -w output when pressing ctrl-h

[richlv]

this is easily reproducible in two cases :

• press download button, dialog opens, click "download";
• select an object, press ctrl-h (note that shift-ctrl-h does not produce and exception)

attached is tcpdump -w output in both cases

[bastiK]

JOSM sends a GET request to ​http://api.openstreetmap.org, the server seemingly answers with 302 REDIRECT to http://wireless.hp.internal:8080/index.asp. JOSM follows the redirect and downloads some kind of wireless login page, which is not the OSM/GPX data it expects.

Please call

wget -SO output.txt http://api.openstreetmap.org/api/0.6/map?bbox=-105.1615114,39.9732381,-105.1580688,39.9748677

on the command line. If you get the same login page, then this is not a JOSM problem, but caused by your local setup (router, firewall) or the OSM server (unlikely).

[stoecker]

This looks much like a login mechanism of mobile web access. You once need to open a browser, request any page and login before you can use the connection for other purposes.

[richlv]

of course. exactly. but josm shouldn't throw an unexpected exception at me and ask me to report a bug ;)
it already knows it's not proper data, it should gracefully fail and tell me that (which it partially does upon download, btw)

[Don-vip]

Ticket #6194 has been marked as a duplicate of this ticket.

[Don-vip]

Ticket #13660 has been marked as a duplicate of this ticket.

[Don-vip]

Some reading: ​https://bugzilla.mozilla.org/show_bug.cgi?id=562917#c5

[Don-vip]

So from ​https://bugzilla.mozilla.org/show_bug.cgi?id=1048131#c45 I understand Firefox detects captive portals by checking if ​http://detectportal.firefox.com/success.txt returns "success":

​https://hg.mozilla.org/integration/mozilla-inbound/rev/5f2ca3ac2fcb
​https://hg.mozilla.org/integration/mozilla-inbound/rev/ddd6b2e3aead

// If minInterval is 0, the check will only happen
// when the service has a strong suspicion we are in a captive portal
pref("network.captive-portal-service.minInterval", 60000); // 60 seconds
pref("network.captive-portal-service.maxInterval", 1500000); // 25 minutes
// Every 10 checks, the delay is increased by a factor of 5
pref("network.captive-portal-service.backoffFactor", "5.0");
pref("network.captive-portal-service.enabled", false);

pref("captivedetect.canonicalURL", "http://detectportal.firefox.com/success.txt");
pref("captivedetect.canonicalContent", "success\n");
pref("captivedetect.maxWaitingTime", 5000);
pref("captivedetect.pollingTime", 3000);
pref("captivedetect.maxRetryCount", 5);

[stoecker]

Another possibility would be to do DNS resolution for two IP addresses (e.g. josm.openstreetmap.de and www.google.de). They will have the same IP in a captive portal.

[stoecker]

P.S. I REALLY want better DNSSEC usage. It would kill such bullshit.

[stoecker]

Hmm, we already use a HTTP connection to josm server for IPv6 detection. We can expand that similar to above Mozilla approach. That would not add another net check.

More important question: What do we do when we detect a portal?

[Don-vip]

Replying to stoecker:

More important question: What do we do when we detect a portal?

I was thinking to display a small panel like Firefox, with a small button that launches an external browser:

[stoecker]

We also need to prevent any background network transmissions or they may get invalid.