Modify

Opened 7 years ago

Closed 7 years ago

#8505 closed defect (fixed)

[Patch] Encoding of URLs

Reported by: roland.olbricht Owned by: team
Priority: normal Milestone:
Component: Core Version: tested
Keywords: URL encoding Cc:

Description

Under Windows 7 with Java 6, when typing an URL into "File > Open address ...", this URL is sent enocded as Windows-1252. It should be sent encoded as UTF8.

Example:
http://overpass-api.de/api/interpreter?data=way[name=Kornstraße];out+meta;
is sent with a single byte 0xDF (which looks like Windows-1252).

I haven't tested it with other systems, so I could be both platform dependend or platform independend.

Attachments (0)

Change History (9)

comment:1 Changed 7 years ago by Don-vip

How do you test this ? There is currently no suitable download task for this kind of URL ?

EDIT: sorry my mistake, there is one, I don't know why I'm getting this error.

EDIT2: Found: I had a zero-width space character at the beginning of the URL, ignoring it in JOSM right now.

Last edited 7 years ago by Don-vip (previous) (diff)

comment:2 Changed 7 years ago by Don-vip

In 5772/josm:

see #8505 - strip of trailing and leading whitespace characters of input URL (proper solution than String.trim(), see Utils.strip()) + unit test

comment:3 Changed 7 years ago by Don-vip

I have tried to change some internal mechanisms of JOSM, now I have this URL:
http://overpass-api.de/api/interpreter?data%3Dway%5Bname%3DKornstra%C3%9Fe%5D%3Bout%2Bmeta%3B
It raises some parser errors on Overpass API side, is it feasible to fix them ?

comment:4 Changed 7 years ago by roland.olbricht

First of all, thank you for fixing the bug. It now works for me with the version from svn, with URL
http://overpass-api.de/api/interpreter?data=(way[name="Kornstraße"];>;);out%20meta;

I'm sorry for the confusion. There are two different problems in this query:

  • Kornstraße has no quotation marks around the word. The syntax parser can be fixed to handle the string also without quotation marks.
  • The equation sign after daata is URL encoded. I would prefer not to fix that. This should be a proper equation mark to make to whole URL properly URL encoded like a GET sent web form.

comment:5 Changed 7 years ago by Don-vip

I have not changed that much :) If it works for you now, can we close this ticket ?

comment:6 Changed 7 years ago by roland.olbricht

Sorry, it still doesn't work. From the logfile of the server:

[13/Mar/2013:18:29:42 +0100] query string: /api/interpreter?data=(way[name=\"Kornstra\xc3\x9fe\"];>;);out%20meta;, user agent: JOSM/1.5 (5772 SVN de)
[14/Mar/2013:10:12:35 +0100] query string: /api/interpreter?data=(way[name=\"Kornstra\xdfe\"];>;);out%20meta;, user agent: JOSM/1.5 (5772 SVN de)

This is the same JAR file, executed from an USB drive. The first line is from a run under Linux, the second from a run under Windows.

comment:7 Changed 7 years ago by roland.olbricht

I suggest the following patch:

Insert into class DownloadOsmTask, in method loadUrl immediately after the begin:

try
{

if (url.matches(PATTERN_OVERPASS_API_URL))
{

int pos = url.indexOf("/interpreter?data=") + 18; encode only the part after the equal sign
url = url.substring(0, pos) + URLEncoder.encode(url.substring(pos), "UTF-8");

}
else if (url.matches(PATTERN_OVERPASS_API_XAPI_URL))
{

int pos = url.indexOf("/xapi?") + 6; encode only the part after the equal sign
url = url.substring(0, pos) + URLEncoder.encode(url.substring(pos), "UTF-8");

}

}
catch (UnsupportedEncodingException e)
{
}

The used classes for import are:
import java.io.UnsupportedEncodingException;
import java.net.URLEncoder;

This encodes the URL properly and platform independend in percent encoding.

I've just added a corresponding patch to the mirrored_download plugin.

comment:8 Changed 7 years ago by Don-vip

Summary: Encoding of URLs[Patch] Encoding of URLs

comment:9 Changed 7 years ago by Don-vip

Resolution: fixed
Status: newclosed

In 5782/josm:

fix #8505 - Encoding of overpass API URLs (based on patch by roland.olbricht)

Modify Ticket

Change Properties
Set your email in Preferences
Action
as closed The owner will remain team.
as The resolution will be set.
The resolution will be deleted.

Add Comment


E-mail address and name can be saved in the Preferences.

 
Note: See TracTickets for help on using tickets.