Command line output unreadable on Windows

[gaben]

What steps will reproduce the problem?

1. Use Windows, set lang to Hungarian
2. Set JOSM language to Hungarian
3. Start JOSM from command line (cmd or powershell, doesn't matter):

java -jar "josm-snapshot-15752.jar"

What is the expected result?

Readable output with valid characters.

What happens instead?

JOSM printing invalid characters and it's not readable easily. See below:

2020-01-23 17:28:11.200 INFO: Naplˇzßsi szint: INFO (INFO, 800)
2020-01-23 17:28:11.370 INFO: Removed preference default remotecontrol.https.enabled since it is no longer used
2020-01-23 17:28:11.428 INFO: Saved updated default preferences.
2020-01-23 17:28:11.853 INFO: IPv6 hßlˇzat Úszlelve; IPv6 preferßlßsa IPv4 ellenÚben.
2020-01-23 17:28:12.777 INFO: 'jaxb' b§vÝtmÚny bet÷ltÚse (35092 verziˇ)
2020-01-23 17:28:13.171 INFO: Reassigning shortcut 'file:download_along' from null to 640 because of conflict with Alt+Shift+D
2020-01-23 17:28:13.172 INFO: Csendes gyorsbillentyű konfliktus: 'file:download_along' ßtmozgatßsra kerŘlt 'continuosdownload:activate'-r§l 'Ctrl+Alt+F1'-re.
2020-01-23 17:28:13.539 INFO: GET https://api.openstreetmap.org/api/0.6/user/details -> HTTP_1 200 (683 B)
2020-01-23 17:28:14.040 INFO: GET https://josm.openstreetmap.de/wiki/Hu:StartupPage -> HTTP_1 200
2020-01-23 17:28:14.120 INFO: Obtained 51 Tag2Link rules from https://query.wikidata.org/sparql
2020-01-23 17:28:14.175 INFO: Obtained 11 Tag2Link rules from https://sophox.org/sparql
2020-01-23 17:28:14.408 INFO: "Bing műholdkÚp" lÚgifelvÚtel frissÝtÚse
2020-01-23 17:28:14.408 INFO: "Esri World Imagery" lÚgifelvÚtel frissÝtÚse
2020-01-23 17:28:14.410 INFO: "Esri World Imagery (Clarity) Beta" lÚgifelvÚtel frissÝtÚse

Please provide any additional information below. Attach a screenshot if possible.

For example, the first entry is
2020-01-23 17:28:11.200 INFO: Naplˇzßsi szint: INFO (INFO, 800)
instead of
2020-01-23 17:28:11.200 INFO: Naplózási szint: INFO (INFO, 800)

Older JOSM versions work fine. I checked r10526 and it printed readable characters.

URL:https://josm.openstreetmap.de/svn/trunk
Repository:UUID: 0c6e7542-c601-0410-84e7-c038aed88b3b
Last:Changed Date: 2020-01-22 22:31:24 +0100 (Wed, 22 Jan 2020)
Build-Date:2020-01-23 02:30:55
Revision:15752
Relative:URL: ^/trunk

Identification: JOSM/1.5 (15752 hu) Windows 10 64-Bit
OS Build number: Windows 10 Pro 1909 (18363)
Memory Usage: 1066 MB / 1820 MB (666 MB allocated, but free)
Java version: 1.8.0_231-b11, Oracle Corporation, Java HotSpot(TM) 64-Bit Server VM
Screen: \Display0 1920x1200
Maximum Screen Size: 1920x1200
VM arguments: [-Djava.security.manager, -Djava.security.policy=file:<java.home>\lib\security\javaws.policy, -DtrustProxy=true, -Djnlpx.home=<java.home>\bin, -Djnlpx.origFilenameArg=%UserProfile%\AppData\LocalLow\Sun\Java\Deployment\cache\6.0\31\583aa85f-3ed5e881, -Djnlpx.remove=false, -Djava.util.Arrays.useLegacyMergeSort=true, -Djnlpx.heapsize=NULL,2048m, -Djnlpx.splashport=57967, -Djnlp.application.href=https://josm.openstreetmap.de/download/josm-latest.jnlp, -Djnlpx.jvm=<java.home>\bin\javaw.exe]

[gaben]

Just found JOSM archive so I could do some research :) The last working archive version is r10895 and the first wrong one is r10899.

The r10899 printing this new entry while older versions not:
FINE: System property 'java.locale.providers' set to 'JRE,CLDR'. Old value was 'null'

[gaben]

my first patch ever on the internet :)

[gaben]

I'm not sure if I broke anything, but it works fine for me. Please check your output.

[gaben]

It fixed only the IDE terminal output, the system terminal got new charset which is still invalid. Maybe a terminal encoding check needed?

[simon04]

Hello gaben and welcome to JOSM development! :)

Thank you for your patch. I did some testing and managed to get correct console output on Windows' cmd.exe by doing the following:

Microsoft Windows [Version 10.0.18363.592]
(c) 2019 Microsoft Corporation. Alle Rechte vorbehalten.

C:\Program Files (x86)\JOSM>chcp 65001
Active Codepage: 65001.

C:\Program Files (x86)\JOSM>java -Dfile.encoding=UTF-8 -jar josm-tested.jar --language=ru --debug
...
2020-01-26 23:16:34.435 FINE: Инициализация
2020-01-26 23:16:34.451 FINE: Определение модулей для скачивания...
2020-01-26 23:16:34.451 FINE: Plugins list initialized to []
2020-01-26 23:16:34.451 FINE: Удаление нежелательных модулей...
2020-01-26 23:16:34.451 FINE: Удаление неподдерживаемых модулей...
2020-01-26 23:16:34.451 FINE: Plugins list is finally set to []
2020-01-26 23:16:34.451 FINE: Чтение информации об установленных модулях...
2020-01-26 23:16:34.451 FINE: Модуль обработки файлов кэша сайта...
2020-01-26 23:16:34.467 FINE: Обработка файла 'site-josm.openstreetmap.de-_pluginicons.txt'
2020-01-26 23:16:34.857 FINE: Установка обновлённых модулей
2020-01-26 23:16:34.857 FINE: Загрузка ранних модулей
2020-01-26 23:16:34.857 FINE: Загрузка модулей с сервера...
2020-01-26 23:16:34.873 FINE: Проверка входных условий для модулей...
2020-01-26 23:16:34.873 FINE: Настройки по умолчанию
2020-01-26 23:16:35.201 FINE: Создание интерфейса
2020-01-26 23:16:35.217 FINE: Инициализация формата координат
2020-01-26 23:16:35.232 FINE: Инициализация формата координат завершено за  (15 мс)
2020-01-26 23:16:35.232 FINE: Запуск наблюдения за файлами
2020-01-26 23:16:35.232 FINE: Запуск наблюдения за файлами завершено за  (0 мс)
2020-01-26 23:16:35.232 FINE: File watcher thread started
2020-01-26 23:16:35.232 FINE: Исполнение стартовых процедур
2020-01-26 23:16:35.232 FINE: java.lang.ClassNotFoundException: com.sun.deploy.config.BuiltInProperties
2020-01-26 23:16:35.232 FINE: Исполнение стартовых процедур завершено за  (0 мс)
2020-01-26 23:16:35.232 FINE: Создание основного меню
2020-01-26 23:16:36.467 FINE: Создание основного меню завершено за  (1,2 с)
2020-01-26 23:16:36.467 FINE: Обновление интерфейса пользователя
2020-01-26 23:16:36.529 FINE: Обновление интерфейса пользователя завершено за  (62 мс)
2020-01-26 23:16:36.529 FINE: Инициализация внутренних данных о границах
2020-01-26 23:16:36.560 FINE: Подготовка данных...
2020-01-26 23:16:36.560 FINE: Разбор данных OSM...
...

So basically, one has to invoke chcp 65001 in the cmd.exe window before launching JOSM. The -Dfile.encoding=UTF-8 has a similar effect as your changes from attachment:18608v1.patch​​

I updated the documentation at Help/CommandLineOptions#Javaoptions.

[gaben]

Thanks, Simon! :)

Checked your findings and could confirm the workaround. It's working in cmd but unfortunately not in PowerShell. With the patch, I wanted to replicate the pre r10899 behaviour which works fine without any extra arguments or console tweaking in both environments.

You can download it from here https://josm.openstreetmap.de/download/Archiv/josm-snapshot-10895.jar

(The patch indeed equivalent with the -Dfile.encoding=UTF-8 flag. Nevermind, at least I tried.)

[stoecker]

Is the behaviour wrong since r10899 or is no longer UTF-8 since then? As far as I know in Windows shells there is no proper way to determine the correct encoding, so it can only be guessed.

So I assume output was broken before r10899 for standard Windows.

Your patch is wrong because of this. UTF-8 is not the correct choice for all systems. Either there is a way to autodetect the correct choice in JOSM (BTW: did you try to set proper LANG environment variables like ru_RU.UTF-8 in the system) or you need to pass UTF-8 as argument.

[gaben]

Replying to stoecker:

So I assume output was broken before r10899 for standard Windows.

I just tested the behaviour in a virtual machine with

• an official Windows 10 1909 x64 (Hungarian) ISO
• with JDK 1.8.241 (x64) and JDK 11.0.6 (x64)
• in PowerShell and cmd

and I can say this is a regression. It's broken since r10899.

Updated the steps.