Some tests are failing

[naoliv]

I am seeing some tests failing (when running ant test) with the latest JOSM version (14187):

    [junit] Running org.openstreetmap.josm.data.coor.conversion.ICoordinateFormatTest
    [junit] Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.537 sec
    [junit] TEST org.openstreetmap.josm.data.coor.conversion.ICoordinateFormatTest FAILED

    [junit] Running org.openstreetmap.josm.tools.ExifReaderTest
    [junit] Tests run: 9, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 19 sec
    [junit] TEST org.openstreetmap.josm.tools.ExifReaderTest FAILED

    [junit] Running org.openstreetmap.josm.tools.PlatformHookOsxTest
    [junit] Tests run: 9, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.679 sec
    [junit] TEST org.openstreetmap.josm.tools.PlatformHookOsxTest FAILED

    [junit] Running org.openstreetmap.josm.gui.mappaint.MapCSSRendererTest
    [junit] Tests run: 22, Failures: 21, Errors: 0, Skipped: 0, Time elapsed: 11.204 sec
    [junit] TEST org.openstreetmap.josm.gui.mappaint.MapCSSRendererTest FAILED

    [junit] Running org.openstreetmap.josm.io.MultiFetchServerObjectReaderTest
    [junit] Tests run: 0, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 4.522 sec
    [junit] TEST org.openstreetmap.josm.io.MultiFetchServerObjectReaderTest FAILED

    [junit] Running org.openstreetmap.josm.io.OsmServerBackreferenceReaderTest
    [junit] Tests run: 0, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 1.904 sec
    [junit] TEST org.openstreetmap.josm.io.OsmServerBackreferenceReaderTest FAILED

This one doesn't cause any failure, but gives a warning:

    [junit] Running org.openstreetmap.josm.data.osm.search.SearchCompilerTest
    [junit] WARNING: An illegal reflective access operation has occurred
    [junit] WARNING: Illegal reflective access by nl.jqno.equalsverifier.internal.reflection.FieldAccessor (file:/tmp/josm/test/lib/equalsverifier-2.5.1.jar) to field javax.swing.AbstractAction.enabled
    [junit] WARNING: Please consider reporting this to the maintainers of nl.jqno.equalsverifier.internal.reflection.FieldAccessor
    [junit] WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
    [junit] WARNING: All illegal access operations will be denied in a future release
    [junit] Tests run: 30, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 24.657 sec

The full ant test output and the report dir are attached.

Java:

$ java -version
Picked up _JAVA_OPTIONS: -Dawt.useSystemAAFontSettings=on
openjdk version "10.0.2" 2018-07-17
OpenJDK Runtime Environment (build 10.0.2+13-Debian-1)
OpenJDK 64-Bit Server VM (build 10.0.2+13-Debian-1, mixed mode)

ant:

$ ant -version
Picked up _JAVA_OPTIONS: -Dawt.useSystemAAFontSettings=on
Apache Ant(TM) version 1.10.5 compiled on July 16 2018

locale:

$ locale
LANG=pt_BR.UTF-8
LANGUAGE=pt_BR:pt:en
LC_CTYPE="pt_BR.UTF-8"
LC_NUMERIC="pt_BR.UTF-8"
LC_TIME="pt_BR.UTF-8"
LC_COLLATE="pt_BR.UTF-8"
LC_MONETARY="pt_BR.UTF-8"
LC_MESSAGES="pt_BR.UTF-8"
LC_PAPER="pt_BR.UTF-8"
LC_NAME="pt_BR.UTF-8"
LC_ADDRESS="pt_BR.UTF-8"
LC_TELEPHONE="pt_BR.UTF-8"
LC_MEASUREMENT="pt_BR.UTF-8"
LC_IDENTIFICATION="pt_BR.UTF-8"
LC_ALL=

Timezone is GMT -3

[Don-vip]

​https://github.com/jqno/equalsverifier/issues/206 reported for the equalsverifier problem

[Don-vip]

The bz2 archive is corrupted

[naoliv]

It isn't corrupted :-)
It was compressed with pbzip2; maybe the decompressor that you are using doesn't properly support it (like in #9537)
But I did attach a gzipped file with the same content.

[Don-vip]

OK, thanks!

[Don-vip]

First failures (ICoordinateFormatTest.testFormatting and ExifReaderTest.testReadLatLon) are tracked by #15696

[Don-vip]

The PlatformHookOsxTest failure is new:

Testcase: testOpenUrl took 0.011 sec
	Caused an ERROR
Cannot run program "open": error=2, Arquivo ou diretório inexistente
java.io.IOException: Cannot run program "open": error=2, Arquivo ou diretório inexistente
	at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1128)
	at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1071)
	at java.base/java.lang.Runtime.exec(Runtime.java:635)
	at java.base/java.lang.Runtime.exec(Runtime.java:459)
	at java.base/java.lang.Runtime.exec(Runtime.java:356)
	at org.openstreetmap.josm.tools.PlatformHookOsx.openUrl(PlatformHookOsx.java:206)
	at org.openstreetmap.josm.tools.PlatformHookOsxTest.testOpenUrl(PlatformHookOsxTest.java:66)
Caused by: java.io.IOException: error=2, Arquivo ou diretório inexistente
	at java.base/java.lang.ProcessImpl.forkAndExec(Native Method)
	at java.base/java.lang.ProcessImpl.<init>(ProcessImpl.java:339)
	at java.base/java.lang.ProcessImpl.start(ProcessImpl.java:270)
	at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1107)

This test should only be run on macOS.

[Don-vip]

For MapCSSRendererTest I need additional files. You should have several files named test-output.png and test-differences.png in test/data/renderer subdirectories. Please attach the full directory (test/data/renderer)

[naoliv]

Attached.

[Don-vip]

And for the last two tests (MultiFetchServerObjectReaderTest and OsmServerBackreferenceReaderTest), they require defining a DEV_API login/password on the command line with -Dosm.username and -Dosm.password:

    <target name="test" depends="test-compile" unless="test.notRequired"
        description="Run unit and functional tests. OSM API (TEST) account shall be set with -Dosm.username and -Dosm.password">
        <call-junit testfamily="unit"/>
        <call-junit testfamily="functional"/>
    </target>

We should skip the tests if the credentials are not provided.

[Don-vip]

In 14190/josm:

see #16688 - skip some unit tests if preconditions are not met

[Don-vip]

@naoliv, can you please remove this option _JAVA_OPTIONS: -Dawt.useSystemAAFontSettings=on and tell me if MapCSSRendererTest is still failing?

[Don-vip]

In 14203/josm:

fix #15696, see #16688 - Don't use the localized decimal separator for coordinates

[naoliv]

I did unset _JAVA_OPTIONS, forced it to -Dawt.useSystemAAFontSettings=off, run the tests in another shell but they still fail.
I will need to see if it's getting the anti-aliasing setting from another place maybe.

[Don-vip]

I'm not sure it's linked to antialiasing, it was just an idea. Right now I don't understand why the test fails on your Debian machine while it works on our Ubuntu server. We should have almost the same openjdk, so no difference in rendering.

[naoliv]

I will run the same tests in a clean Ubuntu and Debian chroot and get back with the results later.

[Don-vip]

Did you re-run the tests?

[naoliv]

Algo got failures inside a clean Ubuntu 18.04 chroot with ant 1.10.3 and java 10.0.1:

Testsuite: org.openstreetmap.josm.gui.mappaint.MapCSSRendererTest
Tests run: 22, Failures: 21, Errors: 0, Skipped: 0, Time elapsed: 11.472 sec

And inside a Debian testing chroot with ant 1.10.5 and java 10.0.2:

Testsuite: org.openstreetmap.josm.gui.mappaint.MapCSSRendererTest
Tests run: 22, Failures: 21, Errors: 0, Skipped: 0, Time elapsed: 6.509 sec

[Don-vip]

Mmm I think we will then have the same problem after we fix #16231.

Closing this ticket as fixed, we'll investigate the last failure later.