15 Mapillary unit tests are failing

[Don-vip]

See https://josm.openstreetmap.de/jenkins/job/JOSM-Plugins/jdk=JDK11/2052/testReport/

 org.openstreetmap.josm.plugins.mapillary.cache.MapillaryCacheTest.test()	0.67 s	2
 org.openstreetmap.josm.plugins.mapillary.model.ImageDetectionTest.testGetDetectionsWait(long, int)[8]	0.38 s	2
 org.openstreetmap.josm.plugins.mapillary.model.ImageDetectionTest.testGetDetectionsWait(long, int)[7]	0.46 s	2
 org.openstreetmap.josm.plugins.mapillary.model.ImageDetectionTest.testGetDetectionsWait(long, int)[6]	0.38 s	2
 org.openstreetmap.josm.plugins.mapillary.model.ImageDetectionTest.testGetDetectionsWait(long, int)[5]	0.31 s	2
 org.openstreetmap.josm.plugins.mapillary.model.ImageDetectionTest.testGetDetectionsWait(long, int)[4]	0.37 s	2
 org.openstreetmap.josm.plugins.mapillary.model.ImageDetectionTest.testGetDetectionsWait(long, int)[3]	0.44 s	2
 org.openstreetmap.josm.plugins.mapillary.model.ImageDetectionTest.testGetDetectionsWait(long, int)[2]	0.38 s	2
 org.openstreetmap.josm.plugins.mapillary.model.ImageDetectionTest.testGetDetectionsWait(long, int)[1]	0.34 s	2
 org.openstreetmap.josm.plugins.mapillary.utils.api.JsonImageDetailsDecoderTest.testDecodeImageInfos(boolean)[1]	0.41 s	2
 org.openstreetmap.josm.plugins.mapillary.utils.api.JsonImageDetailsDecoderTest.testDecodeImageInfosWithFetchErrorsApplicationRequestLimitReached()	0.41 s	2
 org.openstreetmap.josm.plugins.mapillary.utils.api.JsonImageDetailsDecoderTest.testDecodeImageInfos(boolean)[2]	0.42 s	2
 org.openstreetmap.josm.plugins.mapillary.utils.api.JsonMapObjectDecoderTest.testNoOverwriteFromInitialVector()	0.31 s	2
 org.openstreetmap.josm.plugins.mapillary.utils.api.JsonMapObjectDecoderTest.testDecodeMapObject()	0.32 s	2
 org.openstreetmap.josm.plugins.mapillary.utils.api.JsonSequencesDecoderTest.testDecodeSequence()

[taylor.smock]

I saw that. :(

And they are passing with Gradle/IDE ( ​https://gitlab.com/smocktaylor/Mapillary/-/jobs/1427528846 for the CI run ).

It looks like they are all having issues reading files from disk (I suspect that the ImageDetectionTest.testGetDetectionsWait is erroring out at ImageDetections#getDetections where it is getting the json file from disk).

EDIT: I've been able to reproduce locally using Java 11 and 15.
JAVA_HOME=$(/usr/libexec/java_home -v 11) ant test (on Mac).

[Don-vip]

Replying to taylor.smock:

EDIT: I've been able to reproduce locally using Java 11 and 15.
JAVA_HOME=$(/usr/libexec/java_home -v 11) ant test (on Mac).

Yes, forgot to mention that the tests are OK with Java 8.

[taylor.smock]

I think I found a workaround: ensure that my custom @MapillaryURLWireMock is run after preferences are initialized (I'm using the @BasicPreferences annotation in @MapillaryURLWireMock).

Hopefully fixed in ​f68d7ea423e0de8346619aab487579f67c732b80.

[taylor.smock]

Replying to taylor.smock:

I think I found a workaround: ensure that my custom @MapillaryURLWireMock is run after preferences are initialized (I'm using the @BasicPreferences annotation in @MapillaryURLWireMock).

Hopefully fixed in ​f68d7ea423e0de8346619aab487579f67c732b80.

I'm kind of wondering if this was a Java bug. Do you happen to know what version is currently installed on the CI?

It failed under Java 11.0.8 (AdoptOpenJDK), but passed in Java 11.0.12 (Eclipse Temurin).

[Don-vip]

I think it's displayed in the job output. We're not targeting a particular version but only "11" so we follow the update train.

[taylor.smock]

Replying to Don-vip:

I think it's displayed in the job output. We're not targeting a particular version but only "11" so we follow the update train.

Build output wasn't helpful, but ​https://packages.ubuntu.com/bionic/openjdk-11-jdk tells me Java 11.0.11. Assuming CI was updated sometime after April.

EDIT: I installed Java 11.0.11 (AdoptOpenJDK) to check. It passed as well. I hate it when tests pass in some environments and fail in others. It makes debugging fun.

[skyper]

At least on Debian, 11.0.11 was released only a few days ago.

[Don-vip]

https://josm.openstreetmap.de/jenkins/job/JOSM-Plugins/jdk=JDK11/2061/testReport/ used:

java -version
openjdk version "11.0.11" 2021-04-20
OpenJDK Runtime Environment (build 11.0.11+9-Ubuntu-0ubuntu2.18.04)
OpenJDK 64-Bit Server VM (build 11.0.11+9-Ubuntu-0ubuntu2.18.04, mixed mode, sharing)

and still failed. Previous run used an older version, but I don't know which one.

[Don-vip]

It was also using Ant 1.10.10 with following options:

clean
test
checkstyle
spotbugs
-Djavadoc.executable=/opt/jdk-11/bin/javadoc
-Djavadoc.link=https://docs.oracle.com/en/java/javase/11/docs/api/
-Dplugin.dist.dir=../dist
-Djosm=/var/lib/jenkins/jobs/JOSM/configurations/axis-jdk/${jdk}/lastSuccessful/archive/dist/josm-custom.jar
-Djosm.ivy=/var/lib/jenkins/jobs/JOSM/workspace/jdk/${jdk}/ivy.xml
-Djosm.ivysettings=/var/lib/jenkins/jobs/JOSM/workspace/jdk/${jdk}/ivysettings.xml
-Djosm.test.build.dir=/var/lib/jenkins/jobs/JOSM/workspace/jdk/${jdk}/test/build

-Dskip-josm -XX:+HeapDumpOnOutOfMemoryError

I just updated to Ant 1.10.11 and launched a new job:
https://josm.openstreetmap.de/jenkins/job/JOSM-Plugins/2062/

[taylor.smock]

I've got it replicated in a docker container, but I've got to attach a debugger somehow. And it doesn't like localhost :(

[taylor.smock]

Ticket #21671 has been marked as a duplicate of this ticket.

[taylor.smock]

OK. I've temporarily disabled the failing tests. I'm going to have to actually fix them, but this is just so we can remove the last log4j jar (build 2033), see comment:29:ticket:21657.

I'll re-enable the tests after Christmas and guess-and-check.

[taylor.smock]

OK. I started looking into this again. I added some debugging statements to figure out what was going on.

[junitlauncher] 2022-03-31 13:24:46.659 SEVERE: <html>
[junitlauncher] <head>
[junitlauncher] <meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/>
[junitlauncher] <title>Error 500 java.io.FileNotFoundException: /var/lib/jenkins/jobs/Mapillary/workspace/jdk/JDK11/test/resources/__files/api/v4/responses/graph/311681117131457/detections.json (No such file or directory)</title>
[junitlauncher] </head>
[junitlauncher] <body><h2>HTTP ERROR 500 java.io.FileNotFoundException: /var/lib/jenkins/jobs/Mapillary/workspace/jdk/JDK11/test/resources/__files/api/v4/responses/graph/311681117131457/detections.json (No such file or directory)</h2>
[junitlauncher] <table>
[junitlauncher] <tr><th>URI:</th><td>/api/v4/graph/311681117131457/detections</td></tr>
[junitlauncher] <tr><th>STATUS:</th><td>500</td></tr>
[junitlauncher] <tr><th>MESSAGE:</th><td>java.io.FileNotFoundException: /var/lib/jenkins/jobs/Mapillary/workspace/jdk/JDK11/test/resources/__files/api/v4/responses/graph/311681117131457/detections.json (No such file or directory)</td></tr>
[junitlauncher] <tr><th>SERVLET:</th><td>com.github.tomakehurst.wiremock.servlet.WireMockHandlerDispatchingServlet-265fae0a</td></tr>
[junitlauncher] <tr><th>CAUSED BY:</th><td>java.io.FileNotFoundException: /var/lib/jenkins/jobs/Mapillary/workspace/jdk/JDK11/test/resources/__files/api/v4/responses/graph/311681117131457/detections.json (No such file or directory)</td></tr>
[junitlauncher] </table>
[junitlauncher] <h3>Caused by:</h3><pre>java.io.FileNotFoundException: /var/lib/jenkins/jobs/Mapillary/workspace/jdk/JDK11/test/resources/__files/api/v4/responses/graph/311681117131457/detections.json (No such file or directory)
[junitlauncher] 	at java.base/java.io.FileInputStream.open0(Native Method)
[junitlauncher] 	at java.base/java.io.FileInputStream.open(FileInputStream.java:219)
[junitlauncher] 	at java.base/java.io.FileInputStream.&lt;init&gt;(FileInputStream.java:157)
[junitlauncher] 	at java.base/java.io.FileInputStream.&lt;init&gt;(FileInputStream.java:112)
[junitlauncher] 	at java.base/sun.net.www.protocol.file.FileURLConnection.connect(FileURLConnection.java:86)
[junitlauncher] 	at java.base/sun.net.www.protocol.file.FileURLConnection.getInputStream(FileURLConnection.java:184)
[junitlauncher] 	at java.base/java.net.URL.openStream(URL.java:1165)
[junitlauncher] 	at com.github.tomakehurst.wiremock.common.BinaryFile.getStream(BinaryFile.java:57)
[junitlauncher] 	at com.github.tomakehurst.wiremock.common.BinaryFile.readContents(BinaryFile.java:34)
[junitlauncher] 	at com.github.tomakehurst.wiremock.common.TextFile.readContentsAsString(TextFile.java:30)
[junitlauncher] 	at com.github.tomakehurst.wiremock.extension.responsetemplating.ResponseTemplateTransformer.transform(ResponseTemplateTransformer.java:125)
[junitlauncher] 	at
[...snip...]
[junitlauncher] </pre>
[junitlauncher] 
[junitlauncher] </body>
[junitlauncher] </html>

So it looks like I need to figure out why the wiremock server is using the parent directory as the root directory for resources. This may end up being a patch for the @BasicWiremock annotation in JOSM core, but I don't know yet.

It also looks like I should drop Mapillary from the javafx targets (once I do that, the tests will probably start failing on Java 8 -- JavaFX targets aren't built on Java 8, but I won't do that until the tests are fixed for JDK11).

EDIT: Patch to reproduce locally (without Jenkins, and without building all the plugins)

plugins/build.xml

@@ -23 +23 @@
     <property name="javafx_plugins" value="javafx/build.xml
                                                                                         Mapillary/build.xml
                                                                                         MicrosoftStreetside/build.xml"/>
+    <property name="failing_plugins" value="Mapillary-git/build.xml"/>
     <macrodef name="iterate">
         <attribute name="target"/>
         <sequential>
             <subant target="@{target}" inheritall="true">
-                <filelist dir="." files="${ordered_plugins}"/>
+                <!--<filelist dir="." files="${ordered_plugins}"/>-->
                 <!-- Build JavaFX plugins only with Java 11+ -->
+                <!--
                 <filelist dir="." files="${javafx_plugins}" if:set="isJava11"/>
                 <fileset  dir="." includes="*/build.xml" excludes="00_*/build.xml *.wip/build.xml ${javafx_plugins} ${ordered_plugins}"/>
+                -->
+                <filelist dir="." files="${failing_plugins}"/>
             </subant>
         </sequential>
     </macrodef>

[taylor.smock]

It looks like everything is working now (or at least there were no test failures -- the Mapillary job did fail, but only after it started building MicrosoftStreetside (/var/lib/jenkins/jobs/Mapillary/workspace/jdk/JDK11/MicrosoftStreetside/build)).

https://josm.openstreetmap.de/jenkins/job/Mapillary/lastCompletedBuild/testReport/