Modify

Opened 7 months ago

Last modified 6 weeks ago

#21121 new defect

15 Mapillary unit tests are failing

Reported by: Don-vip Owned by: taylor.smock
Priority: normal Milestone:
Component: Plugin mapillary Version:
Keywords: Cc:

Description

See https://josm.openstreetmap.de/jenkins/job/JOSM-Plugins/jdk=JDK11/2052/testReport/

 org.openstreetmap.josm.plugins.mapillary.cache.MapillaryCacheTest.test()	0.67 s	2
 org.openstreetmap.josm.plugins.mapillary.model.ImageDetectionTest.testGetDetectionsWait(long, int)[8]	0.38 s	2
 org.openstreetmap.josm.plugins.mapillary.model.ImageDetectionTest.testGetDetectionsWait(long, int)[7]	0.46 s	2
 org.openstreetmap.josm.plugins.mapillary.model.ImageDetectionTest.testGetDetectionsWait(long, int)[6]	0.38 s	2
 org.openstreetmap.josm.plugins.mapillary.model.ImageDetectionTest.testGetDetectionsWait(long, int)[5]	0.31 s	2
 org.openstreetmap.josm.plugins.mapillary.model.ImageDetectionTest.testGetDetectionsWait(long, int)[4]	0.37 s	2
 org.openstreetmap.josm.plugins.mapillary.model.ImageDetectionTest.testGetDetectionsWait(long, int)[3]	0.44 s	2
 org.openstreetmap.josm.plugins.mapillary.model.ImageDetectionTest.testGetDetectionsWait(long, int)[2]	0.38 s	2
 org.openstreetmap.josm.plugins.mapillary.model.ImageDetectionTest.testGetDetectionsWait(long, int)[1]	0.34 s	2
 org.openstreetmap.josm.plugins.mapillary.utils.api.JsonImageDetailsDecoderTest.testDecodeImageInfos(boolean)[1]	0.41 s	2
 org.openstreetmap.josm.plugins.mapillary.utils.api.JsonImageDetailsDecoderTest.testDecodeImageInfosWithFetchErrorsApplicationRequestLimitReached()	0.41 s	2
 org.openstreetmap.josm.plugins.mapillary.utils.api.JsonImageDetailsDecoderTest.testDecodeImageInfos(boolean)[2]	0.42 s	2
 org.openstreetmap.josm.plugins.mapillary.utils.api.JsonMapObjectDecoderTest.testNoOverwriteFromInitialVector()	0.31 s	2
 org.openstreetmap.josm.plugins.mapillary.utils.api.JsonMapObjectDecoderTest.testDecodeMapObject()	0.32 s	2
 org.openstreetmap.josm.plugins.mapillary.utils.api.JsonSequencesDecoderTest.testDecodeSequence()

Attachments (0)

Change History (12)

comment:1 Changed 7 months ago by taylor.smock

I saw that. :(

And they are passing with Gradle/IDE ( https://gitlab.com/smocktaylor/Mapillary/-/jobs/1427528846 for the CI run ).

It looks like they are all having issues reading files from disk (I suspect that the ImageDetectionTest.testGetDetectionsWait is erroring out at ImageDetections#getDetections where it is getting the json file from disk).

EDIT: I've been able to reproduce locally using Java 11 and 15.
JAVA_HOME=$(/usr/libexec/java_home -v 11) ant test (on Mac).

Last edited 7 months ago by taylor.smock (previous) (diff)

comment:2 in reply to:  1 Changed 7 months ago by Don-vip

Replying to taylor.smock:

EDIT: I've been able to reproduce locally using Java 11 and 15.
JAVA_HOME=$(/usr/libexec/java_home -v 11) ant test (on Mac).

Yes, forgot to mention that the tests are OK with Java 8.

comment:3 Changed 6 months ago by taylor.smock

I think I found a workaround: ensure that my custom @MapillaryURLWireMock is run after preferences are initialized (I'm using the @BasicPreferences annotation in @MapillaryURLWireMock).

Hopefully fixed in f68d7ea423e0de8346619aab487579f67c732b80.

comment:4 in reply to:  3 Changed 6 months ago by taylor.smock

Replying to taylor.smock:

I think I found a workaround: ensure that my custom @MapillaryURLWireMock is run after preferences are initialized (I'm using the @BasicPreferences annotation in @MapillaryURLWireMock).

Hopefully fixed in f68d7ea423e0de8346619aab487579f67c732b80.

I'm kind of wondering if this was a Java bug. Do you happen to know what version is currently installed on the CI?

It failed under Java 11.0.8 (AdoptOpenJDK), but passed in Java 11.0.12 (Eclipse Temurin).

comment:5 Changed 6 months ago by Don-vip

I think it's displayed in the job output. We're not targeting a particular version but only "11" so we follow the update train.

comment:6 in reply to:  5 Changed 6 months ago by taylor.smock

Replying to Don-vip:

I think it's displayed in the job output. We're not targeting a particular version but only "11" so we follow the update train.

Build output wasn't helpful, but https://packages.ubuntu.com/bionic/openjdk-11-jdk tells me Java 11.0.11. Assuming CI was updated sometime after April.

EDIT: I installed Java 11.0.11 (AdoptOpenJDK) to check. It passed as well. I hate it when tests pass in some environments and fail in others. It makes debugging fun.

Last edited 6 months ago by taylor.smock (previous) (diff)

comment:7 Changed 6 months ago by skyper

At least on Debian, 11.0.11 was released only a few days ago.

comment:8 Changed 6 months ago by Don-vip

https://josm.openstreetmap.de/jenkins/job/JOSM-Plugins/jdk=JDK11/2061/testReport/ used:

java -version
openjdk version "11.0.11" 2021-04-20
OpenJDK Runtime Environment (build 11.0.11+9-Ubuntu-0ubuntu2.18.04)
OpenJDK 64-Bit Server VM (build 11.0.11+9-Ubuntu-0ubuntu2.18.04, mixed mode, sharing)

and still failed. Previous run used an older version, but I don't know which one.

comment:9 Changed 6 months ago by Don-vip

It was also using Ant 1.10.10 with following options:

clean
test
checkstyle
spotbugs
-Djavadoc.executable=/opt/jdk-11/bin/javadoc
-Djavadoc.link=https://docs.oracle.com/en/java/javase/11/docs/api/
-Dplugin.dist.dir=../dist
-Djosm=/var/lib/jenkins/jobs/JOSM/configurations/axis-jdk/${jdk}/lastSuccessful/archive/dist/josm-custom.jar
-Djosm.ivy=/var/lib/jenkins/jobs/JOSM/workspace/jdk/${jdk}/ivy.xml
-Djosm.ivysettings=/var/lib/jenkins/jobs/JOSM/workspace/jdk/${jdk}/ivysettings.xml
-Djosm.test.build.dir=/var/lib/jenkins/jobs/JOSM/workspace/jdk/${jdk}/test/build
-Dskip-josm -XX:+HeapDumpOnOutOfMemoryError

I just updated to Ant 1.10.11 and launched a new job:
https://josm.openstreetmap.de/jenkins/job/JOSM-Plugins/2062/

Last edited 6 months ago by Don-vip (previous) (diff)

comment:10 Changed 6 months ago by taylor.smock

I've got it replicated in a docker container, but I've got to attach a debugger somehow. And it doesn't like localhost :(

comment:11 Changed 6 weeks ago by taylor.smock

Ticket #21671 has been marked as a duplicate of this ticket.

comment:12 Changed 6 weeks ago by taylor.smock

OK. I've temporarily disabled the failing tests. I'm going to have to actually fix them, but this is just so we can remove the last log4j jar (build 2033), see comment:29:ticket:21657.

I'll re-enable the tests after Christmas and guess-and-check.

Modify Ticket

Change Properties
Set your email in Preferences
Action
as new The owner will remain taylor.smock.
as The resolution will be set.
to The owner will be changed from taylor.smock to the specified user.
The owner will change to Don-vip
as duplicate The resolution will be set to duplicate.The specified ticket will be cross-referenced with this ticket
The owner will be changed from taylor.smock to anonymous.

Add Comment


E-mail address and name can be saved in the Preferences.

 
Note: See TracTickets for help on using tickets.