Modify

Opened 4 months ago

Last modified 4 days ago

#13999 reopened enhancement

New MapCSS test that uses reference images

Reported by: michael2402 Owned by: team
Priority: normal Milestone: 17.03
Component: Unit tests Version:
Keywords: mapcss Cc: Klumbumbus

Description

When debugging MapCSS is often found the need to create custom styles to track down issues.

This is quite annoying and I thought that it could be automated.

I wrote a test that uses sample OSM files and styles and renders them to small (256 x 256 px) images. Those images are then compared to a reference image. If the images differ, the rendered result is stored for the user to inspect.

This works as a good unit test for the renderer classes - we can detect changes and see the impact of changes visually. If we change the renderer in a good way we need to re-create the reference files.

If the team is fine with it, I would like to write some more tests and commit it.

While I was on it, I found the first potential bug (ws.lowerIndex < offset in StyledMapRenderer does not make sense)

Attachments (3)

mapcss-reference-test.zip (11.7 KB) - added by michael2402 4 months ago.
test-differences.png (1.1 KB) - added by michael2402 3 weeks ago.
test-output.png (2.7 KB) - added by michael2402 3 weeks ago.

Download all attachments as: .zip

Change History (19)

Changed 4 months ago by michael2402

Attachment: mapcss-reference-test.zip added

comment:1 Changed 4 months ago by Klumbumbus

Cc: Klumbumbus added

comment:2 Changed 4 months ago by bastiK

+1

comment:3 Changed 4 months ago by Don-vip

Milestone: 16.1116.12

Milestone renamed

comment:4 Changed 3 months ago by Don-vip

Milestone: 16.1217.01

comment:5 Changed 3 months ago by Don-vip

Resolution: fixed
Status: newclosed

In 11433/josm:

fix #13999 - New MapCSS test that uses reference images (patch by michael2402)

comment:6 Changed 3 months ago by Don-vip

Keywords: mapcss added

comment:7 Changed 3 months ago by Don-vip

The tests fail on Jenkins.

comment:8 Changed 2 months ago by Don-vip

In 11449/josm:

see #13999 - skip unit tests (do not work on Jenkins)

comment:9 Changed 2 months ago by Don-vip

Resolution: fixed
Status: closedreopened
Summary: [Patch] New MapCSS test that uses reference imagesNew MapCSS test that uses reference images

comment:10 Changed 2 months ago by Don-vip

Milestone: 17.0117.02

comment:11 Changed 4 weeks ago by Don-vip

Milestone: 17.0217.03

comment:12 Changed 3 weeks ago by michael2402

I traced this a bit more.

The problem is with the way OpenJDK and Sun handle the corner cases when drawing. There are differences in their implementations when handling sub-pixels.

Although you don't see that much of a visual difference, there are many small differences, especially in the anti-aliasing region.


So we are left with two options:

  • Do not include any rendering tests
  • Use a pure java implementation (the best one I found is PJA toolkit, last version is of 2004)
  • Make the tests depend on the current JVM. We can add reference images for Sun and OpenJDK. Or we only add OpenJDK reference images, since this is used on our Jenkins server.

The main goal of those tests for me would be to quickly see if something was broken while working on the MapCSS code. So the last option would be fine with me. I'd have to test if there are many changes in the different OpenJDK versions, but I did not find any so far.

Last edited 3 weeks ago by michael2402 (previous) (diff)

Changed 3 weeks ago by michael2402

Attachment: test-differences.png added

Changed 3 weeks ago by michael2402

Attachment: test-output.png added

comment:13 Changed 3 weeks ago by Don-vip

ok for option 3 :) did you find the class which differs between Oracle and OpenJDK ?

comment:14 Changed 3 weeks ago by michael2402

In 11691/josm:

See #13999: Use MapCSS test only for OpenJDK

comment:15 Changed 3 weeks ago by michael2402

No, I did not dig into that code. There is a lot of native code.

And there seems to be an issue with text drawing, I'll have to trace that one... (See Jenkins)

comment:16 Changed 4 days ago by michael2402

In 11762/josm:

See #13999: Add reference test for area icons.

Modify Ticket

Change Properties
Set your email in Preferences
Action
as reopened The owner will remain team.
as The resolution will be set. Next status will be 'closed'.
to The owner will be changed from team to the specified user. Next status will be 'new'.
Next status will be 'needinfo'.The owner will change to michael2402
as duplicate The resolution will be set to duplicate. Next status will be 'closed'.The specified ticket will be cross-referenced with this ticket

Add Comment


E-mail address and name can be saved in the Preferences.

 
Note: See TracTickets for help on using tickets.