New MapCSS test that uses reference images

[michael2402]

When debugging MapCSS is often found the need to create custom styles to track down issues.

This is quite annoying and I thought that it could be automated.

I wrote a test that uses sample OSM files and styles and renders them to small (256 x 256 px) images. Those images are then compared to a reference image. If the images differ, the rendered result is stored for the user to inspect.

This works as a good unit test for the renderer classes - we can detect changes and see the impact of changes visually. If we change the renderer in a good way we need to re-create the reference files.

If the team is fine with it, I would like to write some more tests and commit it.

While I was on it, I found the first potential bug (ws.lowerIndex < offset in StyledMapRenderer does not make sense)

[bastiK]

+1

[Don-vip]

Milestone renamed

[Don-vip]

In 11433/josm:

fix #13999 - New MapCSS test that uses reference images (patch by michael2402)

[Don-vip]

The tests fail on Jenkins.

[Don-vip]

In 11449/josm:

see #13999 - skip unit tests (do not work on Jenkins)

[michael2402]

I traced this a bit more.

The problem is with the way OpenJDK and Sun handle the corner cases when drawing. There are differences in their implementations when handling sub-pixels.

Although you don't see that much of a visual difference, there are many small differences, especially in the anti-aliasing region.

So we are left with two options:

• Do not include any rendering tests
• Use a pure java implementation (the best one I found is PJA toolkit, last version is of 2004)
• Make the tests depend on the current JVM. We can add reference images for Sun and OpenJDK. Or we only add OpenJDK reference images, since this is used on our Jenkins server.

The main goal of those tests for me would be to quickly see if something was broken while working on the MapCSS code. So the last option would be fine with me. I'd have to test if there are many changes in the different OpenJDK versions, but I did not find any so far.

[Don-vip]

ok for option 3 :) did you find the class which differs between Oracle and OpenJDK ?

[michael2402]

In 11691/josm:

See #13999: Use MapCSS test only for OpenJDK

[michael2402]

No, I did not dig into that code. There is a lot of native code.

And there seems to be an issue with text drawing, I'll have to trace that one... (See Jenkins)

[michael2402]

In 11762/josm:

See #13999: Add reference test for area icons.

[michael2402]

It will take some more time to get full (or at least most) coverage.

[bastiK]

I quite like this test, maybe I'll add some too.

[michael2402]

Feel free to add as many as you want.

If you search it: I moved it to functional tests, since it is not really a unit test.

In addition to rendering, it can be used to test MapCSS selectors and other MapCSS features.

[Don-vip]

This ticket was only open because the tests were failing. Thanks for fixing them. A new ticket can be created if needed for new tests.