Modify

Opened 7 years ago

Closed 7 years ago

#13999 closed enhancement (fixed)

New MapCSS test that uses reference images

Reported by: michael2402 Owned by: team
Priority: normal Milestone: 17.04
Component: Unit tests Version:
Keywords: mapcss Cc: Klumbumbus

Description

When debugging MapCSS is often found the need to create custom styles to track down issues.

This is quite annoying and I thought that it could be automated.

I wrote a test that uses sample OSM files and styles and renders them to small (256 x 256 px) images. Those images are then compared to a reference image. If the images differ, the rendered result is stored for the user to inspect.

This works as a good unit test for the renderer classes - we can detect changes and see the impact of changes visually. If we change the renderer in a good way we need to re-create the reference files.

If the team is fine with it, I would like to write some more tests and commit it.

While I was on it, I found the first potential bug (ws.lowerIndex < offset in StyledMapRenderer does not make sense)

Attachments (3)

mapcss-reference-test.zip (11.7 KB ) - added by michael2402 7 years ago.
test-differences.png (1.1 KB ) - added by michael2402 7 years ago.
test-output.png (2.7 KB ) - added by michael2402 7 years ago.

Download all attachments as: .zip

Change History (23)

by michael2402, 7 years ago

Attachment: mapcss-reference-test.zip added

comment:1 by Klumbumbus, 7 years ago

Cc: Klumbumbus added

comment:2 by bastiK, 7 years ago

+1

comment:3 by Don-vip, 7 years ago

Milestone: 16.1116.12

Milestone renamed

comment:4 by Don-vip, 7 years ago

Milestone: 16.1217.01

comment:5 by Don-vip, 7 years ago

Resolution: fixed
Status: newclosed

In 11433/josm:

fix #13999 - New MapCSS test that uses reference images (patch by michael2402)

comment:6 by Don-vip, 7 years ago

Keywords: mapcss added

comment:7 by Don-vip, 7 years ago

The tests fail on Jenkins.

comment:8 by Don-vip, 7 years ago

In 11449/josm:

see #13999 - skip unit tests (do not work on Jenkins)

comment:9 by Don-vip, 7 years ago

Resolution: fixed
Status: closedreopened
Summary: [Patch] New MapCSS test that uses reference imagesNew MapCSS test that uses reference images

comment:10 by Don-vip, 7 years ago

Milestone: 17.0117.02

comment:11 by Don-vip, 7 years ago

Milestone: 17.0217.03

comment:12 by michael2402, 7 years ago

I traced this a bit more.

The problem is with the way OpenJDK and Sun handle the corner cases when drawing. There are differences in their implementations when handling sub-pixels.

Although you don't see that much of a visual difference, there are many small differences, especially in the anti-aliasing region.


So we are left with two options:

  • Do not include any rendering tests
  • Use a pure java implementation (the best one I found is PJA toolkit, last version is of 2004)
  • Make the tests depend on the current JVM. We can add reference images for Sun and OpenJDK. Or we only add OpenJDK reference images, since this is used on our Jenkins server.

The main goal of those tests for me would be to quickly see if something was broken while working on the MapCSS code. So the last option would be fine with me. I'd have to test if there are many changes in the different OpenJDK versions, but I did not find any so far.

Last edited 7 years ago by michael2402 (previous) (diff)

by michael2402, 7 years ago

Attachment: test-differences.png added

by michael2402, 7 years ago

Attachment: test-output.png added

comment:13 by Don-vip, 7 years ago

ok for option 3 :) did you find the class which differs between Oracle and OpenJDK ?

comment:14 by michael2402, 7 years ago

In 11691/josm:

See #13999: Use MapCSS test only for OpenJDK

comment:15 by michael2402, 7 years ago

No, I did not dig into that code. There is a lot of native code.

And there seems to be an issue with text drawing, I'll have to trace that one... (See Jenkins)

comment:16 by michael2402, 7 years ago

In 11762/josm:

See #13999: Add reference test for area icons.

comment:17 by michael2402, 7 years ago

Milestone: 17.0317.04

It will take some more time to get full (or at least most) coverage.

comment:18 by bastiK, 7 years ago

I quite like this test, maybe I'll add some too.

comment:19 by michael2402, 7 years ago

Feel free to add as many as you want.

If you search it: I moved it to functional tests, since it is not really a unit test.

In addition to rendering, it can be used to test MapCSS selectors and other MapCSS features.

comment:20 by Don-vip, 7 years ago

Resolution: fixed
Status: reopenedclosed

This ticket was only open because the tests were failing. Thanks for fixing them. A new ticket can be created if needed for new tests.

Modify Ticket

Change Properties
Set your email in Preferences
Action
as closed The owner will remain team.
as The resolution will be set.
The resolution will be deleted. Next status will be 'reopened'.

Add Comment


E-mail address and name can be saved in the Preferences .
 
Note: See TracTickets for help on using tickets.