tag_regex from fixAdd -> NullPointerException: Cannot invoke "org.openstreetmap.josm.data.osm.IPrimitive.getKeys()" because "env.osm" is null

[Famlam]

What steps will reproduce the problem?

1. Add the following mapcss rule to a custom mapcss rules file and load it in JOSM:

   *[/^(.+):surface$/] {
     throwWarning: tr("The tag is {0}", "{0.tag}");
     fixAdd: concat("test_element=", get(tag_regex("^(.+):surface$"), 0));
   }
   

(This rule is a simplified dummy rule. Any rule calling tag_regex in fixAdd will trigger the exception)

What is the expected result?

tag_regex can be used to obtain data in a fixAdd statement

What happens instead?

java.lang.NullPointerException: Cannot invoke "org.openstreetmap.josm.data.osm.IPrimitive.getKeys()" because "env.osm" is null in the call to tag_regex

Please provide any additional information below. Attach a screenshot if possible.

No need to load data. Starting JOSM is enough.

I see most other mapcss functions in Functions.java are wrapped in a block similar to this:

if (env.osm != null) {
  // Code calling env.osm.*
}
return null;

so I suspect the same should happen here?

Revision:18822
Build-Date:2023-08-30 11:44:32

Identification: JOSM/1.5 (18822 nl) Windows 10 64-Bit
OS Build number: Windows 10 Home 2009 (19045)
Memory Usage: 287 MB / 2012 MB (94 MB allocated, but free)
Java version: 17.0.8+7-LTS, Azul Systems, Inc., OpenJDK 64-Bit Server VM
Look and Feel: com.sun.java.swing.plaf.windows.WindowsLookAndFeel
Screen: \Display0 1920×1080 (scaling 1.10×1.10)
Maximum Screen Size: 1920×1080
Best cursor sizes: 16×16→32×32, 32×32→32×32
System property file.encoding: Cp1252
System property sun.jnu.encoding: Cp1252
Locale info: nl_NL
Numbers with default locale: 1234567890 -> 1234567890
VM arguments: [-Djpackage.app-version=1.5.18822, --add-modules=java.scripting,java.sql,javafx.controls,javafx.media,javafx.swing,javafx.web, --add-exports=java.base/sun.security.action=ALL-UNNAMED, --add-exports=java.desktop/com.sun.imageio.plugins.jpeg=ALL-UNNAMED, --add-exports=java.desktop/com.sun.imageio.spi=ALL-UNNAMED, --add-opens=java.base/java.lang=ALL-UNNAMED, --add-opens=java.base/java.nio=ALL-UNNAMED, --add-opens=java.base/jdk.internal.loader=ALL-UNNAMED, --add-opens=java.base/jdk.internal.ref=ALL-UNNAMED, --add-opens=java.desktop/javax.imageio.spi=ALL-UNNAMED, --add-opens=java.desktop/javax.swing.text.html=ALL-UNNAMED, --add-opens=java.prefs/java.util.prefs=ALL-UNNAMED, -Djpackage.app-path=%UserProfile%\AppData\Local\JOSM\JOSM.exe]

Plugins:
+ OpeningHoursEditor (36126)
+ imagery_offset_db (36126)
+ pbf (36126)
+ pt_assistant (2.1.15-19-g9aeec3c-SNAPSHOT)
+ reverter (36126)
+ tageditor (36126)
+ turnlanes-tagging (0.0.5)
+ undelete (36126)
+ utilsplugin2 (36134)

Map paint styles:
- https://josm.openstreetmap.de/josmfile?page=Styles/Lane_and_Road_Attributes&zip=1
- https://josm.openstreetmap.de/josmfile?page=Styles/PublicTransport&zip=1
+ %UserProfile%\Documents\tijdelijke bestanden\josm-eigen.mappaint.mapcss
+ https://josm.openstreetmap.de/josmfile?page=Styles/Sidewalks&zip=1
- https://josm.openstreetmap.de/josmfile?page=Styles/ParkingLanes&zip=1

Validator rules:
+ %UserProfile%\Documents\tijdelijke bestanden\josm-eigen.validator.mapcss
+ https://josm.openstreetmap.de/josmfile?page=Rules/SuspiciousSwimming_Pool&zip=1
+ https://raw.githubusercontent.com/Famlam/OsmMapcssValidationNL/main/netherlands.validator.mapcss
+ https://raw.githubusercontent.com/osm-fr/osmose-backend/master/plugins/TagFix_Destination.validator.mapcss
+ https://raw.githubusercontent.com/osm-fr/osmose-backend/master/plugins/Colour.validator.mapcss
+ https://raw.githubusercontent.com/osm-fr/osmose-backend/master/plugins/notprefix.validator.mapcss
+ https://raw.githubusercontent.com/osm-fr/osmose-backend/master/plugins/TagFix_MultipleTag2.validator.mapcss

Last errors/warnings:
- 00000.609 W: extended font config - overriding 'filename.Myanmar_Text=mmrtext.ttf' with 'MMRTEXT.TTF'
- 00000.611 W: extended font config - overriding 'filename.Mongolian_Baiti=monbaiti.ttf' with 'MONBAITI.TTF'
- 00001.199 E: java.security.KeyStoreException: Windows-ROOT not found. Oorzaak: java.security.NoSuchAlgorithmException: Windows-ROOT KeyStore not available
- 00027.834 E: Handled by bug report queue: java.lang.NullPointerException: Cannot invoke "org.openstreetmap.josm.data.osm.IPrimitive.getKeys()" because "env.osm" is null



=== REPORTED CRASH DATA ===
BugReportExceptionHandler#handleException:
No data collected.

Warning issued by: BugReportExceptionHandler#handleException

=== STACK TRACE ===
Thread: File Watcher (27)
java.lang.NullPointerException: Cannot invoke "org.openstreetmap.josm.data.osm.IPrimitive.getKeys()" because "env.osm" is null
	at org.openstreetmap.josm.gui.mappaint.mapcss.Functions.tag_regex(Functions.java:411)
	at org.openstreetmap.josm.gui.mappaint.mapcss.Functions.tag_regex(Functions.java:394)
	at org.openstreetmap.josm.gui.mappaint.mapcss.ExpressionFactory$Factory.lambda$ofEnv$22(ExpressionFactory.java:137)
	at org.openstreetmap.josm.gui.mappaint.mapcss.ExpressionFactory$Factory.lambda$of$13(ExpressionFactory.java:94)
	at org.openstreetmap.josm.gui.mappaint.mapcss.ExpressionFactory$Factory.lambda$ofObjectVarargs$7(ExpressionFactory.java:81)
	at java.base/java.util.stream.ReferencePipeline$3$1.accept(Unknown Source)
	at java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(Unknown Source)
	at java.base/java.util.stream.AbstractPipeline.copyInto(Unknown Source)
	at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(Unknown Source)
	at java.base/java.util.stream.AbstractPipeline.evaluate(Unknown Source)
	at java.base/java.util.stream.AbstractPipeline.evaluateToArrayNode(Unknown Source)
	at java.base/java.util.stream.ReferencePipeline.toArray(Unknown Source)
	at org.openstreetmap.josm.gui.mappaint.mapcss.ExpressionFactory$Factory.lambda$ofObjectVarargs$9(ExpressionFactory.java:82)
	at org.openstreetmap.josm.data.validation.tests.MapCSSTagCheckerRule.ofMapCSSRule(MapCSSTagCheckerRule.java:119)
	at org.openstreetmap.josm.data.validation.tests.MapCSSTagCheckerRule.readMapCSS(MapCSSTagCheckerRule.java:199)
	at org.openstreetmap.josm.data.validation.tests.MapCSSTagChecker.addMapCSS(MapCSSTagChecker.java:291)
	at org.openstreetmap.josm.data.validation.tests.MapCSSTagChecker.addMapCSS(MapCSSTagChecker.java:268)
	at org.openstreetmap.josm.data.validation.tests.MapCSSTagChecker.reloadRule(MapCSSTagChecker.java:359)
	at org.openstreetmap.josm.io.FileWatcher.processEvents(FileWatcher.java:160)
	at java.base/java.lang.Thread.run(Unknown Source)

[taylor.smock]

It looks like there are a total of 22 functions where this might occur. I'll have to investigate this.

[Famlam]

It looks like there are a total of 22 functions where this might occur.

Good call, I checked the other functions (those that call env.* in Functions.java)

I can verify that the exact same issue (NPE with Cannot X because "env.osm" is null) occurs with these functions:

• tag_regex
• has_tag_key
• osm_id
• osm_user_name
• osm_user_id
• osm_version
• osm_changeset_id
• osm_timestamp
• number_of_tags
• center

---

A very similar issue with the same cause occurs for this function:

• JOSM_search

  java.lang.NullPointerException: Cannot invoke "org.openstreetmap.josm.data.osm.Tagged.hasKeys()" because "osm" is null
  	at org.openstreetmap.josm.data.osm.search.SearchCompiler$Any.match(SearchCompiler.java:1074)
  	at org.openstreetmap.josm.gui.mappaint.mapcss.Functions.JOSM_search(Functions.java:774)
  

---

And a slightly different error with a similar cause ("env.source" is null) for this function:

• setting

[taylor.smock]

I was lazy and wrote a quick non-regression test to find all of the env.osm ones. I'll probably do the same for env.source.

EDIT: I'll probably just do it for all Environment fields, since it seems like it might be a problem.

[taylor.smock]

In 18875/josm:

Fix #23238: Some MapCSS functions cause NPEs when part of fixAdd

[Famlam]

Thank you for the very quick fix, great! I'll try it in the coming days when the dev version is build.

I think there is one NullPointerException-candidate left in Functions.java: JOSM_search, which is not found by the regression tests as it'll only cause an error if the string argument != ""

A test case that pops up a NPE with JOSM_search:

*[JOSM_search("test")] {
  throwWarning: "x";
  suggestAlternative: JOSM_search("test");
}

[Famlam]

I can confirm the issue has been resolved with r18875, except for function JOSM_search (see previous comment)

[taylor.smock]

In 18876/josm:

See #23238: Some MapCSS functions cause NPEs when part of fixAdd

JOSM_Search doesn't trigger the issue unless the search string is non-empty.

[taylor.smock]

Ticket retargeted after milestone deleted