Modify

Opened 9 months ago

Closed 8 months ago

Last modified 5 months ago

#14425 closed enhancement (fixed)

[patch] Improve Wikipedia tag validators

Reported by: nyurik Owned by: nyurik
Priority: normal Milestone: 17.03
Component: Core validator Version:
Keywords: , wikipedia Cc:

Description (last modified by nyurik)

Wikipedia tags need a few more validators with fixers:

  • update the list of valid Wikipedia languages
  • auto-replace language be-x-old with be-tarask
  • auto-replace language cz with cs
  • remove space before page title ("en: Foo" -> "en:Foo")
  • replace "_" with spaces ("en:Foo_bar" -> "en:Foo bar")
  • capitalize first letter if it's not "jbo" language ("en:foo" -> "en:Foo")
  • double language link ("en:en:Foo" -> "en:Foo")

This patch also adds a few additional validator functions: trim, upper and lower

Attachments (5)

wikipedia-validations.patch (8.3 KB) - added by nyurik 9 months ago.
patch to fix this issue
wikipedia-validations.2.patch (10.0 KB) - added by nyurik 9 months ago.
overrides previous patch
wikipedia-validations.3.patch (10.9 KB) - added by nyurik 9 months ago.
wikipedia-validations.4.patch (12.3 KB) - added by nyurik 9 months ago.
wikipedia-validations.5.patch (12.4 KB) - added by nyurik 9 months ago.

Download all attachments as: .zip

Change History (22)

Changed 9 months ago by nyurik

Attachment: wikipedia-validations.patch added

patch to fix this issue

comment:1 Changed 9 months ago by nyurik

Description: modified (diff)

comment:2 Changed 9 months ago by nyurik

Description: modified (diff)

comment:3 Changed 9 months ago by Don-vip

Component: CoreCore validator
Keywords: added; patch validator removed
Milestone: 17.03
Summary: (patch) Improve Wikipedia tag validators[patch] Improve Wikipedia tag validators

Changed 9 months ago by nyurik

overrides previous patch

comment:4 Changed 9 months ago by nyurik

Description: modified (diff)

Changed 9 months ago by nyurik

Changed 9 months ago by nyurik

comment:5 Changed 9 months ago by Don-vip

Please don't write this:

 	29	  throwWarning: tr("wikipedia ''be-x-old'' language is obsolete, use ''be-tarask'' instead");
 	36	  throwWarning: tr("wikipedia ''cz'' language is invalid, use ''cs'' instead");

This makes an entry to translate for each value we will list in an individual check. Instead, this only creates a generic string to translate:

 	29	  throwWarning: tr("wikipedia ''{0}'' language is obsolete, use ''{1}'' instead", "be-x-old", "be-tarask");
 	36	  throwWarning: tr("wikipedia ''{0}'' language is invalid, use ''{1}'' instead", "cz", "cs");

I didn't check the other strings, you see the point :)

Changed 9 months ago by nyurik

comment:6 Changed 9 months ago by nyurik

@Don-vip, please recheck. I so wish we used git for this. Attaching and revising patches like this is very arcane.

comment:7 Changed 8 months ago by nyurik

@don-vip, is something still missing for this ticket to be merged?

comment:8 Changed 8 months ago by Don-vip

Sorry I forgot this one I'll review it tonight :)

comment:9 Changed 8 months ago by Don-vip

Resolution: fixed
Status: newclosed

In 11756/josm:

fix #14425 - Improve Wikipedia tag validators (patch by nyurik, modified)

comment:10 Changed 8 months ago by Don-vip

Thanks for the patch! :)
However for the next time, please test it with JOSM first, I had to fix many syntax issues (wrong quote characters, missing semicolons, etc.)

comment:11 Changed 8 months ago by Klumbumbus

Resolution: fixed
Status: closedreopened

"wikipedia tag is set, but there is no wikidata tag. Wikipedia plugin might help with wikidata id lookups" does this really fit into the category warning? wikidata tag seems to me an optional tag.

comment:12 Changed 8 months ago by Don-vip

Resolution: fixed
Status: reopenedclosed

In 11771/josm:

fix #14425 - reduce wikidata warning to info level

comment:13 Changed 8 months ago by Klumbumbus

@nyurik: please add documentation for the new functions at wiki:Help/Styles/MapCSSImplementation

comment:14 Changed 8 months ago by Don-vip

Resolution: fixed
Status: closedreopened

I didn't notice that the assertions are not met:

GRAVE: Expecting test 'wikipedia tag has no language given, use 'wikipedia'='language:page title'' (i.e., [*[wikipedia][wikipedia'NREGEX'^[-a-z]{2,12}:]]) to not match node wikipedia=en-GB:Foobar (i.e., TagMap[wikipedia=en-GB:Foobar])
GRAVE: Expecting test 'wikipedia tag has an unknown language prefix' (i.e., [*[wikipedia'REGEX'^[-a-z]{2,12}:][wikipedia'NREGEX'^https?:\/\/][wikipedia'NREGEX'^(aa|ab|ace|ady|ady-cyrl|aeb|aeb-arab|aeb-latn|af|ak|aln|als|am|an|ang|anp|ar|arc|arn|arq|ary|arz|as|ase|ast|av|avk|awa|ay|az|azb|ba|ban|bar|bat-smg|bbc|bbc-latn|bcc|bcl|be|be-tarask|be-x-old|bg|bgn|bh|bho|bi|bjn|bm|bn|bo|bpy|bqi|br|brh|bs|bto|bug|bxr|ca|cbk-zam|cdo|ce|ceb|ch|cho|chr|chy|ckb|co|cps|cr|crh|crh-cyrl|crh-latn|cs|csb|cu|cv|cy|cz|da|de|de-at|de-ch|de-formal|din|diq|dsb|dtp|dty|dv|dz|ee|egl|el|eml|en|en-ca|en-gb|eo|es|et|eu|ext|fa|ff|fi|fit|fiu-vro|fj|fo|fr|frc|frp|frr|fur|fy|ga|gag|gan|gan-hans|gan-hant|gd|gl|glk|gn|gom|gom-deva|gom-latn|gor|got|grc|gsw|gu|gv|ha|hak|haw|he|hi|hif|hif-latn|hil|ho|hr|hrx|hsb|ht|hu|hy|hz|ia|id|ie|ig|ii|ik|ike-cans|ike-latn|ilo|inh|io|is|it|iu|ja|jam|jbo|jut|jv|ka|kaa|kab|kbd|kbd-cyrl|kea|kg|khw|ki|kiu|kj|kk|kk-arab|kk-cn|kk-cyrl|kk-kz|kk-latn|kk-tr|kl|km|kn|ko|ko-kp|koi|kr|krc|kri|krj|krl|ks|ks-arab|ks-deva|ksh|ku|ku-arab|ku-latn|kv|kw|ky|la|lad|lb|lbe|lez|lfn|lg|li|lij|liv|lki|lmo|ln|lo|loz|lrc|lt|ltg|lus|luz|lv|lzh|lzz|mai|map-bms|mdf|mg|mh|mhr|mi|min|mk|ml|mn|mo|mr|mrj|ms|mt|mus|mwl|my|myv|mzn|na|nah|nan|nap|nb|nds|nds-nl|ne|new|ng|niu|nl|nl-informal|nn|no|nod|nov|nrm|nso|nv|ny|nys|oc|olo|om|or|os|ota|pa|pag|pam|pap|pcd|pdc|pdt|pfl|pi|pih|pl|pms|pnb|pnt|prg|ps|pt|pt-br|qu|qug|rgn|rif|rm|rmy|rn|ro|roa-rup|roa-tara|ru|rue|rup|ruq|ruq-cyrl|ruq-latn|rw|rwr|sa|sah|sat|sc|scn|sco|sd|sdc|sdh|se|sei|ses|sg|sgs|sh|shi|shi-latn|shi-tfng|shn|si|simple|sje|sk|sl|sli|sm|sma|smj|sn|so|sq|sr|sr-ec|sr-el|srn|srq|ss|st|stq|su|sv|sw|szl|ta|tcy|te|tet|tg|tg-cyrl|tg-latn|th|ti|tk|tl|tly|tn|to|tokipona|tpi|tr|tru|ts|tt|tt-cyrl|tt-latn|tum|tw|ty|tyv|tzm|udm|ug|ug-arab|ug-latn|uk|ur|uz|uz-cyrl|uz-latn|ve|vec|vep|vi|vls|vmf|vo|vot|vro|wa|war|wo|wuu|xal|xh|xmf|yi|yo|yue|za|zea|zh|zh-classical|zh-cn|zh-hans|zh-hant|zh-hk|zh-min-nan|zh-mo|zh-my|zh-sg|zh-tw|zh-yue|zu):]]) to match node wikipedia=X-Y-Z:Foobar (i.e., TagMap[wikipedia=X-Y-Z:Foobar])
GRAVE: Expecting test 'wikipedia title should not have 'wiki/' prefix' (i.e., [*[wikipedia'REGEX'^[-a-z]{2,12}:wiki\/]]) to match node wikipedia=en: foo (i.e., TagMap[wikipedia=en: foo])

Can you please submit a patch that fix them? To check it you can simply run MapCSSTagCheckerTest.testInit() as JUnit test.

comment:15 Changed 8 months ago by Don-vip

Owner: changed from team to nyurik
Status: reopenedneedinfo

comment:16 Changed 8 months ago by bastiK

Resolution: fixed
Status: needinfoclosed

In 11800/josm:

fixed #14425 - fix wikipedia tag validator regexes

comment:17 Changed 5 months ago by Klumbumbus

In 12415/josm:

fix #14947, see #14425 - fix wikipedia warning message

Modify Ticket

Change Properties
Set your email in Preferences
Action
as closed The owner will remain nyurik.
as The resolution will be set.
The resolution will be deleted.

Add Comment


E-mail address and name can be saved in the Preferences.

 
Note: See TracTickets for help on using tickets.