Syntax Highlighting Checker

The KTextEditor Framework uses the syntax highlighting files provided by the KSyntaxHighlighting Framework since the  KDE Frameworks release 5.28.

The KSyntaxHighlighting Framework implements Kate’s highlighting system and meanwhile is used in quite some applications (e.g. LabPlot, KDE PIM). What is quite nice is that the KSyntaxHighlighting framework is nicely unit tested. And while we do not have tests for all highlighting files, we still provide some quality assurance through a compile time checker.

How does it work? Well – in former times, Kate loaded all highlighting .xml files from disk (through the KTextEditor framework). This lead to a slow startup over time, since there are >250 .xml files that needed a stat system call at startup.

With the KSyntaxHighlighting Framework, all these xml files are compiled into a Qt resource (qrc file), that then is included into the KSyntaxHighlighting library.

In order to create the Qt resource file, we need to iterate over all available xml files anyways. So what happens is that we take this opportunity and also scan the highlighting files for common mistakes.

As of today, we are checking the following:

  1. RegExpr: A warning is raised, if a regular expression has syntax errors.
  2. DetectChars: A warning is raised, if the char=”x” attribute contains more or less than one character, e.g. when char=”xyz”, or char=”\\” (no escaping required), or similar.
  3. Detect2Chars: Same as DetectChars, just for char=”x” and char1=”y”.
  4. Keyword lists: A warning is raised, if a keyword entry contains leading or trailing spaces. Additional trimming just takes time.
  5. Keyword lists: A warning is raised if a keyword list is unused.
  6. Keyword lists: A warning is raised if multiple keyword lists use the same me (=identifier).
  7. Keyword lists: A warning is raised if a non-existing keyword list is used.
  8. Contexts: A warning is raised, if a non-existing context is referenced.
  9. Contexts: A warning is raised, if a context is unused.
  10. Contexts: A warning is raised, if multiple contexts have the same name (identifier clash).
  11. Attributes: A warning is raised, if non-existing itemData is used.
  12. Attributes: A warning is raised, if multiple itemDatas use the same name (identifier clash).
  13. Attributes: A warning is raised, if an itemData is unused.

This list helps us nicely to catch many mistakes at compile time even before running unit tests.

However, there is still work to do, since some of the files still contain one or more of the above issues. The current todo-list looks as follows (updated on 2017-12-04):

Reference of non-existing keyword list:
- "asp.xml": QSet("Others")
- "template-toolkit.xml": QSet("Others")

Duplicate contexts:
- "diff.xml": "File"
- "objectivecpp.xml": "Preprocessor"

Reference of non-existing itemData attributes:
- "haml.xml": QSet("Array", "Escaped Text", "Ruby embedded in haml")
- "lilypond.xml": QSet("Tremolo")
- "metafont.xml": QSet("Verbatim", "Tex", "Bullet")
- "relaxng.xml": QSet("Entity Reference")
- "rhtml.xml": QSet("RUBY RAILS ERB Text")
- "rmarkdown.xml": QSet("Markdown", "Document Headers")
- "stata.xml": QSet("String2")

Incorrect DetectChar and Detect2Chars:
- "xmldebug.xml" line 130 'char' must contain exactly one char: "\"(\\s+|$)"
- "xmldebug.xml" line 135 'char' must contain exactly one char: "'(\\s+|$)"
- "xmldebug.xml" line 373 'char' must contain exactly one char: "\"(\\s+|$)"
- "xmldebug.xml" line 378 'char' must contain exactly one char: "'(\\s+|$)"
- "xmldebug.xml" line 415 'char' must contain exactly one char: "\"(\\s+|$)"
- "xmldebug.xml" line 420 'char' must contain exactly one char: "'(\\s+|$)"
- "xmldebug.xml" line 467 'char' must contain exactly one char: "\"(\\s+|$)"
- "xmldebug.xml" line 472 'char' must contain exactly one char: "'(\\s+|$)"

Unused contexts:
- "ample.xml": QSet("Preprocessor", "Define")
- "gcc.xml": QSet("GNUMacros")
- "ruby.xml": QSet("RDoc Label")
- "tcsh.xml": QSet("HereDocIQ", "HereDocNQ", "HereDocINQ", "HereDocQ", "FindCommentsParen")
- "xmldebug.xml": QSet("44:Enumeration")

Unused keyword lists:
- "ample.xml": QSet("dvafct", "icprops", "sgfct")
- "ansforth94.xml": QSet("attention")
- "css.xml": QSet("mediatypes_op")
- "dosbat.xml": QSet("not", "else")
- "euphoria.xml": QSet("constants")
- "freebasic.xml": QSet("Assembly Operators")
- "fsharp.xml": QSet("symbols")
- "ilerpg.xml": QSet("pkeywords", "evalopcodes8")
- "metafont.xml": QSet("EnvDelimiters", "notDefined")
- "pango.xml": QSet("int_attributes", "plain_attributes", "endtags", "color_attributes", "tags")
- "pony.xml": QSet("types", "literal")
- "powershell.xml": QSet("operators", "attributes")
- "scss.xml": QSet("mediatypes_op")
- "vhdl.xml": QSet("forOrWhile", "directions")
- "xonotic-console.xml": QSet("Aliases")
- "build/frameworks/syntax-highlighting/data/css-php.xml": QSet("mediatypes_op")
- "prolog.xml": QSet("list+is_list non-ISO", "lists ISO", "listing non-ISO", "directives non-ISO", "lists non-ISO", "terms non-ISO", "streams deprecated")

Fixing these issues is quite some work and sometimes even a bit tricky. So if you want to get your hands dirty, help is always welcome. Please send your patches to us using the syntax-highlighting repository on Phabricator 🙂

New Kate on Windows installer

We have a new round of Kate installers for Windows. This round comes with some installer-bug fixes (all project plugin dlls) and support for editorconfig files

Project plugin analyzing codeIn the above screen-shot you can see the project plugin analyzing Kate code. This works if you have cppcheck installed and added to the path.

Grab the installers now at  Kate-setup-17.08.1-KF5.38-32bit or Kate-setup-17.08.1-KF5.38-64bit

Kate 17.04.1 available for Windows

Installers for Kate 17.04.1 are now available for download!

This release includes, besides bug-fixing and features, an update to the search in files plugin. The search-while-you-type in the current file should not “destroy” your last search in files results as easily as previously. The search-combo-box-history handling is also improved.

Grab it now at  Kate-setup-17.04.1-KF5.34-32bit or Kate-setup-17.04.1-KF5.34-64bit