MIT licensed KSyntaxHighlighting usage

With the KDE Frameworks 5.50 release, the KSyntaxHighlighting framework was re-licensed to the MIT license.

This re-licensing only covers the actual code in the library and the bundled themes but not all of the syntax highlighting definition data files.

One of the main motivation points was to get QtCreator to use this, if possible, instead of their own implementation of the Kate highlighting they needed to create in the past due to the incompatible licensing of KatePart at that time (and the impossibility to do a quick split/re-licensing of the parts in question).

We talked about that possibility on Akademy this year and it seems, that if time permits, this will be picked up by the QtCreator team.

The current state allows the use of this tier 1 framework by projects like Qt(Creator), that require non-copyleft licenses for bundled 3rd-party source components, but in addition also for commercial applications that do static linking against a commercial Qt version.

Whereas at the moment, the QtCreator integration has not yet started (at least I am not aware of any work for that), a first commercial consumer already exists.

The company AbsInt I work at does develop both binary and source level analysis tools. Our GUI is Qt based, statically linked with a commercial license.

Before the current release, our GUI used a handcrafted highlighter for our own annotation languages and the programming languages we support (e.g. C and C++). After the release of the 5.50 MIT licensed KSyntaxHighlighting, this was changed to use the framework through its QSyntaxHighlighter implementation.

The framework was easy to integrate into our static build process.  To make it possible to be used without violating licensing for the bundled highlighting definitions that are not MIT and ensure no other installed instances of the framework will mess up the shipped highlighting definitions, the following two changes were contributed upstream.

A CMake switch to disable the bundling of the syntax definition data files into the library. This avoids mixing non-MIT files into the created static library, which then only contains MIT licensed code and data. One can then let people either download the definitions or ship some as extra data files with an extra licensing.

cmake -DQRC_SYNTAX=OFF

A CMake switch to disable the lookup for syntax and theme definitions in the normal locations via QStandardPaths. This allows the user of the library to only load definitions from search paths specified manually. No definitions that e.g. are installed by users for Kate or other applications using the framework will mess up your lookup, which is really important if you rely on exactly your files to be used.

cmake -DNO_STANDARD_PATHS=ON

These two options might be interesting for the QtCreator people, too. If they need additional configurability, I am sure we can find ways to integrate that.

After the transition, my colleagues compared the speed of the old implementation versus the new generic highlighting engine. At first, they were not that impressed, which did lead to several performance improvements to be implemented and up-streamed.

All directly visible bottle-necks got perf’d away. The most CPU consumption now more or less boils down to the costs of the used regular expressions via QRegularExpression. Same for the allocations, we reduced them by taking a look on the heaptrack profiles for the KSyntaxHighlighting benchmark suite.

But as always, performance work is never done, if you have time, you can take a look by profiling the “highlighter_benchmark” autotest, that applies the shipped highlightings to the test files we have in the repository.

There is no divergence in the local git clone at AbsInt at the moment, nor is there any plan to have that in the future. Both sides profit from up-streaming the changes. Other consumers of the framework get improvements and AbsInt doesn’t need to maintain a patched version.

Starting with the 18.10 release of our tools, all highlighting is handled by the framework, no more error-prone handcrafting of QSyntaxHighlighter implementations.

Thanks to all people that helped making this possible ;=)

I hope more projects/companies will pick up the use of this pure qt-dependent tier 1 framework in the future and up-stream their improvements. Be welcome.