Category Archives: Common

MIT licensed KSyntaxHighlighting usage

With the KDE Frameworks 5.50 release, the KSyntaxHighlighting framework was re-licensed to the MIT license.

This re-licensing only covers the actual code in the library and the bundled themes but not all of the syntax highlighting definition data files.

One of the main motivation points was to get QtCreator to use this, if possible, instead of their own implementation of the Kate highlighting they needed to create in the past due to the incompatible licensing of KatePart at that time (and the impossibility to do a quick split/re-licensing of the parts in question).

We talked about that possibility on Akademy this year and it seems, that if time permits, this will be picked up by the QtCreator team.

The current state allows the use of this tier 1 framework by projects like Qt(Creator), that require non-copyleft licenses for bundled 3rd-party source components, but in addition also for commercial applications that do static linking against a commercial Qt version.

Whereas at the moment, the QtCreator integration has not yet started (at least I am not aware of any work for that), a first commercial consumer already exists.

The company AbsInt I work at does develop both binary and source level analysis tools. Our GUI is Qt based, statically linked with a commercial license.

Before the current release, our GUI used a handcrafted highlighter for our own annotation languages and the programming languages we support (e.g. C and C++). After the release of the 5.50 MIT licensed KSyntaxHighlighting, this was changed to use the framework through its QSyntaxHighlighter implementation.

The framework was easy to integrate into our static build process.  To make it possible to be used without violating licensing for the bundled highlighting definitions that are not MIT and ensure no other installed instances of the framework will mess up the shipped highlighting definitions, the following two changes were contributed upstream.

A CMake switch to disable the bundling of the syntax definition data files into the library. This avoids mixing non-MIT files into the created static library, which then only contains MIT licensed code and data. One can then let people either download the definitions or ship some as extra data files with an extra licensing.

cmake -DQRC_SYNTAX=OFF

A CMake switch to disable the lookup for syntax and theme definitions in the normal locations via QStandardPaths. This allows the user of the library to only load definitions from search paths specified manually. No definitions that e.g. are installed by users for Kate or other applications using the framework will mess up your lookup, which is really important if you rely on exactly your files to be used.

cmake -DNO_STANDARD_PATHS=ON

These two options might be interesting for the QtCreator people, too. If they need additional configurability, I am sure we can find ways to integrate that.

After the transition, my colleagues compared the speed of the old implementation versus the new generic highlighting engine. At first, they were not that impressed, which did lead to several performance improvements to be implemented and up-streamed.

All directly visible bottle-necks got perf’d away. The most CPU consumption now more or less boils down to the costs of the used regular expressions via QRegularExpression. Same for the allocations, we reduced them by taking a look on the heaptrack profiles for the KSyntaxHighlighting benchmark suite.

But as always, performance work is never done, if you have time, you can take a look by profiling the “highlighter_benchmark” autotest, that applies the shipped highlightings to the test files we have in the repository.

There is no divergence in the local git clone at AbsInt at the moment, nor is there any plan to have that in the future. Both sides profit from up-streaming the changes. Other consumers of the framework get improvements and AbsInt doesn’t need to maintain a patched version.

Starting with the 18.10 release of our tools, all highlighting is handled by the framework, no more error-prone handcrafting of QSyntaxHighlighter implementations.

Thanks to all people that helped making this possible ;=)

I hope more projects/companies will pick up the use of this pure qt-dependent tier 1 framework in the future and up-stream their improvements. Be welcome.

Kate projects and out-of-source builds

During Akademy I once more was a bit disappointed how bad the project plugin of Kate can cope with out-of-source builds.

At work, we use in-source-builds, as we normally only build in one configuration and have no issues with left-overs in the source directories locally. For this use-case, the project plugin works really well. You have your project local terminal view and that allows you all normal things you need during work, e.g. building + using the git command line client for the version control work.

On the other side, with out-of-source builds, that no longer is that nice to use. Either you use the .kateproject generated by the “Kate – Ninja” or “Kate – Unix Makefiles” CMake generators, then your terminal defaults to the build directory, which allows building just fine, but no version control stuff, or you use the .kateproject (or auto-project creation) in the source directory, which doesn’t allow you to build nicely inside the terminal prompt of Kate. There are workaround for that, like having shell magic to switch between source and build directory with ease, but that all feels a bit unnatural.

Therefore, I added today a very simple “fix” for the issue: If you have a .kateproject that has a different base directory (the toplevel “directory” entry) than the directory the .kateproject file is located in, you will get two terminal tabs in the project view. One is the “old one” that has the directory of the .kateproject as base, the other has the base directory of the project as base.

With these two views, you can easily switch between build/source directory without any hassle or extra setup for any properly setup .kateproject as generated by CMake.

I hope this improves the usability of the project plugin for the normal setup of out-of-source builds with CMake. If this sparked your interest: any further improvement ideas are welcome, best as patches submitted on phabricator.kde.org.

I think this small change is something that shows how many open source contributions work: You have some itch to scratch and you share your solution to help others that have a similar issue.

If you look at the open bugs & wishes for Kate/KWrite/KTexteditor/… you will see that there are still a lot things that need some scratching. It might look like the developers don’t care for the issues of their users, but that is not correct. We just don’t have the time to scratch all these itches (nor are all that easy solvable). Sometimes we unfortunately did even lack time or motivation to do proper reviews for some proposed solutions, I hope we improve on that in the future. Any volunteers that help us taking care are always welcome. The addition of the inline notes interface is a nice example. Michal Srb provided an initial solution for his own needs to us and sparked some new development with that.

Akademy 2018 Wrap-Up

The Akademy 2018 ends today.

Like each Akademy I attended, it was an interesting experience. As the location switches around each year, so does the set of people attending change every year, too.

That is actually nice, as you get always to meet some of your old “friends” but additionally new members of the KDE community. I think this kind of “conferences” or “meetings” are an important way to get some more cohesion in the community, which is sometimes a bit lacking between people only meeting online via mail/…

Beside the presentation tracks and the e.V. meeting, several of the BoFs did spark my interest.

In the KDevelop BoF, Sven talked about what could be done to give the current KDevelop project a bit more focus on the parts it does well to polish them more for a better user experience. The idea is that if you get KDevelop shining even more in the areas it is good at and perhaps cut off some parts that are really given bad impressions, one might attract more people to both use it and contribute. It is still to be discussed if this idea is shared with the other KDevelop contributors.

In the kdesrc-build BoF Michael talked about the current state and collected pain points from the audience and potential future extensions. For example an API to allow to build a light-weight GUI tooling around kdesrc-build to ease the entry to the KDE development was one topic of interest.

Between the conference/BoF/socializing parts of Akademy, I got plenty of time to finally work again on some KTextEditor/Kate related tasks.

With help of Dominik and Volker I got to integrate the KSyntaxHighlighting framework and we even got at least some initial contact with the QtCreator team about the topic of integrating this framework to replace their own implementation of the Kate syntax definition handling. If you experience any issues with the highlighting or folding in the master branch, please file a bug or even better provide some patch on phabricator.

In addition some small KTextEditor and Kate bugs got either solved or at least started to be worked on again. Help with any bug fixing is always welcome!

As small but perhaps for users important step was to actually link to the new and shiny Windows installers that the Binary Factory for KDE produces. Thanks to the team behind that, once more. Hopefully that will lead to more users and developers for Kate on Windows.

Thanks to the organizers to make this Akademy happen and all people that volunteered! Great job! The sponsors are highly appreciated for their contributions, too.

So, thanks for all the fish (or Krapfen), lets see how Akademy next year will be :=)

Akademy & Binary Factory

During Akademy it was brought to my (and the other Kate developers) attention, that we should take a closer look on the Binary Factory for KDE. There were some blogs about the Binary Factory in the past but we somehow never really linked it on our homepage as potential source for up-to-date installers for the different operating systems. I feel a bit sorry for neglecting that area in the past year.

Therefore, as we have now some time during Akademy together as team, we did take a look at the current state of the installers there for Windows and macOS.

The Windows installer is working fine. Kåre’s manually created installers are better optimized for size by stripping out things not that commonly used, on the other side the Binary Factory installers bundle more stuff like all translations/dictionaries that might be interesting for some people, too. One other thing that might be fine tuned is the used frameworks. For example the KActivities is not useful for Kate on Windows and we normally disable that for our own build, at the moment the factory still includes that (as it is an optional dependency). If we fine tune that a bit more in the future, I think we can close the gap between the hand-tuned installer and the autogenerated one even more. In other parts like the KHotNewStuff integration the current factory installer even provides a better experience than our last published own one. And last but not least, the binary factory creates an installer every night for the stable and the development version, so we get up-to-date installers with no manual work.

The macOS installer is on-par with what I did manually a year ago (but than gave up due to loss of interest). Therefore we now link the nightly build for that on our homepage as primary preview build. The state is still not that good, but that is not a fault of the installer creation. The macOS port just lacks manpower, as myself has no longer that much interest in improving it and it still is in a “usable” but non-polished state that will still crash in a few situations, which makes using it not that nice. Any help there would be appreciated.

As a conclusion I must say, I am personally impressed how well the Binary Factory installers work. They are a real big step towards having people use up-to-date versions of our tools on operating systems other than the usual unices. Thanks to the team behind that and all others that contributed, including our sysadmins that keep all the stuff running the whole time! I hope we make good use of that infrastructure in the future. Thanks to Hannah, Kevin, Ben and all others that contribute to this effort! And thanks to Kåre and everybody else taking care of issues on Windows or macOS.

The download page is now updated to provide links to both the latest successful Kate release and nightly builds of the Binary Factory.