Category Archives: Developers

MIT licensed KSyntaxHighlighting usage

With the KDE Frameworks 5.50 release, the KSyntaxHighlighting framework was re-licensed to the MIT license.

This re-licensing only covers the actual code in the library and the bundled themes but not all of the syntax highlighting definition data files.

One of the main motivation points was to get QtCreator to use this, if possible, instead of their own implementation of the Kate highlighting they needed to create in the past due to the incompatible licensing of KatePart at that time (and the impossibility to do a quick split/re-licensing of the parts in question).

We talked about that possibility on Akademy this year and it seems, that if time permits, this will be picked up by the QtCreator team.

The current state allows the use of this tier 1 framework by projects like Qt(Creator), that require non-copyleft licenses for bundled 3rd-party source components, but in addition also for commercial applications that do static linking against a commercial Qt version.

Whereas at the moment, the QtCreator integration has not yet started (at least I am not aware of any work for that), a first commercial consumer already exists.

The company AbsInt I work at does develop both binary and source level analysis tools. Our GUI is Qt based, statically linked with a commercial license.

Before the current release, our GUI used a handcrafted highlighter for our own annotation languages and the programming languages we support (e.g. C and C++). After the release of the 5.50 MIT licensed KSyntaxHighlighting, this was changed to use the framework through its QSyntaxHighlighter implementation.

The framework was easy to integrate into our static build process.  To make it possible to be used without violating licensing for the bundled highlighting definitions that are not MIT and ensure no other installed instances of the framework will mess up the shipped highlighting definitions, the following two changes were contributed upstream.

A CMake switch to disable the bundling of the syntax definition data files into the library. This avoids mixing non-MIT files into the created static library, which then only contains MIT licensed code and data. One can then let people either download the definitions or ship some as extra data files with an extra licensing.

cmake -DQRC_SYNTAX=OFF

A CMake switch to disable the lookup for syntax and theme definitions in the normal locations via QStandardPaths. This allows the user of the library to only load definitions from search paths specified manually. No definitions that e.g. are installed by users for Kate or other applications using the framework will mess up your lookup, which is really important if you rely on exactly your files to be used.

cmake -DNO_STANDARD_PATHS=ON

These two options might be interesting for the QtCreator people, too. If they need additional configurability, I am sure we can find ways to integrate that.

After the transition, my colleagues compared the speed of the old implementation versus the new generic highlighting engine. At first, they were not that impressed, which did lead to several performance improvements to be implemented and up-streamed.

All directly visible bottle-necks got perf’d away. The most CPU consumption now more or less boils down to the costs of the used regular expressions via QRegularExpression. Same for the allocations, we reduced them by taking a look on the heaptrack profiles for the KSyntaxHighlighting benchmark suite.

But as always, performance work is never done, if you have time, you can take a look by profiling the “highlighter_benchmark” autotest, that applies the shipped highlightings to the test files we have in the repository.

There is no divergence in the local git clone at AbsInt at the moment, nor is there any plan to have that in the future. Both sides profit from up-streaming the changes. Other consumers of the framework get improvements and AbsInt doesn’t need to maintain a patched version.

Starting with the 18.10 release of our tools, all highlighting is handled by the framework, no more error-prone handcrafting of QSyntaxHighlighter implementations.

Thanks to all people that helped making this possible ;=)

I hope more projects/companies will pick up the use of this pure qt-dependent tier 1 framework in the future and up-stream their improvements. Be welcome.

Who is Hiring?

Just as quick info: For some time, there is a sticky thread on r/cpp about who is hiring C++ developers. This thread gets cleaned quarterly, so all the open jobs listed there are likely still open. The same was just started on reddit for r/Qt5: Who’s Hiring Qt Devs – Q4 2018: So if you are looking for either C++ or Qt jobs, this is a good place to have a look at from time to time. When I just looked for r/Qt5, the list was still empty, but this hopefully changes soon, given that the post was added just some hours ago. 🙂

Kate projects and out-of-source builds

During Akademy I once more was a bit disappointed how bad the project plugin of Kate can cope with out-of-source builds.

At work, we use in-source-builds, as we normally only build in one configuration and have no issues with left-overs in the source directories locally. For this use-case, the project plugin works really well. You have your project local terminal view and that allows you all normal things you need during work, e.g. building + using the git command line client for the version control work.

On the other side, with out-of-source builds, that no longer is that nice to use. Either you use the .kateproject generated by the “Kate – Ninja” or “Kate – Unix Makefiles” CMake generators, then your terminal defaults to the build directory, which allows building just fine, but no version control stuff, or you use the .kateproject (or auto-project creation) in the source directory, which doesn’t allow you to build nicely inside the terminal prompt of Kate. There are workaround for that, like having shell magic to switch between source and build directory with ease, but that all feels a bit unnatural.

Therefore, I added today a very simple “fix” for the issue: If you have a .kateproject that has a different base directory (the toplevel “directory” entry) than the directory the .kateproject file is located in, you will get two terminal tabs in the project view. One is the “old one” that has the directory of the .kateproject as base, the other has the base directory of the project as base.

With these two views, you can easily switch between build/source directory without any hassle or extra setup for any properly setup .kateproject as generated by CMake.

I hope this improves the usability of the project plugin for the normal setup of out-of-source builds with CMake. If this sparked your interest: any further improvement ideas are welcome, best as patches submitted on phabricator.kde.org.

I think this small change is something that shows how many open source contributions work: You have some itch to scratch and you share your solution to help others that have a similar issue.

If you look at the open bugs & wishes for Kate/KWrite/KTexteditor/… you will see that there are still a lot things that need some scratching. It might look like the developers don’t care for the issues of their users, but that is not correct. We just don’t have the time to scratch all these itches (nor are all that easy solvable). Sometimes we unfortunately did even lack time or motivation to do proper reviews for some proposed solutions, I hope we improve on that in the future. Any volunteers that help us taking care are always welcome. The addition of the inline notes interface is a nice example. Michal Srb provided an initial solution for his own needs to us and sparked some new development with that.

Kate gains Support for Inline Notes

Thanks to Michal Srb and Sven Brauch for pioneering the work an a new KTextEditor interface that allows applications like Kate, KDevelop, etc. to display inline notes in a text document. As demo, we quickly prototyped one application to display colors in CSS documents:

Clicking on the color rectangle will launch the color chooser:

Choosing a color and clicking OK finally adapts the color in the CSS document:

The code for this is just a demo and looks as follows:

class NoteProvider : public KTextEditor::InlineNoteProvider {
public:
    QVector<int> inlineNotes(int line) const override
    {
        if (line == 1) return { 29 };
        if (line == 11) return { 29 };
        if (line == 12) return { 29 };
        if (line == 13) return { 29 };

        return {};
    }

    QSize inlineNoteSize(const KTextEditor::InlineNote& note) const override
    {
        return QSize(note.lineHeight(), note.lineHeight());
    }

    void paintInlineNote(const KTextEditor::InlineNote& note, QPainter& painter) const override
    {
        const auto line = note.position().line();
        const auto color = QColor(note.view()->document()->text({line, 22, line, 29}));
        painter.setPen(color);
        painter.setBrush(color.lighter(150));
        painter.drawRoundedRect(1, 1, note.width() - 2, note.lineHeight() - 2, 2, 2);
    }

    void inlineNoteActivated(const KTextEditor::InlineNote& note, Qt::MouseButtons buttons, const QPoint& globalPos) override
    {
        const int line = note.position().line();
        const auto oldColor = QColor(note.view()->document()->text({line, 22, line, 29}));
        const auto newColor = QColorDialog::getColor(oldColor);
        note.view()->document()->replaceText({line, 22, line, 29}, newColor.name(QColor::HexRgb));
    }

    void inlineNoteFocusInEvent(const KTextEditor::InlineNote& note, const QPoint& globalPos) override
    {} // unused in this example

    void inlineNoteFocusOutEvent(const KTextEditor::InlineNote& note) override
    {} // unused in this example

    void inlineNoteMouseMoveEvent(const KTextEditor::InlineNote& note, const QPoint& globalPos) override
    {} // unused in this example
};

// later in code:
auto provider = new NoteProvider();
view->registerInlineNoteProvider(provider);
// final cleanup
view->unregisterInlineNoteProvider(provider);

As you can see, it’s actually not much code at all: We have to derive a class from KTextEditor::InlineNoteProvider, and then register an instance of our Note Provider in the KTextEditor::View. In a next step, we implement the inlineNotes(), inlineNoteSize(), and the paintInlineNote() functions to get basic visual drawing at the desired location. The above code is just a tech-demo, since it uses hard-coded lines and color positions. Additionally, one can also track mouse events (unused in the example above). On mouse click, we open the QColorDialog to let the user choose a new color.

To give more examples of what’s possible, the initial Phabricator review requests contained many other interesting examples (the examples were really implemented). From review request D12662:

Kate showing additional information for loops and structs.

Or a KDevelop addition that adds a lot of meta information on the current code if desired:

KDevelop showing detailed code meta information

We believe this addition to the KTextEditor component has a lot of potential for nice features and plugins. Feel free to use this interfaces starting with KDE Frameworks 5.50. Happy coding! 🙂

A big thanks also goes to this year’s Akademy organizers. Thanks to this event, we could meet up in person and also finalize the InlineNoteInterface, InlineNoteProvider, and InlineNote class to make it ready for public release. This again shows the importance of the yearly KDE conferences since it enables us to significantly push things forward.