Tag Archives: planet

Kate: Search & Replace Notifications in KDE 4.11

In KDE 4.10, the “Find All” and “Replace All” highlights all matches and at the same time shows a passive notification in a bar below the view. This bar is animated, and takes quite a lot of place in addition to the search & replace bar.

Since some days, Kate Part can also show passive notifications floating in the view. Hence, we’ve changed the passive notification to appear on the bottom right as a small info message, showing the number of matches. However, in order to make this passive notification as small as possible, we removed the “Close” button, since the notification is hidden after 3 seconds anyway. Further, we removed the “Keep Highlighting” button. If you want to keep the highlights, just do not close the search & replace bar. The following video demonstrates this behavior, first for KDE 4.10, then how it currently will be in KDE 4.11 (watch the video in 720p):

New Text Folding in kate.git master

In the kate.git master branch the text folding is now new and shiny.

In addition to be faster and less memory hungry (no folding tree is around if you fold nothing, it is only created on-demand exactly for the folded regions), the new code is less complex and smaller (and hopefully better documented + unit tested, it actually has a test for most internal operations).

There is actually now a clean separation, the folding does not mix with highlighting and can be used without it, too.

This allows for a better integration of indentation based foldings like in Python (and already fixes the most ugly miss-placements of folding regions there) and empowers the user to fold arbitrary regions manually.

This can be done at the moment with two kate commands (for the F7 command line):

  • “fold” will create persistent fold for the current selection, this fold will stay until next reload of document (will implement some dfold command later, to delete it)
  • “tfold” will create temporary fold for the current selection, this fold will disappear again on unfolding

This should allow better vi mode integration, too, as vi is able to fold arbitrary regions, too. (There is an internal API to fold any KTextEditor::Range you pass in, which should allow to fold by regex and other ways, too, if somebody steps up to implement it.)

If you find glitches, please report them as bugs and please attach some file to reproduce, I am very interested to get this all in even better shape for KDE 4.11, be folding crashs a thing of the past.

P.S.

For all people not using folding at all, be happy, it no longer will crash you (and me), as it is a nop if no folding ranges are created by the user and some extreme cases got a speedup of 1000% because of less overhead, like VERY large C++ sources ;)

Text Folding Reloaded

If we look at the incoming bug reports, a lot still seem to be located in our code folding implementation.

The problem of the current code folding code, even after the GSoC rewrite of large parts, is that it is inter weaved with the syntax highlighting engine.

During highlighting, we try to build up some tree containing all folding start/ends. To accommodate languages that have indentation based folding like Python, we started to fake such starts/ends which are normally tokens like { or }. As these tree can be unbalanced and contain overlapping regions, the implementation is quiet complex.

Additionally, as the folding relies heavily on the syntax highlighting, which is done per document, it is impossible to have different folding per view.

After looking again at the code to solve some issues, it got more and more clear to me, that we need to restart that from scratch.

Instead of feeding the highlighting info into a tree, I started to implement text folding that just allows you to mark regions as folding ranges manually. Then internally a non-overlapping tree of ranges is created that uses our moving cursors to keep track of changes in the document. Each view will have an instance of such a tree, which allows to have folding per view.

These folds are only created on demand, e.g. if you decide to fold some region via selection + “fold” command. This already works quiet well in my Kate repository clone. The missing part is now the integration with the highlighting, that should use this well defined API to only create a folding region in the moment the user wants to fold it. In addition some API for the highlighters is needed to indicate that for some line there is the fold action. After that is done, I might consider to replace the current implementation with the novel one, which is btw. much easier to unit test (which is already done for most API primitives).

Btw., the work is done in this repository, if somebody wants to contribute, patches are welcome .P

Kate: New contributors, Valgrind, and a fixed bug

In the recent months Kate saw quite some contributions from developers other than “the usual kate developers”. This is really cool, areas include more work on python plugins, a lot of bug fixing and fine tuning in the vi input mode, an improved python indenter, and more unit tests. Especially one unit test turned out to be immensely useful and resulted in a fix for the infamous “rangesForLine” crash (about 40 times reported). We even decided to backport this to KDE 4.10.2, so KDevelopers and Kile LaTeXers please keep an eye open on whether this has any side effects!

Basically, the unit test provided us with a valgrind trace. Since valgrind is a very useful tool for all developers, we’ll have a closer look at the valgrind trace:

==10276== Invalid read of size 4
==10276== at 0x62F2116: Kate::TextCursor::lineInternal() const (katetextcursor.h:134)
==10276== by 0x62F9035: Kate::TextBlock::updateRange(Kate::TextRange*) (katetextblock.cpp:572)
==10276== by 0x62F8C57: Kate::TextBlock::mergeBlock(Kate::TextBlock*) (katetextblock.cpp:522)
==10276== by 0x62EFAA1: Kate::TextBuffer::balanceBlock(int) (katetextbuffer.cpp:494)
==10276== by 0x62EF1B2: Kate::TextBuffer::unwrapLine(int) (katetextbuffer.cpp:298)
==10276== by 0x637BB9C: KateBuffer::unwrapLines(int, int) (katebuffer.cpp:292)
==10276== by 0x6359DE9: KateDocument::editRemoveLines(int, int) (katedocument.cpp:1321)
==10276== by 0x6359AD6: KateDocument::editRemoveLine(int) (katedocument.cpp:1293)
==10276== by 0×6357841: KateDocument::removeLine(int) (katedocument.cpp:732)
==10276== by 0x6394C6A: KateScriptDocument::removeLine(int) (katescriptdocument.cpp:513)
==10276== Address 0×14353470 is 96 bytes inside a block of size 152 free’d
==10276== at 0x4C2A44B: operator delete(void*) (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==10276== by 0x62FDE4D: Kate::TextRange::~TextRange() (katetextrange.cpp:66)
==10276== by 0x6470B1E: KateOnTheFlyChecker::deleteMovingRange(KTextEditor::MovingRange*) (ontheflycheck.cpp:535)
==10276== by 0×6470657: KateOnTheFlyChecker::rangeEmpty(KTextEditor::MovingRange*) (ontheflycheck.cpp:489)
==10276== by 0x62FE641: Kate::TextRange::checkValidity(int, int, bool) (katetextrange.cpp:196)
==10276== by 0x62F6C84: Kate::TextBlock::wrapLine(KTextEditor::Cursor const&) (katetextblock.cpp:168)
==10276== by 0x62EEF1D: Kate::TextBuffer::wrapLine(KTextEditor::Cursor const&) (katetextbuffer.cpp:231)
==10276== by 0x637BA6D: KateBuffer::wrapLine(KTextEditor::Cursor const&) (katebuffer.cpp:275)
==10276== by 0x635972E: KateDocument::editInsertLine(int, QString const&) (katedocument.cpp:1246)
==10276== by 0×6357690: KateDocument::insertLine(int, QString const&) (katedocument.cpp:706)
==10276== by 0x6394C32: KateScriptDocument::insertLine(int, QString const&) (katescriptdocument.cpp:508)

The trace tells us that somewhere in TextCursor::lineIntern() we read a variable/address that is invalid. In fact, we are reading a pointer called m_block that should be valid. In this case, as we know the pointer should be valid, we can conclude that the entire TextCursor object is then corrupted/invalid. Since the cursor is part of a TextRange, it means the TextRange is already invalid. Interestingly, this is exactly what valgrind is telling us in the lower part: The TextRange destructor that lead to the invalid read above is called in the 2nd half of the valgrind trace. From this, we can conclude that the function KateTextBuffer::mergeBlock() accesses a dangling pointer to a TextRange.

That’s a good start, but in fact, this is where the hard work begins. I’ve invesed about 2 hours to find that probably the dangling pointer is in a cache that is supposed to make the access of the TextRanges (MovingRanges to be precise) faster. Christoph then looked into it and tried several attempts to fix the nasty bug, finally showing us that the cache was indeed wrong but due to a too-late fixup of the start/end text cursors with respect to the TextBlocks. In total, Christoph probably invested about 6 hours, and given that Gerald certainly took quite a lot of time too to write the unit test, we probably have more than 10 ours just to get this bug fixed. Still, the good news is that we got it! And if we are lucky, this was the last bug in  Kate’s text buffer. So on to the next crashes… Help welcome!