Tag Archives: planet

Kate: New contributors, Valgrind, and a fixed bug

In the recent months Kate saw quite some contributions from developers other than “the usual kate developers”. This is really cool, areas include more work on python plugins, a lot of bug fixing and fine tuning in the vi input mode, an improved python indenter, and more unit tests. Especially one unit test turned out to be immensely useful and resulted in a fix for the infamous “rangesForLine” crash (about 40 times reported). We even decided to backport this to KDE 4.10.2, so KDevelopers and Kile LaTeXers please keep an eye open on whether this has any side effects!

Basically, the unit test provided us with a valgrind trace. Since valgrind is a very useful tool for all developers, we’ll have a closer look at the valgrind trace:

==10276== Invalid read of size 4
==10276== at 0x62F2116: Kate::TextCursor::lineInternal() const (katetextcursor.h:134)
==10276== by 0x62F9035: Kate::TextBlock::updateRange(Kate::TextRange*) (katetextblock.cpp:572)
==10276== by 0x62F8C57: Kate::TextBlock::mergeBlock(Kate::TextBlock*) (katetextblock.cpp:522)
==10276== by 0x62EFAA1: Kate::TextBuffer::balanceBlock(int) (katetextbuffer.cpp:494)
==10276== by 0x62EF1B2: Kate::TextBuffer::unwrapLine(int) (katetextbuffer.cpp:298)
==10276== by 0x637BB9C: KateBuffer::unwrapLines(int, int) (katebuffer.cpp:292)
==10276== by 0x6359DE9: KateDocument::editRemoveLines(int, int) (katedocument.cpp:1321)
==10276== by 0x6359AD6: KateDocument::editRemoveLine(int) (katedocument.cpp:1293)
==10276== by 0x6357841: KateDocument::removeLine(int) (katedocument.cpp:732)
==10276== by 0x6394C6A: KateScriptDocument::removeLine(int) (katescriptdocument.cpp:513)
==10276== Address 0x14353470 is 96 bytes inside a block of size 152 free’d
==10276== at 0x4C2A44B: operator delete(void*) (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==10276== by 0x62FDE4D: Kate::TextRange::~TextRange() (katetextrange.cpp:66)
==10276== by 0x6470B1E: KateOnTheFlyChecker::deleteMovingRange(KTextEditor::MovingRange*) (ontheflycheck.cpp:535)
==10276== by 0x6470657: KateOnTheFlyChecker::rangeEmpty(KTextEditor::MovingRange*) (ontheflycheck.cpp:489)
==10276== by 0x62FE641: Kate::TextRange::checkValidity(int, int, bool) (katetextrange.cpp:196)
==10276== by 0x62F6C84: Kate::TextBlock::wrapLine(KTextEditor::Cursor const&) (katetextblock.cpp:168)
==10276== by 0x62EEF1D: Kate::TextBuffer::wrapLine(KTextEditor::Cursor const&) (katetextbuffer.cpp:231)
==10276== by 0x637BA6D: KateBuffer::wrapLine(KTextEditor::Cursor const&) (katebuffer.cpp:275)
==10276== by 0x635972E: KateDocument::editInsertLine(int, QString const&) (katedocument.cpp:1246)
==10276== by 0x6357690: KateDocument::insertLine(int, QString const&) (katedocument.cpp:706)
==10276== by 0x6394C32: KateScriptDocument::insertLine(int, QString const&) (katescriptdocument.cpp:508)

The trace tells us that somewhere in TextCursor::lineIntern() we read a variable/address that is invalid. In fact, we are reading a pointer called m_block that should be valid. In this case, as we know the pointer should be valid, we can conclude that the entire TextCursor object is then corrupted/invalid. Since the cursor is part of a TextRange, it means the TextRange is already invalid. Interestingly, this is exactly what valgrind is telling us in the lower part: The TextRange destructor that lead to the invalid read above is called in the 2nd half of the valgrind trace. From this, we can conclude that the function KateTextBuffer::mergeBlock() accesses a dangling pointer to a TextRange.

That’s a good start, but in fact, this is where the hard work begins. I’ve invesed about 2 hours to find that probably the dangling pointer is in a cache that is supposed to make the access of the TextRanges (MovingRanges to be precise) faster. Christoph then looked into it and tried several attempts to fix the nasty bug, finally showing us that the cache was indeed wrong but due to a too-late fixup of the start/end text cursors with respect to the TextBlocks. In total, Christoph probably invested about 6 hours, and given that Gerald certainly took quite a lot of time too to write the unit test, we probably have more than 10 ours just to get this bug fixed. Still, the good news is that we got it! And if we are lucky, this was the last bug in  Kate’s text buffer. So on to the next crashes… Help welcome!

Navigation in Okular: Back & Forward

Okular, KDE’s universal document viewer, has a really cool feature I’m using for years already: “Go > Forward” and “Go > Backward”. These two actions allow to quickly jump to positions in the document where you came from in a chronological order. Consider e.g. reading the phrase “As shown in [15], …”, and you want to know quickly lookup reference [15]. So you click on it, and okular will jump to the list of references. And “Go > Back” will bring you back to exactly the position where you came from.

Awesome :-D

In fact, I removed the “Previous” and “Next”-page buttons from the toolbar in favor of the “Back” and “Forward” feature, which is much more useful to me :) Given that my colleagues were not aware of this feature (all experts in LaTeX  and all using Kile and Okular), I thought to praise it here in a dedicated blog. Interestingly, the Okular Handbook is missing this feature. Any takers to fix this? :-) Update: This is also mentioned in the Okular handbook :-)

Idea: Code Completion for JavaScript

In 2012, Kate had a GSoC project with the aim of adding a Scripting IDE plugin. The idea was to make it easy to write Kate Part scripts (command line scripts, indenters), that should include rudimentary code completion and a gui to manage available scripts. As detailed in the blog post, a rudimentary list that showed all available scripts is available. However, due to changes in Kate Part’s scripting framework (e.g. changes in the folder structure, require() function to load libraries), the plugin in its current form is not applicable anymore. Still, the idea of having a better JavaScript support is valid.

So this blog post is about some ideas how we can add possibly really cool JavaScript support to Kate as a plugin. This plugin should provide the following features:

  1. provide code completion
  2. support for loading additional modules (jQuery, Node.js, Prototype, Kate Scripting API, …)
  3. specialized support for Kate Part scripting
  4. outline in terms of a function and prototype browser

JavaScript Code Completion

Providing code completion to JavaScript requires a parser so we have an abstract syntax tree (AST) of the current file. This parser needs to be error-tolerant, since during the process of programming, the source code often violates the syntax. Searching the Web, relatively few information about such a parser exists. But one tiny parser shines: Esprima (source code, more info), written by KDE’s “very own” Ariya :-)

Esprima is a JavaScript parser written in JavaScript. It is (to some degree) error-tolerant and returns an AST as shown in the parse demo. The AST is returned in simple JSON syntax, meaning it can be easily parsed with either the JavaScript JSON functions or JSON in Qt5 (JSON for Qt4). Along the error-tolerant mode, the AST returned by Esprima also contains comments and line/column information, if desired.

Interestingly, the idea of using Esprima for code completion arose about a year ago, and meanwhile there are several projects that successfully use the Esprima AST: Eclipse Orion, a web-based IDE, uses Esprima for code completion and for the outline view (sources). The source code for the code completion is available in the orion client git module: lib, code completion (content assist), unit tests.

So What Next?

As it seems the code is all there, the next step would be to start a small standalone project and write a proof of concept to check whether this approach is feasible. I’d suggest to use Qt’s JavaScript interpreter (QtScript in Qt4, QJS in Qt5). In the long run, the goal would be to have code completion and possibly an outline view. If this works, more functionality like support for Kate Scripting could be added. So there is a lot to play around with.

If you are interested, just start working on it. It might be a good GSoC project (although we’d require a quite skilled developer here) :-P Thoughts? Comments?

Kate Git Statistics

Now the statistics of the kate.git are online for public viewing.
They will be updated daily, located on: http://kate-editor.org/stats/

Unfortunately, the statistics of the last years are not that “representative”, as the moves of Kate around in SVN and to Git biased the statistics, as I did a lot of the commits for syncing and moving and so on.

Still it is amazing how MANY people did contribute during Kate’s history! (see Kate Authors)

A big THANK YOU to all contributors that appear there (and all the people that provided patches others merged in, did the testing, bug reporting, translations, …).

(And thanks to the authors of git_stats.)