A while ago I’ve blogged about the Kate’s most powerful plugins: the XML completion plugin. Thanks to Tomáš Trnka this plugin is back alive with all its features since today and compiled by default in trunk: It will be included in KDE 4.5. Yay! And if you need it already now for KDE 4.4, you can build Kate yourself very easily according to this howto, without touching your global KDE installation (probably interesting for KDE documentation writers!). Have fun!
Since all of the Kate code is now co-hosted on gitorious, it became very easy to build Kate in your stable KDE >= 4.4 environment. This means you can run the newest version of Kate with very few effort. Just give it a try and do the following steps:
- make sure you have the following packages installed: git, cmake and kdelibs development package (on openSUSE this is git, cmake and libkde4-devel)
- create and change into a KDE development directory:
mkdir ~/kde; cd ~/kde
- get a copy of the Kate code:
git clone git://gitorious.org/kate/kate.git
- create and change into a build directory for compilation:
mkdir build; cd build
- run the configure process with cmake:
cmake ../kate -DCMAKE_BUILD_TYPE=debugfull \
- compile Kate:
- finally install Kate:
That’s all! This installs Kate locally into the separate directory ~/kde/usr, so that your global KDE installation will not be touched at all.
Now on to starting the newly compiled Kate. Create a file ~/kde/run.sh with the following content:
#!/bin/bashexport KATE_DIR=~/kde/usr export PATH=$KATE_DIR/bin:$PATH export LD_LIBRARY_PATH=$KATE_DIR/lib:$LD_LIBRARY_PATH export KDEDIR=$KATE_DIR export KDEDIRS=$KDEDIRexport XDG_DATA_DIRS=$XDG_DATA_DIRS:$KATE_DIR/share # update KDE's system configuration cache kbuildsycoca4 # start app $@
Now you can run the compiled Kate version with ~/kde/run.sh kate. Just calling kate directly will start the system version of Kate.
Your copy of Kate contains all of the Kate code, i.e.: the KTextEditor interfaces, Kate Part, KWrite and the Kate Application. Feel free to send patches to our mailing list email@example.com. And join #kate on irc.kde.org
Note for KDE developers: All the changes in git are merged back to KDE’s subversion repository in a timely manner. So don’t worry about Kate moving away from KDE; this is not the case.
(Updated on 2010-04-17: Allow installation into local directory.)
Right now, in Kate’s gitorious repository we, or rather mainly Christoph, is rewriting the heart of Kate: The text buffer. In this blog I’ll explain how the text buffer works in detail. After reading, you’ll be able to understand the code. And be warned: Probably you will be desperately amazed about its awesomeness! So this is a must read :^)
Storage. Kate Part internally stores the content of a text document as a QVector of text lines. Each text line holds the text in a QString object as well as additional data like color attributes for highlighting. Inserting and removing text lines implies inserting or removing items in the QVector of the text lines. This is can get slow when there are thousands of text lines in a document, because QVector has to move the memory every time. Thus, to keep text operations fast, Kate splits all text lines into several text blocks: Each text block contains a certain amount of lines (e.g. 256). The expense of memory movement is then always limited. When a text block grows too much, it is automatically split. When the amount of lines in a text block shrinks, it is merged with the previous text block. In other words, Kate’s text buffer automatically balances the text blocks. The basic idea of Kate’s text buffer, the text blocks and text lines looks like this:
Text Cursors and Ranges. Text manipulation always takes place at certain positions in a text. Such a position is called a text cursor. Each text cursor is defined by a line and a column. Further, to specify e.g. text selections, we need text ranges. A text range has a start cursor and an end cursor. For instance, if you have two views of the same document, you want the cursor in the 2nd view to keep its correct position in the text if you insert text in the 1st view. Thus, text cursors have to be kind of intelligent. Whenever text changes, each text cursor needs to be moved. As a text range just consists of two text cursors, we will focus only on text cursors for now. The question is how to implement this efficiently? If we store all text cursors in a list in the text buffer, we have to iterate over all text cursors on every editing operation. This obviously does not scale for thousands of text cursors (KDevelop for instance uses thousands of text cursors for the highlighting). The solution is the same as with the text content itself: Let’s just put a list of all text cursors in a text block to the text block itself. During text editing, we only have to adapt all text cursors of a single text block instead of all text cursors in a document. This looks as follows:
Editing Operations. When editing text, you usually have only four different types of text manipulation:
- insert text at a text cursor inside one line
- remove text at a text cursor inside one line
- wrap line at a text cursor (e.g. by hitting <return/enter>)
- unwrap line (e.g. by hitting <backspace> at the beginning of a line)
Those editing primitives are implemented in the Kate::TextBuffer and Kate::TextBlock and take care of balancing the text blocks. For each one the set of text cursors in a text block has to be adapted. Of course, corner cases need to be taken into account: For instance, unwrapping the first line in a text block means that cursors in the first line need to be moved to the previous text block. All editing primitives emit a signal, such that additional layers later can track what happens. For instance, the undo/redo system needs to know all editing operations. Or on top of that we could implement vim like swap file support, i.e., track all changes from the beginning, and if Kate crashes, replay all the editing primitives on the original files.
Transactions. A transaction consists of several of the four editing operations. For instance, if you select text and then move it to another location in the document with drag & drop you want this to be grouped together to a single text operation on undo/redo. (The text operations in this case are unwrap lines and remove text, then insert text and wrap line). To be able to specify which parts of text operations belong together, the text buffer provides two functions: startEditing() starts a transaction and finishEditing() closes it. Those functions use reference counting, so you can call startEditing() multiple times and only the last finishEditing() completes a transaction. Again, signals are emitted e.g. in finishEditing() such that other layers (undo/redo system) are notified about this.
Revisions. As an easy way to check whether a document changed you can get the current revision of the text buffer. The revision is simply an int64 starting with 0 after loading a document. The revision is incremented in every of the 4 editing primitives. This way you don’t have to listen to multiple signals like textInserted() and textRemoved(). This could be useful for e.g. KDevelop’s background parser: Parse a file. When done, check whether the revision is the same. If yes, parsing was successful. If not, parse again. This is how QtCreator does it. Easy and straight forward.
Unit Tests. The text buffer, text block, text line, text cursors are all designed such that unit tests can be written. Hence, each of the above implementation details will be covered by unit tests.
Further ideas. As mentioned previously, the design of the text buffer leaves room for further features. For now, let’s face two ideas:
- Vim like swap files: If Kate or the application using Kate Part crashes, the unsaved data in the text document is lost. Vim has a backup mechanism called ‘swap files’ to recover from crashes to avoid data loss since years, and it works as follows: Each of the 4 editing primitives explained above are streamed into a separate file. When you save a file successfully, the swap file is cleared. If a crash happens, vim loads the original text document and then replays all editing operations in the swap file. This way the current unsaved editing state is recovered; no data is lost. Since the text buffer emits all necessary signals, implementing this feature is kind of trivial. Any takers?
- Right now, the highlighting works in an extra class, accessing the text buffer’s text lines and setting the text attributes such as the color. As idea, we could also derive a class Kate::HighlightingBuffer from Kate::TextBuffer and do the additional modifications in the derived class instead of an extra class. We’ll have to see whether that makes sense.
- Basic idea: text stored as lines
- Lines hold text and additional data (like for highlighting)
- Advanced Concept: Stores lines in blocks of xxx lines, to have better performance
- They will move on editing
- Cursors can be combined to ranges
- KateTextCursor will be stored in the buffer blocks
- Each edit action must be encapsuled in a transaction
- startEdit/endEdit will do this
- Signals for starting and ending this transactions
- After loading, buffer has revision 0
- Each edit action which is no nop will lead to increment in revision number
- Successful saving will reset revision back to 0
Editing Operations (only 4 different editing primitives)
- Insert Text (inside one line)
- Remove Text (inside one line)
- Wrap Line (wrap a line at given position, create a new line with content behind wrap position)
- Unwrap Line (unites the line with its predecessor)
- Signals for all of these actions, containing the change (to allow to layer undo/swap file support/…) on top of the buffer
Load/Save & Encodings
- Loading will try to use given codec, if that doesn’t work, try to detect the codec by the BOM at start of file or use fallback codec, if that again doesn’t work, given codec will be used again, and errors reported
- Saving will just use the given codec
- Each of the above implementation details will be covered by unit tests
- KateTextBuffer (+KateTextLine and Cursor) must therefor be usable without other components