Multiple Keyboard Layouts and Shortcuts

KDE has a very handy feature to switch keyboard layouts on the fly. This is for instance useful if you use the German keyboard layout by default, and the US layout for programming. To enable keyboard layout switching, go into System Settings > Input Devices (category Hardware) > Keyboard item > Layouts tab:

Here, ‘[x] Configure layouts‘ is checked, enabling the list view. I added the German keyboard layout, and then the English (US) layout. Notice also, that the shortcut is set to ‘Ctrl+Alt+K‘. Clicking apply, a tiny little indicator appears in the panel:

You now can quickly switch with Ctrl+Alt+K between the German and the US layout. Quite efficient, especially since the keyboard layout config page allows to switch the language on application basis.

Unchanged Keyboard Shortcuts

Switching the keyboard layout has one potential issue, though: The shortcuts remain unchanged. So if undo is mapped to Ctrl+z in the German layout, it is still mapped to Ctrl+z in the US layout. Note that by ‘z’ we refer to the hardware key on the keyboard. As a consequence, in the US layout, hitting the hardware key ‘y’ on the German keyboard inserts the character ‘z’, but the z in Ctrl+z is still on the hardware key ‘z’. This behavior may or may not be wanted.

Getting more into detail reveals that the order of the keyboard layouts in the first screen shot is of importance: If you first add the German ‘de‘ layout, and then the English ‘us‘ layout, then the shortcuts will always use the Germany keyboard layout, independent of what keyboard layout is chosen.

Reversely, if you first add the English ‘us’ layout, and then the German ‘de‘ layout, then the shortcuts will always use the English ‘us‘ keyboard layout.

So it seems that the order defines a priority, and the shortcuts always use the first entry in the list.

The correct solution to fix this would (in my humble opinion) be to add an option ‘[x] Shorcuts follow keyboard layout‘ or similar. But since this option does not exist, let’s do a quick hack to still get what we want here.

A Workaround

First we reset the shortcut in the settings of the keyboard layout options:

Click apply and close the dialog. Now, the shortcut ‘Ctrl+Alt+K‘ is unbound. Our idea is now to create a script that toggles the keyboard layout by calling setxkbmap with the appropriate parameters and bind this script via a global shortcut to ‘Ctrl+Alt+K‘.

To this end, we first have to create the script. So let’s first type `setxkbmap -query` in the console and check the output. For me, this results in:

$ setxkbmap -query
rules: evdev
model: pc101
layout: de,us
variant: nodeadkeys,

From this, we can follow that the current xkb layout is achieved with:

setxkbmap -model pc101 -layout de,us -variant nodeadkeys

Now, let’s switch the de,us to us,de and try the following:

setxkbmap -model pc101 -layout us,de -variant nodeadkeys

Notice, that the keyboard layout indicator in the panel switched to ‘us‘. Calling the first variant with de,us again, we get back to the German layout.

This discovery leads us to the following script switch-keyboard-layout.sh:

#!/bin/sh

# query xkb map: us,de -> us is primary; de,us -> de is primary
dummy=`setxkbmap -query | grep us,de`

# return value 0: us,de; return value != 0, de,us
if [ $? -ne 0 ]; then
  # de is primary, now make us primary in list
  setxkbmap -model pc101 -layout us,de -variant nodeadkeys
else
  # us is primary, now make de primary in list
  setxkbmap -model pc101 -layout de,us -variant nodeadkeys
fi

Save this script somewhere to switch-keyboard-layout.sh and make it executable with

chmod 755 switch-keyboard-layout.sh

Each time we execute this script, the keyboard layout is toggled.

Next, we go into System Settings again and navigate to Shortcuts and Gestures (Common Appearance and Behavior), there in the Custom Shortcuts we add a new Command/URL item named ‘SwitchKeyboardLayout’ as follows:

As a comment for this new item, we write ‘Switch Keyboard Layout‘, in the ‘Trigger‘ tab, bind the global shortcut to ‘Ctrl+Alt+K‘, and in the ‘Action‘ tab, choose the switch-keyboard-layout.sh script. Finally click Apply, and close the dialog.

Now, hitting Ctrl+Alt+K calls our script and correctly toggles the keyboard layout including the shortcuts.

Unfortunately, this approach does not support e.g. switching the keyboard layout on application basis as the switching policy of the Keyboard settings (first screen shot) allows. Still it works.

A final remark, though: For GTK applications this works out of the box. So is there any real reason why this is not the case for KDE / Qt applications? A real fix would be very much appreciated, I’d be also fine with an option. But not providing this feature at all is very thin ice…

Update: This issue was reported as KDE bug #197552 in 2009, and resolved as an upstream issue. However, it never was reported to Qt upstream. If I may say so, this is not how resolving bugs in KDE usually works. Grrr…

Kate in 4.11

Another release cycle gone, and the KDE Software Compilation 4.11 is out in the open (well, for quite some time already), and with that it is time to talk about what changed in Kate the last half year since the 4.10 release. Besides the usual bug fixing (~50 bugs since 4.10), the following sections present some major improvements and features of Kate in 4.11.

Python Plugins

Since KDE 4.10, Kate features the Python »Pâté« plugin. This plugin basically wraps the API of all Kate application interfaces. Therewith, together with the Python KDE (PyKDE) bindings you can develop full-featured Kate plugins in Python. Being relatively new, there are quite some Python plugins available already now (see this blog, this blog, or this blog).

@Python community: You can interpret this as an invitation for writing lots of cool addons for the Kate text editor in python. All contributions are usually very much welcome!

Vi Input Mode

Enabling the vi input mode turns Kate into a full-fleged vim compatible editor: modal editing exactly like in vim. This mode is especially suited for vim users who want a fully KDE integrated text editor with all the beloved vim features. Started as GSoC project in 2010, the vi input mode evolved over the years to become a very mature alternative to vim itself. However, there is always room for improvements. The good news is, that Kate Part’s vi input mode gained a lot of attention thanks to Simon, mostly in the form of small “papercuts” that fixed small bugs and annoyances and made behaviour more compatible with Vim, including:

  • Yank highlighting, which helps you to see what text you just yanked;
  • “Yank to clipboard” (courtesy of Fabian Kosmale!);
  • The “last edit” markers, “.”, “[" and "]“;
  • Recursive vs non-recursive mappings, for if you want to e.g. map “j” to “gj” …
  • … plus plenty of fixes to gj and gk;
  • and countless other small tweaks to things like the “inner block” text object; cursor position after paste; fixes to ctrl-a/x (add to/ subtract from number under cursor); making more motions available in Visual Mode and more motions counted, etc!

One fairly major change is the introduction of an experimental Emulated Vim Command Bar. This, as you will probably have guessed, is a replacement for the current Search dialog that behaves more like Vim’s, allowing one to e.g. insert the contents of registers via ctrl-r; dismiss via ctrl-c/ ctrl-[ as well as ESC; work properly with both forward (“/”) and backward (“?”) incremental searches; use smart case for case-sensitivity; allow the use of Vim-style regex’s etc. As an extension to Vim, I’ve also added the ability to auto-complete words from the document via ctrl-space: not that useful for searching, but very useful for when you want to do a search+replace command (command mode (“:”) did not make it into 4.11, alas).

In 4.11, it is disabled by default as it is not yet ready for primetime, but may be enabled by setting the hidden config option “Vi Input Mode Emulate Command Bar” to “true”. It is much more powerful and featureful in current master (see the blog here!) and will likely be the default in 4.12.

New Text Folding

Back in KDE SC 4.8, Kate got a new code folding implementation as part of the yearly GSoC project. Compared to the previous implementation, the amount of bugs indeed reduced, but during the 4.9 and 4.10 release it became apparent, that the code folding was still not where it should be. Maybe writing a good code folding implementation was also a bit too much in terms of a newby developer in a GSoC project.

So Christoph sat down and wrote it completely new from scratch, this time very clean and simple. And as a new feature, the text folding is now per view, so the folded parts can be different in each view of a document. And as another feature for those who do not use text folding at all: Not using text folding means you have zero overhead, no matter how large the file. Besides that, code folding in Python works now better than ever before, in fact, it should be bug-free.

Passive Notifications & KTextEditor Interfaces

Passive notifications are meant to provide a non-intrusive way showing notifications (replacing KPassivePopup). Used e.g. by the data recovery, these notifications were also introduced for the search & replace feature (most of this is already available in KDE SC 4.10.3). In KDE SC 4.11, the interface for showing these passive notifications got officially included into the KTextEditor interfaces, meaning that applications like Kile or KDevelop can show notifications as well. (Issues in KDE 4.11.0 should be fixed for 4.11.1).

Next to the KTextEditor::MessageInterface, Kate has a so-called TextHintInterface since year 2003, but it seems its implementation never got finished. For 4.11, Sven Brauch stepped up and finally fixed the text hint interface. So showing tool tips for text under the mouse (section Text tooltips) is as easy as it was never before:

Saving Files

When saving files, Kate uses the class KSaveFile (in Qt5 available as QSaveFile thanks to David Faure). In short, to avoid data loss, KSaveFile saves the file to a temporary file in the same directory as the target file, and on successful write finally moves the temporary file to the target filename. If the folder of the file does not allow creating new files, KSaveFile automatically falls back to writing the file directly. In this case, data loss could happen if e.g. the system crashes in this moment, but saving the file would otherwise be not possible at all.

What Comes Next?

Announced just a few days ago, the KDE community prepares for KDE Frameworks 5 and much more: The KDE libraries and the KDE Plasma Workspace are now in long term support mode, meaning that bugs will get fixed for another two years. That means that the KTextEditor interfaces are frozen as well, there will not be any feature updates at all in the KTextEditor interfaces itself. Since Kate Part and the Kate Application are developed outside of kdelibs, Kate Part and Kate will most certainly see another feature release. Especially, since there are already a lot of improvements for the vi input mode in the pipe for Kate in KDE SC 4.12.

After that, we plan to port Kate to Qt5 and the new frameworks 5 libraries. We already added a lot of “KDE 5″ todo-markers to the Kate source code, meaning that we will probably work on a 5 port rather sooner than later. We will address this schedule in a separate post later this year, so stay tuned!

Kate Vim Progress

Hi all,

Just an eye-glazingly brief blog post about some of the stuff I’ve been working on in the Kate Vim emulation mode since my last blog. Once more, I’ll mostly be dumping them all as bullet points, Gräßlin-Style™, with some longer sections on the more important changes, but this time around I’ve added some nifty animated gifs which you can skip to should the full text prove too harrowing!

In roughly chronological order, I:

    • Fixed regression with gv (“re-select last selection”).
    • Laid the groundwork for improved behaviour when repeating a change that involved an auto-completion, for now just fixing a bug where if auto-completion was automatically invoked and we repeat the change via “.”, only a portion of what we typed gets repeated.
    • As promised, implemented “yank highlighting”, which shows the portion of the text that you just yanked; very handy if you do complex yanks like e.g “y2f)”:

“Yank Highlighting”, 444KB, Click to Play.
 

  • Fixed a bug with keypad handling.
  • Made ctrl-p and ctrl-n wrap around in completion lists.

… and a biggie:

Emulated Vim Command Bar

There are quite a lot of issues with the interaction between Vim mode and Kate’s own Find dialog:

  • The cursor is not positioned at the beginning of the match.
  • The full match is not highlighted.
  • “?” does not search backwards(!)
  • The Find dialog cannot be used in mappings.
  • The Find dialog could not be used to either record or playback macros.
  • Aborts Visual Mode when summoned.

etc. On top of that, we miss out on some nice features of Vim’s search bars, such as ctrl-r for insertion of register contents; ctrl-w for deleting previous word; ctrl-h for delete; ctrl-b/e for jumping to the beginning or end (more conveniently located, IMO, than HOME or END); plus the Qt regex syntax is different from Vim’s. In light of this, I decided to add a new KateViewBarWidget called “KateViEmulatedCommandBar”:

“Emulated Command Bar Search”, 697KB, Click to Play.
This behaves much more like Vim, and has the features and bugfixes mentioned above; it even does a reasonable job (in current master; it missed 4.11) of translating Vim regexes to Qt ones e.g. with Vim, a “+” is a literal “+”, not an “at least one of the preceding” marker; with Qt it is the reverse. The emulated command bar performs this, and other, conversions for you.

The searches carried out can be repeated correctly via “n” and “N”. Search history can be navigated via ctrl-p and ctrl-n, and searches “external” to the emulated command bar (e.g. those carried out by “*” and “#”) are added to this history. “Smart Case” is used for specifying case-sensitivity – if the search term has any capitals, then the search is assumed to be case-sensitive; else it is case-insensitive. More on the emulated command bar later! Back to the micro-fixes:

  • Distinguished recursive vs non-recursive mappings; inspired by this bug report which highlighted a nasty crashing problem if we tried to map j -> gj and k -> gk …
  • … and relatedly, completely reworked gk and gj, fixing a bunch of bugs in the process.
  • Yet more fixes to the “inner block” text object, which is much more complex than you would credit!
  • A bunch of fixes and patches to ctrl-a/ctrl-x (“increment/ decrement number under-or-to-the-right-of cursor”) culminating in me just writing the whole thing from scratch :)
  • Indented paste (]p, ]P etc).

Everything above made it for 4.11, I think. The Emulated Command Bar was disabled by default but can be enabled by setting the hidden config option “Vi Input Mode Emulate Command Bar” to “true”. Continuing:

  • Made “/” and “?”, when using the Emulated Command Bar, usable as (countable) motions: so one can, for example, do d3/foo<enter> to delete up to just before the third occurence of “foo”.
  • Allowed one to set the whole selection to a given text object even after we’ve formed a selection in Visual Mode e.g. you can start visual mode, navigate to inside a pair of brackets, and do “ib” to re-set the current selection to just the inside of that pair of brackets. I think this worked once upon a time and I broke it, so this is more of a regression fix than a new feature :/
  • Made the emulated command bar usable for Commands (like the “F7″ command box in normal Kate) as well as for forwards and backwards searches when summoned via “:”. Note that since the emulated command bar is usable in mappings, we can now incorporate the execution of your Javascript script(s) into the Vim key mapping system, which could end up being a very powerful addition.
  • Allowed mappings for InsertMode and VisualMode as well as for NormalMode. Oddly, this was accomplished mainly as a side-effect of a fix for a tiny bug with mappings. There is no gui for this, yet, but I’ve implemented imap, vmap, inoremap, etc so they can be set from the command line. I’ve also added the <space> special key for use in mappings, so that you can do e.g. imap a<space>b c<space>d to specify a mapping containing spaces from the command line.
  • Implemented “g~” (“toggle case” with motion).
  • A bunch more fixes for t/T/f/F to bring them more in line with Vim’s behaviour, especially when there aren’t enough characters to find.
  • Made ‘r’ (“replace single character”) counted, so it can actually replace several characters.
  • Made “\C” (“case-sensitivity marker”) work in the emulated command-bar search: this marker is used to specify that the current search should be case-sensitive (handy if you want a case-sensitive search for an all-lower-case search term) after which it is stripped out and ignored completely.
  • Made “//” and “??” (“repeat last search forward/ backwards”) work in the emulated command bar, and also allow the “/e” marker (“place the cursor at the end of the match”), although this latter is a little buggy at the moment.
  • Added fix for the usage of Alt-0, Alt-1 etc shortcuts in normal mode; this is for use with KDevelop’s automatic refactoring support which relies on these shorcuts (e.g. “Rename [variable] to [newvariable] Alt-0 | Hide Alt-1″, etc).
  • A bunch of fixes for J (“join lines”), including making it work in Visual Mode when the selection runs from top to bottom; not crashing if all lines are empty; fixing countedness (it would join one too many lines); cursor positioning after the join; etc.
  • Fixed occasional bug where counted inserts e.g. 5ixyz<esc> would start each repetition on a new line.
  • Implemented iW and aW (“inner/ a WORD”) text objects.

… and then on to:

Making the Emulated Command Bar Suitable for Default

The emulated command bar has a lot of advantages over the default Find dialog and command-line for Kate Vim users, but there are drawbacks, too: firstly, it can only perform regex searches and not literal searches, which is awkward when searching for complex expressions pasted from your document that may contain characters that have a special meaning when interpreted as regex’s (e.g. “*”) or which have special meaning in a Vim search bar (e.g. “/” when searching forward, and “?” when searching backwards). The necessity of going through the search term and escaping all such characters is a big strike against its use.

Secondly, Vim’s Find/ Replace dialog has most of the same flaws, when used in Vim mode, as its Find dialog has: in fact, incremental Find & Replace doesn’t work at all with Vim mode(!), and the emulated command bar is of no use here.

Since I’d really like to make the emulated command bar the default (and possibly only) option for use in Vim mode, I really needed to sort out a solution for these two dealbreakers.

For the first, there are basically two options: one is to add some kind of flag that says “look for literal matches for this string; don’t treat it as a regex!”; the other is to make it easy to perform the escaping of special characters. I eventually opted for the second, as it allowed one to treat portions of an expression as a regex if one desired, plus it allowed one to always use the same delimiters in a sed-replace expression (i.e. :s/find/replace/gci) without having to worry about whether the “find” term contained “/”‘s or not. So I added a shortcut (ctrl-g) to the emulated command bar that behaved much like ctrl-r (“insert contents of the following register”) except that it escapes the inserted register contents in such a way that a regex search for the contents would behave the same as a literal search for the contents. The gif below highlights the problem and the solution; the “:) :) :)” bit used ctrl-g to insert the (correctly escaped) yanked expression:

“Escape Expression for Regex”, 483KB, Click to Play.
For the second problem, improving the good old sed-replace seemed the obvious choice, so I added some code in the emulated command bar to translate the regex in the “find” term from Vim style to Qt style. Kate’s own Find/ Replace dialog maintains separate histories for both the “find” and “replace” terms, so I needed to follow suit if I wanted the emulated command bar to be a credible replacement: if the cursor is placed over the “find” or “replace” terms in a sed-replace expression, pressing ctrl-p will summon the relevant history:

“Sed Search/ Replace History”, 259KB, Click to Play.
Kate’s own Find/ Replace dialog makes it easy to focus on the “find” and “replace” fields, so I also added shortcuts “ctrl-d” and “ctrl-f” to the emulated command bar that clears and places the cursor at the “find” and “replace” portions of a sed-replace expression, respectively. As an extra bonus, ctrl-space can be used anywhere in the emulated command bar to complete words from the document, which is handy when filling out your find/ replace terms.

Still missing, though, were interactive search and replace; multi-line search; usage of \U and \L in the “replace” expression to make regex captures upper/ lower cased; etc. Multi-line search and \U and \L codes are available in a class called KateRegExpSearch, so I initially pulled out the guts of the SedReplace code and replaced it with that, and then made the whole thing interactive:

“Sed Multi-Line”, 399KB, Click to Play.
(in the gif above, I constructed my regex with the standard “/” search bar so I could get immediate feedback as I am a giant regex baby who still needs his stabilisers; this final regex was then added to the “find” history, which I then added to the sed-replace search expression with a ctrl-d + ctrl-p).

The emulated command bar now seemed to me to be a decent alternative to Kate’s own Find/ Find & Replace dialogs, so with that out of the way, I then moved on to …

Macro Support!

The lack of macro support is what got me into Kate Vim hacking in the first place, so it seems weird that it ended up being almost the last thing I tackled. The reason, of course, is that macro support would be a bit crippled if we couldn’t run searches or execute commands in our macros, so I really needed to finish the emulated command bar first. It ended up being quite easy in the end; the main difficulty was in interacting with mappings, especially the situation where a mapping triggered a macro which in turn needed to execute another mapping! Anyway, here’s a small, real-life example of a Kate Vim macro in action:

“Macro”, 300KB, Click to Play.
One difficulty with macros comes from recording and replaying auto-completions. In Vim itself, macros are “dumb” sequence of keypresses that are recorded and replayed verbatim, including those that summon, navigate and execute lists of completions. In Kate Vim, though, this approach doesn’t work so well, partly because the available completions at any time are less predictable (e.g. we could have a function appear in the list of completions when we record the macro, but not when we replay it, due to e.g. problems with the C++ parser etc) and also because, in KDevelop at least, the added completions are context-sensitive: for example, a completion of a function will add its own opening bracket if there is not one immediately after the cursor position, but will re-use the existing one otherwise. The “repeat last change” runs into a similar problem and has some measures to mitigate this, but is still far from fool-proof. The gif below illustrates the problem:

“Old Completion”, 184KB, Click to Play.
In what probably represents the worst effort:payoff ratio in all of history, I spent a while re-working the macro recording and playback system to store attributes of completions as they occur, and then attempt to play them back using the same context-sensitive system; the “repeat last change” mechanism was re-worked to also use this system, so I’ll use that to demonstrate:

“New Completion”, 176KB, Click to Play.
 

That’s about it for macro support; now back to the small changes!

  • Fixed regression where opening a view and immediately giving it a selection (for example, by clicking an entry in KDevelop’s grepview) didn’t switch to Visual mode, leaving us with a “stuck” selection that we needed to manually enter & exit Visual Mode to clear.
  • Got rid of the baffling “No more chars for bookmark” message every time we set a breakpoint in KDevelop ;)
  • Fixed crashes with guu & gUU on empty line.
  • Implemented command-mode mappings (cmap etc) for use in the emulated command bar; this was prompted by my preference for using “ctrl-e” instead of “ctrl-g” for the “insert escaped register contents” (‘e’ is right next to ‘r’ on my keyboard) and for “ctrl-m” instead of “ctrl-e” (“jump to the end of the command bar”).
  • Implemented the *unmap (vunmap, iunmap, cunmap etc) family of commands for removing mappings from the command line.

… and that’s about it! I’m going to take a break from Kate stuff for a while to work on some other projects I’ve been neglecting, but will return to finish off some odds and ends later :)

The Kate Vim mode is in pretty good shape nowadays, so if you were thinking of trying it out, now (with the current master) would be a good time :)