[Request] Input Grabbing API

[Ph4ntomas]

TL;DR Add a way to redirect all input events to the config for processing, instead of only bound inputs.
My main motivation is for modal behaviors, but it could be used for cli/widget inputs

Rationale

I'm a huge user of modal behavior in software, and not so much a fan of multi key combo.

Anatomy of a Mode

A Mode is usually started by a keystroke/keycombo, and is quite similar to the concept of 'binding layer' already present. Where they differ is in the presence of arbitrary action modifiers (not to be confounded with key modifiers), that can alter the action being performed.

Modes can be thought as an arbitrary string of inputs in the form (this is heavily simplified):
/M?AmT/ or /M?TmA/
M -> enter a given mode (optional if already in that mode)
A/T -> Action & Target
m* -> modifiers. Could be target modifiers, repetition modifiers, etc.

As an example, here are some of the way I usually interact with my WM:
normal mode (act on focus & clients)
/\d*[hjkl]/ -> move focus in direction (hjkl), stepping over \d* clients
/\d*[HLg]/ -> focus tag by direction (HL) stepping over \d* tag, or explicit tag focus when using g
/s\d*[hjkl]/ -> focus Screen in direction (hjkl), stepping over \d+ screen
/m\d+[hlg]/ -> Move client to Tag by direction|globally.

Existing state

Pinnacle's binding layers only partially address this use-case. While its a good start for basic interactions, you can't encode action modifiers (at least not easily, see Workaround section).

Enhancement

I'm don't think the compositor should handle all of that, but instead expose an API to request for all input to be redirected to the config/client, upon which we can build things.

While the modal behavior is my primary concern, the same facilities would also enable rudimentary text inputs for (e.g.) a pseudo cli to start processes without having to open a terminal/menu. (as an example, awesomewm has a ctrl+x binding by default).

This would not replace bindings either. The idea would have binding as they exist now, intercepting inputs with the highest priority, then the active grabber, then the windows.

Workaround

This should already be somewhat achievable through the use of layers. While I haven't tried it yet, you could in theory bind every inputs possible, and process that in a handler for a given layer.

I'm not sure however that you could cleanly have both keybinds and the pseudo grabber at the same time.
The other issue with that workaround is that it adds extra delay

[Ottatop]

I've added an experimental API for input grabbing in e0c71e6. Basically we're just (ab)using the fact that exclusive overlay layers steal keyboard focus and spawning an invisible one with Snowcap. There's a small example in the doc comments if you'd like to test it out. I believe it currently doesn't show up in the Lua docs, should be under require("pinnacle.experimental").input_grab.

[Ph4ntomas]

Oh thanks !

I was planning on working on that later, but you beat me to it. I'll give it a try later today.

Wrt the implementation, I believe there should be a way to prevent input stealing by another surface.
(The spec says it's implementation defined which surface get the input when more than one are requiring it).

As an example workflow where there could be an issue:
-> setup input grabbing for modal behavior
-> setup a second one for a prompt, and disables/pause the first one
-> from the prompt start an application that may request exclusive input
-> restart the grabber because the prompt is done

The second step should not cause issues (i.e. when starting a prompt, you want to stop the current grabber anyway), but I feel step 3/4 might.

I'd rather have something well defined in case the started app also request exclusive input at some point, if only to prevent weird race conditions (imo a snowcap grabber should always be given priority over an application).

Anyway, that's theoretical and can implemented later imo, I'll first check the current implementation

[Ph4ntomas]

Hey, did a small test, for now just a prompt. To have visual feedback, I've added a quick & dirty redraw function on the Layer object :

function LayerHandle:redraw(widget_def)
    local layer_id = self.id

    if not widget_def then
        return
    else
        client:snowcap_layer_v1_LayerService_UpdateLayer({
            layer_id = layer_id,
            widget_def = widget.widget_def_into_api(widget_def),
        })
    end
end

Then I call it like this:

function RunPrompt:show()
   local Layer = require("snowcap.layer" )
   local prompt = Layer.new_widget({
       program = self,
       anchor = nil,
       keyboard_interactivity = Layer.keyboard_interactivity.EXCLUSIVE,
       exclusive_zone = "respect",
       layer = Layer.zlayer.TOP,
   })

   if not prompt then
       return
   end

   prompt:on_key_press(function(mods, key)
       self.content = self.content or ""

       if key == Input.key.Escape then
           prompt:close()
       elseif key == Input.key.Return then
           Process.spawn(self.content)
           prompt:close()
       else
           self.content = self.content .. utf8.char(key)
           prompt:redraw(self:view())
       end
   end)
end

IMO the object returned from Layer.new_widget should either have the args stored, or at least the 'program', so we can call redraw without parameters.

@Ottatop I can clean that up and apply the same idea on the Rust side. Let me know if you have a better idea :)

[Ph4ntomas]

After playing a bit more with that, it works rather well for modal input (I will need to build a cleaner framework before sharing that code/make a full setup with it),
It's a bit weird to use as a prompt however, mainly due to the lack of cursor and key repetition. It' s still usable though, so that' s nice !

I'm also wondering if we could make use of the 'on demand' exclusivity (on widgets in general) so the grabber can be set-up once, and enable/disabled at will. Maybe through a focus()/unfocus() set of RPCs on the Layer.

Side note, in building the prompt widget to get feedback, I was expecting the top layer with exclusive zone to shrink my window, it seems not to be the case. Did I miss something ?

[Ottatop]

I'd rather have something well defined in case the started app also request exclusive input at some point, if only to prevent weird race conditions (imo a snowcap grabber should always be given priority over an application).

Agree, I was thinking we could set the layer's namespace to some special string and have Pinnacle auto-focus any layer with that string.

IMO the object returned from Layer.new_widget should either have the args stored, or at least the 'program', so we can call redraw without parameters.

Ah, this is the send_message method for DecorationHandles, which triggers the update method on the program along with a redraw. I think I forgot to add it for layers, whoops.

It's a bit weird to use as a prompt however, mainly due to the lack of cursor and key repetition.

Iced has text input, just need to add it as a widget. Though I suppose that makes input grabbing moot for prompts.

Maybe through a focus()/unfocus() set of RPCs on the Layer.

I'm open to this idea, but we'll have to add a new layer API to the pinnacle side and translate between snowcap and pinnacle LayerHandles.

Side note, in building the prompt widget to get feedback, I was expecting the top layer with exclusive zone to shrink my window, it seems not to be the case. Did I miss something ?

For a layer to shrink the working area, it needs to be both anchored to an edge and have a non-zero exclusive_zone. You want to change anchor = nil to anchor = Layer.anchor.<whatever_edge_here> and exclusive_zone = "respect" to exclusive_zone = <some_number>.

[Ph4ntomas]

Agree, I was thinking we could set the layer's namespace to some special string and have Pinnacle auto-focus any layer with that string.

Make sense. Maybe if there are more than one we should focus the one on top/last one to get focus.

Iced has text input, just need to add it as a widget. Though I suppose that makes input grabbing moot for prompts.

That remain to be seen. While the look&feel of text input (+key repetition) would be nice, it's still useful to be able to handle input key by key for stuff like history management and autocompletion when hitting tab.
Plus the widget has to be keyboard focusable.

Depending on what's possible, the text input might just act as a 'presentation' helper.

(Haven't checked, this is pure speculation)

But yeah it might work a bit differently.

Ah, this is the send_message method for DecorationHandles, which triggers the update method on the program along with a redraw. I think I forgot to add it for layers, whoops.

Gotcha, then I'll keep the same name for consistency. Small nitpick, but I'd never have guessed send_message was calling the update function (but to be fair, I did not read the doc this time around) That's fully consistent with iced doc, I should've read the doc

[Ph4ntomas]

Iced has text input, just need to add it as a widget. Though I suppose that makes input grabbing moot for prompts.

That remain to be seen. While the look&feel of text input (+key repetition) would be nice, it's still useful to be able to handle input key by key for stuff like history management and autocompletion when hitting tab.
Plus the widget has to be keyboard focusable.

Re prompt, since I'm now implementing TextInput and playing with it at the same time.

I confirm the input grabbing facility is needed if we want to do anything fancy since the text-input widget only handle 'text' (i.e. displayable characters), unless we want to implement our own iced widget (we might want that at some point, but that's beyond the point).

[Ph4ntomas]

It'd be nice to have the KeyPressed text field sent to the layers on_key_pressed callback.

The way it's setup right now, I can't easily use quotation mark (amongst other) because I'm using an us(intl) mapping.

It would also be nice to have the keyrelease event as well.

(I will do these changes, but I'm documenting these right now)

[Ph4ntomas]

Yet another TODO, but I don't know if it should be handled server side:
Since we're using layer with keyboard exclusivity, pinnacle.window.get_focus() no longer returns the focused window.

An easy workaround is to temporarily pause the input grabbing when we want to interact with the api (which I'll add in my utility lib). Since this is easy to implement a workaround, I don't really mind for my usecase.