Listen to user input using the “Raw Input API” to manage the background application.

Perhaps, there are almost no people left who do not know what Ctrl + C and Ctrl + V are. More experienced users know the hotkeys of frequently used applications. There are those who use more complex combinations: for example, to control the player in the background. For developers, the implementation of such functionality usually does not cause great difficulties, since This problem is widespread, and much has already been written about its solution. But what if you need to listen to user input from a joystick or a presenter in a minimized state, and also understand which device came from what particular device? Let's be honest, for us this task turned out to be something new. Under the cat, we will describe how we solved it in C # in a WPF application using the " Raw Input API ".

Prehistory

Our application Jalinga Studio is used to shoot video, webinars and online broadcasts, and they are managed primarily with the help of a presenter or a joystick (why we needed this, you can read in our previous article “How we animate the presentation” ). Most presenters are made to work with Power Point, so they generate the usual keyboard input: F5, Page Up, Page Down, etc. WPF has a standard mechanism for working with keyboard input. We used them at first and used them until we came across a significant drawback for us. The fact is that this mechanism works only when the application is active (is in the foreground), but some of our clients would like to have parallel access, for example, to a browser or other program, which by no means prevents us from receiving keyboard input. At first we tried to work around this problem by creating an additional small window in the foreground, similar to how it was done in Skype. This window displays the status of the program and several buttons for controlling if it is more convenient for the user to control the mouse. This approach was not the most convenient - the control window must be activated. If the user forgot to switch focus, the keyboard input from the presenter went away to the current active application. For example, F5 or Page Down in the browser. On top of that, at some point we started to miss the buttons on the presenter, and we decided to use joysticks that the standard WPF mechanism does not support.

Finding a solution

First, we formulated the requirements for the new mechanism:

• getting information about user input if the application is running in the background;
  
  
• the ability to work with presenters, joysticks, gamepads;
  
  
• there is a way to distinguish between input devices.
  
  

The first thing that came to mind was the hooks that can be delivered using the SetWindowsHookEx function. But here it is still an open question to support joysticks and gamepads. I am silent about antiviruses that can take us for keyloggers, the influence of hooks on the work of other programs, the creation of 32-bit and 64-bit dll-ek, interaction with our application from another process and the overall complexity of support.

')

Considered the option of using DirectInput or XInput . DirectInput is deprecated, Microsoft recommends using XInput instead . Using them, you can receive user input from joysticks even in the background if you set the Background flag using the SetCooperativeLevel method. But XInput does not support keyboards and mice. I still didn’t like the pull-use model, because of which I’ll have to interrogate the devices of interest with some frequency.



They continued to dig further. One good friend from Parallels advised to look in the direction of the " Raw Input API ". After analyzing the capabilities of this API, we realized that everything we need is there - working with various devices of the HID class, and the ability to receive input without being an active window, and the available device ID from which the event came. There are also limitations - if the input is carried out in the admin process, and our process is not admin, then the input events will not come. But we do not need it. In a pinch, you can always run our application with administrator rights.

Implementation

Generally, the process of getting user input using the Raw Input API consists of the following steps:

1. We register the types of devices from which we will receive input events using RegisterRawInputDevices .
   
   
2. We listen WM_INPUT events in a window procedure .
   
   
3. Parse incoming events using GetRawInputData .
   
   
4. We determine the type of event (RAWMOUSE, RAWKEYBOARD, RAWHID) and parse it according to its type.
   
   

All this sequence needed to be implemented on C # in WPF application. In order not to write independently a large number of Win API-shnyh functions, it was decided to use SharpDX.RawInput .



This is what a simplified C # code looks like if you use Windows.Forms:

public class RawInputListener { public void Init(IntPtr hWnd) { Device.RegisterDevice(UsagePage.Generic, UsageId.GenericGamepad, DeviceFlags.InputSink, hWnd); Device.RegisterDevice(UsagePage.Generic, UsageId.GenericKeyboard, DeviceFlags.InputSink, hWnd); Device.RawInput += OnRawInput; Device.KeyboardInput += OnKeyboardInput; } public void Clear() { Device.RawInput -= OnRawInput; Device.KeyboardInput -= OnKeyboardInput; } private void OnKeyboardInput(object sender, KeyboardInputEventArgs e) { } private void OnRawInput(object sender, RawInputEventArgs e) { } } 

The DeviceFlags.InputSink flag is needed in order for an application to receive a message, even if it is not in the foreground. When using this flag it is necessary to specify hWnd.



If you use WPF, then the OnRawInput and OnKeyboardInput methods will not be called in this form, since Inside the Device class, the IMessageFilter interface from Windows.Forms is implemented. If you look at the source code for the Device , you can see there that HandleMessage is called in the PreFilterMessage method.



Simplified C # code if you are using WPF:

 public class RawInputListener { private const int WM_INPUT = 0x00FF; private HwndSource _hwndSource; public void Init(IntPtr hWnd) { if (_hwndSource != null) { return; } _hwndSource = HwndSource.FromHwnd(hWnd); if (_hwndSource != null) _hwndSource.AddHook(WndProc); Device.RegisterDevice(UsagePage.Generic, UsageId.GenericGamepad, DeviceFlags.InputSink, hWnd); Device.RegisterDevice(UsagePage.Generic, UsageId.GenericKeyboard, DeviceFlags.InputSink, hWnd); Device.RawInput += OnRawInput; Device.KeyboardInput += OnKeyboardInput; } public void Clear() { Device.RawInput -= OnRawInput; Device.KeyboardInput -= OnKeyboardInput; } private IntPtr WndProc(IntPtr hWnd, int msg, IntPtr wParam, IntPtr lParam, ref bool handled) { if (msg == WM_INPUT) { Device.HandleMessage(lParam, hWnd); } return IntPtr.Zero; } private void OnKeyboardInput(object sender, KeyboardInputEventArgs e) { } private void OnRawInput(object sender, RawInputEventArgs e) { } } 

To find out which device the event came from, you can use the Device property of the RawInputEventArgs class, with which you can get a DeviceName of the form:

 "\\?\HID#{00001124-0000-1000-8000-00805f9b34fb}_VID&000205ac_PID&3232&Col02#8&26f2f425&7&0001#{884b96c3-56ef-11d1-bc8c-00a0c91405dd}" 

You can find the Vendor ID and Product ID in this name or use the GetRawInputDeviceInfo or HidD_GetAttributes functions . You can read more about VID and PID here and here .



We now turn to the analysis of events. With the keyboard, everything turned out to be simple: the information already comes disassembled, described in the KeyboardInputEventArgs class. But with a gamepad, everything turned out to be more difficult. In OnRawInput comes the argument of the base type RawInputEventArgs. This argument must be cast to the HidInputEventArgs type and, if it works, continue to work with it. In HidInputEventArgs there is only an array of bytes coming from the device, and the number of bytes in this array is different for different joysticks and gamepads.



Unfortunately, we managed to find little documentation that tells how to parse this data, and that was usually found in a brief form (even in MSDN, there were continuous understatements on this issue). The most useful was this project in C. He had to first bring to working condition, but it was already a great start. After it was necessary to transfer the necessary parts to C #.



First of all, I had to wrap the native functions to call them from C # code. Here, as always, pinvoke.net helped, something needed to be described most. You can read Marshal, PInvoke and unsafe C # code here .



The next step is to transfer the message parsing algorithm, which is as follows:

1. get PreparsedData devices ( GetRawInputDeviceInfo or HidD_GetPreparsedData );
   
   
2. learn about device capabilities ( HidP_GetCaps );
   
   
3. learn about device buttons ( HidP_GetButtonCaps );
   
   
4. get a list of pressed buttons ( HidP_GetUsages ).
   
   

Below is a part of the data parsing code from the gamepad, which at the output produces a list of pressed buttons:

 public static class RawInputParser { public static bool Parse(HidInputEventArgs hidInput, out List<ushort> pressedButtons) { var preparsedData = IntPtr.Zero; pressedButtons = new List<ushort>(); try { preparsedData = GetPreparsedData(hidInput.Device); if (preparsedData == IntPtr.Zero) return false; HIDP_CAPS hidCaps; CheckError(HidP_GetCaps(preparsedData, out hidCaps)); pressedButtons = GetPressedButtons(hidCaps, preparsedData, hidInput.RawData); } catch (Win32Exception e) { return false; } finally { if (preparsedData != IntPtr.Zero) { Marshal.FreeHGlobal(preparsedData); } } return true; } } 

PreparsedData is easier to get using GetRawInputDeviceInfo , because The device handle you need is already in RawInputEventArgs. The HidD_GetPreparsedData function does not accept this handle, it requires a handle that can be obtained using CreateFile .



To get the values ​​of discrete buttons, the procedure is similar, but instead of HidP_GetButtonCaps, you must first call HidP_GetValueCaps , and then HidP_GetUsageValue to get the discrete value of the button.



Now, when the set of bytes from HidInputEventArgs is converted into data about which buttons are pressed, you can make a mechanism similar to that in WPF for working with the keyboard and mouse.



The full code of the application that parses user input from gamepads and keyboards can be found in the RawInputWPF project on GitHub.

Total

In this way, using the "Raw Input API", you can get user input from the keyboard, mouse, joystick, gamepad, or other user input device, even if your application is in the background.



And what to do with the data on the pressed buttons, you decide.