Simplify user action log

In previous articles we have done a great and good deed - we learned how to automatically collect the background of the fall of the program and send it with crash report. For WinForms , WPF , ASP, Javascript . Now we learn to show all these mountains of information in a digestible form.

It would seem that what is still missing, sit carefully and read the log, everything is written in sufficient detail. Even too detailed, perhaps. Directly debug log of some kind. The same thing is certainly useful, but we wanted something different. We wanted to know what the user was doing before the program crashed. Steps to play. How will a person describe in a letter to the developer his actions in human language? Probably something like:


«Edit Something»
«Edit»
«Name»

Enter
-

What about us? In fact, it seems to be the same, but it is replete with a mass of “redundant” details: the mouse is clicked, the mouse is released, the button is pressed, the character is entered, the button is released, and so on.

Let's try on the server side to convert the logs to a more convenient form, just before showing on the page. We will implement several “simplistic” ones, which we will set on the source log one by one, and see what happens.
')
Let's start with the mouse.
This is what the double-click with the left mouse button for the WinForms platform looks like in the log:



It looks simple - we are looking for a sequence of events down / up / doubleClick / up in the log and replace it with one doubleClick entry. At the same time, it is necessary to make sure that in the sequence found all the events have the same mouse button (the left one in our example), and that all events occur over the same window. Also, in order to avoid misunderstandings, it is better to check that the coordinates of all four events do not go beyond the square by the size of SystemInformation.DragSize . We will do this on the server, and DragSize should be used in a good way from the client machine. In reality, even if they manage to catch a situation in which they differ, this will have little effect. Therefore, we deliberately neglect this and use fixed values.

Mouse coordinates will have to be sent from the client:

 void AppendMouseCoords(ref Message m, Dictionary<string, string> data) { data["x"] = GetMouseX(m.LParam).ToString(); data["y"] = GetMouseY(m.LParam).ToString(); } int GetMouseX(IntPtr param) { int value = param.ToInt32(); return value & 0xFFFF; } int GetMouseY(IntPtr param) { int value = param.ToInt32(); return (value >> 16) & 0xFFFF; } 

The following "simplifier" will deal with single clicks:



All the same as for a double click, only easier and shorter. The only thing that should not be forgotten is that you first need to process all double clicks, and already in what happened, look for single ones.

Let's go to the keyboard input.

A single alphanumeric key press:



At first glance, everything is the same as with the mouse, what problems could there be? We have 3 events recorded here: WM_KEYDOWN , WM_CHAR and WM_KEYUP . What do they have in common to understand that all 3 entries relate to one event - pressing the F button? But they have in common only scan codes in bits 16 to 23, since WM_KEYDOWN and WM_KEYUP contain a Virtual Key Code , which does not depend on the current input language. But WM_CHAR contains an unicode character that corresponds to the key pressed (or key combination, or IME input sequence in general). The scancode must also be sent from the client, while not forgetting to hide it when entering passwords:

 void ProcessKeyMessage(ref Message m, bool isUp) { // etc data["scanCode"] = maskKey ? "0" : GetScanCode(m.LParam).ToString(); // etc } int GetScanCode(IntPtr param) { int value = param.ToInt32(); return (value >> 16) & 0xFF; } 

Well, turn off the gas, remove the kettle from the stove, pour out the water and reduce the problem to an already solved one, now you can, as for the mouse, roll the triples of events into one. “Yeah, so big, but you still believe in fairy tales!” Said the reality:



It turns out that with speed dialing, keyUp may not be so far behind the corresponding keyDown and keyPress. Therefore, just look for 3 consecutive records of a certain type will not work. It is necessary to find the corresponding keyPress from the found starting point (keyDown) and then keyUp. To the end of the list to search? So it immediately quadratic complexity turns out, hello brakes! Yes, and keyPress events may not be at all if a non-alphanumeric key was pressed. Therefore, we will look no further than a fixed number of records from the initial one; we will choose the number based on common sense and correct it if necessary. Let it be 15, 2-3 entries per key, this will mean 5-8 simultaneously pressed and unstressed keys. For normal "civil" input is more than enough.

So, individual keystrokes turned up before entering individual characters (type char). Now you can try to combine the inseparable type char sequences into one large type text. Although about the continuity, I, perhaps, got excited - before, inside and after the type char sequence it is quite possible for myself to press and release the Shift keys, which also needs to be taken into account when folding the chain of events.

With alphanumeric input everything, now you can minimize the remaining keyDown / keyUp pairs, there are no big surprises there.

What we still have there? Focus.

Change focus with the mouse:



We are looking for and replacing the top three records with one, we control the coordinates of the mouse.

Change focus from keyboard:



It is very similar to the mouse, but it is important not to pin up on the fact that keyDown comes to the window that loses focus, and keyUp to the window that receives it.

Now let's try to put it all together and see.

It was:



It became:



Feel the difference, as they say. The information is the same, but it is framed much more compact and readable more naturally.

In the first approximation with WinForms finished. Go to WPF.

To a certain extent, WPF is similar to WinForms, and almost all of the above rules can be used. But, as we remember from the school mathematics course, in order to solve the problem in a typical way, we must first bring it to this method. Let's take a look at what we have at the entrance:



It can be seen that we have several logs for each action, and the usual mouse down event turns into 4 records, as it is sent to us from each heir Control on the visual tree from the window to the button itself. With a rather complex UI without such a path, it can be difficult to understand exactly where the click occurred. However, such ways are pretty littered with the kind of log that discourages any desire to read it.

As a result of the discussions, we concluded that the optimal solution would be to reduce this list of logs to one, the last and, optionally, allow us to show everything. Why the latter? Because a click message in a button is more useful than a click in a window.

But how to understand that before you the block from the logs relating to one action? And elementary, all events from the same action get the same EventArgs object. Therefore, we simply assign a unique identifier to each object of the arguments and write it to the logs.

 class PreviousArgs { public EventArgs EventArgs { get; set; } public string Id { get; set; } } PreviousArgs previousArgs = null; short currentId = 0; Dictionary<string, string> CollectCommonProperties(FrameworkElement source, EventArgs e) { Dictionary<string, string> properties = new Dictionary<string, string>(); properties["Name"] = source.Name; properties["ClassName"] = source.GetType().ToString(); if(previousArgs == null) { previousArgs = new PreviousArgs() { EventArgs = e, Id = (currentId++).ToString("X") }; } else { if(e == null || !Object.ReferenceEquals(previousArgs.EventArgs, e)) { previousArgs = new PreviousArgs() { EventArgs = e, Id = (currentId++).ToString("X") }; } } properties["#e"] = previousArgs.Id; } 

After that, on the server, we discard all logs related to one event, except the last. We get:



Well, the size of the log has been greatly reduced, and now it’s almost like WinForms, which means you can try to apply the rules described above.

To earn the rules for folding mouse clicks, we lack the collection of coordinates. The coordinates relative to the root element will be obtained using the MouseEventArgs.GetPosition method, and the coordinates relative to the window will be obtained using the Visual.PointToScreen :

 void CollectMousePosition(IDictionary<string, string> properties, FrameworkElement source, MouseButtonEventArgs e) { IInputElement inputElement = GetRootInputElement(source); if(inputElement != null) { Point relativePosition = e.GetPosition(inputElement); properties["x"] = relativePosition.X.ToString(); properties["y"] = relativePosition.Y.ToString(); if(inputElement is Visual) { Point screenPosition = (inputElement as Visual).PointToScreen(relativePosition); properties["sx"] = screenPosition.X.ToString(); properties["sy"] = screenPosition.Y.ToString(); } } } IInputElement GetRootInputElement(FrameworkElement source) { return GetRootInputElementCore(source, null); } IInputElement GetRootInputElementCore(FrameworkElement source, IInputElement lastInputElement) { if(source is IInputElement) lastInputElement = source as IInputElement; if(source != null) { return GetRootInputElementCore(source.Parent as FrameworkElement, lastInputElement); } return lastInputElement; } 

Now we have quite successfully minimized click and double click.

Now take a look at moving the focus. Unlike WinForms in WPF, we use two events: GotFocus and LostFocus, respectively, and we have two log entries. Well, it's okay, just correct the logic, so that when you minimize it, either FocusChanged or a couple of Got and Lost focus events are searched.

In principle, with WPF on it also finished. Unfortunately, we cannot minimize text input, since the TextInput event is completely detached from the keyboard, and there are no analogs of the scan code there. As a result, we managed to achieve this appearance:



Everything is convenient, understandable and much clearer than at the very beginning.

This concludes the breadcrumbs article series. As developers, we will be grateful if you share your opinion about this feature. Or ideas, how else can you simplify the log bredkrambsov to Logify spoke completely human language.

And, of course, this is not the last article from our team. Feel free to ask us what you are interested in and we will answer all your questions. And the most interesting of them may well become the basis for a new article.