Terminal access to the Caché DBMS - now in the browser

With the development of web technologies, more and more useful services, applications, programs and even games appear in the browser window. The time has come for the Caché DBMS terminal .

Under the cat you will find a description of all the charms of the application and the history of its development.
')

Functionality

The web terminal can do the following:

• Execution of arbitrary code and Caché Object Script commands, terminal utilities and programs
• Convenient SQL mode for quick access to the database
• Auto-completion of Caché Object Script keywords: classes and their methods, properties, parameters, globals and variables in the current namespace
• Monitoring changes in globals and files (like tail -f)
• Command history
• Caché Object Script syntax highlighting
• Definition of acronyms
• Multiline editing
• Customize the behavior and themes of the application

All of the above works on any Caché server where WebSockets support is present. Just follow the link to the web application and get started.

A detailed description of all the features you can find on the project page .

Security

Naturally, the main goal of any web application should be its security. Since communication with the server occurs through an open port, the latter needs protection. The first thing that the terminal port requires you to do when entering a connection is a key of arbitrary length. The server, receiving the wrong key, immediately disconnects. Now the GUID is used, which is generated each time when the main CSP page is accessed or in case of unsuccessful connection to the open port. That is, every time an attempt is made to connect to a socket with the wrong key, a new one will be generated, and so over and over again, which makes it impossible to select it. The same thing happens when you call the CSP page - only now the key is given to the client and is used to establish a connection with the socket.

Simply put, it remains only to install authorization on the CSP page, thereby not giving the opportunity to get the key to unwanted visitors of the open port.

The history of software implementation

If you look at the terminal from the side, it may seem to someone that its mechanics is quite simple. At first I thought so too. Understanding the behavior of such applications, along with acquaintance with Caché, there were many interesting moments and difficulties that will be discussed further. The main task was to make the terminal familiar for geeks and at the same time friendly for ordinary users, providing new features and finally to embellish the black and white graphical interface of a regular terminal, because the window is already 2013!

Further I will describe mainly reprisals with a rake, which I had to stumble upon and the very ones that all designers-programmers who were familiar with the web or Caché looked like. Maybe the following methods and solutions can be made more interesting, and if you know how - it will be very interesting to hear.

Starting to create a new software product you need to think about, and how it all should work. Since I was just getting acquainted with Caché, at first I thought that the terminal was functioning quite simply - all that was needed was to execute commands on the server and read the answer from it. I didn’t worry about the front-end at all, since I had previously worked with it. But as soon as I began to delve more and more into development, it turned out that it was necessary not only to execute commands, but also all terminal utilities, set up reading information from the client (for example, when processing the read command), achieve streaming I / O and decide some more small tasks.

After some thought with colleagues, it became absolutely clear that the WebSocket protocol would be used for data transfer - a relatively new, stable and reliable way to transmit data via the web. And on the client side, of course, all the charms of HTML5. This and CSS-animation, and clean-transparent JavaScript.

First of all, sitting down at the client side, I did not want to use ready-made solutions or use any frameworks, since the terminal, ideally, is a lightweight application that does not need advanced access to DOM like JQuery and various magic JavaScript animations. No, in fact, the animations could have been useful, but in the format only a pack of CSS3 properties, no more. The application promised to come out easy, both for the browser and for the user.

I didn’t bother with displaying information for a long time - it’s a monospaced font, light block layout and familiar styles. But over the field to enter information I had to think: it was necessary to implement syntax highlighting. Many solutions with plugins were discarded, so the last option, it seemed, was a concise editable HTML5 div. But even there they found their own charms - the content needed to be transferred from HTML to plaintext and vice versa, to set and get the position of the carriage, to copy the original, not decorated text into the buffer - these problems are not solved in a few lines. In addition, you also need to insert your own, flashing terminal carriage and implement the usual system Ctrl-C, Ctrl-V, Right click + Paste, Insert, ...

The implementation of the “backlight and carriage” in the input field was very tricky and simple. In fact, we only need to highlight the text, well, insert the caret. Suppose we have plain and highlighted text. The first one is contained in the most typical textarea, and the highlighted one is in a block of exactly the same size. Catch? So simply using several CSS tricks we make the input field transparent with a transparent font, under which we place a block with highlighted text. The result is a visible selection of the text and complete freedom to edit it. Here only Internet Explorer, as always, distinguished itself - the carriage in it still remains visible when it is transparent in other browsers. Therefore, the usual flashing terminal carriage had to be abandoned in it - let it remain with its own, dear one.

An interesting moment also arose with the processing of keystrokes - it was necessary to highlight the input data, which means that, by pressing, to process the contents of the input field. Yes, but it’s impossible to get the content of this field at the moment of clicking (more precisely, but without the last “pressed” sign) - the DOM does not have time to update before the keydown, keypress events, and it’s not interesting to update the keyup, though also another way out. The second way out is to add the character to the string manually. But the latter immediately disappears in the case of Ctrl + V. Let's make the third method - let's call the click handler function 1ms after the click itself. Yes, now we have received input, but the ability to control the event passed to the handler has disappeared, for example, to disable the default key action. The output was a breakdown of pressing into two events - by pressing the processing of the event and combinations itself, and after 1ms - updating the entered text.

Input parsing, syntax highlighting and inserting carriages into the form were easy to implement - first you need to replace what can spoil the HTML formatting, namely the characters "<", ">" and "&" with the corresponding "<", ">" and "&". Then - perform the syntax highlighting by ultra-regular expression (which, in fact, inserts only tags into the text) and only then insert the caret, defining its “present” position (without taking into account tags and HTML entities), for which more one method. Yes, all of the above is done only in this order, otherwise either the carriage itself will be highlighted, or a lot of broken HTML markup will appear.

But with autocompletion it was interesting to work. I rewrote it three times. And the progress of the algorithm was as follows:

1. Just get a piece of string from the carriage to the nearest left separator or space and look for matches with the available options in the array of all options.
2. The same piece of string, already including, possibly, the characters "$", "%", "##" and others to determine the type of add-ons we are looking for in a special object, divided into "categories".
3. Parsing of the entire left part of the carriage by “masks” - inverse regular expressions, which are contained in a specially structured object of the “terminal dictionary”.

I don’t know if anyone will find the third method familiar to which I gradually approached, but it was he who showed the smartest and fastest results.

So, how does it work? Very simple. All that is needed to create almost any type of autocompletion is correctly composed regular expressions in the “dictionary” object. Here is what it might look like:


Each object inside a language can have a special property-object "! Autocomplete". If it is present, the auto-completion parser will pay attention to this object, namely, to read its reversedRegExp and child properties.

As it was already possible to guess, reversedRegExp is compiled in a special way, and it is he who determines whether it is appropriate to use the properties of the current “vocabulary” object (hereinafter - just a “dictionary”) for autocompletion. Retention brackets in a regular expression serve to select a part of the searched string, which will be checked against the names of dictionary properties (“terms”). This allows you to find in any syntactic structure the key by which the available options will be selected.

With classes, the task is a bit different - you need to get the name of the class and suggest its corresponding properties. This was solved by adding the property “! Autocomplete” to a similar property object “child”, which also contains a reversedRegExp - a prefix to the parent regular expression, which will be considered when the latter matches. The verification algorithm is quite simple. If you are interested in how this algorithm works, you can find it inside the project repository.

The advantages of this approach are obvious - this is the visual structure of the dictionary of all syntactic constructions, and a fairly smart way of auto-completion, which, if desired, can be expanded in every possible way. Yes, the number that goes as the value of the property of a “term” is its supposed “frequency of use”. It is by this figure that the proposed options will be sorted.

On the server side, the entire dictionary of autocompletion of classes and methods of the current namespace, in turn, is generated and stored in a JSON file containing an object of objects that, if necessary, will be loaded and “merged” with the class dictionary object on the client within the main dictionary object. This is how it is.

The server itself was previously taught to send the entire write immediately to the client, using this thing. But for read, as it turned out, doing something simple like “+ T” will not work. The problem is that when the user tried to execute either a terminal utility, or compose a script with a rev read server, processing them in xecut simply hung or corrupted the input data.

Well, let's say we put the terminal in the processing mode of standard terminators at the input (“+ T”), and we will send them from the client. Ok, now read does not freeze, but a different situation arises - we after all read the package received from the client, and not its body. The packet itself contains some “garbage” for us - these are the first few bytes that serve as its header. They needed to be dropped somehow.

To make it easier to imagine what is needed and how it should all happen, consider the intended sequence for executing the “read a write a” command on the server.

• Receiving a team from a client
• The transition of the terminal application in the execution mode - setting input / output "directly"
• Pass command to xecute
• read a : read the data from the client, ending terminator
• We take the body of the read package and place it in a
• write a : send the value a to the client
• Completion of execution mode

But how can you also get this body so that the extra bytes (the header of the WebSocket package) do not fall into a ? Logically, you need to make some kind of read handler. And not simple, but at the system level: after all, terminal utilities also use read.

Fortunately, experienced guys from the sql.ru forum suggested a wonderful undocumented Caché opportunity - I / O redirection. With its help, it was possible to do exactly what was needed - to process all the arriving / departing data in its own way. Input redirection is done by writing seven subroutines that will take over the functions of the primitive read and write commands with the ## class (% Device) .ReDirectIO enabled. If you are interested in the detailed implementation of this charm, this thread can be useful to you.

I hope the experience outlined above, someone will definitely come in handy and will be no less useful than the terminal itself. On the way from the concept to the already functional application, many new ideas have appeared, and this is not yet the limit - here you can limit yourself only to imagination. Follow or participate in the development of the project on GitHub , suggest and discuss ideas, any of your feedback will be helpful. Enjoy your administration!