NODE.JS + Windows: look inside

An article for node.js programmers who understand the principles of asynchronous event programming, but do not know how it works from the inside. If a standard picture with “looped” circles is no longer enough for you and you want to at least take a look at what’s happening in the cycle of events under the hood, then you are under the cat.

Limitation

For ease of description, I decided to build my article around a simple file open operation, i.e. open functions from the fs module. Naturally, because of this, VERY much will remain outside the article, but you have to compromise between simplicity of perception and technical details.

require('fs').open("c:\\1.txt", 'r', function onOpen(err, result){ console.log("Result: ", result); }); 

IOCP

To understand how Node.js works under Windows, you need to understand IOCP technology. Input / output completion port - a technology designed to perform asynchronous input / output operations used in Windows. The main object in this technology is the IOCP port created using the CreateIoCompletionPort () function. We are primarily interested in the fact that the IOCP port encapsulates an event queue created in the operating system. The PostQueuedCompletionStatus () function places the event in a queue, and the GetQueuedCompletionStatus () function retrieves. Moreover, if the queue is empty, then the thread that caused GetQueuedCompletionStatus pauses until the first event occurs.
')

Initialization

Now, before proceeding directly to our example, consider some moments of node.js initialization. At launch, a special structure is created that describes the event loop , let's call it loop. Among other fields, the structure contains a link to the IOCP port, and an asynchronous request counter. For simplicity, we denote it as an integer variable req_count. When the loop is initialized, the IOCP port is created:

 iocp _handle= CreateIoCompletionPort(INVALID_HANDLE_VALUE, NULL, 0, 1); 

Next comes the launch of the event loop, which we will talk about later.

Open function

Well, finally, the algorithm of the open function itself.
First, a structure describing an asynchronous request is created and initialized, let's call it fs_open_req. It contains a link to the callback onOpen, the path and modifiers of access to the file and other information describing the request. In addition, the structure contains a field for storing the result of the query or a link to it.
Second, the asynchronous request counter in the loop structure is incremented.
Thirdly, a separate stream is created in which the file will be opened. In this case, the main stream node.js, returns from the open function and then goes to the next iteration of the event loop. In the generated thread, the file 1.txt is opened using the operating system tools. Its descriptor is written to the fs_open_req structure.



Fourth, after opening the file and completing all the necessary operations, the spawned thread triggers PostQueuedCompletionStatus (), thereby placing the file open event on the IOCP queue. Moreover, through one of the PostQueuedCompletionStatus parameters, a link to the fs_open_req structure is attached to the generated event.

The cycle of events.

At the entrance to the event loop, the asynchronous request counter is checked. If there are no registered requests, the program ends. If it does, the GetQueuedCompletionStatus () function is called, which either returns the next event or, if there are no events, pauses the work of the thread until they appear.
At one of the iterations, the GetQueuedCompletionStatus function will return an event of opening the file and with it a link to the fs_open_req structure. Next, node.js will reduce the asynchronous request counter and start callback onOpen, passing the file opening result as a parameter to it.

Conclusion

That's all. I wanted to show only the basic principles, so much remains to be described. For example, network I / O operations are organized somewhat differently and make fuller use of IOCP capabilities. But let's leave it for the next time.