Interview with hidnplayr

We bring to your attention an interview with Jeffrey Amelynck aka hidnplayr (on Habré hidnplayr ), a Belgian developer in the KolibriOS project. Previously, some of his developments have already been an article . Let's talk about Google Summer of ode, Belgium, the network stack and the prospects of the network programs for KolibriOS.

The interview was conducted by Sergei Kuzmin aka Wildwest (on Habré W__W )

Introduce yourself: where did you study, what did you study, in what operating systems did you have to work.

I studied at the academy, formerly known as KAHO Sint-Lieven , which is now a division of Belgium's largest university, KU-Leuven . There I received a bachelor's degree in the specialty “Electronics and information and communication technologies” (specialization “Electronics”).
')


After that, I tried to get a master's degree, but recently I lost interest in this training. I will say that this training is too theoretical for me and I decided that it would be better to start establishing myself in the real world.

Did working on KolibriOS help you in your education, work or career?

My professional career is just beginning, but I’m sure I can apply some of the knowledge gained here. KolibriOS gave me some experience in working with networks and in programming, but also the importance of documentation, self-motivation and the like. Working with real equipment showed me the importance of documentation and standardization.



For example, some network cards (* coughed * Tulip * coughed *) are noteworthy because you need a lot of software hacks to make all available cards of this family work.



Tell me about Belgium. Tell me that in Belgium there is one that people don’t know about abroad (example: a person from Ecuador recently said that they have a lot of bamboo growing in their country. I didn’t know about it.).

Belgium is widely known for its beer, chocolate and french fries, but our love for gastronomic cuisine does not end there. Do you know that we have as many Michelin stars per inhabitant as in France?

Did not know this. How and when did you find out about MenuetOS and KolibriOS?

At the young age of 13, I was looking for alternative operating systems. I tried several Linux distributions, but there were problems running on my old computer. Configuring, especially the video card drivers and the X Window System, caused headaches. I found MenuetOS and was immediately impressed. Because of MenuetOS, I decided to learn programming in assembly language.

Some time ago your nickname was profkid13, but then you changed it to hidnplayr. Tell me about these nicknames, the reason for the shift.

Profkid13 was my first nickname, as you guess, at the age of 13. He reflected the idea of ​​who I wanted to be at that time: a smart newcomer. Of course, none of this was true, I did not know how to study on the Internet and asked the wrong questions in various forums. Later, when I realized this, I decided that it was time for a new nickname to get rid of a bad image.
Nick came from a chat in which I spent many hours of my youth. It was a public chat where you could play games with other users. I found a way to hide myself from the list of users and therefore I chose the name “hidden player”, which was later reduced to “hidnplayr”. This nickname caught on and I never thought to change it.

One of the controversial topics is the history of MenuetOS and the subsequent division of the development team into those who supported Menuet64 and those who supported KolibriOS. Tell us your version of the events of the period 2003-2007. Why did you support KolibriOS?

I do not remember exactly when I found KolibriOS, but it was obvious that it was better than MenuetOS, at least in my eyes. I have always had an interest in computer networks and I decided to study this at KolibriOS.
KolibriOS was better for me because it had such qualities as (best) memory manager and a more vibrant community.
The first program I committed to SVN was the autodhcp program. It combined the two existing “DHCP client” and “network configuration” programs to automatically receive network settings from a DHCP server and apply them after booting.



Tell us about your programs for KolibriOS, including the irc client and the program for working with the COM port.

I wrote a simple VNC client and wanted to write more network programs, but then I noticed the limitations of the existing network stack in KolibriOS.



A few simple changes (reducing the data copy by the OS kernel) greatly improved performance, but it was still suboptimal. I never publicly showed these kernel fixes. One thing that was quickly incorporated into the kernel was a new system function that allowed you to read more than 1 byte at a time from a socket.
Instead of improving the old code, I decided to start from scratch a new network stack. I wanted to get rid of intranuclear network drivers and replace the existing network API with the industry standard BSD sockets .

This also meant that all network programs had to be updated or even completely rewritten. Some programs, such as the IRC client, were completely rewritten because there was a lot of confusion in their code. One of my maxims : “Write the lasagna code , not the spaghetti code .”







The terminal program was written after a forum request for such an application. I wrote it in a few hours, showing how simple such a program could be.



Unfortunately, it was impossible to assign an interrupt request (IRQ) to the COM port in the program without writing a special driver.
I decided that I would not do this, because it is better to wait for the support of the COM port at the OS level.
And fortunately, this may soon happen when the gtament programmer (who wrote the driver for the FTDI USB-RS232 adapter; on Habré gtament ) finishes this API.

Tell us about paste.kolibrios.org and a program for sending texts from KolibriOS to this site.

The paste.kolibrios.org site is a publicly available pastebin , the same as pastie .
My program "pasta" was written to demonstrate how to use the http library (note - the library will be discussed below) with cookies and the post method from HTTP. It allows you to send text from a file or clipboard to the site.



What network programs are still missing in KolibriOS? Will there ever appear analogs of the netstat command (Note is a command in cmd.exe from Windows, there are programs with the same function - TCPView and Anvir) and a program for displaying a list of open and closed ports with the ability to control their state? How are things going with the server software?

Such programs will definitely be written, the only question is when it will be. Since this is a hobby for me, I can’t name a timeline.
The Socketdbg program can read all the values ​​in the kernel for a particular socket, but this is a very low level of course.
The FTP daemon (or server, if you will) is available and must work.



The HTTP server is also available, but not in the main distribution, because it uses a special library that is not available on SVN.
It is called photonweb and is actually a multi-platform HTTP server, written in “Fresh IDE” by the johnfound programmer. Writing an HTTP server specifically for KolibriOS is not in a high place on my personal list of priorities.

How about Samba, print over the network?

Samba apparently requires a lot of work and therefore also not in a high place on my personal list of priorities. But it would definitely be nice to have her support.
Some network printers allow you to send documents to them via FTP, so this is already possible if you can figure out how to do it.
But since there is no support for the spooler and all other required services for printers, it makes no sense for me to investigate the protocols of printers.

What is included in the list of priorities?

It is constantly changing and in general it is something in my head that I cannot write.

Tell me about the brunch of the network stack, which was later integrated into the kernel. What are its main features and advantages, its current status and plans for the near future.

I support the wiki page where you can find information and track progress: wiki.kolibrios.org/wiki/New_stack
The main objectives for the new network stack were higher performance (bandwidth), support for many network interfaces and an API compatible with BSD sockets to simplify writing / porting network programs.

To increase performance, I reduced the copying of network packets (in MenuetOS, they were copied by each protocol handler).

I also chose a driver implementation that supports interrupts, although I recently learned that a polling approach can provide better bandwidth with traffic from small packets. (Fortunately, polling can be easily implemented in current drivers when needed.)

Currently, work with several network cards is supported, but self-tuning (DHCP) works only on the first detected network card.

Core routing also needs to be improved to work with multiple network cards. (Some functions are rigidly tied to the first network card).
TCP code, which is not so complicated but great, generally works, but is not complete and must be properly tested. The current TCP code is a manually rewritten implementation of TCP code from 4.4BSD, and therefore is a bit old for the modern Internet.

Later, I want to explore different saturation prevention and other optimization algorithms .



Are there any problems or limitations in the network stack? How many connections can be started simultaneously? What is the significance of DDoS traffic for KolibriOS?

Problems can be bugs or unrealized properties.
Full testing is necessary and scheduled after implementing certain properties, such as TCP re-assembly queue.
The maximum number of network buffers and connections is configured in the kernel code. Extract from stack.inc:
NET_DEVICES_MAX = 16
NET_BUFFERS = 512
NET_BUFFER_SIZE = 2048
The number of sockets is limited only by the available memory at the appropriate moment, but in the future it can be easily limited to the number set by the user.

Tell me how the network stack works.

This topic in itself could be a complete article, so I will limit myself here to the input and output of user data.

When a packet is received over the wire by a network card, an interrupt is generated.

This interrupt is processed by the driver, which places the incoming packet into the computer’s RAM (if the hardware has not already done so) and sends a pointer to this network packet to the kernel. The kernel places this pointer in a queue of packets waiting to be processed as soon as possible, but not in the interrupt handler. After a short time, the kernel will begin processing incoming packets and invoke the appropriate protocol handlers in the kernel. For example, the response to an ICMP echo request (ICMP echo request) response will be made by the kernel, ARP packets will also be processed by the kernel, but UDP / TCP packets / segments may result in data queued to the socket.

When such data arrives, the owner of the socket process will be notified that data is available and data can be received through the system function 75, 7 (Get from socket).

This system call will copy data from the network packet buffer (in kernel space) to the location selected by the program (in user space).

The network packet buffer is then marked as empty and ready for reuse by the system.



If the program wants to send data, then it calls the system function 75, 6 (Send to socket).
This will cause the kernel to allocate space for the network buffer, generate the necessary protocol headers in this buffer, and copy the data from user space to the network buffer.
The kernel will then call the appropriate network driver, which will copy the network buffer to the hardware buffer and finally send the packet over the wire.

As soon as the driver signals that copying to the hardware buffer is complete, the network buffer is marked empty so that it can be reused later.



How to use HTTPlib (http.obj)?

So, first, you need to know how to use libraries in KolibriOS. This article on KolibriOS should be a good start: wiki.kolibrios.org/wiki/Libraries .
When you figure this out and know the basics of how HTTP works, you can find the http library library API: websvn.kolibrios.org/filedetails.php?repname=Kolibri+OS&path=%2Fprograms%2Fdevelop%2Flibraries%2Fhttp%2Fhttp_en.txt
and some examples: websvn.kolibrios.org/listing.php?repname=Kolibri+OS&path=%2Fprograms%2Fdevelop%2Flibraries%2Fhttp%2Fexamples%2F&#afd06bd681cc25e97cbe85f78ea95257a .



(Note - for a change in this article, screenshots of the NetSurf browser. Screenshots of the WebView browser and Liza mail client in the relevant forum topics)

What network protocols are supported besides HTTP?

The kernel supports network and transport protocols such as Ethernet, ipv4, ARP, UDP, TCP .





HTTP is technically considered an application layer protocol and this is how it is implemented in KolibriOS (in the library).
There are programs or libraries that use the FTP, TFTP, DNS, DHCP, Telnet and Synergy protocols (for sharing keyboard and mouse over the network).











The VNC client needs some work to be really useful.



What about working in the local network? Is it possible to limit the traffic of a network connection (set quota)?

Computers with KolibriOS can communicate via daemons and HTTP and FTP clients.
Before, there were more programs, but they were not ported by me to a new network stack.





Restriction of traffic in any form is not currently supported.

What network cards are supported? Are multiple network cards supported? How is the network card chosen in this case?

Yes, I took on the task of supporting existing network drivers and even wrote some new ones. Supported
3Com Fast EtherLink (3c59x / 450/555/556/575/900/905/980), dec21x4x (called Tulip in linux), nVidia nForce chipsets (forcedeth)
Intel eepro / 100 (i8255x, i82562, i82801, ..), Intel PRO / 1000 (i8254x), Myson MTD803, AMD PCnet (79c970, 79c978, 79c973, ..),
RDC Semiconductors R6040, VIA Rhine II and III, Realtek 8029/8019, ne2000 PCI, Realtek 8139 and clones, Realtek 8110/8111/8167/8168/8169
Silicon Integrated Systems SiS900, 7016

(Note - photo taken 2.5 years ago. On the left are supported network cards, on the right are non-supported network cards) .


(Note - current photo. The number of supported network cards has increased significantly) .





What about software modems, Wi-Fi?

Both are not supported now and are not on my priority list.
Soft modems (I suspect that you mean the ancient modems of the V92 standard) are of little interest to me, I don't even have a telephone line at home.
I can start learning WiFi when I’m completely satisfied with the performance and security of the main network stack. (But I hope we will not wait for this for so long and someone else will take on this task.)

Does your IPv6 network stack support? What with the translation of domain names with Cyrillic?

Some small parts of the IPv6 code are written, but disabled, because it is not fully implemented yet. This is exactly on my priority list.

Cyrillic domain names are an interesting question. I did not even know that such things exist.
Apparently there is a Punycode system for decoding Cyrillic URLs, but it is not supported now.

Do you often read RFCs concerning network standards?

RFC documents are long and not particularly attractive for reading, so I rarely read them from beginning to end before starting the implementation of the protocol.
It is often easier to use a short description of the protocol and contact the RFC when information is missing or when in doubt. There are many books on this subject that are often easier to read.

What do you think about the new standards WebSocket, SPDY, QUIC, HTTP / 2?

A successor to the current HTTP running over TCP is definitely needed. I will try to follow industry standards.

You were a mentor at Google Summer of Code. Tell us about your experiences with this event and your visit to Google last year.

I enjoyed working with students and will do it again if we find another sponsor and I have time.
The opportunity to visit Google Summer of Code Reunion - it was excellent. This was my first experience with the open source community in real life.


The main essence of the "Summer of Code Reunion" was that the developers come together and meet in real life. There were conference rooms available for those who wanted to program (do some hacking) or give / attend a lecture. But none of this was mandatory.



Unfortunately, neither the dunkaist (another mentor in the Summer of Code from KolibriOS), nor Yogev (on Habré yogev_ezra ) could not attend this reunion because they could not make visas in time.
There were some planned activities, such as traveling to museums, amusement parks, Google offices, etc.
During breaks, I spent most of my time in the main hall, where I communicated with other developers.



I haven't been to the USA before, so this was an adventure in itself. Silicon Valley was nice to see, but I would not want to live here. The food is not so bad, but not so good, and so highly praised the freedom there feels much more limited than in Belgium.

After reunification, I attended other open source community events, such as FOSDEM in Belgium (where I finally met Yogev ), and I plan to continue attending such events whenever possible.