xinetd + netcat → pitfalls
If you need to do something on a remote server, but you are too lazy to bother with writing a network service, xinetd comes to the rescue.
The ease of writing servers for xinetd is attractive, it is really simple: we write a simple script in any language that works with stdin and stdout (in the simplest case, this is a regular REPL ) and we get a console utility and a network server in one bottle at the same time.
After one minute of editing the xinetd config, we get a working server to which you can connect via telnet or netcat and see the result on the console.
Everything is easy and beautiful. But since there is a (remote) console utility, I want to go further - automate the work with it. Do not drive hundreds of requests with your hands and not search for the same on the screen as it worked. That is, speaking the network language, you need a client part.
')
Here, naturally, I want to use netcat (nc): we write a simple script that will issue commands to stdout and interpret the answers received from stdin and connect it to netcat, thus turning it into a network client (although the script itself will not know about it).
Having such “server-side” and “client-side” scripts, with a bonus, we are able to launch them locally. Not to mention the fact that in coding we did not write a single line concerning the work with the network.
In most cases, this is enough to work happily ever after.
But there are nuances .
Having written with joy several such remote utilities, I found that sometimes the process of communication written in this way by the client and server hangs.
There was nothing to google on this topic, I had to figure it out myself, I present the results to you.
How xinetd works: opens a socket, hooks it to standard I / O streams, and starts an external programmer.
How netcat works (-e, for example): opens a socket, hooks it to standard I / O streams and starts an external programmer.
That is the same.
Accordingly, if both programmers are sharpened to work with the terminal, no one checks the possibility of writing or reading. If you need to read, just read; need to write - just write.
Only instead of the terminal there is already a network socket, and no one works with the network.
With enough traffic, there may be a situation of dragons devouring each other from the tail: the reading buffers on both sides are full, but according to the logic of the work cycles, both sides just want to say something to each other.
As a result, both processes hang in write () forever.
Attempts to solve the problem cheaply by subtracting the maximum amount of data do not cure the problem, but only postpone it.
But most of all in this story I was disappointed by netcat.
Not finding any calls in strace nc to set up a non-blocking socket mode, I looked in apt-get source and ... didn’t find anything like that.
Netcat works with sockets in a synchronous mode, provoking the possibility of a hangup, even without any “external program written for the console”.
Sadness adds the fact that netcat is in fact a standard, although, in fact, unsuitable for use in the general case.
There are two ways out:
1. Write a server for xinetd, considering that this is a network (fcntl O_NONBLOCK, select to stdin and stdout and all that), but then xinetd is lost as a simple wrapper for simple things. You can already write a full server.
2. Write a synchronous client that will wait for the server's response and prevent buffer overflows for reading and writing - here netcat is lost and you have to do something with performance so that you can send several requests at the same time without overflowing the buffer and be able to process several at the same time replies.
In my case, I chose a semi-second synchronous option:
• send 10 requests
• while there is something to send {send one request; read one answer}
• read 10 replies
... what worked in my particular case, but as a general solution, of course, is no good.
The ease of writing servers for xinetd is attractive, it is really simple: we write a simple script in any language that works with stdin and stdout (in the simplest case, this is a regular REPL ) and we get a console utility and a network server in one bottle at the same time.
After one minute of editing the xinetd config, we get a working server to which you can connect via telnet or netcat and see the result on the console.
Everything is easy and beautiful. But since there is a (remote) console utility, I want to go further - automate the work with it. Do not drive hundreds of requests with your hands and not search for the same on the screen as it worked. That is, speaking the network language, you need a client part.
')
Here, naturally, I want to use netcat (nc): we write a simple script that will issue commands to stdout and interpret the answers received from stdin and connect it to netcat, thus turning it into a network client (although the script itself will not know about it).
Having such “server-side” and “client-side” scripts, with a bonus, we are able to launch them locally. Not to mention the fact that in coding we did not write a single line concerning the work with the network.
In most cases, this is enough to work happily ever after.
But there are nuances .
Having written with joy several such remote utilities, I found that sometimes the process of communication written in this way by the client and server hangs.
There was nothing to google on this topic, I had to figure it out myself, I present the results to you.
Who is guilty?
How xinetd works: opens a socket, hooks it to standard I / O streams, and starts an external programmer.
How netcat works (-e, for example): opens a socket, hooks it to standard I / O streams and starts an external programmer.
That is the same.
Accordingly, if both programmers are sharpened to work with the terminal, no one checks the possibility of writing or reading. If you need to read, just read; need to write - just write.
Only instead of the terminal there is already a network socket, and no one works with the network.
With enough traffic, there may be a situation of dragons devouring each other from the tail: the reading buffers on both sides are full, but according to the logic of the work cycles, both sides just want to say something to each other.
As a result, both processes hang in write () forever.
Attempts to solve the problem cheaply by subtracting the maximum amount of data do not cure the problem, but only postpone it.
But most of all in this story I was disappointed by netcat.
Not finding any calls in strace nc to set up a non-blocking socket mode, I looked in apt-get source and ... didn’t find anything like that.
Netcat works with sockets in a synchronous mode, provoking the possibility of a hangup, even without any “external program written for the console”.
Sadness adds the fact that netcat is in fact a standard, although, in fact, unsuitable for use in the general case.
What to do?
There are two ways out:
1. Write a server for xinetd, considering that this is a network (fcntl O_NONBLOCK, select to stdin and stdout and all that), but then xinetd is lost as a simple wrapper for simple things. You can already write a full server.
2. Write a synchronous client that will wait for the server's response and prevent buffer overflows for reading and writing - here netcat is lost and you have to do something with performance so that you can send several requests at the same time without overflowing the buffer and be able to process several at the same time replies.
In my case, I chose a semi-second synchronous option:
• send 10 requests
• while there is something to send {send one request; read one answer}
• read 10 replies
... what worked in my particular case, but as a general solution, of course, is no good.
Source: https://habr.com/ru/post/114853/
All Articles