The difference between nginx and apache with examples

During interviews for the linux / unix admin role, many IT companies ask what load average is, how nginx differs from apache httpd, and what fork is. In this article I will try to explain what they expect to hear in response to these questions, and why.

It is important to understand the basics of administration very well. In an ideal situation, when setting a task, the system administrator sets a number of requirements. If the situation is not ideal, then, in fact, the requirement for the administrator is one: "I want everything to work." In other words, the service should be available 24/7 and, if some solution does not meet these requirements (scaling and fault tolerance refer to accessibility), then we can say that the administrator has done a poor job. But if the different solutions of the two administrators work 24/7, how to understand which one is better?

A good system administrator, when choosing a solution with given requirements, is guided by two conditions: minimum resource consumption and their balanced distribution.

The variant, when one specialist needs 10 servers to complete the task, and the second is only 2, we will not consider what’s better here — obviously. Further, under the resources I will understand the CPU (cpu), RAM (ram) and disk (hdd).

Let's consider the situation: one administrator created a solution that requires 10% cpu, 5% ram and 10% hdd from all your equipment, and the second used 1% cpu, 40% ram and 20% hdd for this. Which of these solutions is better? Here everything is no longer so obvious. Therefore, a good administrator should always be able to correctly select a solution based on the available resources.

Imagine that we are entry-level programmers, and we are asked to write an elementary program for working with the network. The requirements are simple: you need to process two connections simultaneously using the tcp protocol and write what we have accepted into a file.

Before developing an application, you need to remember what tools the Linux operating system provides for us (further in the article all examples are based only on this OS). In Linux, we have a set of system calls (that is, functions in the OS kernel that we can call directly from our program, thereby forcibly giving the processor time to the kernel):

1) socket - allocates space in the OS kernel buffer for our socket. The address of the allocated space is returned from the function to the program;
2) bind - allows you to change the information in the structure of the socket, which was allocated to us by the linux OS on the command socket;
3) listen - just like bind, it changes the data in our structure, allowing the OS to indicate that we want to accept connections on this socket;
4) connect - tells our OS that it should connect to another remote socket;
5) accept - tells our OS that we want to accept a new connection from another socket;
6) read - we ask the OS to give us from its buffer a certain number of bytes, which it received from a remote socket;
7) write - we ask the OS to send a certain number of bytes to the remote socket.

In order to establish a connection, we need to create a socket in the linux memory, set the necessary data in it and connect to the remote side.

socket → bind → connect → read / write

But if you trust the OS to make the selection of the outgoing port for you (as well as the ip address), then bind is optional:

socket → connect → read / write

In order to receive incoming messages, we need to run:

socket → bind → listen → accept → read / write

Now we know enough to write a program. We proceed directly to writing using si. Why si? Because in this language, commands are called the same as system calls (with rare exceptions, such as fork).

//,    #define PORT_NO 2222 #include <stdio.h> #include <stdlib.h> #include <string.h> #include <netinet/in.h> int main(int argc, char *argv[]) { //,        long buffersize=50; int sockfd, newsockfd; socklen_t clilen; // ,        char *buffer; struct sockaddr_in serv_addr, cli_addr; FILE * resultfile; //   buffer = malloc (buffersize+1); //      resultfile = fopen("/tmp/nginx_vs_apache.log","a"); bzero((char *) &serv_addr, sizeof(serv_addr)); bzero(buffer,buffersize+1); serv_addr.sin_family = AF_INET; serv_addr.sin_addr.s_addr = INADDR_ANY; serv_addr.sin_port = htons(PORT_NO); //  (),  SOCK_STREAM  tcp/ip . sockfd = socket(AF_INET, SOCK_STREAM, 0); if (sockfd < 0) error("ERROR opening socket"); //   ,    2222   ip  if (bind(sockfd, (struct sockaddr *) &serv_addr, sizeof(serv_addr)) < 0) error("ERROR on binding"); //   ,        ,  50 listen(sockfd,50); while (1) { //          newsockfd = accept(sockfd, (struct sockaddr *) &cli_addr, &clilen); if (newsockfd < 0) error("ERROR on accept"); read(newsockfd,buffer,buffersize); fprintf(resultfile, buffer); fflush (resultfile); } close(sockfd); return 0; } 

Compile and run our daemon:

 [tolik@ localhost]$ [tolik@localhost]$ ./differ 

Look what happened:

 [root@ localhost]# ps axuf | grep [d]iffer tolik 45409 0.0 0.0 4060 460 pts/12 S+ 01:14 0:00 | \_ ./differ [root@localhost ]# netstat -tlnp | grep 2222 tcp 0 0 0.0.0.0:2222 0.0.0.0:* LISTEN 45409/./differ [root@localhost ]# ls -lh /proc/45409/fd  0 lrwx------ 1 tolik tolik 64  19 01:16 0 -> /dev/pts/12 lrwx------ 1 tolik tolik 64  19 01:16 1 -> /dev/pts/12 lrwx------ 1 tolik tolik 64  19 01:16 2 -> /dev/pts/12 l-wx------ 1 tolik tolik 64  19 01:16 3 -> /tmp/nginx_vs_apache.log lrwx------ 1 tolik tolik 64  19 01:16 4 -> socket:[42663416] [root@localhost ]# netstat -apeen | grep 42663416 tcp 0 0 0.0.0.0:2222 0.0.0.0:* LISTEN 500 42663416 45409/./differ [root@localhost ]# strace -p 45409 Process 45409 attached - interrupt to quit accept(4, ^C <unfinished ...> Process 45409 detached [root@localhost ]# 

The process is in the sleep state (S + in the ps command).

This program will continue to run (will get processor time) only when a new connection appears on port 2222. In all other cases, the program will never receive processor time: it will not even require it from the OS and, therefore, will not affect load avarage (hereafter LA ), consuming only memory.

From the other console, launch the first client:

 [tolik@localhost ]$ telnet localhost 2222 Connected to localhost. Escape character is '^]'. test client 1 

See the file:

 [root@localhost ]# cat /tmp/nginx_vs_apache.log test client 1 

Open the second connection:

 [tolik@localhost ]$ telnet localhost 2222 Connected to localhost. Escape character is '^]'. test client 2 

See the result:

 [root@localhost ]# cat /tmp/nginx_vs_apache.log test client 1 

From the contents of the file you can see that only the first message from the first client arrived. But at the same time, we have already sent the second message, and it is somewhere. All network connections are made by the OS, which means that the test client 2 message is now in the operating system's buffer, in a memory that is not available to us. The only way to get this data is to process the new connection with the accept command, then call read.

Let's try to write something in the first client:

 [tolik@localhost ]$ telnet localhost 2222 Connected to localhost. Escape character is '^]'. test client 1 blablabla 

Check the log:

 [root@localhost ]# cat /tmp/nginx_vs_apache.log test client 1 

A new message was not logged. This is due to the fact that we call the read command only once, therefore, only the first message gets into the log.

Let's try to close our first connection:

 [tolik@localhost ]$ telnet localhost 2222 Connected to localhost. Escape character is '^]'. test client 1 bla bla bla ^] telnet> quit Connection closed. 

At this moment, our program starts the following accept and read loop, therefore, accepts the message from the second connection:

 [root@localhost ]# cat /tmp/nginx_vs_apache.log test client 1 test client 2 

Our message bla bla bla did not appear anywhere, we already closed the socket, and the OS cleared the buffer, thereby deleting our data. It is necessary to modernize the program - read from the socket until information comes from there.

 #define PORT_NO 2222 #include <stdio.h> #include <string.h> #include <netinet/in.h> int main(int argc, char *argv[]) { int sockfd, newsockfd; socklen_t clilen; char buffer; char * pointbuffer = &buffer; struct sockaddr_in serv_addr, cli_addr; FILE * resultfile; resultfile = fopen("/tmp/nginx_vs_apache.log","a"); bzero((char *) &serv_addr, sizeof(serv_addr)); serv_addr.sin_family = AF_INET; serv_addr.sin_addr.s_addr = INADDR_ANY; serv_addr.sin_port = htons(PORT_NO); sockfd = socket(AF_INET, SOCK_STREAM, 0); if (sockfd < 0) error("ERROR opening socket"); if (bind(sockfd, (struct sockaddr *) &serv_addr, sizeof(serv_addr)) < 0) error("ERROR on binding"); listen(sockfd,50); while (1) { newsockfd = accept(sockfd, (struct sockaddr *) &cli_addr, &clilen); if (newsockfd < 0) error("ERROR on accept"); while (read(newsockfd, pointbuffer,1)) { fprintf(resultfile, pointbuffer); fflush (resultfile); } } close(sockfd); return 0; } 

The program is not much different from the previous one. We added another loop before the read command in order to take data from the socket as long as it goes there. We are checking.

Clear the file:

 [root@localhost ]# > /tmp/nginx_vs_apache.log 

Compile and run:

 [tolik@localhost ]$ gcc -o differ differ2.c [tolik@localhost ]$ ./differ 

First client:

 [tolik@localhost ]$ telnet localhost 2222 Connected to localhost. Escape character is '^]'. client test 1 yoyoyo 

Second client:

 [tolik@localhost ]$ telnet localhost 2222 Connected to localhost. Escape character is '^]'. client test 2 yooyoy 

Checking what happened:

 [root@localhost ]# cat /tmp/nginx_vs_apache.log client test 1 yoyoyo 

This time everything is fine, we took all the data, but the problem remained: two connections are processed sequentially, in turn, and this does not fit our requirements. If we close the first connection (ctrl +]), then the data from the second connection will go directly to the log:

 [root@localhost ]# cat /tmp/nginx_vs_apache.log client test 1 yoyoyo client test 2 yooyoy 

Data came from. But how to handle two connections in parallel? This is where the fork command comes to the rescue. What does the fork system call on linux do? The correct answer to this question at any interview is nothing. Fork is an obsolete call, and in linux it is present only for backward compatibility. In fact, by calling the fork command, you call the clone system call. The clone function creates a copy of the process and queues both processes on the processor. The difference between them is that fork copies data (variables, buffers, etc.) directly to the memory area of ​​the child process, and clone copies the data to the child process only when trying to change it (see memory access restrictions in the MMU). That is, if you call fork 10 times and use data only for reading, you will receive 10 identical copies of data in memory. And this is clearly not what you need, especially in multi-threading applications. Clone runs a copy of your application, but does not copy data right away. If you run clone 10 times, then you will have 10 executable processes with one memory block, and the memory will be copied only when you try to change it by a child process. Agree, the second algorithm is much more efficient.

 #define PORT_NO 2222 #include <stdio.h> #include <string.h> #include <netinet/in.h> int main(int argc, char *argv[]) { int sockfd, newsockfd; socklen_t clilen; char buffer; char * pointbuffer = &buffer; struct sockaddr_in serv_addr, cli_addr; FILE * resultfile; int pid=1; resultfile = fopen("/tmp/nginx_vs_apache.log","a"); bzero((char *) &serv_addr, sizeof(serv_addr)); serv_addr.sin_family = AF_INET; serv_addr.sin_addr.s_addr = INADDR_ANY; serv_addr.sin_port = htons(PORT_NO); sockfd = socket(AF_INET, SOCK_STREAM, 0); if (sockfd < 0) error("ERROR opening socket"); if (bind(sockfd, (struct sockaddr *) &serv_addr, sizeof(serv_addr)) < 0) error("ERROR on binding"); listen(sockfd,50); while (pid!=0) { newsockfd = accept(sockfd, (struct sockaddr *) &cli_addr, &clilen); if (newsockfd < 0) error("ERROR on accept"); pid=fork(); if (pid!=0) { close(newsockfd); fprintf(resultfile,"New process was started with pid=%d\n",pid); fflush (resultfile); } } while (read(newsockfd, pointbuffer,1)) { fprintf(resultfile, pointbuffer); fflush (resultfile); } close(sockfd); return 0; } 

In this program, everything is the same - we accept, accept a new connection. Next, we run fork. And if this is a master process (fork returned the pid of the created process), then we close the current connection in the parent process (it is available both in the parent and in the child process). If this is a child process (fork returns 0), then we begin to read from the open socket, which we opened with the accept command in the parent process. In fact, it turns out that the parent process only accepts connections, and we do read / write in the child processes.

Compile and run:

 [tolik@localhost ]$ gcc -o differ differ3.c [tolik@localhost ]$ ./differ 

Clearing our log file:

 [root@localhost ]# > /tmp/nginx_vs_apache.log 

We look at the processes:

 [root@localhost ]# ps axuf | grep [d]iffer tolik 45643 0.0 0.0 4060 460 pts/12 S+ 01:40 0:00 | \_ ./differ 

Client1:

 [tolik@localhost ]$ telnet localhost 2222 Connected to localhost. Escape character is '^]'. client 1 test megatest 

Client2:

 [tolik@localhost ]$ telnet localhost 2222 Connected to localhost. Escape character is '^]'. client2 test yoyoyoy 

We look at the processes:

 [root@localhost ]# ps axuf | grep [d]iffer tolik 45643 0.0 0.0 4060 504 pts/12 S+ 01:40 0:00 | \_ ./differ tolik 45663 0.0 0.0 4060 156 pts/12 S+ 01:41 0:00 | \_ ./differ tolik 45665 0.0 0.0 4060 160 pts/12 S+ 01:41 0:00 | \_ ./differ 

We do not close both connections and we can add something else there, see our log:

 [root@localhost ]# cat /tmp/nginx_vs_apache.log New process was started with pid=44163 New process was started with pid=44165 client 1 test megatest client2 test yoyoyoy 

Two connections are processed simultaneously - we get the desired result.

The program works, but not fast enough. It first accepts the connection, and only then runs the fork command, and the connection processes only one process. The question arises: can multiple processes in the Linux OS work with the same tcp port? We try.

 #define PORT_NO 2222 #include <stdio.h> #include <string.h> #include <netinet/in.h> int main(int argc, char *argv[]) { int sockfd, newsockfd, startservers, count ; socklen_t clilen; char buffer; char * pointbuffer = &buffer; struct sockaddr_in serv_addr, cli_addr; FILE * resultfile; int pid=1; resultfile = fopen("/tmp/nginx_vs_apache.log","a"); bzero((char *) &serv_addr, sizeof(serv_addr)); serv_addr.sin_family = AF_INET; serv_addr.sin_addr.s_addr = INADDR_ANY; serv_addr.sin_port = htons(PORT_NO); sockfd = socket(AF_INET, SOCK_STREAM, 0); if (sockfd < 0) error("ERROR opening socket"); if (bind(sockfd, (struct sockaddr *) &serv_addr, sizeof(serv_addr)) < 0) error("ERROR on binding"); listen(sockfd,50); startservers=2; count = 0; while (pid!=0) { if (count < startservers) { pid=fork(); if (pid!=0) { close(newsockfd); fprintf(resultfile,"New process was started with pid=%d\n",pid); fflush (resultfile); } count = count + 1; } //sleep (1); } newsockfd = accept(sockfd, (struct sockaddr *) &cli_addr, &clilen); if (newsockfd < 0) error("ERROR on accept"); while (read(newsockfd, pointbuffer,1)) { fprintf(resultfile, pointbuffer); fflush (resultfile); } close(sockfd); return 0; } 

As you can see, the program still hasn't changed much, we just run fork in a loop. In this case, we create two child processes, and only then in each of them we accept to accept a new connection. We are checking.

Compile and run:

 [tolik@localhost ]$ gcc -o differ differ_prefork.c [tolik@localhost ]$ ./differ 

We look, that at us in processes:

 [root@localhost ]# ps axuf | grep [d]iffer tolik 44194 98.0 0.0 4060 504 pts/12 R+ 23:35 0:07 | \_ ./differ tolik 44195 0.0 0.0 4060 152 pts/12 S+ 23:35 0:00 | \_ ./differ tolik 44196 0.0 0.0 4060 156 pts/12 S+ 23:35 0:00 | \_ ./differ 

We have not connected to any client yet, and the program has already made fork for two times. What is happening with the system now? To begin, the master process: it is in a closed loop and checks whether it is necessary to fork more processes. If we do this without stopping, then, in fact, we will constantly require CPU time from the OS, since our cycle must always be executed. This means that we consume 100% of one core - in the ps command, the value is 98.0%. The same can be seen in the top command:

 [root@localhost ]# top -n 1 | head top - 23:39:22 up 141 days, 21 min, 8 users, load average: 1.03, 0.59, 0.23 Tasks: 195 total, 2 running, 193 sleeping, 0 stopped, 0 zombie Cpu(s): 0.3%us, 0.2%sy, 0.0%ni, 99.3%id, 0.2%wa, 0.0%hi, 0.0%si, 0.0%st Mem: 1896936k total, 1876280k used, 20656k free, 151208k buffers Swap: 4194296k total, 107600k used, 4086696k free, 1003568k cached PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 44194 tolik 20 0 4060 504 420 R 98.9 0.0 4:10.54 differ 44255 root 20 0 15028 1256 884 R 3.8 0.1 0:00.03 top 1 root 20 0 19232 548 380 S 0.0 0.0 2:17.17 init 

If we connect with the strace command to the parent, we will not see anything, since our process does not invoke any kernel functions:

 [root@localhost ]# strace -p 44194 Process 44194 attached - interrupt to quit ^CProcess 44194 detached [root@localhost ]# 

What do child processes do? This is where the fun begins. Judging by the code, all of them after the fork should hang in the accept state and wait for new connections from the same port, in our case 2222. We check:

 [root@localhost ]# strace -p 44195 Process 44195 attached - interrupt to quit accept(4, ^C <unfinished ...> Process 44195 detached [root@localhost ]# strace -p 44196 Process 44196 attached - interrupt to quit accept(4, ^C <unfinished ...> Process 44196 detached 

At the moment, they do not require processor time from the OS and consume only memory. But the question is: which one of them will accept my connection if I make telnet? Checking:

 [tolik@localhost ]$ telnet localhost 2222 Connected to localhost. Escape character is '^]'. client 1 test hhh 

 [root@localhost ]# strace -p 44459 Process 44459 attached - interrupt to quit read(5, ^C <unfinished ...> Process 44459 detached [root@localhost ]# strace -p 44460 Process 44460 attached - interrupt to quit accept(4, ^C <unfinished ...> Process 44460 detached 

We see that the process that was created earlier (with a smaller pid) processed the connection first, and is now in the read state. If we run the second telnet, then our connection will process the next process. After we have finished working with the socket, we can close it and go back to the accept state (I did not do this, so as not to complicate the program).

The last question remains: what do we do with the parent process so that it does not consume so much cpu and continue to work? We need to give time to other processes on a voluntary basis, that is, to “tell” our OS that we do not need cpu for some time. For this task, the sleep 1 command is appropriate: if you uncomment it, you will see in strace a picture that repeats once every second:

 [root@localhost ]# strace -p 44601 ….. rt_sigprocmask(SIG_BLOCK, [CHLD], [], 8) = 0 rt_sigaction(SIGCHLD, NULL, {SIG_DFL, [], 0}, 8) = 0 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 nanosleep({1, 0}, 0x7fff60a15aa0) = 0 …. rt_sigprocmask(SIG_BLOCK, [CHLD], [], 8) = 0 rt_sigaction(SIGCHLD, NULL, {SIG_DFL, [], 0}, 8) = 0 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 nanosleep({1, 0}, 0x7fff60a15aa0) = 0 … 

etc.

Our process will receive the processor about once a second, or at least require it from the OS.

If you still do not understand what this long article is for, then look at the apache httpd operating in prefork mode:

 [root@www /]# ps axuf | grep [h]ttpd root 12730 0.0 0.5 271560 11916 ? Ss Feb25 3:14 /usr/sbin/httpd apache 19832 0.0 0.3 271692 7200 ? S Apr17 0:00 \_ /usr/sbin/httpd apache 19833 0.0 0.3 271692 7212 ? S Apr17 0:00 \_ /usr/sbin/httpd apache 19834 0.0 0.3 271692 7204 ? S Apr17 0:00 \_ /usr/sbin/httpd apache 19835 0.0 0.3 271692 7200 ? S Apr17 0:00 \_ /usr/sbin/httpd 

Child processes in accept:

 [root@www /]# strace -p 19832 Process 19832 attached accept4(3, ^CProcess 19832 detached <detached ...> [root@www /]# strace -p 19833 Process 19833 attached accept4(3, ^CProcess 19833 detached <detached ...> 

Master process with a second pause:

 [root@www /]# strace -p 12730 Process 12730 attached select(0, NULL, NULL, NULL, {0, 629715}) = 0 (Timeout) wait4(-1, 0x7fff4c9e3fbc, WNOHANG|WSTOPPED, NULL) = 0 select(0, NULL, NULL, NULL, {1, 0}) = 0 (Timeout) wait4(-1, 0x7fff4c9e3fbc, WNOHANG|WSTOPPED, NULL) = 0 select(0, NULL, NULL, NULL, {1, 0}) = 0 (Timeout) wait4(-1, 0x7fff4c9e3fbc, WNOHANG|WSTOPPED, NULL) = 0 … 

When starting the httpd master process spawns its child processes, it is easy to see if you run strace on the master process at the time of launch:
Let's start a web server with the following settings:

 StartServers 1 MinSpareServers 9 MaxSpareServers 10 ServerLimit 10 MaxClients 10 MaxRequestsPerChild 1 

These settings indicate that each child process will process only one request, then the process will be killed. The minimum number of processes in accept is 9 and the maximum is 10.

If you run strace on the master process at the time of start, we will see how the master calls clone until it reaches MinSpareServers.

We look at how Apache starts - for this you can simply watch ps axuf | grep [h] ttp every second, immediately after the start.

So, we have several child processes that are ready to accept our http request. Let's try to send a request:

 [root@www /]# wget -O /dev/null http://localhost --2017-01-25 14:04:00-- http://localhost/ Resolving localhost... ::1, 127.0.0.1 Connecting to localhost|::1|:80... failed: Connection refused. Connecting to localhost|127.0.0.1|:80... connected. HTTP request sent, awaiting response... 403 Forbidden 2017-01-25 14:04:00 ERROR 403: Forbidden. 

Apache answered 403, we look at the processes:

 root 13342 0.0 0.4 271084 9384 ? Ss 14:12 0:00 /usr/sbin/httpd apache 13348 0.0 0.2 271084 5232 ? S 14:12 0:00 \_ /usr/sbin/httpd apache 13352 0.0 0.2 271084 5232 ? S 14:12 0:00 \_ /usr/sbin/httpd apache 13353 0.0 0.2 271084 5232 ? S 14:12 0:00 \_ /usr/sbin/httpd apache 13357 0.0 0.2 271084 5232 ? S 14:12 0:00 \_ /usr/sbin/httpd apache 13358 0.0 0.2 271084 5232 ? S 14:12 0:00 \_ /usr/sbin/httpd apache 13359 0.0 0.2 271084 5232 ? S 14:12 0:00 \_ /usr/sbin/httpd apache 13360 0.0 0.2 271084 5232 ? S 14:12 0:00 \_ /usr/sbin/httpd apache 13364 0.0 0.2 271084 5232 ? S 14:12 0:00 \_ /usr/sbin/httpd apache 13365 0.0 0.2 271084 5232 ? S 14:12 0:00 \_ /usr/sbin/httpd 

As you can see, the process with the minimum pid processed the request and completed its work:

 apache 13344 0.0 0.2 271084 5232 ? S 14:12 0:00 \_ /usr/sbin/httpd 

We have 9 child processes left, which fits into our MinSpareServers limit.

We try to send the request again:

 [root@www /]# wget -O /dev/null http://localhost --2017-01-25 14:15:47-- http://localhost/ Resolving localhost... ::1, 127.0.0.1 Connecting to localhost|::1|:80... failed: Connection refused. Connecting to localhost|127.0.0.1|:80... connected. HTTP request sent, awaiting response... 403 Forbidden 2017-01-25 14:15:47 ERROR 403: Forbidden. [root@www /]# ps axuf | grep [h]ttp root 13342 0.0 0.4 271084 9384 ? Ss 14:12 0:00 /usr/sbin/httpd apache 13352 0.0 0.2 271084 5232 ? S 14:12 0:00 \_ /usr/sbin/httpd apache 13353 0.0 0.2 271084 5232 ? S 14:12 0:00 \_ /usr/sbin/httpd apache 13357 0.0 0.2 271084 5232 ? S 14:12 0:00 \_ /usr/sbin/httpd apache 13358 0.0 0.2 271084 5232 ? S 14:12 0:00 \_ /usr/sbin/httpd apache 13359 0.0 0.2 271084 5232 ? S 14:12 0:00 \_ /usr/sbin/httpd apache 13360 0.0 0.2 271084 5232 ? S 14:12 0:00 \_ /usr/sbin/httpd apache 13364 0.0 0.2 271084 5232 ? S 14:12 0:00 \_ /usr/sbin/httpd apache 13365 0.0 0.2 271084 5232 ? S 14:12 0:00 \_ /usr/sbin/httpd apache 13373 0.0 0.2 271084 5232 ? S 14:15 0:00 \_ /usr/sbin/httpd 

This time our request processed the process.

 apache 13348 0.0 0.2 271084 5232 ? S 14:12 0:00 \_ /usr/sbin/httpd 

since it now has a minimum pid.

But we have 8 free child processes left in accept, one is missing before MinSpareServers, so the master process has created a new process for us:

 apache 13373 0.0 0.2 271084 5232 ? S 14:15 0:00 \_ /usr/sbin/httpd 

Let's tell our OS that it does not give the processor time to the master Apache process:

 [root@www /]# kill -SIGSTOP 13342 

We look:

 [root@www /]# ps axuf | grep [h]ttp | grep ^root root 13342 0.0 0.4 271084 9384 ? Ts 14:12 0:00 /usr/sbin/httpd 

The status of the process has changed, now it does not work.

Check if our web server works:

 [root@www /]# wget -O /dev/null http://localhost --2017-01-25 14:20:12-- http://localhost/ Resolving localhost... ::1, 127.0.0.1 Connecting to localhost|::1|:80... failed: Connection refused. Connecting to localhost|127.0.0.1|:80... connected. HTTP request sent, awaiting response... 403 Forbidden 2017-01-25 14:20:12 ERROR 403: Forbidden. 

Oh yes, it still works, the web server is still responding.

We look at what we have with the processes:

 root 13342 0.0 0.4 271084 9384 ? Ts 14:12 0:00 /usr/sbin/httpd apache 13352 0.0 0.0 0 0 ? Z 14:12 0:00 \_ [httpd] <defunct> apache 13353 0.0 0.2 271084 5232 ? S 14:12 0:00 \_ /usr/sbin/httpd apache 13357 0.0 0.2 271084 5232 ? S 14:12 0:00 \_ /usr/sbin/httpd apache 13358 0.0 0.2 271084 5232 ? S 14:12 0:00 \_ /usr/sbin/httpd apache 13359 0.0 0.2 271084 5232 ? S 14:12 0:00 \_ /usr/sbin/httpd apache 13360 0.0 0.2 271084 5232 ? S 14:12 0:00 \_ /usr/sbin/httpd apache 13364 0.0 0.2 271084 5232 ? S 14:12 0:00 \_ /usr/sbin/httpd apache 13365 0.0 0.2 271084 5232 ? S 14:12 0:00 \_ /usr/sbin/httpd apache 13373 0.0 0.2 271084 5232 ? S 14:15 0:00 \_ /usr/sbin/httpd 

Our next request was processed by another child process, which worked and left. But he left the exit code, which must be processed by the master process. Since the master process is stopped at us, the exit code is still in the kernel in the process table, and even though we don’t already have the process, it is in the table marked as zombies.

 apache 13352 0.0 0.0 0 0 ? Z 14:12 0:00 \_ [httpd] <defunct> 

8, 9 , .

http :

 [root@www /]# wget -O /dev/null http://localhost --2017-01-25 14:25:03-- http://localhost/ Resolving localhost... ::1, 127.0.0.1 Connecting to localhost|::1|:80... failed: Connection refused. Connecting to localhost|127.0.0.1|:80... connected. HTTP request sent, awaiting response... 403 Forbidden 2017-01-25 14:25:03 ERROR 403: Forbidden. [root@www /]# ps axuf | grep [h]ttp root 13342 0.0 0.4 271084 9384 ? Ts 14:12 0:00 /usr/sbin/httpd apache 13352 0.0 0.0 0 0 ? Z 14:12 0:00 \_ [httpd] <defunct> apache 13353 0.0 0.0 0 0 ? Z 14:12 0:00 \_ [httpd] <defunct> apache 13357 0.0 0.2 271084 5232 ? S 14:12 0:00 \_ /usr/sbin/httpd apache 13358 0.0 0.2 271084 5232 ? S 14:12 0:00 \_ /usr/sbin/httpd apache 13359 0.0 0.2 271084 5232 ? S 14:12 0:00 \_ /usr/sbin/httpd apache 13360 0.0 0.2 271084 5232 ? S 14:12 0:00 \_ /usr/sbin/httpd apache 13364 0.0 0.2 271084 5232 ? S 14:12 0:00 \_ /usr/sbin/httpd apache 13365 0.0 0.2 271084 5232 ? S 14:12 0:00 \_ /usr/sbin/httpd apache 13373 0.0 0.2 271084 5232 ? S 14:15 0:00 \_ /usr/sbin/httpd 

, .

, :

 [root@www /]# kill -SIGCONT 13342 [root@www /]# ps axuf | grep [h]ttp root 13342 0.0 0.4 271084 9384 ? Ss 14:12 0:00 /usr/sbin/httpd apache 13357 0.0 0.2 271084 5232 ? S 14:12 0:00 \_ /usr/sbin/httpd apache 13358 0.0 0.2 271084 5232 ? S 14:12 0:00 \_ /usr/sbin/httpd apache 13359 0.0 0.2 271084 5232 ? S 14:12 0:00 \_ /usr/sbin/httpd apache 13360 0.0 0.2 271084 5232 ? S 14:12 0:00 \_ /usr/sbin/httpd apache 13364 0.0 0.2 271084 5232 ? S 14:12 0:00 \_ /usr/sbin/httpd apache 13365 0.0 0.2 271084 5232 ? S 14:12 0:00 \_ /usr/sbin/httpd apache 13373 0.0 0.2 271084 5232 ? S 14:15 0:00 \_ /usr/sbin/httpd apache 13388 0.0 0.2 271084 5232 ? S 14:26 0:00 \_ /usr/sbin/httpd apache 13389 0.0 0.2 271084 5232 ? S 14:26 0:00 \_ /usr/sbin/httpd apache 13390 0.0 0.2 271084 5232 ? S 14:26 0:00 \_ /usr/sbin/httpd 

exit code , , — 10 accept, .

nginx? , accept , . , . Or?

:

 #define PORT 2222 #include <stdio.h> #include <string.h> #include <stdlib.h> #include <errno.h> #include <unistd.h> #include <arpa/inet.h> #include <sys/types.h> #include <sys/socket.h> #include <netinet/in.h> #include <sys/time.h> int main(int argc , char *argv[]) { int opt = 1; int master_socket , addrlen , new_socket , client_socket[30] , max_clients = 30 , activity, i , valread , sd; int max_sd; FILE * resultfile; struct sockaddr_in address; char buffer[50]; fd_set readfds; resultfile = fopen("/tmp/nginx_vs_apache.log","a"); //     for (i = 0; i < max_clients; i++) client_socket[i] = 0; if( (master_socket = socket(AF_INET , SOCK_STREAM , 0)) == 0) error("socket failed"); address.sin_family = AF_INET; address.sin_addr.s_addr = INADDR_ANY; address.sin_port = htons( PORT ); if (bind(master_socket, (struct sockaddr *)&address, sizeof(address))<0) error("bind failed"); if (listen(master_socket, 3) < 0) error("listen"); addrlen = sizeof(address); while(1) //     { FD_ZERO(&readfds); FD_SET(master_socket, &readfds); max_sd = master_socket; for ( i = 0 ; i < max_clients ; i++) { sd = client_socket[i]; if(sd > 0) FD_SET( sd , &readfds); if(sd > max_sd) max_sd = sd; } //        activity = select( max_sd + 1 , &readfds , NULL , NULL , NULL); if ((activity < 0) && (errno!=EINTR)) printf("select error"); //   if (FD_ISSET(master_socket, &readfds)) { if ((new_socket = accept(master_socket, (struct sockaddr *)&address, (socklen_t*)&addrlen))<0) error("accept"); for (i = 0; i < max_clients; i++) if( client_socket[i] == 0 ) { client_socket[i] = new_socket; break; } } //    ,           CPU for (i = 0; i < max_clients; i++) { sd = client_socket[i]; if (FD_ISSET( sd , &readfds)) { if ((valread = read( sd , buffer, 1024)) == 0) { close( sd ); client_socket[i] = 0; } else { buffer[valread] = '\0'; fprintf(resultfile, buffer); fflush (resultfile); } } } } return 0; } 

, , . , 30 . . , , . , . accept? linux 3 : select, poll epoll. freebsd epoll kqueue (kernel queue). ? select – , , , ( accept). , cpu, . , select . , select : 2222 , . , :

 [tolik@101host nginx_vs_apache]$ ./differ & [1] 44832 [tolik@101host nginx_vs_apache]$ ps axuf | grep [.]/differ tolik 44832 0.0 0.0 4060 448 pts/0 S 22:47 0:00 \_ ./differ [root@localhost ]# strace -p 44832 Process 44832 attached - interrupt to quit select(5, [4], NULL, NULL, NULL) = 1 (in [4]) 

telnet 2222 :

 accept(4, {sa_family=AF_INET, sin_port=htons(41130), sin_addr=inet_addr("127.0.0.1")}, [16]) = 5 select(6, [4 5], NULL, NULL, NULL^C <unfinished ...> Process 44832 detached [root@localhost ]# ls -lh /proc/44832/fd  0 lrwx------ 1 tolik tolik 64  19 00:26 0 -> /dev/pts/12 lrwx------ 1 tolik tolik 64  19 00:26 1 -> /dev/pts/12 lrwx------ 1 tolik tolik 64  19 00:21 2 -> /dev/pts/12 l-wx------ 1 tolik tolik 64  19 00:26 3 -> /tmp/nginx_vs_apache.log lrwx------ 1 tolik tolik 64  19 00:26 4 -> socket:[42651147] lrwx------ 1 tolik tolik 64  19 00:26 5 -> socket:[42651320] [root@localhost ]# netstat -apeen | grep 42651147 tcp 0 0 0.0.0.0:2222 0.0.0.0:* LISTEN 500 42651147 44832/./differ [root@localhost ]# netstat -apeen | grep 42651320 tcp 0 0 127.0.0.1:2222 127.0.0.1:41130 ESTABLISHED 500 42651320 44832/./differ 

select 4 ( ). /proc , 4 - — 42651147. netstat , — listen 2222. , tcp handshake telnet , select. . , , accept, , , . accept, .

, , linux, , — 4 5, strace ([4 5]). nginx: . read/write, accept. Select — , : , ( ). poll, . epoll kqueue ( freebsd). .

nginx? Nginx .

. , , .. .

Nginx fork , . nginx , select, ( linux epoll). We look:

 [root@localhost ]# ps axuf| grep [n]ginx root 232753 0.0 0.0 96592 556 ? Ss Feb25 0:00 nginx: master process /usr/sbin/nginx -c /etc/nginx/nginx.conf nginx 232754 0.0 0.0 97428 1400 ? S Feb25 5:20 \_ nginx: worker process nginx 232755 0.0 0.0 97460 1364 ? S Feb25 5:02 \_ nginx: worker process [root@localhost ]# strace -p 232754 Process 232754 attached - interrupt to quit epoll_wait(12, ^C <unfinished ...> Process 232754 detached [root@localhost ]# strace -p 232755 Process 232755 attached - interrupt to quit epoll_wait(14, {}, 512, 500) = 0 epoll_wait(14, ^C <unfinished ...> Process 232755 detached 

nginx?

 [root@localhost ]# strace -p 232753 Process 232753 attached - interrupt to quit rt_sigsuspend([]^C <unfinished ...> Process 232753 detached 

, . nginx , , , , .

nginx unix :

 [root@localhost ]# ls -lh /proc/232754/fd  0 lrwx------ 1 nginx nginx 64  8 13:20 0 -> /dev/null lrwx------ 1 nginx nginx 64  8 13:20 1 -> /dev/null lrwx------ 1 nginx nginx 64  8 13:20 10 -> socket:[25069547] lrwx------ 1 nginx nginx 64  8 13:20 11 -> socket:[25069551] lrwx------ 1 nginx nginx 64  8 13:20 12 -> anon_inode:[eventpoll] lrwx------ 1 nginx nginx 64  8 13:20 13 -> anon_inode:[eventfd] l-wx------ 1 nginx nginx 64  8 13:20 2 -> /var/log/nginx/error.log lrwx------ 1 nginx nginx 64  8 13:20 3 -> socket:[25069552] l-wx------ 1 nginx nginx 64  8 13:20 5 -> /var/log/nginx/error.log l-wx------ 1 nginx nginx 64  8 13:20 6 -> /var/log/nginx/access.log lrwx------ 1 nginx nginx 64  8 13:20 9 -> socket:[25069546] [root@localhost ]# netstat -apeen | grep 25069547 tcp 0 0 172.16.0.1:80 0.0.0.0:* LISTEN 0 25069547 232753/nginx [root@localhost ]# netstat -apeen | grep 25069551 unix 3 [ ] STREAM CONNECTED 25069551 232753/nginx 

Total

, . apache httpd nginx – . , ( ), nginx nginx.

PS

, , 2222. , - . , , .