The progress of the heavy task in PHP

It happened to me to somehow deal with a heavy PHP script. It was necessary to somehow display in the browser the progress of the task at the time, while in a sufficiently long cycle on the PHP side calculations were performed. In such cases, they usually resort to occasional line breaks like this:

<script>document.getElementById('progress').style.width = '1%';</script> 

This option did not suit me for several reasons, besides, I basically do not like this approach.

I had about 3000-5000 iterations. I figured that the traffic is too big for such a simple undertaking. In addition, this option seemed to me very ugly from a technical point of view, and the appearance of the page was completely ugly: the footer would not come soon - after the last notification of the 100% completion of the task.

Dodging the problem of an ugly page was not a big deal, but the remaining drawbacks made me glad and start searching for a more elegant solution.
')
Some leading questions. Are asynchronous HTTP requests possible? - Yes. Is it possible to report using a single byte that a part of a large task has been completed? - Yes. Can we gradually (consistently) receive and process data using XMLHttpRequest.onreadystatechange ? - Yes. We can even use HTTP headers to send a pre-notification of the total duration of the task being performed (if this is possible in principle).

The solution is simple. The founded page is the control panel. With the remote, you can start and stop the task. This page initiates XMLHttpRequest - starts the execution of the main task. In the process of performing this task (inside the main loop), the script sends the client one byte - a space character. On the console in the onreadystatechange handler onreadystatechange we, by getting byte by byte, will be able to conclude about the progress of the task.

The scheme is as follows. Operation script:

 <?php set_time_limit(0); for ($i = 0; $i < 50; $i++) // ,    50 { sleep(1); //   echo ' '; } 

XMLHttpRequest.onreadystatechange handler:

 xhr.onreadystatechange = function() { if (this.readyState == 3) { var progress = this.responseText.length; document.getElementById('progress').style.width = progress + '%'; } }; 

However, there are only 50 iterations. We know about this because we ourselves have determined their number in the script file. And if we do not know, or the amount can vary? With readyState == 2 we can get information from the headers. Let's take this and use to determine the number of iterations:

 header('X-Progress-Max: 50'); 

And on the remote get and remember this value:

 var progressMax = 100; xhr.onreadystatechange = function() { if (this.readyState == 2) { progressMax = +this.getResponseHeader('X-Progress-Max') || progressMax; } else if (this.readyState == 3) { var progress = 100 * this.responseText.length / progressMax; document.getElementById('progress').style.width = progress + '%'; } }; 

The overall scheme should be clear. Let's talk now about the pitfalls.

First, if the output_buffering option is enabled in PHP, you need to take this into account. Everything is simple here: if it is enabled, then when the script is run ob_get_level() will be greater than 0. It is necessary to bypass buffering. Also, if you use the Nginx FastCGI PHP bundle, you need to consider that both FastCGI and Nginx itself will buffer the output. The latter will do this if it is going to compress the data for sending. The problem is solved simply:

 header('Content-Encoding: none', true); 

If the problem with gzip can be solved inside the PHP script itself, then you can force FastCGI to transfer data immediately by correcting the server configuration:

 fastcgi_keep_conn on; 

In addition, either Nginx, or FastCGI, or Chrome itself consider that initiating the reception and transmission of the response body, which contains only one byte, is too wasteful. Therefore, you need to precede the entire operation with additional bytes. We need to agree, let's say that the first 20 spaces should not mean anything at all. On the PHP side, they just need to be spat out into the output, but in the onreadystatechange handler onreadystatechange they need to be ignored. In my opinion, since the entire configuration component is transmitted in the headers, then this number of ignored spaces is also better to be conveyed in the header. Let's call it padding .

 <?php header('X-Progress-Padding: 20', true); echo str_repeat(' ', 20); flush(); // ... 

On the client side, this also needs to be taken into account:

 var progressMax = 100, progressPadding = 0; xhr.onreadystatechange = function() { if (this.readyState == 2) { progressMax = +this.getResponseHeader('X-Progress-Max') || progressMax; progressPadding = +this.getResponseHeader('X-Progress-Padding') || progressPadding; } else if (this.readyState == 3) { var progress = 100 * (this.responseText.length - progressPadding) / progressMax; document.getElementById('progress').style.width = progress + '%'; } }; 

Where does the number 20 come from? If you tell me - I will be very grateful. I installed it experimentally.

By the way, about setting PHP output_buffering . If you have complex buffering and you do not want to break it, you can use this function:

 function ob_ignore($data, $flush = false) { $ob = array(); while (ob_get_level()) { array_unshift($ob, ob_get_contents()); ob_end_clean(); } echo $data; if ($flush) flush(); foreach ($ob as $ob_data) { ob_start(); echo $ob_data; } return count($ob); } 

With it, you can bypass all levels of buffering, output data directly, after which all buffers are restored.

By the way, why is the space used to notify you about the completed part of the task? Simply because almost any format of data presentation on the web cannot be spoiled by such spaces. You can apply this method of sending a notification of the progress of an operation, and after all this, display a report on the results in JSON.

If you put everything in order, slightly optimize and supplement the code with all the features that can be useful, you will get this:




Instead of introducing before the cut: Please do not kick. Googling before working on the scheme did not give any sane results, so it was necessary to invent it, which is why I decided to present it in this first publication.