Acceleration of loading AJAX applications, + preloading images
It all started with the creation of complex AJAX applications using java technologies GWT, GXT, Spring, Hibernate, Terracota, AndroMDA, ActiveMQ and many other magic sounds that hide all the power and power of java technologies created by tens of thousands of brilliant programmers for the second millennium in a row ...
But the article is not about that. It was necessary to solve a modest, but very interesting task - without mastering all the manufacturability, forethought and perfection of solutions based on the java platform, to reduce the load time of the client part of the application.
Input data: text data size js, css, xml, html, images about ~ 1.2MB (+ flash), download time in Moscow more than one and a half minutes, when following links, noticeable time (1-15 seconds) for downloading images, re-accessing the picture was downloaded again.
The data are approximate for the following reasons.
1) GWT for each browser generates its js size, which is about 700KB
2) At first optimization was done and then the idea came that it could be useful for the community
')
As it turned out, nothing terrible happened to me, they were afraid of the eyes, and their hands boldly did and reworked their work, but first things first.
Content compression by web server
The server turned out to be Tomcat. The settings file was intuitive, made in compliance with the xml format and everything in it was good, compression was turned on, an effective nio connector was turned on - but this wonderful connector in all descriptions did not want to use compression.
Changing the connector to the default HTTP / 1.1 made compression work wonderfully. A thought settled in my head, how is it that there is a compression in the good old connector, but not in the hi-tech technology? - there is something wrong.
In the process of studying the documentation for Tomkat it turned out that there is compression. Also in the documentation there were some more interesting options allowing to compress files according to various features such as file size and type, you can also specify the quality of compression.
Compression nevertheless did not work from the acquired knowledge.
Further reflections on the depth, breadth and high integration of the java platform prompted me that the application may be able to independently configure the server to fit its needs.
It was decided to check this assumption by asking the developers, as it turned out java programmers really turned off this feature for the entire application for the nio connector as it interfered with the chat.
Further in this direction everything went predictably. Java developers are still very busy and have not corrected the bug. Compression works with HTTP / 1.1 and in the configuration file the commented nio connector flaunts and the 700KB js file already has 190KB of message and it was perceived as a mountain from its shoulders.
A piece of configuration file configuring connector.
Disable extra files
The second thing, it was decided to see that for js are loaded in such quantities
They loaded about 10 pieces and about 7 of them did not belong to GWT in any way. As usual curiosity got the better and all incomprehensible js were turned off. What a surprise when after cleaning the cache and reloading the page everything worked fine (in firefox 3.1 under Linux). This fact was reported to the developers of the application, with the question of whether all this is really used? After a brief reflection, the developers left only one of them proposed by me to delete the files, saying that this one is necessary for IE. Nevertheless, discarding 6 unnecessary requests and about 70KB of jammed js code significantly reduced the load time.
File association
The possibility of merging js files into one file was also studied, this possibility was not convenient at this stage of development of the project, since new builds sometimes run 3 per day, and the possibility of merging files was limited to two files of insignificant size. Perhaps this will be done later by means of a simple script on bash delivered in kroon :) `cat file1 file2> file3; mv file 3 file1; egrep -v -o "file2" file.html> tmp.file.html; mv tmp.file.html file.html`, well, or something like that with the necessary checks and backups.
Compress js by means of yuicompressor
developer.yahoo.com/yui/compressor
In order to reduce it even more, the optimizer from Yahoo YUI was studied; it really reduced, albeit unimportantly, the size of the uncompressed js file passed through obfuscation using GWT tools, but after compression by the connector the file size was slightly larger than the uncompressed file by this tool. For this reason, further excavations in this direction were stopped.
Compressing CSS with yuicompressor
There is only one CSS file, there are many whitespace characters in it and it was written without optimizations, for this reason it was driven through yuicompressor, but the result was only 700 bytes in the end, which was not significant in my scales, for this reason it was decided not to use yuicompressor in the current actively developing version in general.
Image Optimization
There are about 600 of them. It was decided to drive these images through automated services and this gave a gain of approximately 15–18%, about 1.5 megabytes out of 10 possible.
Unfortunately, the source code of most of these images has not been preserved, and it may not have been originally since they were immediately purchased in png and jpg formats.
Further experiments showed that the brute force method could achieve another 1% reduction in png files which are most likely to be done before sending the project to the voyage.
I didn’t optimize the pictures for this reason
Optimization was carried out with knowledge and understanding
www.artlebedev.ru/tools/technogrette/img
Geographical distribution of servers
At this point, the boot time ranged from 15 to 30 seconds in Moscow and about 4-6 seconds in California. This injustice was due to the location of the amazon data center. In the storehouse of amazon s3 by this time all the images were already stored.
aws.amazon.com/s3 .
Another solution of Amazon has helped to eliminate this annoying misunderstanding in reasonable days. Aws.amazon.com/cloudfront
After installing cloudfront, the load time in Moscow was equal to the load time in California, from which it can be assumed that in most places on the planet the load time was 4-6 seconds if the “last mile” allowed.
Preloading images.
There was one more significant problem, noticeable loading time of images in the process of using the AJAX application.
It was solved through the creation of a script pre-loading images. The tasks of the script included a minimum of tasks.
1) Download images from a predefined list
2) Delete images from the list after downloading, even if it is unsuccessful
3) Stop the download
4) Continue downloading
In the process of writing this script opened some interesting features of js.
Uploading occurs asynchronously, so the usual loop (while, for) was not suitable, since the 600 simultaneously uploaded files hung the browser for some time, depending on the power of the computer.
This was done by inserting an anonymous function into onload and onerror which caused the parent function, which turned out to be a recursion that did not allow freeing up unused memory. This was manifested in IE through an “out of memory” error.
This recursion had to be broken by inventing an effective variant of tail recursion, which allows freeing up unused memory. The solution to this problem was found through the use of a timer with a value of 1.
preloadimg.js (changed on Wed Oct 14 07:53:31 PDT 2009)
The image preloading function was supposed to be used as follows.
After the introduction of this code into operation, the loading of images ceased to be felt at all. The only case where you can see the download of pictures, if you very quickly enter your login / password and go to the heaviest page, it is expected that in real life this will be almost incredible.
Total
Today's application check showed
Downloadable size with flash 591KB, of which
Flash 348KB - this part has been optimized, but the details are not fully known to me to write about it.
Html + js + xml + css = 243KB
Download time 5 - 10 seconds (there is a difference from the stated above 4-6 seconds, apparently there were changes on the channel).
When navigating through internal pages, image uploading is rarely seen.
Opportunities for further optimization
css - 700byte
further image optimization - from 1%
Enabling protocol-level compression introduces additional load on the CPU; this can be circumvented in at least two ways.
1) - precompression of files, and distribution of correct headers. As far as I understand in the world of java, this is done easily and without additional costs.
2) - Caching on a proxy server or load balancer (nginx, haproxy, ...)
But this relates to load testing and this is a completely different story, though no less fascinating. They have their own tools, their own colossal problems and interesting solutions :)
ps: I will not give a link to an optimized site, it is still under development and a lot of things are changing.
pps: reloading of images was associated with a temporary S3 glitch, they kind of updated the software and they didn’t get it as expected - the problem was observed for hours 6 and exactly at that moment when the question of how to optimize was studied.
But the article is not about that. It was necessary to solve a modest, but very interesting task - without mastering all the manufacturability, forethought and perfection of solutions based on the java platform, to reduce the load time of the client part of the application.
Input data: text data size js, css, xml, html, images about ~ 1.2MB (+ flash), download time in Moscow more than one and a half minutes, when following links, noticeable time (1-15 seconds) for downloading images, re-accessing the picture was downloaded again.
The data are approximate for the following reasons.
1) GWT for each browser generates its js size, which is about 700KB
2) At first optimization was done and then the idea came that it could be useful for the community
')
As it turned out, nothing terrible happened to me, they were afraid of the eyes, and their hands boldly did and reworked their work, but first things first.
Content compression by web server
The server turned out to be Tomcat. The settings file was intuitive, made in compliance with the xml format and everything in it was good, compression was turned on, an effective nio connector was turned on - but this wonderful connector in all descriptions did not want to use compression.
Changing the connector to the default HTTP / 1.1 made compression work wonderfully. A thought settled in my head, how is it that there is a compression in the good old connector, but not in the hi-tech technology? - there is something wrong.
In the process of studying the documentation for Tomkat it turned out that there is compression. Also in the documentation there were some more interesting options allowing to compress files according to various features such as file size and type, you can also specify the quality of compression.
Compression nevertheless did not work from the acquired knowledge.
Further reflections on the depth, breadth and high integration of the java platform prompted me that the application may be able to independently configure the server to fit its needs.
It was decided to check this assumption by asking the developers, as it turned out java programmers really turned off this feature for the entire application for the nio connector as it interfered with the chat.
Further in this direction everything went predictably. Java developers are still very busy and have not corrected the bug. Compression works with HTTP / 1.1 and in the configuration file the commented nio connector flaunts and the 700KB js file already has 190KB of message and it was perceived as a mountain from its shoulders.
A piece of configuration file configuring connector.
< Connector port ="8080" protocol ="HTTP/1.1"
maxThreads ="64000"
connectionTimeout ="20000"
redirectPort ="8443"
proxyPort ="80"
compression ="on"
compressionMinSize ="4000"
noCompressionUserAgents ="gozilla, traviata"
compressableMimeType ="text/html,text/xml,text/javascript,text/css" />
<!--Connector port="8080" compression="on"
protocol="org.apache.coyote.http11.Http11NioProtocol"
connectionTimeout="20000"
backlog="500"
maxThreads="4"
redirectPort="8443"
proxyPort="80"
compressionMinSize="2048"
noCompressionUserAgents="gozilla, traviata"
compressableMimeType="text/html,text/xml,text/javascript,text/css,application/x-javascript"
socketBuffer="64000"/-->
* This source code was highlighted with Source Code Highlighter .Disable extra files
The second thing, it was decided to see that for js are loaded in such quantities
They loaded about 10 pieces and about 7 of them did not belong to GWT in any way. As usual curiosity got the better and all incomprehensible js were turned off. What a surprise when after cleaning the cache and reloading the page everything worked fine (in firefox 3.1 under Linux). This fact was reported to the developers of the application, with the question of whether all this is really used? After a brief reflection, the developers left only one of them proposed by me to delete the files, saying that this one is necessary for IE. Nevertheless, discarding 6 unnecessary requests and about 70KB of jammed js code significantly reduced the load time.
File association
The possibility of merging js files into one file was also studied, this possibility was not convenient at this stage of development of the project, since new builds sometimes run 3 per day, and the possibility of merging files was limited to two files of insignificant size. Perhaps this will be done later by means of a simple script on bash delivered in kroon :) `cat file1 file2> file3; mv file 3 file1; egrep -v -o "file2" file.html> tmp.file.html; mv tmp.file.html file.html`, well, or something like that with the necessary checks and backups.
Compress js by means of yuicompressor
developer.yahoo.com/yui/compressor
In order to reduce it even more, the optimizer from Yahoo YUI was studied; it really reduced, albeit unimportantly, the size of the uncompressed js file passed through obfuscation using GWT tools, but after compression by the connector the file size was slightly larger than the uncompressed file by this tool. For this reason, further excavations in this direction were stopped.
$ java -jar yuicompressor-2.4.2.jar -o modified_script_yui.js --charset utf-8 --type js modified_script.js
$ ls -lh modified_script*
-rw-r--r-- 1 user user 725K 2009-10-13 11:49 modified_script.js
-rw-r--r-- 1 user user 719K 2009-10-13 11:51 modified_script_yui.js
$ gzip modified_script*
$ ls -lh modified_script*
-rw-r--r-- 1 user user 199K 2009-10-13 11:49 modified_script.js.gz
-rw-r--r-- 1 user user 203K 2009-10-13 11:51 modified_script_yui.js.gz
* This source code was highlighted with Source Code Highlighter .Compressing CSS with yuicompressor
There is only one CSS file, there are many whitespace characters in it and it was written without optimizations, for this reason it was driven through yuicompressor, but the result was only 700 bytes in the end, which was not significant in my scales, for this reason it was decided not to use yuicompressor in the current actively developing version in general.
$ java -jar yuicompressor-2.4.2.jar file.css -o file.css.yuic --type css
$ ls -lh file*
-rw-r--r-- 1 User User 38K 2009-10-06 08:26 file.css
-rw-r--r-- 1 User User 5.7K 2009-10-08 01:18 file.css.gz
-rw-r--r-- 1 User User 30K 2009-10-08 01:18 file.css.yuic
-rw-r--r-- 1 User User 5.0K 2009-10-08 01:19 file.css.yuic.gz
* This source code was highlighted with Source Code Highlighter .Image Optimization
There are about 600 of them. It was decided to drive these images through automated services and this gave a gain of approximately 15–18%, about 1.5 megabytes out of 10 possible.
Unfortunately, the source code of most of these images has not been preserved, and it may not have been originally since they were immediately purchased in png and jpg formats.
Further experiments showed that the brute force method could achieve another 1% reduction in png files which are most likely to be done before sending the project to the voyage.
I didn’t optimize the pictures for this reason
Optimization was carried out with knowledge and understanding
www.artlebedev.ru/tools/technogrette/img
Geographical distribution of servers
At this point, the boot time ranged from 15 to 30 seconds in Moscow and about 4-6 seconds in California. This injustice was due to the location of the amazon data center. In the storehouse of amazon s3 by this time all the images were already stored.
aws.amazon.com/s3 .
Another solution of Amazon has helped to eliminate this annoying misunderstanding in reasonable days. Aws.amazon.com/cloudfront
After installing cloudfront, the load time in Moscow was equal to the load time in California, from which it can be assumed that in most places on the planet the load time was 4-6 seconds if the “last mile” allowed.
Preloading images.
There was one more significant problem, noticeable loading time of images in the process of using the AJAX application.
It was solved through the creation of a script pre-loading images. The tasks of the script included a minimum of tasks.
1) Download images from a predefined list
2) Delete images from the list after downloading, even if it is unsuccessful
3) Stop the download
4) Continue downloading
In the process of writing this script opened some interesting features of js.
Uploading occurs asynchronously, so the usual loop (while, for) was not suitable, since the 600 simultaneously uploaded files hung the browser for some time, depending on the power of the computer.
This was done by inserting an anonymous function into onload and onerror which caused the parent function, which turned out to be a recursion that did not allow freeing up unused memory. This was manifested in IE through an “out of memory” error.
This recursion had to be broken by inventing an effective variant of tail recursion, which allows freeing up unused memory. The solution to this problem was found through the use of a timer with a value of 1.
preloadimg.js (changed on Wed Oct 14 07:53:31 PDT 2009)
* This source code was highlighted with Source Code Highlighter .
- stopPreloadImg = false ;
- arrayImgCached = new Array ();
- arrayImg = null ;
- baseUrl = null ;
- function preloadImg (preloadArray, baseUrl) {
- this .arrayImg = preloadArray;
- this .baseUrl = baseUrl;
- cacheImage ();
- }
- function safeMemoryInRecursion () {window.setTimeout (cacheImage, 0)}
- function cacheImage ()
- {
- if (arrayImg.length> 0 && stopPreloadImg == false ) {
- var img = new Image ();
- img.onerror = function () {safeMemoryInRecursion ()};
- img.onload = function () {safeMemoryInRecursion ()};
- img.src = baseUrl + arrayImg [0];
- arrayImg.shift ();
- arrayImgCached [window.arrayImgCached.length] = img;
- }
- }
The image preloading function was supposed to be used as follows.
After the introduction of this code into operation, the loading of images ceased to be felt at all. The only case where you can see the download of pictures, if you very quickly enter your login / password and go to the heaviest page, it is expected that in real life this will be almost incredible.
Total
Today's application check showed
Downloadable size with flash 591KB, of which
Flash 348KB - this part has been optimized, but the details are not fully known to me to write about it.
Html + js + xml + css = 243KB
Download time 5 - 10 seconds (there is a difference from the stated above 4-6 seconds, apparently there were changes on the channel).
When navigating through internal pages, image uploading is rarely seen.
< html >
< head >
< meta http-equiv ="Content-Type" content ="text/html; charset=utf-8" />
</ head >
< body >
< h1 > </ h1 >
< p >
.
. </ p >
< ol >
< li > </ li >
< li > </ li >
</ ol >
< p >
,
.
</ p >
< script type ="text/javascript" src ="preloadimg.js" ></ script >
<script type= "text/javascript" >
var srcArray= new Array(
"/photo_com/650/3257.jpg" ,
"/photo_com/650/3298.jpg" ,
"/images/404_habra_error.png"
);
preloadImg(srcArray, "http://asha-city.ru" );
</ script >
</ body >
</ html >
* This source code was highlighted with Source Code Highlighter .Opportunities for further optimization
css - 700byte
further image optimization - from 1%
Enabling protocol-level compression introduces additional load on the CPU; this can be circumvented in at least two ways.
1) - precompression of files, and distribution of correct headers. As far as I understand in the world of java, this is done easily and without additional costs.
2) - Caching on a proxy server or load balancer (nginx, haproxy, ...)
But this relates to load testing and this is a completely different story, though no less fascinating. They have their own tools, their own colossal problems and interesting solutions :)
ps: I will not give a link to an optimized site, it is still under development and a lot of things are changing.
pps: reloading of images was associated with a temporary S3 glitch, they kind of updated the software and they didn’t get it as expected - the problem was observed for hours 6 and exactly at that moment when the question of how to optimize was studied.
Source: https://habr.com/ru/post/72151/
All Articles