How fast is AMP really?
Google's Accelerated Mobile Pages (AMP) project has caused some confusion for ideological reasons, but the technology itself has not been disassembled in detail. A few weeks ago, Ferdy Cristant wrote about the unfair advantage that AMP content gets by preloading . It made me think: how well does the AMP really work ? I have seen tests like Ferdi’s when comparing one or two pages, but haven’t yet met with comprehensive objective tests.
Assessing AMP performance is actually not as simple as it seems. You need to consider at least four contexts:
As Ferdy noted, when you navigate the AMP article from Google search, it loads instantly. A small lightning icon AMP is more appropriate here than ever. But you do not notice that Google provides such an instantaneous transition by actively preloading AMP documents in the background.
')
In the search carousel, this is literally a frame where the entire AMP document is loaded. If you click on this AMP page, then it is already loaded in the background - and immediately displayed on the screen.
AMP works great in the context of Google search. Like any page preloaded in the background. The only advantage of AMP in this context is the initial launch provided by Google .
In other words, an AMP performance assessment based on how these pages are loaded in the search results does not say anything about the effectiveness of AMP itself, but rather about the effectiveness of preloading content.
In his post, Ferdy analyzed a page from Scientas. He found that without preloading it opens far from instantly. On the 3G emulation article Scientas AMP shows you a white screen for 3.3 seconds.
You might think that this happens with only one page. Like, there is a large variation in speed and, perhaps, Scientas is only a single example. Just a remark, so let's see.
The first thing I looked at the news. I do not recommend it to anyone, but I had to.
When the AMP article came across, I added its URL to the table. It doesn't matter which topic or from which site: if AMP is indicated, the link was placed in a table. The only filtering was to not include more than two URLs from one domain.
In the end, after this filtering turned out a list of 50 different articles of AMP. I drove them through WebPageTest to emulate a 3G connection on Nexus 5. Each page was built via AMP, and also downloaded from the source server .
AMP consists of three main parts:
When we talk about the AMP library, we mean AMP JS and AMP HTML together. HTML AMP is both a subset of HTML (there are limitations that you can and can’t use) and an addition to it (HTML AMP includes a number of custom AMP components and properties). AMP JS is a library that gives you these custom elements, and also produces various optimizations for AMP-based documents. Since HTML, CSS and JS are the basis, you can completely build a document using the AMP library without using the Google AMP cache.
The AMP library must provide some level of consistency in performance. For the most part she copes.
Most test pages loaded within a reasonable range of each other. But at both ends of the spectrum, some deviations were observed: the minimum values turned out to be rather low, and the maximum values - frighteningly large.
Basically, AMP performance is relatively predictable. However, the numbers also indicate that AMP support does not give a 100% guarantee that the page will be quick or easy. As with any other technology, it is possible to create an AMP document that will be slow and heavy.
Any statement that AMP provides a certain level of performance depends on the attitude to extremes and on what you consider “performance”. If you are trying to build an entire site on the basis of AMP, then it should be understood: although it is unlikely to turn out to be too bloated, it will not immediately load at lightning speed. This will require some work.
At least this applies to the library itself. Perhaps the AMP cache will show a significant increase in speed?
The AMP library itself helps, but not as much as one might think. Maybe Google cache will add speed?
The Google AMP Cache is an AMP document delivery CDN. Like most CDNs, it caches documents and applies a number of optimizations to them. The cache also includes a verification system that the document is a valid AMP document. When you see an AMP icon, for example, in a Google search, the document is downloaded from Google AMP Cache.
I again drove the same 50 pages through WebPagetest. This time I downloaded every page from Google AMP CDN. Pete Minen did me a favor and shared a script for WebPagetest, which preheats connections with Google CDN, so that the result would look more like what you see in reality.
When loaded from the cache, AMP pages showed a noticeable performance improvement in all respects.
In general, the benefits of the cache are quite substantial. Although the maximum values are still depressing (the slightly increased maximum is explained mainly by the difference in banners in different tests). But the median average, which corresponds to most AMP documents, is better for all indicators.
The improvement is not surprising, given the various performance optimizations that the CDN automatically performs, including:
Once again, it is worth noting that none of these optimizations require the use of AMP. Most of the major CDN providers do the same. You yourself can automate all these optimizations during the build process.
I'm not saying that you need to drop the Google cache, just keep in mind that you can and should use these optimization methods, whether you use AMP or not. There is nothing unique about AMP itself or even AMP cache.
We made sure that the AMP library itself gives only a small performance boost, but the cache with all the optimizations takes the download speed to a new level.
One of the arguments in favor of AMP is that it does not need to be an “expert” to make a productive website. Although I would argue with the fact that many of the sites we have seen can be called "productive", it makes sense to compare AMP documents with their canonical equivalents.
For the next test round, I took the canonical version of each page and checked it under the same conditions. It turned out that AMP documents more often outperform their equivalents without AMP, which indicates insufficient optimization of the latter.
For a second, let's forget about the Google cache and again compare the AMP library with the canonical pages.
The metrics "Start Download" and "Speed Index" do not notice much benefit from the library. In fact, the start of rendering in AMP documents is even slower .
This is not too surprising. As mentioned above, AMP documents use the AMP JS library for many optimizations and resource loading. When you rely on this JavaScript to display the page, you immediately get hit on the visualization indicators. This is observed until the AMP cache comes into play, which returns normal values for the start of rendering and the speed index.
However, according to other indicators, AMP is clearly superior to the canonical version.
The verdict on the effectiveness of AMP is a bit ambiguous. On the one hand, for all equal, the use of AMP does not necessarily accelerate page loading. There is no guarantee that AMP will not slow down or wipe your data.
On the other hand, AMP documents are usually faster than the original versions. More efficient distribution of load in AMP significantly reduces overhead. Suddenly, publishers who like to include third-party scripts on canonical pages dramatically reduce (at least, they are forced to do so) the number of third-party scripts for AMP pages.
The biggest advantage of AMP is not the library, because its functions can be implemented independently. This is not a cache - many of its optimizations are implemented through a good build script, and any decent CDN provider has all of them. This does not mean that in the AMP JS library or the cache there are not really the right things - they are. They just do not give the biggest difference in terms of performance.
The main advantage of AMP is restrictions on how much content can be squeezed into one page.
For example, here is a graph showing all the requests for the same page, written under the AMP requirements (right), compared to the canonical version (left). Sorry for scrolling.
Comparison of load charts for the canonical version of the article (left) and the AMP version (right). AMP restrictions minimize the number of requests
The page size in the 90th percentile for the canonical version is 5229 KB. The size in the 90th percentile for AMP documents from the same source is 1553 KB, the difference is about 70%. The number of requests in the 90th percentile for canonical versions of pages is 647, for AMP documents - 151. A decrease of almost 77%.
The limitations of AMP really reduce the size of the pages and remove extraneous trash from them. Publishers are ready to make this concession for the sake of content distribution services from Google, but are hesitant to do so for canonical versions of pages.
If we evaluate AMP in terms of speeding up the Web, then the evidence is not particularly convincing. For all sites, the AMP version works in addition to the non-AMP version.
Absolutely. Everyone has. Sites.
And usually these versions without AMP are heavy and slow. If you read the news on these sites and did not specifically click on the AMP button, then AMP will not improve the page loading. AMP does not solve the main problem; he just hides her a little.
Time will tell if anything will change. Perhaps, as in the beginning of the transition from mobile sites
Assessing AMP performance is actually not as simple as it seems. You need to consider at least four contexts:
- How well does AMP work in the context of Google search?
- How well does the AMP library work as a standalone framework?
- How well does AMP work when loading from cache?
- How well does AMP work compared to a canonical document?
How well does AMP work in the context of Google search?
As Ferdy noted, when you navigate the AMP article from Google search, it loads instantly. A small lightning icon AMP is more appropriate here than ever. But you do not notice that Google provides such an instantaneous transition by actively preloading AMP documents in the background.
')
In the search carousel, this is literally a frame where the entire AMP document is loaded. If you click on this AMP page, then it is already loaded in the background - and immediately displayed on the screen.
AMP works great in the context of Google search. Like any page preloaded in the background. The only advantage of AMP in this context is the initial launch provided by Google .
In other words, an AMP performance assessment based on how these pages are loaded in the search results does not say anything about the effectiveness of AMP itself, but rather about the effectiveness of preloading content.
How well does the AMP library work as a standalone framework?
In his post, Ferdy analyzed a page from Scientas. He found that without preloading it opens far from instantly. On the 3G emulation article Scientas AMP shows you a white screen for 3.3 seconds.
You might think that this happens with only one page. Like, there is a large variation in speed and, perhaps, Scientas is only a single example. Just a remark, so let's see.
The first thing I looked at the news. I do not recommend it to anyone, but I had to.
When the AMP article came across, I added its URL to the table. It doesn't matter which topic or from which site: if AMP is indicated, the link was placed in a table. The only filtering was to not include more than two URLs from one domain.
In the end, after this filtering turned out a list of 50 different articles of AMP. I drove them through WebPageTest to emulate a 3G connection on Nexus 5. Each page was built via AMP, and also downloaded from the source server .
AMP consists of three main parts:
- AMP HTML
- AMP JS
- Amp cache
When we talk about the AMP library, we mean AMP JS and AMP HTML together. HTML AMP is both a subset of HTML (there are limitations that you can and can’t use) and an addition to it (HTML AMP includes a number of custom AMP components and properties). AMP JS is a library that gives you these custom elements, and also produces various optimizations for AMP-based documents. Since HTML, CSS and JS are the basis, you can completely build a document using the AMP library without using the Google AMP cache.
The AMP library must provide some level of consistency in performance. For the most part she copes.
Most test pages loaded within a reasonable range of each other. But at both ends of the spectrum, some deviations were observed: the minimum values turned out to be rather low, and the maximum values - frighteningly large.
| Metrics | Min | Max. | Median | 90th percentile |
|---|---|---|---|---|
| Start rendering | 1765 ms | 8130 ms | 4617 ms | 5788 ms |
| Visually complete view | 4604 ms | 35,096 ms | 7475 ms | 21,432 ms |
| Speed index | 3729 | 16 230 | 6171 | 10,144 |
| The size | 273 KB | 10,385 KB | 905 KB | 1,553 KB |
| Requests | 14 | 308 | 61 | 151 |
Basically, AMP performance is relatively predictable. However, the numbers also indicate that AMP support does not give a 100% guarantee that the page will be quick or easy. As with any other technology, it is possible to create an AMP document that will be slow and heavy.
Any statement that AMP provides a certain level of performance depends on the attitude to extremes and on what you consider “performance”. If you are trying to build an entire site on the basis of AMP, then it should be understood: although it is unlikely to turn out to be too bloated, it will not immediately load at lightning speed. This will require some work.
At least this applies to the library itself. Perhaps the AMP cache will show a significant increase in speed?
How well does AMP work when loading from cache?
The AMP library itself helps, but not as much as one might think. Maybe Google cache will add speed?
The Google AMP Cache is an AMP document delivery CDN. Like most CDNs, it caches documents and applies a number of optimizations to them. The cache also includes a verification system that the document is a valid AMP document. When you see an AMP icon, for example, in a Google search, the document is downloaded from Google AMP Cache.
I again drove the same 50 pages through WebPagetest. This time I downloaded every page from Google AMP CDN. Pete Minen did me a favor and shared a script for WebPagetest, which preheats connections with Google CDN, so that the result would look more like what you see in reality.
logdata 0
navigate https://cdn.ampproject.org/c/www.webpagetest.org/amp.html
logdata 1
navigate %URL%When loaded from the cache, AMP pages showed a noticeable performance improvement in all respects.
| Metrics | Min | Max. | Median | 90th percentile |
|---|---|---|---|---|
| Start rendering | 1427 ms | 4828 ms | 1933 ms | 2291 ms |
| Visually complete view | 2036 ms | 36,001 ms | 4924 ms | 19,626 ms |
| Speed index | 1966 | 18,677 | 3277 | 9004 |
| The size | 177 KB | 10,749 KB | 775 KB | 2.079 KB |
| Requests | 13 | 305 | 53 | 218 |
In general, the benefits of the cache are quite substantial. Although the maximum values are still depressing (the slightly increased maximum is explained mainly by the difference in banners in different tests). But the median average, which corresponds to most AMP documents, is better for all indicators.
The improvement is not surprising, given the various performance optimizations that the CDN automatically performs, including:
- Caching images and fonts
- Limit the maximum size of images
- Compression of images on the fly, as well as the generation of options of different sizes with the addition of srcset for them
- Using HTTP / 2 and HTTPS
- Clearing HTML Comments
- Auto-enable hints for resources like
dns-prefetchandpreconnect
Once again, it is worth noting that none of these optimizations require the use of AMP. Most of the major CDN providers do the same. You yourself can automate all these optimizations during the build process.
I'm not saying that you need to drop the Google cache, just keep in mind that you can and should use these optimization methods, whether you use AMP or not. There is nothing unique about AMP itself or even AMP cache.
How well does AMP work compared to a canonical document?
We made sure that the AMP library itself gives only a small performance boost, but the cache with all the optimizations takes the download speed to a new level.
One of the arguments in favor of AMP is that it does not need to be an “expert” to make a productive website. Although I would argue with the fact that many of the sites we have seen can be called "productive", it makes sense to compare AMP documents with their canonical equivalents.
For the next test round, I took the canonical version of each page and checked it under the same conditions. It turned out that AMP documents more often outperform their equivalents without AMP, which indicates insufficient optimization of the latter.
| Metrics | Min | Max. | Median | 90th percentile |
|---|---|---|---|---|
| Start rendering | 1763 ms | 7469 ms | 4227 ms | 6298 ms |
| Visually complete view | 4231 ms | 108 006 ms | 20,418 ms | 54,546 ms |
| Speed index | 3332 | 45 362 | 8152 | 21 495 |
| The size | 251 KB | 11,013 KB | 2762 KB | 5229 KB |
| Requests | 24 | 1743 | 318 | 647 |
For a second, let's forget about the Google cache and again compare the AMP library with the canonical pages.
The metrics "Start Download" and "Speed Index" do not notice much benefit from the library. In fact, the start of rendering in AMP documents is even slower .
This is not too surprising. As mentioned above, AMP documents use the AMP JS library for many optimizations and resource loading. When you rely on this JavaScript to display the page, you immediately get hit on the visualization indicators. This is observed until the AMP cache comes into play, which returns normal values for the start of rendering and the speed index.
However, according to other indicators, AMP is clearly superior to the canonical version.
Increased productivity ... but for whom?
The verdict on the effectiveness of AMP is a bit ambiguous. On the one hand, for all equal, the use of AMP does not necessarily accelerate page loading. There is no guarantee that AMP will not slow down or wipe your data.
On the other hand, AMP documents are usually faster than the original versions. More efficient distribution of load in AMP significantly reduces overhead. Suddenly, publishers who like to include third-party scripts on canonical pages dramatically reduce (at least, they are forced to do so) the number of third-party scripts for AMP pages.
The biggest advantage of AMP is not the library, because its functions can be implemented independently. This is not a cache - many of its optimizations are implemented through a good build script, and any decent CDN provider has all of them. This does not mean that in the AMP JS library or the cache there are not really the right things - they are. They just do not give the biggest difference in terms of performance.
The main advantage of AMP is restrictions on how much content can be squeezed into one page.
For example, here is a graph showing all the requests for the same page, written under the AMP requirements (right), compared to the canonical version (left). Sorry for scrolling.
Comparison of load charts for the canonical version of the article (left) and the AMP version (right). AMP restrictions minimize the number of requests
The page size in the 90th percentile for the canonical version is 5229 KB. The size in the 90th percentile for AMP documents from the same source is 1553 KB, the difference is about 70%. The number of requests in the 90th percentile for canonical versions of pages is 647, for AMP documents - 151. A decrease of almost 77%.
The limitations of AMP really reduce the size of the pages and remove extraneous trash from them. Publishers are ready to make this concession for the sake of content distribution services from Google, but are hesitant to do so for canonical versions of pages.
If we evaluate AMP in terms of speeding up the Web, then the evidence is not particularly convincing. For all sites, the AMP version works in addition to the non-AMP version.
Absolutely. Everyone has. Sites.
And usually these versions without AMP are heavy and slow. If you read the news on these sites and did not specifically click on the AMP button, then AMP will not improve the page loading. AMP does not solve the main problem; he just hides her a little.
Time will tell if anything will change. Perhaps, as in the beginning of the transition from mobile sites
http://m. to adaptive sites, publishers slowly harness. But right now it seems that the calls for the transition to AMP do exactly what you think: they promote AMP, not productivity.Source: https://habr.com/ru/post/353256/
All Articles