JavaScript and SEO in 2016

Are you still giving an HTML version of your SPA application to search engines? 2016 is the yard, times have changed.


AJAX as a technology was finally formed by 2005. In 2006, the W3C released the first preliminary specification of the XMLHttpRequest object in an attempt to create an official web standard. In 2009, Google made a proposal to make AJAX-pages accessible to search engines. Then they could not interpret JavaScript, so a number of practices were proposed that ensured that search engines could index an AJAX application.

Webmasters had to generate the HTML version of their site, and from the pages with the address? _Escaped_fragment = page to give a rendered HTML version of the page with the processed JavaScript. On this occasion, there is a detailed guide from Yandex and Google .
')
However, since October 2015, Google has marked its manual as obsolete, deprecated. Google's blog entry claims that the search engine spider is able to interpret web pages in the same way as modern browsers.

Practice

So, there is a one-page website on AngularJS. We need to ensure that it is indexed in Google with minimal gestures and at the same time check whether Google actually executes all the JavaScript code.

First of all, enable HTML5 History API Mode for the UI-router:

angular.module('app', ['ui.router']) .config(function($locationProvider) { $locationProvider.html5Mode(true); }); 

Now you need to configure nginx so that all requests are sent to index.html:

 location / { try_files $uri $uri/ /index.html =404; } 

Actually, everything. This is enough for Google to start indexing the site. As a proof I can bring my personal project seo11.ru.

This is a SPA application on Angular 1.5, but all pages are indexed, even without sitemap.xml.

The content of the pages is inserted via ui-view, but it is easily accessible by Google Spider:


With the help of $ stateProvider, for each page of the site, unique title and description are registered:

 <title data-ng-bind="title">SEO 1:1</title> <meta name="description" content="{{description}}"> 

Both options work:


Google works out the page completely, exactly as your browser does. Up to the fact that the text in this block is searchable:

 <p ng-show="isFirefox">    Firefox...</p> 

And in this block is not available:

 <p ng-if="isFirefox">    ...</p> 

The difference is that ng-show just adds display: hide to the element, and ng-if cuts the element straight out of the DOM.

And Yandex?

The Yandex robot can also index JavaScript, but for now as part of the experiment. This is described in detail in the blog . Unfortunately, this does not work for all sites and at the moment you cannot be sure that your JS site will be indexed.

Alas, but in this case, the only solution that can be proposed for correct site indexing is the use of HTML copies according to the scheme mentioned earlier.

However, it is not advisable to spend significant resources on this, since sooner or later, Yandex will fully execute the JS code for all sites.

Today, the easiest way to generate an HTML version of a site is to use a headless browser for page ranking, for example, PhantomJS. You can do this on the fly or give away cached pages in advance (there are even special services, for example, prerender.io).

Personally, I use the html-snapshots plugin, which is launched by Gulp during the build phase of the project.