React and SEO: how to make friends?

As you know, search engine optimization of single-page applications is not an easy task and its solution may require significant labor costs, special skills from developers and financial costs from the customer. What solutions does the community propose to, as quickly as possible and at minimal cost, make the React website search engine friendly?

Some banal about SEO

About 92% of traffic comes from the first page of search engine results , and 75% of traffic comes from the first five websites.

Google occupies more than 90% of the search engine market .

Search Engine Optimization (SEO) is the process of structuring and organizing a site in order to increase the volume and improve the quality of its traffic by increasing its position and frequency in search engine results, focusing on keywords that reveal the specifics of the site.

The main goal of the SEO process is to improve the visibility of a website on the Internet and increase website traffic.

The work of search engines is based on the relentless caching of website content. Due to the fact that the process is automated, it is very important that the content be structured and formatted in a manner understood by the machine. The SEO process includes optimizing the website's performance, reworking its content in order to increase the relevance of the content to keywords, as well as providing the site with tools that help search robots understand it.

This may seem fairly simple, but in the case of sites built on React, this is not always the case.

React and SEO

SPA in brief

A one-page application or SPA is a web application or website that interacts with the user by dynamically changing the current pages, rather than downloading new pages from the server. SPA work quickly, because most resources (HTML + CSS + Scripts) are downloaded only once during the entire period of their life. The application receives and gives only data.

One example of technology that can be used to create a SPA is ReactJS , a library that is well optimized to run on user browsers ( more here. )

SPA's main problems with SEO

SPA is, in fact, a JavaScript program running in a browser, and if a search web bot cannot execute scripts, it will not be able to get a properly rendered page and index it.

In October 2015, Google made an important announcement that its bots would run JS files on websites if they were allowed to do so. And, although this statement sounds very positive, and its search bots themselves become more sophisticated, relying on their ability to execute JavaScript is risky . Google bot can refuse to perform javascript for a variety of reasons ( read more here ):

• non-compliance with the rule of "gold 5 sec" (more here and here);
• inability to crawl a site: Google systems should be able to crawl a site, given its structure (more here)
• Impossibility to analyze the site: Google systems should not experience problems during the analysis of the site using the technologies used to form its pages (more here);
• errors in the code, etc.

Other search engine bots such as Yahoo, Bing, Baidu, and others do not support JavaScript and see the SPA pages empty.

For these reasons, you have to look for ways to render the SPA site on the server side in order to give guaranteed bots the opportunity to see their contents in the correct form.

Solutions SEO problems for SPA websites

There are two main ways to solve SEO problems facing SPA websites:

1. Isomorphic (universal) SPA;
2. Pre-rendering.

Additional SEO Improvements for React Web Sites:

• React Router v4 : component for creating SEO-friendly "routing" in React applications;
• React Helmet : perhaps the most important component for providing SEO React-applications, which makes it possible to manage meta-tags of an application and is notable for its ease of use.

How isomorphic React helps with SEO?

Isomorphic React site automatically determines if JavaScript is disabled on the client side. If JavaScript is disabled, the script is executed on the server and the result (static HTML + CSS) is sent to the client. In this case, all the attributes and content necessary for SEO are present during the download.

If JavaScript is enabled, an isomorphic React site in the simplest case can be loaded in the usual way, when everything is rendered in the browser, or cleverly : partial rendering is performed on the server side, which means that only the first DOM rendering is done on the server, and all subsequent ones are produced directly in browser. In other words, the browser receives the fully rendered initial DOM and JavaScript, and when the state of the application changes, it is responsible for all subsequent updates.

It is believed that an isomorphic solution is the best method for solving SEO problems of SPA web applications, but unfortunately, unfortunately, similar solutions for React SPA, at the moment, are difficult to implement :

• Popular React boilers, for example, create-react-app , react-boilerplate do not support isomorphism. Dan Abramov , developer of create-react-app :
  
  

  Ultimately, server-side rendering is very difficult to add in a meaningful way, without making responsible decisions either. At the moment we are not going to make such decisions;

• existing isomorphic boilerplates are difficult to master;
• existing solutions are not perfect:
  
  
  • often the code on the server and on the client has many differences;
  • in the case of almost complete coincidence of the code base, the code becomes slow and prone to failure.
  
  
  

  It seems that many developers will support the opinion expressed by MrCheater in the discussion of the article "Isomorphic React-applications: performance and scaling" :
  
  

  The “isomorphic web applications” theme has a lot of haters. Alas. Although it is easy to understand why, everything is very difficult to program, hundreds of articles on this topic, and the output still makes a terrible client with a weight of 3 megabytes. But sometime soon all the problems and approaches to their solution will be standardized, and we will have lightweight isomorphic single page application.

  
  
  

  Solving SEO problem by pre-rendering

  
  
  

  Pre -rendering is the process of pre-rendering pages on a website in order to prepare them for viewing by a search-based web bot. The prerender service intercepts requests to the website, determines the type of client using the user-agent request, and if the request was made by a web bot, the service sends back the previously rendered cached static version of the site. If the request was made by a non-search web-bot, the usual page is loaded. Rendering is used only to optimize work with web bots and, possibly, with outdated browsers. Prerender services, such as Prerender.cloud and others like it, use “headless” browsers, most often Headless Chrome . This approach allows you to use the latest JavaScript frameworks such as React, Ember, and Angular to build a website and not rely on server rendering.

  
  
  

  Advantages of pre-rendering:

  
  
  
  • The prerender is capable of performing all types of modern JavaScript and provides static HTML;
  • prerender supports the latest web innovations;
  • Minimal modification of the code base of the web application is required, if at all required ;
  • reduced costs for the development and maintenance of the website ;
  • ease of implementation.
  
  
  

  Disadvantages of pre-rendering :

  • not suitable for pages that display frequently changing data;
  • not acceptable for pages that contain personal user data. However, such pages are of little use for SEO and should not be indexed;
  • in the case of a large site with a significant number of pages, the process of pre-rendering may take considerable time;
  • Prerender services are paid.
  
  
  

  Prerender where to get them

  
  
  

  The community has developed a significant number of prerenders: see, for example, here.

  
  
  

  Among the many options, it seems interesting OpenSource Prerender is an open source version of the prerender used in the prerender.io service, which can be implemented on your own server with the supporting Node.js.

  
  
  

  Another interesting fact is that it does not require a server, the prerender is the Prerender SPA Plugin , which is connected by a simple modification of webpack.config.js, for example, to create-react-app or react-boilerplate . The pre-rendering process is carried out in the process of “building” the site and the result is placed in index.html.

  
  
  

  You can also use ready-made services that provide pre-rendering services, a list of some is given here and here.

  
  
  

  Here you can also get acquainted with the practical experience of working with presenters.

  
  
  

  Of particular interest is the Roast.io service, which provides both CDN and pre-rendering services (!) . The service also automatically (if there are none) adds “screenshot link unfurls” meta tags that improve the display of the website in social networks.
  
  The service is located on AWS CDN , works on the HTTPS protocol, is configured for SPA and React in particular, is very easy to use. There is a free hosting option (10GB bandwidth, 500 pages SSR).

  
  
  

  So:

  
  
  

  Pre-listing, being not an ideal solution, looks more preferable for a significant part of developers of React-applications due to the fact that:
  
  

  • No need to modify the application code - you can safely use your favorite boilerplate;
  • there is a wide range of implementations - “middleware”, “webpack”, “CDN + prerender”, “opensource” and paid;
  • implementation of pre-rendering is a simple process, especially in the case of “CDN + prerender” implementations;
  • there is strong support from Google in the form of Headless Chrome ;
  • reduced development and support costs;
  • There are options for using free prerender web services.

  
  
  However , you need to think about the isomorphic (universal) version of the site , if:
  
  

  • the web application displays frequently changing data;
  • The web application contains hundreds of pages, because the pre-rendering process can take a considerable time.

  
  
  If the website contains only static pages (HTML + CSS), then neither isomorphism nor re-rendering is required .