How to make AJAX indexed
My free translation of yesterday's Google announcement on how to make Ajax indexed.
The Google team is in a hurry to introduce you to a new approach that will allow indexing sites made using AJAX. The developed standard will help both webmasters and users by making the content of interactive AJAX sites available for indexing to any search engine that agreed to participate in this project. We believe that if interactive content becomes available for indexing, this will only improve the content of the Internet.
AJAX sites are quite popular among users, but search engines, as a rule, cannot access the content of such a site. According to our latest survey, about 70% of all sites we checked use some form of JavaScript. Of course, most JavaScript is not AJAX, but still, it would be nice if search engines could index AJAX. In this case, the developers could make their sites more interesting and supply them with a large number of “tricks”, and the search engines, in turn, indexed such sites and displayed them in the search results.
')
The goals we want to achieve with this standard are:
• Minimum website changes.
• The user and the search engine receive the same content (no cloaking)
• The search engine sends the user straight to the page with AJAX (not to a static copy)
• Site owners can verify that their AJAX site is presented correctly and that the search spider has access to all content.
As we present the AJAX indexation by search engines:
• We slightly change the structure of the part of the URL for the page with AJAX
If an AJAX page has the same content each time it is requested, then these pages can appear in search results. We suggest adding a symbol instead of a URL, for example, example.com/page?query#state , which will help to highlight and understand the URL of the pages with AYAX: example.com/page?query# [SYMBOL] state. Based on the study of existing URLs we suggest using "!" For this purpose. (exclamation mark). The proposed URL that will be displayed in the search results will look like this: example.com/page?query# ! State.
• Uses a browser that does not accept headers to generate an HTML page from your web server.
A browser that does not receive headers is used to access the AJAX page and generate HTML code based on the final state of the page in the browser. The browser processes only specially designated URLs. By processing pages on the server side in this way, the site owner can manage the generated HTML code and make sure that all the JavaScript is executed correctly. An example of such a browser is HtmlUnit, an open source browser for Java programs without GUI.
• By removing part of the address, we give access to these URLs to search spiders.
Since some fragments of the address will never be transmitted to the server (since a request that adds a fragment to the address will never be executed), it is necessary to slightly modify the URL used to access the page. This modification, in turn, tells the server, which uses a special browser, to generate HTML code, and not to give a page with JavaScript. The remaining URLs - those used by the user - will be processed normally, without using a special browser. We propose not to use part of the information, but add it as a query parameter along with a special character. Using the previous example, the URL would look like this: example.com/page?query& [SYMBOL] = state. After analyzing today's URLs on the Internet, we suggest using "_escaped_fragment_" as a template. The suggested URL will now look like this: example.com/page?query&_escaped_fragment_=state .
• Show the user the real URL in the search results
It makes sense to direct users directly to AJAX pages, and not to static versions. And this can be done by displaying the actual URL in the search results (such as example.com/page?query# ! State from the example above). The search engine can verify that the text given to it is the same or part of the text given to the user.
(Chart: Katharina Probst)
As a result, we started with a URL that has the following structure:
example.com/dictionary.html#AJAX , the page can be accessed by both the search spider and the user in the form:
example.com/diction.html# ! AJAX which will be indexed as
example.com/dictionary.html?_escaped_fragment_=AJAX and which will be shown to the user as
example.com/dictionary.html# ! AJAX
The proposal of the standard was received and is being finalized: Katharina Probst, Bruce Johnson, Arup Mukherjee, Erik van der Poel and Li Xiao, Google
original: googlewebmastercentral.blogspot.com/2009/10/proposal-for-making-ajax-crawlable.html
The Google team is in a hurry to introduce you to a new approach that will allow indexing sites made using AJAX. The developed standard will help both webmasters and users by making the content of interactive AJAX sites available for indexing to any search engine that agreed to participate in this project. We believe that if interactive content becomes available for indexing, this will only improve the content of the Internet.
AJAX sites are quite popular among users, but search engines, as a rule, cannot access the content of such a site. According to our latest survey, about 70% of all sites we checked use some form of JavaScript. Of course, most JavaScript is not AJAX, but still, it would be nice if search engines could index AJAX. In this case, the developers could make their sites more interesting and supply them with a large number of “tricks”, and the search engines, in turn, indexed such sites and displayed them in the search results.
')
The goals we want to achieve with this standard are:
• Minimum website changes.
• The user and the search engine receive the same content (no cloaking)
• The search engine sends the user straight to the page with AJAX (not to a static copy)
• Site owners can verify that their AJAX site is presented correctly and that the search spider has access to all content.
As we present the AJAX indexation by search engines:
• We slightly change the structure of the part of the URL for the page with AJAX
If an AJAX page has the same content each time it is requested, then these pages can appear in search results. We suggest adding a symbol instead of a URL, for example, example.com/page?query#state , which will help to highlight and understand the URL of the pages with AYAX: example.com/page?query# [SYMBOL] state. Based on the study of existing URLs we suggest using "!" For this purpose. (exclamation mark). The proposed URL that will be displayed in the search results will look like this: example.com/page?query# ! State.
• Uses a browser that does not accept headers to generate an HTML page from your web server.
A browser that does not receive headers is used to access the AJAX page and generate HTML code based on the final state of the page in the browser. The browser processes only specially designated URLs. By processing pages on the server side in this way, the site owner can manage the generated HTML code and make sure that all the JavaScript is executed correctly. An example of such a browser is HtmlUnit, an open source browser for Java programs without GUI.
• By removing part of the address, we give access to these URLs to search spiders.
Since some fragments of the address will never be transmitted to the server (since a request that adds a fragment to the address will never be executed), it is necessary to slightly modify the URL used to access the page. This modification, in turn, tells the server, which uses a special browser, to generate HTML code, and not to give a page with JavaScript. The remaining URLs - those used by the user - will be processed normally, without using a special browser. We propose not to use part of the information, but add it as a query parameter along with a special character. Using the previous example, the URL would look like this: example.com/page?query& [SYMBOL] = state. After analyzing today's URLs on the Internet, we suggest using "_escaped_fragment_" as a template. The suggested URL will now look like this: example.com/page?query&_escaped_fragment_=state .
• Show the user the real URL in the search results
It makes sense to direct users directly to AJAX pages, and not to static versions. And this can be done by displaying the actual URL in the search results (such as example.com/page?query# ! State from the example above). The search engine can verify that the text given to it is the same or part of the text given to the user.
(Chart: Katharina Probst)
As a result, we started with a URL that has the following structure:
example.com/dictionary.html#AJAX , the page can be accessed by both the search spider and the user in the form:
example.com/diction.html# ! AJAX which will be indexed as
example.com/dictionary.html?_escaped_fragment_=AJAX and which will be shown to the user as
example.com/dictionary.html# ! AJAX
The proposal of the standard was received and is being finalized: Katharina Probst, Bruce Johnson, Arup Mukherjee, Erik van der Poel and Li Xiao, Google
original: googlewebmastercentral.blogspot.com/2009/10/proposal-for-making-ajax-crawlable.html
Source: https://habr.com/ru/post/275845/
All Articles