An example of using the Product API from Fetchee for parsing products online store

In this manual, we will describe how to get product data by URL using the Fetchee Product API using the example of the lamoda online store.



For those who have not read our last note - the Product API will be useful for developers who need to get product data from any store, but who do not want to waste time creating their own parsing system or have already realized that open-source libraries have significant limitations and require a lot of time to support. Our automatic and non-configurable API for parsing eCommerce data allows you to focus on developing the core functions of your application. In addition, it is very easy to try. Details under the cut.



First you need an API access key. Leave a request for https://fetch.ee/ru/developers/ . We give access to everyone and usually respond to applications within 24 hours.

')

1. Sending a request to parse the goods by URL

Please note that the API accepts only POST requests in the application / json format at the URL https://fetch.ee/api/v1/product :

{ "url":"<URL >", "api_key":"<API >", "callback_url":"<URL    >" } 

An example of a request for a product at http://www.lamoda.ru/p/ug174awohj47/shoes-uggaustralia-uggi/ will look like this (we added a demo API key that will live for a couple of days):

 curl -X POST -H "Content-type: application/json" -d "{\"url\": \"http://www.lamoda.ru/p/ug174awohj47/shoes-uggaustralia-uggi/\", \"api_key\": \"0e2bc30838d8bbbbbec787667c4ff34b\", \"callback_url\": \"http://requestb.in/onpdf2on\"}" https://fetch.ee/api/v1/product 

All parameters are required, if one of them does not exist, the server will return a 403 error, as in the case of an incorrect API key.



IMPORTANT: In the example, a temporary callback_url is specified. It will cease to operate in a couple of hours, replace it with your URL, which can be obtained at http://requestb.in .



The Product API processes requests asynchronously; the response time can vary from a couple of seconds to several tens of minutes. This is due to the load on the system in a specific geographic region. The API will send the data to the URL you specify upon completion of the parsing (callback_url).



The maximum waiting time is 30 minutes, after which, in any case, a response will be sent with the result or a message about the inability to complete the parsing of the goods at the specified URL.



For testing the API, it is convenient to use the http://requestb.in service. Get a new container on it (Create a RequestBin) and specify the resulting URL as callback_url. All responses from the API will come to him, just refresh the browser page.

2. Fields and API response options

The response uses the following fields:

• success - the result of sending the URL for parsing;
• id - unique ID of the request;
• already_processed - the request has recently been processed;
• not_shop - the URL does not belong to the store;
• not_product - at this URL there is no data about the product.

Any field other than success may be absent.



Immediately after the request, you will receive a response in JSON format. The ID is needed to match the requests you send and the responses received on callback_url. In this case, everything went fine and the product was accepted for parsing.

 { "success": true, "id": "583ea83a09e9497a0eb1b82a" } 

Such a response will follow if you try to send a parsing URL that has already been processed less than 6 hours ago. In this case, the call to callback_url will not follow.

 { "success":true, "id":"583ea83a09e9497a0eb1b82a", "already_processed":true } 

You will receive this answer if the API already knows that the specified URL does not belong to the website of the online store. This product is not added to parsing and there will be no callback_url. We keep a black list of sites, including dubious and partner stores (acting as showcases with goods from other stores).

 { "success":false, "not_shop":true } 

When the system knows that there is no product at this URL (category page, main store page, etc.). This product is not added to parsing and there will be no callback_url.

 { "success":false, "not_product":true } 

3. Getting the result of parsing

The API sends a POST request with a response in JSON to the address specified in callback_url. After 20 seconds we got the result of parsing:

 { "id":"583ea83a09e9497a0eb1b82a", "url":"http://www.lamoda.ru/p/ug174awohj47/shoes-uggaustralia-uggi/", "created_at":"2016-11-30T10:21:46.865Z", "title":"  UGG Australia", "price":22800, "currency":"RUB", "img_url":"https://fetch.ee/assets/item-images/583e/a843a9436d700e54ef37.jpg", "brand":"UGG Australia" } 

Consider the resulting JSON in more detail. There are always 3 required parameters in the response:

• id - unique ID of the request;
• url - final URL of the product (if there were redirects, or we cut out various unnecessary query_params from the URL (UTM tags, for example);
• created_at - time to create a request for parsing goods;

Any of the following parameters may be missing:

• title - the name of the product;
• price - price;
• currency - currency code (for example, RUB, USD, EUR);
• img_url - link to the product image;
• brand - the manufacturer of the goods;
• out_of_stock: true - when the product is not on sale;
• removed: true - when the product disappeared from the store altogether and is redirected to the category page, the store's main page, or we get a 404 response from the server;
• not_shop: true - when during parsing the API determined that the URL does not belong to the store, although it had previously accepted the request for processing;
• not_product: true - the goat during the parsing API determined that the URL does not contain information about the product, although the processing request was accepted;
• unprocessed: true - when the Product API failed to get the key parameters of the product title or price.

If the Product API can be useful to you, leave a request for fetch.ee/en/developers ! Write in the comments what other parameters of the goods you would like to receive in addition to those already available?