Product API from Fetchee: automatic parsing of goods online stores
UPD: Today we sent API access keys to everyone who submitted a request for https://fetch.ee/ru/developers/ .
UPD2: Placed a tutorial on using the Product API in a new note on Habré.
From the beginning of work on Fetchee, we had a plan to offer a solution for parsing data about goods from online stores of corporate clients and start-up community. Testimonials that we collected during our work spoke about its potential relevance and led to the launch of a new product that will help developers quickly create their own applications based on one of the main functions of Fetchee - collecting eCommerce data and tracking prices in any online stores in the world .
At the start of the project, we wanted to show the finished product to users as quickly as possible, so we tried to simplify the application by all means. One of the options was to refuse to develop your own parser of online stores and introduce a ready-made solution.
In order to reduce the amount of narration, I will miss the comparison of popular applications for extracting data from sites (if you want to see this comparison, leave your vote in the poll at the end of the article). Suffice it to say that diffbot turned out to be the most suitable for our task - an extremely expensive service, popular among large companies and startups with decent funding. It is used by many applications for creating bookmarks, because high-quality parsing is difficult.
')
First, the solution was beyond our means, and second, diffbot did not work well with Russian online stores. I had to invent my own parsing technology, which should be able to:
We managed to implement all the principles described above. The system is learning from new data and getting smarter before our eyes. The more queries - the more accurate it becomes. Our desire to make the sample more diverse and has become one of the reasons for providing access to the Product API to third-party developers, because the rate of new data from users has slowed down - 90% buy in the same online stores.
So, meet the Product API from Fetchee . With it, you can, specifying any URL of the product in the online store, to obtain basic information about things, including name, image, price and currency. In the case of our API, you do not have to configure the parser for each site, the system independently analyzes the page and determines the necessary data.
For example, here are the results of parsing a bike with Avito .
Fashionable bag with wildberries .
Or an SUV from a car site .
On any page where there is a price, the image and the name of the product - the Product API will find this data and return it to you in JSON for further processing.
You can try the API directly on the Beta-program page . If you like the result of the test drive, register as a developer - the more applications there are, the sooner we will provide full access to the API.
We are very interested to see how your ideas come to life with the help of the Product API.
UPD2: Placed a tutorial on using the Product API in a new note on Habré.
From the beginning of work on Fetchee, we had a plan to offer a solution for parsing data about goods from online stores of corporate clients and start-up community. Testimonials that we collected during our work spoke about its potential relevance and led to the launch of a new product that will help developers quickly create their own applications based on one of the main functions of Fetchee - collecting eCommerce data and tracking prices in any online stores in the world .
At the start of the project, we wanted to show the finished product to users as quickly as possible, so we tried to simplify the application by all means. One of the options was to refuse to develop your own parser of online stores and introduce a ready-made solution.
In order to reduce the amount of narration, I will miss the comparison of popular applications for extracting data from sites (if you want to see this comparison, leave your vote in the poll at the end of the article). Suffice it to say that diffbot turned out to be the most suitable for our task - an extremely expensive service, popular among large companies and startups with decent funding. It is used by many applications for creating bookmarks, because high-quality parsing is difficult.
')
First, the solution was beyond our means, and second, diffbot did not work well with Russian online stores. I had to invent my own parsing technology, which should be able to:
- Maximum automatically determine the basic parameters of the product;
- Work with any currency and language;
- Take into account the geographic location of users and the peculiarities of stores with this region (language, currency and country);
- Identify discriminatory pricing stores (operating system version and city within country).
We managed to implement all the principles described above. The system is learning from new data and getting smarter before our eyes. The more queries - the more accurate it becomes. Our desire to make the sample more diverse and has become one of the reasons for providing access to the Product API to third-party developers, because the rate of new data from users has slowed down - 90% buy in the same online stores.
So, meet the Product API from Fetchee . With it, you can, specifying any URL of the product in the online store, to obtain basic information about things, including name, image, price and currency. In the case of our API, you do not have to configure the parser for each site, the system independently analyzes the page and determines the necessary data.
For example, here are the results of parsing a bike with Avito .
{ "id":"58234b6cfd920b507bfd6b1f", "url":"https://www.avito.ru/sankt-peterburg/velosipedy/velosiped_author_basic_rama_19_i_21_835103333", "title":" Author Basic, 19\" 21\"", "price":14900, "currency":"RUB", "img_url":"https://fetch.ee/assets/item-images/5823/4b78475d39467b4b25eb.jpg", "created_at":"2016-11-09T16:14:36.542Z", "last_track_at":"2016-11-09T16:14:48.061Z" } Fashionable bag with wildberries .
{ "id":"5824212c65ef60477b38b890", "url":"https://www.wildberries.ru/catalog/3095060/detail.aspx?targetUrl=GP", "title":", GUESS", "price":13560, "currency":"RUB", "img_url":"https://fetch.ee/assets/item-images/5824/2144475d39467b4b26de.jpg", "created_at":"2016-11-10T07:26:36.368Z", "last_track_at":"2016-11-10T07:27:00.544Z" } Or an SUV from a car site .
{ "id":"5824219b65ef60477b38b8be", "url":"https://auto.ru/cars/new/sale/bmw/x6_m/1044423007-94d1a/", "title":" BMW X6 M II (F86) ", "price":8099000, "currency":"RUB", "img_url":"https://fetch.ee/assets/item-images/5824/21ad475d39467b4b26df.jpg", "created_at":"2016-11-10T07:28:27.038Z", "last_track_at":"2016-11-10T07:28:45.516Z" } On any page where there is a price, the image and the name of the product - the Product API will find this data and return it to you in JSON for further processing.
You can try the API directly on the Beta-program page . If you like the result of the test drive, register as a developer - the more applications there are, the sooner we will provide full access to the API.
We are very interested to see how your ideas come to life with the help of the Product API.
Source: https://habr.com/ru/post/314960/
All Articles