Some time ago I read the topic of
Weekday online store: why the product is not available? how hard it is for an online store to keep all goods and prices up to date. Especially if suppliers wanted to put a huge bolt on the relevance of their own price lists. The problem is of course correct, but far from new and well-known.
Personally, I was embarrassed by the lack of any technical data and advice. Everything in the topic came down to the fact that you need to download and update regularly - and then there will be happiness. But this is understandable. We all know what we need. But how to achieve this?
Strictly speaking, this is the topic of this topic -
what tools to achieve maximum automation when working with price lists and online stores . Which in turn will save you a lot of good time for your business.
Let's miss the reasoning about how bad the suppliers are. They are even worse. Relevance is a painful topic, but not ours.
Our task is to make the data get to the site as soon as they are updated at the supplier and with minimal delay. And this is very real. Of course, not instantly, but in minutes 3 everything fits. Although time is very dependent on many factors. But about them later.
')
Another important point: I am writing about fully automated autonomous "systems" that require human intervention only for the initial setup and then control how things are going. These solutions work themselves, download themselves, update themselves - all without the intervention of managers and responsible persons. And you can not worry about the fact that someone will forget something or get sick, or quit. You may not know what is happening there. But it will happen as it was once set up.
So let's get started.
A heart
There is an incredibly convenient task scheduler for Windows -
nnCron . In our case, he will be the heart of everything that governs. For complex tasks, of course, you'll have to learn the language of Fort, in which crown scripts are written. And for simple ones, the minimum programming skills are enough. There are still moments with setting up and running crown tasks on Win 2003/2008 - but these are nuances.
I will not describe all the charms of this planner, but if you still cope without him, then apparently you just do not need it.
For example, if it is known that the supplier updates the price every 3 hours, you can easily create these tasks.
But there are moments that the price is updated unpredictably. For such cases, we use downloading every 5 minutes, after which we check the file for changes, and if they take place, this means that the price is already with new data - therefore, the site must also be updated. By the way, crowns are also involved in this.
To compare files, use the crc32 plugin.It happens that the prices come in the mail.
Usually we do this: when a price arrives by mail, it is automatically saved to the correct folder. Crown sees that the file has been updated, determines if there have been any changes, and if so, updates the site.
Well, of course, it is cron that keeps track of the unarchiving of prices. Why cron? Because sometimes it happens that prices come in the form of price.zip, and inside there is a file (or files) that will each time have a new name that the supplier’s robot generates. And in this case it would be difficult to start processing this price list, since the name will change from download to download. A cron just helps in the manipulation of files and their names.
In general, all this is done by crowns. Want full automation - use
nnCron .
I also tried using
Windows PowerShell . A handy thing and does everything you need, and FORT does not need to learn. But they were not impressed with the ability to configure the very scheduler in Windows, which should start scripts very flexibly. Therefore, we decided to use
nnCron .
Getting prices
You definitely remembered several favorite console utilities for downloading. But our choice is
wget ported to windows. We can say that this is a matter of taste, but in one of the recent tasks, only
wget helped us get to the price list. Although I do not exclude the possibility that the rest with this, too, would be fair.
The download process looked like this: the user needs to get to the login page, enter the login and password, then redirect it to the data verification script and, if everything is correct, then the second redirect goes to the portal index page, where you can download the price list if the person is already authorized. There we click on the link of the form getprice.php (download price list) - again redirect to the download script, the formation of the current price list starts there and after 20 seconds php gives out a ready price list that can be saved.
The first urge was to use the login and password + direct link. It did not work. In general, a lot of time was spent, but only
wget helped, although with its help it was necessary to smash a lot. In short:
logged in logged in, used the login cookie and downloaded everything that is required.Another option is mail
There are usually no problems here. Sometimes it happens that your mail is not local. But, in principle, it is easy to manage by downloading letters via POP3 or by forwarding to a box from which you will already receive letters from a local client. There is nothing complicated. The main thing is that the email client can, when receiving a letter, save its attachment to the folder you need.
So, at the moment we must have a system that at each point in time will provide us with current prices. And you will not need to spend 30 minutes to download and check 10-20 files.
But just get new prices - not cool. But without human intervention to update them - it's cool.Parsing
Parsing is a scary word, but incredibly useful. This is so called an analysis and import of the price list with the definition of what you need and what is not.
Usually, after receiving a new price list, the manager saves it to xml, csv, or just to a text file, and imports data via the interface of his web store. And it becomes joyful. But not all.
There are poor fellows who have 5 thousand lines in the price list, of which 3 thousand are superfluous, but these 3 thousand need to be taken from the second price list, where 10 thousand positions, and everything is scattered there so that the devil will break his leg. And you need to check the prices and correct. You have to sacrifice either time or assortment.
And you say that suppliers are slowly updating their prices. I saw a company where, from the arrival of the price list, it took 5 days before it reached the site. For this is a manual, stupid and ungrateful work, from which we must get rid of.
One of the solutions is suitable for small and small entrepreneurs who simply want to transfer data from the eksel to the site. Even if you need to perform hundreds of operations on the positions in the price list (to discard half, and make a margin of + 50% for the second), this is easily implemented on the VBA in the macro, which will automatically start with your price list and create everything you want there. In the meantime, you go to a business meeting -
nnCron knows best for you what to do and when.
After processing the price list - it will be downloaded to your store. But about the process a little later.
But if you are not a small-small entrepreneur, then you probably have a warehouse. And if there is a warehouse, then it should be 1C 7 or 8. Better 8. No 1c ?? You are probably insane. In the trade - it is irreplaceable. And with automation too.
I will not dwell on versions 1C, but I want to make one clarification - for tasks that will be discussed further, 1c8 is much more convenient. At a minimum, in order to implement most of these solutions, you do not have to make changes to the configuration itself. Automation will work and so. But in 1c7 - you have to rotate the main configuration, which is not always valid.
Basically, the parsing of the price is done using 1C. I will not go into details exactly how, for this is not an article about how you can bend any price list using 1c, but just describe the main features.
nnCron - provides auto start of our treatments. As already mentioned, your entire office at this time can walk on corporate.
Processing determines what the price is, the set of rules for it and starts the import. You can ignore any product, any group, etc. Parallel to put down any markup and update all positions in your database. In addition, if desired, processing can create a new nomenclature, which is not in the database, but has already appeared in the price list. There may be a hundred of these items - this will significantly shorten your time. Processing can import any number of prices in a row and take care of matching articles.
In general, she can do everything that you can, but she never gets sick, and hundreds of times faster.By the way, it is from the complexity of this stage - the parsing - that basically depends on how long the whole procedure of updating the site takes. In particularly complex systems, the process of parsing will take about 10 minutes. Although it would take at least a day for a person to do this.
But, in this article, we are not interested in the automation of importing and parsing the price list, but in getting information to the site. Therefore, about the possibilities and methods - another time. Unless I remind you that for true parsing, 1C programmers should be able to use
regular expressions .
When the goods hit the base 1C, it must be uploaded to the site. Actually, this interested us initially. Although it will be enough for someone to import the prices into 1C itself - this is actually also a very relevant topic for many entrepreneurs.
Uploading
Why are we talking about options for uploading to the site? After all, many developers of Internet solutions themselves care about the tools that allow you to update the product.
It just happens (even very often) that people use different free online stores, which sometimes do not have the capabilities that are required by the owner.
Here we need to make a reservation that there is a conversation about “universal” ways of updating. There are several of them.
The main thing that you need to realize, whatever your site is, you can update the data on it. Sometimes people just think that an Internet shop must necessarily be on Bitrix or on some CMS thread in order to be able to update the product without any problems. No, it is not. It does not matter what your site is - everything can be updated.
First you need to have what to unload. I assume that you have already solved this problem. Often met with the fact that from 1C unload the structure on the site 1 to 1. But sometimes the situation happens, as with the prices - you only need half of the kettles to unload or ignore vacuum cleaners, or shove the goods into the "non-native" category, etc.
As for me, the most prehistoric and stupid way to update is manually. That is, even if you import the file for updating through the admin panel of the site and there it’s just “update”, it is still manually.
Your updates will depend on the health and memory of your manager . And you definitely do not need this.
For automation, there are such options.
Xml, cml, csv and company
This is an option when 1c or Excel creates the file you need, and cron uploads it to the site's ftp. Unloading we usually do
wput for Windows. But this is a matter of taste.
On the hosting works
cron (not the one that we have installed), which every 3 minutes checks for the availability of update files. When they are updated, it launches a script that updates the database and wipes the file. And waiting again.
With such rates in an hour, you will be able to update the price exactly 10 times, no matter how many changes you send, then how to configure
nnCron . Will be relevant? Of course.
And for this you need:
wpu t,
cron on hosting.
Direct connection to the base
This option is highly dependent on the amount of data to update and connection speed. But it has its charms. And also it should be noted that your hoster should allow you to connect directly. This is usually done through
ODBC for MySQL.
If they do not give a direct connection, then probably
ssh connections will still be allowed. In our case, we work this way. For
ssh connections,
putty is used, then
ODBC to configure the connection to the base.
In this way you can connect to the base of the site from Excel and update the price list. And you can directly from 1C.We use this method so that the manager directly from the MySQL database can see what kind of new orders there are and import them directly into 1C, creating order documents, consumables, etc. Conversely, by creating a document in 1C, you can immediately transfer it to the site database directly.
Sometimes this way you can implement a
“robot” . That is, a visitor on the site performs certain actions, for example, sending an invoice. But the account should be sent exactly from 1C, because you need to "stake out" the number and write out on behalf of the specific manager at that time. It will look like this:
1. In mysql an action record is created.
2. 1C automatically using nnCron checks a direct connection to the base of the site for "actions"
3. Seeing the action, the 1C processing starts, which automatically generates an account for the necessary person and with the necessary parameters, and then sends it.
In addition, in this way, you can link your online customers with real ones in the 1C database by specifying the matching codes in order to reduce the amount of manual work when transferring orders.
That is the “plus” of the direct connection method: you can read data directly from your site, as well as bring it back.Just ssh
If you do not want direct connections, you can use the
putty utility to transfer files via
ssh connections:
pscp . In addition, this utility allows you to get files from your server.
This is another version of
ftp connection, which will fit even if there is no connection to the base for any reason.
Then, using the
plink utility,
you can run a remote script on the site, which will do what is necessary.
But this is already a “perverted” way for special cases when nothing else helps.
Update the price lists
Well, once we have updated the base, then the prices need to be updated too. There are problems - just spit. VBA of Excel forms a new price list for you, if you are not a shark business. And if the shark, then your 1C should give you a price of any complexity of your choice, image and likeness.
After that, everything is archived and thrown using
wput / pscp to the site. And as it is already known,
nnCron manages everything: archiving, checking, uploading.
Online store and relevance
Such developments will make it possible to keep the base of the site always up to date. In addition, if you use 1C, it is possible to update not only the availability in the price list, but also the physical availability of at least 10 warehouses.
Thus, on each product you can make the status
“available for order” if it is just in the price list.
"In stock" - if the goods are physically in your warehouse; You can also specify the quantity and which warehouse it is. In general, everything that comes up is possible. The main thing is to organize everything.
Process
Approximately the scheme will look like this:
1. nnCron downloads the price, for example, every 10 minutes.
2. There is a check whether it is a new price or is still an old version.
3. If new - nnCron starts processing in 1C for importing the price list into the database.
4. 1C parsit and imports the price in its base.
5. After that, processing starts to create an insert file for the MySQL online store: availability, warehouse, new products, prices - in short, everything you need.
6. After creating the file, it will automatically upload to your server.
7. When updating a file on the server, your hosting script with cron updates the database with new data.
Alternatively, instead of 6 and 7, you can simply connect to the MySQL database directly and update directly from 1C8. 1C creates the .xls file with the necessary parameters.
9. nnCron seeing the signal that the price has been updated, uploads it to the server.
What do you need for happiness?
For this process, you will need the following tools:
1.
nnCron is a powerful scheduler. + preferably knowledge of the Fort language. It is possible and Windows Power Shell, but this is in case there are no “tricky” requirements for schedules.
2.
wget - download files
3.
wput - uploadim files
4.
putty -
ssh client. + its related utilities
pscp, plink .
5.
ODBC Connector for MySQL - to configure a connection to
MySQL6.
1C8 / 17 - for happiness in trade and warehouse accounting.
7.
Blat - console mailer. About him the speech did not go, but it can be useful.
8.
AutoIt - allow automate complex tasks, and then compile it into exe. Especially useful if you use automation on programs that do not provide for such a possibility.
All of this set ensures that all your automation will work flawlessly. Your prices will always be updated on time, and availability will always be relevant. And from updating the price to updating the database will take not an hour or two, but a few minutes - no matter how many prices you have.
And the entire workflow will not rely on people who are notoriously unreliable. No need to worry that someone forgot, did not have time, fell ill, shook himself - everything will work. All these scripts are designed to remove the weak link - the person.
At one time I wondered about automation, because the human factor began to annoy me. He - the most important obstacle in the relevance.
I hope that someone will push all this information to the right decisions and ways. These are just tools. Most of the work is still yours.
Successes in business!