Slow and / or resource-intensive tasks in the code: deferred tasks, queues, tasks with manual processing
Publish at the request of eugenioz .
When working with large Web projects, there are tasks that are not necessary to be done right now, but the actions are critical and it is necessary to perform them. Such actions can be carried out directly in the code and in case of an unsuccessful attempt to repeat. But, in my opinion, it is better to do this immediately outside the main task code: this unloads the code and achieves the monotony of execution.
A common approach here is to create execution queues and pending tasks. Key examples: forwarding / verifying data from third-party sites; transfer of data from the reseller panel to the main site; the transfer of tasks that can not be performed automatically for manual processing.
')
As a means of distributing tasks, I bring to your attention the PHP class Tasks.
Class code
And now the explanation.
Tasks are stored in a temporary directory as files. The script, launched by crown, gets a list of tasks and executes the execute method for each task. After successful execution, the task file is deleted. The class can be reworked to work with MySQL database. This makes sense, because you can do DELAYED INSERT, which does not block execution, but now the file system is used because of its greater reliability.
The task is created as a normal element of the algorithm, without changing the logic.
For example, it was:
has become
Tasks have a priority level, so that you can perform tasks in order of “importance” each time the cron-script is run. The larger the priority parameter, the further the task will be postponed.
Variant of cron-script: code .
Next to the Tasks class in the code there is a task_handlers class - in task_handlers you can create a handler.
The name of the function of this class is the name of the processor that can be specified at the time of creating the task. As an example, the handlers 'mysql' and 'mail' are given - their code is small, therefore, it is convenient for illustration.
The author of the code, eugenioz , permits the use of a code under the MIT license.
Despite the simplicity and lightness of this class, it allows you to very effectively distribute the load without changing the logic of the algorithms.
Once again, examples of application:
eugenioz : I will carefully listen to constructive criticism, questions and suggestions for improvement.
When working with large Web projects, there are tasks that are not necessary to be done right now, but the actions are critical and it is necessary to perform them. Such actions can be carried out directly in the code and in case of an unsuccessful attempt to repeat. But, in my opinion, it is better to do this immediately outside the main task code: this unloads the code and achieves the monotony of execution.
A common approach here is to create execution queues and pending tasks. Key examples: forwarding / verifying data from third-party sites; transfer of data from the reseller panel to the main site; the transfer of tasks that can not be performed automatically for manual processing.
')
As a means of distributing tasks, I bring to your attention the PHP class Tasks.
Class code
And now the explanation.
Tasks are stored in a temporary directory as files. The script, launched by crown, gets a list of tasks and executes the execute method for each task. After successful execution, the task file is deleted. The class can be reworked to work with MySQL database. This makes sense, because you can do DELAYED INSERT, which does not block execution, but now the file system is used because of its greater reliability.
The task is created as a normal element of the algorithm, without changing the logic.
For example, it was:
$q_upd=update_order_data($params);has become
$q_upd=tasks::create("update_order_data", 'api', 0, $params);Tasks have a priority level, so that you can perform tasks in order of “importance” each time the cron-script is run. The larger the priority parameter, the further the task will be postponed.
Variant of cron-script: code .
Next to the Tasks class in the code there is a task_handlers class - in task_handlers you can create a handler.
The name of the function of this class is the name of the processor that can be specified at the time of creating the task. As an example, the handlers 'mysql' and 'mail' are given - their code is small, therefore, it is convenient for illustration.
The author of the code, eugenioz , permits the use of a code under the MIT license.
Despite the simplicity and lightness of this class, it allows you to very effectively distribute the load without changing the logic of the algorithms.
Once again, examples of application:
- On the profile settings page of one of the services, you need to interact with another site, confirming its data. This site could also answer "not immediately." Conclusion concise "checked" in one of the blocks - much better than the generation of the entire page in a few seconds due to a delay in one line;
- At the moment, for all visitors who have entered the wap site from a mobile phone, and whose IP is not on the list of IP gateway operators, an IP verification task is being created. In his spare time, whois IP information is taken and checked for the presence of words that may indicate that they belong to a cellular operator. If there are such words, IP is added to the “for review” list and, after “manual” verification, this IP can be added to the list of IP-gates of operators;
- working with a remote API that may not be available, but the action may be delayed and must be executed. For example, often reseller panels do not repeat orders, which leads to a large number of complaints and lost profits.
eugenioz : I will carefully listen to constructive criticism, questions and suggestions for improvement.
Source: https://habr.com/ru/post/95984/
All Articles