
Almost in every more or less dynamic project there is a need to execute task queues in the background (sending email, cache updates, search re-indexing, etc.). Job servers (Gearman, etc.) are good, but for most simple tasks they are redundant. The classic implementation of queues in MySQL (using SELECT ... LOCK FOR UPDATE) as the load grows with time begins to lead to problems with locking. Therefore, as is usually the case, I had to write my own “bicycle” to work with background tasks, which would “work exactly” and be extremely simple.
Basis: Cron, PHP 5.3 (mysqli), MySQL> 5.1 - it is easy to stick to almost any hosting.
The operation of obtaining (capturing) a task is atomic (one UPDATE request). No problems with locking and RC.
The possibility of assigning tasks to groups among workers and groups of priorities, transferring an array of data to an executable method (function).
Three modes of processing completed tasks: move the record to a separate table, delete the record, leave a record and mark as successfully processed.
Processing unfinished tasks or tasks processed with an error is on the developer’s conscience.
All about all 400 lines of code (with full PHPDOC).
Restrictions: The current implementation is not suitable for persistent connections, but if someone needs it, it's easy to finish. Even if you want to rewrite to another language :)
The possibility of non-blocking work with the queue is realized through the use of user variables in the UPDATE request with their subsequent selection. To devote a whole article to this technique is silly. Much nicer is the final implementation that can be applied to the business (We are practicing with you, aren't we?). In all other respects, an exclusively classic lineup with groups and priorities.
Example of use (client):
$task_server = \DBTaskServer::create('localhost', 'root', '', 'testDB', 'jobs_queue'); $task_server->addTask('mywork', $data);
mywork is a function that should be available to a worker. An array of
$ data will be passed to it. It is also possible to specify a call to the static methods of the class.
$task_server->addTask('MyWork::doWork', $data);
')
Example worker:
\DBTaskServer::create('localhost', 'root', '', 'testDB', 'jobs_queue')
Run the worker from the console with parameters:
/path/to/script/worker.php [max_tasks_per_lifecycle] [comma_separated_group_ids]
As the name implies, the first option tells how many tasks a worker can perform before completing work (if of course those are available for him), the second option is the group_id values ​​of tasks that this worker must process. If no groups are specified, the worker will process any groups.
For example:
/path/to/script/worker.php 100 3,5,6
Complete 100 tasks from groups 3, 5 and 6.
If no task is found, the worker will immediately complete his work.
Add a worker to the cron:
0-59/5 * * * * /path/to/script/worker.php 5 3 >/dev/null 2>&1
Every 5 minutes, process 5 tasks with group_id = 3.
In the archive, examples of the client, the worker, the server class itself (documented), the sql file with the task table.
Download here (as much as 5kB).Enjoy your code.