We write our handler task on .NET
Hello.
In your company do not like open-source?
Do you like bicycles?
It was always interesting how task planners work?
Under the cut, the story of how I had to make my analogue of the well-known open source scheduler quartz.net .
It all started about a year and a half ago, when I moved to a new job. The project was interesting, it is now launched and even keeps a good load, in the web part there are several hundred hits per second, but this, I believe, will be the topic of a separate article. At a certain stage of development, there appeared a requirement to perform a “couple” of tasks asynchronously. Naturally, it immediately became clear that a couple of tasks would not be enough and a bunch of new requirements would come out. And so it happened, now the number of parallel tasks exceeded 20 and the component was deployed on several servers. In this article I will talk about the very first implementation, quick and dirty solution, which allowed a quick start.
I already had experience using quartz.net - this is the port of the famous java quartz component on the .net platform and, frankly, the only less serious implementation of the scheduler on .net that I know. I also heard about Castle.Scheduling, but my hands did not get around to tinker. I will be glad to hear about other solutions in the comments. I, as if nothing had happened, came to my team-leader with a proposal to use this solution, and here I was in for an unpleasant surprise.
It turned out that we cannot take and use some third-party component like this. After all, he did not pass the approval of the board of architects! You may have heard about these people spending half a day at meetings, and at other times drawing futuristic pictures that no one except them looks into. It also requires the approval of the security department, which must check if there are any bookmarks in the code. And here is another good lawyers to see the license agreement, you never know what is there. I felt like Don Quixote, who goes into battle with windmills and decided to go his own way.
')
Of course, I understood that the dates were not rubber, so I climbed into the corporate FishEye and quickly realized that I was not alone. I found two implementations of this functionality and even wanted to use one of them. After talking with the author, I realized that he did not mind, but I would have to sharpen several libraries created by their team and independently deal with all the bugs, because I don’t have time (read desire) to support me.
As any self-respecting programmer, I do not like poking around in other people's bugs, I love to create my own. So I rolled up my sleeves and got down to business.
Time was running out, it was necessary to release a prototype. I’ll say right away that I really didn’t want to get involved with the Task Scheduler built into Windows, although they also considered this option. It was possible to make an image similar to the GAE and use the intervals to invoke the web service methods. It was possible to pick open quartz, but this is not a noble cause.
I wanted to do it in a simple way so that everything would work during the working day.
But as? Well, let's set our task in the simplest way.
Now we define the simplest trigger - a class that will store information about the execution interval of the task.
After that, you need to set in the config something in the spirit.
And let it run every 10 seconds. Let's see how we achieve this.
To make our tasks run in parallel, we need a custom thread pool.
Immediately make a reservation that .net 4.0 chips were not used, because The code was written before the release of this in the broad masses.
Writing your own pool is not an easy task, it will take a lot of time and most likely will lead to failure, so I took it ready from a recognized guru in these matters - Jon Skeet (there are many other utilities here)
By itself, the handler will wrap the windows service, this is trivial, so we will not dwell on it. Look at the picture, which will explain the work of the planner.
Let's understand what's going on here.
First, we initialize our pool with any number of threads, which directly depends on the number of processor cores on our server. Do not forget that, in addition to the scheduler, CPU time and consumes the OS, and other services, so it is better to restrict a reasonable amount.
Tasks we put in the dictionary, so that if necessary it is easy to track them by identifier. In the future, this may be useful to us for persistence. The general execution scheme looks like this:
To stop the execution, we use Monitor and some boolean variable aka Halted, which will signal the main loop to exit.
An experienced eye probably already peered into the code using spring.net. Indeed, tasks are defined as poco, managed by spring. This will simplify our dynamic loading and will give a number of advantages. For example, through interceptors we will be able to log time to perform tasks and use other AOP buns.
Let's talk a little about implementation flaws.
First, we do not limit the execution time of each task, which can lead to hangs, timeouts must always be set.
Secondly, our code is very straightforward as regards the definition of triggers. He will not allow us to perform the task at a specified time, for example, every first day of the month or every Sunday.
I can not help but notice that in .net 4.0 a lot of tasty things appeared for multithreaded work, which will also help to improve and simplify our code.
In conclusion, a few words about concurrency. Practice shows that it is necessary to design tasks taking into account the deployment on multiple servers. For this task should not have a state and the elements of your workflow should also be as independent as possible. Also try to avoid prolonged tasks, it is better to make a lot of nimble.
If readers are interested, I plan to write about the implementation of features that I find interesting. Among them, the GUI, which will allow to steer the farm from such planners, to see which tasks are performed, the average time of task execution and the problems that arise. And also to stop / resume execution of tasks on any scheduler in the farm, again through the GUI.
The source code for the article can be taken here .
I sincerely recommend using quartz.net and functionality from .net 4.0 instead of samopisny solutions.
In your company do not like open-source?
Do you like bicycles?
It was always interesting how task planners work?
Under the cut, the story of how I had to make my analogue of the well-known open source scheduler quartz.net .
Prehistory
It all started about a year and a half ago, when I moved to a new job. The project was interesting, it is now launched and even keeps a good load, in the web part there are several hundred hits per second, but this, I believe, will be the topic of a separate article. At a certain stage of development, there appeared a requirement to perform a “couple” of tasks asynchronously. Naturally, it immediately became clear that a couple of tasks would not be enough and a bunch of new requirements would come out. And so it happened, now the number of parallel tasks exceeded 20 and the component was deployed on several servers. In this article I will talk about the very first implementation, quick and dirty solution, which allowed a quick start.
I already had experience using quartz.net - this is the port of the famous java quartz component on the .net platform and, frankly, the only less serious implementation of the scheduler on .net that I know. I also heard about Castle.Scheduling, but my hands did not get around to tinker. I will be glad to hear about other solutions in the comments. I, as if nothing had happened, came to my team-leader with a proposal to use this solution, and here I was in for an unpleasant surprise.
It turned out that we cannot take and use some third-party component like this. After all, he did not pass the approval of the board of architects! You may have heard about these people spending half a day at meetings, and at other times drawing futuristic pictures that no one except them looks into. It also requires the approval of the security department, which must check if there are any bookmarks in the code. And here is another good lawyers to see the license agreement, you never know what is there. I felt like Don Quixote, who goes into battle with windmills and decided to go his own way.
')
Of course, I understood that the dates were not rubber, so I climbed into the corporate FishEye and quickly realized that I was not alone. I found two implementations of this functionality and even wanted to use one of them. After talking with the author, I realized that he did not mind, but I would have to sharpen several libraries created by their team and independently deal with all the bugs, because I don’t have time (read desire) to support me.
As any self-respecting programmer, I do not like poking around in other people's bugs, I love to create my own. So I rolled up my sleeves and got down to business.
Story
Time was running out, it was necessary to release a prototype. I’ll say right away that I really didn’t want to get involved with the Task Scheduler built into Windows, although they also considered this option. It was possible to make an image similar to the GAE and use the intervals to invoke the web service methods. It was possible to pick open quartz, but this is not a noble cause.
I wanted to do it in a simple way so that everything would work during the working day.
But as? Well, let's set our task in the simplest way.
public interface ITask { void Execute(); } * This source code was highlighted with Source Code Highlighter .
public interface ITask { void Execute(); } * This source code was highlighted with Source Code Highlighter .public interface ITask { void Execute(); } * This source code was highlighted with Source Code Highlighter .public interface ITask { void Execute(); } * This source code was highlighted with Source Code Highlighter .public interface ITask { void Execute(); } * This source code was highlighted with Source Code Highlighter .public interface ITask { void Execute(); } * This source code was highlighted with Source Code Highlighter .
Now we define the simplest trigger - a class that will store information about the execution interval of the task.
public interface ITrigger
{
Guid Id { get ; set ; }
DateTime ? NextProcessTime { get ; set ; }
TimeSpan Interval { get ; set ;}
ITask Task { get ; set ;}
}
* This source code was highlighted with Source Code Highlighter .After that, you need to set in the config something in the spirit.
* This source code was highlighted with Source Code Highlighter .
- < objects xmlns = "http://www.springframework.net" xmlns: aop = "http://www.springframework.net/aop" >
- < object id = "SampleTrigger1" type = "TaskHandler.BusinessLogic.Impl.GenericTrigger, TaskHandler.BusinessLogic" >
- < property name = "Interval" value = "10s" > </ property >
- < property name = "Task" ref = "SampleTask1" > </ property >
- </ object >
- < object id = "SampleTask1" type = "TaskHandler.BusinessLogic.Impl.SampleTask1, TaskHandler.BusinessLogic" > </ object >
- < object id = "ITriggerLoader" type = "TaskHandler.BusinessLogic.Impl.TriggerLoaderImpl, TaskHandler.BusinessLogic" > </ object >
- </ objects >
And let it run every 10 seconds. Let's see how we achieve this.
Parse the code
To make our tasks run in parallel, we need a custom thread pool.
Immediately make a reservation that .net 4.0 chips were not used, because The code was written before the release of this in the broad masses.
Writing your own pool is not an easy task, it will take a lot of time and most likely will lead to failure, so I took it ready from a recognized guru in these matters - Jon Skeet (there are many other utilities here)
By itself, the handler will wrap the windows service, this is trivial, so we will not dwell on it. Look at the picture, which will explain the work of the planner.
Let's understand what's going on here.
First, we initialize our pool with any number of threads, which directly depends on the number of processor cores on our server. Do not forget that, in addition to the scheduler, CPU time and consumes the OS, and other services, so it is better to restrict a reasonable amount.
Tasks we put in the dictionary, so that if necessary it is easy to track them by identifier. In the future, this may be useful to us for persistence. The general execution scheme looks like this:
- Download all our Tasks
- Run main loop
- In the loop, we run through all the tasks and put them in the queue of our pool.
- When a task appears in the queue, the pool immediately sends it to the free stream for execution.
- The thread completes the task and returns to the “ready” state.
To stop the execution, we use Monitor and some boolean variable aka Halted, which will signal the main loop to exit.
An experienced eye probably already peered into the code using spring.net. Indeed, tasks are defined as poco, managed by spring. This will simplify our dynamic loading and will give a number of advantages. For example, through interceptors we will be able to log time to perform tasks and use other AOP buns.
Let's talk a little about implementation flaws.
First, we do not limit the execution time of each task, which can lead to hangs, timeouts must always be set.
Secondly, our code is very straightforward as regards the definition of triggers. He will not allow us to perform the task at a specified time, for example, every first day of the month or every Sunday.
I can not help but notice that in .net 4.0 a lot of tasty things appeared for multithreaded work, which will also help to improve and simplify our code.
In conclusion, a few words about concurrency. Practice shows that it is necessary to design tasks taking into account the deployment on multiple servers. For this task should not have a state and the elements of your workflow should also be as independent as possible. Also try to avoid prolonged tasks, it is better to make a lot of nimble.
If readers are interested, I plan to write about the implementation of features that I find interesting. Among them, the GUI, which will allow to steer the farm from such planners, to see which tasks are performed, the average time of task execution and the problems that arise. And also to stop / resume execution of tasks on any scheduler in the farm, again through the GUI.
The source code for the article can be taken here .
I sincerely recommend using quartz.net and functionality from .net 4.0 instead of samopisny solutions.
Source: https://habr.com/ru/post/111415/
All Articles