Implementing a pool of objects in C #

Good day!
In this article, I will discuss how to improve the performance of a multi-threaded (and not only) C # application, in which objects are often created for "one-off" work.
A little about multithreading, non-blocking synchronization, use of the profiler built into VS2012 and a small benchmark.

Introduction

The object pool is a parent design pattern, a set of objects initialized and ready to use.
Why is it needed? In short - to improve performance when initializing a new object leads to high costs. But here it is important to understand that the garbage collector built into .NET does an excellent job with the destruction of light short-lived objects, therefore the applicability of the pool is limited by the following criteria:

• expensive to create and / or destroy objects (examples: sockets, streams, unmanaged resources);
• clearing objects for reuse is cheaper than creating a new one (or nothing);
• very large objects.

I will explain the last point a little. If your object occupies 85,000 bytes or more in memory, it falls into a large object heap in the second generation of garbage collection, which automatically makes it a “long-lived” object. Add to this the fragmentation (this heap is not compressed) and we get the potential problem of lack of memory with constant allocation / destruction.
The idea of ​​the pool is to organize the reuse of "expensive" objects using the following scenario:

var obj = pool.Take(); //    .        obj.DoSomething(); pool.Release(obj); //  ("")   ,      

Problems of this approach:

• after performing work with an object, it may be necessary to reset it to its initial state so that previous use does not affect subsequent ones in any way;
• the pool must provide thread safety, because it is used, as a rule, in multi-threaded systems;
• the pool must handle the situation when there are no objects available for issuing in it.

Given these problems, requirements for a new class were drawn up:

1. Type safety pool at compile time.
2. Work pool with any classes, including third-party.
3. Simple use in code.
4. Auto-selection of new objects in case of shortage, their user initialization.
5. Limit the total number of selected objects.
6. Auto-clean object when it is returned to the pool.
7. Thread safety (preferably with minimal synchronization costs).
8. Support for multiple instances of the pool (this implies at least the simplest control that objects return to their own pools).

Problem solving use

In some implementations, to support a pool of an object, an object must implement an IPoolable interface or similar, but my task was to ensure that the pool works with any classes, even if they are closed for inheritance. For this was created a generic shell PoolSlot, which inside contains the object itself and a link to the pool. The pool itself is an abstract generic class for storing these slots, with two unimplemented methods for creating a new object and clearing the old one.
')

 public abstract class Pool<T> { public PoolSlot<T> TakeSlot() {...} //  "  " public void Release(PoolSlot<T> slot) {...} //  "  " /* ... */ //   : protected abstract T ObjectConstructor(); //   ,    protected virtual void CleanUp(T item) {} //   ,   } 

Using the example of the class SocketAsyncEventArgs

Use in code:

 var pool = new SocketClientPool(1024, 5, 10); //   ,  /* ...-  ... */ var slot = pool.TakeSlot(); //     var args = slot.Object; //    -  pool.Release(slot); //     

Or even like this:

 using(var slot = pool.TakeSlot()) //  PoolSlot  IDisposable { var args = slot.Object; } 

Those who are familiar with the asynchronous .NET model and / or asynchronous methods of the same Socket class know that using such an implementation is difficult, because the Socket.XxxAsync methods take SocketAsyncEventArgs as input, and not some PoolSlot <SocketAsyncEventArgs> . It doesn't matter to call a method, but where do you get the slot in the end handler?
One option is to save the slot in the SocketAsyncEventArgs.UserToken property when creating an object; for this, there is a method in the pool for overriding HoldSlotInObject.

Of course, not every object provides the user with such a property. And if your class is still not closed from inheritance, then a special IPoolSlotHolder interface with a single property for storing the slot is offered. And if I know that my object is guaranteed to contain a slot, then it would be logical to add TakeObject / Release methods that return / accept objects themselves (and get their slot inside), which was done in the descendant of the pool.

Next, I suggest to get acquainted with the development of the internal "kitchen".

Storage

To store objects "in the pool" is a collection of ConcurrentStack. The possible use of multiple pool instances required keeping track of which of the objects was created by this particular pool.
This was how the “registry” was entered based on the ConcurrentDictionary, which contains the ID of the slots ever created by the pool and the accessibility flag of the object (true - “in the pool”, false - “not in the pool”).
This made it possible to immediately kill 2 birds with one stone: to prevent the erroneous multiple return of the same object (after all, the stack does not ensure the uniqueness of the objects stored in it) and to prevent the return of objects created in another pool. This approach was a temporary solution, and then I got rid of it.

Multithreading

The classic implementation of the pool involves using a semaphore (in .NET, this is Semaphore and SemaphoreSlim) for tracking the number of objects, or other synchronization primitives in conjunction with a counter, but ConcurrentStack, like ConcurrentDictionary, is thread-safe collections, so the input-output of objects is no longer required . I will note only that the call to the ConcurrentStack.Count property causes a complete enumeration of all the elements, which takes considerable time, so it was decided to add my own element counter. As a result, two “atomic” pool operations were obtained - Push and TryPop, on the basis of which all the others were built.

In addition to the I / O of existing objects, it is necessary to synchronize the allocation of new ones to the specified upper limit.
Here we use a semaphore initialized by the maximum number of elements in the pool (upper limit) and subtract one each time when creating a new object, but the problem is that when it reaches zero, it will simply block the stream. A call to the SemaphoreSlim.Wait (0) method, which, given the current value of the “0” semaphore, returns false almost without delay, could be a way out of this situation, but it was decided to write a lightweight analog of this functionality. This is how the LockFreeSemaphore class appeared, which returns false when zero is reached without delay. For internal synchronization, it uses Interlocked.CompareExchange fast CAS operations.

Thus, the pool operation “take an object” works according to the following algorithm:

1. We are trying to take an object from the repository, if it is not there - point 2.
2. We are trying to create a new object if the semaphore is zero (the upper limit is reached) - point 3.
3. The worst scenario - we expect the object to return to the bitter end.

First results, optimization and refactoring

Do you need a pool of objects? Depends on the situation. Here are the results of a small test using the “typical server object”, SocketAsyncEventArgs with a buffer of 1024 bytes (time in seconds, pooling is on):

New object requests	One thread, no pool	One thread, with a pool	25 tasks * without pool	25 tasks *, with a pool
1,000	0.002	0.003	0.027	0.009
10,000	0.010	0.001	0.272	0.039
25,000	0.030	0.003	0.609	0.189
50,000	0.048	0.006	1.285	0.287
1,000,000	0.959	0.125	27.965	8.345

* task - the System.Threading.Tasks.Task class from the TPL library, starting with .NET 4.0
The results of the passage of the VS2012 profiler in a multi-threaded test with a pool:


As you can see, everything depends on the ConcurrentStack.TryPop method, which (we will assume) has nowhere to accelerate. In second place is an appeal to the “registry”, which takes about 14% each in both operations.
In principle, the support of the second collection inside the pool seemed to me like a crutch, so the sign “in the pool / not in the pool” was transferred to the slot itself, and the registry was safely removed. Test results after refactoring (increase, as expected, 30-40%):

New object requests	25 tasks with a pool
25,000	0.098
1,000,000	5.751

I think this can be stopped.

Conclusion

In brief, let me remind you how the tasks were solved:

1. Type safety at compile time - using generic classes.
2. Working with a pool with any classes - using generic shell without inheritance.
3. Ease of use - using construction (implementation by the shell IDisposable interface).
4. Auto-selection of new objects is the abstract Pool.ObjectConstructor method, in which the object is initialized as you please.
5. Limiting the number of objects - a lightweight version of the semaphore.
6. Auto-clearing an object when it is returned is a virtual method called Pool.CleanUp, which is automatically called by the pool when it returns.
7. Thread Safety — using a collection of ConcurrentStack and CAS operations (methods of the Interlocked class).
8. Support for multiple pool instances — the Pool class is not static, not singleton, and provides validation checks.

Source code with unit tests and test application: Github
If interested, I can continue the article with the implementation of asynchronous TCP and UDP socket servers for which this pool was written.