Why, why, and when you need to use ValueTask

^{This translation appeared thanks to a good comment 0x1000000 .}

In the .NET Framework 4, the System.Threading.Tasks space appeared, and with it the Task class. This type and Task <TResult> spawned from it waited for a long time until they were recognized as standards in .NET as key aspects of the asynchronous programming model that was introduced in C # 5 with its async / await operators. In this article, I will talk about new types of ValueTask / ValueTask <TResult> designed to improve the performance of asynchronous methods in cases where the cost of memory allocation needs to be taken into account.

Task

Task performs different roles, but the main one is a “promise” (promise), an object representing the possible completion of some operation. You initiate an operation and get a Task object for it, which will be executed when the operation is completed, which can occur in synchronous mode as part of the initialization of the operation (for example, receiving data that is already in the buffer), in asynchronous mode with execution at the moment when you receive a Task (receiving data not from the buffer, but very quickly), or in asynchronous mode, but after the Task you already have (receiving data from a remote resource). Since the operation may complete asynchronously, you either block the thread of execution, waiting for the result (which often makes the asynchronous call pointless), or create a callback function that will be activated after the operation is completed. In .Net 4, the creation of a callback is implemented by the ContinueWith methods of the Task object, which explicitly demonstrate this model by accepting the delegate function (delegate) to start it after Task execution:

SomeOperationAsync().ContinueWith(task => { try { TResult result = task.Result; UseResult(result); } catch (Exception e) { HandleException(e); } }); 

But in the .NET Framework 4.5 and C # 5, Task objects can simply be called by the await operator, which makes it easy to get the result of an asynchronous operation, and the generated code that is optimized for the above options will work correctly in all cases of completing the operation in synchronous mode, fast asynchronous or asynchronous with callbacka execution:

 TResult result = await SomeOperationAsync(); UseResult(result); 

Task is a very flexible class and it has several advantages. For example, you can perform await several times for any number of consumers at the same time. You can put it in a dictionary for repeated awaits in the future to use it as a cache of asynchronous call results. You can block the execution, waiting for the Task to complete if necessary. And you can write and apply various operations on Task objects (sometimes called “combinators”), for example, “when any” to asynchronously wait for the first completion of several Task.
But this flexibility becomes superfluous in the most common case: just call an asynchronous operation and wait for the task to complete:

 TResult result = await SomeOperationAsync(); UseResult(result); 

Here we do not need to wait for execution several times. We do not need to provide competitive expectations. We do not need to perform synchronous locking. We will not write combinators. We are just waiting for the promise of an asynchronous operation. After all, this is how we write synchronous code (for example, TResult result = SomeOperation ();), and this is translated into async / await in the usual way.

Moreover, Task has a potential weakness, especially when a large number of instances are created, and high throughput and performance are key requirements — Task is a class. This means that any operation that Task needed is forced to create and place an object, and the more objects are created, the more work for the garbage collector (GC), and this work consumes resources that we could spend on something more useful.

The runtime and system libraries help mitigate this problem in many situations. For example, if we write this method:

 public async Task WriteAsync(byte value) { if (_bufferedCount == _buffer.Length) { await FlushAsync(); } _buffer[_bufferedCount++] = value; } 

as a rule, there will be enough free space in the buffer, and the operation will be executed synchronously. When this happens, there is no need to do anything with the Task, which should be returned, since the return value is missing, this is using Task as the equivalent of a synchronous method that returns a null value (void). Therefore, the environment can simply cache one non-generic (Non-generic) Task and use it again and again as the execution result for any async method that is completed synchronously (this cached Singleton can be obtained via Task.CompletedTask). Or, for example, you write:

 public async Task<bool> MoveNextAsync() { if (_bufferedCount == 0) { await FillBuffer(); } return _bufferedCount > 0; } 

and generally expect the data to be already in the buffer, so the method simply checks the _bufferedCount value, sees that it is greater than 0, and returns true; and only if there is no data in the buffer yet, you need to perform an asynchronous operation. And since there are only two possible results of the Boolean type (true and false), there are only two possible Task objects that are needed to represent these results, the environment can cache these objects and return them with the corresponding value without memory allocation. Only in the case of asynchronous completion, the method will need to create a new Task, because it will need to be returned before the result of the operation is known.

The environment provides caching for some other types, but it is unrealistic to cache all possible types. For example, the following method:

 public async Task<int> ReadNextByteAsync() { if (_bufferedCount == 0) { await FillBuffer(); } if (_bufferedCount == 0) { return -1; } _bufferedCount--; return _buffer[_position++]; } 

will also often be performed synchronously. But unlike the variant with the Boolean result, this method returns an Int32, which has about 4 billion values, and caching all Task <int> variants will require hundreds of gigabytes of memory. The environment provides a small cache for Task <int>, but a strongly limited set of values, for example, if this method completes synchronously (data is already in the buffer) with a return value of 4, it will be a cached Task, but if it returns 42, you will need to create a new Task <int>, like calling Task.FromResult (42).

Many library methods attempt to smooth this by providing their own cache. For example, the overload in the .NET Framework 4.5 of the MemoryStream.ReadAsync method always ends synchronously, since it reads data from memory. ReadAsync returns a Task <int>, where the result of the type Int32 shows how many bytes have been read. This method is often used in a loop, often with the same required number of bytes for each call, and often this need is met in full. So for repeated calls to ReadAsync, it is reasonable to expect Task <int> to return synchronously with the same value as in the previous call. Therefore, a MemoryStream creates a cache for a single object that is returned in the last successful call. And in the next call, if the result is repeated, it will return the cached object, and if not, create a new one with Task.FromResult, save it to the cache and return it.

And yet there are many other cases where the operation is performed synchronously, but the Task <TResult> object is forcedly created.

ValueTask <TResult> and synchronous execution

All this required the implementation of a new type in .NET Core 2.0, which is available in previous versions of .NET in the NuGet System.Threading.Tasks.Extensions: ValueTask <TResult> package.
ValueTask <TResult> is created in .NET Core 2.0 as a structure that can wrap both TResult and Task <TResult>. This means that it can be returned from the async method, and if this method executes synchronously and successfully, no object should be placed on the heap: you can simply initialize this ValueTask <TResult> structure to TResult and return it. Only in the case of asynchronous execution, the Task <TResult> object will be placed, and ValueTask <TResult> will wrap it (to minimize the size of the structure and optimize the case of successful execution, the async method, which ends with an unsupported exception, will also allocate Task <TResult>, so ValueTask <TResult> will also simply wrap Task <TResult>, and will not carry an extra field for storing Exception).

Based on this, a method like MemoryStream.ReadAsync, but returning ValueTask <int>, should not be caching, but instead can be written like this:

 public override ValueTask<int> ReadAsync(byte[] buffer, int offset, int count) { try { int bytesRead = Read(buffer, offset, count); return new ValueTask<int>(bytesRead); } catch (Exception e) { return new ValueTask<int>(Task.FromException<int>(e)); } } 

ValueTask <TResult> and asynchronous execution

The ability to write an async method that can complete synchronously without the need for additional placement for the result is a big win. That's why ValueTask <TResult> was added to .NET Core 2.0, and new methods that are likely to be used in applications that require performance are now announced with the return of ValueTask <TResult> instead of Task <TResult>. For example, when we added a new ReadAsync overload to the Stream class in .NET Core 2.1, in order to be able to pass Memory instead of byte [], we return the ValueTask <int> type in it. In this form, Stream objects (in which very often the ReadAsync method is executed synchronously, as in the earlier example for the MemoryStream) can be used with much less memory allocation.

However, when we work with services with very high bandwidth, we still want to avoid allocating memory as much as possible, which means reducing and eliminating memory allocations along the asynchronous execution route.
In the await model, for any operation that terminates asynchronously, we need the ability to return an object that represents the possible completion of the operation: the caller needs to redirect the callback that will be triggered upon completion of the operation, and this requires a unique object on the heap that can serve as a transmission channel for this particular operation. This, at the same time, does not mean whether the object will be used again after the operation is completed. If this object can be reused, the API can organize a cache for one or several such objects, and use it for sequential operations, in the sense of not using the same object for several intermediate async operations, but using it for noncompetitive access.
In .NET Core 2.1, the ValueTask <TResult> class has been enhanced to support this kind of pooling and reuse. Instead of simply wrapping a TResult or Task <TResult>, the refined class can wrap a new IValueTaskSource <TResult> interface. This interface provides the basic functionality that is required to support an asynchronous operation with a ValueTask <TResult> object just as Task <TResult> does:

 public interface IValueTaskSource<out TResult> { ValueTaskSourceStatus GetStatus(short token); void OnCompleted(Action<object> continuation, object state, short token, ValueTaskSourceOnCompletedFlags flags); TResult GetResult(short token); } 

The GetStatus method is used to implement properties like ValueTask <TResult> .IsCompleted, which returns information as an asynchronous operation or completed, and how it is completed (successfully or not). The OnCompleted method is used by the waiting object to attach a callback to continue execution from the await point when the operation completes. And the GetResult method is needed to get the result of the operation, so that after the end of the operation, the calling method can get a TResult object or pass any exception that was thrown.

Most developers do not need this interface: the methods simply return a ValueTask <TResult> object, which can be created as a wrapper for an object that implements this interface, and the calling method will remain in the dark. This interface is for developers who need to avoid memory allocation when using a performance-critical API.

There are several examples of such an API in .NET Core 2.1. The most well-known methods are Socket.ReceiveAsync and Socket.SendAsync with new overloads added in 2.1, for example

 public ValueTask<int> ReceiveAsync(Memory<byte> buffer, SocketFlags socketFlags, CancellationToken cancellationToken = default); 

This overload returns ValueTask <int>. If the operation completes synchronously, it can simply return a ValueTask <int> with the appropriate value:

 int result = …; return new ValueTask<int>(result); 

For asynchronous completion, it can use an object from the pool that implements the interface:

 IValueTaskSource<int> vts = …; return new ValueTask<int>(vts); 

The implementation of the Socket supports one such object in the pool for receiving, and one for transfer, since there can not be more than one object for each direction waiting to be executed at a time. These overloads do not allocate memory, even in the case of asynchronous execution of the operation. This behavior manifests itself further in the NetworkStream class.
For example, in .NET Core 2.1 Stream provides:

 public virtual ValueTask<int> ReadAsync(Memory<byte> buffer, CancellationToken cancellationToken); 

which is overridden in NetworkStream. The NetworkStream.ReadAsync method simply uses the Socket.ReceiveAsync method, so the Socket win is translated to the NetworkStream, and NetworkStream.ReadAsync also does not actually allocate memory.

Unsubscribed ValueTask

When ValueTask <TResult> appeared in .NET Core 2.0, only the case of synchronous execution was optimized in it in order to exclude the placement of the Task <TResult> object if the TResult value is already ready. This meant that the non-generic ValueTask class was not needed: for the case of synchronous execution, the singletonton Task.CompletedTask could simply be returned from the method, and this was done by the environment implicitly in the async methods that return Task.

However, with the receipt of asynchronous operations without memory allocation, the use of a non-generic ValueTask has become relevant again. In .NET Core 2.1, we introduced the non-generic ValueTask and IValueTaskSource. They provide direct equivalents for generic versions, for similar use, only with an empty return value.

Implementing IValueTaskSource / IValueTaskSource <T>

Most developers should not implement these interfaces. Besides, it's not so easy. If you decide to do this, several implementations in .NET Core 2.1 can serve as a starting point, for example:

• AwaitableSocketAsyncEventArgs
• AsyncOperation <TResult>
• DefaultPipeReader

To make it easier, in .NET Core 3.0 we plan to provide all the necessary logic included in the type of ManualResetValueTaskSourceCore <TResult>, a structure that can be embedded into another object that implements IValueTaskSource <TResult> and / or IValueTaskSource so that you can delegate to This structure is the main part of the functionality. You can learn more about this from https://github.com/dotnet/corefx/issues/32664 in the dotnet / corefx repository.

ValueTasks Application Patterns

At first glance, the scope of ValueTask and ValueTask <TResult> is much more limited than Task and Task <TResult>. This is good, and even expected, since the main way to use them is simply to use it with the await operator.

However, since they can wrap objects that are reused, there are significant restrictions on their use compared to Task and Task <TResult> if you deviate from the usual simple await method. In general, the following operations should never be performed with ValueTask / ValueTask <TResult>:

• Repeating Wait ValueTask / ValueTask <TResult> The result object can already be disposed of and used in another operation. In contrast, Task / Task <TResult> never transitions from a completed state to an incomplete state, so you can re-wait it as many times as needed and get the same result every time.
• Parallel wait for ValueTask / ValueTask <TResult> The result object waits for processing by only one callback from one consumer at a time, and trying to wait for it from different threads at the same time can easily lead to racing and subtle program errors. In addition, it is also a more specific case of the previous invalid operation “re-wait”. By comparison, Task / Task <TResult> provides any number of concurrent awaits.
• Using .GetAwaiter (). GetResult () when the operation is not completed. Implementing IValueTaskSource / IValueTaskSource <TResult> does not need support for locking until the end of the operation, and most likely will not do this, so this operation will definitely lead to racing and probably will not be executed as the caller expects. Task / Task <TResult> blocks the calling thread until the task is executed.

If you received a ValueTask or ValueTask <TResult>, but you need to perform one of these three operations, you can use .AsTask (), get Task / Task <TResult> and then work with the received object. After that, you can no longer use that ValueTask / ValueTask <TResult>.

In short, the rule is this: when applying ValueTask / ValueTask <TResult>, you must either either await it directly (possibly with .ConfigureAwait (false)) or call AsTask () and not use it again:

 //   ,  ValueTask<int> public ValueTask<int\> SomeValueTaskReturningMethodAsync(); ... // GOOD int result = await SomeValueTaskReturningMethodAsync(); // GOOD int result = await SomeValueTaskReturningMethodAsync().ConfigureAwait(false); // GOOD Task<int> t = SomeValueTaskReturningMethodAsync().AsTask(); // WARNING ValueTask<int> vt = SomeValueTaskReturningMethodAsync(); //       , //     // BAD: await   ValueTask<int> vt = SomeValueTaskReturningMethodAsync(); int result = await vt; int result2 = await vt; // BAD: await  (    ) ValueTask<int> vt = SomeValueTaskReturningMethodAsync(); Task.Run(async () => await vt); Task.Run(async () => await vt); // BAD:  GetAwaiter().GetResult(),     ValueTask<int> vt = SomeValueTaskReturningMethodAsync(); int result = vt.GetAwaiter().GetResult(); 

There is one more advanced pattern that programmers can use, I hope, only after careful measurement and obtaining significant advantages. The ValueTask / ValueTask <TResult> classes have several properties that report the current status of the operation, for example, the IsCompleted property returns true if the operation has been completed (that is, it is no longer running and completed successfully or not successfully), and the IsCompletedSuccessful property returns true, only if completed successfully (when waiting and receiving the result did not throw an exception). For the most intense execution threads, where the developer wants to avoid the costs that appear in asynchronous mode, these properties can be checked before an operation that actually destroys a ValueTask / ValueTask <TResult> object, such as await, .AsTask (). For example, in the implementation of SocketsHttpHandler in .NET Core 2.1, the code reads from the connection and gets the ValueTask <int>. If this operation is performed synchronously, we should not worry about early interruption of the operation. But if it is executed asynchronously, we have to hook up the interrupt processing in order for the interrupt request to break the connection. Since this is a very tight section of code, if profiling reveals the need for the next small change, it can be structured like this:

 int bytesRead; { ValueTask<int> readTask = _connection.ReadAsync(buffer); if (readTask.IsCompletedSuccessfully) { bytesRead = readTask.Result; } else { using (_connection.RegisterCancellation()) { bytesRead = await readTask; } } } 

Should each asynchronous API method return a ValueTask / ValueTask <TResult>?

In short: no, the default is to still choose Task / Task <TResult>.
As highlighted above, Task and Task <TResult> are correctly used more easily than ValueTask and ValueTask <TResult>, and as long as performance requirements do not outweigh practicality requirements, Task and Task <TResult> are preferred. In addition, there are small costs associated with returning ValueTask <TResult> instead of Task <TResult>, that is, micro-benchmarks show that await Task <TResult> is faster than await ValueTask <TResult>. So, if you use task caching, for example, your method returns a Task or Task, for performance it is worth staying with Task or Task. ValueTask / ValueTask <TResult> objects occupy several words in memory, so when they are expected and fields are reserved for them in the state machine that calls the async method, they will occupy more memory in it.

- ValueTask/ValueTask<TResult> : ) , await, ) , ) , . , / .

ValueTask ValueTask<TResult>?

.NET , Task/Task<TResult>, , ValueTask/ValueTask<TResult>, , . – IAsyncEnumerator<T>, .NET Core 3.0. IEnumerator<T> MoveNext, bool, IAsyncEnumerator<T> MoveNextAsync. , , Task, . , , , ( ), await foreach, ValueTask. , . C# , , , .