In this article, I put the benchmarks of the most private calls to the loggers. I conducted all experiments on log4net and NLog, on Windows 10 x64 Intel with M.2 SSD.
To not read for a long time, the results table is immediately on top.
Raw results can be viewed on github . In the same repository code (to run, you need .Net 4.7.2 + Microsoft Visual Studio 2017+).
What, how and why - under the cut.
Method | Mean | Error | Stddev | Median | Ratio | Rank |
---|---|---|---|---|---|---|
KeepFileOpen = true, ConcurrentWrites = false, Async = true | 1,835.730 ns | 55.3980 ns | 163.3422 ns | 1,791.901 ns | 1.00 | one |
KeepFileOpen = true, ConcurrentWrites = true, Async = true | 1,910.814 ns | 37.9116 ns | 90.1008 ns | 1,908.513 ns | 1.00 | one |
KeepFileOpen = false, ConcurrentWrites = false, Async = true | 1,765.390 ns | 34.5893 ns | 33.9713 ns | 1,764.598 ns | 1.00 | one |
KeepFileOpen = false, ConcurrentWrites = true, Async = true | 1,834.002 ns | 36.2599 ns | 56.4523 ns | 1,838.500 ns | 1.00 | one |
KeepFileOpen = true, ConcurrentWrites = false, Async = false | 5,387.220 ns | 77.9523 ns | 69.1027 ns | 5,376.298 ns | 1.00 | one |
KeepFileOpen = true, ConcurrentWrites = true, Async = false | 11,171.048 ns | 58.5253 ns | 54.7446 ns | 11,186.697 ns | 1.00 | one |
KeepFileOpen = false, ConcurrentWrites = false, Async = false | 652,512.923 ns | 8,856.6000 ns | 8,284.4691 ns | 650,545.605 ns | 1.00 | one |
KeepFileOpen = false, ConcurrentWrites = true, Async = false | 642,003.054 ns | 12,750.3183 ns | 31,515.6277 ns | 645,749.609 ns | 1.00 | one |
CreateLog4NetFromString | 1,271.456 ns | 3.9287 ns | 3.4827 ns | 1,271.722 ns | 1.00 | one |
CreateNLogFromString | 199.004 ns | 0.3101 ns | 0.2901 ns | 199.046 ns | 1.00 | one |
CreateLog4NetLogger | 18,564.228 ns | 44.6344 ns | 41.7510 ns | 18,564.598 ns | 1.00 | one |
CreateNLogTypeOfLogger | 140,220.404 ns | 188.8802 ns | 176.6787 ns | 140,235.303 ns | 1.00 | one |
CreateNLogDynamicLogger | 115,329.549 ns | 243.0537 ns | 227.3526 ns | 115,361.597 ns | 1.00 | one |
FileLoggingLog4NetNoParams | 7,076.251 ns | 41.5518 ns | 38.8676 ns | 7,075,394 ns | 1.00 | one |
FileLoggingLog4NetSingleReferenceParam | 7,464.427 ns | 36.0445 ns | 33.7161 ns | 7,470.789 ns | 1.00 | one |
FileLoggingLog4NetSingleValueParam | 7,684.635 ns | 49.2505 ns | 46.0690 ns | 7,693.219 ns | 1.00 | one |
FileLoggingLog4NetMultipleReferencesParam | 8,207.387 ns | 79.5855 ns | 74.4443 ns | 8,220.897 ns | 1.00 | one |
FileLoggingLog4NetMultipleValuesParam | 8,477.657 ns | 63.4105 ns | 59.3143 ns | 8,472.385 ns | 1.00 | one |
FileLoggingNLogNetNoParams | 5,438.306 ns | 42.0170 ns | 37.2470 ns | 5,427.805 ns | 1.00 | one |
FileLoggingNLogNetSingleReferenceParam | 5,734.572 ns | 46.0770 ns | 40.8461 ns | 5,729.974 ns | 1.00 | one |
FileLoggingNLogNetSingleValueParam | 5,834.548 ns | 40.4125 ns | 35.8246 ns | 5,838.905 ns | 1.00 | one |
FileLoggingNLogNetMultipleReferencesParam | 6,445.663 ns | 57.5870 ns | 53.8669 ns | 6,440.509 ns | 1.00 | one |
FileLoggingNLogNetMultipleValuesParam | 6,784.489 ns | 43.9255 ns | 38.9388 ns | 6,782.898 ns | 1.00 | one |
NoOpLog4NetNoParams | 11.063 ns | 0.0141 ns | 0.0125 ns | 11.065 ns | 1.00 | one |
NoOpLog4NetSingleReferenceParam | 9.206 ns | 0.0321 ns | 0.0300 ns | 9.203 ns | 1.00 | one |
NoOpLog4NetSingleValueParam | 11.423 ns | 0.0147 ns | 0.0131 ns | 11.421 ns | 1.00 | one |
NoOpLog4NetMultipleReferencesParam | 44.472 ns | 0.0474 ns | 0.0396 ns | 44.475 ns | 1.00 | one |
NoOpLog4NetMultipleValuesParam | 58.138 ns | 0.1598 ns | 0.1416 ns | 58.139 ns | 1.00 | one |
NoOpNLogNetNoParams | 1.045 ns | 0.0037 ns | 0.0033 ns | 1.045 ns | 1.00 | one |
NoOpNLogNetSingleReferenceParam | 1.994 ns | 0.0033 ns | 0.0028 ns | 1.994 ns | 1.00 | one |
NoOpNLogNetSingleValueParam | 2.025 ns | 0.0044 ns | 0.0042 ns | 2.024 ns | 1.00 | one |
NoOpNLogNetMultipleReferencesParam | 34.800 ns | 0.0374 ns | 0.0312 ns | 34.798 ns | 1.00 | one |
NoOpNLogNetMultipleValuesParam | 47.509 ns | 0.1199 ns | 0.1063 ns | 47.511 ns | 1.00 | one |
First, we estimate how much time we spend on calling the logging method, which ultimately leads to nothing. In most cases (in my experience) the verbose Debug is disabled on the combat servers, but no one removes calls.
First result:
Method | Mean | Error | Stddev | Median |
---|---|---|---|---|
NoOpLog4NetNoParams | 11.063 ns | 0.0141 ns | 0.0125 ns | 11.065 ns |
NoOpLog4NetSingleReferenceParam | 9.206 ns | 0.0321 ns | 0.0300 ns | 9.203 ns |
NoOpLog4NetSingleValueParam | 11.423 ns | 0.0147 ns | 0.0131 ns | 11.421 ns |
NoOpLog4NetMultipleReferencesParam | 44.472 ns | 0.0474 ns | 0.0396 ns | 44.475 ns |
NoOpLog4NetMultipleValuesParam | 58.138 ns | 0.1598 ns | 0.1416 ns | 58.139 ns |
NoOpNLogNetNoParams | 1.045 ns | 0.0037 ns | 0.0033 ns | 1.045 ns |
NoOpNLogNetSingleReferenceParam | 1.994 ns | 0.0033 ns | 0.0028 ns | 1.994 ns |
NoOpNLogNetSingleValueParam | 2.025 ns | 0.0044 ns | 0.0042 ns | 2.024 ns |
NoOpNLogNetMultipleReferencesParam | 34.800 ns | 0.0374 ns | 0.0312 ns | 34.798 ns |
NoOpNLogNetMultipleValuesParam | 47.509 ns | 0.1199 ns | 0.1063 ns | 47.511 ns |
And the code:
void Log4NetNoParams() => _log4Net.Debug("test"); void Log4NetSingleReferenceParam() => _log4Net.DebugFormat("test {0}", _stringArgument); void Log4NetSingleValueParam() => _log4Net.DebugFormat("test {0}", _intArgument); void Log4NetMultipleReferencesParam() => _log4Net.DebugFormat( "test {0} {1} {2} {3} {4} {5} {6} {7} {8}", _stringArgument, _stringArgument, _stringArgument, _stringArgument, _stringArgument, _stringArgument, _stringArgument, _stringArgument, _stringArgument); void Log4NetMultipleValuesParam() => _log4Net.DebugFormat( "test {0} {1} {2} {3} {4} {5} {6} {7} {8}", _intArgument, _intArgument, _intArgument, _intArgument, _intArgument, _intArgument, _intArgument, _intArgument, _intArgument); void NLogNetNoParams() => _nlog.Debug("test"); void NLogNetSingleReferenceParam() => _nlog.Debug("test {0}", _stringArgument); void NLogNetSingleValueParam() => _nlog.Debug("test {0}", _intArgument); void NLogNetMultipleReferencesParam() => _nlog.Debug( "test {0} {1} {2} {3} {4} {5} {6} {7} {8}", _stringArgument, _stringArgument, _stringArgument, _stringArgument, _stringArgument, _stringArgument, _stringArgument, _stringArgument, _stringArgument); void NLogNetMultipleValuesParam() => _nlog.Debug( "test {0} {1} {2} {3} {4} {5} {6} {7} {8}", _intArgument, _intArgument, _intArgument, _intArgument, _intArgument, _intArgument, _intArgument, _intArgument, _intArgument);
First, let's decide why these tests were chosen:
NLog and log4net have different function signatures for a small number of arguments:
void DebugFormat(string format, object arg0)
void Debug(string message, string argument) void Debug<TArgument>(string message, TArgument argument)
And now the analysis of the results:
Log4NetSingleReferenceParam
and Log4NetSingleValueParam
)Most programs (according to my observations) still log the results to a file, so for comparison we will select this operation. For simplicity, let's just take logger configurations when writing to a file without buffering, without additional locks, etc.
Results:
Method | Mean | Error | Stddev | Median |
---|---|---|---|---|
FileLoggingLog4NetNoParams | 7,076.251 ns | 41.5518 ns | 38.8676 ns | 7,075,394 ns |
FileLoggingLog4NetSingleReferenceParam | 7,464.427 ns | 36.0445 ns | 33.7161 ns | 7,470.789 ns |
FileLoggingLog4NetSingleValueParam | 7,684.635 ns | 49.2505 ns | 46.0690 ns | 7,693.219 ns |
FileLoggingLog4NetMultipleReferencesParam | 8,207.387 ns | 79.5855 ns | 74.4443 ns | 8,220.897 ns |
FileLoggingLog4NetMultipleValuesParam | 8,477.657 ns | 63.4105 ns | 59.3143 ns | 8,472.385 ns |
FileLoggingNLogNetNoParams | 5,438.306 ns | 42.0170 ns | 37.2470 ns | 5,427.805 ns |
FileLoggingNLogNetSingleReferenceParam | 5,734.572 ns | 46.0770 ns | 40.8461 ns | 5,729.974 ns |
FileLoggingNLogNetSingleValueParam | 5,834.548 ns | 40.4125 ns | 35.8246 ns | 5,838.905 ns |
FileLoggingNLogNetMultipleReferencesParam | 6,445.663 ns | 57.5870 ns | 53.8669 ns | 6,440.509 ns |
FileLoggingNLogNetMultipleValuesParam | 6,784.489 ns | 43.9255 ns | 38.9388 ns | 6,782.898 ns |
Code used:
var roller = new RollingFileAppender(); roller.ImmediateFlush = true; roller.RollingStyle = RollingFileAppender.RollingMode.Once; roller.MaxFileSize = 128 * 1000 * 1000;
new FileTarget($"target_{_logIndex++}") { ArchiveAboveSize = 128 * 1000 * 1000, MaxArchiveFiles = 16, AutoFlush = true, ConcurrentWrites = false, KeepFileOpen = false };
As you can see, the configuration of the loggers is more or less similar, but according to the results:
Method | Mean | Error | Stddev | Median |
---|---|---|---|---|
KeepFileOpen = true, ConcurrentWrites = false, Async = false | 5,387.220 ns | 77.9523 ns | 69.1027 ns | 5,376.298 ns |
KeepFileOpen = true, ConcurrentWrites = true, Async = false | 11,171.048 ns | 58.5253 ns | 54.7446 ns | 11,186.697 ns |
KeepFileOpen = false, ConcurrentWrites = false, Async = false | 652,512.923 ns | 8,856.6000 ns | 8,284.4691 ns | 650,545.605 ns |
KeepFileOpen = false, ConcurrentWrites = true, Async = false | 642,003.054 ns | 12,750.3183 ns | 31,515.6277 ns | 645,749.609 ns |
Source:
new FileTarget($"target_{_logIndex++}") { ArchiveAboveSize = 128 * 1000 * 1000, MaxArchiveFiles = 16, AutoFlush = true, ConcurrentWrites = XXXXX, KeepFileOpen = YYYYY };
If in place of XXXXX and YYYYY to put all possible combinations, we get a test from the table.
The results are pretty predictable:
KeepFileOpen=true
, we created a file for each logging operation (along with Handle), made a record on the disk, called Flush, gave it to Handle, and also made a lot of engine operations. As a result, the speed drops hundreds of times.The NLog library is also able to perform all IO operations on another thread, immediately freeing the current one. Moreover, it does this competently, preserving the order of events, dropping all the data in blocks, and in each block an integer number of events (so as not to get truncated strings), and so on.
The results of different ways of non-blocking work:
Method | Mean | Error | Stddev | Median |
---|---|---|---|---|
KeepFileOpen = true, ConcurrentWrites = false, Async = true | 1,835.730 ns | 55.3980 ns | 163.3422 ns | 1,791.901 ns |
KeepFileOpen = true, ConcurrentWrites = true, Async = true | 1,910.814 ns | 37.9116 ns | 90.1008 ns | 1,908.513 ns |
KeepFileOpen = false, ConcurrentWrites = false, Async = true | 1,765.390 ns | 34.5893 ns | 33.9713 ns | 1,764.598 ns |
KeepFileOpen = false, ConcurrentWrites = true, Async = true | 1,834.002 ns | 36.2599 ns | 56.4523 ns | 1,838.500 ns |
Comparison of blocking and asynchronous approaches will be further, and here - only the latter.
AsyncTargetWrapper
code:
new AsyncTargetWrapper(fileTargetWithConcurrentWritesAndCloseFileAsync) { OverflowAction = AsyncTargetWrapperOverflowAction.Block, BatchSize = 4096, FullBatchSizeWriteLimit = 128 }
As you can see, the settings of the wrapper are such that a direct reset to the file does not take a long time. Thus, a large buffer is accumulated, which means all resource-intensive operations like “open file” are performed once for the whole block. However, this algorithm requires additional memory (and quite a lot).
Findings:
Results: | Method | Mean | Error | Stddev | Median |
---|---|---|---|---|---|
KeepFileOpen = true, ConcurrentWrites = false, Async = true | 1,835.730 ns | 55.3980 ns | 163.3422 ns | 1,791.901 ns | |
FileLoggingLog4NetNoParams | 7,076.251 ns | 41.5518 ns | 38.8676 ns | 7,075,394 ns | |
FileLoggingNLogNetNoParams | 5,438.306 ns | 42.0170 ns | 37.2470 ns | 5,427.805 ns | |
NoOpLog4NetNoParams | 11.063 ns | 0.0141 ns | 0.0125 ns | 11.065 ns | |
NoOpNLogNetNoParams | 1.045 ns | 0.0037 ns | 0.0033 ns | 1.045 ns |
Findings:
Results: | Method | Mean | Error | Stddev | Median |
---|---|---|---|---|---|
CreateLog4NetFromString | 1,271.456 ns | 3.9287 ns | 3.4827 ns | 1,271.722 ns | |
CreateNLogFromString | 199.004 ns | 0.3101 ns | 0.2901 ns | 199.046 ns | |
CreateLog4NetLogger | 18,564.228 ns | 44.6344 ns | 41.7510 ns | 18,564.598 ns | |
CreateNLogTypeOfLogger | 140,220.404 ns | 188.8802 ns | 176.6787 ns | 140,235.303 ns | |
CreateNLogDynamicLogger | 115,329.549 ns | 243.0537 ns | 227.3526 ns | 115,361.597 ns |
What tested:
[Benchmark] public object CreateLog4NetFromString() { return LogManager.GetLogger("my-logger_" + (Interlocked.Increment(ref _log4NetStringLogIndex) % 1000)); } [Benchmark] public object CreateNLogFromString() { return NLog.LogManager.GetLogger("my-logger_" + (Interlocked.Increment(ref _nLogStringLogIndex) % 1000)); } [Benchmark] public object CreateLog4NetLogger() { return new [] { LogManager.GetLogger(typeof(BaseTest)), // x16 times }; } [Benchmark] public object CreateNLogTypeOfLogger() { return new[] { NLog.LogManager.GetCurrentClassLogger(typeof(BaseTest)), // x16 times }; } [Benchmark] public object CreateNLogDynamicLogger() { return new[] { NLog.LogManager.GetCurrentClassLogger(), // x16 times }; }
An important remark: unfortunately, it was difficult for me to make a reproducible benchmark that does not lead to Out Of Memory, but that would create different loggers (i.e., for different types, for different lines, and so on).
However, having studied the work of the libraries, I found that perhaps the most ponderous operations are performed to create the logger key (i.e., name definition, clearing of Generic arguments, and so on).
Moreover, in order to stabilize the benchmark for creating a logger for log4net, I had to perform not one operation, but 16 (that is, an array of 16 identical objects is returned). If you do not return anything, then .Net for me optimized execution (apparently, just not returning the result), which led to incorrect results.
And the conclusions:
Debug
, Info
, etc. directly. In fact, each ILogger
knows the answer for itself: whether the following methods should be called or not (and this requires at least some sort of binding to the general configuration). Because of this scheme of work, I went out of memory on most tests (if you use different lines, etc.).LogManager.GetCurrentClassLogger()
is even slower than LogManager.GetLogget(typeof(XXX))
. This is logical, even NLog developers do not recommend calling the first method in a loop.private static readonly ILogger Log = LogManager.GetCurrentClassLogger()
. That is, it does not affect the performance of the system itself.How best to handle logs:
Stream
's). However, the program can work faster.LogManager.GetLogger("123")
. Such calls work faster, which means that the logger can be created for large instances of objects (for example, "one logger for the context of the query execution").Conclusions for your code:
object
) and in most cases does nothing (which is true for contracts / validators), then it is more correct to wrap the calls in a generic form (that is, to make methods of the form Something<TArg>(TArg arg)
). This will really work faster.Source: https://habr.com/ru/post/456488/
All Articles