.NET development: nine questions for adults
.NET becomes truly cross-platform: after a long wait, the release date of ASP.NET Core is finally announced, JetBrains is preparing an alternative to Visual Studio based on ReSharper and IDEA, Microsoft acquired Xamarin, made the Xamarin Community free, and Mono transferred to the MIT license and finally, Windows Server 2016 will receive support for Windows containers in Docker.
With new opportunities we face new challenges:
In just three weeks, at the DotNext conference in St. Petersburg, 20 speakers will give presentations on the present and future of the .NET platform, on optimizing performance and multithreading, on the internal structure of the .NET platform and the CLR, on profiling and debugging the .NET code.
In the meantime, we asked four of them to share their experiences and opinions about future changes in the world of .NET. Our questions were answered:
Sasha Goldstein
For Microsoft, Roslyn is a project of immense importance. Almost everyone in the C # team has been working on it for seven years, and probably they couldn’t release it for a very long time. And now he is on github in open-source and the whole process of language changes is before his eyes. On the one hand, many people came up with strange ideas to add constructions from other languages ​​to C #, on the other hand, the whole process takes place openly and in two hours you can answer any question to three people who write the compiler. For the community, it's very cool.
')
Andrey Akinshin
Overall I like Roslyn very much. As a compiler, it is better than the old one: it compiles many code sections more competently and more successfully in terms of performance. But if we are talking about compiling C # code, then I’m more interested in the next step: JIT compilation. Not so long ago, Microsoft introduced a new 64-bit JIT compiler called RyuJIT.
One of its main objectives is to reduce JIT compilation time. We give the user a very quick start, but because of this we can’t do cool optimizations, so the code itself runs slower than it could, and sometimes even slower than the old JIT. At the same time, RyuJIT is able to use SSE and AVX instructions well, which allows, with due diligence, to implement highly effective programs using the power of one C # without referring to native code.
In terms of performance, it seems to me, Microsoft is betting on another technology that is now actively developing: .NET Native. While it is still in a very raw state, but it looks promising.
Nikita Tsukanov
The main feature of Roslyn is that Microsoft can now add new features with acceptable speed. There used to be one compiler for the build, the other for the studio. Now this is not - there is only one compiler left and it is written in C #.
Dmitry Ivanov
From the point of view of the API, it has certainly become better, but there are problems with performance and excess memory, and they are far from being resolved. Now we cannot open huge projects in Visual Studio 2015 together with ReSharper, because there is not enough memory.
Sasha Goldstein
Mono moves to MIT license. So Microsoft will be able to copy pieces of Mono to CoreFx and use them. It seems to me that Mono should slowly fade away. Mono will have some niches on which .NET Core will not work yet. But Mono's runtime itself is so-so: GC only recently switched to Generational GC, JIT doesn't work very well. And the framework itself, which they wrote in many cases is not production-ready: there are a lot of bugs, bugs are detected and not repaired for a long time. Almost all my clients who used Mono in production faced problems. This does not mean that in the end the system did not work at all, but this is not the level of quality we are used to in .NET.
But with .NET Core, too, everything is not smooth. The project was supposed to end in January, then in March, just a few days ago, they said that it would be ready at the end of June. API, project model, half the code is constantly changing.
Last week, one of the clients with a project on ASP.NET MVC 6 asked how I thought it was necessary to repair the code every two weeks or it was better to backport to ASP.NET MVC 5 and wait. The question is not simple.
Andrey Akinshin
.NET Core is certainly a very interesting project, a big step forward, but today it is still raw. Let's not forget that many libraries have not yet come out under it or are in RC. Plus, Microsoft constantly renames something, every day everything changes, some bugs are repaired, others are added. For two years, a new project system was developed and promoted on the basis of project.json files, and just recently it was thrown into the dustbin and decided to return to the good old csproj. Such situations discourage a desire in the foreseeable future to try to transfer the production system to the .NET Core.
Mono has been in development for over 15 years. This is an adult, solid runtime, which runs a lot of production-code. For example, I am working on a project Rider (cross-platform .NET IDE) in JetBrains. Backend Rider (this is R # in its pure form) was launched under Mono, and under .NET Core it’s not really possible to launch. It works quite well: a healthy application from hundreds of projects and very nontrivial logic, with all the features of .NET and all this under * nix starts up normally. There are, of course, their own problems, but they are completely resolved in working order. And about the transition to .NET Core: you need to wait for this new cross-platform runtime to become more stable.
Nikita Tsukanov
Mono is a fairly mature environment, and .NET Core is still in beta and by the number of available libraries. NET Core is definitely losing. For example, in .NET Core there is currently no support for System.Drawing, and in Mono it appeared in the 2000th shaggy year.
Regarding the performance and quality of the .NET Framework coverage, Mono actively brings to itself those pieces that Microsoft opens. Given that Microsoft acquired Xamarin, the process should go even faster. At some point, Mono will become an add-on over .NET Core, but for now it makes sense to use Mono.
Dmitry Ivanov
We probably almost all have an opinion that Mono at some point will drop. At one time, Oracle also said that it would merge the HotSpot JVM with JRockit, but in the end it dropped the last one.
So, probably, it is worth waiting a bit, and switching from Mono to .NET Core. We in Rider work a lot with Mono and see that there are a lot of errors and the code is much lower quality than in the .NET Framework.
Andrey Akinshin
I believe that there is no silver bullet, there are no universal libraries that suit everyone. It is necessary to select a solution for your task. If you work with a database, there are databases that work faster than others on certain indicators; if you work with graphics, there are various good libraries for rendering 2D / 3D, but each is good for its own spectrum of tasks. It must be remembered that each project is unique in its own way. Most often, some solutions are suitable for some solutions, for others - others.
A very popular mistake: a person has heard that the ABC library is good, it takes and uses it, without thinking about what tasks it is intended for, on what scenarios it behaves quickly ...
An important point: many people bother with performance and spend a lot of time optimizing non-essential sections of code. There will be some speed increase from this, but no one can notice it. When we want to improve performance, we need to conduct research, do benchmarks, and profile.
Dmitry Ivanov
Even within one .NET-team, we have several solutions for interprocess communication, so yes, we have to often. Now, high-performance solutions usually mean some kind of server load.
If we talk about performance problems on one machine, it is also constantly being solved, because, let's say, we quickly go beyond the framework of the idiomatic C # code. Pretty quickly, you have to switch from a large number of references to arrays, so go to unsafe. Without this, it simply does not work. And we do our protocol in the Rider Framework with an eye on high performance.
Sasha Goldstein
Many developers find it difficult to imagine what exactly is happening in a multi-threaded program, that memory accesses can be moved, what to synchronize, on the other hand, if there is a lot of synchronization, bottlenecks appear. There are many tools for this, but you need to know and be able to use them.
Many developers do not have access to multiprocessor systems. When the same code is executed not on Core i7, but on the server, at least with 64 processors, completely different problems appear. Some small lock, which took 2% of the time, suddenly starts to occupy 50%, there are problems with the memory bandwidth (the memory system is not able to respond to requests quickly enough), not efficient use of the cache. .NET-developers most often do not even face such tasks.
Andrey Akinshin
I think that the complexity of multithreaded programming on .NET is the same as on any other platform. You need to understand well how a multi-threaded world works. As for the synchronization primitives and concurrent data structures, everything is fine with .NET. Plus in C # 5 appeared async / await, which allow you to write multi-threaded code in a beautiful and understandable.
Nikita Tsukanov
There is no way to prevent one stream from being able to access data that is in the possession of another. In Rust, for this purpose, a more or less normal system was made (destructive assignment). In .NET there is no such thing and without it is very hard: it may happen that when we transfer something from one stream to another, due to carelessness or other reasons, something superfluous may pass to another stream. And with “this superfluous” they can start working, although the stream owner knows nothing about this. In this case, competitive write requests occur where no one knows. As the project grows, it becomes more and more difficult to track and deal with it.
Dmitry Ivanov
Good question. I have been programming multithreaded for so many years and telling people how to do it, but I haven't figured it out myself. Probably, the main challenge arises at the top level - to come up with a model that is successfully suited for a large development team and a large product and get everyone to follow it and hope that you will never rest against the limitations of this model.
The Roslyn team decided to use the transactional model of multi-threaded interaction, and they came up against Memory Wall. We are faced with pessimistic locks and a read / write lock-model facing a long time not releasing the read lock and not a smooth taiping.
There is no silver bullet; even new-minded actors do not save: Hello World works well for them, but in reality everything is more complicated.
Sasha Goldstein
Do not save, you need. The only question is how much is needed. Do I have to be able to read an assembler, understand the memory model, how does the processor work? Standard tip - understand at least one level below the one you work at. If you are working on .NET you should understand how the OS works, Runtime, GC, Framework. Without it, even if you find a problem, you will not know how to fix it.
Andrey Akinshin
Depends on what tasks the programmer solves. 90% of the time, until there is a task to squeeze the maximum out of the machine, the programmer should think about the code to be reliable, readable, without bugs. In the remaining 10% of cases, when problems have arisen (or we guess that problems will clearly arise), and we try to solve them, then yes, you need to know how everything happens inside.
Many people try to solve performance problems without understanding what is happening in this life: they do wrong benchmarks, crookedly profile, write strange code, which only slows down everything and spend a lot of working time on it. To successfully cope with these 10% performance-critical cases, you need to know a lot about how things work inside.
Nikita Tsukanov
The problem of abstractions is that they have a tendency to flow, so the team needs one person who understands how everything works inside. Roughly with a man who knows math well. Traditional low-level problems are memory leaks, exceptions in the native code, heap corruption.
Dmitry Ivanov
In my opinion, any .NET-engineer, should at least understand how the processor and processor caches work, be a little bit aware of how branch prediction works and understand why sequential access to an array and a disk always works faster than parallel.
Sasha Goldstein
No need to guess. You need to run the tool and get information about what is happening in the process. Look at the memory, CPU, how many threads, what they do. Now there are a lot of free utilities. After that there are a bunch of methods. My favorite is USE (utilization, saturation, errors). He came up with Brandon Greg (Brendan Gregg) - an expert on performance in Linux. Utilization means that we have to determine for each resource (CPU, memory, disk ...) how much it is used. Saturation means that for all these resources we must determine if there is an over subscription, for example, whether we read 30 files from one disk at a time. Errors - the number of errors in the program. After that, you can optimize something: we will add processors or code to improve. This is in general.
Andrey Akinshin
The first thing you need to do is poprofilirti and understand what exactly slows down. A very common mistake is optimization without profiling. After we have found a bottleneck, the second step is to ask yourself the question: “why does this section of the code slow down”? Here, sometimes you need to have really a lot of knowledge: you need to guess what factors are really important and check only them, otherwise the programmer risks spending a lot of time on the unnecessary work. And the third step is to fix the problem and carefully check that the program really works faster now (and also that during the optimization nothing broke off).
Dmitry Ivanov
The classics recommend taking a profiler, find the bottlenecks and fix them. But when you have already done this several times, there are no more open bottlenecks and the program cannot be accelerated several times, having fixed a couple of lines. Sometimes in these cases it is necessary to reconsider the approach, for example, to insert by code Asserts, which make sure that the operations are completed in the allotted time. And to force each member of the team to correct performance-asserts falling in his area of ​​responsibility. It is also useful to look at memory traffic and start a fight with it. Maybe even go to unsafe.
Sasha Goldstein
On Windows, you need to start with Performance Counters. After that depends on the situation. A lot of problems can be solved only with PerfView, but it has big problems with visualization. Visual Studio Profiler is not bad at all, by the way, it has a standalone version. dotMemory or .NET Memory Profiler– for memory profiling.
Andrey Akinshin
I am the maintainer of the BenchmarkDotNet utility. The latest versions of this library already behave quite consistently and provide ample opportunities for benchmarking. Alas, many people from the Internet are constantly trying to make their own self-made benchmarks based on Stopwatch and a couple of cycles. At the same time, they are surrounded at every step by problems that they are unaware of and which are capable of completely spoiling the conclusions from the experiment being conducted. But this is only part of the trouble: after all, you can miraculously pass by all the rakes and in this particular situation write the correct code, which really measures something useful. But only every second novice performance engineer launches his benchmarks in DEBUG, because this configuration is selected by default in the studio.
Whether it is worth explaining that it can completely spoil the carried-out measurements. BenchmarkDotNet scolds the user for the debug configuration and the debugger attached in large red letters, complies with the general benchmarking methodology, makes a warm-up, runs each method in different processes several times, allows you to compare different environments, counts the statistics, etc. A community has now gathered around the project, and the library is becoming a convenient and popular tool for a number of performance studies.
As a profiling, I can recommend the products of the company JetBrains: dotMemory and dotTrace. I often use them, and I really, really like them.
Dmitry Ivanov
All our tools: dotTrace, dotMemory. Almost for everything is enough. Sometimes you need to look at sys internals utilities.
Sasha Goldstein
There are a lot of materials online, there are my courses on pluralsight, quite a lot of information can be found in the old blogs of Microsoft developers (Rico Mariani, Chris Brue, Vance Morisson). About books: there is my book Pro .NET Performance , there is a newer book - Ben Watson “High Performance. NET Applications” . Also quite interesting written. Plus, you need to know something and understand about the device .NET. There is a classic book CLR via C # . It is a bit out of date, but some points are simply not mentioned anywhere else, for example, how exceptions and delegates work.
Andrey Akinshin
I would go from top to bottom and would not go to the CPU level (as some try to do), if there is no basic knowledge of algorithms, data structures and the platform with which you work. In .NET you can read the same Richter, to understand how the base classes work and what they are good for. It is extremely important to understand the device runtime: how does GC work, how do classes differ from structures, what does the JIT compiler look like, etc.
Further, it would be good to understand how modern iron works, in particular, Intel processors (at least in general terms). In this regard, there are many resources on the Internet, but this is often not very simple reading: you cannot read a book in the evening and figure out the details of the CPU device.
However, a person who does not specialize in performance, but who works in an enterprise does not need this. I advise such people to pump the general erudition and broaden their horizons. In this regard, it is very cool to go to such conferences as DotNext. I myself love to watch reports on areas that I do not professionally deal with. In any case, I don’t have time to thoroughly study all areas of programming, but at least I will have a general idea and will be able to talk on relevant topics with other programmers. If I have to solve problems related to a new area for me, I will know which sides to start looking at and what to google.
Dmitry Ivanov
Go to the report of Sasha Goldstein :) Read his book, and then be sure to work on tasks that require optimization of performance, otherwise the brain simply discards the information as unnecessary.
Sasha Goldstein
I have two reports: about the tool for profiling and about the memory model. In the first, we will use PerfView. I have some code samples with performance-problems that we will analyze. PerfView is a frontend for ETW technology. About her, by the way, there is another report on DotNext. ETW is a technology with which you can collect logs. Many components of Windows, .NET, CLR, ASP.NET are written to this log. And write these logs with very high speed. PerfView is an effective analysis of ETW: CPU logs, execution time, database access, allocation profiling.
The second report is about memory models. Memory Model is a “scary” name for all memory operations in all high-level languages. If the program works in one thread, everything is simple, and if there are many threads, the situation changes. Writing and reading may not go in the order we specified; the compiler can optimize a part of the memory accesses in general. The same with processors. And when the port is made from Intel to ARM, which in all iPhones, androids, or Power PC, which in all cars, a lot of things stop working, because the processor allows itself more.
I am going to show several theoretical and practical examples when Intel is working correctly, but on an iPhone it breaks or, when switching from one thread to several, it breaks on Intel, although almost no one, looking at this code, can say that there is any that problem. After we analyze the subtleties of the Memory Model, we turn to examples of how to build the correct multithreaded code with the correct synchronization.
Andrey Akinshin
I will talk about arithmetic. Many times I have seen how programmers make annoying mistakes in simple arithmetic expressions, make bugs that are then searched for weeks (why does the matte part work incorrectly there?) - simply because they don’t understand how the numbers with floating point.
Even fewer people think about the performance of arithmetic and what simplest metamorphosis can be accomplished with a mathematical formula that would make it work faster.
For example, a CPU can parallelize commands that even run in one thread at the hardware level, but for good parallelization you need good code that you need to write correctly. If we have found a bottleneck and need to optimize the calculations, then it makes sense to apply additional knowledge and conjure a little over the code.
Nikita Tsukanov
I plan to talk about Docker as an element of development infrastructure and deployment. How to quickly and effectively deploy microservices and manage them. I will also tell you how to run Windows Server Core in Docker.
Dmitry Ivanov
Rider is a hybrid of Idea and ReSharper. We use a common reactive model. Idea exposes some parameters, R # responds to it and exposes the necessary data for drawing, and this process can be repeated several times.
The Rider Framework is a library for reactive interaction between Java and .NET and a model generator from DSL in target languages. DSL is written in Kotlin, because Kotlin allows for some groovy-style meta-programming, thanks to its syntactic features. The report will be live-demo.
I think my report will be of interest to those who want to bring their application out-of-process and / or are interested in reactive frameworks and the use of a reactive model for a desktop application.
If you have questions for the speakers , you want to get a dose of hard-core .NET and find out how and how many times the weather changes in St. Petersburg, we are waiting for you on June 3 at the DotNext 2016 Piter conference.
With new opportunities we face new challenges:
- How will the same code work under .NET Core and Mono, on Windows and Linux, in a docker container?
- Should I switch to .NET Core now and how to get the most out of the new platform?
- What are the prospects for Mono and Xamarin?
- What changes have occurred "under the hood". NET with the transition to Roslyn and. NET Core?
In just three weeks, at the DotNext conference in St. Petersburg, 20 speakers will give presentations on the present and future of the .NET platform, on optimizing performance and multithreading, on the internal structure of the .NET platform and the CLR, on profiling and debugging the .NET code.
In the meantime, we asked four of them to share their experiences and opinions about future changes in the world of .NET. Our questions were answered:
- The world's leading .NET platform performance expert, eight-time Microsoft MVP, the author of the excellent .NET performance book Pro .NET Performance, Sasha Goldstein ;
- The main developer of the protocol for reactive multiprocess communication in Rider Dmitry Ivanov from JetBrains;
- Another Rider developer from JetBrains, .NET MVP, Ph.D., silver winner of ACM ICPC, postdoc at the Weizmann Institute of Science Andrei Akinshin ;
- CTO Promarket and Nikita Tsukanov, an expert in Mono and Linux.
What has changed in .NET with the transition to Roslyn?
Sasha Goldstein
For Microsoft, Roslyn is a project of immense importance. Almost everyone in the C # team has been working on it for seven years, and probably they couldn’t release it for a very long time. And now he is on github in open-source and the whole process of language changes is before his eyes. On the one hand, many people came up with strange ideas to add constructions from other languages ​​to C #, on the other hand, the whole process takes place openly and in two hours you can answer any question to three people who write the compiler. For the community, it's very cool.
')
Andrey Akinshin
Overall I like Roslyn very much. As a compiler, it is better than the old one: it compiles many code sections more competently and more successfully in terms of performance. But if we are talking about compiling C # code, then I’m more interested in the next step: JIT compilation. Not so long ago, Microsoft introduced a new 64-bit JIT compiler called RyuJIT.
One of its main objectives is to reduce JIT compilation time. We give the user a very quick start, but because of this we can’t do cool optimizations, so the code itself runs slower than it could, and sometimes even slower than the old JIT. At the same time, RyuJIT is able to use SSE and AVX instructions well, which allows, with due diligence, to implement highly effective programs using the power of one C # without referring to native code.
In terms of performance, it seems to me, Microsoft is betting on another technology that is now actively developing: .NET Native. While it is still in a very raw state, but it looks promising.
Nikita Tsukanov
The main feature of Roslyn is that Microsoft can now add new features with acceptable speed. There used to be one compiler for the build, the other for the studio. Now this is not - there is only one compiler left and it is written in C #.
Dmitry Ivanov
From the point of view of the API, it has certainly become better, but there are problems with performance and excess memory, and they are far from being resolved. Now we cannot open huge projects in Visual Studio 2015 together with ReSharper, because there is not enough memory.
Sasha Goldstein
Mono moves to MIT license. So Microsoft will be able to copy pieces of Mono to CoreFx and use them. It seems to me that Mono should slowly fade away. Mono will have some niches on which .NET Core will not work yet. But Mono's runtime itself is so-so: GC only recently switched to Generational GC, JIT doesn't work very well. And the framework itself, which they wrote in many cases is not production-ready: there are a lot of bugs, bugs are detected and not repaired for a long time. Almost all my clients who used Mono in production faced problems. This does not mean that in the end the system did not work at all, but this is not the level of quality we are used to in .NET.
But with .NET Core, too, everything is not smooth. The project was supposed to end in January, then in March, just a few days ago, they said that it would be ready at the end of June. API, project model, half the code is constantly changing.
Last week, one of the clients with a project on ASP.NET MVC 6 asked how I thought it was necessary to repair the code every two weeks or it was better to backport to ASP.NET MVC 5 and wait. The question is not simple.
Andrey Akinshin
.NET Core is certainly a very interesting project, a big step forward, but today it is still raw. Let's not forget that many libraries have not yet come out under it or are in RC. Plus, Microsoft constantly renames something, every day everything changes, some bugs are repaired, others are added. For two years, a new project system was developed and promoted on the basis of project.json files, and just recently it was thrown into the dustbin and decided to return to the good old csproj. Such situations discourage a desire in the foreseeable future to try to transfer the production system to the .NET Core.
Mono has been in development for over 15 years. This is an adult, solid runtime, which runs a lot of production-code. For example, I am working on a project Rider (cross-platform .NET IDE) in JetBrains. Backend Rider (this is R # in its pure form) was launched under Mono, and under .NET Core it’s not really possible to launch. It works quite well: a healthy application from hundreds of projects and very nontrivial logic, with all the features of .NET and all this under * nix starts up normally. There are, of course, their own problems, but they are completely resolved in working order. And about the transition to .NET Core: you need to wait for this new cross-platform runtime to become more stable.
Nikita Tsukanov
Mono is a fairly mature environment, and .NET Core is still in beta and by the number of available libraries. NET Core is definitely losing. For example, in .NET Core there is currently no support for System.Drawing, and in Mono it appeared in the 2000th shaggy year.
Regarding the performance and quality of the .NET Framework coverage, Mono actively brings to itself those pieces that Microsoft opens. Given that Microsoft acquired Xamarin, the process should go even faster. At some point, Mono will become an add-on over .NET Core, but for now it makes sense to use Mono.
Dmitry Ivanov
We probably almost all have an opinion that Mono at some point will drop. At one time, Oracle also said that it would merge the HotSpot JVM with JRockit, but in the end it dropped the last one.
So, probably, it is worth waiting a bit, and switching from Mono to .NET Core. We in Rider work a lot with Mono and see that there are a lot of errors and the code is much lower quality than in the .NET Framework.
Andrey Akinshin
I believe that there is no silver bullet, there are no universal libraries that suit everyone. It is necessary to select a solution for your task. If you work with a database, there are databases that work faster than others on certain indicators; if you work with graphics, there are various good libraries for rendering 2D / 3D, but each is good for its own spectrum of tasks. It must be remembered that each project is unique in its own way. Most often, some solutions are suitable for some solutions, for others - others.
A very popular mistake: a person has heard that the ABC library is good, it takes and uses it, without thinking about what tasks it is intended for, on what scenarios it behaves quickly ...
An important point: many people bother with performance and spend a lot of time optimizing non-essential sections of code. There will be some speed increase from this, but no one can notice it. When we want to improve performance, we need to conduct research, do benchmarks, and profile.
Dmitry Ivanov
Even within one .NET-team, we have several solutions for interprocess communication, so yes, we have to often. Now, high-performance solutions usually mean some kind of server load.
If we talk about performance problems on one machine, it is also constantly being solved, because, let's say, we quickly go beyond the framework of the idiomatic C # code. Pretty quickly, you have to switch from a large number of references to arrays, so go to unsafe. Without this, it simply does not work. And we do our protocol in the Rider Framework with an eye on high performance.
Sasha Goldstein
Many developers find it difficult to imagine what exactly is happening in a multi-threaded program, that memory accesses can be moved, what to synchronize, on the other hand, if there is a lot of synchronization, bottlenecks appear. There are many tools for this, but you need to know and be able to use them.
Many developers do not have access to multiprocessor systems. When the same code is executed not on Core i7, but on the server, at least with 64 processors, completely different problems appear. Some small lock, which took 2% of the time, suddenly starts to occupy 50%, there are problems with the memory bandwidth (the memory system is not able to respond to requests quickly enough), not efficient use of the cache. .NET-developers most often do not even face such tasks.
Andrey Akinshin
I think that the complexity of multithreaded programming on .NET is the same as on any other platform. You need to understand well how a multi-threaded world works. As for the synchronization primitives and concurrent data structures, everything is fine with .NET. Plus in C # 5 appeared async / await, which allow you to write multi-threaded code in a beautiful and understandable.
Nikita Tsukanov
There is no way to prevent one stream from being able to access data that is in the possession of another. In Rust, for this purpose, a more or less normal system was made (destructive assignment). In .NET there is no such thing and without it is very hard: it may happen that when we transfer something from one stream to another, due to carelessness or other reasons, something superfluous may pass to another stream. And with “this superfluous” they can start working, although the stream owner knows nothing about this. In this case, competitive write requests occur where no one knows. As the project grows, it becomes more and more difficult to track and deal with it.
Dmitry Ivanov
Good question. I have been programming multithreaded for so many years and telling people how to do it, but I haven't figured it out myself. Probably, the main challenge arises at the top level - to come up with a model that is successfully suited for a large development team and a large product and get everyone to follow it and hope that you will never rest against the limitations of this model.
The Roslyn team decided to use the transactional model of multi-threaded interaction, and they came up against Memory Wall. We are faced with pessimistic locks and a read / write lock-model facing a long time not releasing the read lock and not a smooth taiping.
There is no silver bullet; even new-minded actors do not save: Hello World works well for them, but in reality everything is more complicated.
Sasha Goldstein
Do not save, you need. The only question is how much is needed. Do I have to be able to read an assembler, understand the memory model, how does the processor work? Standard tip - understand at least one level below the one you work at. If you are working on .NET you should understand how the OS works, Runtime, GC, Framework. Without it, even if you find a problem, you will not know how to fix it.
Andrey Akinshin
Depends on what tasks the programmer solves. 90% of the time, until there is a task to squeeze the maximum out of the machine, the programmer should think about the code to be reliable, readable, without bugs. In the remaining 10% of cases, when problems have arisen (or we guess that problems will clearly arise), and we try to solve them, then yes, you need to know how everything happens inside.
Many people try to solve performance problems without understanding what is happening in this life: they do wrong benchmarks, crookedly profile, write strange code, which only slows down everything and spend a lot of working time on it. To successfully cope with these 10% performance-critical cases, you need to know a lot about how things work inside.
Nikita Tsukanov
The problem of abstractions is that they have a tendency to flow, so the team needs one person who understands how everything works inside. Roughly with a man who knows math well. Traditional low-level problems are memory leaks, exceptions in the native code, heap corruption.
Dmitry Ivanov
In my opinion, any .NET-engineer, should at least understand how the processor and processor caches work, be a little bit aware of how branch prediction works and understand why sequential access to an array and a disk always works faster than parallel.
Sasha Goldstein
No need to guess. You need to run the tool and get information about what is happening in the process. Look at the memory, CPU, how many threads, what they do. Now there are a lot of free utilities. After that there are a bunch of methods. My favorite is USE (utilization, saturation, errors). He came up with Brandon Greg (Brendan Gregg) - an expert on performance in Linux. Utilization means that we have to determine for each resource (CPU, memory, disk ...) how much it is used. Saturation means that for all these resources we must determine if there is an over subscription, for example, whether we read 30 files from one disk at a time. Errors - the number of errors in the program. After that, you can optimize something: we will add processors or code to improve. This is in general.
Andrey Akinshin
The first thing you need to do is poprofilirti and understand what exactly slows down. A very common mistake is optimization without profiling. After we have found a bottleneck, the second step is to ask yourself the question: “why does this section of the code slow down”? Here, sometimes you need to have really a lot of knowledge: you need to guess what factors are really important and check only them, otherwise the programmer risks spending a lot of time on the unnecessary work. And the third step is to fix the problem and carefully check that the program really works faster now (and also that during the optimization nothing broke off).
Dmitry Ivanov
The classics recommend taking a profiler, find the bottlenecks and fix them. But when you have already done this several times, there are no more open bottlenecks and the program cannot be accelerated several times, having fixed a couple of lines. Sometimes in these cases it is necessary to reconsider the approach, for example, to insert by code Asserts, which make sure that the operations are completed in the allotted time. And to force each member of the team to correct performance-asserts falling in his area of ​​responsibility. It is also useful to look at memory traffic and start a fight with it. Maybe even go to unsafe.
Sasha Goldstein
On Windows, you need to start with Performance Counters. After that depends on the situation. A lot of problems can be solved only with PerfView, but it has big problems with visualization. Visual Studio Profiler is not bad at all, by the way, it has a standalone version. dotMemory or .NET Memory Profiler– for memory profiling.
Andrey Akinshin
I am the maintainer of the BenchmarkDotNet utility. The latest versions of this library already behave quite consistently and provide ample opportunities for benchmarking. Alas, many people from the Internet are constantly trying to make their own self-made benchmarks based on Stopwatch and a couple of cycles. At the same time, they are surrounded at every step by problems that they are unaware of and which are capable of completely spoiling the conclusions from the experiment being conducted. But this is only part of the trouble: after all, you can miraculously pass by all the rakes and in this particular situation write the correct code, which really measures something useful. But only every second novice performance engineer launches his benchmarks in DEBUG, because this configuration is selected by default in the studio.
Whether it is worth explaining that it can completely spoil the carried-out measurements. BenchmarkDotNet scolds the user for the debug configuration and the debugger attached in large red letters, complies with the general benchmarking methodology, makes a warm-up, runs each method in different processes several times, allows you to compare different environments, counts the statistics, etc. A community has now gathered around the project, and the library is becoming a convenient and popular tool for a number of performance studies.
As a profiling, I can recommend the products of the company JetBrains: dotMemory and dotTrace. I often use them, and I really, really like them.
Dmitry Ivanov
All our tools: dotTrace, dotMemory. Almost for everything is enough. Sometimes you need to look at sys internals utilities.
Sasha Goldstein
There are a lot of materials online, there are my courses on pluralsight, quite a lot of information can be found in the old blogs of Microsoft developers (Rico Mariani, Chris Brue, Vance Morisson). About books: there is my book Pro .NET Performance , there is a newer book - Ben Watson “High Performance. NET Applications” . Also quite interesting written. Plus, you need to know something and understand about the device .NET. There is a classic book CLR via C # . It is a bit out of date, but some points are simply not mentioned anywhere else, for example, how exceptions and delegates work.
Andrey Akinshin
I would go from top to bottom and would not go to the CPU level (as some try to do), if there is no basic knowledge of algorithms, data structures and the platform with which you work. In .NET you can read the same Richter, to understand how the base classes work and what they are good for. It is extremely important to understand the device runtime: how does GC work, how do classes differ from structures, what does the JIT compiler look like, etc.
Further, it would be good to understand how modern iron works, in particular, Intel processors (at least in general terms). In this regard, there are many resources on the Internet, but this is often not very simple reading: you cannot read a book in the evening and figure out the details of the CPU device.
However, a person who does not specialize in performance, but who works in an enterprise does not need this. I advise such people to pump the general erudition and broaden their horizons. In this regard, it is very cool to go to such conferences as DotNext. I myself love to watch reports on areas that I do not professionally deal with. In any case, I don’t have time to thoroughly study all areas of programming, but at least I will have a general idea and will be able to talk on relevant topics with other programmers. If I have to solve problems related to a new area for me, I will know which sides to start looking at and what to google.
Dmitry Ivanov
Go to the report of Sasha Goldstein :) Read his book, and then be sure to work on tasks that require optimization of performance, otherwise the brain simply discards the information as unnecessary.
Sasha Goldstein
I have two reports: about the tool for profiling and about the memory model. In the first, we will use PerfView. I have some code samples with performance-problems that we will analyze. PerfView is a frontend for ETW technology. About her, by the way, there is another report on DotNext. ETW is a technology with which you can collect logs. Many components of Windows, .NET, CLR, ASP.NET are written to this log. And write these logs with very high speed. PerfView is an effective analysis of ETW: CPU logs, execution time, database access, allocation profiling.
The second report is about memory models. Memory Model is a “scary” name for all memory operations in all high-level languages. If the program works in one thread, everything is simple, and if there are many threads, the situation changes. Writing and reading may not go in the order we specified; the compiler can optimize a part of the memory accesses in general. The same with processors. And when the port is made from Intel to ARM, which in all iPhones, androids, or Power PC, which in all cars, a lot of things stop working, because the processor allows itself more.
I am going to show several theoretical and practical examples when Intel is working correctly, but on an iPhone it breaks or, when switching from one thread to several, it breaks on Intel, although almost no one, looking at this code, can say that there is any that problem. After we analyze the subtleties of the Memory Model, we turn to examples of how to build the correct multithreaded code with the correct synchronization.
Andrey Akinshin
I will talk about arithmetic. Many times I have seen how programmers make annoying mistakes in simple arithmetic expressions, make bugs that are then searched for weeks (why does the matte part work incorrectly there?) - simply because they don’t understand how the numbers with floating point.
Even fewer people think about the performance of arithmetic and what simplest metamorphosis can be accomplished with a mathematical formula that would make it work faster.
For example, a CPU can parallelize commands that even run in one thread at the hardware level, but for good parallelization you need good code that you need to write correctly. If we have found a bottleneck and need to optimize the calculations, then it makes sense to apply additional knowledge and conjure a little over the code.
Nikita Tsukanov
I plan to talk about Docker as an element of development infrastructure and deployment. How to quickly and effectively deploy microservices and manage them. I will also tell you how to run Windows Server Core in Docker.
Dmitry Ivanov
Rider is a hybrid of Idea and ReSharper. We use a common reactive model. Idea exposes some parameters, R # responds to it and exposes the necessary data for drawing, and this process can be repeated several times.
The Rider Framework is a library for reactive interaction between Java and .NET and a model generator from DSL in target languages. DSL is written in Kotlin, because Kotlin allows for some groovy-style meta-programming, thanks to its syntactic features. The report will be live-demo.
I think my report will be of interest to those who want to bring their application out-of-process and / or are interested in reactive frameworks and the use of a reactive model for a desktop application.
If you have questions for the speakers , you want to get a dose of hard-core .NET and find out how and how many times the weather changes in St. Petersburg, we are waiting for you on June 3 at the DotNext 2016 Piter conference.
Source: https://habr.com/ru/post/283516/
All Articles