Event model building projects and solutions of Visual Studio for developers

This short article will help:

To get acquainted with the event model of building projects and solutions of MS Visual Studio;
Understand how to get Command-Line support for devenv.exe for VSPackage (where it was not originally provided);
Understand how to emulate a similar event model from MSBuild Tools and broadcast to the main plugin;
Learn how to work on priority subscription;
Find out how to get the build context when handling Visual Studio / MSBuild Tools events;
Learn about MSBuild Property & MSBuild Property Functions;
Get general information about intermodular interaction on the layer of abstraction for heterogeneous components of the system.

Synopsis

I quite often have to engage in the automation of various processes, so it’s not surprising that some of the solutions sooner or later touched Visual Studio.

In fact, this article, or even a note, is the result of a working and long-written plug-in, which only 2 years ago was just a by-product when working on a single C ++ project. However, my debut on Habrahabr will, perhaps, from this.

Recently I was approached with similar needs (the ones that the plug-in was originally intended to solve) a DevDiv guy. In an attempt to solve his problem of highly custom automation, it became clear that it was still extremely necessary to highlight some aspects of the solution and the interaction between VS & MSBuild. After all, this material was not originally there, and it seems to be still not publicly available.
')

A warning

Before you begin, please note that the material is not for Visual Studio users, but for developers (suggesting an extension of Visual Studio itself). And it is intended primarily to highlight some aspects of solutions and interactions for these components, if someone is faced with similar tasks for their own development. This is not a step by step guide, but it is meant to help you understand the minimum principles and patterns of work. For example, interaction with devenv.exe, msbuild.exe, work with priority subscription, and all that is indicated in the list above.

About problems

The initial need, for both of us, was and is to work with the Solution-Context as part of an event-driven studio model.
Those who worked with Visual Studio, of course, need to know about Pre / Post-build project-level events. However, they do not fully satisfy the needs of project management for the entire solution, especially when there are several dozen of these projects, etc., which is typical primarily for C ++ projects.

There are several ways to solve this problem. For example, the above mentioned, is now making the transition from the decision, when the Solution has a project on which all others depend.

So why MS still does not allocate a similar context for its IDE?

In fact, everything is the same as with the .sln format, which still represents hunter’s notes, rather than a valid xml document with project files, or something more flexible and elegant.
Compatibility? Probably, however, it was necessary to break it with the advent of major changes in VS2010.

So with the solution-context. Just to raise events to the upper level will not work, because you need to solve the problem of managing msbuild data and much more for one single context, and no one is in a hurry to add it to your downloaded timeline.

Event handling of projects and solutions of MS Visual Studio (VS2010, VS2012, VS2013, VS2015)

To begin, let's look at the EnvDTE version. For our task, the gaze is primarily focused on BuildEvents , which provides an affordable subscription to basic events, such as OnBuildBegin , OnBuildProjConfigBegin, and the like. That is, they are implemented in the form of public events.

However, no , in our case they will work too late when the alert to all of their DTE listeners goes. We need to get control over the situation as soon as possible in our package.

Fortunately, in VS there is a priority layer for processing such things, they will be executed first of all with all such things.

In general, this is the same classic observer, implemented through Advise methods for specific services. But first things first.

First of all, we need to pay attention to Microsoft.VisualStudio.Shell.Interop . He will help to work with the basic "build-events" VS and other level of the solution, namely IVsUpdateSolutionEvents :

To work with the above-mentioned needs, it suffices to consider only the basic IVsUpdateSolutionEvents2 , which is available up to VS2010. IVsUpdateSolutionEvents4 is actually desirable for working with the build context, however it is available only for VS2012 and later.

Note : IVsSolutionEvents are also likely to be required for full-fledged work, but this point is not covered in this article. If you still need someone, I can show basic techniques for maintaining this layer.

So , we need to implement the following basic things of the IVsUpdateSolutionEvents2 interface:

//         build-action //       int UpdateSolution_Begin(ref int pfCancelUpdate);

 //      .. Cancel / Abort -     int UpdateSolution_Cancel();

 //      . //  ,        ,    Cancel/Abort, ..: // * Begin -> Done // * Begin -> Cancel -> Done int UpdateSolution_Done(int fSucceeded, int fModified, int fCancelCommand);

All these methods are primarily suitable for the implementation of the solution-context level.
For projects we have provided:

 int UpdateProjectCfg_Begin(IVsHierarchy pHierProj, IVsCfg pCfgProj, IVsCfg pCfgSln, uint dwAction, ref int pfCancel); int UpdateProjectCfg_Done(IVsHierarchy pHierProj, IVsCfg pCfgProj, IVsCfg pCfgSln, uint dwAction, int fSuccess, int fCancel);

Which will be called for each, the configuration of which should go to build. Note the pHierProj argument; this is a flexible way to refer to wherever the control came from.

Basic implementation of your package does not mean that VS will be ready to work with IVsUpdateSolutionEvents . As mentioned above, we need to register our listener (who implements this interface):

 public sealed class vsSolutionBuildEventPackage: Package, IVsSolutionEvents, IVsUpdateSolutionEvents2 { ... public int UpdateSolution_Begin(ref int pfCancelUpdate) { return VSConstants.S_OK; } ... }

This must be done with the Advise methods, so for IVsUpdateSolutionEvents2 is provided - AdviseUpdateSolutionEvents .

Registration example:

 /// http://msdn.microsoft.com/en-us/library/microsoft.visualstudio.shell.interop.ivssolutionbuildmanager2.aspx private IVsSolutionBuildManager2 sbm; /// IVsSolutionBuildManager2 / IVsSolutionBuildManager /// http://msdn.microsoft.com/en-us/library/bb141335.aspx private uint _sbmCookie; ... sbm = (IVsSolutionBuildManager2)ServiceProvider.GlobalProvider.GetService(typeof(SVsSolutionBuildManager)); sbm.AdviseUpdateSolutionEvents(this, out _sbmCookie);

It should be noted that access to the SVsSolutionBuildManager service is possible in other ways. Use what is convenient.

The sbm, in the example above, should be part of the class to be protected from the GC.

Now we are ready to listen in the forefront. It is also important to understand that we are not the very first, but we don’t need it.

Support Command-Line mode

No wonder that someone has a need to work with devenv.exe (or rather devenv.com, since it is he who processes in the console mode) with our plugin. However, VSPackages does not have the ability to work somehow in this mode! Accordingly, the plugin will simply remain inactive if we try to build a solution in the console as

 devenv "D:\App1\App1.sln" /Rebuild Debug

I first asked this question in the Q / A gallery and, to be honest, I didn’t have such needs or even desires before (since you can work with msbuild.exe and, besides, automation servers will still provide limited environment without all that).

However, as we see, the needs of each of us are different, especially if this is a part of VS, then there is a disorder, we must support it.

Later, as part of supporting CI servers, I took up a similar issue with a successful solution . That is, we can process the entire event model of projects and solutions in VSPackage ! Who would not say:

 "C:\Program Files (x86)\Microsoft Visual Studio 12.0\Common7\IDE\devenv" "D:\App1.sln" /Rebuild Debug "C:\Program Files (x86)\Microsoft Visual Studio 12.0\Common7\IDE\devenv" "D:\App1.sln" verbosity:diagnostic /Build Release

[?]
But…

The solution involves using Add-Ins wrappers to broadcast events;
It is the Add-Ins that provide the command-line mode, or rather, VS is ready to work with it in this mode;
They also have exactly the same model, so we have nothing easier to simply implement the good old IVsUpdateSolutionEvents2 and register as for VSPackage at the point:

 void OnConnection(object application, ext_ConnectMode connectMode, object addInInst, ref Array custom);

And redirect to the main library, that is, for example:

 ... public int UpdateSolution_Begin(ref int pfCancelUpdate) { return library.Event.onPre(ref pfCancelUpdate); } public int UpdateProjectCfg_Begin(IVsHierarchy pHierProj, IVsCfg pCfgProj, IVsCfg pCfgSln, uint dwAction, ref int pfCancel) { return library.Event.onProjectPre(pHierProj, pCfgProj, pCfgSln, dwAction, ref pfCancel); } ...

But, as everyone has long known, the Add-ins are deprecated in Visual Studio 2013 . That is, such a trick will work for VS2010, VS2012 and VS2013. For the planned VS2015 such games will not work.

I already wrote about this on MS Connect Issue # 1075033 , but you can say goodbye to those who thought it was important. VS2015 is already in RC, and the problem simply closed.

Go ahead.

Emulation of a similar event model from MSBuild tools

Let's start with the fact that MSBuild tools is definitely a more powerful tool and no one should know the above mentioned problems. We can safely handle $ (Configuration) & $ (Platform) level solution, we can work with targets, etc. However, the solution should be the same and we should not notice the difference in work between VS IDE and CI / Special Build Servers builds. Therefore, we consider the possibility of working with the above events within the msbuild tools.

Neither DTE2-context, nor event registration services are available to us, nothing that can communicate with VS, but what else are you waiting for ? Yes, of course, we can get, for example, the same DTE2-context c GetActiveObject , which in turn uses:

 HRESULT GetActiveObject( _In_ REFCLSID rclsid, _Reserved_ void *pvReserved, _Out_ IUnknown **ppunk );

[?]
Those. like this:

 (EnvDTE80.DTE2)System.Runtime.InteropServices.Marshal.GetActiveObject("VisualStudio.DTE.10.0");

But all this will only work if there is a running instance of an IDE studio, which is not possible for limited environments, for example, for CI.

Therefore, I suggest getting control of msbuild.exe by registering the developed translator as a logger. To do this, we need to work with Microsoft.Build.Framework .

Indeed, basic IEventSource is able to provide all of our basic needs:

 public class EventManager: Logger { ... public override void Initialize(IEventSource evt) { ... } }

However, there are a number of features that you need to know.

IEventSource.BuildStarted will not work (see note below), because it works too early, more precisely, in order to get the data of the project that needs to be processed, we will have to wait. In VS, the input context is processed with IVsSolutionEvents;
Therefore, for PRE events, it is possible to use IEventSource.ProjectStarted , and it, in turn, should be used for project-level events.

The fact is that the handler - ProjectStartedEventHandler also sends ProjectStartedEventArgs as an argument and an .sln file for working with it, we can track, for example, like this:

 evt.ProjectStarted += (object sender, ProjectStartedEventArgs e) { e.ProjectFile; // should be .sln ! ... };

The option can be with waiting until we get .sln for processing, and it in turn must be before passing the project files.

The torment on this is just beginning, because For full-fledged work, you will most likely need .sln data, including loading projects to get access to the msbuild engine anywhere else, for example, evaluating new St.-in msbuild, interrupting assembly, etc.

To work with .sln files, alas, you will not see anything normal, or rather choose your own taste:

Either use outdated ie. It is marked as obsolete Microsoft.Build.BuildEngine.Project . In this case, take into account that if you do not use Microsoft.Build.BuildEngine, then this is add. unnecessary reference for your project.
Or use reflection on internal methods - Microsoft.Build. BuildEngine.Shared , for example:
-> void ParseProject (string firstLine)
-> void ParseFirstProjectLine (string firstLine, ProjectInSolution proj)
-> crackProjectLine -> PROJECTNAME + RELATIVEPATH
Or write your own small parser following the example of the same ParseProject.

The choice is not rich, however .sln in an incomprehensible form for a long time ...

Note:

The phrase above “IEventSource.BuildStarted will not work” is not entirely true ... It does work too early to get the necessary data, but it’s a double-edged sword. Here , I described a problem that can be encountered with a similar implementation. So for a multithreaded assembly / m: 2+ (2 or more concurrent processes to use) this can cause quite unpleasant consequences, and a special case of CSC error (CS2001) , when all the goals were defined by the msbuild collector before processing our first events, and The corresponding modifications at runtime could not affect the new ones in any way. Although this coverage is only part of the tasks involved in the construction process, etc.

Therefore, I recommend the most flexible way to use IEventSource.BuildStarted with the preliminary preparation of all the necessary data. How to do this, I described in this material , there you can also find a simple sln parser under MIT or see the parser in the main library.

Go ahead.
The project may need to evaluate the properties and functions of msbuild (MSBuild Property Functions), etc., but what about without them ...

If you need this, in order to work with this, you need to prepare the msbuild engine, and you must first initiate it. Since we are working with an isolated environment, it is also necessary to transfer the properties with which it was initiated from msbuild.exe.

If you are using .NET 4.0, you will have to do it manually, i.e. you need to define a Configuration, Platform, SolutionDir, etc. for the handler. But for the .NET 4.5 platform ProjectStartedEventArgs.GlobalProperties is available.

After the isolated environment is fully initialized, we can finally handle project events. However, we work with msbuild and we work with targets. So you need to subscribe to TargetStarted .

For the broadcast, we will be given PreBuildEvent and PostBuildEvent from the received TargetName, for example:

 protected void onTargetStarted(object sender, TargetStartedEventArgs e) { switch(e.TargetName) { case "PreBuildEvent": { ... return; } } }

But here, not everything goes smoothly. The fact is that we cannot refer to a project that is being processed with TargetStartedEventArgs, however we can get a BuildEventContext , from which we can get ProjectInstanceId , and it in turn exists for ProjectStarted, i.e. we can simply remember the ProjectId and then refer to it wherever a BuildEventContext is available, for example, simply:

 projects[e.ProjectId] = new Project() { Name = properties["ProjectName"], File = e.ProjectFile, Properties = properties };

Actually, now you can fully translate the event model in VSPackage with an isolated environment and work as it is:

 "C:\Program Files (x86)\MSBuild\12.0\bin\msbuild.exe" "app.sln" /l:"<>.dll" /m:12 /t:Rebuild /p:Configuration=<cfg> /p:Platform=<platform>

Get build context

In fact, there are a number of problems or features that I also wanted to highlight. But in general, this is too specific for our project, and for your case you may not need anything like that at all, so you can skip it .

Since our plugin works with build events, it’s not hard to guess what might be required to get the build type with which it all started. But for VS this is not so simple .

You already know and understand that we are working on a priority subscription, and we will not be able to use something like:

 _buildEvents.OnBuildBegin += (vsBuildScope Scope, vsBuildAction Action) => { buildType = (BuildType)Action; };

Because it is too late.

You can try to work with IVsUpdateSolutionEvents4 , however, as I wrote in the question and answer, only if you do not plan to be compatible with older versions.

At that time, the choice was to intercept commands from VS IDE, for example:

 _cmdEvents.BeforeExecute += (string guid, int id, object customIn, object customOut, ref bool cancelDefault) => { if(GuidList.VSStd97CmdID == guid || GuidList.VSStd2KCmdID == guid) { _c.updateContext((BuildType)id); } };

Not really like it, it's better to use IVsUpdateSolutionEvents4 , but the only working version is for compatibility of versions and preservation of priority processing, which I could guess at that moment. And no one is in a hurry to please with other solutions.

Intermodular interaction

Well, in fact there will not be anything remarkable, everything is ordinary, however, there is a short description in the article.

As you have already noticed, the components are versatile and even different in their model of event handling. Using my plugin as an example, it looks like this:

A sheet of what is shown can be found here .

Highlights for your work:

We select a common public interface at the project level, so that all the necessary components can provide an implementation on it. So does Shell.Interop, so does EnvDTE, etc .;
We determine where the main core of the program will be located. The one who will be responsible for the uniform data processing;
We define somewhere library loader. This may also be the case with our external Provider, etc .;
Each of the components, respectively, independently determines how it should connect to where it will be used, and in general, it should deal only with sending and transmitting, if necessary, the processed data. (I suppose you don’t need to give an example of how to work with external libs; you can find either in my source code or anywhere else, this is beyond the scope of the article).

There is an important remark. Our plugin was not originally intended to be used in this volume, so the core of the system is located in the VSPackage.

Yes, it is convenient not to divide into separate components, but why is it bad? The fact is that my VSPackage in the diagram above, you have to pull heavy dependencies on EnvDTE & EnvDTE80 (which, in turn, have even more exotic connections). All this can, and most likely will be absent on servers of continuous integration, since first comes standard msbuild tools, and more. That is, do not forget about light self-sufficient components and transparent interfaces. I'm not in a hurry with the separation, because all this time, and it’s not worth it yet, because it was originally a VSPackage solution, something like that.

Access and rating MSBuild Properties & MSBuild Property Functions

The last point I want to consider is the evaluation of properties. In general, access to properties can occur in several ways, for example:

With Microsoft.Build.BuildEngine
Either with Microsoft.Build.Evaluation

And also others.

The most flexible option is to use Microsoft.Build.Evaluation, since here is the easiest way to get an estimate using the msbuild engine and work with projects, for example:

 public virtual string evaluate(string unevaluated, string projectName = null) { ... lock(_lock) { try { ... project.SetProperty(container, Scripts.Tokens.characters(_wrapProperty(ref unevaluated))); return project.GetProperty(container).EvaluatedValue; } finally { project.RemoveProperty(project.GetProperty(container)); } } }

However, as it was already possible to understand, for a solution-context you will have to decide how to work with data from projects. I personally implemented the add. syntax extension .

That is, you still have to pre-parse with your analyzer, and then send the prepared data that the original msbuild engine can already handle .

Conclusion

Actually, this is the first article on Habrahabr. I hope not the last -_-.

The material covers key points to understand what needs to be done, what solutions there are, and what problems and peculiarities you will have to deal with working with the event model of constructions in VS, as well as its related components devenv & MSBuild tools, using the example of a real working solution .

Implementation details can be found in the source, as well as get extra. information in the comments later if you have questions.

Write about errors and inaccuracies, correct, if they will ... it was very inconvenient without a markdown .

upd.

In my attempts to convey to commentators that:
The material describes - how to work with these elements, as well as solving problems when using them in the described circumstances. In order for the VS developer to be able to familiarize himself with the existing solutions, and to apply the knowledge gained during development - anything and for anything of his own ... alas, they were not crowned with success .

We discuss exclusively - ways to solve problems with the help of these developments for the end user, and also generally try to solve abstract problems that were not presented or described in detail - we decide to solve it without knowing the subject area.

In view of the exclusivity of the formed audience, to continue to comment in this article, I will not present it appropriate.
However , for those who still need help on the topic, I can probably consult as much as I have free time. Contacts in G +.

upd2. Targets & Map of projects.

A small bonus, while I support the material, for those who want, but can not use IVsUpdateSolutionEvents2 / IVsUpdateSolutionEvents due to various circumstances.

Discussing the problem of the 'solution level' events (for simplicity, we continue to refer to this wording) with the person who contacted me, I also considered the possibility of working with targets instead of implementing the specified interfaces, since it was clearly an extra link for him to get rid of. Well, we are trying to achieve some equivalent.

Besides, if you just like something like that - MSBuild: Extending the solution build , (but this option can only work when building from msbuild.exe and not from VS IDE ...)

VS IDE does not consider the .sln file at all when building it. It forms final targets from its loaded environment with EnvDTE, etc.
The .sln file is considered only msbuild.exe - i.e. Before building, it automatically generates .metaproj (in memory, by default), which contains what and when it will be built, including common targets for all projects if it exists. For example:

 ... <Import Project="$(MSBuildExtensionsPath)\$(MSBuildToolsVersion)\SolutionFile\ImportAfter\*" Condition="'$(ImportByWildcardBeforeSolution)' != 'false' and exists('$(MSBuildExtensionsPath)\$(MSBuildToolsVersion)\SolutionFile\ImportAfter')" /> <Import Project="D:\tmp\p\after.Sample.sln.targets" Condition="exists('D:\tmp\p\after.Sample.sln.targets')" /> <Target Name="Build" /> <Target Name="Rebuild" /> <Target Name="Clean" /> <Target Name="Publish" />

At the same time. Metaproj is not considered VS IDE, since he has his own formed goals.
And also we don’t know what is happening and when. projects are built all separately with pre-defined goals VS.

those. to work with common targets for build operations from VS IDE, we can only use project files (meaning without modification / VS extension) with some restrictions.

We consider, respectively, the situation of a general solution, or when we may not know at all about the projects that the end user will have in solution.

As a limitation, you can consider a project map (there are at least 2 options, but only this one showed itself to be more or less stable, therefore the bonus is only about it):

gist.github.com/3F/a77129e3978841241927

 ... <Target Name="_Build" BeforeTargets="Build" DependsOnTargets="ProjectsMap"> <CallTarget Targets="_BuildPRE" Condition="$(ScopeDetectFirst)" /> <CallTarget Targets="_BuildPOST" Condition="$(ScopeDetectLast)" /> </Target> <Target Name="_BuildPRE"> <!-- ... --> </Target> <Target Name="_BuildPOST"> <!-- ... --> </Target> ...

In general, when forming a similar map, we now know what and when it should be (formed with ProjectsMap, the detector with ScopeDetectFirst and ScopeDetectLast respectively) , ie:

Just automatically add (for example with NuGet events , etc.) your common .targets file to all projects, for example:
```
 <Import Project="..\<SolutionFile>.targets" /> 
```
Further, with the restriction in the form of a project map, we can now achieve the following implementation (for Clean, Build, Rebuild, etc. goals) :
- “Before the start of the first project”
- "After the end of the last project"

It is safe for all or most cases (changing the build order or deleting some projects from the solution, everything should work fine). But! This option has a number of inconveniences in the form of import service at the initialization stage, as well as when adding new projects.This is inconvenient, but an option for IVsUpdateSolutionEvents.

In addition, I noted a potential problem with Unload projects (when the IDE allows the user to temporarily unload the project from the solution) in the title - however, in most cases this will be a valid option, since this unload is temporary, and the error is valid in some form, so it can be allowed (besides, the problem can still be solved if desired).

Regarding convenience, a controversial point ... in your VSPackage products can also be an extra link, and as an option you can consider something like that ...
However , with regard to reliability, I would still choose IVsUpdateSolutionEvents with VSPackage! because Such a minimum is standardized, and accordingly predictable on different versions.

upd3.

msbuild , , CSC error (CS2001) .
. MSBuild Plugin, . , . CodeProject . You will learn:
- , + .
- MSBuild Tool , ..
, EnvDTE.CommandEvents , . SO .