Geekly Articles each Day
📜 ⬆️ ⬇️

The history of one configuration "umbrella"


In this article I will talk about the experience of supplying and configuring numerous interconnected products (non-participants call it simply “the creation of installers”).

So imagine a set of applications scattered across several servers: 12 pieces. Applications are heterogeneous, developed by different companies at different times. Various integration methods have been implemented, from primitive exports / imports through a file system to advanced web-based services. Applications use SQL Server and Oracle databases. All of these applications work in hundreds of different end-user environments.

Installing and configuring all this costs a lot of work and resources. Ok, set. And updates that should be issued for accounting systems in accordance with changes in legislation? Scaling applications? And the new functionality? What about new customers with their “special” configurations? Manually is expensive and time consuming if you have, say, 150 clients and at least 2 updates of such a “monster” per year.

Our answer to all these questions is the creation of an installation “umbrella” - a bootstrapper and combing everything under it. Many customers themselves can now install or upgrade without our help and additional payment for the services of the company's consultants. The number of customers is growing, and the amount of work for support is decreasing.
')
Here, in fact, about our "umbrella". Is it a "bicycle"? Yes, to some extent. However, clearly with electric drive and built-in GPS navigation. Although, perhaps, not the latest version.



Is this the perfect solution? No, but it works and brings real benefits from its implementation. Current products cannot be efficiently delivered without it.

And, sorry, "umbrella". Then that's it :)



Why is all this necessary?


The packaging of all these applications has long been engaged in a dedicated team using InstallShield. It was necessary to deliver installation packages in various configurations, customizing them for specific requirements and client environments, and supporting various scaling options.



As a result, it happened and so ...

Comes, it means, the task of packaging. The first question is right there:
- And what is the deadline for this task?
Answer:
- The day before yesterday!

At once all the architectural delights evaporate from the head in such a situation. Not? That is, everything should already be ready. As a surgeon for an emergency operation - the tools on the table and in stock. In this case, the “bad” pekedzhing, as the last link, is to blame for disrupting the deadlines. Sometimes it’s just like a grenade without checks — tell the next one.

The business value of peking, so to say, is not in the first place - “somehow wrap up, so that it works and that's all.” And it is not at all obvious that, due to the lack of any new feature, well, maybe a couple of customers will suffer, and without proper packaging, they will not get everything. Well, if the installer spoils the environment with a huge number of clients?

As a result, each of the application development projects lived its own life and something integrated grew only in the process of packaging the entire solution.

What did not suit us?
  1. Speed ​​and predictability of delivery times. Regardless of how qualified and great the team is, it’s impossible to rewrite half of the application in a week.
  2. Large budget peking did not suit the customer.
  3. The need for costly licensing and specialization of employees in this area.
  4. The process of developing installers was not sufficiently transparent and understandable to the customer.

What happened and how does it work?

Decision itself


From the user's point of view, this is a folder with executable files of the “engine” of an umbrella, several XML “matrices”, utilities for configuration and diagnostics, among them the famous setup.exe.

This is the hierarchy of the directories supplied to the end user of the product.


User view


The user runs the famous setup.exe - our product. Displays the user interface in the corporate style of the company. There is a choice of installation kit. But, in fact, where our matrix works, the system calculates the dependencies by itself: if another product is required to install a product, it adds it as mandatory.

In accordance with the choice of products, the user is asked to enter one or another number of installation parameters in the user interface. Moreover, if a group of packages requires the same parameter, all this is automatically grouped into one UI element. If the user has not selected a product for this setting, it is also not displayed.

During the installation, we generate a batch file with parameterization of each package in ProgramData at the entrance, then after installation we get 2 logs - one from the umbrella in XML, the second from each installer. And in the first there are links for easy diagnosis. There is also a special utility for viewing.

And if the package fell off? Copy-paste into a separate .cmd from the generated command file and comfortable debugging directly on the environment without launching the “umbrella”, but with its parameterization. It is very convenient to cling to the process of .NET Custom Action for debugging. Paradise versus VBS. Especially since our framework when changing the system environment variable all messages from the Custom Actions MSI issues in the message box, cling on - I don't want to. ;)

And immediately the question "What about the upgrade?"
We write to the registry, and in a single system area for our products. So we define the parameters of the installed applications, we migrate the settings from there, if the manual change of parameters is not supported. Otherwise we read from configs or merzhim them.

We use MSI only in Major Upgrade mode. Patches did not take root, although this possibility exists in the system. However, now the size of the package has not become so critical, and with its competent development and testing - a full upgrade will not cause problems either.

From the developer’s side


Now the practice is such that we start peking as early as possible and automate it immediately. Even if the new application has only one page, you can already install it!

The process is not too complicated, described in several points below.
  1. The developer enters the portal, downloads the toolkit, according to the documentation, doesn’t he forget what. Fills in the specification of pekedzhingu, if he does not himself.
  2. It is convinced that the project can collect and all the files in place.
  3. Configures the toolkit in accordance with the required configuration (for example, depending on the type of application - client, web server or just a database) and generates an installer template.
  4. Assembles the installer, tests, automates in accordance with the examples.
  5. Enters the umbrella portal, imports command line parameters from the package, and binds them to the user interface.
  6. Exports a matrix for testing or release on a carrier with our shell engine and dependent products.
  7. Installs.

It is possible to automatically install, done through Silent mode. You can launch it remotely or by event via Windows Scheduler or PowerShell. Now it’s immediately clear where development is delayed: by the application or peking team :)

History of creation


In the beginning it was very important to find successful examples that could be oriented. At that time there were not many of them, and there were virtually no flexible solutions.

We looked at the VS VS, and even wrote to Microsoft, it was not flexible enough. And the scope is different. They did not have plans to enter the market with something customer-oriented. I was a little surprised that they didn’t have a common practice in this area, but we decided that the rich had their quirks. We will go the other way.

We also communicated with Flexera (at that time called Acresso Software) - asked if they could offer something ready within a reasonable time. A large thread was started in the forum with product wishes and a long, representative conversation with the company's management took place. There we described our product requirements and how we see the final decision. Terms from them have not received. Three years later, the InstallShield Suite Flexera appeared, which is now supplied with InstallShield Premier, in which many of our ideas are guessed :)

Prototyping


In the beginning, we made a couple of prototypes that allowed us to understand the budget and technical aspects of the solution.
A very good idea was storing the application dependency matrix in XML, which would be supplied with the product.

The simplicity and compatibility of WIX Deployment Tools Foundation made it possible to integrate .NET code into MSI without any problems. Previously, even if it was possible to pull the .NET method from InstallShield, the problem with the links to the libraries spoiled life. Actually, how to use in the installation process the functionality of libraries not from Global Assembly Cache is a mystery (there are several options, but they are all rather laborious), and inconvenient. The DTF is simply a container, where everything that the library refers to automatically enters. No "manual" work.

The functionality for repacking the Silverlight XAP file to change the configuration inside has become critical for the transition from VBScript, which is still used today. Further .NET Custom Actions began to be used everywhere.

It was suggested to refuse the MSI user interface when installing a set of applications: one absence of an event to change the contents of the text box is worth it. Who, by the way, and still there.

For separate installation of the UI application, the Windows Installer was saved, but strictly standardized. As a result, we got a little hot with the user interface in one of the first prototypes, because it was also supposed to be in XML. The idea was excellent, a year later it was implemented by Microsoft in WPF, therefore it was actually recognized as a result unsuccessful and replaced with it.

Agreed that the sample will take the Microsoft Office installer. The Office team, as far as I remember, developed the Windows Installer - there is no reason not to trust them. The friendliness of its installation at the highest level - you can simply click on Next and get a workable product. We wanted the same thing.

Prototype 1


Just a bootstrapper, with configurable options for running MSI. InstallShield for installers.
It turned out not enough - it will not save from a mess with dependencies, and the interface for all cases does not write.

Prototype 2


.NET, XML everywhere. WIX as an engine for installers.
It turned out that there was too much code (in particular, describing the packages themselves), a rather complicated structure for WIX XML projects. The XML GUI engine was also quite voluminous. In the process of use, it turned out that error diagnostics and solution support are too time consuming. Quickly the source of the problem is not localized and not corrected.

Prototype 3


Database and export from DataSet to XML for the bootstrapper, UI to WPF, own toolkit for creating installation packages.
This decision was already viable - it was decided to stop at it.

Unrealized ideas


Several ideas have never been tested:


Editing the dependency matrix


Let's return to our installers. With the XML matrix, everything was fine, except that it was almost impossible to work with data for several users at the same time. Errors when entering, validation dependencies - need an editor.

We came to the idea of ​​storing and editing the matrix in a SQL Server database and exporting it to the XML that comes with the installer.


According to the results, the “readability” and “modifiability” of the XML file suffered greatly, and the dependence on the editor increased. However, there were specialists who managed to edit this manually if this situation arises. But now there is usually "pens" climb only for experiments.

What are the options on the modern market?


We, of course, see what's new on the market. If we see a solution covering our requirements, we can switch to it.

Here is some comparison of our solution with others available in the market.
Characteristic
InstallShield Suite
WIX Burn
Our product
Product dependency checking
Not
Not
there is
Centralized multi-user editor
Not
Not
there is
Storage of component installation parameters
XML
XML
SQL Server, XML
Developer User Interface
InstallShield
XML
Silverlight client
Dynamically changeable user interface
Not
Not
Yes
Interface availability for automation
Yes, starting from version 2015
No, you can implement XML-based
Yes WCF
Interface for extensions
C ++ plugins
.NET
.NET
Work with the application database
Not
Not
SQL Server
Estimated Integrated Product Installer
InstallShield
WIX
InstallShield + WIX DTF
Built-in Active Directory Configuration
Not
Not
there is

Our solution is in the details.


It was adopted by C #, MVVM and SQL Server - a standard tool, not like VBScript.

InstallShield - MSI Project Editor
Pros:

Minuses:

C # is the main programming language ("inside" and outside MSI, in the "umbrella")
Pros:

Minuses:

WPF - UI Bootstrapper
Pros:

Minuses:

SQL Server - Dependency Matrix Storage
Pros:

Minuses:

Silverlight - UI editor of products and their dependencies
Pros:

Minuses:

All this, of course, we will try to consider in detail if there is such interest among the readers.

Installers - before and after the umbrella


Actually, all of the above was started for the sake of installing and configuring the final products. So let's talk about the installers of products.

Error number 1 that was made - we began to write an “umbrella” before the development of the pekedzha completely settled down. Because of this, there are excessive product requirements. For example, several types of dependencies were implemented, of which only two are actually used. The article will not go this vicious way!

Not everyone likes MSI. And we always had MSI, and even not always InstallShield, and even not always WIX in other cases. In general, a decent solyanka, such a rich ...

Composition: WIX, InstallShield, Wise, InstallScript, VBScript, .NET

From the beginning about why this is bad - bitty packages. And why is this sure to vybesit you, if you start to put it on the stream.

This is what could be ...


And everything is good, but everyone has their own standards and limitations. This results in total chaos in terms of functionality and parameterization. And if the installer is not at all based on MSI, then the popular Orca editor from the Windows SDK will not help to find that cherished parameter. In addition, WISE, for example, is no longer supported.

Typical options
  1. The all-in-one installer based on InstallScript, as a probable interpretation of Wise or on the basis of any other scripting engine. It only works with the GUI and tries to install all related components before installing the product. As a rule, a lot of code inside, completely unpredictable parameterization and support for the “silent” mode. Removal is usually dirty, implemented on the residual basis. Upgrade - the same way. This is a rewrite.
  2. MSI with InstallScript and VBS inserts. Can be created by various development environments from InstallShield to WIX. Strong side - do not depend on the availability of .NET. But who has Windows now without it? Among the shortcomings - the complexity of the development and support. Awkward debug. Parameterization is chaotic. Usually silent mode somehow works. This is left as possible as is. Or rewrite if something does not suit.
  3. A good option. MSI on WIX or InstallShield with WIX DTF. But a mess with the parameters can be. We lead to a common denominator.

As a result, InstallShield was left based on our generated template and WIX installers. Both use the WIX DTF with our wrappers. The container setup.exe was abandoned altogether; you need to be able to see what's inside.

Parameterization


If in .NET we have a real OOP (interfaces, classes that we can override, modify and inherit), then there is no MSI. If not, then you need to come up with something - we have these prefixes and naming conventions. If now in managed languages ​​this is considered a sign of bad tone (but we sin, we still sin in the user interface for some reason), then for the Windows Installer - just right.

That is, so we call the properties (CIF - abbreviated Common Installation Framework) MSI, in simple terms "property".

If the package contains our standard properties, the installer processes them and supports, in this case, the configuration of Active Directory:
CIF_AD_USER
CIF_AD_PASSWORD

Non-standard properties (for example, if an application has several users) are configured by separate input fields, or are overridden manually in advanced settings:
APP_AD_USER
APP_AD_PASSWORD

Why upper case? For those used inside MSI, we allow the use of both registers. For all external interfaces - only the top. Why? InstallShield is not case-sensitive everywhere (system search, for example).

Redundant options will not cause problems if MSI does not use them - just ignore them. Therefore, a standard set of parameters is always transferred, which simplifies the addition of a new product and minimizes the required amount of documentation or related communication.

How do we pass parameters to installers? Considered 2 options:
  1. Via command line
  2. Through transforms

Chose the first option. Cause? Trite easier. Yes, and the batch file is easier to edit. It is better to use one, the most universal, approach. Restrictions on the length of the line we have not reached.

Custom actions, log and exception handling


With input parameters sorted out. And what about the logs? InstallScript is generally binary. How to understand that the installation failed in the process? Error codes may be different. How to understand what went wrong during the execution of the very VBS CA written 3 years ago?

Need standardization:
  1. The need to support a standard set of parameters for installing and updating the product.
  2. Diagnostics and error handling during installation.
  3. Setting the packaging process to flow.
  4. Addition of the built-in InstallShield configuration tools for working with XML, IIS, etc. Avoiding “flashing black windows”.

As a rule of good tone, the recommendation is to use the WIX Deployment Tools Foundation for InstallShield. Then it was not so obvious.

What we lacked in InstallShield Professional was the various IIS settings and the order in which they were applied during the upgrade. We also didn’t like how InstallShield works with XML (many applications didn’t work because the elements were crookedly closed, their order changed) - the functionality of reformatting and working with XML fragments increased. Now we are going to the text file changes and our own XML engine.

Such an architecture is the result.


On the picture:

What is good WIX .NET custom action - interface MSI DLL. As a result of the assembly, the test-box automatically forms a zip container with all associated libraries: if you rename to ZIP, you can see the contents, packed libraries, etc.

The logic and dependencies are stored separately. As a result, the library can be reused from PowerShell, MS Build, or from our configuration umbrella. Not to mention that if someone wants to configure something directly from the application.

How can we debug the installer? Very simple - we set the CIF_DEBUG system environment variable to 1 and each of our CAs will write not to the log already, but to the Message Box. There are a lot of options to pick up and see from the inside what is happening.

How is this done? All MSI service functions are performed by a special Core library that serves as a level of abstraction over the Microsoft Deployment Tools Foundation. It is easy to throw the output on the GUI.

It's still quite simple to make an exe from a custom action - the only difference is in the constructor, and the output is already going to the console! This is in case of "quick check on the environment."

.NET Custom Actions


We have a lot of different .NET Custom Actions, we use the following naming convention:
[Subject] [Action] [Execution Type] ([Parametrization Type])

Example:
XmlMergeNodesDef - deferred execution in system context custom action, merges nodes in configuration files
MsiParseStringImOut - immediate execution, generates several new properties on the output, so the type "out".

What we have not yet fully resolved is that we have not abandoned the heaps of actions that pass the “deferred in system context” parameters to custom actions. They still have and they have the extension “_SetCAProp”

Example:
CIF_XmlMergeNodesDef_SetCAProp - passes parameters to the deferred custom action of the same name.

How did we solve the problem of passing parameters? In the template, a single .NET CA TemplatePrepare manages everything. It is generated automatically by our toolkit by the specified parameters.

We use a using construction that declares the beginning and end of the custom action code after the standard WIX binding. Opening bracket - automatically posts a greeting to the log with the version of the library, closing says goodbye. Each line has a standard prefix with the name custom action and timestamp, for example:
CIF CA NetServicesStopIm 06: 27: 56.1707871: Starts with CIF Core 3.7.0.15 -> Windows Installer Log, parameters in CustomActionData
CIF CA NetServicesStopIm 06: 27: 56.1707871: SplitData splitting property CustomActionData
CIF CA NetServicesStopIm 06: 27: 56.1707871: SplitData found 1 parameter values ​​in "with delimiter" | "
CIF CA NetServicesStopIm 06: 27: 56.1863471: Searching for property references in "AppFabricEventCollectionService | AppFabricWorkflowManagementService | AppFabricCachingService |" ...
CIF CA NetServicesStopIm 06: 27: 56.1863471: Searching for property references in "" ...
CIF CA NetServicesStopIm 06: 27: 56.1863471: No properties detected.
CIF CA NetServicesStopIm 06: 27: 56.1863471: Get property AppFabricEventCollectionService | AppFabricWorkflowManagementService | AppFabricCachingService
CIF CA NetServicesStopIm 06: 27: 56.1863471: Searching for property references in "8" ...
CIF CA NetServicesStopIm 06: 27: 56.1863471: No properties detected.
CIF CA NetServicesStopIm 06: 27: 56.1863471: Get property InstanceId = 8
CIF CA NetServicesStopIm 06: 27: 56.1863471: {InstanceId} replaced with "8"
CIF CA NetServicesStopIm 06: 27: 56.2488510: Ends

Note that it is very easy to read - for the scroll just click on "Find the next entry." By timestamp with milliseconds, you can see how much this or that action takes a lot of time (sometimes because of the wrong parameter, something too voluminous is processed at a higher level, for example).

Conclusion, conclusions, advantages and disadvantages of the decision


As a result - now people on application development projects are engaged in these projects. And the people involved in the configuration, in fact, she and engaged.
If something new needs to be configured, then there is a process from creating a user story in the backlog to getting a result in the form of a new library, described in the documentation with examples.

Minuses:

Pros:

In general, the goal was achieved, although the decision matured a little later than we would like. So we wish you clear weather and do not fall under the rain, but if that happens, then be ready with an umbrella! :)

Source: https://habr.com/ru/post/277733/

More articles:


All Articles