Automate system administration, review problems and solution. Ed wilson
Below is a translation of part 1 of the Powershell and WMI book. It covers the direction of development of information systems in relation to system administration. Ed Wilson gives a look at the problems of the operators, indicates the direction of reducing the costs of infrastructure maintenance.
Ask any Windows administrator about his problems, and at the top of the list there will be a lot of work and a constant lack of time to do it. They know about automation tools, maybe even aware of the capabilities of WMI and Powershell, but they don’t have time to master these technologies. This is a disgraceful situation, because it is generally accepted that up to 70% of an organization’s IT budget is spent on “so that everything is in working order” (example of a translator: in the original “keep the lights on.”) . Automation can reduce costs from these 70%, due to which time and money will be freed up for tasks further down the list of “problems”.
In addition, it is possible that they were interested in WMI or Powershell and came to the conclusion that these tools are too difficult to master. This is understandable, especially given the complexity of working with WMI through VBScript, as well as the absence of examples that would explain the techniques used. Even I am horrified by some examples of Powershell scripts posted on the Internet, what do those who have just started to deal with this topic feel. Administrators, refusing to use these tools, miss opportunities to reduce their workload, miss the opportunity to automate their processes.
')
The goal of this book was to lower the entry bar to WMI usage. The examples provided in it can be used without changes or with a minimum of them. In addition, you will gain a deeper understanding of WMI, which can be used to work with areas previously inaccessible for your control. PowerShell is designed to make WMI simpler and clearer than previously used in scripting languages, Microsoft's PowerShell automation engine, which, among other things, provides easy access to the rich sets of management tools available in WMI.
Together, PowerShell and WMI provide a set of proven methods that allow you to manage your system easily and quickly. You can automate many standard tasks that currently consume too much of your attention.
First of all, we will describe the problem facing the modern system administrator.
There are a number of issues that affect any Windows environment of considerable size.
In the second part of the chapter, I will show why PowerShell and WMI provide a large set of tools to solve these problems. Getting a return on PowerShell implies a small initial investment of time to learn it, especially when using WMI. After studying the language and automating your daily processes, you can achieve an excellent return on the time spent on the learning process.
The chapter ends with two examples of demonstrating the power of this combination of technology. The first example shows how you can turn off all servers in the data center with one command, and the second shows how you can check the settings of many machines in a single pass.
Let's start by looking at the responsibilities of a modern Windows administrator and the problems he faces.
Administrators are very busy people. They seem to constantly need to do more with fewer resources. In Figure 1.1 on the left we see a graph of the cost reduction of hardware. For example, I recently purchased a laptop with a 4-core processor (HyperThreading allows you to see eight cores) and with 16 GB of RAM, I'm going to use it as my mobile lab. So, a few years ago, a machine with these parameters was in the middle segment of servers, not laptops!
The same is true for servers with 4, 8, or even 10 core processors. Computers can be installed more and more with regard to cheap memory. This means that an increasing number of companies can afford to launch applications and business processes that were previously used only by large corporations with huge IT budgets.
Reducing the cost of hardware leads to two other schedules, they reflect a steep increase in the complexity of the infrastructure and even faster growth in administrative costs for maintaining it in working condition. The constant complication of the infrastructure and the increase in its costs are currently the main problem of the IT departments.
To reduce the budget, you need to break the growth curve of operating costs and PowerShell with WMI will help you with this. To begin with, let's see - where does the complexity and the rising costs of administration come from?
1.1.1. Too many cars
Do you really need every server that you have created? Many, if not most organizations have too many servers. This happens for a number of reasons:
The load on the system administrator is growing faster than the growth rate of the power of the machines due to the time that he needs to spend on switching attention between machines (usually a new remote connection is required), the additional complexity of each machine and its applications is brought to life by the system administrator.
Virtualization is in full swing also makes its own. There are several advantages of virtualization:
The organization as a whole benefits from virtualization, but the administrator load increases. If you had 100 servers before virtualization, and you installed 4 physical hosts instead of them and 100 servers were installed on them, then you will have 104 systems that need to be managed. Besides the fact that there are 4 more of them, the complexity will increase due to the fact that the virtualization platform can introduce another operating system into the infrastructure or add new tools. An increase in the total number of (physical plus virtual) systems also means that the likelihood of accidents increases during the evolution of infrastructure.
Change can be considered as the worst headache of the administrator. Unfortunately, the environments are not static, changes always happen:
These problems are superimposed on tens, hundreds or even thousands of machines, superimposed on daily activity, such as monitoring and backup.
This situation can not continue indefinitely. Companies cannot mitigate the ever-increasing administration costs; today's economic realities prohibit the use of alternative mechanisms such as an endless increase in revenues in order to overlook the rising costs of IT departments. The situation should be resolved by reducing operating costs, but administrators cannot do this because a lot of changes are brought by new and new technologies that do not reduce the load, and so it goes on forever.
Difficulty is a real problem. It arises from a number of reasons:
The complexity also increases due to the incompleteness of the knowledge and skills of the administrators themselves. Too often, the project introduces a new technology and system administrators are expected to immediately pick up and start managing the system. Do they have the necessary skills? Do they have time to learn the intricacies of the introduced technology? Unfortunately, the answer to both questions is often negative.
The administrator in such a situation will take the only right decision - he will start to do as he can. Sometimes, if a new technology is a version change from something already, the administrator will continue to use the old methods, even if the new version has a much better way to do the job.
Lack of skills and knowledge leads to mistakes, and these mistakes cost money, often they can be seen as a loss of income for an organization. This puts administrators under stress and leads to distrust from business management. The IT department is often excluded from the discussion on the introduction of new technologies, it is too late that it learns that management has decided to change their environment, and the cycle of errors will repeat again.
In addition, little changes are being made by projects; system administrators are facing many of the smallest changes necessary to keep the environment safe and to ensure stability.
The solution to overcome these problems is to automate routine operations. Making the machine do simple, repetitive work is what we were doing for them!
Automation is understood differently. Below is the automation hierarchy.
To implement a level above this hierarchy, you need to answer the question - “will there be any benefit from moving to a more complex level?”. I know a number of organizations that lack the capabilities of the standard Windows toolkit and several tools for mass distribution of commands (note of the translator: I don’t know how to accurately translate "a few bulk-editing tools") . Others try to make the most of the task scheduler or even create responses to events in the log. Automation for most organizations is a mixture of command line tools, scripts, and tasks in the scheduler.
The next question is “How can we automate my administrative tasks?”. PowerShell provides a set of command line tools (called “cmdlets”) that you can use by typing interactively into the console. As the complexity of the tasks becomes more and more ambitious, the emergence of scripts. In PowerShell, you can use the same commands, one spelling and one style for the command line and when writing scripts.
PowerShell is a great tool (Yes, I'm a fan of PowerShell). If necessary, you can easily move to the next level of difficulty, and so on, and WMI will be at the very top of this staircase. (Note of the translator. Richard Siddaway in the book “powershell in depth” wrote “if you don’t use WMI together with powershell, then you lose yes 60% of power”) . WMI will give you access to a standard set of system management tools that you can use locally or on a remote machine, potentially you can work even with non-Windows systems (for example, the translator means CIM) . Scripts can be run interactively or can be scheduled on time. But before we dive into the details, we need to look at automation as a whole.
In this book, we will focus on scenarios as a primary automation tool. Of course, you could do a lot from the command line by connecting to remote machines. However, the advantage of scripts is that you can reuse the code, each time saving time on writing and debugging code. This topic is discussed in detail in Chapter 4 of the book “PowerShell in Practice” by Manning 2010. Planned tasks and “automatic responses to events” depend too much on your specific infrastructure, in Chapter 3 we will begin to consider how you can make automatic responses to events occurring in your system.
Let's take an example. Suppose you need to determine the amount of free space on the C drive of several machines in your environment. One way is to come to the data center, connect to each machine in turn and see the free space of the C drive. Record the answer and repeat for the next machine.
A slightly simpler option is to use RDP to connect to each machine and manually download the information. This way you will not leave your table. But you still have to do a lot of small actions, you still lose too much time.
And the solution I like is to use PowerShell for this purpose, the code is shown in Listing 1.1. Do not worry if you do not understand it right now. We will return to this scenario in Chapter 6 when we look at how to manage the disk subsystem.
The example begins with a list of computer names in my lab. This list is passed by the pipeline to the ForEach-Object (foreach) cmdlet that calls Get-WmiObject for each server in the list, requesting data on the logical drive C. Then, the received information is formatted and output as a table.
Listing 1.1 Determine free disk space:
Free space is converted from bytes to gigabytes to make the results more understandable. Note that PowerShell understands abbreviations GB, as well as KB, MB, TB and PB.
The result of the script is as follows:
A number of improvements can be made to this scenario:
I have this script with the first two improvements. He regularly reports on the place on the disks, you can see the trends. After writing, I have a tool that can be run in a few seconds, it will poll each machine itself and return the information. In which case I can finish it.
It took me only a few minutes to write it and I save time when I run it again and again. This is the way PowerShell helps save time. Jeffrey Snover, the architect of PowerShell, wrote, “I firmly believe that economics determines what people do and what they don't do. PowerShell is designed from scratch to be extensible, high-level, task-oriented abstraction, reducing the cost of administration and support. ”The full text of his article (here already translated)“ Semantic gap ” is available on the Windows Powershell blog at blogs.msdn. com / b / powershell do a search for the word "semantic gap" and you will come across this article.
In Figure 1.1. we have seen the continuous increase in organization complexity and administration costs. This continuous increase in complexity and expenses will sooner or later lead to a loss of balance and stunting due to management costs; we need a way to break this growth.
PowerShell can help stop spending by providing the following:
PowerShell methods to improve performance and efficiency. And with PowerShell and WMI, you can count on further increasing your growth of control over the system.
P / S / Bottom line: if your infrastructure is facing increasing complexity, you should think about moving to a new level of automation. I hear a lot from friends about the complexity of the infrastructure, constant pressure from marketing departments, constant change of systems. Personally, I feel such pressure, now I’m looking for a way to create automatic bots for admin tasks.
Solving administrative challenges
Ask any Windows administrator about his problems, and at the top of the list there will be a lot of work and a constant lack of time to do it. They know about automation tools, maybe even aware of the capabilities of WMI and Powershell, but they don’t have time to master these technologies. This is a disgraceful situation, because it is generally accepted that up to 70% of an organization’s IT budget is spent on “so that everything is in working order” (example of a translator: in the original “keep the lights on.”) . Automation can reduce costs from these 70%, due to which time and money will be freed up for tasks further down the list of “problems”.
In addition, it is possible that they were interested in WMI or Powershell and came to the conclusion that these tools are too difficult to master. This is understandable, especially given the complexity of working with WMI through VBScript, as well as the absence of examples that would explain the techniques used. Even I am horrified by some examples of Powershell scripts posted on the Internet, what do those who have just started to deal with this topic feel. Administrators, refusing to use these tools, miss opportunities to reduce their workload, miss the opportunity to automate their processes.
')
The goal of this book was to lower the entry bar to WMI usage. The examples provided in it can be used without changes or with a minimum of them. In addition, you will gain a deeper understanding of WMI, which can be used to work with areas previously inaccessible for your control. PowerShell is designed to make WMI simpler and clearer than previously used in scripting languages, Microsoft's PowerShell automation engine, which, among other things, provides easy access to the rich sets of management tools available in WMI.
Together, PowerShell and WMI provide a set of proven methods that allow you to manage your system easily and quickly. You can automate many standard tasks that currently consume too much of your attention.
First of all, we will describe the problem facing the modern system administrator.
There are a number of issues that affect any Windows environment of considerable size.
- Increase the number of systems
- Increasing infrastructure complexity
- Increasing rate of change
In the second part of the chapter, I will show why PowerShell and WMI provide a large set of tools to solve these problems. Getting a return on PowerShell implies a small initial investment of time to learn it, especially when using WMI. After studying the language and automating your daily processes, you can achieve an excellent return on the time spent on the learning process.
The chapter ends with two examples of demonstrating the power of this combination of technology. The first example shows how you can turn off all servers in the data center with one command, and the second shows how you can check the settings of many machines in a single pass.
Let's start by looking at the responsibilities of a modern Windows administrator and the problems he faces.
1.1 Administrative tasks
Administrators are very busy people. They seem to constantly need to do more with fewer resources. In Figure 1.1 on the left we see a graph of the cost reduction of hardware. For example, I recently purchased a laptop with a 4-core processor (HyperThreading allows you to see eight cores) and with 16 GB of RAM, I'm going to use it as my mobile lab. So, a few years ago, a machine with these parameters was in the middle segment of servers, not laptops!
The same is true for servers with 4, 8, or even 10 core processors. Computers can be installed more and more with regard to cheap memory. This means that an increasing number of companies can afford to launch applications and business processes that were previously used only by large corporations with huge IT budgets.
Reducing the cost of hardware leads to two other schedules, they reflect a steep increase in the complexity of the infrastructure and even faster growth in administrative costs for maintaining it in working condition. The constant complication of the infrastructure and the increase in its costs are currently the main problem of the IT departments.
To reduce the budget, you need to break the growth curve of operating costs and PowerShell with WMI will help you with this. To begin with, let's see - where does the complexity and the rising costs of administration come from?
1.1.1. Too many cars
Do you really need every server that you have created? Many, if not most organizations have too many servers. This happens for a number of reasons:
- Reducing the cost of hardware power - this leads to the fact that with high load it is easier to purchase a new server than to look for how to optimize the existing ones.
- Independent procurement of departments or procurement for projects - this is due to server ownership issues, departments or “projectors” do not want anyone to “sit” on their servers. They do not wish to provide others with their resources.
- "One application - one server" - separation of applications so that the problem in one does not affect the other, this rule can still work for critical business applications, but this is not necessary for the second or third tier of applications. And of course, it is not necessary for the purposes of testing and training.
- Slow reaction or rigidity of IT departments - lack of control and neglect of processes in the IT department leads to confusion in projects and random change of systems.
The load on the system administrator is growing faster than the growth rate of the power of the machines due to the time that he needs to spend on switching attention between machines (usually a new remote connection is required), the additional complexity of each machine and its applications is brought to life by the system administrator.
Virtualization is in full swing also makes its own. There are several advantages of virtualization:
- Reduced number of physical servers
- Reduced requirements for data center organizations, including because of the space, electricity and air conditioning costs
- More precisely the power of physical assets is used, which leads to a greater return on investment
The organization as a whole benefits from virtualization, but the administrator load increases. If you had 100 servers before virtualization, and you installed 4 physical hosts instead of them and 100 servers were installed on them, then you will have 104 systems that need to be managed. Besides the fact that there are 4 more of them, the complexity will increase due to the fact that the virtualization platform can introduce another operating system into the infrastructure or add new tools. An increase in the total number of (physical plus virtual) systems also means that the likelihood of accidents increases during the evolution of infrastructure.
1.1.2 too many changes
Change can be considered as the worst headache of the administrator. Unfortunately, the environments are not static, changes always happen:
- Operating system and applications get patches regularly
- New software versions are coming out.
- The storage space needs to be reconfigured to fit the changing typical user tasks.
- Typical application scenarios require hardware changes or hardware upgrades.
- Virtualization and other breakthrough technologies are changing the ecosystem and creating new opportunities and configurations.
These problems are superimposed on tens, hundreds or even thousands of machines, superimposed on daily activity, such as monitoring and backup.
This situation can not continue indefinitely. Companies cannot mitigate the ever-increasing administration costs; today's economic realities prohibit the use of alternative mechanisms such as an endless increase in revenues in order to overlook the rising costs of IT departments. The situation should be resolved by reducing operating costs, but administrators cannot do this because a lot of changes are brought by new and new technologies that do not reduce the load, and so it goes on forever.
1.1.3 Increasing complexity
Difficulty is a real problem. It arises from a number of reasons:
- Several operating systems carry different toolkits and terminology, there is even a difference between the two versions of Windows.
- Different types of applications, such as databases, e-mail, Active Directory directory services, and web applications, require different skills, different tools, have different requirements and create different server load.
- Many machines perform the same or similar roles, but the inconspicuous features in their implementation, undocumented features increase the likelihood of error and make the structure more difficult.
The complexity also increases due to the incompleteness of the knowledge and skills of the administrators themselves. Too often, the project introduces a new technology and system administrators are expected to immediately pick up and start managing the system. Do they have the necessary skills? Do they have time to learn the intricacies of the introduced technology? Unfortunately, the answer to both questions is often negative.
The administrator in such a situation will take the only right decision - he will start to do as he can. Sometimes, if a new technology is a version change from something already, the administrator will continue to use the old methods, even if the new version has a much better way to do the job.
Lack of skills and knowledge leads to mistakes, and these mistakes cost money, often they can be seen as a loss of income for an organization. This puts administrators under stress and leads to distrust from business management. The IT department is often excluded from the discussion on the introduction of new technologies, it is too late that it learns that management has decided to change their environment, and the cycle of errors will repeat again.
In addition, little changes are being made by projects; system administrators are facing many of the smallest changes necessary to keep the environment safe and to ensure stability.
1.2 Automation - a way forward
The solution to overcome these problems is to automate routine operations. Making the machine do simple, repetitive work is what we were doing for them!
Automation is understood differently. Below is the automation hierarchy.
To implement a level above this hierarchy, you need to answer the question - “will there be any benefit from moving to a more complex level?”. I know a number of organizations that lack the capabilities of the standard Windows toolkit and several tools for mass distribution of commands (note of the translator: I don’t know how to accurately translate "a few bulk-editing tools") . Others try to make the most of the task scheduler or even create responses to events in the log. Automation for most organizations is a mixture of command line tools, scripts, and tasks in the scheduler.
The next question is “How can we automate my administrative tasks?”. PowerShell provides a set of command line tools (called “cmdlets”) that you can use by typing interactively into the console. As the complexity of the tasks becomes more and more ambitious, the emergence of scripts. In PowerShell, you can use the same commands, one spelling and one style for the command line and when writing scripts.
PowerShell is a great tool (Yes, I'm a fan of PowerShell). If necessary, you can easily move to the next level of difficulty, and so on, and WMI will be at the very top of this staircase. (Note of the translator. Richard Siddaway in the book “powershell in depth” wrote “if you don’t use WMI together with powershell, then you lose yes 60% of power”) . WMI will give you access to a standard set of system management tools that you can use locally or on a remote machine, potentially you can work even with non-Windows systems (for example, the translator means CIM) . Scripts can be run interactively or can be scheduled on time. But before we dive into the details, we need to look at automation as a whole.
In this book, we will focus on scenarios as a primary automation tool. Of course, you could do a lot from the command line by connecting to remote machines. However, the advantage of scripts is that you can reuse the code, each time saving time on writing and debugging code. This topic is discussed in detail in Chapter 4 of the book “PowerShell in Practice” by Manning 2010. Planned tasks and “automatic responses to events” depend too much on your specific infrastructure, in Chapter 3 we will begin to consider how you can make automatic responses to events occurring in your system.
Let's take an example. Suppose you need to determine the amount of free space on the C drive of several machines in your environment. One way is to come to the data center, connect to each machine in turn and see the free space of the C drive. Record the answer and repeat for the next machine.
A slightly simpler option is to use RDP to connect to each machine and manually download the information. This way you will not leave your table. But you still have to do a lot of small actions, you still lose too much time.
And the solution I like is to use PowerShell for this purpose, the code is shown in Listing 1.1. Do not worry if you do not understand it right now. We will return to this scenario in Chapter 6 when we look at how to manage the disk subsystem.
The example begins with a list of computer names in my lab. This list is passed by the pipeline to the ForEach-Object (foreach) cmdlet that calls Get-WmiObject for each server in the list, requesting data on the logical drive C. Then, the received information is formatted and output as a table.
Listing 1.1 Determine free disk space:
"dc02", "W08R2CS01", "W08R2CS02", "W08R2SQL08", "W08R2SQL08A", "WSS08" | foreach { Get-WmiObject -Class Win32_LogicalDisk -ComputerName $_ -Filter "DeviceId='C:'" } | Format-Table SystemName, @{Name="Free"; Expression={[math]::round($($_.FreeSpace/1GB), 2)}} -auto Free space is converted from bytes to gigabytes to make the results more understandable. Note that PowerShell understands abbreviations GB, as well as KB, MB, TB and PB.
The result of the script is as follows:
SystemName Free ---------- ---- W08R2CS01 119.04 W08R2CS02 118.65 W08R2SQL08 114.8 W08R2SQL08A 115.17 WSS08 111.41 DC02 118.53 A number of improvements can be made to this scenario:
- Put the computer names in the CSV file (as we will in Listing 1.4)
- Add the result of work in Excel, or the database so that you can see the trend of changing disk space
- Schedule a task in the scheduler
I have this script with the first two improvements. He regularly reports on the place on the disks, you can see the trends. After writing, I have a tool that can be run in a few seconds, it will poll each machine itself and return the information. In which case I can finish it.
It took me only a few minutes to write it and I save time when I run it again and again. This is the way PowerShell helps save time. Jeffrey Snover, the architect of PowerShell, wrote, “I firmly believe that economics determines what people do and what they don't do. PowerShell is designed from scratch to be extensible, high-level, task-oriented abstraction, reducing the cost of administration and support. ”The full text of his article (here already translated)“ Semantic gap ” is available on the Windows Powershell blog at blogs.msdn. com / b / powershell do a search for the word "semantic gap" and you will come across this article.
I’m missing a translator here several sections
1.3.3 Break the growth curve
In Figure 1.1. we have seen the continuous increase in organization complexity and administration costs. This continuous increase in complexity and expenses will sooner or later lead to a loss of balance and stunting due to management costs; we need a way to break this growth.
PowerShell can help stop spending by providing the following:
- provides a set of interactive tools for working with the server and applications
- It works in all Windows systems (translator’s example: in the original, the word “estate” is the exact translation - estates, possessions. This implies that this is Microsoft’s core technology and all systems have cmdlets anyway)
- Provides one universal approach to working with different systems (translator's note: all cmdlets are mono-images, no need to learn the list of keys of the console utility)
- Built-in remote control capabilities
- Built-in capabilities to further complicate automation
PowerShell methods to improve performance and efficiency. And with PowerShell and WMI, you can count on further increasing your growth of control over the system.
P / S / Bottom line: if your infrastructure is facing increasing complexity, you should think about moving to a new level of automation. I hear a lot from friends about the complexity of the infrastructure, constant pressure from marketing departments, constant change of systems. Personally, I feel such pressure, now I’m looking for a way to create automatic bots for admin tasks.
Source: https://habr.com/ru/post/301242/
All Articles