Common problems in the IaaS cloud based on the VMware hypervisor and their solutions
Have you ever encountered a situation when a virtual machine (VM) refused to start via a remote console? Or were there difficulties with a slow running VM? In this article we will talk about the problems that can be encountered when working with cloud environments, and their solutions.
/ photo David Cruz CC
Suppose the linux-base-01 virtual machine is used to transfer business-critical information from a research lab to the cloud using the clipboard. A typical task is when a copy operation is invoked on one side, and an paste is called on the other side. The system user found that the clipboard does not work correctly, therefore it does not allow copying data from a laptop to the VM directory.
To solve the problem, you need to open a browser and connect to the vSphere Web Client. In the control panel window that opens, after authentication, we refer to the user directory. The user provided us with a fragment of the study in the form of a file readme.txt. Now we need to solve the following tasks: restore the clipboard, copy the text from the user file to the desktop and paste it into the protocol.txt document of the linux-base-01 machine in the cloud.
')
Buffer usage can be enabled and disabled for a single virtual machine, as well as for all ESXi host machines. To activate the copy and paste features for a specific VM, you need to complete these steps:
In the opened window we allow the use of copy / paste functions:
Then save the changes. Note that after this you need to restart the VM, transferring it to the status of power on. For more information on solving problems with the clipboard, you can use materials from the VMware knowledge base.
Suppose that a large number of requests from users to perform various operations have accumulated in the queue. The employee believes that this was due to the poor performance of the virtual machine, which is unable to cope with the influx of requests. To solve this problem, you should first turn to the troubleshooting methodology to clearly identify the source.
Most often, performance problems are caused by disks (if you're interested, here is a good study on the subject of “disk-dependency”). While the network causes a similar much less. However, it is still necessary to conduct an analysis of the performance of the subsystems, identify bottlenecks, and draw up a plan for their elimination or optimization.
In our case, we first connect to the problematic virtual machine using terminal access. We assume that user statistics collection tools have already been installed on the user VM, so we see information about the host name, CPU load, RAM, IP address, etc.
We will also use the SPECjbb utility, which is a benchmark for measuring performance, created by the Standard Performance Evaluation (SPEC). The results obtained during testing and presented in the figure below indicate that the perf-01 system is experiencing performance problems.
This was caused by excessive use of processor resources. It turned out that on the esx-01a host (dual-core processor), in addition to perf-01a, another virtual machine is deployed - perf-02a. In this case, perf-01a uses 2vCPU, and perf-02a uses 1vCPU, which exceeds the number of available host cores and “clogs” the system.
You can solve the problem in several ways. Firstly, you can transfer perf-01a to a less loaded ESXi host, for example esx-02a, which is practically not loaded. Secondly, do the same, but move perf-02a. Thirdly, you can use the redistribution of computing resources for the perf-02a machine.
The set limit will reduce the load on the system. Also note that increasing the number of virtual processors can improve performance. For this, the Virtual Hardware tab has a special line for adding additional processor resources.
However, it must be remembered that if the host on which the virtual machine resides has only two physical processors, an increase in the number of CPUs can lead to cost overruns and reduced performance. You should also make sure that the operating system and applications can work in multi-threaded mode.
Configuring virtual machines with more virtual CPUs than is actually required for the workload can lead to an increase in resource utilization, potentially affecting the performance of the loaded systems.
Most of the guest operating systems operate idle during idle times, often following HLT or MWAIT instructions, while some operating systems use busy-waiting. All this leads to the consumption of resources that could be available for other tasks.
There are two reasons why a user may experience this problem:
To establish the cause of the fault and eliminate it, we connect to the vSphere Web Client - for access we use an account with administrator rights. In the control window that opens, go to the Administration> Single Sign-On> Users and Groups tab, and then find the problematic user account. The image below shows that the Murphy account has the Disabled attribute, that is, it is disabled and cannot be used for access.
To eliminate the "fault", simply enable the account by calling the context menu and setting the Enable attribute.
There are situations when you or users of the system cannot delete existing virtual machine snapshots. This problem manifests itself in the event that a backup server or another VM is blocking the basic disk or the previous snapshot file, making it impossible to perform the merge. This also happens if the process hangs when creating a snapshot.
To solve the problem, you can combine snapshots by removing .vmdk from the backup system. Another option is to clone the latest snapshot disk to a new one. You can read more about these operations here .
Another frequently encountered problem when working with snapshots is an error that occurs when creating snapshots for ESXi / ESX virtual machines, which states that the file being created exceeds the maximum allowed sizes. It appears when the total size of the snapshot file and the additional space required for the operation exceeds the maximum file size allowed by the virtual machine data store.
To solve this problem, either change the location of the VM configuration files, or install workingDir on the data store with enough space for snapshots. WorkingDir creates snapshots, and by default this directory contains the .vmx configuration files. To move virtual machine disks and / or configuration files, use Storage vMotion or cold migration with file movement.
PS Stay tuned for new content on our IaaS blog, because in the following articles we will continue to look at options for solving common virtual environment problems based on the VMware hypervisor.
PPS What else do we write in our blog:
/ photo David Cruz CC
Clipboard problem: copy / paste does not work
Suppose the linux-base-01 virtual machine is used to transfer business-critical information from a research lab to the cloud using the clipboard. A typical task is when a copy operation is invoked on one side, and an paste is called on the other side. The system user found that the clipboard does not work correctly, therefore it does not allow copying data from a laptop to the VM directory.
To solve the problem, you need to open a browser and connect to the vSphere Web Client. In the control panel window that opens, after authentication, we refer to the user directory. The user provided us with a fragment of the study in the form of a file readme.txt. Now we need to solve the following tasks: restore the clipboard, copy the text from the user file to the desktop and paste it into the protocol.txt document of the linux-base-01 machine in the cloud.
')
Buffer usage can be enabled and disabled for a single virtual machine, as well as for all ESXi host machines. To activate the copy and paste features for a specific VM, you need to complete these steps:
- Make sure you have VMware Tools
- Connect to the vSphere Web Client
- Connect to vDC with linux-base-01
- Turn off the virtual machine, transferring to power off status
- Select a virtual machine and go to the Summary tab
- Go to the Edit Settings window
- Open Virtual Machine Options: VM Options> Advanced> Configuration Parameters> Edit Configuration
In the opened window we allow the use of copy / paste functions:
Then save the changes. Note that after this you need to restart the VM, transferring it to the status of power on. For more information on solving problems with the clipboard, you can use materials from the VMware knowledge base.
Lack of performance
Suppose that a large number of requests from users to perform various operations have accumulated in the queue. The employee believes that this was due to the poor performance of the virtual machine, which is unable to cope with the influx of requests. To solve this problem, you should first turn to the troubleshooting methodology to clearly identify the source.
Most often, performance problems are caused by disks (if you're interested, here is a good study on the subject of “disk-dependency”). While the network causes a similar much less. However, it is still necessary to conduct an analysis of the performance of the subsystems, identify bottlenecks, and draw up a plan for their elimination or optimization.
In our case, we first connect to the problematic virtual machine using terminal access. We assume that user statistics collection tools have already been installed on the user VM, so we see information about the host name, CPU load, RAM, IP address, etc.
We will also use the SPECjbb utility, which is a benchmark for measuring performance, created by the Standard Performance Evaluation (SPEC). The results obtained during testing and presented in the figure below indicate that the perf-01 system is experiencing performance problems.
This was caused by excessive use of processor resources. It turned out that on the esx-01a host (dual-core processor), in addition to perf-01a, another virtual machine is deployed - perf-02a. In this case, perf-01a uses 2vCPU, and perf-02a uses 1vCPU, which exceeds the number of available host cores and “clogs” the system.
You can solve the problem in several ways. Firstly, you can transfer perf-01a to a less loaded ESXi host, for example esx-02a, which is practically not loaded. Secondly, do the same, but move perf-02a. Thirdly, you can use the redistribution of computing resources for the perf-02a machine.
The set limit will reduce the load on the system. Also note that increasing the number of virtual processors can improve performance. For this, the Virtual Hardware tab has a special line for adding additional processor resources.
However, it must be remembered that if the host on which the virtual machine resides has only two physical processors, an increase in the number of CPUs can lead to cost overruns and reduced performance. You should also make sure that the operating system and applications can work in multi-threaded mode.
Configuring virtual machines with more virtual CPUs than is actually required for the workload can lead to an increase in resource utilization, potentially affecting the performance of the loaded systems.
Most of the guest operating systems operate idle during idle times, often following HLT or MWAIT instructions, while some operating systems use busy-waiting. All this leads to the consumption of resources that could be available for other tasks.
Denied access to the vSphere Web Client console
There are two reasons why a user may experience this problem:
- User account disabled administrator.
- The user has entered incorrect authorization data several times and the account has been blocked.
To establish the cause of the fault and eliminate it, we connect to the vSphere Web Client - for access we use an account with administrator rights. In the control window that opens, go to the Administration> Single Sign-On> Users and Groups tab, and then find the problematic user account. The image below shows that the Murphy account has the Disabled attribute, that is, it is disabled and cannot be used for access.
To eliminate the "fault", simply enable the account by calling the context menu and setting the Enable attribute.
Common Snapshot Issues
There are situations when you or users of the system cannot delete existing virtual machine snapshots. This problem manifests itself in the event that a backup server or another VM is blocking the basic disk or the previous snapshot file, making it impossible to perform the merge. This also happens if the process hangs when creating a snapshot.
To solve the problem, you can combine snapshots by removing .vmdk from the backup system. Another option is to clone the latest snapshot disk to a new one. You can read more about these operations here .
Another frequently encountered problem when working with snapshots is an error that occurs when creating snapshots for ESXi / ESX virtual machines, which states that the file being created exceeds the maximum allowed sizes. It appears when the total size of the snapshot file and the additional space required for the operation exceeds the maximum file size allowed by the virtual machine data store.
To solve this problem, either change the location of the VM configuration files, or install workingDir on the data store with enough space for snapshots. WorkingDir creates snapshots, and by default this directory contains the .vmx configuration files. To move virtual machine disks and / or configuration files, use Storage vMotion or cold migration with file movement.
PS Stay tuned for new content on our IaaS blog, because in the following articles we will continue to look at options for solving common virtual environment problems based on the VMware hypervisor.
PPS What else do we write in our blog:
Source: https://habr.com/ru/post/314590/
All Articles