Building Test Environments with OpenStack
Author: Evgeniya Schumacher
Evgenia Schumacher is a business analyst at Mirantis and the author of the proposal: How to Avoid Picking Problems that Can't Solve , Will your Cloud be Compliant? at the OpenStack Summit, which will be held this spring in Atlanta.
A survey of users OpenStack , conducted in October last year, showed that the construction of test environments included in the top 10 of the most popular options for using OpenStack. In this post we will talk about how test environments are built, analyze the ways in which OpenStack can simplify this process and make it more efficient.
')
Perhaps the most important requirement for the test environment is that it is a copy of the corresponding working environment (although in reality this is not always 100% true - see below). Any other requirements and restrictions arise from software use cases (software) and testing methodology.
A test suite that runs before commissioning can begin with functional tests and end with parallel testing.
Consider an approach to testing, which involves the following steps:
• run tests on a previous software version in the first test environment;
• perform tests on the new software version in the second test environment; and
• compare results.
If with this approach, the working version of the application does not work in a virtualized environment, but on baremetal, then a separate hardware cluster is required for each test environment. The hardware, network configuration, software settings and data (of course, impersonal) from the working environment should be mirrored in these two test environments. Ideally, both test environments should run on the same hardware and, if possible, have the same network configuration.
In most cases, system administrators are owners of such test environments — the testing team must create various applications to gain access to or change the test environment.
This approach has the following disadvantages:
• High cost of building test environments. Building a test environment based on hardware costs a lot of money, especially if the software being tested is critical and / or requires a large number of servers. And for two environments, these costs are doubled. And this is without taking into account the additional requirements that arise when testing multiple applications.
• Complexity of implementation. The two test environments should have similar configurations, be replicas of your working settings, and both should work well.
• Difficulty in scaling. What if you need another test environment for user acceptance testing or usability testing, for example? More resources, more time, more dependencies.
• The team of testers is not able to independently manage test environments - they are largely dependent on the IT team.
As key requirements for test environments, the following characteristics can be identified:
• Low construction costs;
• Easy and quick setup and operation;
• Easy and fast scaling;
• Possibility of self-service.
For those familiar with OpenStack and the benefits of this technology, it may seem that this is exactly what you need. OpenStack is a free, open source platform that can be deployed on low-cost general-purpose hardware. It allows you to pre-configure the components of test environments and virtual test environments entirely, and then quickly reproduce them at any desired scale. Finally, OpenStack has a self-service portal (Horizon), through which users can access resources on demand, in order to:
• create virtual servers (Nova);
• create / attach volumes for data storage (Cinder);
• configure networks (Neutron);
• build infrastructures for cloud applications (Heat).
Moving a test environment to the cloud requires a deep understanding of the application under test (s) and technical capabilities for:
• performing benchmarking base platforms and developing protocols that allow you to establish a correspondence between cloud-based performance and baremetal-based performance when they are quite different.
• adaptation of the cloud-based test environment to provide the necessary test coverage, despite performance constraints and other differences. For example, by adjusting the sizes of test databases, queries and other variables so that the cloud-based test environment works in a manner comparable to the baremetal test environment. This can make it possible to perform in a test environment based on a cloud of tests similar to those performed on a baremetal solution in many areas.
• adequate assessment of situations when the test environment does not model the working environment properly (as often happens during load testing).
It must be remembered that cloud technologies are developing very rapidly and there are more and more opportunities to configure the connection between virtual machines and the hardware on which they work. Before you write off the cloud from accounts, you need to check all the available possibilities.
If you are sure that your application will work properly, and that testing in the cloud will give meaningful results, then the cloud (and, at the same time, OpenStack) is the right choice.
Moving the test environment to the cloud changes the picture:
What is required to transfer the stack, on which the application runs, into a virtual environment?
Suppose you build an OpenStack cloud on existing hardware.
The first and most important thing to think about is the cloud architecture. Many aspects need to be taken into account. The more levels the stack has, the more carefully you need to be when designing each level.
There is no one correct way to build cloud test environments, but there are a few design tips that may be helpful.
Firstly, the requirement of high availability of the cloud is one of the main ones. You can read about how to build a highly available OpenStack cluster here: Mirantis OpenStack's implementation of HA ,
The next item is a cluster performance issue. On this topic, a post about ways to improve computing performance in cloud environments based on OpenStack will be helpful.
Other design solutions, such as:
• Network Configuration
o How many networks will you need?
o How will VM traffic be transmitted?
o Will internet access be required and how will it be provided?
• OpenStack component configuration
o What components need to be installed?
o Where should I install OpenStack services (on a compute node, on a controller node, on some other dedicated node, such as a data storage node)?
... depend on the characteristics of the software, data transfer methods and traffic restriction requirements (for example, high bandwidth, low latency, quality of service, response time) and the testing process itself. Try to answer the following questions:
• How are working data copied to the test environment?
• How often is the test environment created or updated?
As a result, the output will be a list of requirements that can be converted into design solutions.
Deciding what the architecture will be, you can begin preparing the images of virtual machines. It is necessary to compile:
• a set of universal images for vCPU, vRAM, vHDD, guest OS and some fixed programs that are installed and configured;
• a set of images for specific vCPU, vRAM, vHDD, possibly unusual operating systems and special programs that are installed and configured.
The goal here is to create a library of images to enable rapid deployment of a variety of test environments, the components of which are strictly configured, pre-documented and tested.
How to ensure the deployment of test environments?
Heat comes to the rescue, a component of OpenStack that is responsible for the orchestration of cloud applications.
Heat offers a template engine . The Heat template describes the infrastructure of a cloud application, for example, a set of servers, volumes and connections between them, network settings, incl. floating IP addresses, security groups, authentication settings, etc. You can use tools like Puppet or Chef to automatically configure the software.
For different types of test environments, one common heat pattern can be created, provided they have similar network settings and virtual cluster configurations. You can then add features that are specific to a specific virtual test environment, either using the custom Heat template or manually setting it up.
Moving test environments to the cloud gives you the opportunity to make the testing team the owners of their test environments: they can create, delete and manage them without IT intervention. In the meantime, the IT team typically manages a cluster of hardware and a cloud environment.
The above describes a general approach to building a virtual test environment. Now let's look at a specific use case.
The company develops applications for WEB.
The application uses the standard server stack LAMP. The working environment worked in a cloud on the basis of VMWare. This solution is relatively expensive due to the need for licensing and usage fees. The test environment was also built on VMWare and managed by the IT team.
The company identified a number of problems:
• A team of testers could not create on-demand local test environments.
• IT professionals took too long to complete requests to create new test environments.
• There were difficulties with scaling the test environment and / or adding a new one.
• The cost of building, maintaining, and scaling test environments was high (in part because of the cost of VMWare licenses).
The company wanted to change the approach to building test environments. The new solution had to meet the following requirements:
• Providing test engineers the ability to create and maintain a test environment through a self-service portal.
• Prompt creation and preparation of test environments.
• Easy scaling and support of test environments.
• Low total cost of solution.
• Allowing other users, such as product consultants, to create a demo environment through a self-service portal, since replicas of the work environment are useful not only to the team of testers.
• Isolation of different test environments.
• Providing the IT team with the ability to monitor the resource consumption of each team of testers.
The solution architect suggested building a test environment in the cloud based on OpenStack. As a result, an OpenStack cluster consisting of 10-20 nodes with a high degree of availability of controller nodes was deployed.
The resulting solution provides the following benefits:
• To ensure isolation, each test environment operates in a separate tenant. For example, one project for a team of testers, another for a team of consultants, etc. (See docs.openstack.org/trunk/openstack-ops/content/projects_users.html )
• Templates Heat allow you to quickly create new environments. A team of testers can create new Heat templates on their own.
• Self-service is provided through the OpenStack CLI / Horizon and Heat templates.
• The total cost of a solution based on OpenStack is lower because Companies do not have to pay for licenses and they can use inexpensive software and general-purpose hardware.
• Testers own their test environments. They can create and delete environments at any time, when necessary.
• OpenStack allows IT staff to set quotas for each project. The OpenStack component called Ceilometer can also be used to monitor resource consumption by cloud instances.
The process of building a test environment can be very complex. It is very important to consider what software is being tested; what testing methodology is used; who is responsible for supporting the test environment and much more.
Of course, not every test environment can be built on the basis of OpenStack and not every test can be performed in a cloud. OpenStack, like any other cloud operating system, has its limitations and problems. For example, when developing and executing performance tests, you need to take into account the performance limitations that it imposes.
Hardware-dependent tests (for example, failover and recovery testing) simply cannot be run in a virtual environment. Of course, even in this case, you can use hardware emulators, but how adequate are the results?
When deciding whether to transfer the test environment to the cloud based on OpenStack, it is important to carefully plan each step.
Original article in English .
Evgenia Schumacher is a business analyst at Mirantis and the author of the proposal: How to Avoid Picking Problems that Can't Solve , Will your Cloud be Compliant? at the OpenStack Summit, which will be held this spring in Atlanta.
A survey of users OpenStack , conducted in October last year, showed that the construction of test environments included in the top 10 of the most popular options for using OpenStack. In this post we will talk about how test environments are built, analyze the ways in which OpenStack can simplify this process and make it more efficient.
')
Current state of affairs
Perhaps the most important requirement for the test environment is that it is a copy of the corresponding working environment (although in reality this is not always 100% true - see below). Any other requirements and restrictions arise from software use cases (software) and testing methodology.
A test suite that runs before commissioning can begin with functional tests and end with parallel testing.
Consider an approach to testing, which involves the following steps:
• run tests on a previous software version in the first test environment;
• perform tests on the new software version in the second test environment; and
• compare results.
If with this approach, the working version of the application does not work in a virtualized environment, but on baremetal, then a separate hardware cluster is required for each test environment. The hardware, network configuration, software settings and data (of course, impersonal) from the working environment should be mirrored in these two test environments. Ideally, both test environments should run on the same hardware and, if possible, have the same network configuration.
In most cases, system administrators are owners of such test environments — the testing team must create various applications to gain access to or change the test environment.
This approach has the following disadvantages:
• High cost of building test environments. Building a test environment based on hardware costs a lot of money, especially if the software being tested is critical and / or requires a large number of servers. And for two environments, these costs are doubled. And this is without taking into account the additional requirements that arise when testing multiple applications.
• Complexity of implementation. The two test environments should have similar configurations, be replicas of your working settings, and both should work well.
• Difficulty in scaling. What if you need another test environment for user acceptance testing or usability testing, for example? More resources, more time, more dependencies.
• The team of testers is not able to independently manage test environments - they are largely dependent on the IT team.
And what if you use OpenStack?
As key requirements for test environments, the following characteristics can be identified:
• Low construction costs;
• Easy and quick setup and operation;
• Easy and fast scaling;
• Possibility of self-service.
For those familiar with OpenStack and the benefits of this technology, it may seem that this is exactly what you need. OpenStack is a free, open source platform that can be deployed on low-cost general-purpose hardware. It allows you to pre-configure the components of test environments and virtual test environments entirely, and then quickly reproduce them at any desired scale. Finally, OpenStack has a self-service portal (Horizon), through which users can access resources on demand, in order to:
• create virtual servers (Nova);
• create / attach volumes for data storage (Cinder);
• configure networks (Neutron);
• build infrastructures for cloud applications (Heat).
Is the cloud right for you?
Moving a test environment to the cloud requires a deep understanding of the application under test (s) and technical capabilities for:
• performing benchmarking base platforms and developing protocols that allow you to establish a correspondence between cloud-based performance and baremetal-based performance when they are quite different.
• adaptation of the cloud-based test environment to provide the necessary test coverage, despite performance constraints and other differences. For example, by adjusting the sizes of test databases, queries and other variables so that the cloud-based test environment works in a manner comparable to the baremetal test environment. This can make it possible to perform in a test environment based on a cloud of tests similar to those performed on a baremetal solution in many areas.
• adequate assessment of situations when the test environment does not model the working environment properly (as often happens during load testing).
It must be remembered that cloud technologies are developing very rapidly and there are more and more opportunities to configure the connection between virtual machines and the hardware on which they work. Before you write off the cloud from accounts, you need to check all the available possibilities.
If you are sure that your application will work properly, and that testing in the cloud will give meaningful results, then the cloud (and, at the same time, OpenStack) is the right choice.
Cloud architecture
Moving the test environment to the cloud changes the picture:
What is required to transfer the stack, on which the application runs, into a virtual environment?
Suppose you build an OpenStack cloud on existing hardware.
The first and most important thing to think about is the cloud architecture. Many aspects need to be taken into account. The more levels the stack has, the more carefully you need to be when designing each level.
There is no one correct way to build cloud test environments, but there are a few design tips that may be helpful.
Firstly, the requirement of high availability of the cloud is one of the main ones. You can read about how to build a highly available OpenStack cluster here: Mirantis OpenStack's implementation of HA ,
The next item is a cluster performance issue. On this topic, a post about ways to improve computing performance in cloud environments based on OpenStack will be helpful.
Other design solutions, such as:
• Network Configuration
o How many networks will you need?
o How will VM traffic be transmitted?
o Will internet access be required and how will it be provided?
• OpenStack component configuration
o What components need to be installed?
o Where should I install OpenStack services (on a compute node, on a controller node, on some other dedicated node, such as a data storage node)?
... depend on the characteristics of the software, data transfer methods and traffic restriction requirements (for example, high bandwidth, low latency, quality of service, response time) and the testing process itself. Try to answer the following questions:
• How are working data copied to the test environment?
• How often is the test environment created or updated?
As a result, the output will be a list of requirements that can be converted into design solutions.
Preparing virtual machine images
Deciding what the architecture will be, you can begin preparing the images of virtual machines. It is necessary to compile:
• a set of universal images for vCPU, vRAM, vHDD, guest OS and some fixed programs that are installed and configured;
• a set of images for specific vCPU, vRAM, vHDD, possibly unusual operating systems and special programs that are installed and configured.
The goal here is to create a library of images to enable rapid deployment of a variety of test environments, the components of which are strictly configured, pre-documented and tested.
Heat it!
How to ensure the deployment of test environments?
Heat comes to the rescue, a component of OpenStack that is responsible for the orchestration of cloud applications.
Heat offers a template engine . The Heat template describes the infrastructure of a cloud application, for example, a set of servers, volumes and connections between them, network settings, incl. floating IP addresses, security groups, authentication settings, etc. You can use tools like Puppet or Chef to automatically configure the software.
For different types of test environments, one common heat pattern can be created, provided they have similar network settings and virtual cluster configurations. You can then add features that are specific to a specific virtual test environment, either using the custom Heat template or manually setting it up.
Moving test environments to the cloud gives you the opportunity to make the testing team the owners of their test environments: they can create, delete and manage them without IT intervention. In the meantime, the IT team typically manages a cluster of hardware and a cloud environment.
Usage example
The above describes a general approach to building a virtual test environment. Now let's look at a specific use case.
The company develops applications for WEB.
The application uses the standard server stack LAMP. The working environment worked in a cloud on the basis of VMWare. This solution is relatively expensive due to the need for licensing and usage fees. The test environment was also built on VMWare and managed by the IT team.
The company identified a number of problems:
• A team of testers could not create on-demand local test environments.
• IT professionals took too long to complete requests to create new test environments.
• There were difficulties with scaling the test environment and / or adding a new one.
• The cost of building, maintaining, and scaling test environments was high (in part because of the cost of VMWare licenses).
The company wanted to change the approach to building test environments. The new solution had to meet the following requirements:
• Providing test engineers the ability to create and maintain a test environment through a self-service portal.
• Prompt creation and preparation of test environments.
• Easy scaling and support of test environments.
• Low total cost of solution.
• Allowing other users, such as product consultants, to create a demo environment through a self-service portal, since replicas of the work environment are useful not only to the team of testers.
• Isolation of different test environments.
• Providing the IT team with the ability to monitor the resource consumption of each team of testers.
The solution architect suggested building a test environment in the cloud based on OpenStack. As a result, an OpenStack cluster consisting of 10-20 nodes with a high degree of availability of controller nodes was deployed.
The resulting solution provides the following benefits:
• To ensure isolation, each test environment operates in a separate tenant. For example, one project for a team of testers, another for a team of consultants, etc. (See docs.openstack.org/trunk/openstack-ops/content/projects_users.html )
• Templates Heat allow you to quickly create new environments. A team of testers can create new Heat templates on their own.
• Self-service is provided through the OpenStack CLI / Horizon and Heat templates.
• The total cost of a solution based on OpenStack is lower because Companies do not have to pay for licenses and they can use inexpensive software and general-purpose hardware.
• Testers own their test environments. They can create and delete environments at any time, when necessary.
• OpenStack allows IT staff to set quotas for each project. The OpenStack component called Ceilometer can also be used to monitor resource consumption by cloud instances.
Conclusion
The process of building a test environment can be very complex. It is very important to consider what software is being tested; what testing methodology is used; who is responsible for supporting the test environment and much more.
Of course, not every test environment can be built on the basis of OpenStack and not every test can be performed in a cloud. OpenStack, like any other cloud operating system, has its limitations and problems. For example, when developing and executing performance tests, you need to take into account the performance limitations that it imposes.
Hardware-dependent tests (for example, failover and recovery testing) simply cannot be run in a virtual environment. Of course, even in this case, you can use hardware emulators, but how adequate are the results?
When deciding whether to transfer the test environment to the cloud based on OpenStack, it is important to carefully plan each step.
Original article in English .
Source: https://habr.com/ru/post/219001/
All Articles