Build solutions with IBM SmartCloud Enterprise. Part 1: Practical Recommendations and Tools

One of the goals of cloud computing is to provide cloud users with a rich choice of work resources; the other is to maintain a manageable amount of workload in solving support tasks for groups of image and software developers. It may seem like these are mutually exclusive goals, but this series of articles demonstrates how you can efficiently use and manage software package and image resources, making both of these goals achievable.

In this article:

• provides an overview of software package and image management concepts;
• explains the resource model;
• compares this model with other software package management platforms;
• discusses examples of practical application;
• available tools are being investigated.

The concepts are illustrated by the example of an electricity calculator application. We show how to create a simple solution with a very small amount of Python code, and give instructions for creating an RPM package. You will learn how to install this software package on a virtual machine in the cloud. Subsequent articles in this series will look at individual tools, APIs, and applications using the same sample application in more detail.
')
This article is intended for users and planners of cloud systems who want to learn how to effectively manage cloud resources on an ongoing basis. No programming knowledge is required. Although here are some very short scenarios, all the necessary explanations are given. The examples use Linux, but they are easy to adapt for Windows.

Preliminary notes

In the context of IBM SmartCloud Enterprise, a software package is a collection of software files, configuration files, and metadata that can be installed on a virtual machine instance.

In SmartCloud Enterprise, software packages can be created and administered using the IBM Image Construction and Composition Tool (ICCT), SmartCloud Enterprise REST-API and third-party tools.

The software for virtual machine images is usually pre-installed and configured by the image developer. The problem with this “sublimated product” approach is that it is difficult to find an image with the right set of software, which can lead to a do-it-yourself approach when choosing ready-made images of a virtual machine and consider them as the basis for installing your own software. This contributes to the reproduction of images of virtual machines, which in IT circles are called sprawl .

Another problem is that virtual machine images intended for use in specific cases are generally not well suited for other scenarios. SmartCloud Enterprise bridges the gap between the “sublimated products” approach and the “do-it-yourself” approach, using virtual machine parameters, activation scripts, and software packages for creating image templates.

Another more portable solution is the stand-alone IBM Image Construction and Composition Tool (ICCT), which allows software to be added to virtual machines. ICCT currently ships with IBM Workload Deployer 3.1, IBM SmartCloud Provisioning 1.2 and IBM SmartCloud Enterprise. ICCT stores images as Open Virtual Format Archive (OVA) packages that can be installed using IBM Workload Deployer 3.1, IBM SmartCloud Enterprise, and VMware products.

Virtual machine images can contain:

• Operating Systems;
• middleware;
• applications;
• user configurations.

As you add new features to your image, it becomes less universal; the less versatile a particular image, the more likely it is that others will be needed. However, the fewer functions added to an image, the less useful it is to cloud users.

Making decisions about what exactly to include in an image and how to make it a template are two fundamental tasks in the art of developing images.

SmartCloud Enterprise Resource Model

Figure 1 shows a simplified logical resource model.



Resource descriptions, attributes, and relationships are listed in Table 1.

Software package management systems

Software package management is an important aspect of managing computing systems, including cloud ones. Package management is specific to the cloud context, because it interacts with virtual machine instances that are dynamically created. This opens up possibilities for adding additional software “on the fly” and gives the user confidence that the system will “just work”.

Cloud package management systems have similar concepts and can include package management systems that come with operating systems and some software development platforms.

This section describes the software package management systems for Linux and their application to cloud systems. An example of creating a program module in the Python language, its binding as an RPM and its introduction into a cloud-based virtual machine at the time of preparation is given.

RPM and YUM Management

Red Hat Package Manager (RPM) was developed by Red Hat in the 1990s and is part of the Linux Standard Base. It has been migrated to other operating systems, such as IBM AIX:

• this helps maintain consistency between packages;
• can be used to obtain information about installed packages;
• checks packets for authenticity using a cryptographic signature;
• updates already installed packages;
• allows you to receive packages from the Internet.

There are several interfaces for RPM, including Yellow Dog Updater Modified (YUM), up2date, Zypper and Smart Package Manager. RPM is mainly used for Red Hat Enterprise Linux, CentOS, and Fedora. Debian-based systems, including SUSE and Ubuntu, use package management systems for Debian.

YUM was developed at Duke University as a Red Hat Linux systems management interface for RPM. YUM provides RPM with additional automatic update and dependency management features and works with central software package repositories. RPM "understands" dependencies, but does not resolve them. This is a task whose solution greatly facilitates YUM.

The package is installed with the rpm command with the -i option:

# rpm -i <package-name> 

If there are required dependencies, the rpm command will fail. You need root authority to install, remove, or upgrade packages.

To install a package using YUM, use the yum command with the install option. For example, here is the command to install dos2unix:

 # yum install dos2unix 

If there are dependencies, YUM will install them automatically.

Package Management for Debian

Debian systems, including Ubuntu and SUSE, use the Advanced Packaging Tool (APT) package management system. As in the case of RPM and YUM, package data is stored in a central database.

The package can be installed with apt-get install. Here is the command to install the dos2unix package:

 # apt-get install dos2unix 

apt-get finds information about the package in the remote repository, determines the dependencies, downloads the package and installs it.

Python modules

Python has built-in features for distributing and installing modules. In terms of installing and using Python software, these features are easily used by loading a module, unpacking it, and executing the setup command. Python is a convenient language for demonstrating software distribution and management. To understand this Python article, you do not need to know anything beyond what is explained in this section.

Before we begin, we will give some basic terms used when working with Python (see Table 2).



The command to install the python module:

 # python setup.py install 

This is usually done with root authority, and the module is installed in the {INSTALL_ROOT} // site-packages directory, for example, /usr/lib/python2.6/site-packages. Otherwise, it can be installed as a regular user in the home directory using the following command:

 $ python setup.py install --user 

Python contains Distutils toolkit that helps module developers distribute their software and makes these commands possible.

Sample Code: Python Distribution Module

Now we will create a Python module, an installation script and install it to check. The ecalc package is a power calculator and contains one energy_calculator module with the EnergyCalculator class. In this simple example, the code simply prints some sample data; however, the same ideas are easily applied to a more complex scenario where a highly organized software package management system is required.

EnergyCalculator class code:

 # energy_calculator.py """ energy_calculator module for computing energy usage """ class EnergyCalculator: """EnergyCalculator class""" def __init__(self): self._base = 200 def energy_used(self): return self._base 

Calling code from the electricity calculator client:

 #!/usr/bin/python # energy_client.py """ EnergyCalculator client """ from energy_calculator import EnergyCalculator ec = EnergyCalculator() print("You used {0} kJ today".format(ec.energy_used())) 

When called, an example displays:

 $ ./energy_client.py You used 200 kJ today 

First, create an installation script:

 # setup.py # Setup script for libenery from distutils.core import setup setup(name='libenergy', version='0.1', py_modules=['energy_calculator'], ) 

Then create a distribution module:

 $ python setup.py sdist 

The setup script command in the example above creates the source distribution package. A dist directory is created with the compressed libenergy-0.1.tar.gz file, which can be used to install the package. It does not include the energy_client.py client script. The script warns about the absence of information about the author, the URL and the README file. In addition, a manifest is created in the MANIFEST file, which is included in the TAR file. To copy the libenergy-0.1.tar.gz file to the target machine, use the secure copy command:

 $ scp -i ${key} libenergy-0.1.tar.gz idcuser@${vm}:libenergy-0.1.tar.gz 

This command assumes that you are working on a Linux machine. On Windows, everything is similar, but the scp command is required. She enters OpenSSH. It uses the SSH key $ {key} to copy the file to a virtual machine with a host name or the $ {vm} IP address of the idcuser type. To avoid multiple input of values, define the corresponding environment variables. Then you can deploy and install the package remotely by running the following command from the dist directory:

 $ ssh -i ${key} idcuser@${vm} 'gunzip libenergy-0.1.tar.gz; tar -xvf libenergy-0.1.tar; cd libenergy-0.1; sudo python setup.py install' 

This SSH script is executed remotely, unpacking the package and installing it as root with the sudo command. Remote commands are enclosed in apostrophes and separated by a semicolon.

Now that the energy_calculator module is installed on the target computer, you can use the client program to call it using the following script:

 $ scp -i ${key} energy_client.py idcuser@${vm}:energy_client.py $ ssh -i ${key} idcuser@${vm} './energy_client.py' You used 200 kJ today 

These commands first copy and then execute the client script. Run them from the directory containing the client script.

So, this is what we have at the moment:

• we wrote an example of a Python module and a client script for its execution;
• created an installation package for the module;
• remotely installed the module on the virtual machine using SSH;
• remotely called a module in a virtual machine using a client script.

Not bad, especially considering that until now we have written and used “our own” software. However, this is a simple example; here a few flaws become apparent.

1. The consumer must know how to install Python modules. I would like to be able to use the same method to install packages written in different languages.
2. Looking for it and, if necessary, editing is not convenient for anyone except programmers who know Python.
3. It does not describe the prerequisites, such as the required type and level of the operating system and Python.
4. In order to avoid the problem of sprawling images, I would like to install the package from the cloud console and deploy it into virtual machines. At the very least, have other options besides writing complex-looking bash scripts, such as in the example above.

It would be optimal to be able to enter the power calculator module into the virtual machine automatically during initialization or when it needs to be updated.

Installing a Python module using RPM

In Python, there are some additional tools that help you create installation packages for Red Hat Package Management (RPM), Windows, and other formats, excluding the disadvantages 1, 2, and 3 mentioned above. These formats are more convenient to use when writing activation and launch scripts.

This section shows how to write an activation script to verify that the electricity calculator is already installed. If it is not installed, you can install it with the rpm command.

Creating an RPM package:

 $ python setup.py bdist_rpm 

This command creates the RPM package libenergy-0.1-1.noarch.rpm in the dist directory. Copy it to the target virtual machine with the scp command:

 $ scp -i ${key} libenergy-0.1-1.noarch.rpm idcuser@${vm}:/tmp/libenergy-0.1-1.noarch.rpm 

Install the power calculator module:

 $ sudo rpm -i /tmp/libenergy-0.1-1.noarch.rpm 

The rpm command checks if it is installed and does not reinstall the package if it is already running.

Make sure the module is installed by running the electricity calculator client energy_client.py, as shown above. This RPM file and the following command will be used to enter the electricity calculator module into virtual machines when they are initialized in the cloud.

Clone an image using the SmartCloud Web portal

The manual process of creating software packages is not as simple as the process using tools like ICCT, but it is clearer for demonstration and is more amenable to automation using the SmartCloud Enterprise API. For these reasons, we will start with it.

One of the first necessary steps is to create an image to which you will have rights. This is necessary for editing metadata and adding files entered into virtual machines that will be initialized based on the image. There are two ways to do this.

1. Start the virtual machine instance and configure the necessary settings, including software installation.
2. Clone an existing image.

Image cloning is described here, because it is the most convenient approach for automating and creating image management tools.

To clone an image, follow these steps.

1. Click the Add Image button on the Control Panel | Images . The Base images window will open (see Figure 2).
   
   
2. Click Next . Step 2 of the wizard opens (see Figure 3).
   
   
3. Enter the name and description of the image being cloned. Check the data in step 3 of the wizard and click Next .
4. Agree to the terms in step 4 and click the Submit button. The cloning process will take a few minutes, then a control panel and a resource directory will appear.

The contents of the image resources in the resource directory look like Figure 4.



Among these files is the parameters.xml file, which describes the input parameters and the files that model the image topology. The most interesting files for us are located in the activation_scripts folder, as shown in Figure 5.

Introduction of installation scripts in the VM during initialization

The content of the cloned image from the previous sections in the resource directory looks like that shown in Figure 6.



Among these files is the parameters.xml file, which describes the input parameters and the files that model the image topology. The most interesting files for us are located in the activation_scripts folder, as shown in Figure 7.



The key to introducing files to the virtual machine during initialization is the scripts.txt file. Here is the scripts.txt file from the cloned RHEL 6 image:

 cloud-startup3.txt=/etc/init.d/cloud-startup3.sh activate.txt=/etc/cloud/activate.sh 

It causes the file cloud-startup3.txt to be entered into the VM as /etc/init.d/cloud-startup3.sh, and the activate.txt file as /etc/cloud/activate.sh.

To embed your own file in a virtual machine, follow these steps:

1. Download files from the activation_scripts folder, placing them in a folder with the same name.
2. Add the file libenergy-0.1-1.noarch.rpm to this folder.
3. Create a libenergy-activate.sh script:
   
   

    /usr/bin/logger 'installing libenergy-0.1' rpm -i /tmp/libenergy-0.1-1.noarch.rpm 2> /tmp/energy.log /usr/bin/logger < /tmp/energy.log 

4. Be careful with UNIX-style string delimiters or run the dos2unix utility. You need to copy this script to the virtual machine and execute it at startup. To do this, add it to the activation_scripts folder. Modify the scripts.txt file as shown below:
   
   

    cloud-startup3.txt=/etc/init.d/cloud-startup3.sh activate.txt=/etc/cloud/activate.sh libenergy-0.1-1.noarch.rpm=/tmp/libenergy-0.1-1.noarch.rpm libenergy-activate.sh=/home/idcuser/libenergy-activate.sh 

5. Call the script from the startup script in the cloud cloud-startup3.sh. Edit cloud-startup3.sh and add the lines in bold:
   
   

    start) echo "== Cloud Starting" if [ ! -e /etc/cloud/idcuser_pw_randomized ]; then echo "Randomizing idcuser password" echo idcuser:'< /dev/urandom tr -dc _A-Zaz-0-9 |head -c16' | /usr/sbin/chpasswd touch /etc/cloud/idcuser_pw_randomized fi /sbin/restorecon -R -v /home/idcuser/.ssh <b>chmod +x /home/idcuser/libenergy-activate.sh /home/idcuser/libenergy-activate.sh ;;</b> 

6. Squeeze the folder and call the resulting activation_scripts file. Edit the image resource for the cloned image and download the zip file. It is unpacked during the boot process.

To check the cloned and modified image, initialize the virtual machine based on the cloned image; make sure the module is installed by running the energy calculator energy_client.py client:

 $ scp -i ${key} energy_client.py idcuser@${vm}:energy_client.py $ ssh -i ${key} idcuser@${vm} './energy_client.py' You used 200 kJ today 

If you do not see the result of You used 200 kJ today, check the event log file / var / log / messages for an error message.

Now you know how to introduce a software module into a virtual machine without installing it on a running VM. You just started the virtual machine to check the result. This is an important step towards automating cloud image management and the foundation of several tools that help manage virtual machine images.

The following sections show how to use some of these tools and how to automate the process more efficiently using the API. Note also that we have not created an independent software package resource in the cloud; we simply combined the RPM installation file and the installation script with the metadata of the virtual image in the resource repository. .

, SmartCloud Asset Catalog. Asset Catalog Composable Software Bundles ( 8). .



, . Asset Manager Clone , . , , . , ICCT .

ICCT

ICCT IBM SmartCloud Enterprise, . , SmartCloud Enterprise.

1. Instance Add Instance .
2. , ICCT .
3. ICCT . Getting Started .
4. ICCT, Build and Manage Software Bundles .
5. Enablement Bundle for SmartCloud Enterprise . ICCT API SmartCloud Enterprise, , .

Resources

: Create solutions on IBM SmartCloud Enterprise: Best practices and tools .

, :

Python ;
Linux, 101: RPM YUM ;
RPM Package Manager ;
- RPM Fedora (Red Hat) ;
Linux, 101: Debian :
( ICCT) .

IBM SmartCloud Enterprise:

Windows ;
Web- IIS Windows 2008 R2 ;
IBM Cloud Linux ;
IBM Cloud Windows ;
IBM Cloud ;
IBM ;
;
Windows- IBM Cloud ;
;
- ;
Linux ;
;
, VLAN ;
IBM Cloud Android- ;
IBM SmartCloud Enterprise ;
.

About the authors

, IBM SmartCloud Enterprise ; , , .

:

• IBM SmartCloud Enterprise;
• IBM SmartCloud Enterprise;
• IBM SmartCloud Enterprise ICCT;
• API IBM SmartCloud Enterprise;
• IBM SmartCloud Enterprise CohesiveFT;
• IBM Workload Deployer IBM SmartCloud Enterprise;
• ;
• IBM SmartCloud Enterprise;
• IBM SmartCloud Enterprise.

: IBM developerWorks