How to make a fuel plugin (for example, NFS). Part 2
Characters and performers:
Alexander Kislitsky - the author of the code,
Evgenia Schumacher, Irina Povolotskaya - useful notes and links,
Vyacheslav Struk - review,
Ilya Stechkin - narrator
Important note: the plug-in world is developing very quickly. What is described below is relevant for plugins created under MOS 6.0 . Recall that MOS is Mirantis OpenStack, our distribution, open, like everything we do. However, last week it was officially announced the official release of MOS 7.0 , and this is a completely different story, which we will also try to tell. But another time. For lazy copy-pasters, we immediately offer a link to GitHub and remind you that laziness is the engine of progress.
So, go to the second part and finally talk about how the specific NFS plugin was made. In our case, the NFS server that was used as a backend for Cinder is already installed. If you are less fortunate, we advise you to read the corresponding manual . His retelling is not included in our plans - so the text was quite voluminous. But in our plans is to share the experience of creating a plug-in and provide you with useful links to additional materials of direct relevance to the topic. Yes, by the way, we are sometimes accused of having too many common words in the texts. This time we tried to correct. Go!
')
To build your plugin, you first need to create a virtual development environment: create a directory, activate it and install the Fuel plugin builder :
The trick is that you do not need to be root on the server to build a new plugin in this virtual development environment.
Now use the Fuel plugin builder, start working on the plugin with the following command in order to generate the future skeleton of the plugin:
As a result, you will receive a set of templates in which the developer can then correct the value of specific fields. For more information on how the Fuel plugin builder works, we recommend that you study the Fuel Plugins wiki .
Set the plugin definition in metadata.yaml as shown below. Here we use Fuel 6.0 with plugins in HA and multinode modes.
The most detailed information on how this is done is available in a document called the OpenStack Cloud Administrator Guide .
Important! Only plugins created in Mirantis OpenStack 6.0 and above can be validated. We wrote about the validation and why it is needed in the first part of this epochal narration. And how to pass validation - we will tell at the end of this text.
And now a few paragraphs for those who have no guides in the subject:
Now download the packages required for NFS Ubuntu and CentOS and move them to the external_nfs / repositories / ubuntu and external_nfs / repositories / centos directories, respectively, to add a task to install the package.
Save the packages to deployment_scripts / install_packages.sh and add the executable permissions for the “chmod + x install_packages.sh” script:
Next, add a shell shell to install the packages. Our tasks.yaml install_packages.sh script file, which runs after deployment on the Cinder node, looks like this:
Now add the checkbox and specific elements for the Settings tab of the Fuel interface. To do this, describe the UI in environment_config.yaml according to the following parameters :
1. NFS shares file
2. NFS mount options
3. Is NFS sparse volumes are used
Add the plugin interface description as shown in the following example:
Puppet manifests present tasks as described in the OpenStack Cloud Administrator Guide and send them to the deployment_scripts / puppet directory on GitHub.
After applying the puppet manifests, notice the permissions in the / etc / nfsshares files and run the following command to restart the Cinder service:
Puppet apply external_nfs / deployment_scripts / puppet / modules / cindernfs / manifests / backend / nfs.pp.
Now add the task to apply the manifests to tasks.yaml:
Let's combine pre_build_hook with the appropriate version of fuel-library. In our case, it looked like this:
Now you can build your package as follows:
The package will be saved as external_nfs / external_nfs-1.0.0.fp, and now you can deploy the module using VirtualBox, as shown in the Mirantis Quick Start manual.
Important! Don't forget to take snapshots of your virtual machines after deploying OpenStack so that you can come back if something goes wrong.
So, the plugin is created. Copy it to the Fuel Master node and install it for deployment:
Make sure the plugin is installed:
After installation, you can add the environment to the Fuel interface and configure the NFS plugin in the Settings tab. You can also add nodes and check the plugin. Before we proceed to testing, let's look at the procedure for diagnosing and troubleshooting possible problems.
Here, for example, a frequently encountered situation: the file exists, but cannot be found. For example, let's assume that you did not specify the path for the install_packages.sh script:
In this case, deploy cloud will not succeed, the system will give the following error:
It is assumed that in this case you want to look at the logs that will be located on /var/log/docker-logs/astute.log on the Fuel master node, and in our particular case will look like this:
This shows that the install_packages.sh path to tasks.yaml is incorrect. You can fix it like this:
After making corrections, rebuild the package and update it on the Fuel Master node:
Having solved this problem, you can create a new environment, configure and deploy cloud. After a successful deployment, make sure that Cinder works properly with the NFS backend. This is easy to do:
1. Login to the Controller and create a cinder volume:
2. Check the volume status through the Fuel CLI. For more information, see the Mirantis OpenStack User Guide .
Cinder-list
3. Verify that the ID from the cinder list output created on NFS is being scanned as a file with the name corresponding to the volume-Id value:
If you want to fix the Puppet manifests on the Cinder node, you can find them in the /etc/fuel/plugins/external_nfs-1.0.0/puppet directory
Additional information is available here .
After creating the plugin, any of the options are available to you:
1. Deploy and use the plugin yourself, without any contact with the Mirantis team. This scenario has two minuses: your cloud will not be supported by our support and, therefore, you act at your own peril and risk. Well, the community will not know about your feat, will not be able to use the results of your work and appreciate the functionality of the created plug-in, since the plug-in will not be published in our catalog .
2. Place plugin for MOS without validation. That is, you allow other people who download Mirantis OpenStack to download your plugin as well, but this installation will not be supported by the Mirantis team. To do this, it is enough to lay out the entry of the plugin on the driverlog .
3. Post the plug-in, become Mirantis partners (that is, meet the specific requirements of the Mirantis Unlocked project) and send a request for validation , certify it, and IF the plug-in quality is confirmed during the validation procedure, we will publish it in the catalog and will support MOS cloud with this plugin.
The validation process is optional and takes some time, but validated plugins are added in the Fuel plugin catalog and supported by Mirantis support. And this is important, especially for large enterprises that do not have a large number of OpenStack specialists on their staff.
If you do decide to validate your plugin , we will now briefly describe the procedure. But before this we recall once again that you must be a partner of Mirantis. Any integration of plug-ins (and validation is one of the steps towards integration with MOS) is possible only as part of an affiliate program.
First, you need to provide a specific set of documents to Mirantis:
- The project specification (or “spec,” as this document is called in a get-together),
- Installation and Deployment Guide for the plugin,
- User's manual,
- Test plan,
- Test results.
Details on how to structure the manuals can be found here .
Secondly, the certification process itself consists of the following steps:
- Review of project specs,
- Review code
- Testing plan review,
- Review plugin deployment instructions,
- User Guide Review,
- Analysis of test results submitted by the developer
- Own testing of the plugin team Mirantis.
Important: you can work with the plugin without Fuel, and certification is not a mandatory procedure at all. It is required if you want to guarantee the support of your MOS-cloud with the author plugin by the Mirantis team. However, it seems, we repeat. But this is really important!
We showed you how to create a plugin using the example of NFS, but you can create a plugin that meets your unique needs. Fuel flexibility allows you to comfortably deploy and manage your cloud. In addition, if you are not too lazy to undergo the certification procedure, then get the support of Mirantis specialists for your plugin.
When we talk about support, we are talking about the fact that the support of the plug-in itself - its code - is carried out by the developers, and the Mirantis team can support MOS-clauses with validated plug-ins.
By creating plugins for Fuel, you help expand the functionality of the OpenStack ecosystem. Chewing food thoroughly - helping the community!
Alexander Kislitsky - the author of the code,
Evgenia Schumacher, Irina Povolotskaya - useful notes and links,
Vyacheslav Struk - review,
Ilya Stechkin - narrator
Important note: the plug-in world is developing very quickly. What is described below is relevant for plugins created under MOS 6.0 . Recall that MOS is Mirantis OpenStack, our distribution, open, like everything we do. However, last week it was officially announced the official release of MOS 7.0 , and this is a completely different story, which we will also try to tell. But another time. For lazy copy-pasters, we immediately offer a link to GitHub and remind you that laziness is the engine of progress.
So, go to the second part and finally talk about how the specific NFS plugin was made. In our case, the NFS server that was used as a backend for Cinder is already installed. If you are less fortunate, we advise you to read the corresponding manual . His retelling is not included in our plans - so the text was quite voluminous. But in our plans is to share the experience of creating a plug-in and provide you with useful links to additional materials of direct relevance to the topic. Yes, by the way, we are sometimes accused of having too many common words in the texts. This time we tried to correct. Go!
')
Creating a development environment
To build your plugin, you first need to create a virtual development environment: create a directory, activate it and install the Fuel plugin builder :
$ mkdir plugins $ cd plugins $ virtualenv .venv $ source .venv/bin/activate $ pip install fuel-plugin-builder The trick is that you do not need to be root on the server to build a new plugin in this virtual development environment.
Creating a plugin
Now use the Fuel plugin builder, start working on the plugin with the following command in order to generate the future skeleton of the plugin:
$ fpb -create external_nfs
As a result, you will receive a set of templates in which the developer can then correct the value of specific fields. For more information on how the Fuel plugin builder works, we recommend that you study the Fuel Plugins wiki .
Plugin backend
Set the plugin definition in metadata.yaml as shown below. Here we use Fuel 6.0 with plugins in HA and multinode modes.
The most detailed information on how this is done is available in a document called the OpenStack Cloud Administrator Guide .
Important! Only plugins created in Mirantis OpenStack 6.0 and above can be validated. We wrote about the validation and why it is needed in the first part of this epochal narration. And how to pass validation - we will tell at the end of this text.
And now a few paragraphs for those who have no guides in the subject:
$ Plugin name name: external_nfs title: External NFS backend for Cinder $ Plugin version version: 1.0.0 $ Description description: External NFS backend for Cinder $ Required fuel version fuel_version: ['6.0'] $ The plugin is compatible with releases in the list releases: - os: ubuntu version: 2014.2-6.0 mode: ['ha', 'multinode'] deployment_scripts_path: deployment_scripts/ repository_path: repositories/ubuntu - os: centos version: 2014.2-6.0 mode: ['ha', 'multinode'] deployment_scripts_path: deployment_scripts/ repository_path: repositories/centos $ Version of plugin package package_version: '1.0.0'
Add shell task
Now download the packages required for NFS Ubuntu and CentOS and move them to the external_nfs / repositories / ubuntu and external_nfs / repositories / centos directories, respectively, to add a task to install the package.
Save the packages to deployment_scripts / install_packages.sh and add the executable permissions for the “chmod + x install_packages.sh” script:
$ !/bin/bash OS_NAME='' if grep -i CentOS /etc/issue.net >/dev/null; then OS_NAME='centos'; elif grep -i Ubuntu /etc/issue.net >/dev/null; then OS_NAME='ubuntu'; fi function install_package { if [ $OS_NAME == 'ubuntu' ]; then apt-get install -y rpcbind nfs-common libevent-2.0 \ libgssglue1 libnfsidmap2 libtirpc1 elif [ $OS_NAME == 'centos' ]; then yum install -y rpcbind nfs-utils nfs-utils-lib libevent \ key-utils libtirpc libgssglue fi } install_package
Next, add a shell shell to install the packages. Our tasks.yaml install_packages.sh script file, which runs after deployment on the Cinder node, looks like this:
- role: ['cinder'] stage: post_deployment type: shell parameters: cmd: ./install_packages.sh timeout: 180
Frontend plugin
Now add the checkbox and specific elements for the Settings tab of the Fuel interface. To do this, describe the UI in environment_config.yaml according to the following parameters :
1. NFS shares file
2. NFS mount options
3. Is NFS sparse volumes are used
Add the plugin interface description as shown in the following example:
attributes: endpoint: value: '' label: 'NFS endpoints' description: 'comma separated HOST:SHARE values' weight: 25 type: "text" mount_options: value: '' label: 'NFS mount options' description: 'optional NFS mount parameters' weight: 25 type: "text" nfs_sparsed_volumes: type: "checkbox" weight: 30 value: false label: "NFS sparsed volumes" description: ""
Adding puppet puppies
Puppet manifests present tasks as described in the OpenStack Cloud Administrator Guide and send them to the deployment_scripts / puppet directory on GitHub.
After applying the puppet manifests, notice the permissions in the / etc / nfsshares files and run the following command to restart the Cinder service:
Puppet apply external_nfs / deployment_scripts / puppet / modules / cindernfs / manifests / backend / nfs.pp.
Now add the task to apply the manifests to tasks.yaml:
- role: ['cinder'] stage: post_deployment type: puppet parameters: puppet_manifest: puppet/site.pp puppet_modules: puppet/modules/:/etc/puppet/modules timeout: 360
Plugin build
Let's combine pre_build_hook with the appropriate version of fuel-library. In our case, it looked like this:
#!/bin/bash set -eux ROOT="$(dirname `readlink -f $0`)" MODULES="${ROOT}"/deployment_scripts/puppet/modules TMP_DIR="${ROOT}"/tmp mkdir -p "${MODULES}" mkdir -p "${TMP_DIR}" REPO_PATH='https://github.com/stackforge/fuel-library/tarball/f43d885914d74fbd062096763222f350f47480e1' wget -qO- "${REPO_PATH}" | \ tar -C "${MODULES}" --strip-components=3 -zxvf - \ stackforge-fuel-library-f43d885/deployment/puppet/{inifile,stdlib}
Now you can build your package as follows:
fpb --build external_nfs
The package will be saved as external_nfs / external_nfs-1.0.0.fp, and now you can deploy the module using VirtualBox, as shown in the Mirantis Quick Start manual.
Important! Don't forget to take snapshots of your virtual machines after deploying OpenStack so that you can come back if something goes wrong.
Deploying the plugin
So, the plugin is created. Copy it to the Fuel Master node and install it for deployment:
scp external_nfs/external_nfs-1.0.0.fp root@master-node: ssh root@master-node fuel plugins --install external_nfs-1.0.0.fp
Make sure the plugin is installed:
$ fuel plugins
After installation, you can add the environment to the Fuel interface and configure the NFS plugin in the Settings tab. You can also add nodes and check the plugin. Before we proceed to testing, let's look at the procedure for diagnosing and troubleshooting possible problems.
troubleshooting
Here, for example, a frequently encountered situation: the file exists, but cannot be found. For example, let's assume that you did not specify the path for the install_packages.sh script:
- role: ['cinder'] stage: post_deployment type: shell parameters: cmd: install_packages.sh timeout: 180
In this case, deploy cloud will not succeed, the system will give the following error:
Deployment has failed. Method deploy. Failed to deploy plugin external_nfs-1.0.0. Inspect Astute logs for the details.
It is assumed that in this case you want to look at the logs that will be located on /var/log/docker-logs/astute.log on the Fuel master node, and in our particular case will look like this:
2014-12-23T16:31:54 debug: [395] c5d1e2b5-56bb-4428-9666-8c5574cb06a4: MC agent 'execute_shell_command', method 'execute', results: {:sender=>"3", :statuscode=>0, :statusmsg=>"OK", :data=>{:stdout=>"", :exit_code=>127, :stderr=>"sh: 1: install_packages.sh: not found\n"}} 2014-12-23T16:31:54 debug: [395] c5d1e2b5-56bb-4428-9666-8c5574cb06a4: cmd: cd /etc/fuel/plugins/external_nfs-1.0.0/ && install_packages.sh cwd: /etc/fuel/plugins/external_nfs-1.0.0/ stdout: stderr: sh: 1: install_packages.sh: not found
This shows that the install_packages.sh path to tasks.yaml is incorrect. You can fix it like this:
- role: ['cinder'] stage: post_deployment type: shell parameters: cmd: ./install_packages.sh timeout: 180
After making corrections, rebuild the package and update it on the Fuel Master node:
fpb --build external_nfs scp external_nfs/external_nfs-1.0.0.fp root@master-node: ssh root@master-node fuel plugins -- install --force external_nfs-1.0.0.fp
Checking results
Having solved this problem, you can create a new environment, configure and deploy cloud. After a successful deployment, make sure that Cinder works properly with the NFS backend. This is easy to do:
1. Login to the Controller and create a cinder volume:
ssh root@master-node ssh controller-node source openrc cinder create 1
2. Check the volume status through the Fuel CLI. For more information, see the Mirantis OpenStack User Guide .
Cinder-list
3. Verify that the ID from the cinder list output created on NFS is being scanned as a file with the name corresponding to the volume-Id value:
nfs-server$ ls volume-d9382a2b-fd13-424b-9eba-3a77a1d16008
If you want to fix the Puppet manifests on the Cinder node, you can find them in the /etc/fuel/plugins/external_nfs-1.0.0/puppet directory
ssh root@master-node ssh cinder-node cd /etc/fuel/plugins/external_nfs-1.0.0/ puppet apply --modulepath=/etc/fuel/plugins/external_nfs-1.0.0/puppet/modules site.pp
Additional information is available here .
Plugin release
After creating the plugin, any of the options are available to you:
1. Deploy and use the plugin yourself, without any contact with the Mirantis team. This scenario has two minuses: your cloud will not be supported by our support and, therefore, you act at your own peril and risk. Well, the community will not know about your feat, will not be able to use the results of your work and appreciate the functionality of the created plug-in, since the plug-in will not be published in our catalog .
2. Place plugin for MOS without validation. That is, you allow other people who download Mirantis OpenStack to download your plugin as well, but this installation will not be supported by the Mirantis team. To do this, it is enough to lay out the entry of the plugin on the driverlog .
3. Post the plug-in, become Mirantis partners (that is, meet the specific requirements of the Mirantis Unlocked project) and send a request for validation , certify it, and IF the plug-in quality is confirmed during the validation procedure, we will publish it in the catalog and will support MOS cloud with this plugin.
The validation process is optional and takes some time, but validated plugins are added in the Fuel plugin catalog and supported by Mirantis support. And this is important, especially for large enterprises that do not have a large number of OpenStack specialists on their staff.
Validation of the plugin
If you do decide to validate your plugin , we will now briefly describe the procedure. But before this we recall once again that you must be a partner of Mirantis. Any integration of plug-ins (and validation is one of the steps towards integration with MOS) is possible only as part of an affiliate program.
First, you need to provide a specific set of documents to Mirantis:
- The project specification (or “spec,” as this document is called in a get-together),
- Installation and Deployment Guide for the plugin,
- User's manual,
- Test plan,
- Test results.
Details on how to structure the manuals can be found here .
Secondly, the certification process itself consists of the following steps:
- Review of project specs,
- Review code
- Testing plan review,
- Review plugin deployment instructions,
- User Guide Review,
- Analysis of test results submitted by the developer
- Own testing of the plugin team Mirantis.
Important: you can work with the plugin without Fuel, and certification is not a mandatory procedure at all. It is required if you want to guarantee the support of your MOS-cloud with the author plugin by the Mirantis team. However, it seems, we repeat. But this is really important!
Instead of a conclusion: extend the capabilities of OpenStack using plugins
We showed you how to create a plugin using the example of NFS, but you can create a plugin that meets your unique needs. Fuel flexibility allows you to comfortably deploy and manage your cloud. In addition, if you are not too lazy to undergo the certification procedure, then get the support of Mirantis specialists for your plugin.
When we talk about support, we are talking about the fact that the support of the plug-in itself - its code - is carried out by the developers, and the Mirantis team can support MOS-clauses with validated plug-ins.
By creating plugins for Fuel, you help expand the functionality of the OpenStack ecosystem. Chewing food thoroughly - helping the community!
Source: https://habr.com/ru/post/268315/
All Articles