Puppet Virtual Resources
It seems to me that the basic meaning of vitrual resources becomes clearer already with specific examples of exported resources — when virtual resources are placed in the database and used to exchange information between agents, but in order to understand recursion, you need to understand recursion , so let's start with a local application. For example.
The example will be a bit synthetic. It was difficult for me to come up with a rather short example, while demonstrating the meaning of virtual resources. In practice, such examples with sewn-in user names are rare. At least they should.
There is a server with Apache installed. Installation and configuration is made conveniently and fashionably puppet-class apache. For simplicity, everything will be stored in the main manifest site.pp. All emerging problems during the development of the example are relevant in the case of the separation of pieces of logic on the modules.
Suppose a class needs a unix user, in this example a webUser whose home directory will be the document root for the web. Then we get the following site.pp skeleton:
')
It's simple. Now we have decided to add nginx to our infrastructure no matter for what purposes. The main thing is that he also needs a webUser user to upload content. Add a class:
Run:
For obvious reasons, it does not work. It turns out that in the same scope we have two resources with the same name name value. You can solve the problem, for example, by taking out the user's resource in a separate class:
Run - works:
Suppose that we needed to add a new user, cacheUser , in the folder of which we will store a cache. Both Apache and nginx use this cache, so we add the corresponding user to the users class:
Next, we decided to add php5-fpm and uwsgi, which need webUser , but do not need cacheUser . In such a situation, it is necessary to allocate cacheUser to a separate class in order to connect it separately only in the classes apache and nginx. It is not comfortable. In addition, there are no guarantees that a little later it is not necessary to allocate another user to a separate class. This is where virtual resources come to the rescue.
If you add an @ to the definition of a resource:
The resource will be considered virtual. Such a resource will not be added to the agent directory until we explicitly determine. From the documentation:
Therefore, if you execute the code below even if there are no webUser and cacheUser users in the system, they will not be added:
Checking:
Users, as expected, were not added.
But be careful. Although the virtual resource is not added to the directory, this does not mean that the following code will work:
It will still produce a compilation error. This happens because, at first, the puppet parser iterates over all resources, adding even vitrual to the directory. At this stage, the stage and an error occurs due to duplication of names. The next step is the processing of the implementation of virtual types: the collector searches the directory for places in which virtual resources are defined and the found marks as non-virtual. And only at the very end there is a cleanup of the directory from virtual resources that would not be implemented.
To determine the resource, use either the spaceship operator <| | > either by using realize function. Rewrite our manifest using one or the other syntax:
You can transfer several resources to realize at once, and <| |> You can specify several conditions by which a search is made for the resources to be determined.
Besides the syntactic difference in realize and <| |> there are differences in behavior. If the resource with the specified name does not exist, realize will generate an error:
Operator <| |> in this case does not give an error, because it is a kind of superstructure over the function realize . The function realize is applied to all found resources according to the search query specified in its body. Accordingly, if no resource was found for the specified error criteria does not occur, since the function realize is not called.
By the way, the operator <| |> There are still two fairly good uses. It can be used to override the status of a resource in a class. For example:
Exclude the /etc/apache2.con f file for the s2.example.com node.
It can also be used with the ~> and -> operators. Thus, we can notify all services about any changes, or request to add all yum repositories before installing any package:
It seems to me that the main advantage of virtual resources is that they can be exported and made available to other agents. To export a virtual resource, you need to add another @ sign before its description.
A classic example from the Puppet documentation:
In this example, we have defined the sshkey virtual resource. Operator-collector << | | >> contains an empty body, so it unloads all exported objects of the Sshkey class. Thus, any agent in the manifest of which the ssh class is connected exports its public key ( @@ sshkey ) and then imports all the keys added by other agents ( Sshkey << | | >> ) to itself.
Exported resources are stored in PuppetDB - DB from PuppetLabs. After connecting PuppetDB, each folder copied by the master master is put into the PuppetDB database, which in turn provides a search interface for searching through catalogs.
By specifying @@ , we mark the resource as exportable and inform puppet that the resource must be added to the directory and tag it exported . When the puppet master sees the operator << | | >> , it makes a search query to PuppetDB and adds all the exported resources found that match the search criteria.
It is important that the exported resources are in the global scope, so their names must be unique.
This functionality has a huge potential and I often have to use it. Automation of adding servers to monitoring or nginx backends.
It is better to use existing modules, but to demonstrate the principle, this example will do:
More information about syntax and usage patterns can be found at the following links:
The example will be a bit synthetic. It was difficult for me to come up with a rather short example, while demonstrating the meaning of virtual resources. In practice, such examples with sewn-in user names are rare. At least they should.
There is a server with Apache installed. Installation and configuration is made conveniently and fashionably puppet-class apache. For simplicity, everything will be stored in the main manifest site.pp. All emerging problems during the development of the example are relevant in the case of the separation of pieces of logic on the modules.
Suppose a class needs a unix user, in this example a webUser whose home directory will be the document root for the web. Then we get the following site.pp skeleton:
')
class apache { user { 'webUser' : ensure => present } ... } node default { include apache } It's simple. Now we have decided to add nginx to our infrastructure no matter for what purposes. The main thing is that he also needs a webUser user to upload content. Add a class:
class apache { user { 'webUser' : ensure => present } } class nginx { user { 'webUser' : ensure => present } } node default { include apache include nginx } Run:
root@puppet:/vagrant# puppet apply ./site.pp --noop Error: Duplicate declaration: User[webUser] is already declared in file /vagrant/site.pp:17; cannot redeclare at /vagrant/site.pp:11 on node puppet.example.com Error: Duplicate declaration: User[webUser] is already declared in file /vagrant/site.pp:17; cannot redeclare at /vagrant/site.pp:11 on node puppet.example.com For obvious reasons, it does not work. It turns out that in the same scope we have two resources with the same name name value. You can solve the problem, for example, by taking out the user's resource in a separate class:
class users { user { 'webUser' : ensure => present } } class nginx { include users } class apache { include users } node default { include apache include nginx } Run - works:
root@puppet:/vagrant# puppet apply ./site.pp --noop Notice: Compiled catalog for puppet.example.com in environment production in 0.07 seconds Notice: /Stage[main]/users/User[webUser]/ensure: current_value absent, should be present (noop) Notice: Class[users]: Would have triggered 'refresh' from 1 events Notice: Stage[main]: Would have triggered 'refresh' from 1 events Notice: Finished catalog run in 0.02 seconds Suppose that we needed to add a new user, cacheUser , in the folder of which we will store a cache. Both Apache and nginx use this cache, so we add the corresponding user to the users class:
class users { user { 'webUser': ensure => present } user { 'cacheUser': ensure => present } } Next, we decided to add php5-fpm and uwsgi, which need webUser , but do not need cacheUser . In such a situation, it is necessary to allocate cacheUser to a separate class in order to connect it separately only in the classes apache and nginx. It is not comfortable. In addition, there are no guarantees that a little later it is not necessary to allocate another user to a separate class. This is where virtual resources come to the rescue.
If you add an @ to the definition of a resource:
@user { 'webUser': ensure => present } The resource will be considered virtual. Such a resource will not be added to the agent directory until we explicitly determine. From the documentation:
The virtual resource declaration
Therefore, if you execute the code below even if there are no webUser and cacheUser users in the system, they will not be added:
class users { @user { 'webUser': ensure => present } @user { 'cacheUser': ensure => present } } class nginx { include users } class apache { include users } node default { include apache include nginx } Checking:
root@puppet:/vagrant# puppet apply ./site.pp Notice: Compiled catalog for puppet.example.com in environment production in 0.07 seconds Notice: Finished catalog run in 0.02 seconds Users, as expected, were not added.
But be careful. Although the virtual resource is not added to the directory, this does not mean that the following code will work:
class apache { @user { 'webUser' : ensure => present } } class nginx { @user { 'webUser' : ensure => present } } node default { include apache include nginx } It will still produce a compilation error. This happens because, at first, the puppet parser iterates over all resources, adding even vitrual to the directory. At this stage, the stage and an error occurs due to duplication of names. The next step is the processing of the implementation of virtual types: the collector searches the directory for places in which virtual resources are defined and the found marks as non-virtual. And only at the very end there is a cleanup of the directory from virtual resources that would not be implemented.
To determine the resource, use either the spaceship operator <| | > either by using realize function. Rewrite our manifest using one or the other syntax:
class users { @user { 'webUser': ensure => present } @user { 'cacheUser': ensure => present } } class nginx { include users realize User['webUser'], User['cacheUser'] } class apache { include users User <| title == 'webUser' or title == 'cacheUser' |> } node default { include apache include nginx } You can transfer several resources to realize at once, and <| |> You can specify several conditions by which a search is made for the resources to be determined.
Besides the syntactic difference in realize and <| |> there are differences in behavior. If the resource with the specified name does not exist, realize will generate an error:
Error: Failed to realize virtual resources User[nonExistingUser] on node puppet.example.com Operator <| |> in this case does not give an error, because it is a kind of superstructure over the function realize . The function realize is applied to all found resources according to the search query specified in its body. Accordingly, if no resource was found for the specified error criteria does not occur, since the function realize is not called.
By the way, the operator <| |> There are still two fairly good uses. It can be used to override the status of a resource in a class. For example:
class configurations { file { '/etc/nginx.conf' : ensure => present } file { '/etc/apache2.conf' : ensure => present } } node s1.example.com { include configurations } node s2.example.com { include configurations File <| title == '/etc/apache2.conf' |> { ensure => absent } } Exclude the /etc/apache2.con f file for the s2.example.com node.
It can also be used with the ~> and -> operators. Thus, we can notify all services about any changes, or request to add all yum repositories before installing any package:
Yumrepo <| |> -> Package <| |> It seems to me that the main advantage of virtual resources is that they can be exported and made available to other agents. To export a virtual resource, you need to add another @ sign before its description.
A classic example from the Puppet documentation:
class ssh { # Declare: @@sshkey { $hostname: type => dsa, key => $sshdsakey, } # Collect: Sshkey <<| |>> } In this example, we have defined the sshkey virtual resource. Operator-collector << | | >> contains an empty body, so it unloads all exported objects of the Sshkey class. Thus, any agent in the manifest of which the ssh class is connected exports its public key ( @@ sshkey ) and then imports all the keys added by other agents ( Sshkey << | | >> ) to itself.
Exported resources are stored in PuppetDB - DB from PuppetLabs. After connecting PuppetDB, each folder copied by the master master is put into the PuppetDB database, which in turn provides a search interface for searching through catalogs.
By specifying @@ , we mark the resource as exportable and inform puppet that the resource must be added to the directory and tag it exported . When the puppet master sees the operator << | | >> , it makes a search query to PuppetDB and adds all the exported resources found that match the search criteria.
It is important that the exported resources are in the global scope, so their names must be unique.
This functionality has a huge potential and I often have to use it. Automation of adding servers to monitoring or nginx backends.
It is better to use existing modules, but to demonstrate the principle, this example will do:
# "server IP:PORT;" upstream nginx class nginx::backend($ip = $::ipaddress, $port = 8080) { @@concat::fragment { "$::fqdn" : content => "server $ip:$port;", tag => 'nginx-backend', target => '/etc/nginx/conf.d/backend.conf' } } # , concat, nginx::backend class nginx::frontend { concat { '/etc/nginx/backend.conf' : ensure => present, force => true, ensure_newline => true } ~> Class['::nginx::service'] concat::fragment { 'upstream_header': content => 'upstream backend { ', order => '01', target => '/etc/nginx/backend.conf', } concat::fragment { 'upstream_footer' : content => '}', order => '03', target => '/etc/nginx/backend.conf' } # Concat::Fragment <<| tag == 'nginx-backend' |>> { target => '/etc/nginx/backend.conf', order => '02' } } class nginx::install { package { 'nginx' : ensure => present } } class nginx::service { service { 'nginx' : ensure => running, require => Class['nginx::install'] } } class nginx { class { 'nginx::install' : } -> class { 'nginx::service': } } node 'back1.example.com' { class { 'nginx' : } class { 'nginx::backend' : port => 8083 } } node 'back2.example.com' { class { 'nginx' : } class { 'nginx::backend' : port => 8084 } } node 'front1.example.com' { class { 'nginx' : } class { 'nginx:::frontend' : } } More information about syntax and usage patterns can be found at the following links:
Source: https://habr.com/ru/post/242123/
All Articles