In short:

1. cfnetwork - Puppet API for full network configuration and filtering through Puppet resources. Friendly with Hiera and potentially other "data providers" in the Puppet concept.
2. cffirehol - the "meta-provider" of the specific implementation of filter settings for cfnetwork based on the wonderful FireHOL generator
3. So far, only Debian 8+ (Jessie and above) and Ubuntu 14.04+ (Trusty and above) are supported.

Thematic cycle:

• Part I: network and network filter (cfnetwork + cffirehol)
• Part II: Access and Standard Environment (cfauth + cfsystem)
• Part III: Install Puppet Server (cfpuppetserver)
• Part IV: Centralized Management (cftotalcontrol)
• Part V: Databases (cfdb)
• Part VI: Topical Blacklists and Secure Knock

Lyrical introduction: it so happened that the author is extremely paranoid about the control of deployed systems and automation. For many years, accumulated experience of the problems encountered and with respect to local solutions. After leaving the previous job, it became clear that in general, there was no tangible baggage in the administration sphere. However, the fact that it was, especially to drag with them further and did not want to. So was born a new bike. And now to the point.

Note: According to the text, a or a instead of another “Russian” term that violates the author’s children's ear, eyes and brain of the author.

What is not satisfied with the existing solutions?

• Weak integration of network configuration and network filter - additional fussing with (re-) configuration and high risk of errors; difficulties in creating plug & play modules of individual services that are friendly with the security system.
• Lack of visualization for auditing - the configuration is either not concise and smeared over many files, or it exists in general in an unreadable form for a person.
• Unnecessarily low-level filter configuration without abstractions - the same problems as the item above. It can be compared to writing a web page in assembly language.
• The lack of "intelligence" of the default settings - for some clumsiness is an advantage, but not for the author.
• The network stack configuration is generally bypassed - vanilla kernel settings are not always what you want to see on the combat system even before testing and optimization, not to mention security.

The general concept of network settings and filter

No new theories - ordinariness from different places.

1. Each logical network interface has a unique meaningful name like world , dmz , office , etc. local reserved for the loopback interface.
2. The settings of the standard logical interfaces are set and accessible to the filter rule generator.
3. Support for a special type of interface any - the filter generator must be intelligent enough so as not to produce unnecessary rules where they will never work. For example, if a list of allowed addresses is specified for outgoing or incoming, then the rules should not be added to interfaces where such connections are not assumed to be a network configuration in principle.
4. Instead of directly specifying ports, an associative name is used, which can hide a whole set of ports and protocols.
5. Temporary donation of the network and filter should be easily performed on the target machine without centralized control when recovering from failures.
6. Sufficient network security must be achieved before enabling AppArmor or SeLinux.
7. Dynamic protection must be implemented separately, but the blacklist interface is defined at this level.

Choice of technology

• Puppet 4 + Puppet DB + Hiera - the author honestly tried to instill in himself a love for at least one of Ansible and Chef, but the fourth version of Puppet took its toll. Although, Ansible looks interesting for periodic tasks on system maintenance and initial deployment of Puppet.
• Ruby is essentially a predefined choice for Puppet extensions. By the way, the author had to study this PL during the project, which he does not regret at all.
• FireHOL is the first third-party iptables generator that the author was able to entrust his network filter for more than 10 years of active server administration. All other generators subjectively pale.

What happened

The interface itself consists of the main cfnetwork class and the cfnetwork::* type set for specifying network settings and network filter. It is possible to set all settings programmatically through Puppet DSL or through a data provider like Hiera.

A brief description of the API with an incomplete list of parameters. The full is available in English .

cfnetwork class

• main - settings of the cfnetwork::iface for the main interface.
• dns - list of DNS servers or special values:
  
  • '$recurse' - put the local server.
  • '$serve' is the same, but also serve clients on $service_face .
• is_router - whether this machine performs the function of a network router.
• optimize_10gbe - adjust TCP default settings for maximum performance of connections over 10 + Gbit interfaces instead of public "Internet" ones with an estimated delay of 50-100ms.
• Convenience to use Hiera.
  All values ​​except ifaces have lookup_options: { merge: hash } (documentation) .
  
  • ifaces is a set of configurations of minor interfaces like cfnetwork::iface .
  • describe_services - a set of resource descriptions of the type cfnetwork::describe_service (description of services).
  • service_ports - dialing * cfnetwork::service_port (incoming connections).
  • client_ports - set * cfnetwork::client_ports (outgoing connections).
  • dnat_ports - set * cfnetwork::dnat_ports .
  • router_ports - set * cfnetwork::router_ports .

type cfnetwork::iface - interface configuration.

• title - associative identifier that will be used in other resources.
• device - system network interface.
• address - the primary IPv4 / IPv6 address along with the network mask in the format "address/cidr" .
• extra_addresses - additional addresses in the same format.
• extra_routes - additional routing settings (also important for the filter generator).
• gateway - implies a default route that is used in the filter generator.
• force_public = auto is an extremely important setting for the filter:
  
  • By default, if $address belongs to 10/8, 172.16 / 12 or 192.168 / 16, then false , otherwise true .
  • If true :
    
    • Automatically adds SNAT or MASQUERADE for outgoing connections.
    • Automatically enables TCP SYNPROXY for incoming connections, including DNAT.
    • Sets the DROP policy instead of the default REJECT .
    • Limits incoming ping to 1 / sec. via hashlimit for one ip.
    • Sets the global black list of incoming IPs, except for the special whitelist.

type cfnetwork::describe_service - description of the service (protocols and ports).

• title - the resource name is used for all port names.
• server — list of server ports in the proto/portnum . Example: [ 'tcp/80', 'tcp/443' ] .

type cfnetwork::client_port - description of outgoing connections.

Terminology taken from FireHOL ..

• title = '<iface>:<service>[:<tag>]'
• src , dst , user , group , comment

type cfnetwork::service_port - description of incoming connections.

• title = '<iface>:<service>[:<tag>]'
• src , dst , comment

type cfnetwork::router_port - description of the allowed routed connection.

• title = '<iface>/<outface>:<service>[:<tag>]'
• src , dst , comment

type cfnetwork::dnat_port - description of the simultaneously routed connection and translation of the destination address

• title = '<iface>/<outface>:<service>[:<tag>]'
• src , dst , comment
• to_dst - redirect address (IPv4 and IPv6)
• to_port - port of redirection (optional)

Description of unified parameters:

• <iface> is the name of the associated cfnetwork::iface or:
  
  • 'local' - as already mentioned above - only local traffic, but keep in mind that traffic to your own external IP also goes through local !
  • 'any' is a special intricate login based on src , dst and to_dst so as not to create obviously unused rules. In the absence of these parameters, it is added to all possible interfaces where it makes sense. (For example, local does not make sense in router_port )
• <outface> - the same, but for the second interface in the case of dnat_port and router_port
• <service> - the name of the service description in cfnetwork::describe_service
• <tag> is an optional part that goes to comment .
  Added to avoid resource name conflicts without the need to explicitly use "virtual resources"
• src , dst - lists of outgoing and target IPv4 / IPv6 addresses
• comment - any single-line comment (forced line breaks are cut)
• user , group - check user and group for outgoing connections (the real paranoid is obliged to use them even for local )

class cfnetwork::sysctl - the ability to fine-tune the network stack, the standard keys are displayed as class parameters.

class cffirehol - filter generator

• enable=false - you must force it to turn on after you make sure that the filter config meets the expectations
• synproxy_public=true - SYNPROXY enable flag on public interfaces
• ip_whitelist / ip_blacklist - static lists. ip_blacklist should not be asked here at all, but should be dynamically ipset into ipset from constantly updated databases and dynamic protection systems, but this is a separate story.
  Predefined sets:
  
  • whitelist4 and whitelist6 - IPv4 and IPv6 network whitelist
  • blacklist4 - individual IPv4 blacklist4 addresses
  • blacklist4net and blacklist6net - IPv4 and IPv6 network blacklist

Since there was not enough fresh FireHOL package in Debian and Ubuntu, and since the standard ones run only AFTER the elevation of the network interfaces, I had to make my own .deb package builds .

Note: in the description of each Puppet module from the cfxxx series cfxxx is a section "Implicitly created resources", which describes all detectable network filter resources.

Living example

The full deployment of the infrastructure in Vagrant, using the modules not covered in this article, can be found here .

For clarity, the network configuration and the filter of the router are given:

Hiera settings

 classes: - cfnetwork #  ,     `/sbin/firehol try` #cffirehol::enable: true cfnetwork::is_router: true cfnetwork::main: device: eth1 address: '192.168.1.30/24' extra_addresses: '192.168.1.40/24' gateway: '192.168.1.1' #     force_public: true cfnetwork::ifaces: vagrant: device: eth0 method: dhcp #     extra_routes: ['10.0.1.1/25'] infradmz: device: eth2 address: '10.10.1.254/24' dbdmz: device: eth3 address: '10.10.2.254/24' webdmz: device: eth4 address: '10.10.2.254/24' cfnetwork::describe_services: testdb: server: 'tcp/1234' cfhttp: server: - 'tcp/80' - 'tcp/443' # DNAT   HTTP  (     ) cfnetwork::dnat_ports: 'main/webdmz:cfhttp': dst: '192.168.1.40' to_dst: '10.10.2.10' cfnetwork::router_ports: #   NTP, DNS, APT     'infradmz/main:cfhttp:apt': src: 'maint.example.com' 'infradmz/main:ntp': src: 'maint.example.com' #  Puppet Server (r10k)   'infradmz/main:cfhttp:puppet': {} #    DMZ     'any/infradmz:ntp': src: '10.10.0.0/16' dst: 'maint.example.com' 'any/infradmz:dns': src: '10.10.0.0/16' dst: 'maint.example.com' 'any/infradmz:aptproxy': src: '10.10.0.0/16' dst: 'maint.example.com' 'any/infradmz:puppet': src: '10.10.0.0/16' dst: 'puppet.example.com' #        'webdmz/dbdmz:testdb': {} 

generated filter generator config (this is how it comes out)

Yes, there is a small percentage of redundancy, but it is possible to optimize in future versions. For example, it makes no sense to have two rules, one of which is a special case of the other. In principle, the generation code is already overgrown with a multitude of "intellectual" conditions as it is implemented.

This configuration takes effect automatically when cffirehol::enable=true !

 # This file is autogenerated by cffirehol Puppet Module # Any changes made here may be overwritten at any time version 6 # Defaults #---------------- DEFAULT_INTERFACE_POLICY="DROP" DEFAULT_ROUTER_POLICY="DROP" FIREHOL_LOG_MODE="NFLOG" FIREHOL_TRUST_LOOPBACK="0" FIREHOL_DROP_ORPHAN_TCP_ACK_FIN="1" FIREHOL_INPUT_ACTIVATION_POLICY="DROP" FIREHOL_OUTPUT_ACTIVATION_POLICY="DROP" FIREHOL_FORWARD_ACTIVATION_POLICY="DROP" # Custom Services #---------------- server_dns_ports="tcp/53 udp/53" client_dns_ports="any" # Use to open all TCP ports (eg for local) server_alltcp_ports="tcp/1:65535" client_alltcp_ports="any" # Use to open all UDP ports (eg for local) server_alludp_ports="udp/1:65535" client_alludp_ports="any" # Use to open all TCP and UDP ports (eg for local) server_allports_ports="udp/1:65535 tcp/1:65535" client_allports_ports="any" server_cfhttp_ports="tcp/80 tcp/443" client_cfhttp_ports="default" server_testdb_ports="tcp/1234" client_testdb_ports="default" server_cfssh_ports="tcp/22" client_cfssh_ports="default" server_smtp_ports="tcp/25" client_smtp_ports="default" server_cfsmtp_ports="tcp/25 tcp/465 tcp/587" client_cfsmtp_ports="default" server_puppet_ports="tcp/8140" client_puppet_ports="default" # Setup of ipsets #---------------- ipset4 create whitelist4 hash:net ipset6 create whitelist6 hash:net ipset4 create blacklist4 hash:ip ipset4 create blacklist4net hash:net ipset6 create blacklist6net hash:net # note: hardcoded list is not expected to be large ipset4 add whitelist4 "10.0.0.0/8" # Protection on public-facing interfaces #---------------- # main blacklist4 input inface "eth1" ipset:blacklist4net ipset:blacklist4 except src ipset:whitelist4 blacklist6 input inface "eth1" ipset:blacklist6net ipset:blacklist6 except src ipset:whitelist6 iptables -t raw -N cfunroute_main iptables -t raw -A cfunroute_main -s "10.0.0.0/8,172.16.0.0/12,224.0.0.0/4,127.0.0.1/8" -j DROP iptables -t raw -A cfunroute_main -d "10.0.0.0/8,172.16.0.0/12,224.0.0.0/4,127.0.0.1/8" -j DROP iptables -t raw -A PREROUTING -i "eth1" -j cfunroute_main # cfauth: synproxy4 input inface main dst "192.168.1.30/24" dport "22" src "192.168.0.0/16" accept synproxy4 forward inface main dst "192.168.1.40" dport "80" dnat to "10.10.2.10" synproxy4 forward inface main dst "192.168.1.40" dport "443" dnat to "10.10.2.10" iptables -t nat -N cfpost_snat_main iptables -t nat -A cfpost_snat_main -s 192.168.1.30,192.168.1.40 -j RETURN iptables -t nat -A cfpost_snat_main -j SNAT --to-source=192.168.1.30 iptables -t nat -A POSTROUTING -o "eth1" -j cfpost_snat_main # vagrant blacklist4 input inface "eth0" ipset:blacklist4net ipset:blacklist4 except src ipset:whitelist4 blacklist6 input inface "eth0" ipset:blacklist6net ipset:blacklist6 except src ipset:whitelist6 # cfauth: iptables -t nat -A POSTROUTING -o "eth0" -j MASQUERADE # Custom Headers #---------------- # NAT #---------------- dnat4 to "10.10.2.10" inface "eth1" proto "tcp" dport "80" dst "192.168.1.40" dnat4 to "10.10.2.10" inface "eth1" proto "tcp" dport "443" dst "192.168.1.40" # Interfaces #---------------- interface "eth1" "main" policy deny protection bad-packets client icmp accept server4 ping accept with hashlimit ping upto 1/s burst 2 # cfauth: server4 "cfssh" accept src "192.168.0.0/16" # cfsystem: client "http" accept uid "root" # cfsystem: client "https" accept uid "root" # cfsystem: client "ntp" accept uid "root ntpd" # cfsystem: client "cfsmtp" accept uid "root Debian-exim" # cfsystem: client "puppet" accept uid "root" # cfnetwork: client "dns" accept interface "eth0" "vagrant" policy deny protection bad-packets client icmp accept server4 ping accept with hashlimit ping upto 1/s burst 2 # cfauth: server4 "cfssh" accept src "10.0.0.0/8 192.168.0.0/16 172.16.0.0/12" # cfsystem: client "http" accept uid "root" # cfsystem: client "https" accept uid "root" # cfsystem: client "ntp" accept uid "root ntpd" # cfsystem: client "cfsmtp" accept uid "root Debian-exim" # cfsystem: client "puppet" accept uid "root" # cfnetwork: client4 "dns" accept dst "10.10.1.10" interface "eth2" "infradmz" policy reject client icmp accept server icmp accept # cfauth: server4 "cfssh" accept src "10.0.0.0/8" # cfsystem: client "http" accept uid "root" # cfsystem: client "https" accept uid "root" # cfsystem: client "ntp" accept uid "root ntpd" # cfsystem: client "cfsmtp" accept uid "root Debian-exim" # cfsystem: client "puppet" accept uid "root" # cfnetwork: client4 "dns" accept dst "10.10.1.10" interface "eth3" "dbdmz" policy reject client icmp accept server icmp accept # cfauth: server4 "cfssh" accept src "10.0.0.0/8" # cfsystem: client "http" accept uid "root" # cfsystem: client "https" accept uid "root" # cfsystem: client "ntp" accept uid "root ntpd" # cfsystem: client "cfsmtp" accept uid "root Debian-exim" # cfsystem: client "puppet" accept uid "root" interface "eth4" "webdmz" policy reject client icmp accept server icmp accept # cfauth: server4 "cfssh" accept src "10.0.0.0/8" # cfsystem: client "http" accept uid "root" # cfsystem: client "https" accept uid "root" # cfsystem: client "ntp" accept uid "root ntpd" # cfsystem: client "cfsmtp" accept uid "root Debian-exim" # cfsystem: client "puppet" accept uid "root" interface "lo" "local" policy reject client icmp accept server icmp accept # cfauth: server4 "cfssh" accept src "10.0.0.0/8 192.168.0.0/16" # cfsystem: client "http" accept uid "root" # cfsystem: client "https" accept uid "root" # cfsystem: client "ntp" accept uid "root ntpd" # cfsystem: server "smtp" accept # cfsystem: client "smtp" accept # cfsystem: client "cfsmtp" accept uid "root Debian-exim" # cfsystem: client "puppet" accept uid "root" # Routers #---------------- router "main_infradmz" inface "eth1" outface "eth2" policy drop client icmp accept # apt: client4 "cfhttp" accept src "10.10.1.10" client4 "ntp" accept src "10.10.1.10" # puppet: client "cfhttp" accept router "main_webdmz" inface "eth1" outface "eth4" policy drop client icmp accept server4 "cfhttp" accept dst "10.10.2.10" router "vagrant_infradmz" inface "eth0" outface "eth2" policy drop client icmp accept server4 "ntp" accept dst "10.10.1.10" src "10.10.0.0/8" server4 "dns" accept dst "10.10.1.10" src "10.10.0.0/8" server4 "aptproxy" accept dst "10.10.1.10" src "10.10.0.0/8" server4 "puppet" accept dst "10.10.1.11" src "10.10.0.0/8" router "infradmz_infradmz" inface "eth2" outface "eth2" policy reject server icmp accept client icmp accept server4 "ntp" accept dst "10.10.1.10" src "10.10.0.0/8" server4 "dns" accept dst "10.10.1.10" src "10.10.0.0/8" server4 "aptproxy" accept dst "10.10.1.10" src "10.10.0.0/8" server4 "puppet" accept dst "10.10.1.11" src "10.10.0.0/8" router "dbdmz_infradmz" inface "eth3" outface "eth2" policy reject server icmp accept client icmp accept server4 "ntp" accept dst "10.10.1.10" src "10.10.0.0/8" server4 "dns" accept dst "10.10.1.10" src "10.10.0.0/8" server4 "aptproxy" accept dst "10.10.1.10" src "10.10.0.0/8" server4 "puppet" accept dst "10.10.1.11" src "10.10.0.0/8" router "webdmz_infradmz" inface "eth4" outface "eth2" policy reject server icmp accept client icmp accept server4 "ntp" accept dst "10.10.1.10" src "10.10.0.0/8" server4 "dns" accept dst "10.10.1.10" src "10.10.0.0/8" server4 "aptproxy" accept dst "10.10.1.10" src "10.10.0.0/8" server4 "puppet" accept dst "10.10.1.11" src "10.10.0.0/8" router "webdmz_dbdmz" inface "eth4" outface "eth3" policy reject server icmp accept client icmp accept server "testdb" accept 

network configuration

The module itself will not attempt to change the network settings on the fly - this will need to be done with pens or restarts.

 # # Generated by cfnetwork::iface puppet module # auto lo iface lo inet loopback source /etc/network/interfaces.d/* # # Generated by cfnetwork::iface puppet module # auto eth3 iface eth3 inet static address 10.10.2.254 netmask 24 up sysctl --ignore net.ipv6.conf.eth3.disable_ipv6=1 # # Generated by cfnetwork::iface puppet module # auto eth2 iface eth2 inet static address 10.10.1.254 netmask 24 up sysctl --ignore net.ipv6.conf.eth2.disable_ipv6=1 # # Generated by cfnetwork::iface puppet module # auto eth1 iface eth1 inet static address 192.168.1.30 netmask 24 gateway 192.168.1.1 dns-nameservers 10.10.1.10 dns-search example.com up ip addr add 192.168.1.40/24 dev eth1 up sysctl --ignore net.ipv6.conf.eth1.disable_ipv6=1 # # Generated by cfnetwork::iface puppet module # auto eth0 iface eth0 inet dhcp netmask 255.255.255.0 up ip route add 10.0.1.1/25 dev eth0 up sysctl --ignore net.ipv6.conf.eth0.disable_ipv6=1 # # Generated by cfnetwork::iface puppet module # auto eth4 iface eth4 inet static address 10.10.2.254 netmask 24 up sysctl --ignore net.ipv6.conf.eth4.disable_ipv6=1 

Conclusion

As you can see, the configuration of the network and the filter is elementary, clean and concise, and most importantly, magic numbers are convenient for changes without a mountain.

There is no long history in combat mode. Running in runs on a pair of real servers and about a dozen virtual locks without a serious load. Therefore, volunteers are interested in who have expanded the fleet of systems, and have not yet managed to tune out the approach to administration or are not completely satisfied.