"Perfect" cluster. Part 2.1: Virtual hetzner cluster
I want to note that this instruction was born in the process of examining various virtualization systems within Acronis .
Proxmox showed itself from the good side, perhaps our experience will be useful for solving your problems.
Renting another server in the data center, everyone thinks about the rationality of its use.
After all, it is no secret to anyone that a well-tuned server should not be too heavy and that it should have enough resources to do other work. In addition to the above, fault tolerance is important and therefore keeping multiple copies of the same server as a hot swap seems like a great idea.
To solve these problems and need virtualization.
')
Now I will tell you how you can quickly make from a single server an entire cluster of servers based on linux and windows.
In future articles, I will try to explain how to raise a secure web cluster and use all the advantages of modern virtualization technologies.
In this manual, we will focus on the free Proxmox virtualization system, it is freely available, but requires support for support. We will try to do without the support and commercial repository of Proxmox. Here is what Wikipedia says about this product.
Proxmox Virtual Environment (Proxmox VE) is an open source virtualization system based on Debian GNU / Linux. Developed by the Austrian company Proxmox Server Solutions GmbH, sponsored by the Internet Foundation Austria.
It uses KVM and OpenVZ as hypervisors. Accordingly, it is able to perform any supported KVM OS (Linux, * BSD, Windows and others) with minimal loss of performance and lossless Linux.
Virtual machine management and server administration are performed via the web interface or via the standard Linux command line interface.
There are many options available for the virtual machines being created: used hypervisor, storage type (image file or LVM), type of emulated disk subsystem (IDE, SCSI or VirtIO), type of network card emulated, number of processors available, and others.
Key features
- Simple web-based management;
- Load monitoring in real time;
- Library of installation images (in local or remote storage);
- Connecting to the "physical" console of guest systems directly from the browser (via VNC);
- Combining servers into a cluster with the possibility of live migration of virtual machines (without stopping the guest system);
- Quick deployment of guest systems from templates (available only for OpenVZ);
- Automatic backup of virtual machines.
The first thing you need to order a server with debian 7 64 on board, the more memory the better! Take care of the safety of your data, RAID 1 is not at all superfluous, although by itself it carries a number of risks. We are optimists, take with RAID1.
As soon as we have root access to our new server, we proceed to work:
# Before installing the proxmox itself, you need to decide on the hostname and specify it
nano /etc/hosts 127.0.0.1 localhost xxxx test.xxxx.info test # # IPv6 ::1 ip6-localhost ip6-loopback fe00::0 ip6-localnet ff00::0 ip6-mcastprefix ff02::1 ip6-allnodes ff02::2 ip6-allrouters ff02::3 ip6-allhosts x:x:x:4105::2 test.xxxx.info nano /etc/hostname test # Change the time zone
echo "Europe/Moscow" > /etc/timezone dpkg-reconfigure -f noninteractive tzdata # Create a folder for repositories
mkdir -p /etc/apt/sources.list.d/ # Download repositories
cd /etc/apt/sources.list.d/ wget http://sycraft.info/share/debian7/sources.list.d/debian7.list wget http://sycraft.info/share/debian7/sources.list.d/dotdeb7.list wget http://sycraft.info/share/debian7/sources.list.d/neurodebian.sources7.list wget http://sycraft.info/share/debian7/sources.list.d/proxmox7.list # Install the keys
cd /root/ wget http://www.dotdeb.org/dotdeb.gpg cat dotdeb.gpg | apt-key add - apt-key adv --keyserver keyserver.ubuntu.com --recv-keys A040830F7FAC5991 apt-key adv --recv-keys --keyserver pgp.mit.edu 2649A5A9 wget -O- "http://download.proxmox.com/debian/key.asc" | apt-key add - rm *.gpg # Update the system
apt-get update && apt-get upgrade -f -y # We set the required minimum
apt-get install ntp screen mc git ntpdate sudo zip unzip pigz locales tzdata nano aptitude htop iotop sysstat rkhunter chkrootkit nscd lsof strace subversion multitail -y -f # Install the core from proxmox
apt-get install pve-firmware pve-kernel-2.6.32-26-pve -y -f apt-get install pve-headers-2.6.32-26-pve -y -f # Cleaning up the system from old cores
apt-get remove linux-image-amd64 linux-image-3.2.0-4-amd64 -y -f # Generate grub
update-grub # Reload
reboot # We were lucky, our server was loaded and now you can install proxmox itself
apt-get install proxmox-ve-2.6.32 ntp ssh lvm2 postfix ksm-control-daemon vzprocps open-iscsi bootlogd -y # Remove the repository for paid proxmox
rm -fr /etc/apt/sources.list.d/pve-enterprise.list # Add iptables modules for all occasions
nano /etc/vz/vz.conf IPTABLES="ipt_owner ipt_REDIRECT ipt_recent ip_tables iptable_filter iptable_mangle ipt_limit ipt_multiport ipt_tos ipt_TOS ipt_REJECT ipt_TCPMSS ipt_tcpmss ipt_ttl ipt_LOG ipt_length ip_conntrack ip_conntrack_ftp ipt_state iptable_nat ip_nat_ftp" # Add modules when loading the kernel (extra step, but just in case)
nano /etc/modules ipt_MASQUERADE ipt_helper ipt_REDIRECT ipt_state ipt_TCPMSS ipt_LOG ipt_TOS tun iptable_nat ipt_length ipt_tcpmss iptable_mangle ipt_limit ipt_tos iptable_filter ipt_helper ipt_tos ipt_ttl ipt_REJECT loop Next, a few words about the proposed architecture:
- Together with the server, we order 2 external (public) IP addresses, on the first one there is the service port of the web-panel proxmox, ssh, mysql and other service ports about which no one should know
- The second address serves the ports, which should be accessible to all. For example, 80 and 443 and all. In addition, this address is picked up on an empty virtual machine that is devoid of unnecessary services. The rest will be resolved by forwarding ports.
# Save current iptables rules
iptables-save > /etc/iptables.up.rules # Add rules to the * nat section for our external business address
nano /etc/iptables.up.rules *nat :PREROUTING ACCEPT [2164:136969] :POSTROUTING ACCEPT [58:3659] :OUTPUT ACCEPT [0:0] -A PREROUTING -d xx16.182/32 -p tcp -m tcp --dport 22 -j DNAT --to-destination 192.168.8.2:22 -A POSTROUTING -o vmbr0 -j MASQUERADE -A POSTROUTING -d xx16.182 -p tcp -s 192.168.8.0/24 --dport 22 -j SNAT --to-source xx16.182 COMMIT # Checking the rules, there should be no errors
iptables-restore < /etc/iptables.up.rules A very important rule of POSTROUTING, If you want from within one of the virtual mashes to access the forwarded port of the external address, nothing will work without this rule!
# Load images of openvz containers
cd /var/lib/vz/template/cache/ wget http://download.openvz.org/template/precreated/debian-7.0-x86_64.tar.gz wget http://download.openvz.org/template/precreated/centos-6-x86_64.tar.gz wget http://download.openvz.org/template/precreated/ubuntu-13.10-x86_64.tar.gz Drivers in case we need windows
cd /var/lib/vz/template/iso/ wget http://alt.fedoraproject.org/pub/alt/virtio-win/latest/images/virtio-win-0.1-74.iso # Next, go to the external address of our server xx16.182 : 8006 /
After authorization, we see a message about the use of the free version, and we need it. Just get used to click OK or buy a subscription.
Network Configuration for Hetznet
# We are starting to reconfigure the network, setting up network bridges may seem strange, but Hetzner has a limit on the number of MAC addresses on the switch port, we will provide all external addresses with 1 MAC
This setting works equally well in DC without such restrictions, just a universal option.
There is also a private network 192.168.8.0/16 - we use it for the internal network between our virtual machines
# Next, restart our server and view our network settings.
cat /etc/network/interfaces auto lo iface lo inet loopback auto eth0 iface eth0 inet static address xx16.182 netmask 255.255.255.224 pointopoint xx16.129 gateway xx16.129 dns-nameservers 8.8.8.8 auto vmbr0 iface vmbr0 inet static address xx16.182 netmask 255.255.255.224 bridge_ports none bridge_stp off bridge_fd 0 pre-up iptables-restore < /etc/iptables.up.rules up ip route add xx150/32 dev vmbr0 # auto vmbr1 iface vmbr1 inet static address 192.168.8.100 netmask 255.255.0.0 bridge_ports none bridge_stp off bridge_fd 0 # We write the external service IP gateway in pointopoint and gateway, in vmbr0 the exact same addresses are specified but without gateway, the second address on which the public ports should be up ip route add route and firewall rules pre-up iptables-restore
For general development, here is an example of network setup for DC without restrictions on the number of MAC addresses.
nano /etc/network/interfaces # network interface settings auto lo iface lo inet loopback auto eth0 iface eth0 inet manual auto vmbr0 iface vmbr0 inet static address xx16.182 netmask 255.255.255.0 gateway xx16.1 dns-nameservers 8.8.8.8 bridge_ports eth0 bridge_stp off bridge_fd 0 pre-up iptables-restore < /etc/iptables.up.rules auto vmbr1 iface vmbr1 inet static address 192.168.8.100 netmask 255.255.0.0 bridge_ports none bridge_stp off bridge_fd 0 # Here is an example of how we can install our Windows (if necessary)
In the settings of the video card I specify SPICE and here is the client for it www.spice-space.org/download.html
Network and disk - virtio, to install the drivers immediately do the second cd-rom for downloaded virtio iso
Well and the last in this article - the gw setup. a virtual machine that will forward us public ports. This virtual machine will not have SSH or other services that are listening to the network - it is a firewall node
You create a CT with a debian image with a Network Device network
In the container itself, it will look like this:
nano /etc/network/interfaces auto lo iface lo inet loopback auto eth0 iface eth0 inet static address xxx150 netmask 255.255.255.255 pointopoint xx16.182 gateway xx16.182 pre-up iptables-restore < /etc/iptables.up.rules auto eth1 iface eth1 inet static address 192.168.8.1 netmask 255.255.0.0 # Pay attention to the mask, gateway and pointopoint for this interface - the address of our public service network.
# Add rules to * nat section for our external public address
nano /etc/iptables.up.rules *nat :PREROUTING ACCEPT [2164:136969] :POSTROUTING ACCEPT [58:3659] :OUTPUT ACCEPT [0:0] -A PREROUTING -d xxx150/32 -p tcp -m tcp --dport 80 -j DNAT --to-destination 192.168.8.5:22 -A POSTROUTING -o eth0 -j MASQUERADE -A POSTROUTING -d xxx150 -p tcp -s 192.168.8.0/24 --dport 80 -j SNAT --to-source xxx150 COMMIT # Allow traffic forwarding during masquerading
echo "net.ipv4.ip_forward = 1" >> /etc/sysctl.conf sysctl -p # In this container, delete all unnecessary!
apt-get purge -y -f openssh-server postfix ssh samba bind9 sendmail apache2*apt-get autoremove -y If you have difficulties or need a special person to make a fairy tale come true - I will always be happy to help! my contacts are welcome
In continuation of the topic my article "Ideal" www cluster. Part 1. Frontend: NGINX + Keepalived (vrrp) on CentOS
There will be, I hope, many more and many articles on this topic! Thanks for attention
Source: https://habr.com/ru/post/204190/
All Articles