Distributed cluster of two Fortigate

Recently completed a project with a client to build a fault-tolerant network infrastructure. While the memories are alive - and the documentation is at hand, I want to share a unique know-how. What is unique? The fact that such a configuration is not mentioned anywhere in official documents Fortinet is not recommended and does not officially exist at all. Also, no mention of this configuration was found on the Internet.

So, what we have available:
Two spaced sites, office and data center (DC). Each site has independent access to the Internet and local subnet. Between sites there is a dedicated connection with the ability to create your own VLANs. From the equipment, there are two Fortigate 100D and four Cisco Catalyst 2960XR managed switches with a Stacking Module. Those. two switches and one fortigge to the site.

Task:
Provide fault tolerance and continuity of communication
1) in case of failure of any one piece of iron.
2) in the case of the fall of any of the communication channels.
3) in case of failure of any one port on any of the glands
')
If we had 2 forgates per site - there would be no problem at all - we set up a standard cluster of two forgates at each site, dynamic routing and enjoying life. However, with the equipment we have what we have, so we have to get bogged down.

At the beginning of the equipment, there was even less - one switch and one fortigate per platform, but in this article I will describe the final configuration.

So let's start:

Let's designate SW1 and FG1 as the stack of switches in the office and fortigate in the office. SW2 and FG2 are similar glands in the datacenter.

For the correct configuration of all we need 5 logical ports on each forgate (5 logical = 10 physical). Denote them as:

• Office_Internet
• DC_Internet
• Office_Lan
• DC_Lan
• Interconnect

We also use both HA ports to ensure cluster synchronization, which adds another 2 ports to those used on the switch. In total, 6 ports on each physical switch (or 12 ports on a stack of switches) will be used only to connect forgates.

Layer1 configuration (physical connections):
I assume that the stack of switches is already assembled, since no difficulty there.
We connect fortigates to switches according to the following scheme:

For DC_Internet

FG1-WAN1 - SW1-1/0/1 FG1-WAN2 - SW1-2/0/1 

For Office_Internet

 FG1-DMZ - SW1-1/0/2 FG1-Port1 - SW1-2/0/2 

For Office_Lan

 FG1-DMZ - SW1-1/0/3 FG1-Port1 - SW1-2/0/3 

For DC_Lan

 FG1-Port2 - SW1-1/0/4 FG1-Port3 - SW1-2/0/4 

For interconnect

 FG1-Port4 - SW1-1/0/5 FG1-Port5 - SW1-2/0/5 

For HA1 and HA2

 FG1-HA1 - SW1-1/0/6 FG1-HA2 - SW1-2/0/6 

Repeat the operation similarly for FG2 and SW2.

Now you need to plug in switches links coming from the provider. In my case, these are 2 connections to the Internet and the channel between the DC and the office, respectively, the configuration will be slightly different between the two sites. Again, in my particular case, the providers provided two links to the Internet and 2 links for interconnect on each side.

If you can’t get 2 links to each channel - then one of the requirements will remain partially unfulfilled - in case of a port exit on the switch where the link from the provider is connected, there will be a loss of communication.

Here we will connect downlinkas to Access switches in the office and DC.

We connect in the office:

 SW1-1/0/22 -   Office_LAN  1 SW1-2/0/22 -   Office_LAN  2 SW1-1/0/23 -       1 SW1-2/0/23 -       2 SW1-1/0/24 -  -  1 SW1-2/0/24 -  -  2 

We connect to DC:

 SW1-1/0/22 -   DC_LAN  1 SW1-2/0/22 -   DC_LAN  2 SW2-1/0/23 -       1 SW2-2/0/23 -       2 SW2-1/0/24 -  -  1 SW2-2/0/24 -  -  2 

With Layer 1 finished, you can navigate to the Port Aggregation and VLAN configuration.

Layer 2
At this stage, you need to configure a pair of ports on switches and fortigets, and assign a VLAN to limit traffic.

- Create the necessary VLANs on the switches and assign ports in them:

on switches in configuration mode we create the necessary VLANs

 vlan 10 name DC_Internet vlan 20 name Office_Internet vlan 30 name Office_Lan vlan 40 name DC_LAN vlan 50 name Interconnect vlan 60 name HA 

In the same place on the switches we create a port-channel and assign ports in them:

For DC_Internet:

 Interface port-channel1 switchport access vlan 10 # VLAN 10 = DC_Internet switchport mode access Interface GigabitEthernet1/0/1 switchport access vlan 10 switchport mode access channel-group 1 mode passive Interface GigabitEthernet2/0/1 switchport access vlan 10 switchport mode access channel-group 1 mode passive 

Repeat for the above pairs of ports and get the port-channel 2-4

For Interconnect:

 Interface port-channel5 switchport access vlan 50 switchport mode access Interface GigabitEthernet1/0/5 switchport access vlan 50 switchport mode access channel-group 5 mode passive Interface GigabitEthernet2/0/5 switchport access vlan 50 switchport mode access channel-group 5 mode passive 

Slightly different configuration for HA1 and HA2 there is no port aggregation
For HA1

 Interface GigabitEthernet1/0/6 switchport access vlan 60 # Vlan 60 = HA switchport mode access 

For HA2

 Interface GigabitEthernet2/0/6 switchport access vlan 60 # Vlan 60 = HA switchport mode access 

For the Office-DC Channel:

 Interface port-channel6 switchport mode trunk # !!! Trunk Interface GigabitEthernet1/0/24 switchport mode trunk channel-group 6 mode passive Interface GigabitEthernet2/0/24 switchport mode trunk channel-group 6 mode passive 

The above configuration for Layer2 is the same for the office and DC, below are the different parts of the config.

In the office:

 Interface port-channel7 switchport access vlan 30 # VLAN 30 = Office_Lan switchport mode access Interface GigabitEthernet1/0/22 switchport access vlan 30 switchport mode access channel-group 7 mode passive Interface GigabitEthernet2/0/22 switchport access vlan 30 switchport mode access channel-group 7 mode passive Interface port-channel8 switchport access vlan 20 # VLAN 20 = Office_Internet switchport mode access Interface GigabitEthernet1/0/23 switchport access vlan 20 switchport mode access channel-group 8 mode passive Interface GigabitEthernet2/0/23 switchport access vlan 20 switchport mode access channel-group 8 mode passive 

In DC:

 Interface port-channel7 switchport access vlan 40 # VLAN 40 = DC_Lan switchport mode access Interface GigabitEthernet1/0/22 switchport access vlan 40 switchport mode access channel-group 7 mode passive Interface GigabitEthernet2/0/22 switchport access vlan 40 switchport mode access channel-group 7 mode passive Interface port-channel8 switchport access vlan 10 # VLAN 10 = DC_Internet switchport mode access Interface GigabitEthernet1/0/23 switchport access vlan 10 switchport mode access channel-group 8 mode passive Interface GigabitEthernet2/0/23 switchport access vlan 10 switchport mode access channel-group 8 mode passive 

From the side of forgates:

1. Turn on VDOM support
2. Configure Virtual Cluster Support
3. Scatter ports - each in its own VDOM
4. Create Port Aggregation 802.3ad Port Pairs

This is how the code looks like:


This is all for Layer 2, go to Layer 3+ . Here we need:

- enable marshurt
- configure routing monitoring
- configure firewall rules
- set up VPN in case the channel falls between the office and the DC

On the switches do not need anything. On fortigeit (after setting up the cluster, we manage one piece of iron):


Now, if any Internet channel falls, the static route to the Internet will be deleted and the route will be activated through the official channel, if the channel is restored, everything will return as it was.

It remains to create firewall rules (I hope this will be mastered by yourself). You just need to remember that the same traffic can come from different interfaces (Internet or interconnect).

And also create a VPN connection for connecting sites in the event of a DC channel falling - office.


And the corresponding firewall rules:

As a result, VPN will rise only in the event of a fall in the office-dts channel.

What we got in the end:

1. Aggregation of ports protects us from the fall of either one port or the entire switch (with the exception of channels with one wire from the provider).

2. A virtual cluster of fortigets will work as a router for the network, and in a normal situation, the fortigate in the office will always redirect packets for its network zone, and the forgigate to the DC for its own. In the event of a fall of one of the forgates, all traffic will be routed by the second forgate.

3. In the case of the fall of one of the Internet channels - all traffic will go through another channel, and come back as soon as the channel rises to the Internet.

4. In the case of the fall of the office-dz channel, the traffic will go through the automatically rising VPN and will follow it until the channel is restored.

It turned out a lot and probably at first sight difficult - but if you have questions, I am ready to answer them.