PBX-Sphere Online - How We Did It
Perhaps, it is worth starting with the fact that, initially, in 2011, the PBX-Sphere project was planned as server proprietary software for SMBs, which was supposed to solve the problem of implementation and maintenance of complex VoIP technologies in a simple and understandable software product. And so, on July 1, 2011, the first stable release of PBX-Sphere Enterprise 1.0 was released. Then we got a lot of beta testers around the world and a few big customers. Up until October 15, 2013, we released 3 more stable versions of the PBX-Sphere Enterprise, including the latest - 2.2.0. Still, within half a year, we actively promoted our product to large customers. As a result, as of mid-2014, we had less than 1,000 active end users. This is not to say that it was a failure, but we expected a much better result.
And then, having analyzed the market that had changed over the course of several years, we realized that we had to switch to the “Software as a Service” (SaaS) model, to cloud technologies. Within a few months, we developed a new product concept and started refactoring PBX-Sphere Enterprise in PBX-Sphere Online.
The first issue was the choice of platform deployment system. After some research and experimentation, we decided to choose from three: Hetzner Dedicated Servers, Microsoft Azure, a cluster of our own servers.
')
I'll tell you more:
Hetzner Dedicated Servers
It seemed to be the perfect solution ... Physical servers with Intel Core i7, 48 GB of RAM, 2 x 2 Tb HDD in RAID1 and FlexiPack - for 75 euros per month with a guaranteed bandwidth of 200 Mbps. What more could you want? We started the deployment. In the beginning, everything was great. We deployed dozens of servers with CentOS 7 for different PBX-Sphere Online deployment production roles. Load testing showed excellent results. Inspired by this state of affairs, we opened PBX-Sphere Online 1.0 on this configuration for industrial use for one large client. Everything worked fine. There were several complaints that during a conversation there may be short-term "fading" sound. We blamed it on the instability of the Internet connection of users, since no channel losses were detected on our side, and the Load Average on our servers was minimal, as was the size of the used RAM and Swap. But, soon, we began to notice that short-term "fading" sound is almost always observed when using the Internet over a 3G network, and several times in a minute, regardless of the codecs used. We decided to understand what was happening, because such a problem had never arisen when using test servers located in our office. Imagine our surprise when we discovered that Hetzner Edge Equipment incorrectly uses shaping, as a result of which, RTP packets are queued for delivery to our servers and back with a delay that is higher than the TTL of the RTP packets. It turns out that if the user has a good Internet channel and the TCP exchange protocol is selected, then there is no problem, as it were. A week of negotiations with the support Hetzner did not give any results. Further more. Within 3 days on our servers 5 hard drives failed! Moreover, 2 of them - on a central SQL server. Fortunately, there was a duplicate SQL server, so for users it went unnoticed. To the question “WTF?” Support Hetzner answered that this is normal and in the order of things. After the death of two more hard drives and incessant network problems, it became clear that you need to look for another platform.
Microsoft Azure
First impressions of Microsoft Azure can be described in three words: cool, unusual, expensive. This is not just virtual machines in the usual sense of the word, but a whole separate environment with its own laws and subtleties. We decided to try. We deployed PBX-Sphere Online and launched it in test mode. They watched for a long time, did a lot of how to load tests of all systems, and tested in real conditions. We waited for the problems that we encountered at Hetzner to manifest. This did not happen, there was no "fading" sound. But at once the potential problem with IP addresses was obvious. As you know, Azure has 3 types of IP addresses: the local IP address of the instance and 2 external IP addresses (VIP and PIP). The work of a VIP address is similar to the work of a home router, when an external IP address comes to the WAN interface and in order to “get out of the world” into the internal network, you need to “forward” a specific port. Same mechanism for VIP in Azure. This is acceptable for all instances of PBX-Sphere Online deployment, except for Backend servers, which require the entire range of ports for direct voice transmission during a call (RTP). Azure allows for “forwarding” a maximum of 150 ports per instance. Therefore, for RTP you need to use a different type of external IP address - PIP, all ports of which fall into the instance. It works. But VIP and PIP addresses are dynamic, by default. That is, after rebooting the server, the VIP / PIP address may change. And if the VIP-address is easy to determine by DNS, then the PIP address can only be determined from the inside of the server by performing an external request. How, then, set up your domain zone? To authorize the end user equipment, you can use a domain name that is an alias (CNAME) of the instance host. But Azure does not provide an Azure PIP address. Therefore, the same instance authorizes users using the VIP address, and the voice (RTP) is sent via the PIP address. For such an implementation, after each reboot, each backend instance must externally request its PIP address and only then reconfigure for RTP. Fortunately, for RTP a separate entry in the DNS is not required. In general, such a mechanism is terribly inconvenient, since The server is directly dependent on an external resource that reports the IP address. How to fix a copy of the PIP-address, I did not find. Another unpleasant moment is when Microsoft performs maintenance in its data centers and, as a result, some servers become inaccessible. Rescues redundant computing resources in different regions, so that such actions remain transparent to our users and without loss of quality. But this is achieved at the expense of our direct costs, since we must ensure such reservations at our own expense. In all other respects, the quality of the service is simply excellent, but, as I have already mentioned, quite expensive.
Own cluster
This option was calculated only theoretically. The fact is that most of the PBX-Sphere team is located in Ukraine, and the rest is in Russia and Bulgaria. PBX-Sphere focuses not only on the CIS countries, but, for the most part, on the US and EU countries. Therefore, as you understand, neither Russia, nor Ukraine, nor Bulgaria is suitable for hosting servers. And it is very expensive to place servers in other countries and provide them support with frequent flights of technical specialists, even taking into account the possible hedging of currency risks. Perhaps in the future we will return to the consideration of this option.
Thanks to the support of Microsoft, we became members of the BizSpark program, which, among other things, provides Azure services for free for $ 750 per month for a period of 2 years. Thanks to this support, as well as direct contacts with Azure engineers, we chose this platform for our PBX-Sphere Online. And, so far, very pleased with this.
Summary
On August 27, 2015, we upgraded PBX-Sphere Online on the Microsoft Azure platform to version 2.0.0 for all users and opened public registration for the first time. Now each new account is granted 25 free licenses for life.
And then, having analyzed the market that had changed over the course of several years, we realized that we had to switch to the “Software as a Service” (SaaS) model, to cloud technologies. Within a few months, we developed a new product concept and started refactoring PBX-Sphere Enterprise in PBX-Sphere Online.
The first issue was the choice of platform deployment system. After some research and experimentation, we decided to choose from three: Hetzner Dedicated Servers, Microsoft Azure, a cluster of our own servers.
')
I'll tell you more:
Hetzner Dedicated Servers
It seemed to be the perfect solution ... Physical servers with Intel Core i7, 48 GB of RAM, 2 x 2 Tb HDD in RAID1 and FlexiPack - for 75 euros per month with a guaranteed bandwidth of 200 Mbps. What more could you want? We started the deployment. In the beginning, everything was great. We deployed dozens of servers with CentOS 7 for different PBX-Sphere Online deployment production roles. Load testing showed excellent results. Inspired by this state of affairs, we opened PBX-Sphere Online 1.0 on this configuration for industrial use for one large client. Everything worked fine. There were several complaints that during a conversation there may be short-term "fading" sound. We blamed it on the instability of the Internet connection of users, since no channel losses were detected on our side, and the Load Average on our servers was minimal, as was the size of the used RAM and Swap. But, soon, we began to notice that short-term "fading" sound is almost always observed when using the Internet over a 3G network, and several times in a minute, regardless of the codecs used. We decided to understand what was happening, because such a problem had never arisen when using test servers located in our office. Imagine our surprise when we discovered that Hetzner Edge Equipment incorrectly uses shaping, as a result of which, RTP packets are queued for delivery to our servers and back with a delay that is higher than the TTL of the RTP packets. It turns out that if the user has a good Internet channel and the TCP exchange protocol is selected, then there is no problem, as it were. A week of negotiations with the support Hetzner did not give any results. Further more. Within 3 days on our servers 5 hard drives failed! Moreover, 2 of them - on a central SQL server. Fortunately, there was a duplicate SQL server, so for users it went unnoticed. To the question “WTF?” Support Hetzner answered that this is normal and in the order of things. After the death of two more hard drives and incessant network problems, it became clear that you need to look for another platform.
Microsoft Azure
First impressions of Microsoft Azure can be described in three words: cool, unusual, expensive. This is not just virtual machines in the usual sense of the word, but a whole separate environment with its own laws and subtleties. We decided to try. We deployed PBX-Sphere Online and launched it in test mode. They watched for a long time, did a lot of how to load tests of all systems, and tested in real conditions. We waited for the problems that we encountered at Hetzner to manifest. This did not happen, there was no "fading" sound. But at once the potential problem with IP addresses was obvious. As you know, Azure has 3 types of IP addresses: the local IP address of the instance and 2 external IP addresses (VIP and PIP). The work of a VIP address is similar to the work of a home router, when an external IP address comes to the WAN interface and in order to “get out of the world” into the internal network, you need to “forward” a specific port. Same mechanism for VIP in Azure. This is acceptable for all instances of PBX-Sphere Online deployment, except for Backend servers, which require the entire range of ports for direct voice transmission during a call (RTP). Azure allows for “forwarding” a maximum of 150 ports per instance. Therefore, for RTP you need to use a different type of external IP address - PIP, all ports of which fall into the instance. It works. But VIP and PIP addresses are dynamic, by default. That is, after rebooting the server, the VIP / PIP address may change. And if the VIP-address is easy to determine by DNS, then the PIP address can only be determined from the inside of the server by performing an external request. How, then, set up your domain zone? To authorize the end user equipment, you can use a domain name that is an alias (CNAME) of the instance host. But Azure does not provide an Azure PIP address. Therefore, the same instance authorizes users using the VIP address, and the voice (RTP) is sent via the PIP address. For such an implementation, after each reboot, each backend instance must externally request its PIP address and only then reconfigure for RTP. Fortunately, for RTP a separate entry in the DNS is not required. In general, such a mechanism is terribly inconvenient, since The server is directly dependent on an external resource that reports the IP address. How to fix a copy of the PIP-address, I did not find. Another unpleasant moment is when Microsoft performs maintenance in its data centers and, as a result, some servers become inaccessible. Rescues redundant computing resources in different regions, so that such actions remain transparent to our users and without loss of quality. But this is achieved at the expense of our direct costs, since we must ensure such reservations at our own expense. In all other respects, the quality of the service is simply excellent, but, as I have already mentioned, quite expensive.
Own cluster
This option was calculated only theoretically. The fact is that most of the PBX-Sphere team is located in Ukraine, and the rest is in Russia and Bulgaria. PBX-Sphere focuses not only on the CIS countries, but, for the most part, on the US and EU countries. Therefore, as you understand, neither Russia, nor Ukraine, nor Bulgaria is suitable for hosting servers. And it is very expensive to place servers in other countries and provide them support with frequent flights of technical specialists, even taking into account the possible hedging of currency risks. Perhaps in the future we will return to the consideration of this option.
Thanks to the support of Microsoft, we became members of the BizSpark program, which, among other things, provides Azure services for free for $ 750 per month for a period of 2 years. Thanks to this support, as well as direct contacts with Azure engineers, we chose this platform for our PBX-Sphere Online. And, so far, very pleased with this.
Summary
On August 27, 2015, we upgraded PBX-Sphere Online on the Microsoft Azure platform to version 2.0.0 for all users and opened public registration for the first time. Now each new account is granted 25 free licenses for life.
Source: https://habr.com/ru/post/266169/
All Articles