Atypical story "house" provider
The story originates back in 2004, when affordable Internet access in cities with a population of less than a million was rare. In those days, the “home” networks in which “providers” existed were usually very popular — usually students who installed a satellite dish and set up a server to distribute traffic. I also wanted to participate in this ...
2004
It will be about the city of N-sk, in which lives a little more than half a million people. I will not go into the base part of the organization of access in detail - everything was standard: I wanted to earn money and gain experience in administration after entering university. He connected to the local home network, in which there were about 150 people. The houses already had ADSL, installed a satellite dish, bought the necessary equipment, connected to the traffic tariff from a satellite provider (PlanetSky). Since I was familiar with Linux, the server was organized under Debian. I used a free solution for billing - Neon Internet Billing System.
2005 year
The home network was growing, and a couple of nearby networks were connected. Therefore, the number of subscribers has increased - it was necessary to refine the billing. Instead of the traffic tariff from the satellite operator, it became more profitable to take the conditional unlimited limit for several megabits (SatGate). For the final consumer, prices were from 30 kopecks to a ruble, depending on the time of day (for comparison, local providers offered from 3 to 7 rubles). Reception of payment was still made according to the principle came with a money - put on the account.
')
2006
The home network continued to grow. Also, several networks were connected via radio. The most expensive conditional unlimited satellite provider has ceased to cope - it was decided to buy a dedicated band. The choice fell on RuSat, as the most affordable. Due to this, there was a great opportunity to install a set of ADSL + antenna + server in other remote networks of the city - that point “flopped” in the same dedicated band. So several more networks were connected. Subscribers became even more, connection points too. Under the previous scheme, it was impossible to accept payment. I modified the billing system - an automated process for generating access cards was organized. Cards were printed on thick paper with a color laser printer, laminated, sheets were cut, the corners were rounded. Scratch-panel was attached to the place of the password. They looked like this (the photo was taken in those years on the E-ten M600 communicator, therefore the quality is appropriate):
2007
Subscriber base has become even more. It took expansion of the channel from the satellite operator. RuSat refused, because they had the entire band on the satellite already distributed. It was decided not to resort to satellite providers, but to install a server in the data center and conclude an agreement with the teleport to get its own band on the satellite. It already allowed to connect clients across all Russia. No sooner said than done. The first connections went, but in this situation the card went wrong. After a couple of months, TV people came to the satellite and drove everyone away, as they were willing to pay much more. In addition, local providers began to launch the first unlimited - 128 kbit / s of honest no limit for 1000 rubles. With satellite "ping" and the prices they will not make a competition.
One of the local federal Ethernet providers allowed to conclude several contracts for one address, and also allowed to “raise” a PPPoE session with a login from another address. The picture in my head took shape quickly. I sent 30 friends to conclude ten contracts. In the end, I received 300 PPPoE connections at 128 kbit / s each. It remained to somehow combine them into one big channel.
I rented a physical server in a data center in Moscow and wrote a tun / tap-based Linux kernel module that added a virtual network interface to the system. One such interface was on my server, another on the Moscow server, all client traffic was sent to them. Every first network packet that got into this interface was sent on the first PPPoE connection from one server to another, every second packet was sent on the second PPPoE connection, and so on cyclically. Thus, I received an unlimited channel a little more than 30 Mbps (taking into account the overhead of the tunnel).
On the way I met three major problems:
The first is mixing packages. That is, from the Moscow server the packages went in the order of 1, 2, 3, 4, 5, 6 ..., and on my server because of the shaper of the local provider they came at different times and the order could turn out like this: 3,2,5,1 , 6.4 ... As a result, many users received the same 128 kbps as a download, because the Windows TCP stack digests this sequence badly. A packet sorter was written to the kernel module, which accumulated a small buffer from the packets and sent them to clients in the correct order.
The second problem was that the standard Linux shaper with a large number of static rules, a large amount of traffic and a large bandwidth (rules added at 800 Mb / s, and the channel is only 30 Mb / s) started shaping incorrectly. I had to write a module for Linux tc, which dynamically added / deleted rules in the shaper on the Moscow server, depending on its activity on the network interface.
The third problem was the monthly ratio of outgoing / incoming traffic on the Moscow server - at that time they demanded 1 to 3. For deviation from this ratio to the lower side, they demanded a surcharge of 30 rubles for each gigabyte of excess, and the traffic was measured in terabytes. Since the server was just a gateway, the traffic ratio was 1 to 1. Torrents were a simple solution. Popular content was uploaded to the server and distributed in the console torrent client. Terabytes of sideroar traffic on popular trackers of the country were provided.
The result was excellent prices for the end user: 256 and 512 kbps of honest no-limit for 500 and 700 rubles, respectively. In such a light, 128 kbps on average for 1000 rubles from local operators looked unattractive. I also had traffic and conditionally unlimited tariffs.
2008
In the city, I was seriously competing with local providers - they began an active offensive through the authorities. In addition, for the year of my work, the provider through which my channel worked began to notice that among his clients there are a couple dozen people who have ten contracts each, and each contract pumps the most volume around the clock, which makes up a decent share of their traffic. At the same time in the city there was a big drop in prices for no limit. All these circumstances forced me to stop the activity.
UPD: According to numerous requests in the near future the article will be supplemented with detailed information, photos, graphs and diagrams. Thanks for the feedback!
2004
It will be about the city of N-sk, in which lives a little more than half a million people. I will not go into the base part of the organization of access in detail - everything was standard: I wanted to earn money and gain experience in administration after entering university. He connected to the local home network, in which there were about 150 people. The houses already had ADSL, installed a satellite dish, bought the necessary equipment, connected to the traffic tariff from a satellite provider (PlanetSky). Since I was familiar with Linux, the server was organized under Debian. I used a free solution for billing - Neon Internet Billing System.
2005 year
The home network was growing, and a couple of nearby networks were connected. Therefore, the number of subscribers has increased - it was necessary to refine the billing. Instead of the traffic tariff from the satellite operator, it became more profitable to take the conditional unlimited limit for several megabits (SatGate). For the final consumer, prices were from 30 kopecks to a ruble, depending on the time of day (for comparison, local providers offered from 3 to 7 rubles). Reception of payment was still made according to the principle came with a money - put on the account.
')
2006
The home network continued to grow. Also, several networks were connected via radio. The most expensive conditional unlimited satellite provider has ceased to cope - it was decided to buy a dedicated band. The choice fell on RuSat, as the most affordable. Due to this, there was a great opportunity to install a set of ADSL + antenna + server in other remote networks of the city - that point “flopped” in the same dedicated band. So several more networks were connected. Subscribers became even more, connection points too. Under the previous scheme, it was impossible to accept payment. I modified the billing system - an automated process for generating access cards was organized. Cards were printed on thick paper with a color laser printer, laminated, sheets were cut, the corners were rounded. Scratch-panel was attached to the place of the password. They looked like this (the photo was taken in those years on the E-ten M600 communicator, therefore the quality is appropriate):
2007
Subscriber base has become even more. It took expansion of the channel from the satellite operator. RuSat refused, because they had the entire band on the satellite already distributed. It was decided not to resort to satellite providers, but to install a server in the data center and conclude an agreement with the teleport to get its own band on the satellite. It already allowed to connect clients across all Russia. No sooner said than done. The first connections went, but in this situation the card went wrong. After a couple of months, TV people came to the satellite and drove everyone away, as they were willing to pay much more. In addition, local providers began to launch the first unlimited - 128 kbit / s of honest no limit for 1000 rubles. With satellite "ping" and the prices they will not make a competition.
One of the local federal Ethernet providers allowed to conclude several contracts for one address, and also allowed to “raise” a PPPoE session with a login from another address. The picture in my head took shape quickly. I sent 30 friends to conclude ten contracts. In the end, I received 300 PPPoE connections at 128 kbit / s each. It remained to somehow combine them into one big channel.
I rented a physical server in a data center in Moscow and wrote a tun / tap-based Linux kernel module that added a virtual network interface to the system. One such interface was on my server, another on the Moscow server, all client traffic was sent to them. Every first network packet that got into this interface was sent on the first PPPoE connection from one server to another, every second packet was sent on the second PPPoE connection, and so on cyclically. Thus, I received an unlimited channel a little more than 30 Mbps (taking into account the overhead of the tunnel).
On the way I met three major problems:
The first is mixing packages. That is, from the Moscow server the packages went in the order of 1, 2, 3, 4, 5, 6 ..., and on my server because of the shaper of the local provider they came at different times and the order could turn out like this: 3,2,5,1 , 6.4 ... As a result, many users received the same 128 kbps as a download, because the Windows TCP stack digests this sequence badly. A packet sorter was written to the kernel module, which accumulated a small buffer from the packets and sent them to clients in the correct order.
The second problem was that the standard Linux shaper with a large number of static rules, a large amount of traffic and a large bandwidth (rules added at 800 Mb / s, and the channel is only 30 Mb / s) started shaping incorrectly. I had to write a module for Linux tc, which dynamically added / deleted rules in the shaper on the Moscow server, depending on its activity on the network interface.
The third problem was the monthly ratio of outgoing / incoming traffic on the Moscow server - at that time they demanded 1 to 3. For deviation from this ratio to the lower side, they demanded a surcharge of 30 rubles for each gigabyte of excess, and the traffic was measured in terabytes. Since the server was just a gateway, the traffic ratio was 1 to 1. Torrents were a simple solution. Popular content was uploaded to the server and distributed in the console torrent client. Terabytes of sideroar traffic on popular trackers of the country were provided.
The result was excellent prices for the end user: 256 and 512 kbps of honest no-limit for 500 and 700 rubles, respectively. In such a light, 128 kbps on average for 1000 rubles from local operators looked unattractive. I also had traffic and conditionally unlimited tariffs.
2008
In the city, I was seriously competing with local providers - they began an active offensive through the authorities. In addition, for the year of my work, the provider through which my channel worked began to notice that among his clients there are a couple dozen people who have ten contracts each, and each contract pumps the most volume around the clock, which makes up a decent share of their traffic. At the same time in the city there was a big drop in prices for no limit. All these circumstances forced me to stop the activity.
UPD: According to numerous requests in the near future the article will be supplemented with detailed information, photos, graphs and diagrams. Thanks for the feedback!
Source: https://habr.com/ru/post/152077/
All Articles