The parochial "Pionernet" with the eyes of a technician. Part two
So, the notes of the technician about the realities of the small town “Pionernet” are continuing.
The first part turned out to be interesting, albeit with a bias in “organizational problems”.
I will try to fill the gap and recall the technical aspects of the work. In particular: the organization of the overall network structure, the modernization of the segments without decommissioning, laying and terminating optical cables.
Welcome under the cat:
Perhaps it is worth noting that the city is relatively modern, the number of floors of the building is mainly 5-9-16 floors. Geographically, it consists of two “macro-districts” distant from each other, divided into “micro-districts” of 20-40 houses each.
When I first started working, the optical backbone network was made in the form of two rings covering “macro districts”, with one point of presence per “microdistrict”, from which chains of blunt switches on copper twisted pair diverged from house to house. Connectivity between the "macro-regions" was organized by the leased fiber of one of the "opsos".
If you try to recall the scheme, then it resembled the scheme of the Moscow metro, from which a part was removed within the circle line, and looked like this:
At that time, we wanted to get a stable, mainline star, from which only later to organize a smart level of access.
The optical network itself was a terrifying view. Multimode fiber in places already crumbling from old age, covered with splice plates. Long probes could easily be made from several pieces of cable of different fibers, manufacturer, and even b / usnosti. Couplings at the welding sites were rare - most of the connections were packed in a cassette and put in a “beer bottle”, hanging upside down.
The ending in the boxes also did without crosses. Cable, fiber, splice plates and tails were tied with twisted pair pieces to plywood, and went into a switch. At points that were considered “exemplary,” a cassette was used instead of plywood.
The situation was quite normal when a single-mode eight-fiber two-module cable leaving a point, on the way, in a bottle, turned into a single-module one, then, in another bottle, it diverged into two multi-mode 4 fiber cables - one went further into the ring, and another one of some kind of organization ...
Naturally, this structure did not suffer from quality and stability since birth.
')
There were no circuits for optical connections and cross-connects. We collected fragmentary information about the cable structure, which was stored in the heads of installation and engineering, and began to think.
At first, a week without getting out of the roofs, on a satellite image of the city in the “SAS Planet”, they estimated with the routes an approximate diagram of the passage of cables, climbed into all points and bottles, as they could trace the cross. Then they forced the authorities to order a used reflectometer and welding machine.
Dumas ended with the purchase of a bay of 8 fiber single-mode and single-module cables, couplings, 19 inch equipment boxes, crosses and piles of optical passivation.
A month later, we received a technique: an Anritsu reflectometer and an Ericsson FSU 905 welding machine, which to this day serve us faithfully. A month later came the materials.
From that day on, it was announced “Absolutely not” to bottles and splice plates.
Jointly, we decided that we would fix the 8 fiber cable in separate sections along the route of the old rings, and unfold the rings into stars as they were laid, with the collection at the aggregation points. In total, 15 rays of a star in one center and 11 in another were counted, the rings were disconnected on the opposite diameter from the center of aggregation, and each point was switched through its WDM-SFP into its fiber.
They did it as follows: In the center, two ports with conventional duplex SFPs were allocated - in them, already with new fiber, the old ring was decreasing as it worked, and 16 ports with WDM modules. The cable in the aggregation center was welded to the cross, and the first fibers were connected to the six ports of the switch, the last pair was used to support the old structure.
Then, next to the existing one on the ring point, they installed a new box and an electrician for equipment transfer, hung a section of cable, scalded it, transferred equipment, switched a point to one fiber in a new cable, the rest of the fibers were crossed through, except for the last pair that went to the old cable and closed the ever decreasing ring.
Both structures worked simultaneously, only one grew and the second squeezed:
When three points remained in the ring, and the last two fibers were involved on each side, they did not disturb them, laid, terminated and crossed the last sections of the new cable, then the engineers and installers went to the night shift, and in 5 hours of a tight schedule moved and switched all three points.
The second "makrorayon" was remade in the same way. There we did without night work, because the star had 11 rays, and 16 fibers were enough to narrow the ring to two points, and then open the ring and switch them to 1 fiber. We even had a reserve of 4 fibers, well, he does not pull a pocket. True, in the first “macro-district” we have no spare fibers left, since 15 of them were used in the star, and one more from each side we used to connect aggregation centers to each other, but this problem is now being solved.
Thus, we got two logical stars on virtually the same rings that were in the beginning. But now each point was completely independent of the performance of the others, equipped with uninterrupted power, optical cross-country, plus we got rid of “jinn in bottles”. In addition, they made their own link between the aggregation points, leaving the rented fiber as a hot reserve.
The idle time of the point and the microdistrict dependent on it, on average, 2-4 hours is quite an acceptable result, given the volume and complexity of the work.
As they were laid, the old sections, and the boxes where the equipment was located, were dismantled. The removed pieces of optics, organoleptically and by reflectograms, were checked for aptitude, and sent to the warehouse. The boxes and the best part of the cables were later used when replacing copper inter-house communications.
After putting the stars into operation, there were practically no problems on the highways. To this day, the inconvenience is delivered only by long-term power outages, which the bespereboyniki cannot cope with, but this is apparently chronic.
But with the growth of the network and the number of subscribers other problems began to emerge - stupid switches on a twisted pair did not cope with the load from one and a half - two hundred clients in the segment. There are customers who have organized their own "access" networks, with rings and broadband storms from all kinds of borg and ice chats.
There is a need to solve customer access problems.
For this it was necessary to “umnify” at home. And at the same time really do FTTB. At one to five access houses decided to put one switch in the middle of the house, in the attic. In multiple-access buildings - one switch to five entrances.
It was decided to connect the switches into rings, which are closed on the rays of the main star.
For tests and development of technology, we chose one of the most densely populated, and therefore problematic areas, consisting of three huge round houses in 25-30 entrances, and several 3-4 family access dormitories. By approximate calculations it turned out 23 points, in fact it turned out 25.
For a couple of months, we purchased a cable, the same as for the highways — I liked it very much, the switches — by the decision of the authorities — dlink, 3526 models on houses and 3028G on a star beam, and proceeded to installation.
Physically it was impossible to close the cable to the ring in any way, so they were laid in both directions to the extreme points, but the switches switched on overlapping switches.
The link, coming out of the root switch to the first horn, fell not on the next in order - but through one from it and so on until the end of the “horn”. There he turned around and walked back through the missed switches of the first half, past the root, to the other end of the horn. There he turned around again and, with passes in the second horn, returned to the root.
The resulting ring, according to RSTP, is opened by one of the switches in the middle, which in the event of a signal loss from one side turns to the other, if it is turned off by itself, it simply physically breaks it.
The circuit is simple and stable. When de-energized on one switch, we lose only it. To lose the whole ring is entirely at risk only if two adjacent points are disconnected simultaneously. But this was in my memory once when half of the neighborhood was disconnected.
Client transfer was carried out gradually, after commissioning of the entire ring. As the transfer proceeded, the copper structure was dismantled, all customers were reduced to smart equipment. It was partially used, so we could not immediately turn it off, we did the same as with the optics in the old ring. When customers were carried away from the previous switch, the next one was switched on with a temporary cable from the optical one closest to it.
All work took two months, fixing the erection teams of the assembly team allowed during the transfer - about a month. After that, the neighborhood from the accident generator turned into one of the most silent harbors of the network. For the second year, 5-7 emergency tickets are received from there per month, no more.
As a result, according to the results of the installation and a couple of months of operation, the scheme was approved for further use.
In the same way, we remade another 9 micro-districts in the first “macro-district”, and 7 in the second. The average length of the rings is from 12 to 30 switches. Given that the rings are mostly passive, and only pass BPDU packets - the convergence time is about 5 seconds.
In one neighborhood, the scheme had to be slightly changed. The development did not allow one ring to cover the entire neighborhood. Therefore, a small ring of 12 switches covering half the microdistrict was made, and a separate cable was connected to seven houses remote from the rest of the houses. For them purchased a media converter chassis, and a Dlink 3100-24 switch. Each of the switches on these houses was switched on via a SFP bundle converter to 3100-24. It turned out a kind of regional "mini star".
But the feature on this site is not in the circuit, but in the method of unwinding the cable. Since it was laid in one piece with end-to-end plants to the places of future installation of equipment, it was decided not to cut it at the points, but to get one fiber each. As a result, two centimeter notches were made directly on the cable at a distance of 40-50 cm and the module was opened. In one incision, the desired fiber was sought and eaten, after which the incision was sealed. The cut fiber was carefully removed in the second cut and terminated. The incision site and the fiber supply were retracted to the cassette, and the end switch went to the switch.
The work is difficult and almost jewelry. I spent a week and a half on splitting seven points, but it was worth it. When in the future, when repairing roofs, our cable was bent almost 180 degrees, and judging by the trace of free fiber, in this place, even after extension, there was a decent signal drop - none of the points stopped working. If there were 5-7 connectors on the way, it would hardly have been so rosy.
There are always enough fronts, plans for the summer to transfer four more microdistricts to FTTB (Building), to start changing the ring in one of the “macro districts” for 24 fiber cables, plus the whole administration plans to connect a couple of cottage villages around the city to the network. In addition, recently, businessmen became more active citizens, who in their dreams see unlimited gigabit Internet directly to the office, which does not reduce our troubles.
According to general results, over the last year we had one emergency visit to repair optics, which was torn down by a tree that fell in a thunderstorm, whereas three years ago we went to emergency heels once a month - the bottle flooded after the rain, the signal disappeared, and other troubles welcomed us almost weekly.
The average number of emergency tickets per week, in the past two years, if it does not fall, it does not grow exactly, this is with a triple increase in the subscriber base. In addition, the initial diagnostics on the ticket is now often reduced to viewing the log and the state of the switch port. That is much easier and faster to go to the place of an emergency or assembly crew.
Intra-district rings and trunk lines designed for skipping 1 Gbit / s are now loaded at peak hours by 30-45%, so we have room to grow and the network can work for another couple of years according to the transport capability, although it is already preparing an adequate change ...
Still in the plans, notes on copper house communications, and met interesting in setting up the equipment.
The first part turned out to be interesting, albeit with a bias in “organizational problems”.
I will try to fill the gap and recall the technical aspects of the work. In particular: the organization of the overall network structure, the modernization of the segments without decommissioning, laying and terminating optical cables.
Welcome under the cat:
Network structure
Perhaps it is worth noting that the city is relatively modern, the number of floors of the building is mainly 5-9-16 floors. Geographically, it consists of two “macro-districts” distant from each other, divided into “micro-districts” of 20-40 houses each.
When I first started working, the optical backbone network was made in the form of two rings covering “macro districts”, with one point of presence per “microdistrict”, from which chains of blunt switches on copper twisted pair diverged from house to house. Connectivity between the "macro-regions" was organized by the leased fiber of one of the "opsos".
If you try to recall the scheme, then it resembled the scheme of the Moscow metro, from which a part was removed within the circle line, and looked like this:
At that time, we wanted to get a stable, mainline star, from which only later to organize a smart level of access.
First stage. How it was.
The optical network itself was a terrifying view. Multimode fiber in places already crumbling from old age, covered with splice plates. Long probes could easily be made from several pieces of cable of different fibers, manufacturer, and even b / usnosti. Couplings at the welding sites were rare - most of the connections were packed in a cassette and put in a “beer bottle”, hanging upside down.
The ending in the boxes also did without crosses. Cable, fiber, splice plates and tails were tied with twisted pair pieces to plywood, and went into a switch. At points that were considered “exemplary,” a cassette was used instead of plywood.
The situation was quite normal when a single-mode eight-fiber two-module cable leaving a point, on the way, in a bottle, turned into a single-module one, then, in another bottle, it diverged into two multi-mode 4 fiber cables - one went further into the ring, and another one of some kind of organization ...
Naturally, this structure did not suffer from quality and stability since birth.
')
There were no circuits for optical connections and cross-connects. We collected fragmentary information about the cable structure, which was stored in the heads of installation and engineering, and began to think.
At first, a week without getting out of the roofs, on a satellite image of the city in the “SAS Planet”, they estimated with the routes an approximate diagram of the passage of cables, climbed into all points and bottles, as they could trace the cross. Then they forced the authorities to order a used reflectometer and welding machine.
Dumas ended with the purchase of a bay of 8 fiber single-mode and single-module cables, couplings, 19 inch equipment boxes, crosses and piles of optical passivation.
A month later, we received a technique: an Anritsu reflectometer and an Ericsson FSU 905 welding machine, which to this day serve us faithfully. A month later came the materials.
From that day on, it was announced “Absolutely not” to bottles and splice plates.
Laying a new structure.
Jointly, we decided that we would fix the 8 fiber cable in separate sections along the route of the old rings, and unfold the rings into stars as they were laid, with the collection at the aggregation points. In total, 15 rays of a star in one center and 11 in another were counted, the rings were disconnected on the opposite diameter from the center of aggregation, and each point was switched through its WDM-SFP into its fiber.
They did it as follows: In the center, two ports with conventional duplex SFPs were allocated - in them, already with new fiber, the old ring was decreasing as it worked, and 16 ports with WDM modules. The cable in the aggregation center was welded to the cross, and the first fibers were connected to the six ports of the switch, the last pair was used to support the old structure.
Then, next to the existing one on the ring point, they installed a new box and an electrician for equipment transfer, hung a section of cable, scalded it, transferred equipment, switched a point to one fiber in a new cable, the rest of the fibers were crossed through, except for the last pair that went to the old cable and closed the ever decreasing ring.
Both structures worked simultaneously, only one grew and the second squeezed:
When three points remained in the ring, and the last two fibers were involved on each side, they did not disturb them, laid, terminated and crossed the last sections of the new cable, then the engineers and installers went to the night shift, and in 5 hours of a tight schedule moved and switched all three points.
The second "makrorayon" was remade in the same way. There we did without night work, because the star had 11 rays, and 16 fibers were enough to narrow the ring to two points, and then open the ring and switch them to 1 fiber. We even had a reserve of 4 fibers, well, he does not pull a pocket. True, in the first “macro-district” we have no spare fibers left, since 15 of them were used in the star, and one more from each side we used to connect aggregation centers to each other, but this problem is now being solved.
What happened.
Thus, we got two logical stars on virtually the same rings that were in the beginning. But now each point was completely independent of the performance of the others, equipped with uninterrupted power, optical cross-country, plus we got rid of “jinn in bottles”. In addition, they made their own link between the aggregation points, leaving the rented fiber as a hot reserve.
The idle time of the point and the microdistrict dependent on it, on average, 2-4 hours is quite an acceptable result, given the volume and complexity of the work.
As they were laid, the old sections, and the boxes where the equipment was located, were dismantled. The removed pieces of optics, organoleptically and by reflectograms, were checked for aptitude, and sent to the warehouse. The boxes and the best part of the cables were later used when replacing copper inter-house communications.
Second phase. Access level.
After putting the stars into operation, there were practically no problems on the highways. To this day, the inconvenience is delivered only by long-term power outages, which the bespereboyniki cannot cope with, but this is apparently chronic.
But with the growth of the network and the number of subscribers other problems began to emerge - stupid switches on a twisted pair did not cope with the load from one and a half - two hundred clients in the segment. There are customers who have organized their own "access" networks, with rings and broadband storms from all kinds of borg and ice chats.
There is a need to solve customer access problems.
For this it was necessary to “umnify” at home. And at the same time really do FTTB. At one to five access houses decided to put one switch in the middle of the house, in the attic. In multiple-access buildings - one switch to five entrances.
It was decided to connect the switches into rings, which are closed on the rays of the main star.
For tests and development of technology, we chose one of the most densely populated, and therefore problematic areas, consisting of three huge round houses in 25-30 entrances, and several 3-4 family access dormitories. By approximate calculations it turned out 23 points, in fact it turned out 25.
For a couple of months, we purchased a cable, the same as for the highways — I liked it very much, the switches — by the decision of the authorities — dlink, 3526 models on houses and 3028G on a star beam, and proceeded to installation.
Physically it was impossible to close the cable to the ring in any way, so they were laid in both directions to the extreme points, but the switches switched on overlapping switches.
The link, coming out of the root switch to the first horn, fell not on the next in order - but through one from it and so on until the end of the “horn”. There he turned around and walked back through the missed switches of the first half, past the root, to the other end of the horn. There he turned around again and, with passes in the second horn, returned to the root.
The resulting ring, according to RSTP, is opened by one of the switches in the middle, which in the event of a signal loss from one side turns to the other, if it is turned off by itself, it simply physically breaks it.
The circuit is simple and stable. When de-energized on one switch, we lose only it. To lose the whole ring is entirely at risk only if two adjacent points are disconnected simultaneously. But this was in my memory once when half of the neighborhood was disconnected.
Client transfer was carried out gradually, after commissioning of the entire ring. As the transfer proceeded, the copper structure was dismantled, all customers were reduced to smart equipment. It was partially used, so we could not immediately turn it off, we did the same as with the optics in the old ring. When customers were carried away from the previous switch, the next one was switched on with a temporary cable from the optical one closest to it.
All work took two months, fixing the erection teams of the assembly team allowed during the transfer - about a month. After that, the neighborhood from the accident generator turned into one of the most silent harbors of the network. For the second year, 5-7 emergency tickets are received from there per month, no more.
As a result, according to the results of the installation and a couple of months of operation, the scheme was approved for further use.
In the same way, we remade another 9 micro-districts in the first “macro-district”, and 7 in the second. The average length of the rings is from 12 to 30 switches. Given that the rings are mostly passive, and only pass BPDU packets - the convergence time is about 5 seconds.
In one neighborhood, the scheme had to be slightly changed. The development did not allow one ring to cover the entire neighborhood. Therefore, a small ring of 12 switches covering half the microdistrict was made, and a separate cable was connected to seven houses remote from the rest of the houses. For them purchased a media converter chassis, and a Dlink 3100-24 switch. Each of the switches on these houses was switched on via a SFP bundle converter to 3100-24. It turned out a kind of regional "mini star".
But the feature on this site is not in the circuit, but in the method of unwinding the cable. Since it was laid in one piece with end-to-end plants to the places of future installation of equipment, it was decided not to cut it at the points, but to get one fiber each. As a result, two centimeter notches were made directly on the cable at a distance of 40-50 cm and the module was opened. In one incision, the desired fiber was sought and eaten, after which the incision was sealed. The cut fiber was carefully removed in the second cut and terminated. The incision site and the fiber supply were retracted to the cassette, and the end switch went to the switch.
The work is difficult and almost jewelry. I spent a week and a half on splitting seven points, but it was worth it. When in the future, when repairing roofs, our cable was bent almost 180 degrees, and judging by the trace of free fiber, in this place, even after extension, there was a decent signal drop - none of the points stopped working. If there were 5-7 connectors on the way, it would hardly have been so rosy.
Future plans.
There are always enough fronts, plans for the summer to transfer four more microdistricts to FTTB (Building), to start changing the ring in one of the “macro districts” for 24 fiber cables, plus the whole administration plans to connect a couple of cottage villages around the city to the network. In addition, recently, businessmen became more active citizens, who in their dreams see unlimited gigabit Internet directly to the office, which does not reduce our troubles.
According to general results, over the last year we had one emergency visit to repair optics, which was torn down by a tree that fell in a thunderstorm, whereas three years ago we went to emergency heels once a month - the bottle flooded after the rain, the signal disappeared, and other troubles welcomed us almost weekly.
The average number of emergency tickets per week, in the past two years, if it does not fall, it does not grow exactly, this is with a triple increase in the subscriber base. In addition, the initial diagnostics on the ticket is now often reduced to viewing the log and the state of the switch port. That is much easier and faster to go to the place of an emergency or assembly crew.
Intra-district rings and trunk lines designed for skipping 1 Gbit / s are now loaded at peak hours by 30-45%, so we have room to grow and the network can work for another couple of years according to the transport capability, although it is already preparing an adequate change ...
Still in the plans, notes on copper house communications, and met interesting in setting up the equipment.
Source: https://habr.com/ru/post/95751/
All Articles