IBM Lotus Domino / Notes Outsourcing
This time we decided to diversify our blog and introduce an element of practice. Previously published articles dealt with various technologies, but now it’s time to tell what projects we are implementing. The first will be the project of outsourcing IBM Lotus Domino / Notes infrastructure of a large European financial holding with branches in different parts of the world, which started in 2010. Our company fulfilled the order for this project, having received as a result a successful case in the portfolio. How we do our work, and what knowledge and experience in outsourcing IBM Lotus Domino / Notes are ready to use in the following projects, we will describe further.
Background to the beginning of the project
The customer has been working with the Lotus Domino / Notes product since the early 2000s, and was accompanied by a large staff of its own administrators and developers. For several years after the introduction of IBM Lotus Domino / Notes, the IT service of the customer refined the standard Lotus Notes applications and developed new ones that automated all of the main business processes assigned to Lotus. This idyll lasted until the well-known 2008 economic crisis, when financial problems pushed the company's management to optimize financial performance by offloading its IT staff from non-core and routine tasks and concentrating their forces on the main business processes.
')
As a result, the customer reserved for himself only configuration tasks in the field of information security, audit of the outsourcer’s activities, improvement of the mailbox design to the specifics of its business processes, and was responsible for the business logic of the applications. All the rest was transferred to IT outsourcing.
First stage. On your marks. What was to be done?
The customer's IBM Lotus Domino / Notes infrastructure included dozens of servers running mail, Sametime, and applications. Servers and several thousand users were located in locations around the world on several continents. All this we had to remotely accompany and develop. We had to provide the second and third levels of support, and the first level (Service Desk) the customer decided to give to one of the Eastern European countries, due to insufficient (at that time) "multilingual" specialists of our company.
For the start of the project, only the signed contract was missing, which was realized by the management of our company and passed the baton to the administrators who were to meet the customer and get the first portion of the service.
The first stage began with daily periodic tasks, managing users and groups, solving incidents of the second level of support and providing the service 24x7 . And six months after the start of the project, we began to provide all types of work specified in the contract, including the third level of support, including communication with the vendor.
Second phase. Project implementation.
Almost immediately after gaining access to the IBM Lotus Domino server infrastructure, we found that most of the key business clusters of Domino servers are overloaded and work at their limits. The slightest failure on one of the cluster members led to the fact that user requests were automatically redirected to another server, which soon also became unavailable due to this additional load.
Cluster "fell". And this is despite the fact that at the conclusion of the contract the customer stated that everything works fine for him and server incidents are extremely rare.
As practice shows, when describing the scale of work that is outsourced to a service, customers tend to overestimate the volume. For example, they indicate only the number of incidents, modestly forgetting about service requests or maintenance work. And this must be prepared. The transition to the virtual environment was in the customer's plans, but the ever-growing volumes of information required not only increasing server performance, but also changing the architecture of the existing IBM Lotus Domino / Notes infrastructure while ensuring the smooth operation of the customer's business processes, which we did.
To ensure high availability, scalability, and load balancing, each branch of the customer’s company has one or two dedicated physical servers that host mail, archives, and replicas of mission-critical applications. These servers are clustered with the corresponding servers in different data centers. Thus, in the case of local problems with the server in the branch office, their Notes clients will automatically switch to the server in one of the data centers.
To ensure security, Domino servers are combined into two different Notes domains: “internal” and “external” (mail gateway).
Thanks to optimization, the number of Domino servers has been reduced. Most of them were transferred from locations to data centers at the location of the head office. The locations left the minimum number of servers, which allowed the customer to minimize the cost of their support. The user mode for mailboxes was changed from “On Server” to “On Local”, which led to a significant reduction in the load on mail servers.
The correctness of the concept embodied in the new architecture, which guarantees the continuity of service provision, was also confirmed by regular DR (Disaster Recovery) tests conducted by the customer.
As a result, after some period of adaptation (note that, in general, all customer requirements were met in stages over 8 months), the number of server incidents with Domino servers decreased 2.5 times .
The third stage. Unexpected surprise.
Having dealt with critical server tasks and successfully survived the transition to the new infrastructure in the main locations, we took up proactive work. Under the contract, regular weekly and monthly work proceeded to our area of ​​responsibility at the very end of the transfer of service to us. This was due to the fact that they contained not only the actions recommended by IBM, but also specific actions aimed at maintaining the efficiency of business processes and the integrity of customer information.
And here we were waiting for a surprise. In connection with the optimization of the staff, the remaining administrators of the customer’s job became so much so that they only had time to do critical weekly and monthly tasks. Thus, the rest of the burden fell on thefragile shoulders of the outsourcer, that is, on us.
What was there to do with a lot of extra work? It was decided to urgently automate the processes. In the shortest possible time, we wrote scripts for additional log analysis, comparison of clusters, mail file quotas, group analysis, reporting. All this allowed for 3-4 months to put things in order and, most importantly, the regular full implementation of all maintenance work has led to a reduction not only in the number of server, but also several types of user incidents.
The success of our work was also facilitated by the presence of mature business processes for administering and resolving incidents, their management and monitoring of performance. At first, the specialists had to spend a lot of time studying and understanding them, but later we could appreciate their undoubted benefits by creating new ones and optimizing the old ones on our own.
Results of the project. What did the customer get?
As a result of the project, our company's specialists provided the customer with a stable, properly developing IT infrastructure and the ability to transfer their own IT resources to solve core business problems.
Throughout the project we:
All this allowed our company to significantly expand the scope of services provided to the customer and, after five years of work, extend the contract.
Background to the beginning of the project
The customer has been working with the Lotus Domino / Notes product since the early 2000s, and was accompanied by a large staff of its own administrators and developers. For several years after the introduction of IBM Lotus Domino / Notes, the IT service of the customer refined the standard Lotus Notes applications and developed new ones that automated all of the main business processes assigned to Lotus. This idyll lasted until the well-known 2008 economic crisis, when financial problems pushed the company's management to optimize financial performance by offloading its IT staff from non-core and routine tasks and concentrating their forces on the main business processes.
')
As a result, the customer reserved for himself only configuration tasks in the field of information security, audit of the outsourcer’s activities, improvement of the mailbox design to the specifics of its business processes, and was responsible for the business logic of the applications. All the rest was transferred to IT outsourcing.
First stage. On your marks. What was to be done?
The customer's IBM Lotus Domino / Notes infrastructure included dozens of servers running mail, Sametime, and applications. Servers and several thousand users were located in locations around the world on several continents. All this we had to remotely accompany and develop. We had to provide the second and third levels of support, and the first level (Service Desk) the customer decided to give to one of the Eastern European countries, due to insufficient (at that time) "multilingual" specialists of our company.
For the start of the project, only the signed contract was missing, which was realized by the management of our company and passed the baton to the administrators who were to meet the customer and get the first portion of the service.
The first stage began with daily periodic tasks, managing users and groups, solving incidents of the second level of support and providing the service 24x7 . And six months after the start of the project, we began to provide all types of work specified in the contract, including the third level of support, including communication with the vendor.
Second phase. Project implementation.
Almost immediately after gaining access to the IBM Lotus Domino server infrastructure, we found that most of the key business clusters of Domino servers are overloaded and work at their limits. The slightest failure on one of the cluster members led to the fact that user requests were automatically redirected to another server, which soon also became unavailable due to this additional load.
Cluster "fell". And this is despite the fact that at the conclusion of the contract the customer stated that everything works fine for him and server incidents are extremely rare.
As practice shows, when describing the scale of work that is outsourced to a service, customers tend to overestimate the volume. For example, they indicate only the number of incidents, modestly forgetting about service requests or maintenance work. And this must be prepared. The transition to the virtual environment was in the customer's plans, but the ever-growing volumes of information required not only increasing server performance, but also changing the architecture of the existing IBM Lotus Domino / Notes infrastructure while ensuring the smooth operation of the customer's business processes, which we did.
To ensure high availability, scalability, and load balancing, each branch of the customer’s company has one or two dedicated physical servers that host mail, archives, and replicas of mission-critical applications. These servers are clustered with the corresponding servers in different data centers. Thus, in the case of local problems with the server in the branch office, their Notes clients will automatically switch to the server in one of the data centers.
To ensure security, Domino servers are combined into two different Notes domains: “internal” and “external” (mail gateway).
Thanks to optimization, the number of Domino servers has been reduced. Most of them were transferred from locations to data centers at the location of the head office. The locations left the minimum number of servers, which allowed the customer to minimize the cost of their support. The user mode for mailboxes was changed from “On Server” to “On Local”, which led to a significant reduction in the load on mail servers.
The correctness of the concept embodied in the new architecture, which guarantees the continuity of service provision, was also confirmed by regular DR (Disaster Recovery) tests conducted by the customer.
As a result, after some period of adaptation (note that, in general, all customer requirements were met in stages over 8 months), the number of server incidents with Domino servers decreased 2.5 times .
The third stage. Unexpected surprise.
Having dealt with critical server tasks and successfully survived the transition to the new infrastructure in the main locations, we took up proactive work. Under the contract, regular weekly and monthly work proceeded to our area of ​​responsibility at the very end of the transfer of service to us. This was due to the fact that they contained not only the actions recommended by IBM, but also specific actions aimed at maintaining the efficiency of business processes and the integrity of customer information.
And here we were waiting for a surprise. In connection with the optimization of the staff, the remaining administrators of the customer’s job became so much so that they only had time to do critical weekly and monthly tasks. Thus, the rest of the burden fell on the
What was there to do with a lot of extra work? It was decided to urgently automate the processes. In the shortest possible time, we wrote scripts for additional log analysis, comparison of clusters, mail file quotas, group analysis, reporting. All this allowed for 3-4 months to put things in order and, most importantly, the regular full implementation of all maintenance work has led to a reduction not only in the number of server, but also several types of user incidents.
The success of our work was also facilitated by the presence of mature business processes for administering and resolving incidents, their management and monitoring of performance. At first, the specialists had to spend a lot of time studying and understanding them, but later we could appreciate their undoubted benefits by creating new ones and optimizing the old ones on our own.
Results of the project. What did the customer get?
As a result of the project, our company's specialists provided the customer with a stable, properly developing IT infrastructure and the ability to transfer their own IT resources to solve core business problems.
Throughout the project we:
- provide service with quality and on time;
- used administration automation tools;
- used third-party Domino server monitoring software;
- lined up good relationships with customer representatives;
- prioritized tasks, their solutions and the order of interaction with the customer, based on his business processes and missions;
- worked with the most detailed Service Level Agreement (agreement on the level of service provision);
- Constantly improved service, offering customers new solutions for optimizing IT processes.
All this allowed our company to significantly expand the scope of services provided to the customer and, after five years of work, extend the contract.
Source: https://habr.com/ru/post/255971/
All Articles