Optimization of the git-server
On Habré and in our corporate IaaS blog, we write a lot about cloud technologies and consider interesting infrastructure projects of various companies. Today we will talk about improving the performance of the git-server and take a look at the history and experience of the Clever Cloud project.
/ photo Karl Baron CC
At the start of the project, the Clever Cloud team used gitolite , a utility to manage Git repositories. A ready-made functional solution with the ability to control its configuration made it possible to describe each user and each repository.
')
To automatically update these settings, the Clever Cloud developers even independently created a tool called etilotig. As a result, gitolite itself ceased to satisfy all requests.
There were several reasons: starting with the fact that all repositories were created in one directory (a serious performance problem) and up to the need to rewrite the entire settings file to make even minor changes. Moreover, the creation of a new repository could take up to 5 minutes, which went beyond the limits of reasonable time.
As a result, it was decided to modify its etilotig utility. She was supposed to manage SSH-keys, authorization, creating repositories and installing interceptors.
When etilotig managed only the configuration, it only redirected the gitolite SHH keys, which in turn processed them. Now he needed to also independently process the file authorized_keys. Every time an SSH key was added or deleted, the utility created a new file with keys and replaced the old one.
To manage the repositories, etilotig wrote configuration bash files, thereby defining the main points, for example, the directory in which the new repository will be created. Next came the small bash script .
Authorization management in etilotig is implemented quite simply: when creating an internal configuration, a Perl script is generated that checks the SSH key for each connection attempt. If the key is correct, then the transaction is allowed, otherwise it is prohibited.
A similar approach to increasing productivity reduced the processing time to less than a second (it was 3-5 minutes), and the size of etilotig decreased by 50%. In its personal blog, the Clever Cloud team announced that the average performance increase was 30,000%.
PS But what else do we write in our blog:
/ photo Karl Baron CCAt the start of the project, the Clever Cloud team used gitolite , a utility to manage Git repositories. A ready-made functional solution with the ability to control its configuration made it possible to describe each user and each repository.
')
To automatically update these settings, the Clever Cloud developers even independently created a tool called etilotig. As a result, gitolite itself ceased to satisfy all requests.
There were several reasons: starting with the fact that all repositories were created in one directory (a serious performance problem) and up to the need to rewrite the entire settings file to make even minor changes. Moreover, the creation of a new repository could take up to 5 minutes, which went beyond the limits of reasonable time.
As a result, it was decided to modify its etilotig utility. She was supposed to manage SSH-keys, authorization, creating repositories and installing interceptors.
How did they do it
When etilotig managed only the configuration, it only redirected the gitolite SHH keys, which in turn processed them. Now he needed to also independently process the file authorized_keys. Every time an SSH key was added or deleted, the utility created a new file with keys and replaced the old one.
To manage the repositories, etilotig wrote configuration bash files, thereby defining the main points, for example, the directory in which the new repository will be created. Next came the small bash script .
Authorization management in etilotig is implemented quite simply: when creating an internal configuration, a Perl script is generated that checks the SSH key for each connection attempt. If the key is correct, then the transaction is allowed, otherwise it is prohibited.
Conclusion
A similar approach to increasing productivity reduced the processing time to less than a second (it was 3-5 minutes), and the size of etilotig decreased by 50%. In its personal blog, the Clever Cloud team announced that the average performance increase was 30,000%.
PS But what else do we write in our blog:
- Digest: All about working with IaaS and examples of practical cases for IT-GRAD clients
- VMware NSX: A New Approach to Health Security
- VMware Cloud Foundation: Deploying Software Data Center and Hybrid Cloud
- Comparison of four top disaster recovery solutions
- Feasibility study of IaaS on the example of a large company case
- The evolution of ERP-systems: the rejection of on-premises-installations in favor of the cloud
- IT Forecast 2017: review of key technologies that expects significant growth
Source: https://habr.com/ru/post/265957/
All Articles