How not to waste time on continuous integration
Any software company sooner or later comes to the need to introduce continuous integration, whether it is developing a mobile application, creating a regular website, or even more so if it is a complex micro-service structure.
Earlier, I wrote that our team is developing iOS and turnkey Android applications, including backend and frontend. This implies an urgent need for a scalable distribution process and well-established project management standards.
But one thing is when you have a single product where you can do it somehow, and another is if a new team with special requirements appears every month with a big team for which you need to quickly get a CI / CD.
')
You, probably, thought that now there will be an instruction for setting up this integration itself, where I will share the best practices, scripts, recommendations on the infrastructure and other life hacking.
I thought so too.
But suddenly I realized that everything is just copy-paste on an industrial scale, and the right decision lies not behind the instruction, but behind the tool. A tool that analyzes, configures and assembles projects independently. In the end, continuous integration was designed to simplify life, and not to add new problems with the need to create huge tutorials for writing scripts of the same exorbitant size. And then also support.
For this reason, I am greatly encouraged by services such as Travis and Circle for the capacity of their language, the severity and availability of infrastructure, but nevertheless, this is not enough. All this imposes responsibilities for the study of their guides, requires a deep understanding of the features of the platform on which the product is developed. It is not enough just to be able to do iOS or Android applications, you need to know the features of the SDK and the device they have under the hood.
You can say that it is normal that every developer should know his platform and be able to configure the build from the command line, but I do not agree. The developer should be able to develop, write the product, know the methods of the SDK, suitable libraries and be able to use them. But do not understand the distribution systems, scripting languages and infrastructure. It's like before to create a simple site you had to be a little more system administrator. I exaggerate a little, but I think you understand.
In order not to reinvent the wheel, I had to go deep into the Internet, where I hoped to find existing services and test their strength. As everyone knows, the idea never comes only in one head.
I decided to take a test project, which I will give to the experiences of all the services found.
The criterion of success will be the ability to conduct a full cycle of continuous integration - from receiving the source code to deploying to Crashlytics.
For a start, we will focus on iOS, as I personally, as an author, have more expertise in this area, and secondly, the creation of apple has always been famous for its puzzling setting due to certificates and other obstacles.
As a sample for testing, let's take the good old project that I am mobbing for in each of my articles. And so, let's go.
After a long dive right up to the 20th page of Google, I found three products that fit the description. Two of them did not start up in principle, so we will not even consider them, and the third one, although it worked, but somehow: I could not determine the appropriate version of Xcode, inexplicably took Carthage and Cocoapods for an inexplicable time (dependencies for the uninitiated), and it seemed That about a cache of any kind basically did not hear. And in the end zafeil assembly, as expected:
Oh, those tales. Without false modesty, I realized that I can do better now. Because besides the one ambitious wording, there was also a clear understanding that competitors do not work where, how to fix it and do it as a whole better and faster.
Inspired by the idea, together with colleagues who supported the idea, we sat down to develop a prototype. I’ll omit the pompous story about how we worked day and night under 'Eye of the tiger', and instead of lush marketing I’ll tell you better what we have achieved, which chips have been implemented and what is really working now.
The main goal was to fully automate the setting of continuous integration for all, so that even the manager could build a normal business process in his company, having only a link to the repository and with a complete lack of technical knowledge. We even conducted an experiment by asking a non-iOS developer to configure continuous integration using our creation. We were very pleased that he had succeeded. It was a signal that we are moving in the right direction.
A small demo video about how everything happens.
Compared to Travis, with equal resources, projects are collected 4-5 times faster due to optimization of dependencies and no need to virtualize each individual worker and deliver the missing software to him.
I forgot to say. The project decided to call Buildben. At first they wanted just Ben, in honor of our wonderful office cat, and then the word-play between Big Ben and Build Ben was born.
But we digress. You probably already understood that the class does everything on its own, the cat is awesome, we are great and all that. As they say - what will surprise?
And here is what. We have specially prepared several special features that you will like:
At one point, we realized that we were cooking in our own boiler for too long, and almost no one heard about the service, except for a couple of acquaintances and friends. And the money is not rubber, the cloud in Amazon does not pay for itself, you know.
It was decided to go look for an investor in the best traditions of startups. It did not take long to search, which is surprising, they became interested in us and made an appointment.
They asked a lot of questions, asked about experience as teams, which projects were developed,who we see ourselves after 5 years , what our target audience was, asked to count many different complex numbers, and a dozen other strange indicators, reading in parallel lectures on how to correctly live. In the end, we were told the following:
" - Gentlemen, everything is great, the project is interesting. But there is one thing. You need users. Without them, there is no certainty that the project is really in demand and should be developed. Find at least a couple of dozen people and companies who want to use your services and we continue the conversation. "
And where can a techie find a techie who loves new products and is ready to go first? Of course here on Habré. My little acquaintance with the audience on several written articles showed that the people here, although they like to criticize, are sensible and inquisitive.
I am not a salesman and a marketer and I don’t know how to talk about love beautifully, but if you’ve read it up to this point, then you’ve found the topic interesting and we’ll be happy if you use it.
We have a small landing site where you can submit an application for beta testing, you are welcome. Each application triggers my alarm clock, and I run immediately to respond. But, unfortunately, our resources are now very limited, and we cannot invite everyone, but we will try to the maximum.
At the end of the post I will add a FAQ, which will be replenished as communication in the comments.
Security issues can be found there now. This is a particularly important aspect that everyone understands.
I also want to say, and even duplicate, that the development of the boxed version, which you can deploy in yourself and forget about paranoia, is among our top priorities and we know its necessity.
Perhaps this is all I wanted to tell. I will be happy to answer questions, listen to advice and sanctify in more detail any aspects.
Oh, by the way, here is the cat I spoke of:
Miracle, isn't it?
And traditionally I want to share useful links:
UPD1: Oops! Unexpectedly for us, we received dozens of applications and a confident list of bug reports.
Forced to suspend the admission of new members until next Monday.
We will pay attention to all new users, thank you for being with us.
Question: Why can I be sure that my code will not leak anywhere?
Answer: First of all, this is our main primary responsibility. We do not read your code and no one outside can access it, everything is hidden in an individual sandbox for each project. And even more, we immediately remove it after the assembly, leaving only the logs. This is written in our confidentiality agreement, which you sign up automatically by registering with our service. Those. We write in the confidentiality agreement that we delete the source code after assembly.
Q: Well, what about access to iTunes Connect? How can I be sure that you will not break our application?
Answer: We recommend using a separate account for external access, regardless of whether you use our service or your own script. You can define the border where you want our product to go. For our part, we are more interested in the preservation of information.
Question: Do you plan to release a boxed version that you could deploy and not worry about security?
Answer: We plan very soon. This is one of the priorities.
Earlier, I wrote that our team is developing iOS and turnkey Android applications, including backend and frontend. This implies an urgent need for a scalable distribution process and well-established project management standards.
But one thing is when you have a single product where you can do it somehow, and another is if a new team with special requirements appears every month with a big team for which you need to quickly get a CI / CD.
')
You, probably, thought that now there will be an instruction for setting up this integration itself, where I will share the best practices, scripts, recommendations on the infrastructure and other life hacking.
I thought so too.
Start
But suddenly I realized that everything is just copy-paste on an industrial scale, and the right decision lies not behind the instruction, but behind the tool. A tool that analyzes, configures and assembles projects independently. In the end, continuous integration was designed to simplify life, and not to add new problems with the need to create huge tutorials for writing scripts of the same exorbitant size. And then also support.
For this reason, I am greatly encouraged by services such as Travis and Circle for the capacity of their language, the severity and availability of infrastructure, but nevertheless, this is not enough. All this imposes responsibilities for the study of their guides, requires a deep understanding of the features of the platform on which the product is developed. It is not enough just to be able to do iOS or Android applications, you need to know the features of the SDK and the device they have under the hood.
You can say that it is normal that every developer should know his platform and be able to configure the build from the command line, but I do not agree. The developer should be able to develop, write the product, know the methods of the SDK, suitable libraries and be able to use them. But do not understand the distribution systems, scripting languages and infrastructure. It's like before to create a simple site you had to be a little more system administrator. I exaggerate a little, but I think you understand.
Study
In order not to reinvent the wheel, I had to go deep into the Internet, where I hoped to find existing services and test their strength. As everyone knows, the idea never comes only in one head.
I decided to take a test project, which I will give to the experiences of all the services found.
The criterion of success will be the ability to conduct a full cycle of continuous integration - from receiving the source code to deploying to Crashlytics.
For a start, we will focus on iOS, as I personally, as an author, have more expertise in this area, and secondly, the creation of apple has always been famous for its puzzling setting due to certificates and other obstacles.
As a sample for testing, let's take the good old project that I am mobbing for in each of my articles. And so, let's go.
After a long dive right up to the 20th page of Google, I found three products that fit the description. Two of them did not start up in principle, so we will not even consider them, and the third one, although it worked, but somehow: I could not determine the appropriate version of Xcode, inexplicably took Carthage and Cocoapods for an inexplicable time (dependencies for the uninitiated), and it seemed That about a cache of any kind basically did not hear. And in the end zafeil assembly, as expected:
Oh, those tales. Without false modesty, I realized that I can do better now. Because besides the one ambitious wording, there was also a clear understanding that competitors do not work where, how to fix it and do it as a whole better and faster.
Job
Inspired by the idea, together with colleagues who supported the idea, we sat down to develop a prototype. I’ll omit the pompous story about how we worked day and night under 'Eye of the tiger', and instead of lush marketing I’ll tell you better what we have achieved, which chips have been implemented and what is really working now.
The main goal was to fully automate the setting of continuous integration for all, so that even the manager could build a normal business process in his company, having only a link to the repository and with a complete lack of technical knowledge. We even conducted an experiment by asking a non-iOS developer to configure continuous integration using our creation. We were very pleased that he had succeeded. It was a signal that we are moving in the right direction.
A small demo video about how everything happens.
Compared to Travis, with equal resources, projects are collected 4-5 times faster due to optimization of dependencies and no need to virtualize each individual worker and deliver the missing software to him.
I forgot to say. The project decided to call Buildben. At first they wanted just Ben, in honor of our wonderful office cat, and then the word-play between Big Ben and Build Ben was born.
But we digress. You probably already understood that the class does everything on its own, the cat is awesome, we are great and all that. As they say - what will surprise?
And here is what. We have specially prepared several special features that you will like:
- Caching the Carthage frameworks in the cloud. What brought down once, does not build again.
If someone did not come across, then there is such a problem that many github frameworks do not provide a compiled binary. And if they provide, then only under a certain Swift. We have made our own library repository for each version of Swift, which is regularly updated as we work. - A utility for self-configuration of the developer’s iOS office. When deploying an application, you constantly have to think about certificates, profiles, bundle id, all entitlements, icloud and others. We took it all on ourselves and automated it.
- Tool for preparing Crashlytics application. Everyone who has worked with Crashlytics knows that when you create a new application, or at least just change its bundle id, you have to start it at least once to create it. And if you need an application with a new bundle id check? Or would you just like to use distribution delights in a Krashlik without any hassle with the SDK?
So, everything now works as it should. It is enough to choose Crashlytics in the admin panel, then it will go on.
We plan to make it possible to simply drive in the login-password from Fabric or, if you do not have an account, register it yourself without dancing with tambourines. - Asynchronous build. If something can be done at the same time, we do it. For example, Cocoapods & Carthage are installed in parallel on different machines, and then merge together. As a result, tasks that do not require powerful resources are performed by appropriate micro-processes, allowing you to concentrate all power on compilation.
Next step
At one point, we realized that we were cooking in our own boiler for too long, and almost no one heard about the service, except for a couple of acquaintances and friends. And the money is not rubber, the cloud in Amazon does not pay for itself, you know.
It was decided to go look for an investor in the best traditions of startups. It did not take long to search, which is surprising, they became interested in us and made an appointment.
They asked a lot of questions, asked about experience as teams, which projects were developed,
" - Gentlemen, everything is great, the project is interesting. But there is one thing. You need users. Without them, there is no certainty that the project is really in demand and should be developed. Find at least a couple of dozen people and companies who want to use your services and we continue the conversation. "
And where can a techie find a techie who loves new products and is ready to go first? Of course here on Habré. My little acquaintance with the audience on several written articles showed that the people here, although they like to criticize, are sensible and inquisitive.
I am not a salesman and a marketer and I don’t know how to talk about love beautifully, but if you’ve read it up to this point, then you’ve found the topic interesting and we’ll be happy if you use it.
We have a small landing site where you can submit an application for beta testing, you are welcome. Each application triggers my alarm clock, and I run immediately to respond. But, unfortunately, our resources are now very limited, and we cannot invite everyone, but we will try to the maximum.
Important nuances
At the end of the post I will add a FAQ, which will be replenished as communication in the comments.
Security issues can be found there now. This is a particularly important aspect that everyone understands.
I also want to say, and even duplicate, that the development of the boxed version, which you can deploy in yourself and forget about paranoia, is among our top priorities and we know its necessity.
Conclusion
Perhaps this is all I wanted to tell. I will be happy to answer questions, listen to advice and sanctify in more detail any aspects.
Oh, by the way, here is the cat I spoke of:
Miracle, isn't it?
And traditionally I want to share useful links:
- Crashlytics is an incredibly convenient and free distribution system. It has good tutorials and a distinct SDK for iOS & Android, I recommend.
- Mattermost - chat for team communication, like Slack, only self-hosted and free. Easily integrates with everything you can. If you are still discussing business issues in a telegram, then you are here.
- Fastlane is a swiss knife for continuous integration. If you are doing a deployment yourself, then this is just a must have. True, it requires some knowledge of Ruby.
UPD1: Oops! Unexpectedly for us, we received dozens of applications and a confident list of bug reports.
Forced to suspend the admission of new members until next Monday.
We will pay attention to all new users, thank you for being with us.
FAQ
Question: Why can I be sure that my code will not leak anywhere?
Answer: First of all, this is our main primary responsibility. We do not read your code and no one outside can access it, everything is hidden in an individual sandbox for each project. And even more, we immediately remove it after the assembly, leaving only the logs. This is written in our confidentiality agreement, which you sign up automatically by registering with our service. Those. We write in the confidentiality agreement that we delete the source code after assembly.
Q: Well, what about access to iTunes Connect? How can I be sure that you will not break our application?
Answer: We recommend using a separate account for external access, regardless of whether you use our service or your own script. You can define the border where you want our product to go. For our part, we are more interested in the preservation of information.
Question: Do you plan to release a boxed version that you could deploy and not worry about security?
Answer: We plan very soon. This is one of the priorities.
Source: https://habr.com/ru/post/320404/
All Articles