Interview with Monty Taylor, OpenStack Continuous Integration Project Manager
This is the second interview from our series of interviews with OpenStack project managers on the Mirantis blog. Our goal is to educate the wider technical community and help people understand how they can contribute to and benefit from the OpenStack project. Naturally, below is the point of view of the interviewee, not of Mirantis. The interview is published with bills in connection with the restriction of the length of the article.
Our second interview is with Monty Taylor, OpenStack CI Continuous Integration technical project manager.
Mirantis: Please introduce yourself.
')
Monty Taylor: I work at Hewlett Packard on the OpenStack project, participate in the OpenStack Foundation Board Commission and the OpenStack Technical Committee and lead the project for continuous integration and the OpenStack CI and Infrastructure Project.
Q: How did you get started with the OpenStack project?
Answer: I worked at Rackspace, in the team that initially raised the OpenStack project. I'm here from the beginning. You can still see my name under some projects in Launchpad, since I helped to create them, and you cannot make changes to Launchpad.
Q: Why are you working with OpenStack?
Answer: I firmly believe in the importance of an open cloud and in the fact that everyone will gradually move towards managed services and cloud applications.
Q: What is your responsibility as a project manager?
Answer: Our team is famous for self-motivation, it is a team of independent-minded professionals. Instead of telling someone on the team what to do and how, we work together. In terms of leadership, I rather ambassador, the external interface for communicating with people. I work with companies that provide us with resources, and I make every effort to do this.
Q: Can you explain what Continuous Integration (CI) is? Why is it so important to OpenStack?
Answer: We started the project with the idea of ​​automatic control of the state of the code. Instead of assigning a special person responsible for injecting new code into a master branch, which is a very boring task, we implemented it automatically.
One of the aspects of our system is running tests on every loaded code change. After checking and approving the code, the tests are conducted again, since the development speed of OpenStack is so great that you can be pretty sure that the project you are planning to change has already progressed since you wrote the code. Thus, we want to make sure that nothing that is added to the code will not violate its current state.
Now we have 800 developers actively working on the OpenStack project. If we allow any of them to break something, this one will affect the other 799, which will lead to a waterfall effect. Everyone will be upset, and no one will be able to act further, as they will clean up the rest.
Therefore, our work is important for the OpenStack project for reasons of quality and speed of development.
Q: What is truly unique about the OpenStack CI project compared to other continuous integration systems, such as Jenkins or Buildbot?
Answer: We take away the right to edit the code from everyone, no one can make changes directly. This is a fairly tough position. We have identified it, which in my opinion is unique for such projects.
Jenkins is one of the elements of the system that we use, but it is not fully integrated into what we are trying to do here. Therefore, we have added some tools to build our continuous integration system.
Jim Blair, one of the key members of the infrastructure team, wrote a tool called Zuul, which can be said to manage the embedding of the code for the entire project. It also fulfills our stringent performance requirements. It periodically tracks changes to OpenStack, which is most convenient. Suppose you want to run 150 patches, and checking each patch takes an hour, but you do not have 150 hours per day. Zuul can correctly test these fixes in parallel.
With the Zuul tool, we push the boundaries in terms of adding code to the project. HP now uses Zuul for internal processes, Wikipedia has begun using it, and we see more and more people picking up this tool. We give guidance in the field not only in terms of cloud software development.
Q: What has the Continuous Integration (CI) community achieved at the moment?
Answer: Only four team members keep the project on their shoulders, and we continue to release releases. It's great.
All the infrastructure that is running for the OpenStack project is self-managed in public repositories. Therefore, anyone who wants to make a contribution can do it. I think this is a significant achievement, and it is very good that we were able to attract significant community participation.
Q: What features does continuous integration provide in the Havana release?
Answer: We are currently working on patching Jenkins and Zuul for greater scalability. We build technology called Gearman into this mix, which will allow us to have several master branches in Jenkins - this is something that no one else has. Thus, we have no bottleneck in terms of scalability of a single Jenkins wizard.
We also began to raise Logstash, the central logger. Typically, getting the data to debug the test after a failure was a problem.
In addition, we are working on collecting test results at the output in stream mode, so we can give developers more feedback. As soon as we learn that one of the tests for any reason led to an error, we can inform the developers that the entire series of tests will lead to an error. Thus, they do not have to wait for the end of the hourly test to receive debug information.
The essence of our work is to provide feedback to developers to continue to work flexibly without reducing the requirements for what it means to issue updates in OpenStack.
Question: You talked about the maximum democratization of OpenStack tools. What does this mean to you?
Answer: We must move as far as possible from a situation where one person has too much power or responsibility, and this is important not only for the tools, but also for the entire OpenStack project. This is not only for fear of getting too much influence. Although anyone can come and fix the problem, basically the project is driven by the same four people. This means that the project has a safety net.
It also means that you don’t need to convince someone of prioritizing your problem over the others, which leads to political games. You have five people who need something, and one of them has many friends, so everyone is working on the problem of this person. If everyone has equal access, this person can just do the work - without having to convince anyone else that the work is worth the effort.
More than 200 companies are participating in the OpenStack project. If any of them wants to do more tests, for example, on Solaris, I have to provide them with this opportunity, and not make them convince me that this is important, and then train me. This is unfair for me and for them.
Question: It seems that you managed to achieve a balance between optimism and realism towards people.
Answer: Yes. If it were not for optimism, I would not be engaged in joint development with other people, because I would not be sure that anyone has any good ideas. At the same time, you need to recognize that people may be wrong. I know that I am constantly mistaken and I want to make some tool that would track my errors so that I could concentrate on my task, and not on syntax errors. Computers can work for me.
Question: Speaking from a philosophical point of view, what is your vision of the OpenStack project?
Answer: The Internet itself does an excellent job of distributing applications with a standard interface. There are many web servers, but from a consumer point of view, they all work the same way. In terms of cloud computing, the situation is different. Amazon, GoGrid, Rackspace, Cloud.com, all these different clouds are emerging, and they all want to innovate, which is very good.
There are positive and negative sides to this. The positive side - they all just solve a problem that no one has solved before. Negative - each of these companies wants to retain a customer, similar to Prodigy and CompuServe before standardizing the global network.
I would really like the open cloud to be based on OpenStack and even more so, as one friend of mine on Facebook recently put it: “Everyone who uses the word“ cloud ”to describe a distributed application only in their own data center misunderstand this word. They are actually talking about the cluster. ” And I agree with that. The cloud is 50, 100, 1000 companies, all working on OpenStack. As a consumer, I just want to use this aggregated cloud without having to rewrite my application or even without knowing that my application is running on five networks of different providers.
Q: What would you like people to know about continuous integration (CI)?
Answer: I am not sure that everyone knows that this is an open system into which anyone can join, and that we will gladly accept anyone and we are waiting for everyone.
Question: How can I start working on a project?
Answer: To start, we have the # OpenStack-Intra IRC channel and we are running a bug track on Launchpad . It is best to talk a little there and track code revisions, adding comments or feedback, thus understanding the current issues at the moment.
Q: Do you think the OpenStack project has changed open source projects at all? And if so, how?
Answer: Many companies work on the OpenStack project at the same time and we managed to achieve a lot under such initial conditions, when several corporations are involved. It seems that this is the first such project. Linux involved many corporations, but it took them a lot of time ... IBM and all these corporate participants did not immediately turn on. It was a long way to attract people in their free time.
No one works on OpenStack in their free time. This is a commercial project, the requirements for which are determined by the requirements of the business. At the same time, all these companies merged without a clear leader. As a CI project manager, I think that we are helping to change people's vision regarding the use of open source in a test environment with a tough infrastructure, a manageable state of the code, and all other features. Therefore, I think we explore new lands both technologically and organizationally.
Question: Thank you very much, Monty!
Answer: Not at all.
Our second interview is with Monty Taylor, OpenStack CI Continuous Integration technical project manager.
Mirantis: Please introduce yourself.
')
Monty Taylor: I work at Hewlett Packard on the OpenStack project, participate in the OpenStack Foundation Board Commission and the OpenStack Technical Committee and lead the project for continuous integration and the OpenStack CI and Infrastructure Project.
Q: How did you get started with the OpenStack project?
Answer: I worked at Rackspace, in the team that initially raised the OpenStack project. I'm here from the beginning. You can still see my name under some projects in Launchpad, since I helped to create them, and you cannot make changes to Launchpad.
Q: Why are you working with OpenStack?
Answer: I firmly believe in the importance of an open cloud and in the fact that everyone will gradually move towards managed services and cloud applications.
Q: What is your responsibility as a project manager?
Answer: Our team is famous for self-motivation, it is a team of independent-minded professionals. Instead of telling someone on the team what to do and how, we work together. In terms of leadership, I rather ambassador, the external interface for communicating with people. I work with companies that provide us with resources, and I make every effort to do this.
Q: Can you explain what Continuous Integration (CI) is? Why is it so important to OpenStack?
Answer: We started the project with the idea of ​​automatic control of the state of the code. Instead of assigning a special person responsible for injecting new code into a master branch, which is a very boring task, we implemented it automatically.
One of the aspects of our system is running tests on every loaded code change. After checking and approving the code, the tests are conducted again, since the development speed of OpenStack is so great that you can be pretty sure that the project you are planning to change has already progressed since you wrote the code. Thus, we want to make sure that nothing that is added to the code will not violate its current state.
Now we have 800 developers actively working on the OpenStack project. If we allow any of them to break something, this one will affect the other 799, which will lead to a waterfall effect. Everyone will be upset, and no one will be able to act further, as they will clean up the rest.
Therefore, our work is important for the OpenStack project for reasons of quality and speed of development.
Q: What is truly unique about the OpenStack CI project compared to other continuous integration systems, such as Jenkins or Buildbot?
Answer: We take away the right to edit the code from everyone, no one can make changes directly. This is a fairly tough position. We have identified it, which in my opinion is unique for such projects.
Jenkins is one of the elements of the system that we use, but it is not fully integrated into what we are trying to do here. Therefore, we have added some tools to build our continuous integration system.
Jim Blair, one of the key members of the infrastructure team, wrote a tool called Zuul, which can be said to manage the embedding of the code for the entire project. It also fulfills our stringent performance requirements. It periodically tracks changes to OpenStack, which is most convenient. Suppose you want to run 150 patches, and checking each patch takes an hour, but you do not have 150 hours per day. Zuul can correctly test these fixes in parallel.
With the Zuul tool, we push the boundaries in terms of adding code to the project. HP now uses Zuul for internal processes, Wikipedia has begun using it, and we see more and more people picking up this tool. We give guidance in the field not only in terms of cloud software development.
Q: What has the Continuous Integration (CI) community achieved at the moment?
Answer: Only four team members keep the project on their shoulders, and we continue to release releases. It's great.
All the infrastructure that is running for the OpenStack project is self-managed in public repositories. Therefore, anyone who wants to make a contribution can do it. I think this is a significant achievement, and it is very good that we were able to attract significant community participation.
Q: What features does continuous integration provide in the Havana release?
Answer: We are currently working on patching Jenkins and Zuul for greater scalability. We build technology called Gearman into this mix, which will allow us to have several master branches in Jenkins - this is something that no one else has. Thus, we have no bottleneck in terms of scalability of a single Jenkins wizard.
We also began to raise Logstash, the central logger. Typically, getting the data to debug the test after a failure was a problem.
In addition, we are working on collecting test results at the output in stream mode, so we can give developers more feedback. As soon as we learn that one of the tests for any reason led to an error, we can inform the developers that the entire series of tests will lead to an error. Thus, they do not have to wait for the end of the hourly test to receive debug information.
The essence of our work is to provide feedback to developers to continue to work flexibly without reducing the requirements for what it means to issue updates in OpenStack.
Question: You talked about the maximum democratization of OpenStack tools. What does this mean to you?
Answer: We must move as far as possible from a situation where one person has too much power or responsibility, and this is important not only for the tools, but also for the entire OpenStack project. This is not only for fear of getting too much influence. Although anyone can come and fix the problem, basically the project is driven by the same four people. This means that the project has a safety net.
It also means that you don’t need to convince someone of prioritizing your problem over the others, which leads to political games. You have five people who need something, and one of them has many friends, so everyone is working on the problem of this person. If everyone has equal access, this person can just do the work - without having to convince anyone else that the work is worth the effort.
More than 200 companies are participating in the OpenStack project. If any of them wants to do more tests, for example, on Solaris, I have to provide them with this opportunity, and not make them convince me that this is important, and then train me. This is unfair for me and for them.
Question: It seems that you managed to achieve a balance between optimism and realism towards people.
Answer: Yes. If it were not for optimism, I would not be engaged in joint development with other people, because I would not be sure that anyone has any good ideas. At the same time, you need to recognize that people may be wrong. I know that I am constantly mistaken and I want to make some tool that would track my errors so that I could concentrate on my task, and not on syntax errors. Computers can work for me.
Question: Speaking from a philosophical point of view, what is your vision of the OpenStack project?
Answer: The Internet itself does an excellent job of distributing applications with a standard interface. There are many web servers, but from a consumer point of view, they all work the same way. In terms of cloud computing, the situation is different. Amazon, GoGrid, Rackspace, Cloud.com, all these different clouds are emerging, and they all want to innovate, which is very good.
There are positive and negative sides to this. The positive side - they all just solve a problem that no one has solved before. Negative - each of these companies wants to retain a customer, similar to Prodigy and CompuServe before standardizing the global network.
I would really like the open cloud to be based on OpenStack and even more so, as one friend of mine on Facebook recently put it: “Everyone who uses the word“ cloud ”to describe a distributed application only in their own data center misunderstand this word. They are actually talking about the cluster. ” And I agree with that. The cloud is 50, 100, 1000 companies, all working on OpenStack. As a consumer, I just want to use this aggregated cloud without having to rewrite my application or even without knowing that my application is running on five networks of different providers.
Q: What would you like people to know about continuous integration (CI)?
Answer: I am not sure that everyone knows that this is an open system into which anyone can join, and that we will gladly accept anyone and we are waiting for everyone.
Question: How can I start working on a project?
Answer: To start, we have the # OpenStack-Intra IRC channel and we are running a bug track on Launchpad . It is best to talk a little there and track code revisions, adding comments or feedback, thus understanding the current issues at the moment.
Q: Do you think the OpenStack project has changed open source projects at all? And if so, how?
Answer: Many companies work on the OpenStack project at the same time and we managed to achieve a lot under such initial conditions, when several corporations are involved. It seems that this is the first such project. Linux involved many corporations, but it took them a lot of time ... IBM and all these corporate participants did not immediately turn on. It was a long way to attract people in their free time.
No one works on OpenStack in their free time. This is a commercial project, the requirements for which are determined by the requirements of the business. At the same time, all these companies merged without a clear leader. As a CI project manager, I think that we are helping to change people's vision regarding the use of open source in a test environment with a tough infrastructure, a manageable state of the code, and all other features. Therefore, I think we explore new lands both technologically and organizationally.
Question: Thank you very much, Monty!
Answer: Not at all.
Source: https://habr.com/ru/post/184146/
All Articles