Organization of development and operation of large projects

News from the world of DevOps on Highload ++ is presented by Alexander Titov from Express 42

- I still do not understand what is the difference between DevOps and system administration, what? Why DevOps, did they do it well?

Well, in fact, it makes no sense to look for differences, it's like comparing soft with salty. DevOps is about organizing development and operation entirely, about the fact that exploitation should be included at an early stage of development. This requires new qualities from developers and new abilities from system administrators, developers and administrators should learn to work in parallel, and not as developers have programmed, thrown everything into operation, and then begins a long and unpleasant way of analyzing the problem “who is to blame?”. DevOps should be used where the need for a new approach has matured, many companies still do not need it and they cope as before.

- Where is the technological and software component of exploitation going?
IoT, open flow (right?), Software switches, SDN - what is all this?

Now the main problem for many years - the integration of software and hardware components together, different vendors, different standards lead to long and painful processes of coupling systems together. This process is not possible without highly qualified people who, for obvious reasons, cannot be many in the market. Because of this, there is a request for creating flexible systems and platforms that can integrate with each other on their own in order to provide an infrastructure service. We already see how our life has changed with clouds, a whole layer of network operation, raid settings, cooler changes in data centers, queue systems, databases, load balancing, firewall settings and other things went inside the cloud, but the cloud is not optimal and not completely flexible. , we are waiting for the next turn and the creation of hyperblack technologies, which will include SDN, containers and other things that will make the integration of systems easier and more flexible. And the Internet of things will be the driver of this process, it will not be possible to create it without these technologies.

- You have vast experience, is there a standard de facto for Internet projects now? What to write, in terms of stability, reliability?

It is difficult for me to say here, technologies change once every 5 years and we are now at the turn of the sunset of the last technological stack and at the birth of a new one. If we talk about the future, it is Docker, Mesos, Go, Nomad, Cassandra, but it’s far from stability and reliability, and stable is all that people talk about, for example, from banks - Oracle, JBoss, Java and others.

We are now witnessing the next turn in the development of infrastructure, system administration and the delivery of software. The era of virtualization and configuration management systems has ended, the solutions developed at this stage of development have taken their places. Now we are witnessing the emergence of a new ecosystem - a container software delivery system and container resource management.
')
In just a couple of years, the world will again become different again, as it became different with the appearance of clouds and other modem technologies.

And of course at Highload ++ we are closely following the new one and we have several interesting reports for you.

So, about Docker. The pioneers in Docker from Russia - Anton Turetsky from the company Badoo. The guys have been rolling out their software in a new way for a year now and have seen the pros and cons of the approach and have already begun to develop their harness to Docker. We expect from them a very interesting practical report .




The network infrastructure is also changing, no one wants to wait any longer when networkers configure or reconfigure the network, the network in the data center should change along with the business task. People from Moscow State University jumped on this new wave and told about their development . It is necessary to listen only because in this report you can see fragments of the future.

Another report about SDN will be presented by Andrey Korolev - “Looking for the perfect network, or why we need another SDN” - from it you will learn about what you can already do today using open-source products.

Infrastructure management in Big Data is gradually transferred from this narrow segment to IT as a whole. Apache Mesos manages Docker containers and the entire datacenter. Li Jin from Two Sigma will talk about resource management in a cluster based on Apache Mesos.

Alexander Chistyakov , the frequenter of our conferences, will cover the topic of using BigData technologies, specifically the Hadoop technology stack, for ... tax collection! And we did grow up together, @alexclear , went to the same conferences! And now you are helping to collect taxes from us, oh ...
And also you will find many practical reports on the topic "System Administration" .

Alexey Vladyshev, the developer of Zabbix, will tell you about ... Zabbix :) About monitoring simultaneously a large number of metrics, automation and integration with other systems. After all, really, I want to automate everything as much as possible and forget. Is it possible ?



The monitoring topic will be continued by Nikolay Sivko, HeadHunter Operations Director. Nikolai raises the eternal topic - we monitor it, but how can we remove such metrics in order not only to notice the error, but to immediately find its cause? Using the example of hh.ru, we will discuss how to cover all layers of the project, including even the client-side metrics.

In order to have something to monitor, it is important that the user gets to you. And it begins the way with DNS. If you are a small project, you don’t need to worry effectively, if you are a large portal, even such trifles should be completely under control. Konstantin Novakovsky will tell you about the experience with DNS in one of the largest Selectel hosters.

And finally : For the users of Habrakhabr, the conference offers a special discount of 15%, all you need to do is use the code " IAmHabr " when booking tickets.

And the very last thing : The conference is already next week and we will write less often - we will sleep and rest. But then we will come back, you can find new publications in the blog on Habré and in our free newsletters . Looking forward to stay in touch!