How we implement Open source in state-owned companies and large-scale commerce in Russia

CROC is one of the largest IT infrastructure building companies in Russia. We have 2000+ projects per year, of which 200 are with a budget of more than a million. Dollars. Inside this huge company is my small department , which is committed to putting open source for large customers.

Often one has to come across the opinion that open source is the curves of the handicrafts of garage programmers. We only work with normal working software; we help to find the best solutions for infrastructure and provide support where the manufacturer cannot do this for obvious reasons. This is not the easiest job, mainly related to open source myths and legends.
')
But to hell with details, let's show you what and how you can put on workstations, servers, in the cloud, for DBMS, VDI and application servers. And about the tie.

Below in the lists is what I tested in practice, including in large companies, and I can recommend, based on my experience or the experience of colleagues from my department. If you add proven solutions for your tasks in the comments, it will also help a lot.

Server OS
- CentOS
- Open SUSE and SUSE Linux Enterprise Server
- alt linux
- Red Hat Enterprise Linux Server.

Server OS comes with both commercial support and completely free. SLES and RH are officially supported by the overwhelming majority of hardware manufacturers. The OS theme in the open source is one of the most developed, their functionality in many areas exceeds Windows. OS Redhat, SLES and ALT Linux are FSTEC certified.

We use Red Hat in our public cloud (more than 70 large customers), in general, much has been done in our cloud based on open source solutions by our developers. Plus, we put open source operating systems in Moscow courts, as well as in Moscow DIT for the organization of the Unified Medical Information and Analytical System (EMIAS).

Operating systems for workstations
- alt linux
- Red Hat Enterprise Linux Desktop
- SUSE Linux Enterprise Desktop
- Ubuntu Linux.

Linux-based user OSs usually come with a bunch of server lotions and can be compared with this parameter. It turns out that the user can simply get lost in a heap of different programs and modules if he does not possess at least basic knowledge. Therefore, almost no large-scale installation can do without accurate processing with a jigsaw. But, nevertheless, it is necessary to say all the same that most of the basic functions for users go out of the box. And they can be used after a little training. Education here, by the way, will be one of the most important factors during migration.

For example, in EMIAS more than 20 thousand workplaces with ALT Linux and 1500 with SLED are used. Of course, they were reworked for the needs of the customer, but to a greater extent only to remove too much of them, although there were a number of specific requirements that had to be completed together by the developers of these operating systems. However, this was also done. Therefore, I am sure, and our experience speaks about it, there are all possibilities for using these operating systems in enterprises.

Backup
- Bacula
- Amanda
- bareos
- Comodo.

In the field of backup there are several products that I would like to say a few words. I think everyone heard the names of Bacula and Amanda on the basis of which a number of commercial forks were made with support.

It is clear that these solutions cannot be compared with normal industrial backup tools that require a huge number of connectors compatible with various enterprise-level software. Nevertheless, the solutions listed above are quite suitable for small and medium businesses. They have all the basic functions, although backup of complex applications here and does not smell. But, since this is an open source, we can modify it ourselves, right? Or, if you still need more complicated functions, paid forks offer both agents for applications and advanced features such as deduplication, for example. But this is already for money comparable to commercial solutions.

Therefore, there is no definite answer here, whether it is worth saving on backup, or is it easier to take Russian Acronis somehow and sleep peacefully.

Virtualization
Virtualization actually has now reached such a distribution and has become so commonplace that it turned out that open source is comparable to commercial products. And just in the field of server virtualization, and in the field of cloud infrastructure. I must say that even in the clouds is one of the drivers of development now.

Cloud infrastructure is, of course, Openstack. You can build a private cloud company (Software Defined Datacenter) using almost any hardware available for medium and large businesses. Today this solution is most interesting for large industrial enterprises, banks and retailers. The solution is so developed by the community that many companies in the west replace their commercial installations with it. By the way, almost all major vendors have their own Openstack builds.

And, actually, its trick is that there is everything that is needed for basic virtualization, and there is everything that is needed for “advanced”. Namely, object data storage, storage and network virtualization and automation engine.

And, actually, why I am telling this. Many companies now want this kind of technology to increase the flexibility of infrastructure, cost management, speed up operations and the like. But they cannot afford it simply because it is extremely expensive if we take commercial products.

For example, we are now offering our customers systems to automate the testing environment, which accelerates the development of percentages by 20–30% simply by leveling many manual operations.
By the way, a little about these advanced features themselves.

Block and object data storage
- Ceph (Red Hat / intank)
- GlusterFS.

The same Ceph can be an excellent alternative for small offices and can work directly on the hypervisors themselves, if you put more disks in them. Thus, it is possible to connect the hypervigil with the storage system and significantly save on the equipment. But, of course, all this requires a fairly fine initial setup and sizing. In a large installation, it can become the basis for data storage, and at a speed comparable to mid-range, hi-end arrays if properly tuned. Not for nothing, they are used by many major cloud service providers.

Open source server virtualization is primarily a KVM hypervisor and management based on Virt-Mananger or O-Virt; Red Hat Enterprise Virtualization (RHEV). There are all necessary basic functions like HA, and vmotion, support for Linux and Windows environments, etc. We are ready to create and replicate solutions based on these products for distributed companies with many small branches with a dozen virtual locks. If you have just this - just keep in mind that zero investment in software can significantly save if such sites from 20 pieces or more. We can provide technical support for such installations.

By the way, if you can, as I said above, you can connect block storage to the hypervisor and get a compact system of two three servers at the output that will provide infrastructure software for a small office for 200-300 people. And, perhaps, there will fit some business systems. The plus is that neither the storage system with FC switches, nor the 10GE network will be needed right away.

About virtual workstations (VDI) - there is still little to say. There is Red Hat Enterprise Virtualization (RHEV), with very basic functions and I cannot name large-scale deployment cases so far. However, if you threaten a large system with proper financing, you can build a working industrial solution based on the same Openstack and RHEV.

Infrastructure Monitoring
- Zabbix
- Zenoss
- Nagios.

These solutions I have labeled as “must have” for any enterprise. Free products to create an infrastructure monitoring system. Out-of-box solutions support monitoring of operating system status, and application monitoring is also supported. For specialized applications, packages are finalized, actions to determine the correct operation. We have a couple of years already accumulated a very good experience in the implementation of such systems, for example, in the same DIT of Moscow and other government customers, for whom cost savings are important. These systems are quite competitors with proprietary solutions with zero investments in software. In addition, we often advise the use of such a class of systems in conjunction with BI systems for reporting and data analysis. At the exit a full analytics on the state of the entire infrastructure.

Database management
Postgresql is a good alternative to proprietary SQL database solutions. Mature full-featured relational database management system without significant financial investments. Currently, more than 50% of the projects for the development of new solutions in CROC are based on the use of this DBMS. By the way, we have certified it for use in state organizations.

For NoSQL
- Apache Cassandra
- MongoDB
- Apache Hive.
All this is also quite well-known systems.

Unstructured Data Processing
- Apache Hadoop. If it is necessary to process very large amounts of data, it is logical to use any Hadoop implementation as a large ETL machine, and there is a large number of free ETL engines with advanced visualization tools for processing and maintaining the logic of the “classic” ETL. In general, if you constantly smoke questions from BI, DWH, Big Data and other buzz words, you already know everything.
Immediately after - look at
- ETL Talend DI
- Cloudera Hadoop
- Pentaho DI
- CloverET.

Massively parallel DBMS
- PostgreSQL XL
- Cloudera Impala
- Infobright; InfiniDB.
These DBMSs allow building large analytical systems on the basis of complexes from a variety of inexpensive servers. Requests to them are executed in parallel, on almost all servers at once.

We successfully use all these DB management technologies in our projects. For example, in BI.
And BI is:
- Pentaho BA
- SpagoBI
- JasperSoft BI.

Pentaho has advanced visualization tools. SpagoBI - a tool for creating AdHoc-panels "on the fly" and for high-load systems, JasperSoft is most suitable for building fixed reporting. Any company that needs data analysis tools is interesting. We implemented similar systems in large banks and industrial enterprises. It should be noted that in a number of indicators important for the same NGOs, such open source solutions often surpass much more expensive proprietary solutions in function.

Web servers
- Apache HTTP Server
- nginx.

You know them for sure. Thanks to the history of nginx, by the way, it has become much easier to explain to customers that open source is not some kind of addict under bridges. Our developers use these web servers in many projects.

Application servers
- Apache Tomcat
- WildFly
- GlassFish.

You probably know them too. There is nothing much to tell, too, all three solutions are well known.

Finally, about directory services and the management of workstations and servers . Such solutions also exist. Both Red hat, SUSE, and just open source. For example, the 389 server and OpenLDAP directories. Everything you need to authenticate and maintain a registry of accounts in them. But there is one drawback - there is no group policies, logon scripts, and all that many have become accustomed to.

But these issues are solved in a heterogeneous or Linux environment just differently. Namely - management products, such as Puppet, Chef, Foreman. And here the bonus is that their functionality is much more than just GP or logon scripts. There are software installation and scripts, inventory, image casting and state monitoring, and there are a lot of other things for which there is enough imagination and specific tasks. And again, you can use them for free, or you can with support and advanced functions (you really have to pay for them, though).

Myths and Legends of Open Source

- Open Source - Linux + free office
No, as you can see above, this is the foundation of the IT infrastructure, and the DBMS, and the concrete “household” butt. There are many options for open source-analogs for various areas of software, and there are even areas where open source, by virtue of its flexibility, wins. Usually they save on the office first, and then they suddenly stop. Nevertheless, there is an opportunity to save much more on infrastructure-level software.

- Oh, the office. Is it true that ODF is a certified format?
Yes, it is certified with GOST R ISO / IEC 26300 - the default format for all types of documents. And the transition to its use will allow the migration to be less painful and not to depend on the software provider in the future. State-owned companies understand this, for example, the benefit of backward compatibility with the same DOCX remains.

- There are no system solutions in the open source
This myth is related to the fact that, as a rule, the Linux way is not a monstrous environment that solves all the tasks of the average user at once, but small utilities for specific tasks written by specialists to solve them (and not selling software). That is, open source software has indeed passed relatively recently to the systems approach, but already now there are quite a few solutions of this class. Why - in response to the next myth.

- No one supports open source software.
This is the sore point for a big business. And this is the most common myth. The fact is that yes, some specific open solutions may freeze in 2010, and since then they will not even be updated. But big serious projects like the ones described above are either actively supported by the community, or they are acquiring a company that, for quite reasonable money on a commercial basis, makes 24x7 support for large businesses. Or for the same earthly money leads product refinement. So if you need a new function, there is a high probability that it will be implemented quickly. Sometimes - even very quickly.

- Here's a proprietary software is good because when I have a problem - it will take
Yes, the image requires that someone dealt with any problem from the vendor. It is good that such behavior is enshrined in the license agreement. In the open source, the same problem is usually handled by an implementer, unless there is an organization specializing in supporting a particular system. For example, we support all software implemented by customers either directly or through specialists of the “second line” from such companies.

- Opensource products are a cheaper replacement for expensive solutions
Previously, it was so. Now, many products are compared in functionality with proprietary counterparts. It should be noted that large companies themselves (MS, IBM, HP, Odnoklassniki) often commit to open source, and many large systems are built on open source. This is a mutually beneficial exchange - as a rule, a corporation commits its code into an open source with the condition that you can include the finished result in your systems, and receive detailed testing plus a lot of good code on top.

- I still need a paper license.
I can say right away - during the implementation of a single incident with the requirement of paper licenses for open source was not. On the other hand, if the reviewers came for the purpose of not checking, but “filling up,” I am sure there would be many problems with proprietary software.

- There are problems with updates ...
On the OS, I can say for sure - issues have long been well resolved by regular means. On the butt side - yes, it does not always work without an intermediate agent to correctly distribute updates to 200-300 workstations, but it is also solved, and, in general, it is not particularly difficult.

pros

So, again, in the mind of the customer, open source is usually inextricably linked with an unshaven man in a sweater with deer somewhere in the garage. At first, no one takes such options seriously, but a little later, the economy forces one to look into the issue. It may be that the implementation is cheaper, and the cost of maintaining a service is much less. And then it turns out that if suddenly you have some kind of problem - you can wait for half a year for an update or patch (which, by the way, may not work), but reset the description in the community or get to the bottom. It turns out that your developer can accurately track what, where and how exactly it happens and put the patch on its own, even if no one in the world ever encounters this. They can tell you exactly where and how to dig.

Of course, if the difficulty is that your improvements may not be included in the next release, and with the fact that there may be a fork - but, in general, all this is solved by the community is often much simpler than it seems. On large products, of course - they do not include all the "children's" diseases of open-source disease.

The open-source community is less sending small users, because many are perfectionists, and bugs are not tracked in order of importance to the client. The organizational structure is such that you can safely send one counterparty with support and take another. The community itself is very professional friendly.

Summary

I had my first experience with open source software at the Institute, 8 years ago. I twisted mail and a number of network services like DNS, DHCP, postfix on * nix. Then there was ASP. At the workstation, he used Ubuntu and did not know any problems. At CROC, we started working with open infrastructure software about 3 years ago, when solutions began to appear, seriously competing with enterprise systems. Therefore, I can say for sure that open source solutions are not only for SMB, there are also those that work fine in large companies. And "thanks to" the economic situation and, in general, relations with Western companies, the topic of open source software is developing very well.

Well, by the way, the payback period for such solutions usually ranges from 1 year to 3 years, while the implementation itself is about six months on average. An infrastructure audit is usually required; development of a new ecosystem and training of employees for internal technical support at least first-line. I can show specific calculations for your situation by mail to albelyaev@croc.ru. Or come to our workshop on open-source and vendor replacement. My colleagues and I will tell you in detail about various solutions and examples of their implementations, and even try to show something interesting in the demo zone. Registration is already open .

It is very entertaining to catch a rake on how Linux behaves on several thousand machines: a lot of interesting things appear on such a scale. And it’s very nice that some people from the community who don’t owe us anything are looking for bugs with us at night ... It seems out of sports interest. Or because it was their code. Or just because they can. And it is magical.

And to the word about the bearded men in sweaters with deer. The most frequent question to me after going to KROK - is it true that you need to wear a tie every day? So, yes, I wore it. The entire trial period.