📜 ⬆️ ⬇️

CTOcast # 5: “You don't need a thousand people for a good discussion.”

Introducing the fifth issue of a podcast about technology, processes, infrastructure, and people in IT companies. Today, CTOcast is hosted by Gene Barmash (technical director of Merchantry, co-founder of the CTO School mitap) and Evgeny Babkin (head of the Merchantry development team).

Listen to the podcast

About our interlocutors:
')


Gene Barmash studied at Cooper Union College (1995--1999), New York. In the period from 1999 to 2009, he worked at Trilogy Software, Symantec, Infusion Development and Alfresco, going from a software engineer to a technical director. In 2009, he founded EnergyScoreCards (a service for comparing the energy performance of buildings). Today is the technical director at Merchantry (SaaS e-commerce platform).

Since 2006, he has been actively involved in mentoring programs and accelerators for technology start-ups. He is a co-founder and organizer of the CTO School (New York), which is a community of technical leaders with more than 1,600 members.



Evgeny Babkin graduated from Novosibirsk State University (2004), where he has worked as an infrastructure engineer since 2000. Since 2005 - in the company Ikstens. Today is the head of the development team that is working on the SaaS e-commerce platform for Merchantry.

Text version podcast


Alexander Astapenko: Jean, please tell us a little about yourself and your career.

Gene Barmash: I started my career with Trilogy in Austin (TX). Then there was the height of the first wave startups, the year 1999, and we all cooked in this sauce. Then for several years I was engaged in consulting, training, did all sorts of interesting and not very interesting things.

At some point I decided that I’m thinking and dreaming about these startups, so it’s time to do something concrete. Alfresco — an open-source content management application — was my first full-fledged start-up. I was there somewhere 70th man. After that — EnergyScoreCards, where I became the founder, and my friend was an expert on the energy use of residential buildings.

For the last year and a half I have been leading technologies at Merchantry. Since, apart from the head of our product, I am the only technical person in the team in New York, my role is to understand that the business wants us to present it to our technical teams as efficiently as possible, sometimes to build some technical strategy . And, on the other hand, understand what the technical team needs. The second part of my work is our data center and everything that happens there.

Alexander Astapenko: Marrying, how did you get into Merchantry? What did you do before?

Yevgeny Babkin: I started working and studying at the very beginning of the 2000s. If someone remembers, at that time there was a crisis and everything was quite difficult. More precisely, in 1998 I started. He worked as an administrator and I had some of the infrastructure of the university, and there were several small companies where I did everything I needed.

Time passed, and at some point I got a Java-developer in the company Ikstens, where we made large electronic stores. Then the enterprise-platform Intershop was very popular. And on this platform for clients from the USA and Europe we did, as it was then called, the Internet portals, where the e-commerce of that time was carried out.

Then our customers wanted to integrate with Amazon in order to sell, including there. From this point on, both the company and the product have changed. We began to do integration with Amazon, then with eBay. We have our own platform, which was engaged in these integrations for clients of all sizes. Events began to evolve even more interestingly: we began to advise those who wanted to make their own little Amazon, their own marketplace. At some point, we started making our product, a platform that allowed any more or less large retailer to make Amazon. Now we are going a little further and want to occupy another niche in the market, so the product for this business is also changing.

I started as a developer, then I was an analyst. And now we are working with Gene on the current version of the platform. The same product has been written for many years, and when the market segment changes, the business changes, it turns out quite bizarre layering, which is then very interesting to deal with.

About Merchantry



Alexander Astapenko: What is the Merchantry company doing? I want to understand what is business before we talk about the technical part. Who are the customers? When did the company appear?

Gene Barmash: The company was founded about ten years ago. For the first six to seven years, we mainly dealt with consulting. Worked for large customers and made them client applications. About three or four years ago, we took money from investors and said that we would build a platform.

Initially, we built a platform for marketplaces to make online markets. What is needed for this? First, to be able to take products from suppliers and integrate them through, for example, API. This makes it possible to import products into our platform and then export them to customers who have a marketplace. Buyers go to the marketplace and see products from different suppliers. When they buy these products, after some time a package arrives to them, which may consist of, say, 5 different things from 5 different suppliers. In the first part we have to model, synchronize and describe products. Or, more specifically, to give our suppliers the tools to do this. Then synchronize it to the marketplace, and when orders arrive, correctly deliver these orders electronically to suppliers. This is a marketplaces.

What we are doing right now is called product information management. It looks, frankly, very, very similar. We have a platform where suppliers can add their products, and we allow these products to be exported to their store. When orders arrive, we give these orders to suppliers, we synchronize information (which package and when sent, which number is in this package) and allow us to cancel the order.

Pavel Pavlov: What forces managed to build such a product? How is the team structured? Where is?

Evgeny Babkin: Now 13 people are working on the main platform. In fact, they represent one team. This is not to say that this is an overgrown team, but something similar. At various times, the number of developers, testers, and everyone who worked on the platform and its integration reached, probably, 100 people. Now what we do and how we work is a bit like a scrum. Pure scrum is not possible with us because the owner of the product is far enough away from us and speaks English, and the development team in English does not speak very well. We have sprints, and thanks to Jin, continuous integration has developed quite well.

Pavel Pavlov: Can you tell how many people are in the development team? Technical support?

Gene Barmash: About 30 people work in Novosibirsk. As Zhenya said, 13 — on the main development. Also 5--6 people work at professional services, consultants who support their product for integration. Our support consists of 6--7 people. They talk to customers, look at systems, and so on. And 3--4 people are engaged in DevOps: they monitor our data-center, do transformations and migrations.

About technology



Pavel Pavlov: If you hooked this topic, can you tell us about your data center? Where is the platform at all?

Gene Barmash: Now we have two main data centers. There is a Connectria company in St. Louis (Missouri) with a good level of service — our main production is there. We also have a secondary data center in OVH, in Montreal, Canada. We use it mainly for internal systems. Plus there are several production systems. And we also have a geographical failover at Amazon Web Services (AWS) in case a meteorite falls into Missouri.

Pavel Pavlov: Why not use Amazon as the main platform?

Gene Barmash: Several reasons. Partially, because it happened. We have a fairly large load, and we believe that it is more economical to use our hardware. Every time we looked at Amazon prices, they were significantly higher, especially for the hardware we needed. In principle, we could switch to Amazon, but a rather large transformation would be required, because we have already done everything, we have iron. On the other hand, it would have cost significantly more than we pay now, but we already pay a lot.

Alexander Astapenko: Jin, you communicate with quite a lot of technical leaders at CTO School. If we talk about infrastructure approaches, is it now a general trend for services with heavy workloads to use data centers that are not connected with well-known AWS cloud providers? Or are people still leaning towards cloud services?

Gene Barmash: Basically, they are inclined to cloud services. I also want to note that our load is fairly stable. It may be large, but there is no such thing that next month the load will be 5 times more and you need 5 times more servers. So we can do very specific capacity planning and the cloud does not give us the advantages like, for example, other companies. Beginning startups, of course, do on the cloud. Use Amazon or something like Heroku, which I also used for almost a year. It is terribly convenient, especially if you do things like Ruby on Rails. You do not need to think about deployment at all.

We have a rather complicated infrastructure: we have our own FTP, email server, LDAP for users, and so on. Therefore, to have all the client for us works well. And initially, complete control over the iron was very convenient.

Yevgeny Babkin: We approach the infrastructure in about the same way as it is done in the clouds. When we expand something, all this is automated. Our test environments and our production environments are currently quite similar. That is, from this point of view, it does not matter whether the cloud is or not the cloud, your own hosting. Everywhere you have to watch the load, you have to look at the SLA.

Gene Barmash: Yes, Zhenya is completely right, we have a lot of automation. Continuous integration provides automation, Puppet, which allows you to deploy clusters, we have a Docker, which can raise services very quickly. Now we finish the migration to Docker and then everything will become unified. Developers will be able to deploy full-fledged wizards that are 100% the same as in production.

Pavel Pavlov: The issue of transition to continuous integration and Docker implementation. How to implement? How did this solve problems?

Gene Barmash: We approached the Docker from the continuous deployment first. What is a continuous deployment? This is when each commit diploma production. We do not need every commit, so we do it once a day.

The migration process to the continuous deployment with Docker is contacted, because Docker helped isolate our environment in a test environment and ensure 100% isolation between different tests. This means that we can test different branches completely separately from each other. Each developer sits on his branch and fully scrolls all the tests.

Pavel Pavlov: What was your transition to the Docker related to?

Gene Barmash: Honestly, we didn’t have a huge problem that we solved when we switched to Docker. We saw that there is a new technology and it seems to suit us. What is a docker? This is Linux in a container that provides lightweight isolation between different servers. That gives VMware, but with less cost.

When we were fairly well acquainted with the technology, we began to think, and how do we go to continuous deployment. And Docker became one of the tools that we used to implement it.

Pavel Pavlov: Should such a transition be coordinated with the business? Or are people who invest in a project not look at such things?

Gene Barmash: It was not necessary to ask for money at Docker, since this is an open-source project. The technology works and makes our DevOps efficient. Why not apply it?

When we switched to continuous deployment, then it was already necessary to explain to the business, because everything changed. To this it was necessary to divert the main development team and invest a lot in the tests. We already had quite good tests, but we connected the developers to the writing of functional tests. And I had to tell the business that for the next two weeks, the team would be doing something else. And, accordingly, when you say these things to them, they ask: “What will they do?” And you explain that the process is changing, that the goal is to improve the quality, to make the release faster. Zhenya, tell me how it looked. You were much closer.

Yevgeny Babkin: Yes. I would like to reveal this topic from the other side. We wrote this platform for a long time. We had to deal with enterprise-clients who have special quality requirements, and therefore we wanted to achieve platform stability. We knew what to do, and when we made the next shift, we began to write automatic tests. That is, we had all testing duplicated by automation. It is clear that it is impossible to automate everything, but much has been automated.

By and large, we now have two levels of tests. These are the classic GUnit tests and integration tests of Selenium, which also run through GUnit and work through external interfaces, applications, through browser simulation and so on.

We wrote a massive test suite. Naturally, there were problems with how long they run. And they were executed for a long time. I had to live with it somehow. We had TeamCity as an integration server, several machines that were servers and on which we deployed the whole thing. The trouble was that the master was constantly dismantled. The transition to Docker allowed us to solve this problem, because now we are very quickly and easily deploying the application from scratch for any branch, filling it with data and running tests. The paradigm of work has changed completely. Now we really have always a stable master. That is, dockers dramatically changed the level of quality that we have. Now you don’t have to make any compromises when the release is needed the day after tomorrow and when not very scary bugs are not bugs at all, because there’s no place to go. This is all in the past.

Alexander Astapenko: Jin, do I understand correctly that you don’t have manual testing at all? So all tests — functional testing?

Jin Barmash: No, this is wrong. We have three testers: one fully works on automation and two are combined manually tested and help with automation.

Before the transition to continuous delivery, our testers did tests only manually, but there was one person who chased the team and tried to automate the tests. He always lagged behind, since there were 9 developers, and he was alone. It turned out that many features did not have automation. Due to the fact that he always lagged behind us, a regression appeared only a few weeks after the initial release. It was unpleasant. Therefore, we connected the developers to writing tests.

Alexander Astapenko: Clear. Zhenya, will you add anything else?

Evgeny Babkin: We do not have manual regression. That's right to say. We have manual testing, because it is very expensive to automate everything. How, in fact, to automate, and then run these tests every time. But the regression is completely automatic. Each new feature is tested manually.

It so happened that testers began to write a set of Selenium tests and they wrote them as they could. Therefore, at some point we began to connect developers to writing tests, which is not easy to do. Because the developers do not really want, it’s hard to work with the code that the previous two years were written by testers. They want to rewrite almost all the code, but no one gives them. If I did this the second time, I would probably have the developers write tests on cases from the very beginning that testers give. Then it would be cheaper and a little faster.

About CTO School



Alexander Astapenko: I already briefly mentioned at the beginning of our podcast CTO School Meetup. I heard from several people about this mitap, but I still don't know much about it. What is CTO School Meetup?

Gene Barmash: This is a simple mitap in New York. Although there is one now in Australia, Melbourne. We just get together once a month, sometimes twice a month, and we try to talk about different topics, about how to become the best technical leader.

From my point of view, this requires competence in three areas: technology, you need to understand the various development processes, so that people work effectively with each other, and you also need to be a leader to motivate people, conduct interviews, and so on. When we are going, I try to balance different topics in all these areas.

I hope that when people come to us, they learn something. We have a large meeting, usually once a month, when someone prepares a presentation on a specific topic: how to hire people, how to optimize the process. Approximately 100--150 people come. There is also a second type of assembly. This is about 20 people. We sit around the table and share our experience in a more intimate setting. We also have a list where people ask questions, while others answer them.

I have been doing mitap for 4 years and it is very interesting for me, it helps me gain experience from other people. And I think that people are also interested to know what problems others have.

Alexander Astapenko: Gene, in fact, a brilliant idea. If I had the opportunity to attract the best technical minds of New York to solve their problems, it would be cool. What are the technical leaders on East Coast talking about now? I understand that in addition to the thematic presentations, there are some conversations on the sidelines. What topics are hot now?

Gene Barmash: What is hot is difficult to say, because each company has its own problems. Topics that have evolved over the past year and a half are DevOps, engineering culture and the philosophy of engineering teams. Here I recently interviewed CTO Etsy (marketplace for handicrafts). This company is very respected in New York. They probably have a turnover of a couple of billions at the moment. The company employs about 215 engineers. And they have such a philosophy: we write PHP and such very boring technologies, on the one hand, and on the other, we don’t have to sit and think about which new technologies to apply. New technologies are hard to maintain, because they are new and they are changing. By the way, the founder of PHP works for them, which also helps.

On the other hand, there are Gill who make microservices. Microservices, Docker and event-driven architecture are becoming very popular. According to the technology, we can say that Node.js has risen to its feet now in New York and Go is beginning to be popular.

Alexander Astapenko: A few words at the end of our podcast?

Gene Barmash: First, thank you very much for inviting us. From my own experience of organizing mitaps, I would say that if there is a topic that is really interesting to people, then it can be expanded anywhere. I know that Zhenya has a group of technical leaders in Novosibirsk, in which he participates. The fact that there are a lot of people in New York, it certainly helps. But if there is interest, it can be raised everywhere. You don't need 1,000 people for a good discussion. You need 10 that are smart and who understand the same thing as you. Even less than 10 is also normal.

Yevgeny Babkin: In fact, I am very pleased that such Russian-language meetings as CTOcast appear. It is very often heard that Russian developers are among the best developers in the world. It is clear that we all understand English very well and can do English speaking, but it is still very pleasant to listen to them in Russian.

Source: https://habr.com/ru/post/252517/


All Articles