Darudar - a cloud as a gift from Microsoft

Daudar is a project known in the vastness of Runet, which has become a link between people who want to donate something, but who do not know who, and those who may want it. On the site you can find anything you want - from household and not-so things to the most real cats in bags . About how the service works, where more than 3 million gifts have already been presented, says one of the founders of the service, Anton brutto Karakulov.

Daudar is a real high-load project. Hosted in the cloud and using the full Linux stack, it can withstand ~ 2.5 thousand gifts and ~ 1.5 thousand thanks, ~ 20 thousand comments, ~ 40 thousand notifications and ~ 4.5 thousand files per day. Details under the cut.


A modern person no longer needs to be convinced of the advantages of cloud platforms that are used as the main infrastructure for creating a web service, a mobile application, and finally - an “ordinary” web site. To fix the facts, it would not be superfluous to repeat the main advantages of cloud hosting services:
')
- A wide range of operating systems, PL, frameworks, tools, databases and devices.
- Extensive integration options: Linux containers in Docker; on a par with
creating JavaScript, Python, .NET, PHP, Java and Node.JS applications
- And, of course, the backend for iOS, Android and Windows applications

The lack of need to develop new technologies or solutions / tools, coupled with real-time billing and scalable “on-demand” iron infrastructure is the cornerstone on which all modern clouds stand, and Microsoft Azure is, of course, no exception.

Perhaps another important advantage of cloud platforms in general, and Microsoft Azure in particular, is the ability to transfer an existing project and get all the benefits already mentioned. Actually, now “to transport” your own back-end from the data center, in which you may rent individual racks, to the cloud is not difficult: hybrid databases, storage, secure connection methods - all this is out of the box. And in the case of Microsoft, there is also the Azure Stack , which allows you to bring a model of development and rollout (deployment) of Azure applications in any data center or data center.

Finally, paying only for what you use, and not for the “fact of availability”, has become such an integral value for developers and owners of virtual products that no one is even ready to consider the reverse model seriously.

About why?

We, like many others, received a grant from Microsoft for the startup support program. In our opinion, this is a fat plus, especially for beginner projects or projects that would only like to try the cloud. Although, of course, we were lucky to learn about the benefits of Azure in one of the most difficult moments (our previous site-hoster abruptly ceased operations, and we “saved” our own infrastructure). Our previous hosters have done their time nicely, and I can’t say anything bad about them, but due to the constant lack of a full-fledged system administrator (for which we have no money for salaries) - the cloud has noticeably simplified the moments related to the administration of the infrastructure.

Pro architecture and technology

Daroudar has a classic nix-stack: nginx, php, mysql, memcached, sphinx.

Meets all nginx, then the request goes to the php-backend, which already interacts with all the other internal and external services of the project.

Currently, Azure uses Cloud Services and Ubuntu virtual machines within the same network. For each internal service, an image is created that can be used for emergency
restoration / replacement and it is planned to use also for the possibility of further horizontal scaling of the project services with the help of convenient tools of the Azure Available Sets and Load Balancing set.

The plans are moving the media storage to Azure Blob Storage, as well as using Azure Queue (we are testing).



About stereotypes

“Azure is MS, and MS is Windows” - this bundle also worked in my head. And that was the first thing I checked when I got access to the Azure control panel.

Everything turned out wrong. Azure was loyal to Nix-systems, and we use them.

We have never used Win-servers at Daroudara and we do not use them at all. I know the virtualization technology only by hearsay — mostly on Windows I use VMware for this purpose. In Azure, there is, as far as I know, the ability to deploy a wine server and connect to it remotely, such as a “remote desktop”, but I personally did not work with this and did not even try (note: you can connect using a visual interface).

If we talk separately about the virtualization of nix-machines, then I can’t say anything about the technology itself - I haven’t dug so deeply, but there are a number of “specific” platform features.

Minuses:

- When resizing (reconfiguring) a VM (virtual machine), it fully reboots.
- The VM system disk is extremely slow and must therefore be used carefully or not at all.
- There is a large additional disk, which is present in every machine and at a good speed, but it is considered “temporary” and can be reset when reset, therefore it is not recommended to use it for sensitive data (by the way, now Azure has started to write about it everywhere and not only in the documentation!).
- Not obvious from the first time to understand the structure of Cloud Services.

Pros:

- You can create, and then use your own images (convenient for clustering and scaling).
- Any disks can be added independently, it is possible even to make an image with the necessary number of disks which will then "rise" and "cling" independently.
- Availability of console utility for managing infrastructure (convenient for automating work on / off, adding / deleting).
- Ability to create a VPN.
- Availability of tools for managing traffic / load (Azure Available Sets, Load Balancing Sets, DNS Traffic Manager).

Perhaps, certain things have long been with Amazon or Google, but I have stopped following them for a while, and at Azure we all use it and stay with good impressions.

Pro cost

Azure costs us now at ~ 140 thousand rubles a month. This is enough for us at 90%, since for peak loads it is still not enough.

I think the final cost for the full service of the current state of affairs will be about 160-180 thousand. About the load on the “Daroudar” I will answer as follows:

~ 2.5 thousand gifts and ~ 1.5 thousand thanks a day
~ 20 thousand comments per day
~ 40 thousand notifications per day
~ 4.5 thousand files (images) per day

As for the choice for companies, first we need to determine our goals and scope.

It’s not worth trying because it’s fashionable and “everything is in the clouds now”. For us (Durudar), the key to choosing to go to the cloud or remain on collocation was the absence of a sysadmin in the state, and therefore, in fact, for us the cost of cloud hosting is the cost of infrastructure + basic administrator.

You can talk a lot about whether it is worth it or not at all, for example, like this:

- It is good for projects that just happened and that only work out their model;
- Good for projects that expect to live only on investments (meaning that in the absence of investments the project closes);
- Good for projects that can earn independently and they are interested in growth and development.

About plans

All that is interesting to us and can be planned for implementation, one way or another, leads to vendor lock, demanding, respectively, a careful implementation of the project.

1. For example, queues are very necessary. We are thinking about implementation.
2. It is necessary to resolve the issue with the media storage. As a result, you will need to go to a full cloud solution (and this is 100% vendor lock).
3. A very interesting thing, which we also planned and which was mentioned above - traffic control / load (Azure Available Sets, Load Balancing Sets, DNS Traffic Manager).

About best practices

The problem is in a very slow system disk for Linux instances, so either we don’t use it at all or only for some kind of tasks where it is not critical.

Although the local disk looks attractive in terms of size and speed, and even despite the warning that data may be lost from it, it can save data when restarting instances (!), But storing critical data there is still an error. For example, if the instance is vertically scaled, it will reset to any value.

It happens that a VM can hang so that nothing will help it except a reboot. And sometimes even she may not help. Therefore, the allocation of each internal service in a separate instance or group on the order facilitates the management of infrastructure. In this connection, even thoughts became “cloudy”: it’s better to reboot than to fix. If the reboot is not fixed, it is better to re-create.

Summarizing

After the pain and suffering in the development of the platform (there were such moments) now I feel calm for the future of the project infrastructure. Appeared predictability, which allows you to fully engage in the planning and development of the project. And what else is very valuable: the opportunity to manage the infrastructure has appeared, without having deep knowledge in the field of administration, and to connect additional people to the infrastructure maintenance.