An inside view or infrastructure of the Likeastore project
In a relatively short time, we managed to try and change a lot of decisions that directly or indirectly affect the product. Today, I would like to review the infrastructure around the Likeastore project. This may be interesting to many developers thinking about their launch.
I will go from hardware to software, from low infrastructure levels to higher ones. For all the services that we use by subscription, I will specify prices. There will be a small commentary for each item, but in the future, each of them can be opened more deeply in subsequent posts. Go…
From the very beginning, we started to focus on the clouds, because it is convenient. The main requirement was for it to be PaaS ( AppFog , Nodejitsu , Heroku , Azure ), and not IaaS ( EC2 , DigitalOcean , Rackspace ). Maintaining your own infrastructure is too “great fun” for a startup.
PaaS is a combination of virtual hardware and infrastructure deployment and scaling applications. It was the convenience of deployment that was the main thing, since it was banal to think about big growth.
')
Our first experience was with AppFog . At the time of the first experiments, they looked really attractive - 8 instances, 256 MB of memory for each. The most important thing is “deployment by one team” through the client and cut it all for 20 bucks. At first, everything was okay, but later negative moments began to open: a rather long deployment (30-40 sec), an uncomfortable cheap board, but most unpleasant, due to a problem in their load balancing system, AppFog constantly “lost” server cookies, which made it impossible use the app.
We had to leave for Nodejitsu . 3 instances for $ 33, expensive compared to AppFog, but on the Nodejitsu the application worked, deployments went quickly, the deshborb was convenient and support was at a height. I had to revise our architecture a bit to fit in 3 instances, but it was even for the better. We had no particular complaints about Nodejitsu, except that their servers are located in the us-east-1 zone and for Europe this created noticeable ping latency.
Happiness ended when we entered the public beta phase (summer 2013). We clearly understood that we were working with personal data of the user, so the use of
It was a "bummer" ... It was necessary to rent the server and do everything yourself, i.e. essentially go back to IaaS solutions.
They decided to rent a server from DigitalOcean . We chose by price, but time has shown that DigitalOcean is a reliable and convenient partner.
By the time of our transition to DigitalOcean, they launched support in the AMS1 zone (Amsterdam) and we were one of the first droplets in this zone. Can not say. that everything was perfectly smooth from the very beginning, but after moving the car to the European zone, the responsiveness of the application changed radically (to the best party). This was especially noticeable from mobile devices.
We started with the cheapest option: 512MB, 1CPU, 20GB SSD for just 5 bucks. This was more than enough to start. In this configuration, we have prospered up to the thousandth user. Node.js is not the most "wasteful" platform in terms of resources and 512MB gives the opportunity for normal operation, but up to some load. Over time, the most resource-intensive component of the collector (collection of likes) began to consume a lot of memory and trite to fall.
We took a bigger droplet - 1GB, 1CPU, 30GB SDD for 10 bucks, and left the last one for our “stitch” environment. Thus, the entire hosting cost us 15 bucks, with SSL support. Almost 10 times more profitable than any PaaS solution. Just recently, we switched to droplets with 2GB RAM / 2 CPU, and this configuration will last us for a long time.
"But DigitalOcean, this is not PaaS," you will object, and you will be right. But we were able to bring it closer to that. How?
After “having tasted” to overflow PaaS, it is extremely difficult to refuse them. I could hardly imagine deployment via ftp or ssh, “manually” starting and stopping the node, configuring nginx, etc. But by chance, I came across a project called dokku , which greatly influenced the development of Likeastore .
Dokku allows you to turn a Ubuntu server into a mini-Heroku server, with the ability to deploy any (Node.js, Ruby, PHP, Static etc.) applications via git push. Dokku is very easy to install and requires minimal configuration.
Dokku uses git as source transport to machine A (a development machine or an intergrate server) on machine B (a production machine) and a docker to isolate executable processes and the launch environment. dokku allows you to deploy the application with one command:
Each time an application is deployed from a “clean” environment, the required version will be installed on this environment, all necessary dependencies will be installed, and so on. dokku uses nginx , acting as a proxy for running containers, if necessary, the SSL connection is configured by copying keys to a specific folder on the server.
We deploy about 5-10 times a day for a test and 1-2 times into the working environment, starting using dokku from version 0.1, we did not experience any significant difficulties, but the project brought a lot of benefit. At the same time, both he himself and the technologies underlying it are extremely interesting!
At the very beginning of the project, we used CouchDB , but very quickly switched to MongoDB .
MongoDB is convenient in all respects, this is a schemaless solution, ideal for fast-developing projects, easily upgraded by the developer’s machine. Our current data volumes allow you to work without sharding, with one instance of the database.
The prospects for the MongoDB configuration in production pleased me even less than the manual deployment. We started with MongoLab , but later switched to MongoHQ , primarily because we received good recommendations from colleagues about their support service, and the second is that MongoHQ allows you to deploy a server in Europe.
For the prototype, we took a test connection, 512MB (which then seemed to be an unattainable volume), everything worked with a bang. But I remember how happy I was with the transition to
Running an application into production without proper logging is suicide. Access to the logs should be quick, simple and desirable with the possibility of notification if errors or warnings appear in the logs. Those. again, logs should be placed in the cloud.
The solution we have focused on is Logentries . With our current volume of traffic log, we remain on a free account. The "cardiograms" of all our applications are always in front of our eyes, so we quickly see when something starts to go wrong.
It is also important to monitor the status of your servers. At first, it was ssh to the server and manual monitoring, now, on the advice of colleagues, they were visiting New Relic . I really liked the installation process, good documentation and support. New Relic's premium feature is application monitoring, they have recently got Node.js support.
If the logs are the number one priority for the application, then the metrics are the priority number for the product.
Naturally, we have Google Analytics . Maybe they throw a stone at me, but GA for me is nothing more than tracking general traffic and referrals. Step to the right, step to the left - problems begin that are difficult to notice, since the data in the reports appear only the next day. Difficult to tune and takes a long time to figure it out.
We use our solution Seismo (very early and simple) and more recently, we used MixPanel to build funnels. MixPanel was very pleased with its integration and scared with its price. So we will develop our solution, depending on current needs.
We use 2 solutions for communication with users: for Mandrill transactional letters, and for support, invitation and retention letters of Intercom . Both products are very cool, the pricing policy of Mandrill (12,000 letters for free) allows us to use it at no cost, but with Intercom we just recently had a trial run, and their price tag is quite high - the base price is $ 49 + $ 11 for every 250 active users. We connected Intercom at the end of December and our invoice was $ 82. This is still the most expensive service (convenient), but I would consider alternatives.
So, let's summarize the startup costs for infrastructure: $ 40 (DigitalOcean) + $ 20 (MongoHQ) + $ 82 (Intercom) = $ 152. This is quite a lot, as for a start-up company, we will still look for ways to optimize. However, this is not such a big amount of money that should keep you from trying to start your service. Moreover, we came to these expenses in about half a year of the product’s existence, and the initial ones were at the level of $ 10–20 per month.
Now the most favorable environment and a lot of things being done to help developers, away from doubts - start today!
I will go from hardware to software, from low infrastructure levels to higher ones. For all the services that we use by subscription, I will specify prices. There will be a small commentary for each item, but in the future, each of them can be opened more deeply in subsequent posts. Go…
A little story ..
From the very beginning, we started to focus on the clouds, because it is convenient. The main requirement was for it to be PaaS ( AppFog , Nodejitsu , Heroku , Azure ), and not IaaS ( EC2 , DigitalOcean , Rackspace ). Maintaining your own infrastructure is too “great fun” for a startup.
PaaS is a combination of virtual hardware and infrastructure deployment and scaling applications. It was the convenience of deployment that was the main thing, since it was banal to think about big growth.
')
Our first experience was with AppFog . At the time of the first experiments, they looked really attractive - 8 instances, 256 MB of memory for each. The most important thing is “deployment by one team” through the client and cut it all for 20 bucks. At first, everything was okay, but later negative moments began to open: a rather long deployment (30-40 sec), an uncomfortable cheap board, but most unpleasant, due to a problem in their load balancing system, AppFog constantly “lost” server cookies, which made it impossible use the app.
We had to leave for Nodejitsu . 3 instances for $ 33, expensive compared to AppFog, but on the Nodejitsu the application worked, deployments went quickly, the deshborb was convenient and support was at a height. I had to revise our architecture a bit to fit in 3 instances, but it was even for the better. We had no particular complaints about Nodejitsu, except that their servers are located in the us-east-1 zone and for Europe this created noticeable ping latency.
Happiness ended when we entered the public beta phase (summer 2013). We clearly understood that we were working with personal data of the user, so the use of
http was not acceptable. Nodejitsu does not allow you to use SSL for Personal Plan with a custom domain. Going to the Business Plan cost $ 120 a month.It was a "bummer" ... It was necessary to rent the server and do everything yourself, i.e. essentially go back to IaaS solutions.
Hosting and iron
They decided to rent a server from DigitalOcean . We chose by price, but time has shown that DigitalOcean is a reliable and convenient partner.
By the time of our transition to DigitalOcean, they launched support in the AMS1 zone (Amsterdam) and we were one of the first droplets in this zone. Can not say. that everything was perfectly smooth from the very beginning, but after moving the car to the European zone, the responsiveness of the application changed radically (to the best party). This was especially noticeable from mobile devices.
We started with the cheapest option: 512MB, 1CPU, 20GB SSD for just 5 bucks. This was more than enough to start. In this configuration, we have prospered up to the thousandth user. Node.js is not the most "wasteful" platform in terms of resources and 512MB gives the opportunity for normal operation, but up to some load. Over time, the most resource-intensive component of the collector (collection of likes) began to consume a lot of memory and trite to fall.
We took a bigger droplet - 1GB, 1CPU, 30GB SDD for 10 bucks, and left the last one for our “stitch” environment. Thus, the entire hosting cost us 15 bucks, with SSL support. Almost 10 times more profitable than any PaaS solution. Just recently, we switched to droplets with 2GB RAM / 2 CPU, and this configuration will last us for a long time.
"But DigitalOcean, this is not PaaS," you will object, and you will be right. But we were able to bring it closer to that. How?
After “having tasted” to overflow PaaS, it is extremely difficult to refuse them. I could hardly imagine deployment via ftp or ssh, “manually” starting and stopping the node, configuring nginx, etc. But by chance, I came across a project called dokku , which greatly influenced the development of Likeastore .
Dokku allows you to turn a Ubuntu server into a mini-Heroku server, with the ability to deploy any (Node.js, Ruby, PHP, Static etc.) applications via git push. Dokku is very easy to install and requires minimal configuration.
Deployment process
Dokku uses git as source transport to machine A (a development machine or an intergrate server) on machine B (a production machine) and a docker to isolate executable processes and the launch environment. dokku allows you to deploy the application with one command:
git push production master
Each time an application is deployed from a “clean” environment, the required version will be installed on this environment, all necessary dependencies will be installed, and so on. dokku uses nginx , acting as a proxy for running containers, if necessary, the SSL connection is configured by copying keys to a specific folder on the server.
We deploy about 5-10 times a day for a test and 1-2 times into the working environment, starting using dokku from version 0.1, we did not experience any significant difficulties, but the project brought a lot of benefit. At the same time, both he himself and the technologies underlying it are extremely interesting!
Data storage
At the very beginning of the project, we used CouchDB , but very quickly switched to MongoDB .
MongoDB is convenient in all respects, this is a schemaless solution, ideal for fast-developing projects, easily upgraded by the developer’s machine. Our current data volumes allow you to work without sharding, with one instance of the database.
The prospects for the MongoDB configuration in production pleased me even less than the manual deployment. We started with MongoLab , but later switched to MongoHQ , primarily because we received good recommendations from colleagues about their support service, and the second is that MongoHQ allows you to deploy a server in Europe.
For the prototype, we took a test connection, 512MB (which then seemed to be an unattainable volume), everything worked with a bang. But I remember how happy I was with the transition to
AWS Small:EU , which MongoHQ gave us a loan after we had some problem and we turned in support - the application just “fly”, the response time was close to what we we have on the developer's environment. In addition, they have a pretty good admin panel, tracking slow queries and recommended indexes.Logging and monitoring
Running an application into production without proper logging is suicide. Access to the logs should be quick, simple and desirable with the possibility of notification if errors or warnings appear in the logs. Those. again, logs should be placed in the cloud.
The solution we have focused on is Logentries . With our current volume of traffic log, we remain on a free account. The "cardiograms" of all our applications are always in front of our eyes, so we quickly see when something starts to go wrong.
It is also important to monitor the status of your servers. At first, it was ssh to the server and manual monitoring, now, on the advice of colleagues, they were visiting New Relic . I really liked the installation process, good documentation and support. New Relic's premium feature is application monitoring, they have recently got Node.js support.
Metrics and User Relations
If the logs are the number one priority for the application, then the metrics are the priority number for the product.
Naturally, we have Google Analytics . Maybe they throw a stone at me, but GA for me is nothing more than tracking general traffic and referrals. Step to the right, step to the left - problems begin that are difficult to notice, since the data in the reports appear only the next day. Difficult to tune and takes a long time to figure it out.
We use our solution Seismo (very early and simple) and more recently, we used MixPanel to build funnels. MixPanel was very pleased with its integration and scared with its price. So we will develop our solution, depending on current needs.
We use 2 solutions for communication with users: for Mandrill transactional letters, and for support, invitation and retention letters of Intercom . Both products are very cool, the pricing policy of Mandrill (12,000 letters for free) allows us to use it at no cost, but with Intercom we just recently had a trial run, and their price tag is quite high - the base price is $ 49 + $ 11 for every 250 active users. We connected Intercom at the end of December and our invoice was $ 82. This is still the most expensive service (convenient), but I would consider alternatives.
Total infrastructure costs
So, let's summarize the startup costs for infrastructure: $ 40 (DigitalOcean) + $ 20 (MongoHQ) + $ 82 (Intercom) = $ 152. This is quite a lot, as for a start-up company, we will still look for ways to optimize. However, this is not such a big amount of money that should keep you from trying to start your service. Moreover, we came to these expenses in about half a year of the product’s existence, and the initial ones were at the level of $ 10–20 per month.
Now the most favorable environment and a lot of things being done to help developers, away from doubts - start today!
Source: https://habr.com/ru/post/210000/
All Articles