Codesign.io - architecture that works
Hi guys. My name is Pavel, I am the co-founder of the Codesign service and am responsible for its technical component. Codesign is a web service for creating and discussing feedback when working on websites, design layouts and presentations. If the designers still write edits to you on the layout with solid text in the letter, then it is time to tie and switch to Codesign.
We launched the service on July 22, 2015, and since then we scored 3,500 registered users, experienced a launch on ProductHunt , when 10,000 people called at us in 36 hours, got our first paid users and discuss obtaining funding from European ( I live in Italy ) investors . In this article I will tell how we built the architecture of the service and organized the development process.
When developing Codesign, we aimed for modularity, considering that in this case it would be easier to make edits and replace one unit with another. The first serious decision was a complete separation of the API and web applications, so that they work on different servers. Since the team had experience with Python / Django , it was decided to build an API based on the Django Rest Framework . We originally planned to write a web application on AngularJS , but the hired front-end developer convinced us to implement on React . The server side of the web application is implemented using the Express framework on Node.js.
')
Due to the fact that we wanted to spend as little time as possible on setting up the servers and being able to scale quickly (if we expect a large traffic spike, for example, after publishing on a popular resource), we decided to deploy our Heroku code. Thus, on Heroku we have 2 servers for the API (production / development) and 2 servers for the web application (production / development). In Heroku, at a free rate, the server falls asleep every hour and must be inactive for more than 6 hours a day. I had several servers raised a few months before the appearance of new tariffs, and there is no such restriction on them. The web application and API work on free servers, which we periodically scale when we expect an influx of traffic. I think the main advantage of Heroku is the ease of deployment and its GIT framework, which allows you to automatically deploy code from Github. This process is described in more detail below.
Heroku has a large number of different plug-ins , among which there are also database management systems (DBMS). Before the release on July 22, we had a prototype using MySQL and we decided to continue working on this DBMS and connected ClearDB . The free tariff has a database limit of 5MB, so we first switched to the Punch tariff, and then Drift for $ 50 per month. Now ClearDB is the most expensive component of technology services (only Intercom is more expensive, but I’m not talking about it in this article) for which we pay. I thought that $ 50 a month is expensive and I thought that at Amazon RDS I would get better performance for the same money. After a series of experiments, I realized that this was not the case, because I received the same speed of query execution only on RDS instances that cost more than $ 50, so we stayed with ClearDB.
The basis for Codesign is graphic files that users upload for further discussion. Here we didn’t have a long discussion and stopped at Amazon S3 , where, with our traffic and the number of stored images, we pay $ 3-4 per month. On a slow Internet, downloading images above 4MB took more than 30 seconds and a timeout error from Heroku occurred . To solve this problem, we began to generate on the server only a special link that allows you to upload images directly to Amazon S3 from the browser . So the problem was solved.
When one user made a change to the board (a collection of discussion pictures) on Codesign, other members of this board receive notifications. To implement these notifications, we use two services: Heroku task scheduler to initiate sending notifications every hour, and Mandrill to send emails as such.
Django itself allows you to send reports about 500 errors encountered on the server. It's comfortable. But even more convenient was the service Opbeat . He collects information on errors, shows statistics of their occurrence, assigns each group of errors to the person who commits the code in which this error occurred. In addition, Opbeat monitors server requests and shows performance bottlenecks. There are many similar services , but Opbeat has bribed its simplicity and gratuity for up to 3 users. Once we had a problem with the fact that endpoint to get the list of boards in the folder took 4 seconds. We could not understand what exactly the problem. Opbeat showed that we are very suboptimal making requests to verify user rights. We optimized the code and the execution of endpoint accelerated 20 times.
Our development is currently fully built on GIT:
Next, we plan to implement the following tools:
This article is an example of how various third-party services can be integrated to simplify work on your own product. We didn’t come immediately to the architecture that you see in this article, but gradually - by trial and error. We do not pretend and do not agitate that this architecture is ideal. We are open to discussion and will be very happy to hear feedback on the architecture that we use.
If you like Codesign (or do not like it and you want to change it), come to work with us - we have 4 deleted vacancies .
We launched the service on July 22, 2015, and since then we scored 3,500 registered users, experienced a launch on ProductHunt , when 10,000 people called at us in 36 hours, got our first paid users and discuss obtaining funding from European ( I live in Italy ) investors . In this article I will tell how we built the architecture of the service and organized the development process.
Frameworks
When developing Codesign, we aimed for modularity, considering that in this case it would be easier to make edits and replace one unit with another. The first serious decision was a complete separation of the API and web applications, so that they work on different servers. Since the team had experience with Python / Django , it was decided to build an API based on the Django Rest Framework . We originally planned to write a web application on AngularJS , but the hired front-end developer convinced us to implement on React . The server side of the web application is implemented using the Express framework on Node.js.
')
Deployment
Due to the fact that we wanted to spend as little time as possible on setting up the servers and being able to scale quickly (if we expect a large traffic spike, for example, after publishing on a popular resource), we decided to deploy our Heroku code. Thus, on Heroku we have 2 servers for the API (production / development) and 2 servers for the web application (production / development). In Heroku, at a free rate, the server falls asleep every hour and must be inactive for more than 6 hours a day. I had several servers raised a few months before the appearance of new tariffs, and there is no such restriction on them. The web application and API work on free servers, which we periodically scale when we expect an influx of traffic. I think the main advantage of Heroku is the ease of deployment and its GIT framework, which allows you to automatically deploy code from Github. This process is described in more detail below.
Database
Heroku has a large number of different plug-ins , among which there are also database management systems (DBMS). Before the release on July 22, we had a prototype using MySQL and we decided to continue working on this DBMS and connected ClearDB . The free tariff has a database limit of 5MB, so we first switched to the Punch tariff, and then Drift for $ 50 per month. Now ClearDB is the most expensive component of technology services (only Intercom is more expensive, but I’m not talking about it in this article) for which we pay. I thought that $ 50 a month is expensive and I thought that at Amazon RDS I would get better performance for the same money. After a series of experiments, I realized that this was not the case, because I received the same speed of query execution only on RDS instances that cost more than $ 50, so we stayed with ClearDB.
Storage
The basis for Codesign is graphic files that users upload for further discussion. Here we didn’t have a long discussion and stopped at Amazon S3 , where, with our traffic and the number of stored images, we pay $ 3-4 per month. On a slow Internet, downloading images above 4MB took more than 30 seconds and a timeout error from Heroku occurred . To solve this problem, we began to generate on the server only a special link that allows you to upload images directly to Amazon S3 from the browser . So the problem was solved.
Notifications
When one user made a change to the board (a collection of discussion pictures) on Codesign, other members of this board receive notifications. To implement these notifications, we use two services: Heroku task scheduler to initiate sending notifications every hour, and Mandrill to send emails as such.
Monitoring
Django itself allows you to send reports about 500 errors encountered on the server. It's comfortable. But even more convenient was the service Opbeat . He collects information on errors, shows statistics of their occurrence, assigns each group of errors to the person who commits the code in which this error occurred. In addition, Opbeat monitors server requests and shows performance bottlenecks. There are many similar services , but Opbeat has bribed its simplicity and gratuity for up to 3 users. Once we had a problem with the fact that endpoint to get the list of boards in the folder took 4 seconds. We could not understand what exactly the problem. Opbeat showed that we are very suboptimal making requests to verify user rights. We optimized the code and the execution of endpoint accelerated 20 times.
Development
Our development is currently fully built on GIT:
- First, the developer makes changes and loads the code in the development branch in Github . In the master branch in Github directly push the code is impossible, it is prohibited in the settings of the repository.
- At this point, the webhook is triggered and the repository code is redeployed on the Heroku development server.
- Making sure that the code works adequately, we create a pull request to the master branch, upon which the code is automatically sent to the production Heroku server. If something went wrong and an error was found, then Heroku makes it very easy to roll back to previous versions.
Future
Next, we plan to implement the following tools:
- Heroku review apps - for parallel testing of different features separately following git flow (now we have only one development server)
- Travis - to run unit tests on every push on Github
- CodeClimate - for automatic code analysis (will lead to high quality code)
- Amazon RDS - when we do not have enough tariffs that ClearDB offers, it will be necessary to deploy more powerful RDS instances
- Firebase - for real-time updates of some blocks
- Stripe - for receiving payments (now used by Gumroad)
Conclusion
This article is an example of how various third-party services can be integrated to simplify work on your own product. We didn’t come immediately to the architecture that you see in this article, but gradually - by trial and error. We do not pretend and do not agitate that this architecture is ideal. We are open to discussion and will be very happy to hear feedback on the architecture that we use.
If you like Codesign (or do not like it and you want to change it), come to work with us - we have 4 deleted vacancies .
Source: https://habr.com/ru/post/271461/
All Articles