Back to the Upper Paleolithic technologies, from everyone's favorite REST, STATEless, CRUD, CGI, FastСGI and MVC

"Only with the death of dogma does science begin."

// Galileo Galilei



“I began to envy the slaves. They know everything in advance. They have strong convictions. ”

// x / f Mark Zakharov "Kill the Dragon" based on the play by Evgeny Schwartz



For a couple of years and a day has not passed, so I did not hear (or did not read) from people starting new projects, a phrase like "Take the server engine for the REST API and MVC, and drove off . " At first I thought that these words have one source, maybe a book that was delivered to all the stores or somewhere in the top of the search engines is an article that zombies developers. If you find out from them what they mean by REST and MVC, then you can be damaged by the mind. Well, with MVC everything is already clear, I wrote about this a long time ago , nothing has changed, it only worsened, it’s worth typing in Google Images "mvc" and we will see the scary , arrows in any direction. Well, about the REST answer the following: well, we need to call server methods from the browser GUI and mobile application, for example: setUserCity (userId, cityId) or calculateMatrix (data) or startVideoConverter (options, source, destination) and then we will encounter heavy load and the REST architecture will solve everything. Then I ask questions, from which the eyes are rounded already by those who have recently burned with righteous faith, rushed into battle and knew exactly what was happening in this world. Now we can proceed to the consideration of terminological catastrophe , in the epicenter of which we are staying.

MVC Questions

First question: draw me a “what is MVC?” Diagram. What I get:



For MVP, such understandings and schemes are also sufficient, do not even hesitate.

')

The second question: how can this pattern intended for user interfaces be applied on the server?

I can not list here all the possible answers. But the correct one is very rare and it’s like this: on the MVC server, it can only be if our user interface is fully implemented on the server, i.e. There is no browser code, the interface is generated on the server, and the browser simply shows it.



The third question: how did you manage to combine MVC with REST?

I will not hold back, I will publish common options:

• Well, of course, MVC is implemented using REST
• Well, of course, our REST is implemented using MVC

I can not comment on this, it is absolutely clear that we are dealing with a terminological catastrophe , but more on that later.

REST Questions

The first question is: well, my dear, how can these setUserCity, calculateMatrix and startVideoConverter be implemented on REST?

My answer: the whole point of REST is that it operates on files, each of which has its own unique URL and only HTTP methods can be performed on this file: GET, PUT, POST, DELETE. Therefore, all those of your own setUserCity will be kindly replaced with HTTP POST / user / id and you need to send there a completely serialized user. Or another option, you need for each user parameter to have a separate URL. Obviously, CRUD methods (create, read, update, delete), called via HTTP methods, are enough to manipulate files or objects in the database. For example, to enter forms and forms, save them to tables with further deletion and editing. But any modern application can not be reduced to such primitive challenges. And developers, most often, understand REST as its opposite - RPC (remote procedure call), when we can do as many methods as possible, not just CRUD, and when a single URL can give completely different answers, depending on which user He was called, with what parameters and in what sequence. And this is not a REST at all, for example, Habr issues at http://habrahabr.ru/tracker/ a different list for each of us and this is everywhere, on Google, Facebook, Github, etc. So, remove the misleading word REST, do the API and that's it.



The second question: okay, what is REST for you?

The range of opinions here is not as wide as for MVC, but the terminological catastrophe is this: as I showed above, REST is understood not as REST, but to make everyone go to science and study dogmatic literature on REST, look for frameworks with REST support, and as a result, RPC and REST hybrid come out. And worst of all, that all this happens unconsciously .



The third question: can you explain how exactly REST will save you from large loads?

My answer: he is STATEless , but what does this mean and why should we give up the state on the server - we will discuss below.

What is the meaning of STATEless

When scaling, modern web servers spawn separate processes for each request. If these processes are STATEless, then there is no difference, the system works on one server or hundreds of servers. Each process receives request data via the CGI protocol and must allocate memory, deploy its data structures in it, if necessary, create connections to the database, read files or make calls to external modules. Then the process will perform its logic and all this economy is killed. Such procedures delay the execution of HTTP requests, so there is FastGI, which provides for several always running processes and saves time on their generation, on initial allocation of memory, on its release and on completion of processes. Another FastGI allows you to reuse connections to the database and other descriptors. But the memory structures needed to process each HTTP request still do not live long, they die and are created, and at the same time, queries are executed in the database, accessing files and external modules. But even in this case, it is necessary to adhere to the STATEless principle, because it is not known which process the next HTTP request falls into. And different requests may relate to different sites applications, and even in the same application may relate to different sessions or provide for the deployment in memory of very different data structures.

It is happiness for a programmer from the Upper Paleolithic

At the time when my programming worldview was developing (1997-2002), the three-tier application architecture (window client, application server, DBMS) was the mainstream. And clients interacted with application servers by RPC. Everything was STATEful, but then there were no such tasks that would require scaling the application server on many machines. And so, for many years, from 2002 to 2012, I endured REST and STATEless, dreaming in my heart about full-fledged STATEful programming and RPC for high-load systems. Now, when most languages ​​and platforms have their Event-Loop solutions, like Node.js, Python's Twisted and the Ruby Event Machine, I feel like a completely happy person and I don’t know anymore why STATEless is needed and where phrases like “ Take the server engine for REST API and MVC, and drove . "



Since the opportunity to make highly loaded application servers with a state appeared, the processes began to live again for a long time, all data structures can be left in memory, connections to the database, caches, timers remain, a complex model can be deployed in memory and it will sit there for a month . And the client part can interact with this model through the API. And now a bit of arithmetic, imagine that the user state is 32Kb, and we have 16GB of memory, which is enough for 524288 (more than half a million) users.



The only thing left to solve is how to send all requests related to the same session to the same process. That is, you need to "glue" the session to the server and to the process (the process can be on one of many servers). To do this, we have hardware and software balancers that implement adhesion by IP (ip-sticky) and cookie (cookie-sticky). Of course, each process must mark with a cookie with its identifier all requests that do not have such an identifier. If there is an identifier, then sticking works, and if not, then round robin or a more complex balancing algorithm.

And with MVC, we have not finished

MVC taught everyone to separate the model, the view and the controller, and thoughtlessly where and where not. But a common understanding of each component has not been developed. Someone leaves only the parameters in the model, and the methods brings to the controller, and some models are able to save and restore themselves from the database, others are violent opponents of this approach and allocate another data access layer to access the database. In any case, you need to do something with a terminological catastrophe .



I propose to use the word “model” as it is customary to use it in science and technology, namely, this is a simplified representation of a real system and / or process . From this it follows that the model may contain parameters and methods. But these methods must fit the definition. For example, CRUD methods cannot be part of the model of a real-world object, because no real object can create or delete itself. CRUD is not part of the model, but part of the data storage layer API, just like drawing methods are part of the graphics layer API. The model should not know anything about its storage, transmission protocols over the network, features of rendering in the browser, etc. A good example of a method for a model is: SteppingMotor.setSpeed ​​(x) when the model is a physical device driver or is used to model a physical device. Another good example is mathematical models, for example Equation.calculateRoots (). And the third example is information models, for example: Patient.assignBed (bedId).



The words “view” and “controller” are avoided whenever possible, only because in the minds of individual developers they may have completely unpredictable and strange meanings. Instead, it is better to use more specific concepts such as “template”, “control” (or “user interface control”), “request router”, etc.

Application Tips for Upper Paleolithic Technology

From event-oriented solutions, I personally use Node.js, but I think that they can be universal:

1. Separate client and server (browser application and network API) according to the principles of RPC and STATEful with gluing sessions to processes.

2. Use RAM, do not climb into the database constantly. STATEful is a great opportunity to write fast applications, and not even because Event Loop frameworks offer non-blocking I / O, but because of proper memory usage. Most I / O operations do not even need to be done during query processing, reading can be done proactive and parallel, and writing is lazy (lazy). Expand data into the memory of the application, build hashes, objects, arrays that will live a long and happy life in the STATEful process.

3. Interact between different processes via ZeroMQ (and other MQ), TCP, HTTP, IPC, and anything else. Thus, the data of different processes, depending on what this data is, can either be duplicated in memory (cached if it is common data) or be divided into “sticky” sessions or synchronized with each other through interprocess communication.

Conclusion

Rethink design patterns every time. do not take other people's decisions just because of their credibility, they may not be bad, but not for your case. It is possible and necessary to take decisions, but only with their full digestion and understanding.

Always check the terminology with your colleagues even before the start of the project and specify it up to the creation of the dictionary in the public domain. With the confusion of concepts that has accumulated in our time, it is impossible to be sure that you are talking about the same thing, even if you use the same words.



UPD: The comments below are the best proof of a terminological catastrophe . It turns out there is no common understanding, not only in what REST and MVC are , but even in what stateless and statefull are . And even more interesting, there is no common understanding of what a “state” is. Someone understands this as the state of the process (its memory), someone the status of all servers (then the state may be in another process), and someone the state of the system as a whole (then the stored files and data in the database can be considered as a state) . In such a situation, it is better to discard the old terms , spoiled and stained, the meaning of which no longer has any hope of agreeing to introduce new ones, but making already strict dictionary definitions . Otherwise, there is nothing to talk about at all, it is impossible to build a dialogue, speaking in words, but about completely different concepts.