Apiway - a new way of client-server data transport

“You want to live - not so raskoryachishsya” © Features of national hunting

How often do you have to “co-ordinate” the work of the client and server parts of the application, organize the exchange of messages, its timeliness, provide access to data and their constant relevance on the client?

Often? Then maybe you will be interested in the idea and implementation described in this article.
It's about Javascript, Ruby and Websockets.

Prelude

At the beginning of this year I published an article about one of my inventions, into the code of which I poured all my thoughts and vision (at that time) on the development of both the frontend and backend. I did not call anyone to use it, which, in fact, did not happen, I just asked one question - is this another bike?
')
After some time, I joined Luciding as a frontend-developer (a young startup promoting the idea of lucid dreams). Since then, and to this day, my work is closely connected with React, and thanks to him, my views on the development have changed dramatically. Now I can answer myself - my past invention is just another bicycle. Now I see no point in implementing the client part of the application in accordance with the MVC pattern, the client part is a mirror reflecting the reality occurring on the server, the reality that the user is allowed to see, and all that is needed is data and a tool for visualizing them.

React handles the visualization task perfectly, but the task of delivering data to the client and keeping it up to date is implemented by each programmer in its own way and quite often the following approach is used:

Data access
1. Sending a request to the server to receive actual data.
2. Creating event handlers for their changes, which bring the data (after the next manipulation) to the current state.

Data manipulation
1. Send a request to the server to change / delete data.
2. Sending notifications (events) about the manipulations to the client.

Conception

And everything seems to be good, but if the first points are taken for granted, then the second is a routine that would be nice to automate. That is what gave rise to the idea of a new bicycle in my head - to create a tool that allows you to completely refuse to participate in the delivery / synchronization of data between the client and the server. The developer should only clearly define what and to whom, and not think about how and when.

Childbirth

After the amount of time I wrote ruby gem for the implementation of the server part and javascript npm package for the client. I called my brainchild Apiway .

Server part

Built on top of the well-known Sinatra framework in the ruby community, it has generators for quickly creating the structure of a new application and its components.

Components:

Client - Apiway::Client instance, personal for each connection;
The model is the usual ActiveRecord model;
The controller is designed to determine the actions that manipulate the data;
Resource - responsible for accessing data and providing it to the client.

Details on the components

Customer

Performs parsing of incoming messages, controls the launch of controllers and synchronization of resources. The current client, always available in the controller and resource code, can be obtained by the client method. An array of all connected clients can be obtained by calling Apiway::Client.all (in addition, this method accepts a block of code applied to each client).
Each client has its own personal repository, the data in which is stored throughout the entire connection.

 #     client[:user_id] = 1 #     client[:user_id] # > 1

Custom client event handlers
When generating a new application, the file app/base/client.rb , which allows you to configure client connection / disconnection event handling, as well as receive a new message. Each handler is called in the context of the client whose event is being processed.

 module Apiway class Client on_connected do #       #      end on_message do |message| #       end on_disconnected do #       #       end end end

Model

By default, the new model is inherited from ActiveRecord::Base when it is generated, but this is not necessary, the main thing is that it be extended by the Apiway::Model module. This module adds to it the only sync method, the call of which starts the process of synchronization of resources depending on this model (automatically called on ActiveRecord models after saving / deleting the model).

 class Test < ActiveRecord::Base include Apiway::Model end

Controller

 class UsersController < ApplicationController include Apiway::Controller #     Rails # Before- before_action :method_name before_action :method_name, only: :action_name before_action :method_name, only: [ :action_name, :action_name ] before_action :method_name, except: :action_name before_action :method_name, except: [ :action_name, :action_name ] # After- after_action :method_name after_action :method_name, only: :action_name after_action :method_name, only: [ :action_name, :action_name ] after_action :method_name, except: :action_name after_action :method_name, except: [ :action_name, :action_name ] #   action :auth do #        # Api.query("Users.auth", {name: "Bob", pass: "querty"}) # .then(function( id ){ console.log("User id: ", id) }) # .catch(function( e ){ console.log("Error: ", e) }) begin #  params      user = User.find_by! name: params[ :name ], pass: params[ :pass ] rescue Exception => e #         ,   error #         #  ,    "Error: auth_error" error :auth_error #  error     , , ,  #    before-,      #       else #     client     #  ( Apiway::Client),      # id  client[:user_id] = user.id #          #     ,      id #    "User id: 1" end end end

Resource

 #         # var userMessages = new Resource("UserMessages", {limit: 30}); # userMessages.onChange(function( data ){ console.log("New data", data) }); # userMessages.onError(function( e ){ console.log("Error", e) }); class UserMessagesResource < ApplicationResource include Apiway::Resource #   depend_on Message, User #   ,       Message  User, #      ,    #    #     access do #           #  client ,       , #        :user_id error :auth_error unless client[:user_id] #   error       #  "error"   "auth_error",    #  "Error: auth_error"  ,   "", #  "" ,    ,   #      "change"    #     "New data: [{mgs},{mgs},{mgs}...]" end #    #      ,      #    data do #  params      Message.find_by(user_id: client[:user_id]).limit(params[:limit]).map do |msg| { text: msg.text, user: msg.user.name } end end end

Client part

Provides the components necessary to interact with the server. Each component is inherited from the EventEmitter class, I carried it into a separate package .

Components:

Api - an object that provides methods for connecting to the server and sending requests to its controllers;
Resource - a class whose instances open server resources and always contain relevant data;
Store is an empty object, designed to transfer “common” data between different parts of the client part of the application.

Details on the components

Developments

Each client-side object has the following methods:

 .on(event, callback[, context]) //   callback   event; .one(event, callback[, context]) //   .on(),      ; .off() //     ; .off(event) //     event; .off(event, callback) //   callback  event; .off(event, callback, context) //   callback  event, //    context;

Api

Generated events

ready - generated after setting up the connection and successfully fulfilling the promise beforeReadyPromise ;
unready — generated after the connection is established and the “failure” to fulfill the promise beforeReadyPromise ;
error - generated in case of errors in connection with the server;
disconnect - generated when disconnecting from the server.

Methods

 Api.connect(address[, options ]) //      //  : // aliveDelay -  ( )   "ping" , , //   ,       Api.query( "Messages.new", params ) //     new  MessagesController'a //   params    Api.disconnect() //     Api.beforeReadyPromise( callback ) //  ,   (Promise) //         //        // "ready" -     "unready" -   //    , ,   //     Api.onReady(callback, context) //  Api.on("ready", callback, context) Api.oneReady(callback, context) //  Api.one("ready", callback, context) Api.offReady(callback, context) //  Api.off("ready", callback, context) Api.onUnready(callback, context) //  Api.on("unready", callback, context) Api.oneUnready(callback, context) //  Api.one("unready", callback, context) Api.offUnready(callback, context) //  Api.off("unready", callback, context)

Resource

Generated events

change - generated when updating resource data;
error - generated when an error occurs (call the error method on the server).

Methods

 var resource = new Resource("Messages", {limit: 10}) //   MessagesResource   {limit: 10} resource.name //     resource.data //     resource.get("limit") //    limit,   : 10 resource.set({limit: 20, order: "ask"}) //    limit     order //      resource.unset("order") //   order //      resource.onChange(callback, context) //  resource.on("change", callback, context) resource.oneChange(callback, context) //  resource.one("change", callback, context) resource.offChange(callback, context) //  resource.off("change", callback, context) resource.onError(callback, context) //  resource.on("error", callback, context) resource.oneError(callback, context) //  resource.one("error", callback, context) resource.offError(callback, context) //  resource.off("error", callback, context)

Taming

Consider creating a simplest console chat using Apiway .

Server part

First, install Apiway and generate an application framework:

 $ gem install apiway #  gem'a $ apiway new Chat #    $ cd Chat #

Through migrations, we will create the Messages table in the database:

 $ bundle exec rake db:create_migration NAME=create_messages

Add the code that forms the table:

 # db/mirgations/20140409121731_create_messages.rb class CreateMessages < ActiveRecord::Migration def change create_table :messages do |t| t.text :text t.timestamps null: true end end end

And run the migrations:

 $ bundle exec rake db:migrate

Now create a model:

 $ apiway generate model Message

 # app/models/message.rb class Message < ActiveRecord::Base include Apiway::Model #         #    ,        validates :text, presence: { message: "blank" }, length: { in: 1..300, message: "length" } end

Then the resource:

 $ apiway generate resource Messages

 # app/resources/messages.rb class MessagesResource < ApplicationResource include Apiway::Resource # ,       Message depend_on Message #  ,         data do Message.limit( params[ :limit ] ).order( created_at: :desc ).reverse.map do |message| { id: message.id, text: message.text } end # params - ,    ,     #    params = {limit: 10}      10   # [{id: 10, text: "Hello 10"}, {id: 9, text: "Hello 9"}, {id: 8, text: "Hello 8"}, ...] end end

And finally - the controller:

 $ apiway generate controller Messages

 # app/controllers/messages.rb class MessagesController < ApplicationController include Apiway::Controller #  ,    action :new do begin # params -     current_user.messages.create! text: params[ :text ] rescue ActiveRecord::RecordInvalid => e #       error e.record.errors.full_messages else true #   ,    end end end

On this, the server part is completed, we will start the server with the command:

 $ apiway server

Client part

I think that the overwhelming majority know what npm, gulp, grunt, browserify, etc. is, so I will not write about the intricacies of the build, but I’ll just describe the main points.

Install apiway:

 npm install apiway --save

Actually, the client part itself (as simple as 3 kopecks and fits in a few lines):

 // source/app.js import { Api, Resource } from "apiway"; //    var Chat = { run: function(){ //      Messages   { limit: 10 } var messagesResource = new Resource( "Messages", { limit: 10 } ); //          render messagesResource.onChange( this.render ); //   send   window window.send = this.send; }, render: function( messages ){ //      console.clear(); //      messages.forEach( function( item ){ console.log( item.text ) }); }, send: function( text ){ //        new  Messages Api.query( "Messages.new", { text: text } ) //        console.warn .catch( function( errors ){ console.warn( errors.join( ", " ) ) }); } }; Api //          .connect( "ws://localhost:3000", { aliveDelay: 5000 } ) //      ,    .oneReady( function( e ){ Chat.run() }); // ""       // ,     onReady(),    //

That's all. Now, to see the result, open the browser, then the console and see the last ten chat messages, write send("Hello world") and again see the last ten messages, or rather our new and nine previous ones. We try to send an empty message - we see errors.

Client side using React

First we create the Message component, it will be responsible for displaying the message:

 // ./components/Message.jsx import React from "react"; class Message extends React.Component { render(){ return ( <div> <b>{ this.props.data.user }</b> <p>{ this.props.data.text }</p> </div> ); } } export default Message;

Now the Chat component itself:

 // ./components/Chat.jsx import React from "react"; import Message from "./Message.jsx"; import { Api, Resource } from "apiway"; //  ,     //        let errorsMsg = { "Text blank": "    ", "Text length": "     1  300 " }; class Chat extends React.Component { constructor( props ){ //     super( props ); this.state = { messages: [], errors: [] }; } componentDidMount(){ //     ,  //  Messages   { limit: 30 } this.MessagesResource = new Resource( "Messages", { limit: 30 } ); //       this.MessagesResource.onChange(( messages )=>{ this.setState({ messages }) }); //  callback        } componentWillUnmount(){ //      this.MessagesResource.destroy(); } onKeyUp( e ){ //     Enter'    ,   if( e.keyCode !== 13) return; //     new  Messages,   {text: " "} Api.query( "Messages.new", { text: e.target.value } ) //   (Promise) .then( ()=>{ this.setState({ errors: [] }) }, ( errors )=>{ this.setState({ errors }) }); //    -  ,    -   } render(){ return ( <div> //   <div> { this.state.messages.map( ( message )=>{ return <Message data={ message } key={ message.id } />; } ) } </div> //   <input type="text" onKeyUp={ ( e )=> this.onKeyUp( e ) } /> //   <div> { this.state.errors.map( function( key ){ return errorsMsg[ key ]; }).join( ", " ) } </div> </div> ); } } export default Chat;

And finally, we will edit the app.js file .

 import React from "react"; import Chat from "./components/Chat.jsx"; Api //    .connect( `ws://localhost:3000`, { aliveDelay: 5000 } ) .oneReady( function( e ){ // ,      ,    React.render( <Chat />, document.getElementById( "app" ) ); });

But what is going on then?

As you can see, we have never directly resorted to synchronizing data with the client - Apiway did this work for us. How did and in what magic - consider more:

When sending a request to an action of the server controller, Apiway remembers all the models that have undergone changes during the execution of the action; in our case, a new Message model was created;
after the action is completed, synchronization of all resources dependent on the modified models is started, in our case these are all instances of the MessagesResource resource;
the resource makes a new sample of models, generates the actual data and converts it into a JSON string (it also remembers the JSON string that is currently on the client), then the two strings are compared and the patch is calculated (the minimum instruction for converting old data into new - this allows you to significantly save traffic);
then the weighting of the complete data and the patch occurs, after which what is less in volume is sent to the client;
the client resource, having received a new instruction, applies it to its data and generates a " change " event.

Subtleties of character

What happens if the connection to the server disappears?
Everything will be fine! The client will automatically reconnect and, as soon as the connection is established, all resources synchronize their data to the current state, and during the absence of a “connection”, the data loaded earlier will be available.

What format should be the data generated by the resource?
Anyone! Anything that can be converted to JSON, be it a hash, an array, a string, a number, or any other object that has the as_json method as_json .

Can I change the parameters of an open resource?
Can! A typical example: load message history when scrolling. It is implemented by an elementary change of the resource parameter messageResource.set({limit: 50}) - the resource will load the last 50 messages and generate the " change " event.

Heels of achilles

Many database queries
Yes this is true! Skillful use of caching of frequent requests can help in this case.

The patch calculation procedure is relatively slow.
Yes, the larger the amount of data, the more time is needed for the calculation. You can of course cut out this functionality altogether, but then the amount of “chased” traffic will increase significantly. Here it is necessary to think about a compromise.

I consider the data and drawbacks to be significant, but at the moment Apiway is an example of implementation and is only suitable for projects with a low load.

Give it a touch!

Chat is a slightly more complex example with simple authentication.
Chat source on Github
Apiway server-side sources on Github
Apiway client-side sources on Github
I apologize in advance for the “clumsiness” of English (at the level of intuition) in the readme repositories.

Thank!

I thank all the readers who have found time to read this article. I will be glad to your criticism, any help and just support.
Sincerely, Denis.

Source: https://habr.com/ru/post/268031/

All Articles

Apiway - a new way of client-server data transport

Prelude

Conception

Childbirth

Server part

Customer

Model

Controller

Resource

Client part

Developments

Api

Resource

Taming

Server part

Client part

But what is going on then?

Subtleties of character

Heels of achilles

Give it a touch!

Thank!

More articles: