Unit for node.js
GitHub and NPM libraries.
some unknown aggregate that is not related to node.js. But on Habré is considered a good form to attach a picture
Some time ago I wondered why, in node.js, working with relational databases, such as * SQL, and some noSQL type Mongo, is difficult, and made an alternative solution, tailored to the speed of the programmer’s work (compared to the classic solutions, work with the database) and the straightness and compactness of the API for the minimum threshold of entry. The first source of inspiration was the report "minimal surface API" , the second - the famous quote by Donald Kruta:
')
Dixler 1: The library described here is in the early beta stage. So far you should not use it for commercial or critical projects for your life.
Disclaimer 2: complex terms can be found in the text, and in the code examples there are many ES6 features. If something seems incomprehensible - please write in the comments, I will try to simplify the text and add comments to the code.
When I used sequelize - do not hope, other libraries are not better - I could not help wondering why working with it is so difficult. Not in terms of understanding how it works from the inside, no - I was digging into the library interface. Either the hands are not growing from there, or the developers are hardened DBAs, not like me.
Now I know what they were trying to put in the toolkit that they could. The output is the 15th standard, combining the previous 14. Indicative in this regard is the hellish juggling combine, which is able to make an extremely diverse list of databases - MySQL, SQlite3, Postgres, CouchDB, Mongo, Redis, Neo4j.
But for me for small projects - all sorts of telegram bots, server devs and SPAs - there was no need for a complex part of the functionality, which is under the hood of complex ORM-ok. The basic required functional is the saving and searching of records, sampling by conditions and relationships. I do not need premature optimizations : sampling parts of fields from the database, tricky query optimizations, stored functions. Sampling with respect to (get the object and all the links) can be made a transaction. Due to the loss in speed, we get the absence of a heap of additional entities of declarative syntax. Occam's razor in its purest form.
Lyrical digression: if you look at the history of the development of projects with tens and hundreds of thousands of users - after a certain time, developers run into speed. They change queries, database, languages, platforms, so that it works. If the project shoots out - he will have to replace parts, and the first one will work with the base under the knife - if not enough effort was spent initially “for the future”. At the same time, the heavy, complex ORM syntax complicates the replacement. The conclusion is obvious - if you evaluate the choice of ORM as an uncontested future technical debt, the correct choice may be a solution with less efficiency, but ensuring the speed of the developer and providing a minimal API, which simplifies the transition to another solution.
Not a DB, but a JS-centric ActiveRecord - however, in some places I have moved away from the classic pattern.
It is important to understand that since it is not DB-centric - the database was selected for solution requests, and not the decision was made for a specific database. Storage was chosen Neo4j. This solution has pros and cons, but for now there are more pros.
If you are not familiar with neo4j, this is a popular graph database with a much more comprehensible for the outsider than SQL, a language, a convenient web client and full-text out-of-box indexes (using lucene), and a slightly lower (linear) speed compared to Postgres / Mysql. All installation instructions are here: http://neo4j.com/download-thanks/?edition=community . On the mac it is installed via brew install neo4j
Let's start with a simple connection and entries:
The only thing that stands out from the clarity of the code - call User.register (). In JS, it is impossible to hang up a handler (and thank the developers of the language for this) to create a class, so you have to do it for the language.
The Record.register method does 3 things:
For abstract classes, this method does not need to be called.
In ES2015, static properties are inherited in the same way as properties of an entity — connecton is declared once, in the parent class, as shown. If you have one database, you can assign a connection to Record.connection, although this is incorrect from the development point of view.
Let's complicate the example. Imagine that we are doing an ACL, and we need a relationship:
If you do not look closely - do not immediately see that in fact this.permissions - many-to-many through the relationship. This kind of syntax allows you to build long chains of relationships for which full-fledged queries are available - search, delete, check for availability, everything, except for obvious reasons not working Relation # add.
Relation emulates the Set object built into ES6. The API is different, but immediately familiar and understandable. The difference is that methods return a Promise, which already returns data, and size () a method, not a property. Additionally, there were methods #intersect, which returns the intersection of the transferred array of elements with the related elements, and #where, which does the obvious, but about it below.
Methods with an identical API are available for this: the class method Record.where () and the instance method of the class Relation # where (). Available are offset, limit, order by, search by value, the contents of the array and entry into the array (yes, the typed array is one of the primitives in neo4j) and the substring. There are many possibilities for searching. They cover all the main tasks. Enumerating all the options is quite difficult, so it’s easier to look at the formal description in a typescript-like syntax:
The API described above already allows you to work. Only the question of atomicity remains, which is classically solved with the help of transactions.
In an aggregate, you can work with transactions in two ways - simple or understandable.
The clear way is to use head-on transactions. To do this, pass it with the last argument (among others). All standard database methods support this notation.
The connection object (which is available for both the class and the class instance) can be a connection, transaction, or sub-transaction. There is no difference for use in life, because all three entities provide the same interface with small internal differences. If you call connection.transaction (), the connection returns a transaction, the transaction returns a sub-transaction, the sub-transaction returns another sub-transaction.
The internal difference is the following: the commit and rollback methods for connection will send an error, for a transaction, it will work as expected, for a sub-transaction - commit will do nothing, and rollback will roll back the parent transaction.
This is done because some methods about generate a transaction for themselves and close at the end - for example, Record # save (). To ensure that such methods work correctly within a transaction, the infinite nesting of sub-transactions is implemented.
For the second method - simple - decorator is used:
It turns the code into something like this:
The decorator can be used both directly, as in the example above, and configuring. For the configuration so far, only one flag is available - force, which forcibly creates a transaction - if a transaction is not transferred, it will create it. You need to use this:
Note that now this.connection has become a transaction. When the function runs, the property will return to its previous state, but now it allows you to call other methods of the class, without worrying about committing the transaction. This magic works only within this (which is predictable).
Since transactions are processed in turn, i.e. until one is completed, another is not started, the object is not cloned, so consider: if you wrap the static method in this decorator, you can accidentally “fumble” the transaction. For class instances, this is not a problem because if you work correctly with JS, they are in their field of view, and from other execution threads (such as promises, async, etc.) you cannot access them at the same time. unavailability of the object.
The description of the API and the reasons why it was done this way and not otherwise, is complete.
Probably the only thing worth adding is that I already use it in small projects for myself and friends. I haven’t experienced such a pleasure when working with a database for a long time - I felt such a feeling of "transparency" and clarity of the working mechanisms only when working in Ruby / Rails, and even there I had to suffer from CLI in places.
The unit may lack some capabilities or speed, but if you want this - connect to the project. Now the aggregate is only 608 lines (in shock itself) of fairly well-organized code, and it is very easy to make edits, additions, updates, and do additional tests. I would like to see it uniquely usable in a big production, and if you like it too, connect to the development!
some unknown aggregate that is not related to node.js. But on Habré is considered a good form to attach a picture
Some time ago I wondered why, in node.js, working with relational databases, such as * SQL, and some noSQL type Mongo, is difficult, and made an alternative solution, tailored to the speed of the programmer’s work (compared to the classic solutions, work with the database) and the straightness and compactness of the API for the minimum threshold of entry. The first source of inspiration was the report "minimal surface API" , the second - the famous quote by Donald Kruta:
')
Programmers spend an abnormal amount of time worrying about the speed of non-critical parts of applications, and these attempts to improve efficiency seriously negatively affect the debugging and support of these applications. Premature optimization is the root of all evil .
Dixler 1: The library described here is in the early beta stage. So far you should not use it for commercial or critical projects for your life.
Disclaimer 2: complex terms can be found in the text, and in the code examples there are many ES6 features. If something seems incomprehensible - please write in the comments, I will try to simplify the text and add comments to the code.
When I used sequelize - do not hope, other libraries are not better - I could not help wondering why working with it is so difficult. Not in terms of understanding how it works from the inside, no - I was digging into the library interface. Either the hands are not growing from there, or the developers are hardened DBAs, not like me.
Now I know what they were trying to put in the toolkit that they could. The output is the 15th standard, combining the previous 14. Indicative in this regard is the hellish juggling combine, which is able to make an extremely diverse list of databases - MySQL, SQlite3, Postgres, CouchDB, Mongo, Redis, Neo4j.
But for me for small projects - all sorts of telegram bots, server devs and SPAs - there was no need for a complex part of the functionality, which is under the hood of complex ORM-ok. The basic required functional is the saving and searching of records, sampling by conditions and relationships. I do not need premature optimizations : sampling parts of fields from the database, tricky query optimizations, stored functions. Sampling with respect to (get the object and all the links) can be made a transaction. Due to the loss in speed, we get the absence of a heap of additional entities of declarative syntax. Occam's razor in its purest form.
Lyrical digression: if you look at the history of the development of projects with tens and hundreds of thousands of users - after a certain time, developers run into speed. They change queries, database, languages, platforms, so that it works. If the project shoots out - he will have to replace parts, and the first one will work with the base under the knife - if not enough effort was spent initially “for the future”. At the same time, the heavy, complex ORM syntax complicates the replacement. The conclusion is obvious - if you evaluate the choice of ORM as an uncontested future technical debt, the correct choice may be a solution with less efficiency, but ensuring the speed of the developer and providing a minimal API, which simplifies the transition to another solution.
I made an assembly
Not a DB, but a JS-centric ActiveRecord - however, in some places I have moved away from the classic pattern.
It is important to understand that since it is not DB-centric - the database was selected for solution requests, and not the decision was made for a specific database. Storage was chosen Neo4j. This solution has pros and cons, but for now there are more pros.
If you are not familiar with neo4j, this is a popular graph database with a much more comprehensible for the outsider than SQL, a language, a convenient web client and full-text out-of-box indexes (using lucene), and a slightly lower (linear) speed compared to Postgres / Mysql. All installation instructions are here: http://neo4j.com/download-thanks/?edition=community . On the mac it is installed via brew install neo4j
Let's start with a simple connection and entries:
const {Connection, Record} = require('agregate') const dbPath = 'http://neo4j:password@localhost:7474'; class ConnectedRecord extends Record { static connection = new Connection(dbPath); } class User extends ConnectedRecord {} User.register() // , const user = new User({name: 'foo'}) user.surname = 'bar' user.save() .then(() => User.where({name: 'foo'})) .then(([user]) => console.log(user)) //=> User {name: 'foo', surname: 'bar'} The only thing that stands out from the clarity of the code - call User.register (). In JS, it is impossible to hang up a handler (and thank the developers of the language for this) to create a class, so you have to do it for the language.
The Record.register method does 3 things:
- registers this class for an existing connection to the database. Simply put - in the Map inside the connection the association "label" - "class" is inserted. When resolving associations (about them later), it is this map that is used to turn database objects into JS objects.
- runs internal library processes (indexing and uuid uniqueness for security reasons)
- starts the indexing of user indices (if they have been defined for this class).
For abstract classes, this method does not need to be called.
In ES2015, static properties are inherited in the same way as properties of an entity — connecton is declared once, in the parent class, as shown. If you have one database, you can assign a connection to Record.connection, although this is incorrect from the development point of view.
Relations and Connections
Let's complicate the example. Imagine that we are doing an ACL, and we need a relationship:
const {Connection, Record, Relation} = require('agregate'); // Role Permission const Role = require('./role'); const Permission = require('./permission'); export default class User extends ConnectedRecord { roles = new Relation(this, 'has_role', {target: Role}); permissions = new Relation(this.roles, 'has_permission', {target: Permission}); hasPermission = ::this.permissions.has } If you do not look closely - do not immediately see that in fact this.permissions - many-to-many through the relationship. This kind of syntax allows you to build long chains of relationships for which full-fledged queries are available - search, delete, check for availability, everything, except for obvious reasons not working Relation # add.
Relation emulates the Set object built into ES6. The API is different, but immediately familiar and understandable. The difference is that methods return a Promise, which already returns data, and size () a method, not a property. Additionally, there were methods #intersect, which returns the intersection of the transferred array of elements with the related elements, and #where, which does the obvious, but about it below.
Search by database
Methods with an identical API are available for this: the class method Record.where () and the instance method of the class Relation # where (). Available are offset, limit, order by, search by value, the contents of the array and entry into the array (yes, the typed array is one of the primitives in neo4j) and the substring. There are many possibilities for searching. They cover all the main tasks. Enumerating all the options is quite difficult, so it’s easier to look at the formal description in a typescript-like syntax:
var dbPrimitiveType = bool | string | number | Array<bool> | Array<string> | Array<number> async function where( params?: { [string: queryKey]: dbPrimitiveType | { $gt?: number //greater than - $gte?: number //greater than or equal - $lt?: number //less than - $lte?: number //less than or equal - $exists?: bool // $startsWith?: Array<string> | string // $endsWith?: Array<string> | string // $contains?: Array<string> | string // $has?: Array<dbPrimitiveType> | dbPrimitiveType // $in?: Array<dbPrimitiveType> | dbPrimitiveType // } }, opts?: { order?: string | Array<string>; // - key key DESC key ASC, ['created_at', 'friends DESC'] offset?: number; limit?: number; }, transaction?: Queryable): Array<Record> Transactions
The API described above already allows you to work. Only the question of atomicity remains, which is classically solved with the help of transactions.
In an aggregate, you can work with transactions in two ways - simple or understandable.
The clear way is to use head-on transactions. To do this, pass it with the last argument (among others). All standard database methods support this notation.
class Post extends Record { author = ::new Relation(this, 'created', {direction: -1}).only // only, . , . async static createAndAssign(text, user) { const transaction = this.connection.transaction() const post = await new this({text}).save(transaction) await post.author(user, transaction) await transaction.commit() return post } // , - async static createAndAssign(text, user) { const transaction = this.connection.transaction() try { const post = await new this({text}).save(transaction) await post.author(user, transaction) await transaction.commit() return post } catch (e) { await transaction.rollback() throw e } } } The connection object (which is available for both the class and the class instance) can be a connection, transaction, or sub-transaction. There is no difference for use in life, because all three entities provide the same interface with small internal differences. If you call connection.transaction (), the connection returns a transaction, the transaction returns a sub-transaction, the sub-transaction returns another sub-transaction.
The internal difference is the following: the commit and rollback methods for connection will send an error, for a transaction, it will work as expected, for a sub-transaction - commit will do nothing, and rollback will roll back the parent transaction.
This is done because some methods about generate a transaction for themselves and close at the end - for example, Record # save (). To ensure that such methods work correctly within a transaction, the infinite nesting of sub-transactions is implemented.
For the second method - simple - decorator is used:
import {Record, acceptsTransaction} from 'agregate' class Post extends Record { @acceptsTransaction async static create(text) { return await new this({text}).save(this.connection) } } It turns the code into something like this:
import {Record, acceptsTransaction} from 'agregate' class Post extends Record { async static create(text, transaction) { //Queryable - , Connection, Transaction, SubTransaction if (transaction instanceof Queryable) this.connection = transaction try { const result = await new this({text}).save(this.connection) if (transaction) await transaction.commit() return result } catch (e) { if (transaction) await transaction.rollback() throw e } } } The decorator can be used both directly, as in the example above, and configuring. For the configuration so far, only one flag is available - force, which forcibly creates a transaction - if a transaction is not transferred, it will create it. You need to use this:
@acceptsTransaction({force: true}) ...Note that now this.connection has become a transaction. When the function runs, the property will return to its previous state, but now it allows you to call other methods of the class, without worrying about committing the transaction. This magic works only within this (which is predictable).
Since transactions are processed in turn, i.e. until one is completed, another is not started, the object is not cloned, so consider: if you wrap the static method in this decorator, you can accidentally “fumble” the transaction. For class instances, this is not a problem because if you work correctly with JS, they are in their field of view, and from other execution threads (such as promises, async, etc.) you cannot access them at the same time. unavailability of the object.
That's the whole unit
The description of the API and the reasons why it was done this way and not otherwise, is complete.
Probably the only thing worth adding is that I already use it in small projects for myself and friends. I haven’t experienced such a pleasure when working with a database for a long time - I felt such a feeling of "transparency" and clarity of the working mechanisms only when working in Ruby / Rails, and even there I had to suffer from CLI in places.
The unit may lack some capabilities or speed, but if you want this - connect to the project. Now the aggregate is only 608 lines (in shock itself) of fairly well-organized code, and it is very easy to make edits, additions, updates, and do additional tests. I would like to see it uniquely usable in a big production, and if you like it too, connect to the development!
Source: https://habr.com/ru/post/278871/
All Articles