Maraquia - ORM for MongoDB

After reading the title, many probably have a question - why another bike if there are already run-in Mongoose, Mongorito, TypeORM, etc.? To answer, you need to understand what is the difference between ORM and ODM. Enjoying Wikipedia:

ORM (English Object-Relational Mapping, Russian object-relational mapping, or transformation) is a programming technology that links databases with the concepts of object-oriented programming languages, creating a "virtual object database."

That is, ORM is about relational data representation. Let me remind you that in relational databases there is no possibility to simply take and embed a document in the field of another document (in this article, the records of the tables are also called documents, even though it is incorrect), you can of course be stored in the JSON field as a string, but the index does not make an will come out. Instead, “links” are used - in the field where the attached document should be, its identifier is written instead, and the document with this identifier is stored in the next table. ORM can work with such links - records for them are automatically immediately or lazily taken from the database, and when saving, you do not need to first save the child document, take the identifier assigned to it, write it in the field of the parent document and only then save the parent document. You just need to ask ORM to save the parent document and everything connected with it, and he (the object-relational mapper) will figure out how to do it correctly. ODM, on the contrary, does not know how to work with such links, but it knows about embedded documents.

I think the differences are roughly clear, so all of the above is exactly ODM. Even TypeORM when working with MongoDB has some limitations ( https://github.com/typeorm/typeorm/issues/655 ) which make it again an ordinary ODM.

And then you ask - why? Why working with document-oriented database did I need any links? There is at least one simple, but often occurring situation when they are still necessary: ​​several parents point to a child document, here each parent can write a copy of the child and then suffer ensuring the consistency of the data in these copies, or you can simply save the child document in a separate collection, and give all parents a link to it (you can still embed the parent in the child, but this is not always possible, first, the relationship may be many-to-many, secondly, the child type may be too minor n in the system, and tomorrow may disappear altogether from the database embedded in it something key does not want to).

For a long time I have worked with RethinkDB for which there are some quite good ORMs ( thinky , requelize , ...), but recently the development activity of this database has been quite disheartening. I decided to look in the direction of MongoDB and the first thing I didn’t find was these packages. Why not write it yourself, it will be quite an interesting experience, I thought, and meet - Maraquia.

Installation

npm i -S maraquia 

When used with typescript, you must also add "experimentalDecorators": true to tsconfig.json .

Connection setup

There are two ways, here we consider a simpler one: in the project folder we create the file config/maraquia.json to which we add the following:

 { "databaseUrl": "mongodb://localhost:27017/", "databaseName": "Test" } 

Using

Saving to DB

A simple example of a one-to-many relationship with a link in one direction only (examples will be on typescript, at the end a javascript example):

 import { BaseModel, Field, Model } from 'maraquia'; @Model({ collectionName: 'Pet' }) class Pet extends BaseModel { @Field() name: string | null; } @Model({ collectionName: 'Owner' }) class Owner extends BaseModel { @Field() name: string | null; @Field(() => Pet) pets: Promise<Array<Pet> | null>; } (async () => { //         let pet = new Pet({ name: 'Tatoshka' }); let owner = new Owner({ name: 'Dmitry', pets: [pet] }); await owner.save(); })(); 

Two collections Pet and Owner will appear in the database with entries:

 { "_id": "5a...1f44", "name": "Tatoshka" } 

and

 { "_id": "5a...1f43", "name": "Dmitry", "pets": ["5a...1f44"] } 

The save method was called only on the owner model, Maraquia as it should be, she herself took care of saving the second document.

Let's complicate the example, now the relation of many-to-many and references in both directions:

 @Model({ collectionName: 'User' }) class User extends BaseModel { @Field() name: string | null; @Field(() => Group) groups: Promise<Array<Group> | null>; } @Model({ collectionName: 'Group' }) class Group extends BaseModel { @Field() name: string | null; @Field(() => User) users: Promise<Array<User> | null>; } let user1 = new User({ name: 'Dmitry' }); let user2 = new User({ name: 'Tatoshka' }); let group1 = new Group({ name: 'Admins', users: [user1] }); let group2 = new Group({ name: 'Moderators', users: [user1, user2] }); user1.groups = [group1, group2] as any; user2.groups = [group2] as any; await group1.save(); 

The User collection will appear in the database with the entries:

 { "_id": "5a...c56f", "name": "Dmitry", "groups": ["5a...c56e", "5a...c570"] } { "_id": "5a...c571", "name": "Tatoshka", "groups": ["5a...c570"] } 

and the Group collection with entries:

 { "_id": "5a...c56e", "name": "Admins", "users": ["5a...c56f"] } { "_id": "5a...c570", "name": "Moderators", "users": ["5a...c56f", "5a...c571"] } 

You, probably, have already noticed the absence of decorators with names like hasOne , hasMany , belongsTo as it is usually accepted for ORM. Maraquia copes without this additional information, hasOne or hasMany is determined by the value, the array means hasMany. A built-in document or external (stored in a separate collection) is determined by the presence in its scheme of a filled collectionName . For example, if in the first example you comment out the line collectionName: 'Pet' and restart it, then the entry will appear only in the Owner collection and will look like this:

 { "_id": "5b...ec43", "name": "Dmitry", "pets": [{ "name":"Tatoshka" }] } 

In addition, the type of pets field is no longer promis.
That is, with the help of Maraquia, you can also conveniently work with embedded documents.

Reading from DB

Let's try to read from the database something from the previously saved:

 let user = User.find<User>({ name: 'Dmitry' }); console.log(user instanceof User); // true console.log(user.name); // 'Dmitry' console.log(await user.groups); // [Group { name: 'Admins', ... }, Group { name: 'Moderators', ... }] 

When reading the groups field, the await keyword was used - external documents are retrieved from the database lazily when they first read the corresponding field.

But what if you need to have access to the identifiers stored in the field without pulling out the corresponding documents from the database, but optionally you may need to pull them out? The name of the field in the model corresponds to the name of the field in the document, but using the dbFieldName option dbFieldName can change this correspondence. That is, by defining two fields in the model that refer to one field in the document and without specifying the type for one of them, you can solve this problem:

 @Model({ collectionName: 'Group' }) class Group extends BaseModel { @Field({ dbFieldName: 'users' }) readonly userIds: Array<ObjectId> | null; //    @Field(() => User) users: Promise<Array<User> | null>; //       } 

Deleting a document

The remove method removes the corresponding document from the database. Maraquia does not know where there are links to it and here the programmer needs to work on his own:

 @Model({ collectionName: 'User' }) class User extends BaseModel { @Field() name: string | null; @Field({ dbFieldName: 'groups' }) groupIds: Array<ObjectId> | null; @Field(() => Group) groups: Promise<Array<Group> | null>; } @Model({ collectionName: 'Group' }) class Group extends BaseModel { @Field() name: string | null; @Field({ dbFieldName: 'users' }) userIds: Array<ObjectId> | null; @Field(() => User) users: Promise<Array<User> | null>; } let user = (await User.find<User>({ name: 'Tatoshka' }))!; //     for (let group of await Group.findAll<Group>({ _id: { $in: user.groupIds } })) { group.userIds = group.userIds!.filter( userId => userId.toHexString() != user._id!.toHexString() ); await group.save(); } //    await user.remove(); 

In this example, the userIds array userIds been replaced with a new one created by the Array#filter method, but the existing array can be changed, Maraquia finds such changes. That is, it was possible so:

 group.userIds!.splice( group.userIds!.findIndex(userId => userId.toHexString() == user._id!.toHexString()), 1 ); 

Validation

To validate a field, you must add the validate property to its options:

 @Model({ collectionName: 'User' }) class User extends BaseModel { @Field({ validate: value => typeof value == 'string' && value.trim().length >= 2 }) name: string | null; @Field({ validate: value => { //  false     : return typeof value == 'number' && value >= 0; //     : if (typeof value != 'number' || value < 0) { return '-   '; //  : return new TypeError('-   '); //  : throw new TypeError('-   '); } } }) age: number | null; @Field(() => Account, { validate: value => !!value }) /*    : @Field({ type: () => Account, validate: value => !!value }) */ account: Promise<Account | null>; } 

You can also transfer objects created by the joi library:

 import * as joi from 'joi'; @Model({ collectionName: 'User' }) class User extends BaseModel { @Field({ validate: joi.string().min(2) }) name: string | null; @Field({ validate: joi.number().min(0) }) age: number | null; } 

Hooks

The following methods work according to their name: beforeSave , afterSave , beforeRemove , afterRemove .

Use with javascript

Typescript is great, but sometimes you need it without it. To do this, instead of the object passed to the Model decorator, you need to define a $schema static field, which also has a fields field:

 const { BaseModel } = require('maraquia'); class Pet extends BaseModel { } Pet.$schema = { collectionName: 'Pet', fields: { name: {} } }; class Owner extends BaseModel { } Owner.$schema = { collectionName: 'Owner', fields: { name: {}, pets: { type: () => Pet } } }; let pet = new Pet({ name: 'Tatoshka' }); let owner = new Owner({ name: 'Dmitry', pets: [pet] }); await owner.save(); 

Writing to fields is done via the setField method:

 pet.setField('name', 'Tosha'); 

And reading fields with external documents through the fetchField method:

 await owner.fetchField('pets'); 

The remaining fields are read as usual.

Performance

I wrote a couple of simple benchmarks to compare performance with Mongoose and Mongorito. In the first, instances of the model are simply created. For all three, it looks the same:

 let cat = new Cat({ name: 'Tatoshka', age: 1, gender: '1', email: 'tatoshka@email.ru', phone: '+79991234567' }); 

Result (more is better):

 Mongoose x 41,382 ops/sec ±7.38% (78 runs sampled) Mongorito x 28,649 ops/sec ±3.20% (85 runs sampled) Maraquia x 1,312,816 ops/sec ±1.70% (87 runs sampled) 

In the second the same, but with preservation in the database. Result:

 Mongoose x 1,125 ops/sec ±4.59% (69 runs sampled) Mongorito x 1,596 ops/sec ±4.08% (69 runs sampled) Maraquia x 1,143 ops/sec ±3.39% (73 runs sampled) 

Sources in the perf folder.

Footer

I hope someone will find the library useful, it has not yet been used on real projects, so use it at your own peril and risk. If something doesn't work as expected, create an issue on github .

Basically, I am engaged in front-end development and I am hardly well versed in databases, so if I have written nonsense somewhere, then please understand and forgive :).

Thank you for attention.