Another Reflection Library and ORM for C ++

Immediately warn about cycling issued here for review. If the reading of the headline causes only the hardly suppressed exclamation “Your mother, only not the new ORM ~~taxon~~ !”, Then it is better to refrain from further reading in order not to increase the level of aggression in the cosmological broth in which we are swimming. The reason for the appearance of this article was the fact that for once I had a vacation, during which I decided to try myself in the field of writing blog posts on okolokhabokrovskoy topics, and the proposed topic seemed to me quite suitable for this. In addition, here I am waiting for constructive criticism, and it is possible to understand what else can be done with such interesting things. At the end there will be a link to the github repository, where you can see the code.

What is another ORM library for?

When developing 3-tier applications with separate presentation layers (Presentation tier), business logic (Logic tier) and data storage (Data tier), there is always a problem of limiting the interaction of application components at the interface of these layers. Traditionally, the interface to relational databases is provided based on the language of SQL queries, but its use directly from the level of business logic is usually associated with a number of problems, some of which are easily solved by using ORM (Object-relational mapping):

The need to represent entities in two forms: object-oriented and relational
The need for conversion between these two forms
Exposure to errors when manually writing SQL queries (partly can be solved using various lint-utilities and plug-ins for modern IDE)

The presence of such a simple solution to these problems led to the abundance of various implementations of ORM for every taste and color (the list is on Wikipedia ). Despite the abundance of existing solutions, there will always be ~~perverts~~ “gourmets” (the author is one of them), whose tastes cannot be satisfied with the existing range. But how else, these are consumer goods, and our project is too unique, and the existing solutions simply do not suit us ( this is sarcasm, the signature of KO ).
')

Probably similar maximalistic thoughts led me when a couple of years ago I took up the writing of ORM to fit my needs. In short, I will still describe what was wrong with those ORMs that I tried and what I wanted to fix in them.

Firstly, this is the need for static typing, which would allow to catch most of the errors when writing queries to the DBMS during compilation, and therefore would significantly speed up development speed.
Condition for implementation: this must be a reasonable compromise between the level of query validation, compile time (which in the case of C ++ is also associated with the responsiveness of the IDE) and readability of the code.
Secondly, it is flexibility, the ability to write arbitrary (within reasonable limits) requests. In practice, this clause boils down to the possibility of writing a LMS (create-delete-get-update) queries with arbitrary WHERE sub-expressions and the ability to execute cross-table queries.
This is followed by the support of different vendors' DBMSs at the “program must continue to work correctly when jumping from one DBMS to another”.
The ability to reuse the ORM reflection for other needs (serialization, script-binding, factories untied from the implementation, etc.). What can we say, most often the reflection in existing solutions "nailed" to ORM.
Still, I don't want to depend on code generators a la Qt moc, protoc, thrift. Therefore, we will try to do only with the means of the C ++ templates and the preprocessor C.

Actually implementation

Let's look at it on a “toy” example from a SQL textbook. We have 2 tables: Customer and Booking, related to each other by a one-to-many relationship.

In code, the declaration of classes in the header is as follows:

//    struct Customer : public Object { uint64_t id; String first_name; String second_name; Nullable<String> middle_name; Nullable<DateTime> birthday; bool news_subscription; META_INFO_DECLARE(Customer) }; struct Booking : public Object { uint64_t id; uint64_t customer_id; String title; uint64_t price; double quantity; META_INFO_DECLARE(Booking) };

As you can see, such classes are inherited from the common ancestor of Object (why be original?), And in addition to declaring methods, contains the macro META_INFO_DECLARE. This method simply adds the declaration of the overloaded and overridden Object methods. Some fields are declared via the Nullable wrapper, as it is not difficult to guess, such fields may take the special value NULL. Also, all column fields must be public.

The definition of classes is somewhat more monstrous:

 STRUCT_INFO_BEGIN(Customer) FIELD(Customer, id) FIELD(Customer, first_name) FIELD(Customer, second_name) FIELD(Customer, middle_name) FIELD(Customer, birthday) FIELD(Customer, news_subscription, false) STRUCT_INFO_END(Customer) REFLECTIBLE_F(Customer) META_INFO(Customer) DEFINE_STORABLE(Customer, PRIMARY_KEY(COL(Customer::id)), CHECK(COL(Customer::birthday), COL(Customer::birthday) < DateTime(1998, January, 1)) ) STRUCT_INFO_BEGIN(Booking) FIELD(Booking, id) FIELD(Booking, customer_id) FIELD(Booking, title, "noname") FIELD(Booking, price) FIELD(Booking, quantity) STRUCT_INFO_END(Booking) REFLECTIBLE_F(Booking) META_INFO(Booking) DEFINE_STORABLE(Booking, PRIMARY_KEY(COL(Booking::id)), INDEX(COL(Booking::customer_id)), // N-to-1 relation REFERENCES(COL(Booking::customer_id), COL(Customer::id)) )

The STRUCT_INFO_BEGIN ... STRUCT_INFO_END block creates definitions of the reflection fields of the class fields. The macro REFLECTIBLE_F creates a class descriptor for fields (there is also REFLECTIBLE_M, REFLECTIBLE_FM to create descriptors of classes that support reflection methods, but this is not the post). The META_INFO macro creates definitions for the overloaded Object methods. Finally, the most interesting for us macro DEFINE_STORABLE creates the definition of a relational table based on the reflection of the class and the declared constraints (constraints) that ensure the integrity of our schema. In particular, one-to-many relationships between tables are checked and the check on the birthday field (just for example, we want to serve only adult clients). Creating the necessary tables in the database is simple:

  SqlTransaction transaction; Storable<Customer>::createSchema(transaction); Storable<Booking>::createSchema(transaction); transaction.commit();

SqlTransaction, as it is not difficult to guess, provides isolation and atomicity of the operations performed, and also captures the connection to the database (there can be several named connections to different DBMS, or parallelization of queries to one DBMS - Connection Pooling). In this regard, you should avoid recursive instantiation of transactions - you can get a Dead Lock. All requests must be executed in the context of a transaction.

Requests

Query examples

INSERT

This is the simplest type of query. Simply prepare our object and call the insertOne method on it:

  SqlTransaction transaction; Storable<Customer> customer; customer.init(); customer.first_name = "Ivan"; customer.second_name = "Ivanov"; customer.insertOne(transaction); Storable<Booking> booking; booking.customer_id = customer.id; booking.price = 1000; booking.quantity = 2.0; booking.insertOne(transaction); transaction.commit();

You can also add several entries to the database with one command (Batch Insert). In this case, the request will be prepared only once:

  Array<Customer> customers; //    SqlTransaction transaction; Storable<Customer>::insertAll(transaction, customers); transaction.commit();

SELECT

Data retrieval from the database is generally performed as follows:

  const int itemsOnPage = 10; Storable<Booking> booking; SqlResultSet resultSet = booking.select().innerJoin<Customer>() .where(COL(Customer::id) == COL(Booking::customer_id) && COL(Customer::second_name) == String("Ivanov")) .offset(page * itemsOnPage).limit(itemsOnPage) .orderAsc(COL(Customer::second_name), COL(Customer::first_name)) .orderDesc(COL(Booking::id)).exec(transaction); // Forward iteration for (auto& row : resultSet) { std::cout << "Booking id: " << booking.id << ", title: " << booking.title << std::endl; }

In this case, the paginal output of all orders Ivanov. The alternative is to get all
table entries list:

  auto customers = Storable<Customer>::fetchAll(transaction, COL(Customer::birthday) == db::null); for (auto& customer : customers) { std::cout << customer.first_name << " " << customer.second_name << std::endl; }

UPDATE

One of the scenarios: updating the record just received from the database with the primary key:

  Storable<Customer> customer; auto resultSet = customer.select() .where(COL(Customer::birthday) == db::null) .exec(transaction); for (auto row : resultSet) { customer.birthday = DateTime::now(); customer.updateOne(transaction); } transaction.commit();

Alternatively, you can create a request manually:

  Storable<Booking> booking; booking.update() .ref<Customer>() .set(COL(Booking::title) = "All sold out", COL(Booking::price) = 0) .where(COL(Booking::customer_id) == COL(Customer::id) && COL(Booking::title) == String("noname") && COL(Customer::first_name) == String("Ivanov")) .exec(transaction); transaction.commit();

DELETE

Similarly, with an update request, you can delete the entry by primary key:

  Storable<Customer> customer; auto resultSet = customer.select() .where(COL(Customer::birthday) == db::null) .exec(transaction); for (auto row : resultSet) { customer.removeOne(transaction); } transaction.commit();

Or through the request:

  Storable<Booking> booking; booking.remove() .ref<Customer>() .where(COL(Booking::customer_id) == COL(Customer::id) && COL(Customer::second_name) == String("Ivanov")) .exec(transaction); transaction.commit();

The main thing you need to pay attention to is, where the subquery is a C ++ expression, on the basis of which an abstract syntax tree (AST) is built. Further, this tree is transformed into a SQL expression of a specific syntax. This is what provides the static typing I mentioned at the beginning. Also, an interim request form in the form of AST allows us to describe the request in a unified manner regardless of the supplier of the DBMS; I had to expend a certain amount of effort on this. The current version supports PostgreSQL, SQLite3 and MariaDB. On vanilla MySQL, too, in principle, it should start, but this DBMS otherwise processes some data types, respectively, some of the tests fail.

What else

You can describe custom stored procedures and use them in queries. Now ORM supports some built-in DBMS functions out of the box (upper, lower, ltrim, rtrim, random, abs, coalesce, etc.), but you can define your own. So, for example, the strftime function is described in SQLite:

 namespace sqlite { inline ExpressionNodeFunctionCall<String> strftime(const String& fmt, const ExpressionNode<DateTime>& dt) { return ExpressionNodeFunctionCall<String>("strftime", fmt, dt); } }

In addition, the implementation of ORM is not limited to the possible use of reflection. It seems that we will not soon get correct reflection in C ++ (correct reflection should be static, that is, provided at the level of the compiler, not the library), so you can try to use this rationalization for serialization and integration with script engines. But about this, I may write another time, if someone has an interest.

What not

The main drawback in the SQL module is that I never managed to support aggregated queries (count, max, min) and grouping (group by). Also, the list of supported DBMS is rather poor. Perhaps in the future I will do SQL Server support through ODBC.
In addition, there are thoughts on integration with mongodb, especially since the library allows describing non-planar structures (with substructures and arrays).

Link to the repository.

Source: https://habr.com/ru/post/282660/

All Articles