Practical metametamodeling
The declarative approach and the MDA Ahritiecture have a number of advantages, which allow to significantly reduce the costs for the development and support of information systems (IC: CRM, WMS, Project Management, etc). This approach is already used in a number of products (such as 1C, for example). However, the declarative approach in them is used to solve too narrow a range of tasks. In this article, we will look at the advantages of the declarative approach, show how you can significantly expand its area of ​​application in the construction of IP, check the model built on real problems and demonstrate the work of the prototype.
My bachelor’s and master’s were associated with MDA, and we’ve been working with these former groupmates for a year to apply these ideas to building information systems. We do not represent any commercial product, everything that we did / came up with was developed “on our knees” in our free time.
Our ideas can be applied both in complex information system designers (such as 1C) and in web frameworks (Django, RoR). It is interesting to know your opinion and comments. In addition, we are looking for companies that are interested in cooperation with the aim of using our developments in their products.
It all started with the fact that quite by chance the task arose of creating a small CRM sharpened for a office flow office. I began to look at different solutions, from MS Access, to web frameworks (Django, RoR). But they all turned out to be too low-level. I wanted to work in terms of “users”, “events”, “tasks”, etc., rather than database objects, and to have corresponding visual presentations (calendars, lists with filters, etc.).
')
For the development of information systems, object-relational notation (OORDBMS, ORM) is most often used, within which business rules are imperatively described in one of the software modules. We propose extending the object-relational notation with new concepts, which will allow declaratively describe business rules. We strive to bring the “implementation” language as close as possible to the requirements formalization language.
The use of high-level abstractions, coupled with a declarative approach allows not only to simplify and accelerate the development and adaptation of IP. The transfer of business rules from the code level to the data level, qualitatively affects not only the development process, but also the product properties:
The last point is worth noting especially - at the moment the “configurability” of cloud solutions remains at an extremely low level - at best, you can add a few of your attributes to already existing entities. The transfer of business rules to the data area allows for different users of the service to have this data completely different, without interfering with the server. Each user in the cloud will be able to have a system adapted specifically for him.
Consider a set of characteristic tasks that arise in the design of IP and from which we are repelled in the construction of the language:
Consider the basic concepts that we add to the object-relational model:
Classes already exist in the object-relational language, but we have significantly changed this concept (expanded and truncated at the same time), bringing it closer to the concept of “essence” in the ER notation:
Examples of “inclusions” are links: a class and its fields, a table and its attributes, a user and its profile. The object “class field” is not independent; it cannot exist outside its parent class.
The ability to set an expression in an XPath-like language as a value for an object field. The concept of computable expressions and computable fields is used in all other extensions of our language. It should be noted that users of ORM frameworks or PL / SQL use the capabilities of their language when implementing get / set methods, but this is not at all the same, because information about the calculation is described in the turing-full language. We specifically simplify the expression language to non-turing-complete in order to:
Thus, information about the method of calculating the field is contained in the description of the structure of the object, which allows for this structure to automatically
Interfaces are well known in OOP, but unfairly forgotten when describing databases. We introduce the interfaces as follows:
The concept of "providing the interface" is an important tool for reuse. It, as well as the implementation of the interface, can be compared with relational views. The essential difference between the “presentation” and the presentation is the calculation method: once the same interface can be provided by different entities and not once, then it’s extremely difficult to get a list of all the “submissions” by one SELECT (for example, to display all events on the calendar) .
Consider how the above tasks are implemented using the constructions we introduced:
The remaining items related to the description of business processes and the management of access rights require concepts that we will consider in the next article if this topic interests you.
We reached the level of “first-playable” and created a configuration for our friend’s cleaning company, which implements inventory accounting. You can see it here demo.meta4.info (username / password root / root). The entire form of the forms, the dialogues, the logic of operation are determined by the following configuration (however, it does not include a description of access restrictions):
We created the system to support the SaaS mode, so that one server can have many “projects”, each of which has its own classes of operated objects. That is, one group of users may have CRM, the second has warehouse accounting, and the third also has warehouse accounting, but specialized for the restaurant business, and all this on one server.
As a result of our work, we have formed 3 “artifacts”:
Of course, we are far from the full implementation of our ideas. For example, such a concept as “accumulation register” is not implemented, now it is manually emulated by triggers. Nevertheless, this year we have formed a vision of the language and architecture of the system implementing it.
My bachelor’s and master’s were associated with MDA, and we’ve been working with these former groupmates for a year to apply these ideas to building information systems. We do not represent any commercial product, everything that we did / came up with was developed “on our knees” in our free time.
Our ideas can be applied both in complex information system designers (such as 1C) and in web frameworks (Django, RoR). It is interesting to know your opinion and comments. In addition, we are looking for companies that are interested in cooperation with the aim of using our developments in their products.
How it all began
It all started with the fact that quite by chance the task arose of creating a small CRM sharpened for a office flow office. I began to look at different solutions, from MS Access, to web frameworks (Django, RoR). But they all turned out to be too low-level. I wanted to work in terms of “users”, “events”, “tasks”, etc., rather than database objects, and to have corresponding visual presentations (calendars, lists with filters, etc.).
')
What do we offer
For the development of information systems, object-relational notation (OORDBMS, ORM) is most often used, within which business rules are imperatively described in one of the software modules. We propose extending the object-relational notation with new concepts, which will allow declaratively describe business rules. We strive to bring the “implementation” language as close as possible to the requirements formalization language.
Why is the declarative description of business rules important?
The use of high-level abstractions, coupled with a declarative approach allows not only to simplify and accelerate the development and adaptation of IP. The transfer of business rules from the code level to the data level, qualitatively affects not only the development process, but also the product properties:
- Partial automation of database updates for new versions
- On-the-fly system updates
- Customization of the system in the visual designer by a business analyst or end user
- Within one SaaS service, you can maintain a dynamically expanding set of configurations.
The last point is worth noting especially - at the moment the “configurability” of cloud solutions remains at an extremely low level - at best, you can add a few of your attributes to already existing entities. The transfer of business rules to the data area allows for different users of the service to have this data completely different, without interfering with the server. Each user in the cloud will be able to have a system adapted specifically for him.
Examples of tasks
Consider a set of characteristic tasks that arise in the design of IP and from which we are repelled in the construction of the language:
- Display the cost (price * quantity), price including VAT based on the price and VAT rate, etc. In the user interface, these values ​​should be automatically recalculated, and the server should check the correctness of the received data (or calculate it itself)
- A more complex example: the total cost of purchasing for all goods is based on a given price list. Support should be from the client and the server.
- Display on the calendar events generated by different sources: the user's birthday, the beginning of the rally, the end of the Mile, etc.
- Specialize the “user” structure of the system
- Different set of parameters for different categories of users, for example, for customers and employees
- Reuse of code on objects with different structure.
For example, writing off, moving and posting goods have a different structure but identical actions. - Business rules:
- Charge of a fine for each overdue day of the task to its performer
- Calculation of rent based on the duration of use and daily cost
- Notifications:
- director of the task of its completion
- director when writing off expensive equipment
- Fixing the price list after it is agreed
- Managers can only see the clients that are assigned to them.
New concepts
Consider the basic concepts that we add to the object-relational model:
Classes
Classes already exist in the object-relational language, but we have significantly changed this concept (expanded and truncated at the same time), bringing it closer to the concept of “essence” in the ER notation:
- Classes have fields only (no methods)
- Classes allow inheritance (and, as in the object-relational extension of SQL, when requesting a base class, objects are returned not only of the base class, but also of successor classes)
- Classes are abstract (i.e., which cannot have instances). Abstract classes can have abstract fields that need to be “implemented” in non-abstract child classes: either specify a computable expression or declare a simple stored field.
Field types can be specified during inheritance (if a class field refers to class A, then during inheritance, you can specify a reference to class B, if B is a successor of A.) - Reference fields are classified as
- Links
- Compositions (analogue of compositions in UML) - the “composable” object has a lifetime limited by the lifetime of the “compositing” object
- Inclusions (I have not seen analogs in well-known programming or modeling languages) are an enhancement to the composition — the nested object is not conceived outside its parent
Examples of “inclusions” are links: a class and its fields, a table and its attributes, a user and its profile. The object “class field” is not independent; it cannot exist outside its parent class.
Computable fields
The ability to set an expression in an XPath-like language as a value for an object field. The concept of computable expressions and computable fields is used in all other extensions of our language. It should be noted that users of ORM frameworks or PL / SQL use the capabilities of their language when implementing get / set methods, but this is not at all the same, because information about the calculation is described in the turing-full language. We specifically simplify the expression language to non-turing-complete in order to:
- Effectively interpret it on different platforms (client, server) that use a different way to access data
- Automatically determine dependencies for recomputation of an expression when dependent fields change for dependent objects
Thus, information about the method of calculating the field is contained in the description of the structure of the object, which allows for this structure to automatically
- Validate the object on the server
- Automatically update calculated fields in user interface
Interfaces
Interfaces are well known in OOP, but unfairly forgotten when describing databases. We introduce the interfaces as follows:
- The interface is an abstract class for which all fields are abstract.
- Interface implementation is identical to inheritance
- Interfaces are completely interchangeable with classes: wherever you can use classes, you can use interfaces (by analogy with representations in relational theory).
That is, you can- To do selections on the interface (result of sampling - all objects implementing the given interface)
- Modify an object through an interface (if the interface field is implemented by binding to a class field)
- Create triggers on interfaces (in this case, the trigger will work for all classes-implementations)
- A class can "provide" an interface. For this, a “connected” class is described that implements the specified interface using the fields of the class that “provides” the interface.
- A class can provide the same interface multiple times (“grants” are named)
- Unlike inheritance, “grants” can be described outside the class (an analogy can be drawn with the “Adapter” pattern)
- For the structure of the "connected" class, as with any inheritance, it is acceptable to specify the field types
The concept of "providing the interface" is an important tool for reuse. It, as well as the implementation of the interface, can be compared with relational views. The essential difference between the “presentation” and the presentation is the calculation method: once the same interface can be provided by different entities and not once, then it’s extremely difficult to get a list of all the “submissions” by one SELECT (for example, to display all events on the calendar) .
Test expressive language
Consider how the above tasks are implemented using the constructions we introduced:
Display cost as a product of price by quantity, price including VAT
This completely falls on the concept of "computable fields", and the language of expression description, of course, allows you to describe the sum and product :)A more complex example: the total cost of purchasing for all goods is based on a given price list.
Let the purchase be described as an object with “total value” fields and a “goods” field containing a list of “goods-quantity-value” objects.
Set the field "cost" in each object "product-quantity-value" by the expression:- take the price list from the purchase
- filter the price list by product, which should be the same as my
- take the first and only item from the resulting list
- take the price field from him.
In the class “Purchase” the field “total cost” should be set to the expression “the sum of all records of goods-quantity by a computable field„ cost “”.
The task of the system is to build a transitive closure of dependencies so that a change in “quantity” in one of the “item-quantity-cost” records leads to a recalculation of the total cost.Display on the calendar events generated by a variety of sources
To do this, you must enter the interface "Event" with the fields "name", "date", "period", "duration". UI code always uses the Event interface to search and display events.
In order for the calendar to “find” the birthdays of users, it is necessary to describe the “provision” of the “Event” interface by the user as follows:- "Name" - the expression "Birthday $ name $ last name",
- Date is user.date of birth
- "Period" - the constant expression "1 year"
- "Duration" - the constant expression "all day"
Specialize the “user” structure of the system
To do this, the concept of "user" will describe the interface (for example, with the fields "login" and "password"). Interfaces are indistinguishable from classes in terms of manipulation, and the interface can be implemented by a class with an arbitrary structure.Different set of parameters for different categories of users, for example, for clients and employees of the company
So how to implement the same interface can be different classes, user classes can be an arbitrary number. The only thing that is necessary for this is that the authorization code does all the work with users through the interface.Reuse of code on objects with different structure.
Obviously implemented by interfaces or abstract classes.
It is important to note that the field types can be clarified during the implementation: if there is a “transfer” interface and it has a list field of the “transfer record” type with the fields “inventory” and “quantity”, then the “purchase” entity can have list field field with the type of "purchase record" with the fields "inventory", "quantity", "purchase price", and it does not affect the processing code of the movement.
The remaining items related to the description of business processes and the management of access rights require concepts that we will consider in the next article if this topic interests you.
What we have implemented
We reached the level of “first-playable” and created a configuration for our friend’s cleaning company, which implements inventory accounting. You can see it here demo.meta4.info (username / password root / root). The entire form of the forms, the dialogues, the logic of operation are determined by the following configuration (however, it does not include a description of access restrictions):
XML configuration
<projectDescription name="CleaningWMS"> <!-- --> <interface name="Ware class"> <attribute name="in stock" type="float"/> <attribute name="local count" type="float"/> </interface> <!-- : , , --> <interface name="Wares"> <attribute name="amount" type="float"/> <relation name="clazz" to="WareClass" kind="reference"/> </interface> <!-- . ( ) --> <interface name="Warehouse"> <relation name="entries" to="Wares" kind="embedding" list="true"/> </interface> <!-- --> <interface name="Transfer"> <attribute name="approved" type="boolean"/> <relation name="from" to="Warehouse" kind="reference"/> <relation name="to" to="Warehouse" kind="reference"/> <relation name="entries" to="Wares" kind="embedding" list="true"/> </interface> <!-- / --> <trigger class="com.meta4.cms.warehouse.TransferTrigger" on="Transfer.approved"/> <!-- --> <trigger class="com.meta4.cms.warehouse.WareClassTrigger" on="Transfer.approved"/> <trigger class="com.meta4.cms.warehouse.RequestTrigger" on="."/> <trigger class="com.meta4.cms.warehouse.RequestTrigger" on="."/> <enum name=""> <value name=""/> <value name=""/> <value name=""/> </enum> <enum name=" "> <value name=""/> <value name=""/> <value name=""/> </enum> <enum name=" "> <value name=" "/> <value name=" "/> <value name=""/> <value name=""/> <value name=""/> <value name=""/> </enum> <enum name=""> <value name=""/> <value name=""/> </enum> <enum name=""> <value name=""/> <value name=".."/> <value name=""/> <value name=""/> </enum> <enum name=""> <value name=""/> <value name=""/> <value name=" "/> <value name=" "/> <value name=" "/> <value name=""/> </enum> <entity name="" parent="BaseUser"> <!-- . , _ --> <attribute name="__label" type="string" template="$_" specialize="__label"/> <attribute name=" " optional="false" type="string" specialize="login"/> <attribute name="" type=""/> <attribute name="" type="password" specialize="password"/> <attribute name="" type="" list="true" expressionClass="com.meta4.cms.warehouse.DepartmentExpression"/> <presentation tableColumns="_, "> <l10n> = = </l10n> <table> <row> <subordinate name="_"/> <subordinate name=""/> </row> <row> <subordinate name=""/> </row> </table> </presentation> </entity> <entity name="" parent="Warehouse"> <attribute name="" optional="false" type="string" specialize="__label"/> <attribute name="" type="address"/> <!-- , : Wares - "" --> <relation name="" kind="embedding" list="true" to="" specialize="Warehouse.entries" readonly="true"/> <presentation> <l10n> = = </l10n> </presentation> </entity> <entity name="" parent="WareClass"> <attribute name="" type="" optional="false"/> <attribute name="" type="string" optional="false" specialize="__label" filterable="true"/> <attribute name="" type="" list="true" optional="false"/> <attribute name=" " type="integer" optional="false"/> <attribute pname=" , " name=" " type="float"/> <attribute name=" " type="float" optional="false" specialize="WareClass.in_stock" default="0" readonly="true"/> <attribute name=" " type="float" optional="false" specialize="WareClass.local_count" default="0" readonly="true"/> <attribute name="" type="integer" optional="false"/> <attribute name=" " type="integer"/> <attribute name=" " type="integer" value="it. / it._"/> <attribute name=" " type="integer"/> <attribute name=" " type="integer" value="it. / it._"/> <presentation tableColumns=", , , _, _, , __, __"> <l10n> = </l10n> <table> <row> <subordinate name=""/> <subordinate name="_"/> </row> <row> <subordinate name=""/> <subordinate name="_"/> </row> <row> <subordinate name=""/> </row> <row> <subordinate name="_"/> <subordinate name="_" label=" , "/> </row> <row> <subordinate name=""/> </row> <row> <subordinate name="_"/> <subordinate name="__"/> </row> <row> <subordinate name="_"/> <subordinate name="__"/> </row> </table> <query title=" " ofType=""> <parameter subordinate="entries.clazz" operator="in" value="it"/> </query> </presentation> </entity> <entity name="" parent="Wares"> <attribute name="" type="float" specialize="Wares.amount" optional="false"/> <relation name="" kind="reference" to="" specialize="Wares.clazz" optional="false"/> </entity> <entity name="" parent="Warehouse"> <attribute name="" type="string" optional="false" specialize="__label"/> <attribute name="" type="address"/> <relation name="" kind="embedding" list="true" to="" specialize="Warehouse.entries" readonly="true"/> <presentation> <l10n> = </l10n> </presentation> </entity> <action name="it.approved ? '': ''" class="com.meta4.cms.warehouse.ApproveActionLogic" accept="" logMessage=" "/> <entity name="" parent="Transfer" abstract="true"> <attribute name="" type="" expressionClass="com.meta4.cms.warehouse.OperationTypeExpression"/> <attribute name="" type="boolean" specialize="Transfer.approved" readonly="true"/> <attribute name=" " type="date" readonly="true"/> <relation name="" kind="reference" to="" optional="false"/> <relation name="" kind="reference" to="Warehouse" specialize="Transfer.from"/> <relation name="" kind="reference" to="Warehouse" specialize="Transfer.to"/> </entity> <entity name="" parent=""> <attribute name="" type="boolean" specialize="."/> <relation name="" kind="reference" to="Warehouse" specialize="." optional="false"/> <relation name="" kind="reference" to="Warehouse" specialize="." optional="false"/> <relation name="" list="true" kind="embedding" to="" specialize="Transfer.entries"/> <relation pname="№ " name="" kind="reference" to="" oppositeTo="." readonly="true"/> <relation name="" kind="embedding" list="true"> <to> <entity name=""> <presentation> <l10n>=</l10n> </presentation> <attribute name="" type="integer" default="0"/> <attribute name=" " type="integer" default="7"/> <attribute name=" " type="integer" default="0"/> <attribute name="" type="integer" value="it. * it.__ + it._"/> <relation name="" kind="reference" to="" optional="false"/> </entity> </to> </relation> <presentation tableColumns=", _, , , , "> <l10n>=</l10n> <table> <row> <subordinate name=""/> <subordinate name=""/> </row> <row> <subordinate name=""/> <subordinate name="_"/> </row> <row> <subordinate name=""/> <subordinate name=""/> </row> </table> <single subordinate=""/> <single subordinate=""/> </presentation> </entity> <entity name="" parent=""> <attribute name="" type="boolean" specialize="."/> <attribute name="" type="string" optional="false"/> <relation name="" pname="№ " kind="reference" to="" oppositeTo="." readonly="true"/> <relation name="" list="true" kind="embedding" to="" specialize="Transfer.entries"/> <relation name="" kind="reference" to="Warehouse" specialize="." optional="false"/> <presentation tableColumns=", _, , , , "> <l10n>=</l10n> <table> <row> <subordinate name=""/> <subordinate name=""/> </row> <row> <subordinate name=""/> <subordinate name="_"/> </row> <row> <subordinate name=""/> <subordinate name=""/> </row> </table> <single subordinate=""/> </presentation> </entity> <entity name="" parent=""> <attribute name=" " type="date" optional="false" default="now"/> <attribute name=" " type="date"/> <attribute name="" type="string"/> <!-- --> <attribute name=" " type="float" value="sum(._)"/> <relation name="" kind="reference" to="" specialize="." optional="false"/> <relation name="" list="true" kind="embedding" specialize="Transfer.entries"> <to> <entity name="" parent=""> <relation name="" kind="reference" to="" specialize="Wares.clazz" optional="false"/> <attribute name="" type="float"/> <attribute name="" type="float" specialize="Wares.amount" optional="false" default="1"/> <attribute name=" " type="float" value="it. * it."/> </entity> </to> </relation> <presentation tableColumns=", _, , _, _, _"> <l10n> = = </l10n> <table> <row> <subordinate name="_"/> <subordinate name=""/> </row> <row> <subordinate name="_"/> <subordinate name="_"/> </row> <row> <subordinate name=""/> <subordinate name=""/> </row> <row> <subordinate name="_"/> </row> </table> <single subordinate=""/> </presentation> </entity> <entity name=""> <attribute name=" " type="datetime"/> <attribute name="" type="" optional="false" default=""/> <attribute name="__label" type="string" template="$_" specialize="__label"/> <attribute name=" " type="string" optional="false"/> <attribute name="" type="string"/> <attribute name="" type="string"/> <attribute name="" type="" optional="false" default=" "/> <attribute name=" " type="date" default="now"/> <relation name="" kind="reference" to="" optional="false"/> <relation name="" kind="reference" to=""/> <relation name="" kind="composition" to="" list="true"> <constructor name=' ' class="com.meta4.cms.warehouse.CreateTransfer" primary="true"> <attribute pname=" " name="auto_add" type="boolean" default="true"/> </constructor> </relation> <relation name="" kind="composition" to="" list="true"> <constructor name=' ' class="com.meta4.cms.warehouse.CreateRetirement" primary="true"> <attribute pname=" " name="auto_add" type="boolean" default="true"/> </constructor> </relation> <relation name="" kind="embedding" list="true"> <to> <entity name=""> <attribute name="" type="float" default="0" optional="false"/> <attribute name="" type="float" readonly="true" default="0"/> <attribute name="" type="float" readonly="true" default="0"/> <attribute name="" type="float" readonly="true" default="0"/> <relation name="" kind="reference" to="" optional="false"/> </entity> </to> </relation> <constructor name=' ' class="com.meta4.cms.warehouse.RequestConstructorLogic" primary="true"> <attribute name="" type="" list="true" optional="false" defaultExpression="user."/> <relation name="" kind="reference" to="" optional="false"/> </constructor> <presentation tableColumns="_, , _, , , "> <l10n> = = </l10n> </presentation> </entity> </projectDescription> We created the system to support the SaaS mode, so that one server can have many “projects”, each of which has its own classes of operated objects. That is, one group of users may have CRM, the second has warehouse accounting, and the third also has warehouse accounting, but specialized for the restaurant business, and all this on one server.
As a result of our work, we have formed 3 “artifacts”:
- Information system description language
- Structure description
- View description
- Trigger description
- Server: Java + Jetty + MongoDB. MongoDB is used because of the lack of schema that made it easier for us to store data with different structure in one collection. However, due to significant shortcomings (lack of transactions and JOINs, slow aggregation), we plan to switch to RDBMS when we become “big and adult”
- Client: Javascript + MooTools. The view is formed entirely on the client, accessing data through the server REST API. The client repeats the implementation of working with language models, but already in a simplified form, especially for the formation of the presentation, taking into account the fact that the client is “thin”. Unlike traditional "textual" templating, we use templating at the level of the DOM tree (this approach is very uncommon, and I will write about it if interested)
Of course, we are far from the full implementation of our ideas. For example, such a concept as “accumulation register” is not implemented, now it is manually emulated by triggers. Nevertheless, this year we have formed a vision of the language and architecture of the system implementing it.
Source: https://habr.com/ru/post/150021/
All Articles