Integration of the external object system in Delphi on the example of IBM SOM

4 years ago my article about IBM SOM came out , where I stated an extremely deplorable situation, when significant tools were lost, and the further, the less chances to recover. Since that time, a lot of things happened, there were SOM 3.0 for Windows, SOM 2.1, an open clone somFree, a working compiler DirectToSOM C ++ for Windows, and a bridge in OLE Automation.

One of my projects is implementing SOM support in Delphi. Development began on Delphi, I had to do some of the bindings manually and not so beautifully, in a procedural style, without type checking. Using these bindings, the bindings generator was written in object style, and then the generator itself was rewritten to new bindings, becoming a confirmation of their performance. For the sake of beauty, you had to hack the Delphi object system, and maybe you would be wondering how this can be done at all.

Ugly ("thin") bindings at least somehow allow to interact with the library, and beautiful ("thick") tend to make it so that it is natural from the point of view of ordinary code. IBM SOM (System Object Model) - about objects, and for objects methods are called through a dot. I am aware of the following entities in Delphi that can call object-style methods:

• Objects
• Objects from the old Borland Pascal with Objects object system
• Classes
• Interfaces
• Dispinterfaces
• Options
• Starting with Delphi 2006, the records
• Starting with Delphi XE3, anything you can see the assistant

I usually develop on Delphi or Ada, with a preference for the second, and my interest in the technology of interfacing the heterogeneous components began with the fact that I tried to make friends of these two languages ​​in different ways. In the question of developing bindings, the beginning of SOM support with Delphi, and not Ada, is due to the fact that it is harder to do it in Delphi. And inside Delphi the most difficult thing is to make the type block, and it was from there that the development of the import tool began. To defeat type means to solve basic problems.
')
In principle, it is good that there is modularity in Delphi. This is better than a strife in C ++ or, especially, C, used by emitters supplied with SOM DTK. And where they lack the usual annoyance, they get tons of macros. But at the same time, in a Delphi-like language Ada, “private with”, “type X;”, “limited with” appeared, allowing to connect limited information about types and thus have both modularity and the freedom to make cyclical connections between modules, and even more do cycles within the same package, and in Delphi development in this direction was moving poorly. Worse, the main commercial compiler (at best, one order for FPC versus 6 for Delphi, and Hell has to be pushed, it’s not found itself, you don’t feel like throwing it somewhere for Ada ), so the old versions are used, the Delphi is still relevant. 7, so if I want to write a library on my beloved Ada and put it in order to use it from Delphi, it is desirable that it could be Delphi 7, so the last 2 options are discarded. The variants have no type control, the method list does not appear in the tooltip, this option is also overlooked.

Initially, I wanted to make RAII so that the memory is managed automatically, as with strings and interfaces, and for this you need to wrap the interface or option in the Delphi 2006 record or the old object . This approach has a significant drawback. By the rules of Delphi forward, you can only declare classes, interfaces and pointers, moreover, in the indivisible type block. And now, let's say, we bind to the Container class and the Contained class, and they are interrelated. It would be nice to write Container = record private FSomething: Variant; end; and similarly for Contained, and then clarify what methods they have, because methods can receive references to the SOM classes projected into the records, and if the methods can only be written inside the record, one of the records will not be yet declared.

This means that the entire record cannot be declared due to the arguments of the method. Partly, the situation can be saved if you first make for each class a SOM object from the old system of objects with a hidden field, but without methods, and then inherit it again, and in each method you will receive an uninspired version without methods to the input, to which the inherited with by methods. But the result will be a problem; there, on the contrary, due to the limitations of the Delphi method, the SOM class, the bindings for which are created before the other, cannot return the inherited version overgrown with the methods of the returned SOM class.

Container and Contained are in somewhat more complicated relationships, they do not return each other, but the sequence (the Korbovsky analogue of the dynamic array) from each other, which means that the sequence would need to be projected for objects that were not inherited, not overgrown with methods. It was only in Delphi XE3 that a slightly less complex way of separating the declaration of the structure and methods appeared. First, we wrap the interface or option in the private part of the record, and so on for each projected class, and then we hang the methods with the help of helpers. And these methods of assistants can safely take everything they need for entry and exit.

Understanding the memory management in the SOM, I did a bit of a RAII. The fact is that in the IBM version of SOM there is no reference counter for all objects, as it is in COM and modern Objective-C, and what can you do if the link to the object is copied? The Apple SOM, by the way, was , and from there it went into somFree, so it's not hopeless, but I also have the DirectToSOM C ++ compiler and bridge in OLE Automation, with which I would like my solution to be compatible at the moment, and they are not designed for this mode of operation.

With interfaces and regular wrapper classes, problems arise that are similar to “and what would you do if the link to the object is copied”, only in the case of destruction. After all, a wrapper can transitively destroy an SOM object, or maybe not. This is at least for the wrappers to do the flag of ownership. And for complete happiness, and even get lost in it all. That would be a reference counter, we would always pull it and not reflect, and not confused. Everything would work like a clock. It would be nice to live.

If you touch the old object system, warnings begin to pour. That's how I came to the decision to hack the object system Delphi. My generator projects SOM classes into Delphi classes with conventional methods, and all this is used in a manner customary to the Delphi developer. For classes, a deferred declaration is done remarkably; then you do not need to wrap them up in anything. Since all cycles need to be closed in a single type block, all CORBA modules and all types nested in classes have to be projected into one unit Delphi, so that this unit has a single type block.

It started with the fact that I decided to force Delphi to consider SOM objects as Delphi objects. As long as Delphi does not touch VMT, everything is fine. Each class SOM method is projected onto the class Delphi method. In this case, SOM methods are usually virtual, and are projected onto non-virtual Delphi methods that know how to send a call to the SOM. Non-virtual methods are invoked without touching VMT, and Delphi has no idea that far from there Delphi objects are processed.


Methods in the SOM is easy to call. Here you see 8 instructions (homework - try to understand what they are doing), two of them are enough for the actual call. Before the last call, mov / push is done twice, this is passing the arguments, not the call. Before that, write the address to var_14, and then it is not necessary to call it, you could write the address to edx in the first instruction and at the end make a call to [edx + 1Ch]. Another minus 2, a total of 2 instructions for calling the object's SOM method are similar to the 2 instructions for calling virtual methods from VMT in other development systems. At the received address is a dynamically generated code fragment that knows how best to call the specified method, and it will give additional “hidden” instructions, but what a difference it is! You can read about this difference in the translation of the “Release-to-Release Binary Compatibility” report . If you ever wanted to understand why under each version of Delphi your dcu and bpl sets, now you know.

Let's return to the generation of bindings. Inheritance in SOM is multiple, and it is actively used. For example, when developing a generator, I work with OperationDef (meta information about a method), and it is simultaneously Contained inside the class and Container of its arguments. And in Delphi classes - single. Theoretically, it is possible to do a single inheritance for the first parents in the projection on Delphi, and then add methods to the non-first, as if they reappeared. It seemed to me an ugly, asymmetrical solution. After all, OperationDef (as well as ModuleDef, InterfaceDef) is equally as Container and Contained, and I won’t even remember at once what parent classes are declared in their order. Plus, if you let the hierarchy of Delphi classes grow, then other problems arise, about this - in the paragraph after the next one. So I projected the SOM classes so that they are not parents to each other in Delphi. They all come from the utility class SOMObjectBase, which is needed to hide the TObject methods, and the methods in them are filled from scratch every time. The operations “as” and “is” are, of course, not supported, because they will take the VMT of the SOM object and will think that this is the VMT of the Delphi object. Unfortunately, they cannot be blocked, but for everything to be somehow typed, for each parent class, the As_ ClassN functions are generated to bring the type up and the Supports class functions to bring the type down.

Attentive readers should have strained the phrase “class function”, because before it was written “as long as Delphi does not touch VMT, everything is fine.” Hid class TObject methods and set your own? After all, this is how you can select Supports from a drop-down list from a regular object, and Delphi will climb into VMT. It turns out that it can be elegantly processed. New class methods are also non-virtual, and all that Delphi does to call them on an object is to take a pointer over the object's zero offset, and this becomes Self in the class method. And if you call a class method on a class by contacting it by name, then Self is a VMT Delphi class. And how to distinguish VMT SOM class from VMT Delphi class?

But there is one way. Each class method is re-added each time to the next Delphi class, from which normally no one is inherited by Delphi means, and the projected classes themselves are not inherited by Delphi means. Thus, we can have only one possible variant of VMT Delphi class in Self - this is VMT of that Delphi class itself, otherwise the method was called by the SOM object, and Self is actually the VMT SOM. The SOM VMT device is unknown, it depends on the version of SOM.DLL and may change, but it is known that there is a reference to the object class by zero offset, and we need it. In SOM, all classes are also objects, and class methods are methods of object classes in the most ordinary sense. Thus, by comparing Self with your name, you can further select, either get a reference to the SOM class by zero offset in the ClassData structure, or, if not matched, take the reference to the SOM class, dereference Self. And then do what the class method implies. Actually, there are two class methods that make such a comparison, this is ClassObject and NewClass, the second is different in that if nothing is found in the ClassData structure, the class is not automatically created. The remaining class methods call one of these two. For example, if we need to check whether such an object is a successor of such a class, then if a class is not created, then it is not necessary, and so it is clear that there is not, and if they ask for InstanceSize, then without creating a class, it is not enough. Thus, both “o.InstanceSize” (for “var o: SOMObject”) and “SOMObject.InstanceSize” will work correctly. Just like in Delphi.

There was an idea to project all the methods of the SOM object class into the Delphi class methods, but there were difficulties. Overcome, but it was decided to abandon them to overcome.


Create, inspired by how TLIBIMP.exe works, I made a class function. It turns out that, like in Delphi, we write “repo: = Repository.Create;” But an idea arose, what if the SOM constructors (initializers in SOM terminology) were made constructors from the point of view of Delphi. So that they, being called by the class, create an object, and for the object, work as methods. To show how you can hack Delphi classes, I decided to give a temporary diagram of how objects in Delphi are constructed and destroyed:

Outer-Create Outer-Create => virtual NewInstance Outer-Create => virtual NewInstance => _GetMem Outer-Create => virtual NewInstance Outer-Create => virtual NewInstance => non-virtual InitInstance Outer-Create => virtual NewInstance => non-virtual InitInstance => FillChar(0) Outer-Create => virtual NewInstance => non-virtual InitInstance Outer-Create => virtual NewInstance Outer-Create Outer-Create => Create Outer-Create Outer-Create => virtual AfterConstruction Outer-Create Free Free => Outer-Destroy Free => Outer-Destroy => virtual BeforeDestruction Free => Outer-Destroy Free => Outer-Destroy => Destroy Free => Outer-Destroy Free => Outer-Destroy => virtual FreeInstance Free => Outer-Destroy => virtual FreeInstance => non-virtual CleanupInstance Free => Outer-Destroy => virtual FreeInstance => non-virtual CleanupInstance => _FinalizeRecord Free => Outer-Destroy => virtual FreeInstance => non-virtual CleanupInstance Free => Outer-Destroy => virtual FreeInstance Free => Outer-Destroy => virtual FreeInstance => _FreeMem Free => Outer-Destroy => virtual FreeInstance Free => Outer-Destroy Free 

Outer-Create and Outer-Destroy is the code into which the constructor and destructor calls are automatically wrapped.

As for the SOM, if you need to call a non-standard constructor (not somInit), then the somNewNoInit function is called instead of somNew in the class object, which returns the object, and then it calls the constructor you need, for example, somDefaultCopyInit. Or all the same somInit. The idea is to somehow hack all the TObject methods so that the object creation sequence is recreated in Delphi reals. In particular, we see that TObject.NewInstance is a virtual class function. With tricks with names, it is impossible to deceive, the Delphi compiler calls it from VMT to a specific address. But in SOMObjectBase, where the TObject and NewInstance methods are hidden, not only hide, but also provide a meaningful implementation that will cause somNewNoInit on the corresponding SOM class. Where will she take this class? For example, it is possible through Delphi VMT to stretch a protected virtual class function that will be able to return the corresponding SOM class. Only there is one problem. At the end of the Outer-Create, AfterConstruction is called, the virtual method. It will not work if the object already has a SOM VMT. You can, of course, at the end of Create, temporarily overwrite a VMT object from SOM to Delphi, and back to AfterConstruction, but this is some kind of too acidic scheme. So in this matter had to retreat.

But the rest turned out pretty natural binding.

Inheritance from Delphi is not implemented, but if it does, then it will be nice to make it a little difficult there. Even if we consider the usual emitters for C ++, then their work with SOM objects is similar to the work with C ++ objects, operator new () and operator new (void *) are overloaded, but when inheriting, the implementation of methods of SOM classes does not look like the implementation of methods C ++ class. In addition to the specially modified DirectToSOM C ++ compiler, of course.

This activity is carried out within the framework of an inventive project and at the moment has a research and demonstration character. I need to learn from A to Z pitfalls, others need to show the fundamental feasibility. Maybe it will be useful somewhere, but it’s planned to work with another, a new model that incorporates the best features of SOM, COM and Objective-C, and will be ready to work on current tasks that were not before the previous SOM authors.