Hi Habr, I want to talk about the creation, which spent his youth
, it would be better to drink and smoke .
Msvcore is a cross-platform library for c ++, written from scratch according to the principles of optimality, openness and simplicity. At least, it was laid as a basic idea. What happened in the end ...
A bit of history
It all started back in 2004, when I started working as something like a sysadmin of all trades, but at the same time I started to get involved in c ++. And, as I remember, MFC with its templates and the string CString. Then the idea arose to write your own lines, simple and clear. And away we go.
Unfortunately, I only have the archive from October 2005, I will restore the events according to it. You can look at it on a
githaba . The earliest date in the archive dates back to October 10, 2004, for lack of another, this day can be considered the birthday of the library (Date: Sun, 10 Oct 2004 12:50:42 GMT).
')
Interestingly, the library, unlike the others, was created by evolution. Frequently used code was added to it, and it was added to the current tasks. The tasks were complicated, the library grew. But in this there is a minus, you may not find the functions that it would seem necessary to be in each library. So she and the bike, which is appended during the development process.
The first component in my library was the MString string class, the full name is MyString.
class MString{ char * data;
Binary strings, with one buffer, ending with an additional zero byte, which was required for most standard functions: open, strlen (), ...
After the first big program, with static arrays, illogical logic and other beginner programmer jambs, dynamic arrays were required. The common name of the first dynamic arrays is MMatrix. The basic principle: A parent element with pointers to the first and last elements of the array and, the counter of elements. The elements of the array had pointers to the previous, next elements and data. For each data variant, a copy of the class was made. Templates? No, not heard. Later, classes of arrays were developed almost once a year.
It was also one of the first to create the MSVCF class for working with configuration files.
They wrote their analogs of the standard functions: itos () and stoi (), itos64 () and stoi64 (), ftos () and stof () for translating a number into a string and back for int, int64, float. itob () and btoi () too, but for binary strings. stos () converter short to char. explode () to split the string into parts. rts () (read to sumbol) and company, to search for a character / string in a string. The ILink class was created, which was required for parsing paths and references to parts and is still in use. At some point, the IHeader class was required to work with http headers. The MSVThreads feature set for creating new threads is still relevant. Created MTime class to work with time.
These were the beginnings of the library, which still has a lot to go. The volume of text in the library 115kb.
Basic principles
The library was sharpened for optimality, clarity, consistency, and many other good words. In fact, it is still full of scary code written in one line. Only last year, the code began to be written with the indentation between the characters.
To use the library, you need to include in the project two files MString.cpp and VString.cpp. Attempts to connect the library in the form of a plug-in failed because it was difficult to constantly rebuild two projects instead of one. Also, in the plugin library, you cannot apply changes on the fly. Initially, the library was included in the project itself, but over time, the assembly began to take a long time, so it was divided into parts so that when a change was made, only part of the project was required to be rebuilt. Library-based projects usually consist of three main files: ProjectName.cpp, MString.cpp, VString.cpp. The program code is written in ProjectName.cpp, the following two are connected from the library. It often does not use the standard separation of code into cpp and h files, to speed up the writing of code. The library and programs were sometimes written day and night, and the extra delays were no good.
A word about cross-platform. After exploring Linux in 2006, the library was finished for assembly in gcc. Then under WinCE, Android (Jni) and even Flash (Crossbridge). It is worth noting that all programs are initially written in Windows and MSVS, and only then are transferred to the final platform for adding a platform-specific part and testing. That saves a lot of time and effort.
I basically did not use libraries, except for those that are meaningless and difficult to rewrite: zlib, openssl, pcre, mysql.
Strings
The concept of lines lined up for quite some time. Options using the reference counter, when several variables point to a single memory block, did not work optimally enough, they needed to use locks and more complex logic, and at that time I placed serious emphasis on economy and simplicity.
At some point, virtual strings of VString appeared, first as a separate class, and later, solving the problem of converting strings between classes as a base one.
As a result, today the picture is as follows:
Class
VString (Virtual String) - virtual string, contains a pointer to the data and their size. The class does not work with the allocation / release of memory. Allows you to quickly perform operations with strings that do not require data changes: getting part of a string, searching by string. The class replaces the frequently used pointer to char * strings ending in zero. I consider the last variant of working with strings not optimal and simply dangerous, incorrect data in the string should not cause errors. The main problem in VString is to keep track of the relevance of the data pointed to by the variable of this class.
class VString{ public: unsigned char *data; unsigned int sz; functions... };
The
MString (My String) class is standard strings with memory for them. Memory is reassigned each time a line is resized, which makes the use of these lines extremely slow. These strings should be used where other string options are not appropriate. Used as variables in classes. Here the main problem is the need to use locks when accessing from multiple threads.
class MString: public VString{ functions... };
The
SString (Stack String) class is a string on the stack. The same strings, but with memory allocation on the stack, allocate 1kb by default, MString is used for larger strings. Huge speed, but a large excess amount. Used as temporary variables. Born in pursuit of reducing memory allocation / freeing operations.
class SStringX : public VString{ unsigned char sdata[stacksize]; MString mdata; functions... };
Class
HLString (Line String) - stores strings in a chain of memory blocks. By default, memory blocks of 4kb or by size of data are allocated. Allocation of memory in advance, speeds up work when adding data. The class overloads the + operator and allows you to write code in the form: MString text (HLString () + “text” + a + 111); The same class allocates memory on the stack, for the first memory block, the default is 4kb. Use cases: the addition of multiple lines, numbers in one line. It is also often used feint ears for temporary storage of strings. HLString ls; VString s = ls.addnfr (“Text”); - Adds an unfragmented string. Here there are advantages in allocating / freeing large blocks of memory, which is much faster than using MString for the same number of strings.
TString class (Temp String, Thread String) - Temporary or stream strings. An idea that came just a year ago, and she had to hurry about five years. In principle, this idea is built on HLString, unfragmented strings and __thread variables. Each stream has its own instance of the HLString variable, from which interesting perspectives flow. TString allocates memory in a HLString bound to a thread, which is definitely faster than allocating memory through malloc () / free (). The problem with this class is that it does not free the memory until all the TString variables are destroyed. At certain moments of the program, all variables must be destroyed, otherwise the program gradually uses all the available memory with corresponding consequences.
These are the five types of strings that I use when writing programs. VString - for working with strings and fragments of strings, MString - for storing data in classes, SString - for collecting strings from substrings on the stack, HLString - for collecting large strings on the fly, TString - for temporary strings.
Arrays
The first program, built using static arrays, showed that something had to be done about it. Since then, the idea of ​​writing arrays came almost every year.
MMatrix (My Matrix) - the first attempts to work with pointers, constant drops and an endless search for errors. Consisted of a parent element with pointers to the first and last elements of the array, and, in fact, the elements of the array, with pointers to the previous and next elements, as well as data. Reproduced by simply copying the class and adding the necessary functions. Templates are not our method. We also optimized for the task: let's remove the pointer to the previous element and save as many as four bytes.
LMatrix (Live Matrix) ~ 2007 - forever alive, like grandfather Lenin. You can take a look at the code, but even I don’t want to dig into it and remember how it worked.
UMatrix (Unlimited Matrix) ~ 2008. - a dynamic array of a chain of memory blocks with storage of several elements of the array in one block. Able to combine all elements into one memory block. Here the idea was implemented to allocate memory immediately under the block of elements, reducing the work with memory functions. A bitmask is used to determine free / busy items. These ideas will be used in the following arrays. Templates are still not our method, but it is quite difficult to copy with our hands, so the code generator was written.
IMatrix (Ideal Matrix) ~ 2009 - vector, the entire array in one memory block, with a shortage of space, is moved to a larger memory block. Subsequently, it turned out to be of little use and practically not used.
OMatrix (Object Matrix) ~ 2010 - a general idea repeats UMatrix, but if the idea of ​​the first is a chain of objects, then here is the idea of ​​separateness. Here, in contrast to UMatrix, the list of free objects and the passage through them is implemented. This class is used as a memory allocator, allowing you to quickly get / free memory for variables.
The matrices ended, the sheets began. And along with them, code generators were abandoned and templates were accepted for use.
MList (My List) ~ 2013 - The class automatically wraps the custom class in its own, adding pointers to the previous and next elements.
IList (Ideal List) ~ 2014 - IMatrix with the replacement of the code generator on the templates, all the same vector.
OList (Object List) ~ 2015 - Similarly, OMatrix rewritten for templates.
UList (Unlimited List) ~ 2015 - Replacing UMatrix templates, beautiful and logical code.
AList (Auto List) 2015 - Dynamic array with a set of memory allocators, from standard new / free, UList, HLString, OList to the ability to write your own.
TrieList 2016 - Implement a trie tree for quick searching, based on AList.
Unfortunately, arrays gradually become overgrown with templates, on the one hand, it simplifies development, on the other hand, it complicates understanding and changing for its tasks.
Library use
The simplest example is to create an empty project and add a cpp file with the contents:
#define USEMSV_GENERALCPP #define PROJECTNAME "projectname" #define PROJECTVER PROJECTNAME ## _versions #include "../../opensource/msvcore/msvcore.cpp" Versions PROJECTVER[]={
Add the files ".. \ .. \ opensource \ msvcore \ VString.cpp" and ".. \ .. \ opensource \ msvcore \ MString.cpp" to the project and write the code.
When adding multiple cpp files, use the same code up to #include ... msvcore.cpp inclusively, removing the #define USEMSV_GENERALCPP line. You can also connect additional library extensions by specifying them before connecting to msvcore.cpp, for example #define USEMSV_OLIST allows you to use OList arrays. The list of extensions can be seen in msvcore.cpp
By default in the library:
- Platform-specific files are connected, various defines are defined.
- String classes are available: VString, MString, TString, SString (), HLString.
- The following feature sets are available: rts () - search by string, PartLine () - analog for working with VString.
- The ILink class is for working with paths and links, the MTime class is for working with time, the ConfLine class is for working with configuration files, the Logs class is for logging, the MSVEF class is an analogue of regular expressions.
- The UList class is dynamic arrays, the MMatrix class is the first dynamic arrays, a set of MTE classes of universal classes for storing various data types.
- MRect class for working with regions, MRGB class for working with colors.
- The mbool class is for working with bitmasks, functions for converting strings to numbers, encodings, various formats and vice versa.
- Library sets for wince, flash.
- A set of cross-platform functions for working with files: CreateFile (), ReadFile (), WriteFile (), GetFilePointer () ... GetFileInfo (), CloseHandle (), MkDir (), LoadFile (), SaveFile ().
- The Readdir class to get a list of files in a folder, uses MSVEF as a filter.
- The console output function is print (VString string, VString string1 = VString (), ...).
- Functions for creating threads StartThread ().
- Blocking classes: TLock - critical sections, CLock - condition variables / conditional variables, UGLock - automatic locking / unlocking TLock.
- Buffer classes: SendDataL - linear buffer, SendDataRing - circular buffer, RecvData - buffer for receiving data.
- Networking classes: ConIp - to establish a connection and open a port. Functions: GetIP () - getting ip by domain name, ifrecv () - checking data availability in a socket, ifsend () - checking permission to write to a socket, gettip () and getcip () - returns ip and port, server and client.
Add-ons are specified before enabling msvcore.cpp:
#define USEMSV_ITOS
ITos class, previous version of SString. Outdated
#define USEMSV_INTERCEPT
A set of functions for analyzing machine codes and intercepting functions.
#define USEMSV_CPPXCC
The XCC class is a C ++ code parser.
#define USEMSV_INTERCEPT_MALLOC
Code to intercept system functions working with memory, used to search for memory leaks.
#define USEMSV_XDATACONT
Classes for working with data formats. The XDataCont class parses JSON and XML, the rest of the classes are obsolete.
#define USEMSV_CJX
Class CjxCont with binary data format implementation, binary json.
#define USEMSV_MLIST, USEMSV_ILIST, USEMSV_OLIST, USEMSV_ALIST, USEMSV_TRIELIST
Enables the corresponding classes of dynamic arrays: MList, IList, OList, AList, TrieList.
#define USEMSV_AMF
Classes amfe and amfd to convert / parse AMF format.
#define USEMSV_PCRE
Enables libraries with pcre2 regular expression functions.
#define USEMSV_CONSOLE
Classes PipeLine and PipeLine2 to run other processes.
#define USEMSV_MWND
Almost a separate library for working with windows, graphics, images. Allows you to work with graphics on all supported library platforms. Contains primitive drawing functions that are platform independent. About her worth telling separately. Uses the CxImage library to encode / decode image formats.
#define USEMSV_CONSOLELINE
A set of classes for working with the console.
#define USEMSV_OPENSSL
Functions and classes for working with openssl and encryption. Class MySSL for setting / receiving ssl connections and working with them. Functions: RsaCreateKeys () - creates two Rsa keys, RsaPublicEncode (), RsaPublicDecode (), RSAPrivateEncode (), RsaPrivateDecode () - encrypts / decrypts with an open / private key, AesEncode () and AesDecode () codes / decodes encrypts of algorithms with a public / private key, AesEncode () and AesDecode () functions encode / decodes algorithms (A / D), encrypts / decrypts algorithms with a public / private key. Also contains functions for working with certificates.
#define USEMSV_WEBSOCKETS
The set of functions for working with WebSockets and the WebSocketsCli class are the implementation of WebSockets client.
#define USEMSV_MYSQL
The MySQLCon class is a wrapper for working with MySQL, it uses mysql-connector-c.
#define USEMSV_MSL_FL
Connects the MSL interpreter - my version of a programming language that most resembles php. Msl Fast Line - version 4 msl. Without pseudo code generation, executes text commands. It was written somewhere in a week in October 2013. You should also talk about it separately.
#define USEMSV_MSL_FV
Msl Five is the fifth version of the language. With the generation of code bytes and other buns. Developed in September 2015.
#define USEMSV_HTTP
Classes and functions for working with http requests. Classes GetHttp and GetHttp2. When connecting openssl functions, it supports https requests. The IHeader class for working with http headers. MCookie class for working with cookies.
#define USEMSV_CONFLINE
The ConfLineOptions class for working with configuration files.
#define USEMSV_NESTAPI, USEMSV_NESTAPI2
Classes and functions for the server side of my NestApi protocol.
#define USEMSV_STORMSERVER
Server platform, about which I would like to write a separate post. For about ten years, I tried to write good servers and finally succeeded. Almost perfect solution.
#define USEMSV_LIGHTSERVER
The LightServer class is a simple and easy server under which you can write your handler. The LightServerHttp class is a simple https server that returns a test page.
#define USEMSV_TGBOTS
Classes TgBot and TgBots implementation of bots for telegram.
#define USEMSV_ANDROID
Functions and classes simplify the work when compiling for android.
#define USEMSV_TRAFFIX
Classes for listening to traffic.
#define USEMSV_BUFECHO
Options for changing console options.
Conclusion
I hope you are interested in this bike named after me and I will have something to tell you more. In my opinion, there are quite interesting things here, starting with strings, arrays, wrappers for openssl are written, msl may be interested, or a class for creating bots in a telegram, and finally, stormserver is a platform for creating various servers. I will write a separate article about the latter, about server development, from a simple echo server to complex http and proxy.
As I already said, the library is quite biased towards the programs being developed. As practice shows, in itself it is difficult to develop without specific tasks.
I wrote this library in order to learn how to program well and understand what the code does from beginning to end. And, I must say, there is progress, but still work and work.
In the future plans once again to rewrite from scratch, finally get rid of the old code, put in order. And maybe finally write the documentation. Also in the plans to add the implementation of new variants of strings, as a replacement for the current, and new dynamic arrays.
→
Library code and several projects