We are friends of Total Commander (64 bits) and plugins (32 bits)
I will tell you about a small project that allows you to use 32 bit versions of extensions in the 64 bit version of Total Commander (hereinafter referred to as TC). The project is at the demo stage and allows you to use WCX modules with the minimum necessary set of functions (viewing and extracting the contents of archives, or all that can be represented as archives). Well, at the end of the survey on the relevance of such a solution and bring the project to a certain level, covering the entire possible part of the API and all possible categories of modules.
The modules for TC are DLL files with extensions WCX, WFX, WLX, WDX and containing a specific set of exported functions (according to the category of module). Everything would be fine, but not all authors took care of 64 bit versions. And the source code is not available, as a rule ...
Question - Can I use existing 32 bit versions?
Answer - Yes, but not everything is so simple.
If you summarize before loading a dynamic 32-bit library into a 64-bit process, it turns out that the task is not new and the search for a solution on the Internet will not keep you waiting. It all comes down to creating a surrogate process capable of loading the library and interacting with this process through IPC (inter-process communication). We do not have access to the source code of the TC and we cannot add the mechanism for working with the surrogate process. But we can create a library. The library will impersonate the module and communicate with the surrogate process, which in turn will pull the module's functions and return the result. And ironing it all will be like this:
')
IPC options are available on MSDN - Interprocess Communications . For my project, I chose Pipes. This may not be the fastest way, but it allows you to implicitly monitor the health of the surrogate process. If the surrogate process falls, then the pipe pipe is broken and our library will know about it. Further description of the processes taking place.
When connecting the library
To generate a unique name, we use the UuidCreate () function. It will generate a UUID (GUID). Convert it to a string (UuidToString) and fill the path for the pipe. Create a pipe (CreateNamedPipe) working in blocking mode and sending messages. Run the surrogate process (CreateProcess). The name of the pipe is passed as a command line parameter. And we will wait for the client (ConnectNamedPipe).
When disconnecting the library
Disable the client (DisconnectNamedPipe), complete the surrogate process (TerminateProcess), close the pipe and clean the resources (CloseHandle)
When starting the surrogate process
Connect to the pipe-in (CreateFile) and configure it to work in blocking mode and transfer messages. Load the module (LoadLibrary) and save the addresses of the exported functions (GetProcAddress). Let's go into the message waiting loop. If necessary, complete the process out of the loop.
At the completion of the surrogate process
Disconnect from the pipe (CloseHandle) and unload the module (FreeLibrary).
When calling a function from the library
The function call is considered on the example
OpenArchive is the first function that TC calls after loading a module. It is passed a pointer structure of type tOpenArchiveData.
We can not pass a pointer to the structure, the processes are isolated and do not see the memory of each other. We also cannot transfer the structure simply by copying it into the message, because of the pointer to the string (ArcName) and the alignment of the fields. Plus, some fields are designed to transfer data to a function (ArcName, OpenMode), and some serve as a buffer for returning a result (OpenResult), the latter are not used at all (Cmt *). We must marshal, i.e. convert the data into a format suitable for transmission. For this are a number of written SET_ * macros. SET_INT writes int as a 32 bit number to the buffer. SET_STR_A writes to the buffer a sign of the validity of a pointer to a string, and in the case of validity, writes the size of the string with terminal zero and the array of characters pointed to by the pointer. Two values are placed at the beginning of the buffer: what is this function and what is it - a request. Next, you need to calculate the size of the data recorded in the buffer and write them to the pipe. Wait for a response from the other side. When you receive a response, read two values: what is this function and what is a response or a request to execute a feedback function. If this is the answer, we get the result, write the part into the structure and exit the function. If this is a request to call the feedback function, we get the parameters for it, execute it, return the result and wait for the next answer (all this is hidden in the CALL_PROC macro). We should also mention the type of result of the considered function. This is HANDLE, but in reality is a pointer. You will need it as a parameter for calling the remaining functions by the TC itself. But its importance plays a role only within the module. In 32-bit processes, the pointer is 32-bit, in 64, respectively, 64-bit. And it is created in a 32 bit process. Therefore, converting it to 64 and then to 32 will not lead to data loss.
Two functions (SetChangeVolProc, SetProcessDataProc) register feedback functions in the module. For our part, we will simply remember them, and pass on the fact of registration. They will be needed in CALL_PROC.
When receiving a message
Message retrieval cycle
Everything is about the same. We will receive a message, find out which function is being asked to call, perform the reverse marshaling process (GET_ *), call the function, get the result and send it to the library. During a function call, a callback function call may occur.
Our fake functions are called that will communicate (with the library).
All this is accompanied by error handling in the form of a surrogate process abnormal termination, substitution of stub functions and return of default values.
The negative side of the solution: all this slows down the speed of the module.
Perhaps that's all ...
In fact, there are still a number of issues for which decisions need to be made. Implemented only at least in the framework of the demo. The set of functions within the framework of the expansion module is somewhat larger, and the export table says about the available capabilities of the module. Dynamically adjust to this is impossible. Not everything is clear with the WLX modules, in particular the interaction with the window. Etc.
The full source code can be found at the link source . You can build with Pelles C for Windows . The resulting application and library must be renamed in accordance with the module (example: the msi.wcx module, the msi.exe program, the msi.wcx64 library) and put next to the module.
And I would like to know your opinion
Problem statement and its solution
The modules for TC are DLL files with extensions WCX, WFX, WLX, WDX and containing a specific set of exported functions (according to the category of module). Everything would be fine, but not all authors took care of 64 bit versions. And the source code is not available, as a rule ...
Question - Can I use existing 32 bit versions?
Answer - Yes, but not everything is so simple.
If you summarize before loading a dynamic 32-bit library into a 64-bit process, it turns out that the task is not new and the search for a solution on the Internet will not keep you waiting. It all comes down to creating a surrogate process capable of loading the library and interacting with this process through IPC (inter-process communication). We do not have access to the source code of the TC and we cannot add the mechanism for working with the surrogate process. But we can create a library. The library will impersonate the module and communicate with the surrogate process, which in turn will pull the module's functions and return the result. And ironing it all will be like this:
')
IPC options are available on MSDN - Interprocess Communications . For my project, I chose Pipes. This may not be the fastest way, but it allows you to implicitly monitor the health of the surrogate process. If the surrogate process falls, then the pipe pipe is broken and our library will know about it. Further description of the processes taking place.
When connecting the library
- generating a unique name for the pipe
- create pipe
- creation of a surrogate process
- transfer of the pipe name to the surrogate process
- waiting and connecting the client through the pipe
To generate a unique name, we use the UuidCreate () function. It will generate a UUID (GUID). Convert it to a string (UuidToString) and fill the path for the pipe. Create a pipe (CreateNamedPipe) working in blocking mode and sending messages. Run the surrogate process (CreateProcess). The name of the pipe is passed as a command line parameter. And we will wait for the client (ConnectNamedPipe).
When disconnecting the library
- disconnect client from pipe
- complete the surrogate process
- close the pipe (in general, release the allocated resources)
Disable the client (DisconnectNamedPipe), complete the surrogate process (TerminateProcess), close the pipe and clean the resources (CloseHandle)
When starting the surrogate process
- get pipe name
- connect to pipe as client
- load module
- expect message
Connect to the pipe-in (CreateFile) and configure it to work in blocking mode and transfer messages. Load the module (LoadLibrary) and save the addresses of the exported functions (GetProcAddress). Let's go into the message waiting loop. If necessary, complete the process out of the loop.
At the completion of the surrogate process
- disconnect from pipe
- unload module
Disconnect from the pipe (CloseHandle) and unload the module (FreeLibrary).
When calling a function from the library
- pack parameters to message
- send request through pipe
- get an answer
- unpack the result and exit function
The function call is considered on the example
__declspec(dllexport) HANDLE __stdcall OpenArchive(tOpenArchiveData *ArchiveData) { if (s_init && ArchiveData) { // serialize uint8_t *p = s_buff; SET_FUNC(p, OPENARCHIVE) SET_CALLTYPE(p, CALL_QUERY) SET_STR_A(p, ArchiveData->ArcName) SET_INT(p, ArchiveData->OpenMode) // send DWORD writeSize = p - s_buff; DWORD writedSize; while (WriteFile(s_pipe, s_buff, writeSize, &writedSize, NULL)) { assert(writeSize == writedSize); // recv DWORD readedSize; if (ReadFile(s_pipe, s_buff, PIPE_BUFF_SIZE, &readedSize, NULL)) { // deserialize uint8_t *p = s_buff; uint8_t func; GET_FUNC(p, func) uint8_t callType; GET_CALLTYPE(p, callType) if (callType == CALL_ANSWER) { assert(func == OPENARCHIVE); GET_INT(p, ArchiveData->OpenResult) HANDLE r; GET_HANDLE(p, r) // result return r; } else if (callType == CALL_QUERY) { CALL_PROC } assert(0); } } ArchiveData->OpenResult = E_NOT_SUPPORTED; } return NULL; } OpenArchive is the first function that TC calls after loading a module. It is passed a pointer structure of type tOpenArchiveData.
typedef struct { char* ArcName; int OpenMode; int OpenResult; char* CmtBuf; int CmtBufSize; int CmtSize; int CmtState; } tOpenArchiveData; We can not pass a pointer to the structure, the processes are isolated and do not see the memory of each other. We also cannot transfer the structure simply by copying it into the message, because of the pointer to the string (ArcName) and the alignment of the fields. Plus, some fields are designed to transfer data to a function (ArcName, OpenMode), and some serve as a buffer for returning a result (OpenResult), the latter are not used at all (Cmt *). We must marshal, i.e. convert the data into a format suitable for transmission. For this are a number of written SET_ * macros. SET_INT writes int as a 32 bit number to the buffer. SET_STR_A writes to the buffer a sign of the validity of a pointer to a string, and in the case of validity, writes the size of the string with terminal zero and the array of characters pointed to by the pointer. Two values are placed at the beginning of the buffer: what is this function and what is it - a request. Next, you need to calculate the size of the data recorded in the buffer and write them to the pipe. Wait for a response from the other side. When you receive a response, read two values: what is this function and what is a response or a request to execute a feedback function. If this is the answer, we get the result, write the part into the structure and exit the function. If this is a request to call the feedback function, we get the parameters for it, execute it, return the result and wait for the next answer (all this is hidden in the CALL_PROC macro). We should also mention the type of result of the considered function. This is HANDLE, but in reality is a pointer. You will need it as a parameter for calling the remaining functions by the TC itself. But its importance plays a role only within the module. In 32-bit processes, the pointer is 32-bit, in 64, respectively, 64-bit. And it is created in a 32 bit process. Therefore, converting it to 64 and then to 32 will not lead to data loss.
Two functions (SetChangeVolProc, SetProcessDataProc) register feedback functions in the module. For our part, we will simply remember them, and pass on the fact of registration. They will be needed in CALL_PROC.
When receiving a message
- get the message
- unpack parameters
- call function from extension
- pack result
- send a message with the result
Message retrieval cycle
while (s_loop) { DWORD readedSize; if (ReadFile(s_pipe, s_buff, PIPE_BUFF_SIZE, &readedSize, NULL)) { // deserialize, process, serialize uint8_t *p = s_buff; uint8_t func; GET_FUNC(p, func) uint8_t callType; GET_CALLTYPE(p, callType) assert(callType == CALL_QUERY); if (func == OPENARCHIVE) { tOpenArchiveData openArchiveData = {0}; GET_STR_A(p, openArchiveData.ArcName) GET_INT(p, openArchiveData.OpenMode) HANDLE r = OpenArchive(&openArchiveData); p = s_buff; SET_FUNC(p, OPENARCHIVE) SET_CALLTYPE(p, CALL_ANSWER) SET_INT(p, openArchiveData.OpenResult) SET_HANDLE(p, r) } else ... ... { assert(0); } DWORD writeSize = p - s_buff; DWORD writedSize; if (!WriteFile(s_pipe, s_buff, writeSize, &writedSize, NULL) || writeSize != writedSize) { return -6; } } else if (GetLastError() != ERROR_TIMEOUT) { break; } } Everything is about the same. We will receive a message, find out which function is being asked to call, perform the reverse marshaling process (GET_ *), call the function, get the result and send it to the library. During a function call, a callback function call may occur.
int __stdcall ChangeVolProc(char *ArcName, int Mode) { uint8_t *p = s_buff; SET_FUNC(p, CHANGEVOLPROC) SET_CALLTYPE(p, CALL_QUERY) SET_STR_A(p, ArcName) SET_INT(p, Mode) DWORD writeSize = p - s_buff; DWORD writedSize; assert(WriteFile(s_pipe, s_buff, writeSize, &writedSize, NULL) || writeSize == writedSize); DWORD readedSize; assert(ReadFile(s_pipe, s_buff, PIPE_BUFF_SIZE, &readedSize, NULL)); p = s_buff; uint8_t func; GET_FUNC(p, func) uint8_t callType; GET_CALLTYPE(p, callType) assert(func == CHANGEVOLPROC && callType == CALL_ANSWER); int r; GET_INT(p, r) return r; } Our fake functions are called that will communicate (with the library).
All this is accompanied by error handling in the form of a surrogate process abnormal termination, substitution of stub functions and return of default values.
The negative side of the solution: all this slows down the speed of the module.
Perhaps that's all ...
What's left
In fact, there are still a number of issues for which decisions need to be made. Implemented only at least in the framework of the demo. The set of functions within the framework of the expansion module is somewhat larger, and the export table says about the available capabilities of the module. Dynamically adjust to this is impossible. Not everything is clear with the WLX modules, in particular the interaction with the window. Etc.
The full source code can be found at the link source . You can build with Pelles C for Windows . The resulting application and library must be renamed in accordance with the module (example: the msi.wcx module, the msi.exe program, the msi.wcx64 library) and put next to the module.
And I would like to know your opinion
Source: https://habr.com/ru/post/202604/
All Articles