Telegram bot for Mikrotik with Webhook and JSON parser
What do you think, is it possible, using only the Mikrotik script, to write an interactive Telegram bot, which will work entirely in the environment of the router with support for Webhook, incoming events from the Telegram API?
Preface:
I have been putting off writing this article for a long time, but recent events around the Telegram messenger spurred me with a new energy to take up what was planned. I believe that Telegram, among other things, is a convenient niche platform for informational dialogue services, and it should exist and be cool to compete with other systems without artificial restrictions. May this article be a modest contribution to Telegram support.
Before answering a question, you need to understand what is minimally required from the bot platform for Webhook to work. Here's what: the presence of a WEB server with SSL, a valid SSL certificate or a self-signed certificate that is loaded into the Telegram API, the URL address of the WEB server for processing Webhook. And if access from the Internet (real IP, domain name) to the router can be secured, then with the WEB server (even SSL is not here), Mikrotik has a problem, there is simply no user server. But this problem can be circumvented; a solution will be proposed below.
')
Telegram bot for Mikrotik - this is only the "tip of the iceberg." It is based on the full-featured (as far as possible) JSON parser in the Mikrotik script language. In general, it is not necessary to do a full analysis of JSON to write an average bot, you can easily do by searching and copying in lines, but I chose a different path. Next, I will tell you about the parser and some programming techniques of the Mikrotik script that you learned while working on it.
I admit that creating JSON parser in the Mikrotik script language was a sport for me. It was interesting if this could be done at all, given the limitations of the Mikrotik scripting language. But the further he plunged into the code, the more clearly the paths to the final goal were seen. Earlier, I brought to mind a similar VBScript parser found on the network for the needs of a single SCADA system, so I took the logic of the VBScript implementation as a basis, reworked it taking into account the constructions of the Mikrotik language and designed the code as a library of functions. Along the way, I found several interesting features of the scripting language, which I’ll share with pleasure below. A few words about the limitations. First: the length of the string in Mikrotik variables is 4096 bytes, there's nothing you can do about it, all that is simply not assigned to a variable anymore. Second: Mikrotik does not know anything about real numbers, so the float parser saves as a string variable, the types bool, int, string normally parse into the internal representation.
Functions are represented by the JParseFunctions library file, which "expands" the code of functions into global variables. This library can be called in scripts as many times as necessary, without much loss of performance. For each function, a check is made for its “deployment” in global variables in order to avoid duplication of actions. When editing a library file, you need to delete global variables - the function code so that they are “re-created” with the updates.
JParseFunctions library code:
Consider the work of the parser for example a piece of code Telegram bot. Perform the following commands step by step.
Request the state of the getWebhookInfo API function of the Telegram API, which returns a JSON string to the j.txt file:
Loading JSON string in input variable:
Execution of the $ fJParse parser function and uploading the result to the $ JParseOut variable
In $ JParseOut, you can find an associative array, which is a mapping of the original JSON string to arrays and Mikrotik data types. The content here does not cite, it is given below.
You can set the global variable $ Jdebug (true), then in manual mode, when you call the function in the console of the router, you can get additional output for debugging needs.
In the Mikrotik language, nested (multidimensional) associative arrays are supported.
Here is an example of the output of the global variable $ JParseOut, in which the result of the parser is written:
It can be seen that the key “result” contains also an associative array as a value, the elements of which can be accessed using the "->" chain. And it is important that all elements have their own data type (number, string, boolean, array):
It was experiments with this multi-level construction that gave rise to the idea of ​​creating a JSON parser. The JSON format is nicely shifted to such an internal representation of the Mikrotik script language.
For many, it is no secret that you can define your functions, on the forum site www.mikrotik.com you can find many examples of such structures. My parser is also built on functions, nested and recursive calls. Yes, recursive function calls are supported!
As an example, I will cite the $ fJParsePrint function from the parser set, printing in readable form the contents of the associative array $ JParseOut (or rather, in the form of paths that can be copied and used in their scripts to access the elements of the array) and the result of its work:
You can see in the function code a recursive call that passes the current level of the attachment and the subarray element into the function, thus traversing the entire array tree in the $ JParseOut variable.
For interest, you can call this function with parameters from the console, specify the initial output path, for example, “home”, and an array variable manually:
The function is written to handle the call with and without parameters, i.e. A variable number of parameters are used. Traditionally, before a call, it is necessary to declare (more precisely, declare) global variables and functions inside the block, in this case in the function body. Notice that the declaration ": global fJParsePrint" is present, i.e. the function itself is declared, nothing surprising, it is necessary for a recursive call.
Let's look at the $ fByteToEscapeChar function:
This function converts the parameter $ 1 (byte number) to a string character, i.e. converts an ASCII code to a character. Here, for example, is the code 0x2B, which corresponds to the symbol "+". You can set the character by using the escaping "\ NN", where NN is the ASCII code, but only on the line:
But if the source code is represented by a number (byte), then obtaining a symbol is not an easy task, since there is no ready built-in function for this. Here comes another built-in parse function that allows you to assemble a string — an expression, a control sequence based on the source number, for example, "(\ 2B)".
Expression of the form:
- collects a line of code that must be executed to get a string character on the output. The second execution of the code obtained after the parse is done using the same square brackets [...], so the final expression takes on a rather intricate look, framing with double square brackets [[...]], after which we get the expected symbol:
Now that we can easily access the contents of the JSON responses from the Telegram API, we’ll write the first version of the bot operating in polling mode, i.e. periodic API request Telegram. It will respond to some commands, for example, uptime - request the router operation time, ip - request all DHCP Client IP addresses, parse - output the contents of the $ JParseOut variable, i.e. parse JSON response to the last request. If you enter any other commands or characters, the bot will simply echo out.
This bot is a single script that is called periodically from the scheduler, for example, once a minute and reads the getUpdates API telegram function, after parsing the answer, makes the if-else action on the $ v -> “message” -> “text” variable. I also want to draw attention to the function call “text = $ [$ fJParsePrintVar]” from the set of parser functions, which returns the contents of $ JParseOut in a readable form. The full bot code is shown below.
From the pros: since the script initiates the exchange, it will work through NAT without settings.
The disadvantages of this implementation are: the speed of Mikrotik’s response to a request is determined by the frequency of the script call, each call is made with a getUpdates request, the parsing, in general, the complete request-analysis cycle, which loads the processor; Each call leads to a j.txt file; for a partition on a flash disk, this is bad; for a RAM disk it is not a problem.
Bot Polling script code:
To get rid of these drawbacks, let's create the second version of the script that will process Webhook, i.e. when the Telegram API itself is “hammered” at a given address into the router in order to send new messages.
Mikrotik, of course, does not know how to make a custom Web server inside itself, which is required for the full-fledged operation of Webhook notifications from the Telegram API. But you can slyly get around this problem. To do this, you need to monitor some non-existent TCP socket, which Webhook will “dig” into; this is done using the Mangle (or Firewall) rules. The Telegram API turns on working with Webhook (the setWebhook API function), specifies the domain name of the router and the TCP port, the SSL certificate does not play any role here, i.e. not needed! By changing the value of the packet counter of the Mangle rule, it can be understood that the Webhook (or something else;) gets into the non-existent TCP port (or you can cut off the excess with the src-address filter = 149.154.167.192 / 26). Unfortunately, the Mangle rule cannot directly invoke a custom script (there is no such action), but you can interrogate the packet counter from the script. The script also runs on a schedule, but with a minimum interval of 1 second. In the standby state, only the check of changes in the value of the packet counter is performed. After detecting a new incoming packet, a request is sent to the Telegram API to disable Webhook, and reading and processing of messages is done as in the first variant of the script (polling), then Webhook is turned on again and returned to the standby state. The main steps are illustrated in the diagram of the script.
As already mentioned, the script is run frequently, and to avoid duplication of instances of the called script, protection against duplication is made at the beginning of the script, the name of this script must be indicated there.
Script code Webhook bot:
The “add” command was added to this bot script, which adds the IP address to the ExtAccessIPList allowing address list for 10 minutes.
Sample request and response in Telegram. The last line is the temporary address already added to the IP list:
It remains to point out the disadvantages and advantages of this approach. Cons: for Webhook, you need access to the IP and the specified TCP port of the router from the Internet, in fact, the real IP address, preferably associated with the domain. Regarding the availability of the domain name, I’m not sure if the Telegram API needs to be “smoked”, perhaps it doesn’t allow Webhook to be made on the server’s IP. It works with a dynamic real IP address and a dynamic DNS service.
Pros: the main part of the script actually sleeps all the time, waiting for incoming packets on the surrogate socket. If the script is called frequently (once per second), then Webhook-and are processed very quickly, as in normal Telegram bots.
Also source code can be found here .
And a little video:
Preface:
I have been putting off writing this article for a long time, but recent events around the Telegram messenger spurred me with a new energy to take up what was planned. I believe that Telegram, among other things, is a convenient niche platform for informational dialogue services, and it should exist and be cool to compete with other systems without artificial restrictions. May this article be a modest contribution to Telegram support.
Before answering a question, you need to understand what is minimally required from the bot platform for Webhook to work. Here's what: the presence of a WEB server with SSL, a valid SSL certificate or a self-signed certificate that is loaded into the Telegram API, the URL address of the WEB server for processing Webhook. And if access from the Internet (real IP, domain name) to the router can be secured, then with the WEB server (even SSL is not here), Mikrotik has a problem, there is simply no user server. But this problem can be circumvented; a solution will be proposed below.
')
Telegram bot for Mikrotik - this is only the "tip of the iceberg." It is based on the full-featured (as far as possible) JSON parser in the Mikrotik script language. In general, it is not necessary to do a full analysis of JSON to write an average bot, you can easily do by searching and copying in lines, but I chose a different path. Next, I will tell you about the parser and some programming techniques of the Mikrotik script that you learned while working on it.
JSON parser strings in Mikrotik language
I admit that creating JSON parser in the Mikrotik script language was a sport for me. It was interesting if this could be done at all, given the limitations of the Mikrotik scripting language. But the further he plunged into the code, the more clearly the paths to the final goal were seen. Earlier, I brought to mind a similar VBScript parser found on the network for the needs of a single SCADA system, so I took the logic of the VBScript implementation as a basis, reworked it taking into account the constructions of the Mikrotik language and designed the code as a library of functions. Along the way, I found several interesting features of the scripting language, which I’ll share with pleasure below. A few words about the limitations. First: the length of the string in Mikrotik variables is 4096 bytes, there's nothing you can do about it, all that is simply not assigned to a variable anymore. Second: Mikrotik does not know anything about real numbers, so the float parser saves as a string variable, the types bool, int, string normally parse into the internal representation.
Using JSON parser
Functions are represented by the JParseFunctions library file, which "expands" the code of functions into global variables. This library can be called in scripts as many times as necessary, without much loss of performance. For each function, a check is made for its “deployment” in global variables in order to avoid duplication of actions. When editing a library file, you need to delete global variables - the function code so that they are “re-created” with the updates.
JParseFunctions library code:
JParseFunctions
# -------------------------------- JParseFunctions --------------------------------------------------- # ------------------------------- fJParsePrint ---------------------------------------------------------------- :global fJParsePrint :if (!any $fJParsePrint) do={ :global fJParsePrint do={ :global JParseOut :local TempPath :global fJParsePrint :if ([:len $1] = 0) do={ :set $1 "\$JParseOut" :set $2 $JParseOut } :foreach k,v in=$2 do={ :if ([:typeof $k] = "str") do={ :set k "\"$k\"" } :set TempPath ($1. "->" . $k) :if ([:typeof $v] = "array") do={ :if ([:len $v] > 0) do={ $fJParsePrint $TempPath $v } else={ :put "$TempPath = [] ($[:typeof $v])" } } else={ :put "$TempPath = $v ($[:typeof $v])" } } }} # ------------------------------- fJParsePrintVar ---------------------------------------------------------------- :global fJParsePrintVar :if (!any $fJParsePrintVar) do={ :global fJParsePrintVar do={ :global JParseOut :local TempPath :global fJParsePrintVar :local fJParsePrintRet "" :if ([:len $1] = 0) do={ :set $1 "\$JParseOut" :set $2 $JParseOut } :foreach k,v in=$2 do={ :if ([:typeof $k] = "str") do={ :set k "\"$k\"" } :set TempPath ($1. "->" . $k) :if ($fJParsePrintRet != "") do={ :set fJParsePrintRet ($fJParsePrintRet . "\r\n") } :if ([:typeof $v] = "array") do={ :if ([:len $v] > 0) do={ :set fJParsePrintRet ($fJParsePrintRet . [$fJParsePrintVar $TempPath $v]) } else={ :set fJParsePrintRet ($fJParsePrintRet . "$TempPath = [] ($[:typeof $v])") } } else={ :set fJParsePrintRet ($fJParsePrintRet . "$TempPath = $v ($[:typeof $v])") } } :return $fJParsePrintRet }} # ------------------------------- fJSkipWhitespace ---------------------------------------------------------------- :global fJSkipWhitespace :if (!any $fJSkipWhitespace) do={ :global fJSkipWhitespace do={ :global Jpos :global JSONIn :global Jdebug :while ($Jpos < [:len $JSONIn] and ([:pick $JSONIn $Jpos] ~ "[ \r\n\t]")) do={ :set Jpos ($Jpos + 1) } :if ($Jdebug) do={:put "fJSkipWhitespace: Jpos=$Jpos Char=$[:pick $JSONIn $Jpos]"} }} # -------------------------------- fJParse --------------------------------------------------------------- :global fJParse :if (!any $fJParse) do={ :global fJParse do={ :global Jpos :global JSONIn :global Jdebug :global fJSkipWhitespace :local Char :if (!$1) do={ :set Jpos 0 } $fJSkipWhitespace :set Char [:pick $JSONIn $Jpos] :if ($Jdebug) do={:put "fJParse: Jpos=$Jpos Char=$Char"} :if ($Char="{") do={ :set Jpos ($Jpos + 1) :global fJParseObject :return [$fJParseObject] } else={ :if ($Char="[") do={ :set Jpos ($Jpos + 1) :global fJParseArray :return [$fJParseArray] } else={ :if ($Char="\"") do={ :set Jpos ($Jpos + 1) :global fJParseString :return [$fJParseString] } else={ # :if ([:pick $JSONIn $Jpos ($Jpos+2)]~"^-\?[0-9]") do={ :if ($Char~"[eE0-9.+-]") do={ :global fJParseNumber :return [$fJParseNumber] } else={ :if ($Char="n" and [:pick $JSONIn $Jpos ($Jpos+4)]="null") do={ :set Jpos ($Jpos + 4) :return [] } else={ :if ($Char="t" and [:pick $JSONIn $Jpos ($Jpos+4)]="true") do={ :set Jpos ($Jpos + 4) :return true } else={ :if ($Char="f" and [:pick $JSONIn $Jpos ($Jpos+5)]="false") do={ :set Jpos ($Jpos + 5) :return false } else={ :put "Err.Raise 8732. No JSON object could be fJParseed" :set Jpos ($Jpos + 1) :return [] } } } } } } } }} #-------------------------------- fJParseString --------------------------------------------------------------- :global fJParseString :if (!any $fJParseString) do={ :global fJParseString do={ :global Jpos :global JSONIn :global Jdebug :global fUnicodeToUTF8 :local Char :local StartIdx :local Char2 :local TempString "" :local UTFCode :local Unicode :set StartIdx $Jpos :set Char [:pick $JSONIn $Jpos] :if ($Jdebug) do={:put "fJParseString: Jpos=$Jpos Char=$Char"} :while ($Jpos < [:len $JSONIn] and $Char != "\"") do={ :if ($Char="\\") do={ :set Char2 [:pick $JSONIn ($Jpos + 1)] :if ($Char2 = "u") do={ :set UTFCode [:tonum "0x$[:pick $JSONIn ($Jpos+2) ($Jpos+6)]"] :if ($UTFCode>=0xD800 and $UTFCode<=0xDFFF) do={ # Surrogate pair :set Unicode (($UTFCode & 0x3FF) << 10) :set UTFCode [:tonum "0x$[:pick $JSONIn ($Jpos+8) ($Jpos+12)]"] :set Unicode ($Unicode | ($UTFCode & 0x3FF) | 0x10000) :set TempString ($TempString . [:pick $JSONIn $StartIdx $Jpos] . [$fUnicodeToUTF8 $Unicode]) :set Jpos ($Jpos + 12) } else= { # Basic Multilingual Plane (BMP) :set Unicode $UTFCode :set TempString ($TempString . [:pick $JSONIn $StartIdx $Jpos] . [$fUnicodeToUTF8 $Unicode]) :set Jpos ($Jpos + 6) } :set StartIdx $Jpos :if ($Jdebug) do={:put "fJParseString Unicode: $Unicode"} } else={ :if ($Char2 ~ "[\\bfnrt\"]") do={ :if ($Jdebug) do={:put "fJParseString escape: Char+Char2 $Char$Char2"} :set TempString ($TempString . [:pick $JSONIn $StartIdx $Jpos] . [[:parse "(\"\\$Char2\")"]]) :set Jpos ($Jpos + 2) :set StartIdx $Jpos } else={ :if ($Char2 = "/") do={ :if ($Jdebug) do={:put "fJParseString /: Char+Char2 $Char$Char2"} :set TempString ($TempString . [:pick $JSONIn $StartIdx $Jpos] . "/") :set Jpos ($Jpos + 2) :set StartIdx $Jpos } else={ :put "Err.Raise 8732. Invalid escape" :set Jpos ($Jpos + 2) } } } } else={ :set Jpos ($Jpos + 1) } :set Char [:pick $JSONIn $Jpos] } :set TempString ($TempString . [:pick $JSONIn $StartIdx $Jpos]) :set Jpos ($Jpos + 1) :if ($Jdebug) do={:put "fJParseString: $TempString"} :return $TempString }} #-------------------------------- fJParseNumber --------------------------------------------------------------- :global fJParseNumber :if (!any $fJParseNumber) do={ :global fJParseNumber do={ :global Jpos :local StartIdx :global JSONIn :global Jdebug :local NumberString :local Number :set StartIdx $Jpos :set Jpos ($Jpos + 1) :while ($Jpos < [:len $JSONIn] and [:pick $JSONIn $Jpos]~"[eE0-9.+-]") do={ :set Jpos ($Jpos + 1) } :set NumberString [:pick $JSONIn $StartIdx $Jpos] :set Number [:tonum $NumberString] :if ([:typeof $Number] = "num") do={ :if ($Jdebug) do={:put "fJParseNumber: StartIdx=$StartIdx Jpos=$Jpos $Number ($[:typeof $Number])"} :return $Number } else={ :if ($Jdebug) do={:put "fJParseNumber: StartIdx=$StartIdx Jpos=$Jpos $NumberString ($[:typeof $NumberString])"} :return $NumberString } }} #-------------------------------- fJParseArray --------------------------------------------------------------- :global fJParseArray :if (!any $fJParseArray) do={ :global fJParseArray do={ :global Jpos :global JSONIn :global Jdebug :global fJParse :global fJSkipWhitespace :local Value :local ParseArrayRet [:toarray ""] $fJSkipWhitespace :while ($Jpos < [:len $JSONIn] and [:pick $JSONIn $Jpos]!= "]") do={ :set Value [$fJParse true] :set ($ParseArrayRet->([:len $ParseArrayRet])) $Value :if ($Jdebug) do={:put "fJParseArray: Value="; :put $Value} $fJSkipWhitespace :if ([:pick $JSONIn $Jpos] = ",") do={ :set Jpos ($Jpos + 1) $fJSkipWhitespace } } :set Jpos ($Jpos + 1) # :if ($Jdebug) do={:put "ParseArrayRet: "; :put $ParseArrayRet} :return $ParseArrayRet }} # -------------------------------- fJParseObject --------------------------------------------------------------- :global fJParseObject :if (!any $fJParseObject) do={ :global fJParseObject do={ :global Jpos :global JSONIn :global Jdebug :global fJSkipWhitespace :global fJParseString :global fJParse # Syntax :local ParseObjectRet ({}) don't work in recursive call, use [:toarray ""] for empty array!!! :local ParseObjectRet [:toarray ""] :local Key :local Value :local ExitDo false $fJSkipWhitespace :while ($Jpos < [:len $JSONIn] and [:pick $JSONIn $Jpos]!="}" and !$ExitDo) do={ :if ([:pick $JSONIn $Jpos]!="\"") do={ :put "Err.Raise 8732. Expecting property name" :set ExitDo true } else={ :set Jpos ($Jpos + 1) :set Key [$fJParseString] $fJSkipWhitespace :if ([:pick $JSONIn $Jpos] != ":") do={ :put "Err.Raise 8732. Expecting : delimiter" :set ExitDo true } else={ :set Jpos ($Jpos + 1) :set Value [$fJParse true] :set ($ParseObjectRet->$Key) $Value :if ($Jdebug) do={:put "fJParseObject: Key=$Key Value="; :put $Value} $fJSkipWhitespace :if ([:pick $JSONIn $Jpos]=",") do={ :set Jpos ($Jpos + 1) $fJSkipWhitespace } } } } :set Jpos ($Jpos + 1) # :if ($Jdebug) do={:put "ParseObjectRet: "; :put $ParseObjectRet} :return $ParseObjectRet }} # ------------------- fByteToEscapeChar ---------------------- :global fByteToEscapeChar :if (!any $fByteToEscapeChar) do={ :global fByteToEscapeChar do={ # :set $1 [:tonum $1] :return [[:parse "(\"\\$[:pick "0123456789ABCDEF" (($1 >> 4) & 0xF)]$[:pick "0123456789ABCDEF" ($1 & 0xF)]\")"]] }} # ------------------- fUnicodeToUTF8---------------------- :global fUnicodeToUTF8 :if (!any $fUnicodeToUTF8) do={ :global fUnicodeToUTF8 do={ :global fByteToEscapeChar # :local Ubytes [:tonum $1] :local Nbyte :local EscapeStr "" :if ($1 < 0x80) do={ :set EscapeStr [$fByteToEscapeChar $1] } else={ :if ($1 < 0x800) do={ :set Nbyte 2 } else={ :if ($1 < 0x10000) do={ :set Nbyte 3 } else={ :if ($1 < 0x20000) do={ :set Nbyte 4 } else={ :if ($1 < 0x4000000) do={ :set Nbyte 5 } else={ :if ($1 < 0x80000000) do={ :set Nbyte 6 } } } } } :for i from=2 to=$Nbyte do={ :set EscapeStr ([$fByteToEscapeChar ($1 & 0x3F | 0x80)] . $EscapeStr) :set $1 ($1 >> 6) } :set EscapeStr ([$fByteToEscapeChar (((0xFF00 >> $Nbyte) & 0xFF) | $1)] . $EscapeStr) } :return $EscapeStr }} # ------------------- End JParseFunctions---------------------- Consider the work of the parser for example a piece of code Telegram bot. Perform the following commands step by step.
Request the state of the getWebhookInfo API function of the Telegram API, which returns a JSON string to the j.txt file:
:do {/tool fetch url="https://api.telegram.org/bot$TToken/getWebhookInfo" dst-path=j.txt} on-error={:put "getWebhookInfo error"}; [admin@MikroTik] > :put [/file get j.txt contents]; {"ok":true,"result":{"url":"https://*****:8443","has_custom_certificate":false,"pending_update_count":0,"last_error_date":1524565055,"last_error_message":"Connection timed out","max_connections":4 0}} Loading JSON string in input variable:
:set JSONIn [/file get j.txt contents] Execution of the $ fJParse parser function and uploading the result to the $ JParseOut variable
:set JParseOut [$fJParse]; In $ JParseOut, you can find an associative array, which is a mapping of the original JSON string to arrays and Mikrotik data types. The content here does not cite, it is given below.
You can set the global variable $ Jdebug (true), then in manual mode, when you call the function in the console of the router, you can get additional output for debugging needs.
Multidimensional associative arrays
In the Mikrotik language, nested (multidimensional) associative arrays are supported.
Here is an example of the output of the global variable $ JParseOut, in which the result of the parser is written:
[admin@MikroTik] > :put $JParseOut ok=true;result=has_custom_certificate=false;max_connections=40;pending_update_count=0;url=https://*****.ru:8443 [admin@MikroTik] > :put ($JParseOut->"result") has_custom_certificate=false;max_connections=40;pending_update_count=0;url=https://*****:8443 [admin@MikroTik] > :put ($JParseOut->"result"->"max_connections") 40 It can be seen that the key “result” contains also an associative array as a value, the elements of which can be accessed using the "->" chain. And it is important that all elements have their own data type (number, string, boolean, array):
[admin@MikroTik] > :put [:typeof ($JParseOut->"result")] array [admin@MikroTik] > :put [:typeof ($JParseOut->"result"->"max_connections")] num [admin@MikroTik] > :put [:typeof ($JParseOut->"result"->"url")] str It was experiments with this multi-level construction that gave rise to the idea of ​​creating a JSON parser. The JSON format is nicely shifted to such an internal representation of the Mikrotik script language.
Functions, recursive call
For many, it is no secret that you can define your functions, on the forum site www.mikrotik.com you can find many examples of such structures. My parser is also built on functions, nested and recursive calls. Yes, recursive function calls are supported!
As an example, I will cite the $ fJParsePrint function from the parser set, printing in readable form the contents of the associative array $ JParseOut (or rather, in the form of paths that can be copied and used in their scripts to access the elements of the array) and the result of its work:
:global fJParsePrint :if (!any $fJParsePrint) do={ :global fJParsePrint do={ :global JParseOut :local TempPath :global fJParsePrint :if ([:len $1] = 0) do={ :set $1 "\$JParseOut" :set $2 $JParseOut } :foreach k,v in=$2 do={ :if ([:typeof $k] = "str") do={ :set k "\"$k\"" } :set TempPath ($1. "->" . $k) :if ([:typeof $v] = "array") do={ :if ([:len $v] > 0) do={ $fJParsePrint $TempPath $v } else={ :put "$TempPath = [] ($[:typeof $v])" } } else={ :put "$TempPath = $v ($[:typeof $v])" } } }} [admin@MikroTik] > $fJParsePrint $JParseOut->"ok" = true (bool) $JParseOut->"result"->"has_custom_certificate" = false (bool) $JParseOut->"result"->"last_error_date" = 1524483204 (num) $JParseOut->"result"->"last_error_message" = Connection timed out (str) $JParseOut->"result"->"max_connections" = 40 (num) $JParseOut->"result"->"pending_update_count" = 0 (num) $JParseOut->"result"->"url" = https://*****.ru:8443 (str) You can see in the function code a recursive call that passes the current level of the attachment and the subarray element into the function, thus traversing the entire array tree in the $ JParseOut variable.
$fJParsePrint $TempPath $v For interest, you can call this function with parameters from the console, specify the initial output path, for example, “home”, and an array variable manually:
[admin@MikroTik] > $fJParsePrint "home" $JParseOut home->"ok" = true (bool) home->"result"->"has_custom_certificate" = false (bool) home->"result"->"last_error_date" = 1524483204 (num) home->"result"->"last_error_message" = Connection timed out (str) home->"result"->"max_connections" = 40 (num) home->"result"->"pending_update_count" = 0 (num) home->"result"->"url" = https://*****.ru:8443 (str) The function is written to handle the call with and without parameters, i.e. A variable number of parameters are used. Traditionally, before a call, it is necessary to declare (more precisely, declare) global variables and functions inside the block, in this case in the function body. Notice that the declaration ": global fJParsePrint" is present, i.e. the function itself is declared, nothing surprising, it is necessary for a recursive call.
Parsing a string with on-the-fly code and its execution
Let's look at the $ fByteToEscapeChar function:
:global fByteToEscapeChar :if (!any $fByteToEscapeChar) do={ :global fByteToEscapeChar do={ # :set $1 [:tonum $1] :return [[:parse "(\"\\$[:pick "0123456789ABCDEF" (($1 >> 4) & 0xF)]$[:pick "0123456789ABCDEF" ($1 & 0xF)]\")"]] }} This function converts the parameter $ 1 (byte number) to a string character, i.e. converts an ASCII code to a character. Here, for example, is the code 0x2B, which corresponds to the symbol "+". You can set the character by using the escaping "\ NN", where NN is the ASCII code, but only on the line:
[admin@MikroTik] > :put "\2B" + But if the source code is represented by a number (byte), then obtaining a symbol is not an easy task, since there is no ready built-in function for this. Here comes another built-in parse function that allows you to assemble a string — an expression, a control sequence based on the source number, for example, "(\ 2B)".
Expression of the form:
:put [:parse "(\"\\$[:pick "0123456789ABCDEF" ((0x2B >> 4) & 0xF)]$[:pick "0123456789ABCDEF" (0x2B & 0xF)]\")"] (<%% + ) - collects a line of code that must be executed to get a string character on the output. The second execution of the code obtained after the parse is done using the same square brackets [...], so the final expression takes on a rather intricate look, framing with double square brackets [[...]], after which we get the expected symbol:
[admin@MikroTik] > :put [[:parse "(\"\\$[:pick "0123456789ABCDEF" ((0x2B >> 4) & 0xF)]$[:pick "0123456789ABCDEF" (0x2B & 0xF)]\")"]] + Telegram bot based on JSON parser
Polling bot
Now that we can easily access the contents of the JSON responses from the Telegram API, we’ll write the first version of the bot operating in polling mode, i.e. periodic API request Telegram. It will respond to some commands, for example, uptime - request the router operation time, ip - request all DHCP Client IP addresses, parse - output the contents of the $ JParseOut variable, i.e. parse JSON response to the last request. If you enter any other commands or characters, the bot will simply echo out.
This bot is a single script that is called periodically from the scheduler, for example, once a minute and reads the getUpdates API telegram function, after parsing the answer, makes the if-else action on the $ v -> “message” -> “text” variable. I also want to draw attention to the function call “text = $ [$ fJParsePrintVar]” from the set of parser functions, which returns the contents of $ JParseOut in a readable form. The full bot code is shown below.
From the pros: since the script initiates the exchange, it will work through NAT without settings.
The disadvantages of this implementation are: the speed of Mikrotik’s response to a request is determined by the frequency of the script call, each call is made with a getUpdates request, the parsing, in general, the complete request-analysis cycle, which loads the processor; Each call leads to a j.txt file; for a partition on a flash disk, this is bad; for a RAM disk it is not a problem.
Bot Polling script code:
TelegramPollingBot
/system script run JParseFunctions :global TToken "12312312:32131231231" :global TChatId "43242342423" :global Toffset :if ([:typeof $Toffset] != "num") do={:set Toffset 0} /tool fetch url="https://api.telegram.org/bot$TToken/getUpdates\?chat_id=$TChatId&offset=$Toffset" dst-path=j.txt #:delay 2 :global JSONIn [/file get j.txt contents] :global fJParse :global fJParsePrintVar :global Jdebug false :global JParseOut [$fJParse] :local Results ($JParseOut->"result") :if ([:len $Results]>0) do={ :foreach k,v in=$Results do={ :if (any ($v->"message"->"text")) do={ :if ($v->"message"->"text" ~ "uptime") do={ /tool fetch url="https://api.telegram.org/bot$TToken/sendmessage\?chat_id=$TChatId" http-method=post http-data="text=$[/system resource get uptime]" keep-result=no } else={ :if ($v->"message"->"text" ~ "ip") do={ /tool fetch url="https://api.telegram.org/bot$TToken/sendmessage\?chat_id=$TChatId" http-method=post http-data="text=$[/ip dhcp-client print as-value]" keep-result=no } else={ :if ($v->"message"->"text" ~ "parse") do={ /tool fetch url="https://api.telegram.org/bot$TToken/sendmessage\?chat_id=$TChatId" http-method=post http-data="text=$[$fJParsePrintVar]" keep-result=no } else={ /tool fetch url="https://api.telegram.org/bot$TToken/sendmessage\?chat_id=$TChatId" http-method=post http-data="text=$($v->"message"->"text")" keep-result=no } } } } :set $Toffset ($v->"update_id" + 1) } } else={ :set $Toffset 0 } Webhook bot
To get rid of these drawbacks, let's create the second version of the script that will process Webhook, i.e. when the Telegram API itself is “hammered” at a given address into the router in order to send new messages.
Mikrotik, of course, does not know how to make a custom Web server inside itself, which is required for the full-fledged operation of Webhook notifications from the Telegram API. But you can slyly get around this problem. To do this, you need to monitor some non-existent TCP socket, which Webhook will “dig” into; this is done using the Mangle (or Firewall) rules. The Telegram API turns on working with Webhook (the setWebhook API function), specifies the domain name of the router and the TCP port, the SSL certificate does not play any role here, i.e. not needed! By changing the value of the packet counter of the Mangle rule, it can be understood that the Webhook (or something else;) gets into the non-existent TCP port (or you can cut off the excess with the src-address filter = 149.154.167.192 / 26). Unfortunately, the Mangle rule cannot directly invoke a custom script (there is no such action), but you can interrogate the packet counter from the script. The script also runs on a schedule, but with a minimum interval of 1 second. In the standby state, only the check of changes in the value of the packet counter is performed. After detecting a new incoming packet, a request is sent to the Telegram API to disable Webhook, and reading and processing of messages is done as in the first variant of the script (polling), then Webhook is turned on again and returned to the standby state. The main steps are illustrated in the diagram of the script.
As already mentioned, the script is run frequently, and to avoid duplication of instances of the called script, protection against duplication is made at the beginning of the script, the name of this script must be indicated there.
:if ([:len [/system script job find script=TelegramWebhookBot]] <= 1) do={...} Script code Webhook bot:
TelegramWebhookBot
:if ([:len [/system script job find script=TelegramWebhookBot]] <= 1) do={ #:while (true) do={ :global TelegramWebhookPackets :local TWebhookURL "https://www.yourdomain" :local TWebhookPort "8443" # Create Telegram webhook mangle action :if ([:len [/ip firewall mangle find dst-port=$TWebhookPort]] = 0) do={ /ip firewall mangle add action=accept chain=prerouting connection-state=new dst-port=$TWebhookPort protocol=tcp src-address=149.154.167.192/26 comment="Telegram" } :if ([/ip firewall mangle get [find dst-port=$TWebhookPort] packets] != $TelegramWebhookPackets) do={ /system script run JParseFunctions :local TToken "123123123:123123123123123" :local TChatId "3213123123123" :global TelegramOffset :global fJParse :global fJParsePrintVar :global Jdebug false :global JSONIn :global JParseOut :if ([:typeof $TelegramOffset] != "num") do={:set TelegramOffset 0} :put "getWebhookInfo" :do {/tool fetch url="https://api.telegram.org/bot$TToken/getWebhookInfo" dst-path=j.txt} on-error={:put "getWebhookInfo error"} :set JSONIn [/file get j.txt contents] :set JParseOut [$fJParse] :put $JParseOut :if ($JParseOut->"result"->"pending_update_count" > 0) do={ :put "pending_update_count > 0" :do {/tool fetch url="https://api.telegram.org/bot$TToken/deleteWebhook" http-method=get keep-result=no} on-error={:put "deleteWebhook error"} :put "getUpdates" :do {/tool fetch url="https://api.telegram.org/bot$TToken/getUpdates\?chat_id=$TChatId&offset=$TelegramOffset" dst-path=j.txt} on-error={:put "getUpdates error"} :set JSONIn [/file get j.txt contents] :set JParseOut [$fJParse] :put $JParseOut :if ([:len ($JParseOut->"result")] > 0) do={ :foreach k,v in=($JParseOut->"result") do={ :if (any ($v->"message"->"text")) do={ :if ($v->"message"->"text" ~ "uptime") do={ :do {/tool fetch url="https://api.telegram.org/bot$TToken/sendmessage\?chat_id=$TChatId" http-method=post http-data="text=$[/system resource get uptime]" keep-result=no} on-error={:put "sendmessage error"} } else={ :if ($v->"message"->"text" ~ "ip") do={ :do {/tool fetch url="https://api.telegram.org/bot$TToken/sendmessage\?chat_id=$TChatId" http-method=post http-data="text=$[/ip dhcp-client print as-value]" keep-result=no} on-error={:put "sendmessage error"} } else={ :if ($v->"message"->"text" ~ "parse") do={ :do {/tool fetch url="https://api.telegram.org/bot$TToken/sendmessage\?chat_id=$TChatId" http-method=post http-data="text=$[$fJParsePrintVar]" keep-result=no} on-error={:put "sendmessage error"} } else={ :if ($v->"message"->"text" ~ "add") do={ :local addIP [:toip [:pick ($v->"message"->"text") 4 [:len ($v->"message"->"text")]]] :if ([:typeof $addIP] = "ip") do={ :do {/ip firewall address-list add address=$addIP list=ExtAccessIPList timeout=10m comment="temp"} on-error={:put "ip in list error"} } :local Str1 "" :foreach item in=[/ip firewall address-list print as-value where list=ExtAccessIPList and dynamic] do={:set Str1 ($Str1 . "$($item->"address") $($item->"timeout") $($item->"comment")\r\n")} :do {/tool fetch url="https://api.telegram.org/bot$TToken/sendmessage\?chat_id=$TChatId" http-method=post http-data="text=$Str1" keep-result=no} on-error={:put "sendmessage error"} } else={ :put ($v->"message"->"text") :do {/tool fetch url="https://api.telegram.org/bot$TToken/sendmessage\?chat_id=$TChatId" http-method=post http-data="text=$($v->"message"->"text")" keep-result=no} on-error={:put "sendmessage error"} } } } } } :set $TelegramOffset ($v->"update_id" + 1) } } else={ # :set $TelegramOffset 0 } :put "getUpdates" :do {/tool fetch url="https://api.telegram.org/bot$TToken/getUpdates\?chat_id=$TChatId&offset=$TelegramOffset" keep-result=no} on-error={:put "getUpdates error"} :put "setWebhook" :do {/tool fetch url="https://api.telegram.org/bot$TToken/setWebhook\?url=$TWebhookURL:$TWebhookPort" keep-result=no} on-error={:put "setWebhook error"} } else={ :if ($JParseOut->"result"->"url"="") do={ :put "setWebhook" :do {/tool fetch url="https://api.telegram.org/bot$TToken/setWebhook\?url=$TWebhookURL:$TWebhookPort" keep-result=no} on-error={:put "setWebhook error"} } } :set TelegramWebhookPackets [/ip firewall mangle get [find dst-port=$TWebhookPort] packets] :put "--------------------------------------------------" } } The “add” command was added to this bot script, which adds the IP address to the ExtAccessIPList allowing address list for 10 minutes.
Sample request and response in Telegram. The last line is the temporary address already added to the IP list:
>add 1.1.1.1
>> 90.0.0.97 h*******
100.0.0.157 6*******
90.0.0.2 i*******.ru
100.00.66 b*******.ru
1.1.1.1 00:10:00 tempIt remains to point out the disadvantages and advantages of this approach. Cons: for Webhook, you need access to the IP and the specified TCP port of the router from the Internet, in fact, the real IP address, preferably associated with the domain. Regarding the availability of the domain name, I’m not sure if the Telegram API needs to be “smoked”, perhaps it doesn’t allow Webhook to be made on the server’s IP. It works with a dynamic real IP address and a dynamic DNS service.
Pros: the main part of the script actually sleeps all the time, waiting for incoming packets on the surrogate socket. If the script is called frequently (once per second), then Webhook-and are processed very quickly, as in normal Telegram bots.
Also source code can be found here .
And a little video:
Source: https://habr.com/ru/post/337978/
All Articles