Everything is relative, or the implementation of one simple task in python and tcl
For historical reasons, in our office, we use an old PBX Panasonic TDA200. And, as you know, it displays the call log to the serial port, from which one programmer was used on the server to read data from. This software has a number of restrictions, which makes its use inconvenient (the size of the log file, the size of the database) and in order to overcome these shortcomings and due to natural laziness (to avoid constant cleaning of the log and database manually), it was decided to type something different. And since, for a long time, the word “python” comes across and the inquisitive mind periodically wakes up, it was decided to implement this task in this language and along the way, well familiar to me, tcl. Well, I decided to share the results with the society. Yes, I will immediately note that the problem has been solved and the service has been brought to "industrial" operation. For data storage is used MariaDB DBMS (it already was), as a CentOS 7 host system.
And one more remark - the python player from me is still the one, so you can kick for the quality of the code but not much. Code examples will be given in this order: first the python and then the tickle, the description of the procedures and commands will go in the order necessary for understanding the work of the scripts, and not as they are written in the code.
Python
Tcl
')
Here, in general, everything is clear - we import the necessary and not very modules, and initialize the variables. But in the example on tcl the lines are commented out (these lines need to be moved to another file, see under spoiler), below it will be clear why.
The script can process data from a text file and directly from the serial port. To do this, add keys to run, respectively, -port - read from the port, -file - read from the file:
In the tcl-script, one more option -conf was added , the code was tested on the working server, and there was also a tickle besides the python there, it was already too much. And according to this, I had to build an "executable" on the principle of "all inclusive" and to ensure the flexibility of the settings, this option was added (and even more so).
Go ahead. Serial port functions:
In tcl, operations with files or ports are used the same. Those. first, a so-called pipe is created by the open command, and then the data is read from this “pipe” or written there, be it a file or a serial port.
Team
allows you to hang on the channel event, or rather the reaction to the event, in this case, we indicated that when any data appears in the channel, perform the Read procedure.
That came to the key point - parsing the line. The PBX "throws out" the data in the form of a line where the fields are separated by spaces, more precisely, for each field the size is specified in characters and the missing data is achieved with spaces:
Python function code:
In the string functions between pit and tickle there are some differences, for example line [9:14] returns the contents of the string starting from 9 to 13 characters inclusive, i.e. as the right border is indicated next to the significant symbol. In tcl, for this purpose, the command [string range $ line 9 13] is used .
There is such a great thing in a ticker as an array of variables, in this case, it is arrVariables () , in which all data is stored in corresponding variables defined by the key, for example, arrVariables (call_time) is the time of the call. It was possible to save all this in the form of a list of lists of "key - value", using the example of the previous variable, it would look as follows:
And now we add a line to the database, the structure of which is described below:
The query to the database is built dynamically based on the parameters passed to the function. From the code, in principle, everything is clear - we format the strings in accordance with the requirements of the SQL query, in the right places we insert commas or brackets, etc. And since the request is built dynamically, in some places an extra comma and a space are added that have to be removed with the rstrip (',') command (you can, of course, count the number of fields and add the necessary number of commas, but this does not reduce the overhead and therefore So). Since the data is not often generated, one transaction is performed for each data row (one request), i.e. connected, executed the request, disconnected.
And the function code itself:
And now the same thing on the tickle:
Here you can complete the story. In my opinion, there is no advantage, in this particular case, neither of them has a different language (I cunning a little, for me the tickle is more beautiful, but this is by virtue of habit). Difficulties with python also does not arise. Sources were tested in Centos and Fedore latest versions and Windows 10. Project Designer (in terms of data collection) is brought to a logical conclusion and put into operation, there is still a simple web snout with a phone directory and reports on the collected data, but this is a topic for another article.
Sources are available here: Git repository
And one more remark - the python player from me is still the one, so you can kick for the quality of the code but not much. Code examples will be given in this order: first the python and then the tickle, the description of the procedures and commands will go in the order necessary for understanding the work of the scripts, and not as they are written in the code.
Python
import pymysql import sys, os import re import datetime # db_host = 'host' db_user = 'dbuser' db_pass = 'dbpass' out_dir = '/var/log/ats' Tcl
')
package require mysqltcl # #set db(host) "host" #set db(user) "user" #set db(pass) "password" #set db(dbname) "ats_test" #set out_dir "~/tmp/ats" Here, in general, everything is clear - we import the necessary and not very modules, and initialize the variables. But in the example on tcl the lines are commented out (these lines need to be moved to another file, see under spoiler), below it will be clear why.
config.tcl
# set db(host) "host" set db(user) "user" set db(pass) "password" set db(dbname) "ats_test" set out_dir "~/tmp/ats" The script can process data from a text file and directly from the serial port. To do this, add keys to run, respectively, -port - read from the port, -file - read from the file:
if __name__ == "__main__": if len(sys.argv) > 2: if sys.argv[1] == '-port': #action = 'read_port' port_name = sys.argv[2] port_data_read(port_name) if sys.argv[1] == '-file': #action = 'read_file' log_file_name = sys.argv[2] log = open(log_file_name) for line in log: parce_string(line) log.close() else: print ("\n :\n- \ \n # python data_reader.py -file TDA20013082015_12052016.lg\ \n- com- \ \n # python data_reader.py -port /dev/ttyUSB0\n") sys.exit(1) In the tcl-script, one more option -conf was added , the code was tested on the working server, and there was also a tickle besides the python there, it was already too much. And according to this, I had to build an "executable" on the principle of "all inclusive" and to ensure the flexibility of the settings, this option was added (and even more so).
# if {[llength $argv] >= 2} { if {[lindex $argv 0] == "-conf"} { source [lindex $argv 1] } else { puts " " } if {[lindex $argv 2] == "-port"} { set port_name [lindex $argv 3] PortDataRead $port_name } if {[lindex $argv 2] == "-file"} { set log_file_name [lindex $argv 3] set log [open $log_file_name "r"] # if {[file isdirectory $out_dir] == 0} { file mkdir $out_dir } # while {[gets $log line] >= 0} { ParceString $line } close $log } } else { puts "\n :\n- \ \n # -conf config.tcl \n # tclsh logger.tcl -conf config.tcl -file TDA20013082015_12052016.lg\ \n- com- \ \n # tclsh logger.tcl -conf config.tcl -port /dev/ttyUSB0\n" exit } Go ahead. Serial port functions:
def port_data_read(port_name): global out_dir """ """ import serial ser = serial.Serial(port_name) ser.baudrate = 9600 while True: # line = ser.readline() # line = line.decode() # line = line.rstrip() # parce_string(line) In tcl, operations with files or ports are used the same. Those. first, a so-called pipe is created by the open command, and then the data is read from this “pipe” or written there, be it a file or a serial port.
# proc PortDataRead {portName} { global fh # " " set fh [open $portName RDONLY] # fconfigure $fh -blocking 0 -buffering line -mode 9600,n,8,1 -translation crlf -eofchar {} # "" fileevent $fh readable Read vwait forever } # proc Read {} { global fh if {[gets $fh line] >= 0} { ParceString $line } } Team
fileevent $fh readable Read allows you to hang on the channel event, or rather the reaction to the event, in this case, we indicated that when any data appears in the channel, perform the Read procedure.
That came to the key point - parsing the line. The PBX "throws out" the data in the form of a line where the fields are separated by spaces, more precisely, for each field the size is specified in characters and the missing data is achieved with spaces:
30/09/16 10:44 501 01 <I> 0'00 00:00'13 D0 Python function code:
def parce_string(line): """ """ # if line[:3] == "---" or line == "" or line[3:7] == "Date": print(line) return print(line) # , now = datetime.datetime.now() out_log_name = os.path.join(out_dir, '{}_{}'.format(now.month, now.year)) out_log = open(out_log_name,"a+") out_log.write(line + '\n') out_log.close() # # "//" ( ) call_date = "20{}/{}/{}".format(line[6:8],line[3:5],line[:2]) # call_time = line[9:14].strip() int_number = line[19:22].strip() ext_co_line = line[23:25].strip() dial_number = line[26:51].strip() ring = line[52:56].strip() call_duration = re.sub("'", ":", line[57:65].strip()) acc_code = line[66:77].strip() call_code = line[77:81].strip() # if dial_number == "<I>": call_direct = "" dial_number = "" elif dial_number[:3] == "EXT": call_direct = "" dial_number = dial_number[3:] else: call_direct = "" # insert(call_date=call_date, call_time=call_time, int_number=int_number, ext_co_line=ext_co_line, dial_number=dial_number, ring=ring, call_duration=call_duration, acc_code=acc_code, call_code=call_code, call_direct=call_direct) In the string functions between pit and tickle there are some differences, for example line [9:14] returns the contents of the string starting from 9 to 13 characters inclusive, i.e. as the right border is indicated next to the significant symbol. In tcl, for this purpose, the command [string range $ line 9 13] is used .
proc ParceString {line} { global out_dir arrVariables # if {[string range $line 0 2] == "---" || $line == "" || [string range $line 3 6] == "Date"} { #puts $line return } # , # _ clock set fName [clock format [clock scan "now" -base [clock seconds]] -format %m_%Y] set out_log_name [file join $out_dir $fName] set out_log [open $out_log_name "a+"] puts $out_log "$line" close $out_log # # # "//" set arrVariables(call_date) "20[string range $line 6 7]\/[string range $line 3 4]\/[string range $line 0 1]" set arrVariables(call_time) [string range $line 9 13] set arrVariables(int_number) [string range $line 19 21] set arrVariables(ext_co_line) [string range $line 23 24] set arrVariables(dial_number) [string range $line 26 50] set arrVariables(ring) [string range $line 52 55] set arrVariables(call_duration) [string range $line 57 66] set arrVariables(acc_code) [string range $line 66 76] set arrVariables(call_code) [string range $line 77 81] # if {$arrVariables(dial_number) == "<I>"} { set arrVariables(call_direct) "In" set arrVariables(dial_number) "" } elseif {[string range $arrVariables(dial_number) 0 3] == "EXT"} { set arrVariables(call_direct) "Ext" set arrVariables(dial_number) [string range $arrVariables(dial_number) 3 end] } else { set arrVariables(call_direct) "Out" } InsertData There is such a great thing in a ticker as an array of variables, in this case, it is arrVariables () , in which all data is stored in corresponding variables defined by the key, for example, arrVariables (call_time) is the time of the call. It was possible to save all this in the form of a list of lists of "key - value", using the example of the previous variable, it would look as follows:
lappend lstVar [list call_time [string range $line 9 13]] those. In the lstVar list (more precisely, the variable containing the list), we add a list of two call_time values ​​and the contents of the $ line between 9 and 13 characters inclusive.And now we add a line to the database, the structure of which is described below:
Table structure
CREATE TABLE `cdr` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`call_date` date DEFAULT NULL,
`call_time` time DEFAULT NULL,
`int_number` varchar(11) DEFAULT NULL,
`ext_co_line` char(2) DEFAULT NULL,
`dial_number` varchar(30) DEFAULT NULL,
`ring` varchar(5) DEFAULT NULL,
`call_duration` time DEFAULT NULL,
`acc_code` varchar(20) DEFAULT NULL,
`call_code` char(2) DEFAULT NULL,
`call_direct` varchar(45) DEFAULT NULL,
PRIMARY KEY (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=2775655 DEFAULT CHARSET=utf8 COMMENT='Call Data Records';The query to the database is built dynamically based on the parameters passed to the function. From the code, in principle, everything is clear - we format the strings in accordance with the requirements of the SQL query, in the right places we insert commas or brackets, etc. And since the request is built dynamically, in some places an extra comma and a space are added that have to be removed with the rstrip (',') command (you can, of course, count the number of fields and add the necessary number of commas, but this does not reduce the overhead and therefore So). Since the data is not often generated, one transaction is performed for each data row (one request), i.e. connected, executed the request, disconnected.
And the function code itself:
def insert(**kwargs): """ . """ qwery = 'INSERT INTO `cdr` (' for key in kwargs.keys(): qwery = "{} `{}`, ".format(qwery,key) qwery = qwery.rstrip(', ') + ') VALUES(' for key in kwargs.keys(): #qwery = qwery + '"' + kwargs.get(key) +'", ' qwery = "{} \"{}\",".format(qwery,kwargs.get(key)) And now the same thing on the tickle:
proc InsertData {} { global arrVariables db set qwery "INSERT INTO `cdr` (" # foreach key [array names arrVariables] { set qwery "$qwery `$key`, " } set qwery "[string trimright $qwery ", "]\) VALUES\(" foreach key [array names arrVariables] { set qwery "$qwery \"[string trim $arrVariables($key)]\"," } set qwery "[string trimright $qwery ", "]\);" puts $qwery # set conn [mysql::connect -host $db(host) -db $db(dbname) -user $db(user) -password $db(pass) -encoding utf-8] mysql::exec $conn $qwery mysql::commit $conn mysql::close $conn } Here you can complete the story. In my opinion, there is no advantage, in this particular case, neither of them has a different language (I cunning a little, for me the tickle is more beautiful, but this is by virtue of habit). Difficulties with python also does not arise. Sources were tested in Centos and Fedore latest versions and Windows 10. Project Designer (in terms of data collection) is brought to a logical conclusion and put into operation, there is still a simple web snout with a phone directory and reports on the collected data, but this is a topic for another article.
Sources are available here: Git repository
Source: https://habr.com/ru/post/335842/
All Articles