Dynamic display of logs in the browser on Node.js & WebSocket

Dynamic display of logs in the browser on Node.js & WebSocket

Sometimes there is a need to dynamically track the emergence of new files in certain folders, as well as output content log files to the browser window. With this simplified example, I want to show how this can be done.

Solution tools and modules

Node.js was selected for the server side. But a similar example can be similarly implemented using EM ruby. In this case, the library bribed Socket.IO , as well as several simple modules for working with files.


')
Utilities for watching file trees in node.js - the meaning of which is to track the appearance, deletion, and changes in the files inside the specified folder.

An example of tracking the appearance / deletion of files inside common.NODES_PATH and sending a message about these events.

watch.createMonitor(common.NODES_PATH, function (monitor) { monitor.on("created", function (f, stat) { var file_name = common.removeRoot(f); console.log(file_name + ' created'); var idx = log_files.indexOf(file_name); if (idx < 0) { log_files.push(file_name); io.sockets.emit('log_files_add', [file_name]); watchFile(file_name); } }) monitor.on("removed", function (f, stat) { var file_name = common.removeRoot(f); console.log(file_name + ' removed'); var idx = log_files.indexOf(file_name); if (idx >= 0) { log_files.splice(idx, 1); io.sockets.emit('log_files_remove', [file_name]); unwatchFile(file_name); } }) }) 

A nodejs directory walker - a package that allows you to walk through the insides of the specified folder and receive events to find a particular type of file. Or, if it's simplistic, just recursively go through all the nested folders and see what lies there. (example is slightly lower)

Jade - Template Engine - to quickly jot down a simplified UI and test the idea.

Implementation example

• Create a web server on the specified port;
• When we first connect to the server, we go through the folders with the walker and send the initial piece of information about the existing files (plus the filter by extension) and the last 20 lines.
  
  

   io.sockets.on('connection', function(socket) { var walk = require('walk'); // receive list of log files var walker = walk.walk(common.NODES_PATH, { followLinks: false }); walker.on('file', function(root, stat, next) { if(/log$/.test(stat.name)) { var file_name = common.removeRoot(root + '/' + stat.name); var idx = log_files.indexOf(file_name); if (idx < 0) { log_files.push(file_name); // send initial data portion var tail = spawn('tail', ['-n', 20].concat(common.NODES_PATH + '/' + file_name)); tail.stdout.on('data', function(data) { console.log('emit starting ' + file_name + ':lines' + ' with ' + data.toString('utf-8')); io.sockets.emit(file_name + ':lines', data.toString('utf-8').split('\n')); }); watchFile(file_name); } } next(); }); walker.on('end', function() { socket.emit('log_files_add', log_files); }); }) 

  
  
  
• With watch, we create a monitor that will monitor the appearance of new files or deletion. (When deleting a file, the block displaying the specified log is deleted accordingly)
  
  

   function watchFile(file) { console.log('start watching: ' + file); var tail = spawn('tail', ['-f'].concat(common.NODES_PATH + '/' + file)); watching_processes[file] = tail; tail.stdout.on('data', function(data) { console.log('emit ' + file + ':lines' + ' with ' + data.toString('utf-8')); io.sockets.emit(file + ':lines', data.toString('utf-8').split('\n')); }) } function unwatchFile(file) { console.log('end watching: ' + file); var process = watching_processes[file]; process.kill(); watching_processes[file] = null; } 

  
  
  

Launch

In common.js, which folder to monitor
var NODES_PATH = pathResolver.resolve(__dirname + '/../nodes');

When * .log appears inside folders and subfolders, they will be displayed in the browser window

npm install to install packages

[~/Projects/autotest_node] git:master $ mkdir nodes
[~/Projects/autotest_node] git:master $ mkdir nodes/test1
[~/Projects/autotest_node] git:master $ mkdir nodes/test2
[~/Projects/autotest_node] git:master $ echo "test_message1" >> nodes/test1/first.log
[~/Projects/autotest_node] git:master $ echo "test_message1" >> nodes/test2/first.log
[~/Projects/autotest_node] git:master $ echo "test_message2" >> nodes/test2/first.log
[~/Projects/autotest_node] git:master $ echo "test_message2" >> nodes/test1/first.log

npm start to start the server

Source

https://github.com/catz/autotest_node

Conclusion

This example can be useful, for example, if you want to share the results of automatic test cases and give others the opportunity to follow the results of their execution. Although there are more advanced tools for such tasks, but if you want to make a simple tail -f in the browser, then a similar example might well be appropriate.