Logging anything in Perl

For logging messages, Pearl offers several ready-made solutions. All of them, as usual, are posted on CPAN . On request "log" you can find a bunch of modules for all occasions.

However, among all these modules there is one special one, it is called Log :: Any .
')
The peculiarity of this module for logging is that it does not, in fact, deal with logging. The Log :: Any module provides the program (and programmer) with a universal API for calls to other modules that deal directly with logging.

If you are tormented by the problem of choosing the logging method in Perla - this article is for you.

Problem

Suppose you have a module that downloads a file from the network. You would like to know what time the download started, how long it lasted and how many bytes were loaded. You can do the simplest way - add lines to the module:

print "$time   \n"; #    print "$time   \n"; print "$time  $size \n"; 

Please note - in order to calculate the time $ time, some additional actions will be needed, but I will not even focus on this, because this is not the main problem.

Everything will be fine as long as you run this script with this module with your hands from the command line. Messages will be displayed to you on the console, you will read them and find out everything you wanted.

But at some point it will be necessary to keep these messages for the future. Or it will be necessary to run this script from Krone, and not by hand. Or it will be necessary to write these messages to the database for further analysis. Well, there will be many downloaded files, and it will be difficult for you to read a long log from the console.

And then you want to redirect the log output from the console somewhere else. To a file, to a database, to another program, to some web service via API, to hell with horns, etc. This is where the problem arises - where exactly and how to log messages?

• Write to file? What if your module will be used in an environment where files cannot be written?
• Write to the database? What if the module for working with the database is not installed?
• Redirect to another program? But what if the user of your script prefers a completely different program that you have chosen?

There are many excellent logging modules, but they all have the same common drawback - these modules are more than one. You like one way of logging, and the user likes the other, the customers the third, the superiors the fourth. Fatal flaw ©.

Decision

Use the Log :: Any module.

Log :: Any solves the selection problem described above in such a way that everyone is satisfied - the programmer thinks only of what he wants to send to the log, and the log users themselves decide on where and how to record the received messages.

The idea of ​​this module is to divide the logging procedure into two separate parts, which can work independently of each other. At the same time, the part that generates the messages has no idea how and where these messages will be further recorded. And the part that writes messages somewhere, has no idea about where these messages came from and how they were formed.

This is how it works.

The dotted lines in the diagram indicate the connection of modules through use , and the solid ones indicate the direction of movement of the logged messages.

Sending messages

To send a message, your code (in a module or in a script, it does not matter) calls the standard function provided by the Log :: Any module:

 #    Log::Any     $log,         use Log::Any qw($log); #   -   (  error    ) $log->error("-    - "); 

The above two lines are all that is needed in order to start logging. But the sent message will not be recorded anywhere yet, since we have not yet chosen where to write the log specifically.

Message recording

Now we need to decide where we write the log. To do this, the script needs to use one of the adapters:

 #           use Log::Any::Adapter ('Stdout'); #          use Log::Any::Adapter ('File', '/path/to/file.log'); #               Log::Dispatch use Log::Dispatch; my $log = Log::Dispatch->new(outputs => [[ ... ]]); Log::Any::Adapter->set( { category => 'Foo::Baz' }, 'Dispatch', dispatcher => $log ); 

Included with Log :: Any comes a few simple built-in adapters - File, Stdout and Stderr. As you can guess from the title, the first one writes messages to a file, and the other two send messages to standard outputs.

In addition to the built-in adapters on CPAN, you can find external ones, such as Log4perl or Syslog . External adapters allow you to write logs anywhere - even on Twitter.

And if you need to Facebook? Not a problem either. You can effortlessly write your own adapter to anything. Creating your own adapter is described in the module documentation , or here in Russian.

I will not go into the details of writing the adapter here, because the links above this task is well described. Instead, I will consider another task - the introduction of logging into an existing code.

Logging existing actions

Connectors

If you are writing code from scratch, you can immediately add logging to the right places. But if you already have some amount of code, then adding logging functions may require a lot of work on digging and modifying existing code. Instead, there is a way to add logging without doing anything (almost).

To do this, you need to use a connector - a module that connects to an existing code and adds logging functions to it.

Remark - the term "connector" I coined myself, perhaps there is some other common name.

Connector modules are usually located in the Log :: Any :: For namespace. There are several ready-made connectors, for example, Log :: Any :: For :: DBI or Log :: Any :: For :: LWP .

Writing your own connectors is not formalized, so much depends on what, in fact, the connector is written to. In general, the connector works like this:

• Intercepted event to be logged. For this, different means can be used, such as mocks or tie .
• The event message is sent to the log using the standard $ log-> method ('message').

The use of the connector is as follows (for example, LWP connector):

 #  LWP use LWP::Simple #    LWP use Log::Any::For::LWP; #          LWP get "http://www.google.com/"; 

In this code, no explicit action was taken to send messages to the log, except for connecting the connector. But, however, the get function will magically get logging and start displaying all sorts of useful messages.

Logging warnings and exceptions

Unfortunately, from time to time in the program there are events that are not provided and - for good - should not happen. For example, it suddenly turned out that in some function a call to an uninitialized variable occurs, or the connection to the database falls off, or the program crashes. In such cases, the Pearl interpreter will throw out a warning or an exception.

If such an event happened in a simple script that you launched from the console, then the corresponding message will appear in front of your eyes and you will immediately see it (although this is not a fact either). If the program is launched from Krona or is running under a web server, or something else like this, then it will not be easy to notice such a message.

The correct solution is to write such a message to the log. But how? After all, messages are written to the log that were explicitly sent to the programmer through the $ log object, and the interpreter throws out the varnings and exceptions, who do not know anything about our remarkable logging and dump all their messages simply on STDERR.

This means that everything that goes to STDERR should be forcibly redirected to the log.

To solve this problem, I did not find a suitable connector on CPAN, so I wrote my own - Log :: Any :: For :: Std . This connector sends to the log all possible messages of the interpreter, at any stage of the program execution.

To intercept STDERR, the tie function is used:

 tie *STDERR, __PACKAGE__; 

This construction wraps up absolutely everything that is sent to STDERR in the package we need, and it is no longer difficult to redirect messages to the log using Log :: Any.

If you wish, you can implement any other connector to intercept warnings and exceptions (or you may not intercept them at all).

Message Filtering

Circumstances may be such that the existing code already displays some messages. For example, I have a big project with a legacy code, in which all the logging is done like this:

 print STDERR "$time --- $login --- $pid ---  \n"; 

As you can see, here all messages are sent to STDERR, and in the message text there are all sorts of variables, delimiters, plus a newline. In addition, although it is not visible, all messages are written without using utf8.

Redirecting messages from STDERR to the log is easily solved using the Log :: Any :: For :: Std connector, but you will have to remove extra garbage from the message text separately. To do this, when you connect the Log :: Any module, you need to enable filtering.

This is done like this:

 use Log::Any '$log', filter => sub { my $msg = $_[2]; utf8::decode($msg); return $msg }; 

Each message sent to the log using $ log-> method () will be passed through the function specified in the filter argument. The variable $ _ [2] in this function contains the message to be sent. If you want to do something with the message, then you must take it from this variable, modify it and return it. The returned value will be recorded in the log.

For example, in the above code, the message text is reduced to utf8.

Script example

Let's put it all together in the test.pl script:

 #!/usr/bin/perl use strict; use warnings; use Log::Any '$log', filter => sub { my $msg = $_[2]; $msg =~ s///; return $msg }; use Log::Any::Adapter ('Stdout'); use Log::Any::For::Std; print "  \n"; $log->info("    "); $log->info("     "); Module::func(); warn "    "; die "      "; #  package Module; use Log::Any '$log'; sub func { $log->info("     "); } 

Run and see the following:

 $ ./test.pl                          at ./test.pl line 18.        at ./test.pl line 20. 

All messages will be displayed on the console, but do not believe your eyes - only the first line is the usual output, everything else is a log. Just this log is displayed on the console.

But what if now we suddenly want to send the log not to the console, but to a file? There is nothing simpler with the Log :: Any module:

 #      # use Log::Any::Adapter ('Stdout'); #   use Log::Any::Adapter ('File', 'file.log'); 

Run and see:

 $ ./test.pl    

As expected, only print will be displayed on the console.

But where did all the rest go:

 $ cat file.log [Fri Jun 19 17:25:44 2015]      [Fri Jun 19 17:25:44 2015]       [Fri Jun 19 17:25:44 2015]       [Fri Jun 19 17:25:44 2015]      at ./test.pl line 18. [Fri Jun 19 17:25:44 2015]        at ./test.pl line 20. 

Summary

• The Log :: Any module allows you to add flexible logging to your programs and modules, which you will not need to redo later when changing the method of saving the log
• Adapters Log :: Any :: Adapter allow you to adapt your program to any way to save the log
• Log :: Any :: For Connectors allow you to connect logging to any message source