Relatively recently, I started learning Python. What to beat the bumps was not clear, but doing the exercises for the exercise was bored quickly enough. I wanted to do something useful and do it with enthusiasm. So the idea was born to make a logger with a color output. The project was called "SCLogger", a start was made. What happened in the end and what mistakes in the design were made further under the cut.

If you do not have time to read, the source can be found here .
Link to the source color source.

Start and first mistake

I started by searching for a module that will color the lines, but unfortunately, none of the solutions for colorizing the lines preserved the colors of the already colored lines, or did not color the initial one, so I had to implement this module first .
')
Before writing the code, I thought, “What should a logger be and what should it know how to do?”. The answer turned out like this ...

The logger must write the file in a table view with the specified delimiter, and also have the following functionality:

• Write to file
• Display a message with the record, if there was an indication of this
• Show statistics on the number of messages
• Read log (all; by message type)

And here is my first mistake: do not put extra functionality in the class . It is better to logically differentiate responsibilities for different modules / classes. This is useful for both architecture and further support.

The error type of the logger will be the most standard and create it based on Enum. We import the modules we need and create 2 classes: Logger and MessageType , inherited from Enum .

from datetime import datetime from pycolorize import colorize, cprint from enum import Enum import re #   class MessageType(Enum): INFO = 10 SUCCESS = 20 WARNING = 30 ERROR = 40 CRITICAL = 50 #       message_color = { 'INFO': 'gray', 'WARNING': 'yellow', 'ERROR': 'red', 'SUCCESS': 'cyan', 'CRITICAL': 'magenta' } class Logger: def __init__(self, fp, delimiter=' ', protected_mode=False): if protected_mode: self.__file = open(fp, 'x') else: self.__file = open(fp, 'w') self.__delimiter = delimiter #  #   self.__mcounter = { 'INFO': 0, 'WARNING': 0, 'ERROR': 0, 'SUCCESS': 0, 'CRITICAL': 0 } 

Now initialization does not look so terrible as it was originally. In the first versions, I made a design error by specifying the filemode argument instead of the protected_mode . That is, I added a lot of work to myself, checking whether the specified filemode is in a valid tuple. So do not, it is better to simplify as much as possible .

Unexpected encounter with error # 2: useless code.

Now we will write a method that will create messages. That is, the main functionality of the logger:

  def message(self, message, message_type: MessageType, to_print=True, include_microseconds=False): mt = str(message_type).replace('MessageType.', '') #    INFO, WARNING, etc. #     MessageType.TYPE #    date = datetime.today().date() if not include_microseconds: time = datetime.today().time().replace(microsecond=0) else: time = datetime.today().time() #        self.__mcounter[mt] += 1 #    #     join message = self.__delimiter.join([str(date), str(time), message, mt]) #   ,      self.__file.write(re.sub(r'(\x1b\[.*?m)', '', message + '\n')) #     re #   print      if to_print: cprint(message, message_color[mt]) 

Everything is good, but what if I want to analyze the log with pandas, for example? For this, in the first versions of the logger I had the __check_head method, which I placed at the end of the class initialization.

  def __check_head(self): template = self.__delimiter.join(['Date', 'Time', 'Message', 'Message type\n']) try: with open(self.__fp, 'r') as f: #  ,   if f.readlines()[0] != template: self.__file.write(template) except IndexError: self.__file.write(template) 

This is the wrong code . This error is generated from the very first: allow the user to select the filemode to work with the file. Why does the method look like this? The fact is that if the file is opened for append (append), then there is no need to write the header again, because the file will be incorrect. Therefore, it was necessary to check the file for the presence of a header.

PS In fact, before writing the article, I did not pay attention to this method and the __fp argument, and the class would continue to function with bad code, because this variant is also working, and I just forgot to rewrite it.

But now logger works only in two modes: 'x' - the uniqueness of the file and 'w' - create the file or overwrite the existing one. This change allows you to simplify the code to this:

  def __write_head(self): #         template = self.__delimiter.join(['Date', 'Time', 'Message', 'Message type\n']) self.__file.write(template) 

Finishing work and finding a place to read messages

They finished off the first two points, even did work above it: they added a hat. It's time to realize getting statistics on the number of recorded messages. Create a get_stats method for this:

  def get_stats(self): info = str(self.__['INFO']) warning = str(self.__mcounter['WARNING']) error = str(self.__mcounter['ERROR']) success = str(self.__mcounter['SUCCESS']) critical = str(self.__mcounter['CRITICAL']) #        #    cprint('INFO MESSAGES COUNT: {}\n' 'WARNING MESSAGES COUNT: {}\n' 'ERROR MESSAGES COUNT: {}\n' 'CRITICAL MESSAGES COUNT: {}\n' 'SUCCESS MESSAGES COUNT: {}'.format (colorize(info, 'magenta'), colorize(warning, 'magenta'), colorize(error, 'magenta'), colorize(critical, 'magenta'), colorize(success, 'magenta')), 'green') 

The work is finished, but I didn’t have the desire to read the messages, and I tried to stick the read_messages () method where it shouldn’t follow (the first design error). In the end, I created a new class that called the LogReader :

 class LogReader: def __init__(self, fp): self.__log = open(fp) def read_messages(self, message_type: MessageType=None): if message_type is None: #      message type    #          #    ,   color   cprint() for line in self.__log.readlines()[1:]: line = line.strip() end = '' for char in line[::-1]: if not char.isspace(): end += char else: end = end[::-1] break try: cprint(line.strip(), message_color[end]) except KeyError: cprint('Key not found. Add it to message_color dict in sclogger module', 'yellow') #   ,         #       cprint(line.strip()) #      else: end = str(message_type).replace('MessageType.', '') for line in self.__log.readlines(): line = line.strip() if line.endswith(end): cprint(line.strip(), message_color[end]) 

While testing the solution, I ran into the problem of reading: I forgot about the fact that before you open the file again, you need to close it. That is, if I call the writing and reading of the same log in one script, the reading does not occur. To solve this, I had to add a close method to the Logger class:

  def close(self): #  ,   self.__file.close() 

Perhaps this is not an ideal solution, but nevertheless everything worked out:

Let's sum up.

All errors were somehow related to the design. The bumps are full and the moral is learned. Well, just in case the cheat sheet:

What should be done:

• Simplify - build simple architecture, write simple, easy to read code.
• After simplification or redesign, do refactoring right away - do not forget to delete unnecessary, optimize solutions and improve code readability.
• Design smart and stick to architecture - as I said, build a simple architecture, but don't forget that it should be logical . Once you have decided what the architecture will be, do not change it unnecessarily.
• Break the functionality - one class or module for all tasks is not always good.
• Think about integration - periodically think about whether it is possible and necessary for your project to work / produce results that can be integrated into other products.

What not to do :

• Trying to put everything in one class / module - if you want everything so, then let it all be, but not immediately. Sometimes good ideas come spontaneously.
• To make design mistakes - try to identify in advance what your MVP will look like (minimum viable product).
• Leaving the workers, but useless lines in the code, is the antonym of the second item from the previous list.

Thanks for attention!