Principles of work of one Python-developer

In this publication, I would like to present to the court of a respected reader some of the principles that guide me in fulfilling my responsibilities as a Python developer.

On the one hand, I wanted to share my accumulated experience, which can be useful for novice developers, and on the other hand, to get feedback from more experienced developers, managers and specialists in related fields.

Principles are conditionally grouped into three groups: principles of decision making; principles aimed at improving the quality of the code; principles to improve code performance.
')

• Making decisions
  • Any technical solution must be justified.
  • Responsibility for the decision is always on the one or those who made the decision.
  • When making technical decisions, it is necessary to consider their action in time and their compliance with business needs.
  • One of the main criteria for making technical and other decisions should be their greatest effectiveness.
  • Feel free to deviate from the rules, methodologies, patterns and other restrictions, if the effect of such a deviation exceeds the potential loss (although it’s practical)
  • If necessary, make a working, but perhaps not the best, solution immediately, and later improve it (Now is better than never,
  • If it is difficult to choose between two alternative technical solutions, then you need to choose any and move with it further, when more information appears, then it will be possible to do refactoring if the solution turned out to be non-optimal
  • The flexibility of technical solutions is highly desirable, and versatility is not required
• Source Code Quality
  • The quality of the code should be optimized on the basis of a formed system of criteria that is balanced against costs in the short and long term.
  • Write the optimal code immediately, if it does not increase its complexity and development time (Beautiful is better than ugly)
  • Self-documenting code takes precedence over well commented code (Beautiful is better than ugly)
  • Write TODO and FIXME in code
  • Give variables, functions, methods, classes and other source code objects names that accurately reflect their purpose, despite the increase in the length of the names (Explicit is better than implicit)
  • Fewer lines and code size is preferable, while maintaining the same code readability (Simple is better than complex)
  • Apply code review as a tool for detecting errors, leveling development style, getting to know someone else’s code, and learning in a team
  • Reuse your own and someone else's code
  • Use specialized libraries to solve specific problems, instead of developing their own similar code.
• Performance
  • Code development performance overrides code execution performance.
  • Optimization of the performance of code execution should be justified by the corresponding need.
  • Optimization of code execution performance should be performed by eliminating the most serious bottlenecks.
  • First of all, the most effective methods for optimizing the performance of code execution should be used.

The following is a detailed explanation of each of these principles. For some principles, the postulates of Zen of Python are indicated in parentheses, which in my opinion are related to these principles, or their parts.

Making decisions

Any technical solution must be justified.

Any technical solution must be justified: from the syntax used in the code to the architectural solutions of the highest level. For me, the validity of a technical solution means the existence of arguments confirming its advantage, the inevitability of such a decision, or other expediency of this decision. Justification must be accompanied by the presence of knowledge about the shortcomings of this decision and a plan to overcome them, or justification of their irrelevance. In addition, alternative solutions should be considered with justification for their lower attractiveness compared to the chosen solution. The rationale should take into account both technical and organizational aspects associated with a specific technical solution. If different arguments conflict with each other, then one should be guided by the economic effect of this or that decision for the business, taking into account the short-term and long-term perspectives.

In non-technical, including organizational, issues this principle should be applied in a similar way.

This principle is opposed to the pattern of decision-making, as well as the thoughtless adoption of the first available decision without a proper analysis of alternatives.

Responsibility for the decision is always on the one or those who made the decision.

This principle is aimed at improving the quality of decisions and contrasted with haphazard decision-making.

When making technical decisions, it is necessary to consider their action in time and their compliance with business needs.

One should take into account the impact of this or that decision in both the short and long term, as well as whether this technical solution is useful for business in these periods.

This principle is opposed to one-sided decision making with a concentration of results only in the short term, or only in the long term, and without taking into account business needs and taking into account the impact on business of these decisions with concentration solely on the technical advantages of the solution.

One of the main criteria for making technical and other decisions should be their greatest effectiveness.

By efficiency in this matter, I mean the ratio of the value of the result for the business to the amount of expenses required for this.

This principle is opposed to an approach that takes into account only the value of the result obtained, regardless of the cost required.

Feel free to deviate from the rules, methodologies, patterns and other restrictions, if the effect of such a deviation exceeds the potential loss (although it’s practical)

Rules, methodologies, templates and other restrictions are a generalization of previous experience in order to obtain a stable result of activity in certain conditions. However, in specific cases, conditions may differ, therefore, it is necessary to consciously approach the use of accumulated experience and make its own adjustments if necessary.

This principle is opposed to dogmatic adherence to the rules, methodologies, patterns and other accepted restrictions without taking into account the surrounding reality.

If necessary, you should make a working, but perhaps not the best, solution right away, and later improve it (Now is better than never, although it is often better than * right * now)

There are situations in which it is necessary to implement a certain functionality or fix a defect in a short time, while there are alternative solutions: quick and good. From the quick effect can be obtained immediately, but in the long term, it can create difficulties and lead to additional costs. A good decision to do is long and costly in the short term, but in the long term it will pay off and bring substantially more benefits than it has required. In this case, the combination of both solutions will be effective, namely: we are making a quick decision now and routinely make a good decision. In total, this will turn out to be a bit more expensive than the implementation of only one of the solutions, but the effect of their combination will pay for the costs.

This principle is opposed to trying to do everything at once perfectly.

If it is difficult to choose between two alternative technical solutions, then you need to choose any and move with it further, when more information appears, then it will be possible to do refactoring if the solution turned out to be non-optimal

No comments

The flexibility of technical solutions is highly desirable, and versatility is not required

Flexibility is the ability to adapt a technical solution to new conditions that may appear in the future. Versatility is the actual adaptation of a technical solution to the possibility of using for a set of conditions. Often, ensuring universality requires more effort than flexibility, since it takes into account the greater number of conditions with which a technical solution must be compatible. Also, ensuring universality may be ineffective, as it requires adaptation to conditions that have not yet come and may never come. In addition, sometimes unification requires complex generalizations. Therefore, the flexibility of technical solutions is highly desirable, and universality is not required.

This principle is opposed to an attempt to immediately make universal decisions that later turn out to be unclaimed due to changes in external conditions.

Source Code Quality

The quality of the code should be optimized on the basis of a formed system of criteria that is balanced against costs in the short and long term.

My basic prioritized list of optimization criteria is as follows (if necessary, should be adjusted for specific conditions):

1. High code readability - the easier it is to understand the code, the better (Readability counts)
2. Low cyclomatic code complexity - the smaller the branches, cycles, conditional operations, function calls and methods, the better (Simple is better than complex. Flat is better than nested)
3. High code flexibility - the easier it is to make changes to the code, the better (this optimization criterion increases development costs in the short run, but reduces them in the long run)
4. Small code size - the smaller the code, the better (Simple is better than complex)
5. Code Execution Performance - the faster the code, the better

The prioritization of the criteria is as follows: first, first of all, labor costs are spent on providing criteria with a higher priority; secondly, if the criteria conflict with each other, then a solution is applied that matches the criterion with a higher priority.

This principle is contrasted with the coat optimizing the quality of the code, as well as the use of uncoordinated among the members of the development team optimization criteria and their priorities.

Write the optimal code immediately, if it does not increase its complexity and development time (Beautiful is better than ugly)

Sometimes there are several functionally equivalent code constructs, but with different performance. As a rule, it is enough to find out once which construction has the greatest performance or memory requirements and use it in all similar cases. However, it should be borne in mind that some such constructions may degrade the readability of the code and increase the development time both at the time of writing the code and with its support and modification in the future. This should be taken into account in order not to face the disadvantages of premature optimization.

Self-documenting code takes precedence over well commented code (Beautiful is better than ugly)

Sometimes comments help to understand complex code, but at the same time they are an indicator of low code readability. Therefore, in cases when there is a need to write a comment, you should think about what changes you can make to the code so that the need for comments disappears.

Here are typical, in my opinion, cases where you can do without the commentary:

• The commentary clarifies the meaning of the data contained in the variables — corrects by renaming the variable (similarly applicable to the names of classes, methods, functions, constants, etc.)
• Selection of a block of code with the help of a comment - corrected by moving the block of code into a separate function or method
• The embrace of the action performed at the level of the semantics of the language - corrected by deleting the comment (an example of the comment that needs to be deleted: a = 10 # assign 10 to a)

Cases where comments should be written:

• Public API Documentation
• Explanation of non-obvious reasons, the choice of alternative technical solutions
• Explanation of the optimized and, as a result, difficult for understanding code segment
• TODO and FIXME - micro-planning in the context of source code

Comments can not only help, but hinder the development because they require additional writing time, take up space, reduce the number of lines of code that are simultaneously visible on the screen, require synchronization when changing code, take up additional time to read. Therefore, commenting should be reasonable.

This principle is opposed to excessive use of comments.

Write TODO and FIXME in code

Not everything and not always makes sense to do at the time of writing a particular piece of code, especially this concerns refactoring and making changes, the relevance of which is a question. In such cases, it is reasonable to write a TODO or FIXME comment in code that can be used at a more appropriate time. It is advisable to immediately assign an approximate priority to this comment in order to simplify planning in the future.

This principle is opposed to writing the ideal code without taking into account the real needs and costs of such code at a particular point in time.

Give variables, functions, methods, classes and other source code objects names that accurately reflect their purpose, despite the increase in the length of the names (Explicit is better than implicit)

Saving title length cannot be done at the expense of code clarity.

Fewer lines and code size is preferable, while maintaining the same code readability (Simple is better than complex)

There are studies from which it follows that the number of errors in the code on average is directly proportional to the length of the code and the number of lines of code in particular. This is explained as follows: firstly, the smaller part of the code is placed simultaneously on the screen, the more effort is required to memorize invisible sections of the code that are currently invisible, and secondly, the longer the code, the more typos can be made in it and other human errors.

Apply code review as a tool for detecting errors, leveling development style, getting to know someone else’s code, and learning in a team

In addition to its intended purpose, the code inspection is excellent for leveling the style of development, getting to know someone else's code, and training in a team. In the ideal case, only the code that passed the code inspection should fall into the general branch.

Reuse your own and someone else's code

This principle follows from the DRY-principle. In order to successfully apply this principle, you must have a good knowledge of the code base, and the application of code inspection by all members of the development team helps in this.

Use specialized libraries to solve specific problems, instead of developing their own similar code.

If you need to solve any problem (for example, send an email via smtp), you should familiarize yourself with the availability of ready-made libraries for this task and explore the possibilities of using them. Failure to use specialized solutions should be justified, since specialized solutions have several advantages over the new development.

Specialized solutions have already been developed and tested. This means saving time for developing, testing and correcting defects. Sometimes it seems that it is easier to develop your own code that performs the necessary function, since to use a ready-made solution, it takes time to study and there is a risk that it will be wasted, because it turns out that the solution is not suitable. However, this impression is often deceiving, firstly, because of the underestimation of the complexity of the task, and secondly, because of the presence of other advantages.

The peculiarity of specialized solutions is that they are specifically designed to solve a specific problem. This means that they take into account features that are difficult to take into account when solving a specific task along with the main one. It should be understood that in addition to functionality, the solution must also meet non-functional requirements, such as requirements for performance, safety, ease of maintenance, compatibility and others. All these features require global immersion in the task and, if developed, can be overlooked due to limited time.

Often specialized solutions are public and therefore are used by many developers. This means they have already identified and eliminated the shortcomings that they encountered during practical use. Thus, specialized solutions allow you to use someone else's experience without additional labor costs.

Public specialized solutions often have an open source code. In fact, these are the majority of libraries in Python. Such solutions are constantly being developed by the developer community, so it is possible to improve the quality of your software without additional labor costs by updating the versions used.

Often, different public solutions are easily integrated with each other (for example, Spyne and SQLAlchemy), which significantly reduce development costs.

There are a number of valid reasons for not using a specialized solution. Examples of such reasons:

• The specialized solution lacks the required functionality.
• A specialized solution is poorly documented, and access to the source code is impossible or limited, or the source code is very difficult to use instead of documentation.
• A specialized solution contains serious errors and is poorly supported, or there is no access to the source code or the possibility of self-correction of errors.
• Support for a specialized solution officially discontinued
• The current needs for the functionality of a specialized solution make up an extremely small fraction of all the functionality embedded in a specialized solution.

Performance

Code development performance overrides code execution performance.

In modern conditions, the cost of human resources is usually higher than the cost of hardware, so for me the code development performance (expressed in terms of the developed functionality per unit of time) takes precedence over the code execution performance.

This principle is opposed to premature optimization of code execution performance.

Optimization of the performance of code execution should be justified by the corresponding need.

The reasoning is to formulate performance requirements that are based on the practical or planned scale of the use of the information system, and the subsequent identification of the inconsistency of the information system with these requirements.

This principle is opposed to premature optimization of code execution performance.

Optimization of code execution performance should be performed by eliminating the most serious bottlenecks.

To do this, it is necessary to identify bottlenecks, evaluate the contribution of each bottleneck to the decrease in performance and optimize the bottleneck in the first place.

This principle is opposed to the coat optimizing the performance of code execution.

First of all, the most effective methods for optimizing the performance of code execution should be used.

The effectiveness of optimization methods is the ratio of the share of productivity gains to the total cost of performing optimization work. For me, the following prioritized list of optimization methods is basic:

1. Changing the data processing algorithm (can give an increase in productivity up to several orders of magnitude with relatively low development costs)
2. The increase in hardware power (may give an increase in performance from tens of percent to several dozen times depending on the nature of performance degradation; this method may require making changes to the architecture of the information system to enable scalability)
3. Updating the finished application and system software used in the information system (implies low costs, except in the case of paid software, but this rarely gives a significant performance gain, since application and system software is usually initially optimized)
4. Changing the development language in terms of bottlenecks, or a complete transition to a more productive programming language (high development costs with an increase in productivity from tens of percent to several dozen times depending on the nature of the functionality of the information system and the programming languages ​​used)

The prioritization of methods is that first of all labor and other resources are spent on using higher priority methods, and the transition to the next method is carried out when optimization opportunities are exhausted, or the efficiency of the less priority method is higher.

This principle is opposed to the coat optimizing the performance of code execution.

Conclusion

I deliberately excluded some of the principles from the publication, since they relate more to the management of people, communication between people and the construction of development processes, than directly to the writing of source code. I simply couldn’t identify or formulate some principles, since the translation of unintelligible knowledge into conscious knowledge is a rather complicated process. Therefore, I have high hopes that comments on the article will be no less interesting than the article itself, and will allow me to write a sequel, which will include missing information.