
In the course of one of my Python projects, with a large admixture of OOP and processing a large amount of data, I wondered how efficiently to process lists in a class using calls to its methods, or can it use a call to an external function? For this, 24 tests were written that showed very interesting results. Who cares about this topic, please in details.
Test sample:creates a list of 500 thousand items and is filled with random values
lst = [] for i in range(1, 500000): val = random() + i lst.append(val)
')
Processing in 2 steps:1) first getting a new list by squaring each element of the list (in several ways)
2) then adding to the list of some constant, which can be: local, globally, an attribute (for a class)
Like so
@howlong def process_list_global_func_plus_local_value(): """. """ local_plus_value = GLOBAL_VALUE new_lst = [] for i in lst: new_lst.append(global_func(i)) for v in new_lst: v + local_plus_value
PerformancePerformance was understood to mean the duration of each test, measured by the decorator.
import time def howlong(f): def tmp(*args, **kwargs): t = time.time() res = f(*args, **kwargs) need_time = time.time()-t tmp.__name__ = f.__name__ tmp.__doc__ = f.__doc__
The following results were obtained for two groups of tests:Function. Processing by the Generator without calling external functions and adding a local variable - 0.192 - 100%
Function. Processing by the Generator without calling external functions and adding a global variable - 0.2 - 104%
Function. Processing in a loop without calling external functions and adding a local variable - 0.238 - 123%
Function. Processing in a loop without calling external functions and adding a global variable - 0.245 - 127%
Function. Processing using map and adding a local variable - 0.25 - 130%
Function. Processing using map and adding a global variable - 0.255 - 132%
Function. Processing by the Generator with calling a local function and adding a local variable - 0.258 - 134%
Function. Processing by the Generator with a call to a global function and the addition of a global variable. - 0.274 - 142%
Function. Processing in a loop with calling a local function and adding a local variable - 0.312 - 162%
Function. Handling in a loop with calling a local function and adding a global variable - 0.32 - 166%
Function. Handling in a loop with calling a global function and adding a global variable - 0.327 - 170%
Function. Handling in a loop with calling a global function and adding a local variable - 0.332 - 172%
The class. Processing by the Generator without calling external functions and adding a local variable - 0.191 - 100%
The class. Processing by the Generator without calling external functions and adding the value of the global lane. - 0.20 - 104%
The class. Processing by the Generator without calling external functions and adding the attribute value - 0.213 - 111%
The class. Processing by calling a local function and adding a local variable - 0.312 - 163%
The class. Handling by calling a global function and adding a local variable - 0.318 - 166%
The class. Handling by calling a local function and adding a global variable - 0.318 - 166%
The class. Handling by calling a global function and adding a global variable - 0.328 - 171%
The class. Handling by calling a local function and adding an attribute value - 0.333 - 174%
The class. Handling by calling a global function and adding an attribute value - 0.34 - 178%
The class. Handling by calling a class method and adding a local variable - 0.39 - 204%
The class. Processing by calling the class method and adding a global variable - 0.398 - 208%
The class. Processing by calling the class method and adding the attribute value is 0.411 - 215%
Findings:Of most interest are the percentage differences.
By processing the list function1) The fastest list processing - in the generator without calling external functions and using local variables, using global variables will be ~ 5% longer
2) when processing a list in a loop for a cycle, the operation time will be 23% using local variables and 27% longer using global variables than when processing through a generator
3) when used in a for loop to process a list of external local or global functions, the speed will be lower by more than 60%
4) when using map as a function of processing list elements, it is almost the same as when using the call of an external function in the generator
5) The longest way to handle a list is to loop through an external global function call.
By processing the list in class1) in classes, access to attributes is longer than local variables, about 10%, and 5% longer than global variables
2) in classes, access to methods is longer than local functions by about 22% and 20% longer than global functions
3) when processing the list in the class using the generator is obtained as quickly as using the function with the generator
4) the longest way (more than 2 times longer) was using a method call in a loop and adding an attribute value, which was a great surprise for me
The code is available in one file at http://pastebin.com/rgdXNdXbNext I plan to investigate the speed of access to the elements of lists, types, various types of dictionaries, and then questions of the expediency of caching functions and methods of classes.