Ultimate benchmark five and a half ways to check for an object attribute in Python

This is where the question of how to determine whether an object has an attribute and how to do it as quickly as possible was raised, but the topic has not been studied deep enough.


This, in fact, was the reason for writing this short article. For testing, I chose the following (known to me) ways to determine the presence of an attribute:

1. Perhaps the most obvious is to use the hasattr(obj, name) built-in function.
2. Another common way is to try to access the attribute and, if it fails, take action by processing an AttributeError .
3. Also use AttributeError processing, however, you can access the property not directly, but through getattr(obj, name) . The situation looks far-fetched, but in real-life situations there are situations where the name of the attribute to check is formed dynamically and getattr there.
4. A very fast (see below test results) method is to look at the __dict__ object (if it has one, of course). The problem with applying this method is, perhaps, only that __dict__ 's are separate for an instance of a class, the class itself, and all of its ancestors. If you do not know exactly where the attribute we need is located, this method has no practical value for us. It should be noted that you can also look at __dict__ in two ways - using the methods dict .has_key(key_name) and dict .__contains__(key_name) , which corresponds to the keyword in ( assert 'My Key Name' in my_dict ). Taking into account all the advantages, disadvantages and two options for implementing __dict__ , it does not draw two separate methods, we consider it “one and a half”.
5. The last, most exotic method is to look at the dir(obj) for the name of the attribute we need. By the way, in the process of checking nested __slots__ -classes, some interesting points were discovered related to dir() , but more on that in a separate article :)

With methods, I hope, understood. In order to get a more realistic situation, I created 3 classes that are inherited by the “chain” - TestClass from TestClass2 , which, in turn, from TestClass3 , whose ancestor is object . Each class has “instance attribute” with the name of the form c3_ ia , assigned in the class constructor, and “class attribute” c2_ ca , which is determined at the compilation stage of the class.

In each test, I try to get the “instance attribute” and “class attribute” of the top level TestClass , the “instance attribute” and “class attribute” defined in the class TestClass3 and some non-existent attribute fake .
')
All tests were run 10'000'000 times. The amount of time spent on performing these 10M identical operations, and is considered the time of the test. To no one was hurt, the total time is calculated equally, for existing and non-existing attributes.

Look like that's it. Now the results.

:
dict_lookup_contains : 5.800250 [2 subtests failed]
dict_lookup : 7.672500 [2 subtests failed]
hasattr : 12.171750 [0 subtests failed]
exc_direct : 27.785500 [0 subtests failed]
exc_getattr : 32.088875 [0 subtests failed]
dir : 267.500500 [0 subtests failed]

:
test_dict_lookup_true_this_ca : FAILED [AssertionError()]
test_dict_lookup_true_parent_ca : FAILED [AssertionError()]
test_dict_lookup_contains_true_this_ca : FAILED [AssertionError()]
test_dict_lookup_contains_true_parent_ca : FAILED [AssertionError()]
test_exc_direct_true_this_ca : 5.133000
test_exc_direct_true_parent_ca : 5.710000
test_dict_lookup_contains_true_parent_ia : 5.789000
test_dict_lookup_contains_false : 5.804000
test_dict_lookup_contains_true_this_ia : 5.804000
test_exc_direct_true_this_ia : 6.037000
test_exc_direct_true_parent_ia : 6.412000
test_hasattr_true_this_ca : 6.615000
test_exc_getattr_true_this_ca : 7.144000
test_hasattr_true_this_ia : 7.193000
test_hasattr_true_parent_ca : 7.240000
test_dict_lookup_false : 7.614000
test_dict_lookup_true_this_ia : 7.645000
test_exc_getattr_true_this_ia : 7.769000
test_dict_lookup_true_parent_ia : 7.817000
test_hasattr_true_parent_ia : 7.926000
test_exc_getattr_true_parent_ca : 8.003000
test_exc_getattr_true_parent_ia : 8.691000
test_hasattr_false : 17.100000
test_exc_direct_false : 49.748000
test_exc_getattr_false : 56.276000
test_dir_true_this_ia : 266.847000
test_dir_true_this_ca : 267.053000
test_dir_false : 267.398000
test_dir_true_parent_ca : 267.849000
test_dir_true_parent_ia : 268.663000

In principle, there is nothing special to comment on - the table speaks for itself. Brief summary:

• Search in __dict__ through in - the best solution, if you know exactly where we are looking.
• hasattr shows a consistently smooth operation for any requests, it is very good to use when the probability that the attribute will not exist.
• try/except + a direct property request runs quickly when no exception happens, otherwise it sneezes a lot ( test_exc_direct_false worked for 49.748 seconds!). Conclusion - you can use when the probability that the attribute will be where it should be, is very, very high.
• dir - well-deserved sloppy Python. To use it for the purpose of checking the presence of an attribute is a firing article.

Here is the tester source code:

#!/usr/bin/env python
# coding: utf8

import time

__times__ = 10000000

def timeit ( func, res ) :
'' 'Check if ' func ' returns ' res ', if true, execute it ' __times__ ' times (__times__ should be defined in parent namespace) measuring elapsed time.' ''
assert func ( ) == res

t_start = time . clock ( )
for i in xrange ( __times__ ) :
func ( )
return time . clock ( ) - t_start

# Define test classes and create instance of top-level class.
class TestClass3 ( object ) :
c3_ca = 1

def __init__ ( self ) :
self . c3_ia = 1

class TestClass2 ( TestClass3 ) :
c2_ca = 1

def __init__ ( self ) :
TestClass3. __init__ ( self )
self . c2_ia = 2

class TestClass ( TestClass2 ) :
c1_ca = 1

def __init__ ( self ) :
TestClass2. __init__ ( self )
self . c1_ia = 2

obj = TestClass ( )

# Legend:
#
# hasattr, exc_direct, exc_getattr, dict_lookup, dict_lookup_contains, dir - attribute accessing methods.
# true, false - if 'true' we are checking for really existing attribute.
# this, parent - if 'this' we are looking for attribute in the top-level class, otherwise in the top-level class' parent's parent.
# ca, ia - test class attribute ('ca') or instance attribute ('ia') access.
#
# Note about __dict__ lookups: they are not suitable for generic attribute lookup because instance's __dict__ stores only instance's attributes. To look for class attributes we should query them from class' __dict__.

# Test query through hasattr
def test_hasattr_true_this_ca ( ) :
return hasattr ( obj, 'c1_ca' )

def test_hasattr_true_this_ia ( ) :
return hasattr ( obj, 'c1_ia' )

def test_hasattr_true_parent_ca ( ) :
return hasattr ( obj, 'c3_ca' )

def test_hasattr_true_parent_ia ( ) :
return hasattr ( obj, 'c3_ia' )

def test_hasattr_false ( ) :
return hasattr ( obj, 'fake' )

# Test direct access to attribute inside try/except
def test_exc_direct_true_this_ca ( ) :
try :
obj. c1_ca
return True
except AttributeError :
return False

def test_exc_direct_true_this_ia ( ) :
try :
obj. c1_ia
return True
except AttributeError :
return False

def test_exc_direct_true_parent_ca ( ) :
try :
obj. c3_ca
return True
except AttributeError :
return False

def test_exc_direct_true_parent_ia ( ) :
try :
obj. c3_ia
return True
except AttributeError :
return False

def test_exc_direct_false ( ) :
try :
obj. fake
return True
except AttributeError :
return False

# Test getattr access to attribute inside try/except
def test_exc_getattr_true_this_ca ( ) :
try :
getattr ( obj, 'c1_ca' )
return True
except AttributeError :
return False

def test_exc_getattr_true_this_ia ( ) :
try :
getattr ( obj, 'c1_ia' )
return True
except AttributeError :
return False

def test_exc_getattr_true_parent_ca ( ) :
try :
getattr ( obj, 'c3_ca' )
return True
except AttributeError :
return False

def test_exc_getattr_true_parent_ia ( ) :
try :
getattr ( obj, 'c3_ia' )
return True
except AttributeError :
return False

def test_exc_getattr_false ( ) :
try :
getattr ( obj, 'fake' )
return True
except AttributeError :
return False

# Test attribute lookup in dir()
def test_dir_true_this_ca ( ) :
return 'c1_ca' in dir ( obj )

def test_dir_true_this_ia ( ) :
return 'c1_ia' in dir ( obj )

def test_dir_true_parent_ca ( ) :
return 'c3_ca' in dir ( obj )

def test_dir_true_parent_ia ( ) :
return 'c3_ia' in dir ( obj )

def test_dir_false ( ) :
return 'fake' in dir ( obj )

# Test attribute lookup in __dict__
def test_dict_lookup_true_this_ca ( ) :
return obj. __dict__ . has_key ( 'c1_ca' )

def test_dict_lookup_true_this_ia ( ) :
return obj. __dict__ . has_key ( 'c1_ia' )

def test_dict_lookup_true_parent_ca ( ) :
return obj. __dict__ . has_key ( 'c3_ca' )

def test_dict_lookup_true_parent_ia ( ) :
return obj. __dict__ . has_key ( 'c3_ia' )

def test_dict_lookup_false ( ) :
return obj. __dict__ . has_key ( 'fake' )

# Test attribute lookup in __dict__ through __contains__
def test_dict_lookup_contains_true_this_ca ( ) :
return 'c1_ca' in obj. __dict__

def test_dict_lookup_contains_true_this_ia ( ) :
return 'c1_ia' in obj. __dict__

def test_dict_lookup_contains_true_parent_ca ( ) :
return 'c3_ca' in obj. __dict__

def test_dict_lookup_contains_true_parent_ia ( ) :
return 'c3_ia' in obj. __dict__

def test_dict_lookup_contains_false ( ) :
return 'fake' in obj. __dict__

# TEST
tests = {
'hasattr' : {
'test_hasattr_true_this_ca' : True ,
'test_hasattr_true_this_ia' : True ,
'test_hasattr_true_parent_ca' : True ,
'test_hasattr_true_parent_ia' : True ,
'test_hasattr_false' : False ,
} ,
'exc_direct' : {
'test_exc_direct_true_this_ca' : True ,
'test_exc_direct_true_this_ia' : True ,
'test_exc_direct_true_parent_ca' : True ,
'test_exc_direct_true_parent_ia' : True ,
'test_exc_direct_false' : False ,
} ,
'exc_getattr' : {
'test_exc_getattr_true_this_ca' : True ,
'test_exc_getattr_true_this_ia' : True ,
'test_exc_getattr_true_parent_ca' : True ,
'test_exc_getattr_true_parent_ia' : True ,
'test_exc_getattr_false' : False ,
} ,
'dict_lookup' : {
'test_dict_lookup_true_this_ca' : True ,
'test_dict_lookup_true_this_ia' : True ,
'test_dict_lookup_true_parent_ca' : True ,
'test_dict_lookup_true_parent_ia' : True ,
'test_dict_lookup_false' : False ,
} ,
'dict_lookup_contains' : {
'test_dict_lookup_contains_true_this_ca' : True ,
'test_dict_lookup_contains_true_this_ia' : True ,
'test_dict_lookup_contains_true_parent_ca' : True ,
'test_dict_lookup_contains_true_parent_ia' : True ,
'test_dict_lookup_contains_false' : False ,
} ,
'dir' : {
'test_dir_true_this_ca' : True ,
'test_dir_true_this_ia' : True ,
'test_dir_true_parent_ca' : True ,
'test_dir_true_parent_ia' : True ,
'test_dir_false' : False ,
} ,
}

# Perform tests
results = { }
results_exc = { }

for ( test_group_name, test_group ) in tests. iteritems ( ) :
results_group = results [ test_group_name ] = { }
results_exc_group = results_exc [ test_group_name ] = { }
for ( test_name, test_expected_result ) in test_group. iteritems ( ) :
test_func = locals ( ) [ test_name ]
print '%s::%s...' % ( test_group_name, test_name )
try :
test_time = timeit ( test_func, test_expected_result )
results_group [ test_name ] = test_time
except Exception , exc:
results_group [ test_name ] = None
results_exc_group [ test_name ] = exc

# Process results
group_results = [ ]

for ( group_name, group_tests ) in results. iteritems ( ) :
group_true_time = 0.0
group_true_count = 0
group_false_time = 0.0
group_false_count = 0
group_fail_count = 0

for ( test_name, test_time ) in group_tests. iteritems ( ) :
if test_time is not None :
if tests [ group_name ] [ test_name ] :
group_true_count += 1
group_true_time += test_time
else :
group_false_count += 1
group_false_time += test_time
else :
group_fail_count += 1

group_time = ( group_true_time / group_true_count + group_false_time / group_false_count ) / 2
group_results. append ( ( group_name, group_time, group_fail_count ) )

group_results. sort ( key = lambda ( group_name, group_time, group_fail_count ) : group_time )

# Output results
print
print ' :'

for ( group_name, group_time, group_fail_count ) in group_results:
print '%-25s: %10f [%d subtests failed]' % ( group_name, group_time, group_fail_count )

print ' :'
all_results = [ ]
for ( group_name, group_tests ) in results. iteritems ( ) :
for ( test_name, test_time ) in group_tests. iteritems ( ) :
all_results. append ( ( group_name, test_name, test_time ) )
all_results. sort ( key = lambda ( group_name, test_name, test_time ) : test_time )

for ( group_name, test_name, test_time ) in all_results:
if test_time is not None :
print '%-50s: %10f' % ( test_name, test_time )
else :
print '%-50s: FAILED [%r]' % ( test_name, results_exc [ group_name ] [ test_name ] )


I almost forgot ... The computer on which the test was performed: CPU: Intel Pentium D CPU 3.40GHz (2 cores (but obviously only one was used)); RAM: 2Gb . If it is interesting to someone, of course.

Update # 1 (Test results for __getattribute__ and comparison with previous results):
The following class was used as a replacement for the previous chain:

__attributes__ = ( 'c1_ca' , 'c3_ca' , 'c1_ia' , 'c3_ia' )
class TestClass ( object ) :
def __getattribute__ ( self , name ) :
if name in __attributes__:
return 1
else :
raise AttributeError ( )

getattr(obj, name, None) is not None results (taking into account getattr(obj, name, None) is not None )

:
dict_lookup : n/a [5 subtests failed]
dict_lookup_contains : n/a [5 subtests failed]
hasattr : 20.181182 [0 subtests failed]
getattr : 26.283962 [0 subtests failed]
exc_direct : 41.779489 [0 subtests failed]
exc_getattr : 47.757879 [0 subtests failed]
dir : 98.622183 [4 subtests failed]

:
test_dir_true_parent_ia : FAILED [AssertionError()]
test_dir_true_this_ia : FAILED [AssertionError()]
test_dir_true_this_ca : FAILED [AssertionError()]
test_dir_true_parent_ca : FAILED [AssertionError()]
test_dict_lookup_true_parent_ia : FAILED [AttributeError()]
test_dict_lookup_true_this_ia : FAILED [AttributeError()]
test_dict_lookup_true_this_ca : FAILED [AttributeError()]
test_dict_lookup_true_parent_ca : FAILED [AttributeError()]
test_dict_lookup_false : FAILED [AttributeError()]
test_dict_lookup_contains_true_this_ia : FAILED [AttributeError()]
test_dict_lookup_contains_true_parent_ia : FAILED [AttributeError()]
test_dict_lookup_contains_true_parent_ca : FAILED [AttributeError()]
test_dict_lookup_contains_true_this_ca : FAILED [AttributeError()]
test_dict_lookup_contains_false : FAILED [AttributeError()]
test_exc_direct_true_this_ca : 13.346949
test_exc_direct_true_parent_ca : 13.970407
test_exc_direct_true_this_ia : 14.621696
test_hasattr_true_this_ca : 15.077735
test_exc_direct_true_parent_ia : 15.146182
test_exc_getattr_true_parent_ca : 16.305500
test_getattr_true_this_ia : 16.976973
test_hasattr_true_parent_ia : 17.196719
test_hasattr_true_parent_ca : 17.613231
test_getattr_true_this_ca : 18.331266
test_exc_getattr_true_parent_ia : 18.720518
test_hasattr_false : 21.983571
test_getattr_true_parent_ca : 22.087115
test_exc_getattr_true_this_ca : 23.072045
test_hasattr_true_this_ia : 23.627484
test_getattr_true_parent_ia : 24.474635
test_getattr_false : 32.100426
test_exc_getattr_true_this_ia : 34.555669
test_exc_direct_false : 69.287669
test_exc_getattr_false : 72.352324
test_dir_false : 98.622183

Now we are going to compare the results ...

Key: , [ ] | -- __getattribute__- | , __getattribute__- [ ] , [ ] | -- __getattribute__- | , __getattribute__- [ ]
( ):
dict_lookup : 7.672500 [2] | n/a -- n/a | n/a [5]
dict_lookup_contains : 5.800250 [2] | n/a -- n/a | n/a [5]
hasattr : 12.171750 [0] | 16.176466 -- 1.658035 | 20.181182 [0]
getattr : 15.350072 [0] | 20.817017 -- 1.712302 | 26.283962 [0]
exc_direct : 27.785500 [0] | 34.782495 -- 1.503644 | 41.779489 [0]
exc_getattr : 32.088875 [0] | 39.923377 -- 1.488300 | 47.757879 [0]
dir : 267.500500 [0] | 183.061342 -- 0.368680 | 98.622183 [4]

( ):
dict_lookup : 7.672500 [2] | n/a -- n/a | n/a [5]
dict_lookup_contains : 5.800250 [2] | n/a -- n/a | n/a [5]
dir : 267.500500 [0] | 183.061342 -- 0.368680 | 98.622183 [4]
exc_getattr : 32.088875 [0] | 39.923377 -- 1.488300 | 47.757879 [0]
exc_direct : 27.785500 [0] | 34.782495 -- 1.503644 | 41.779489 [0]
hasattr : 12.171750 [0] | 16.176466 -- 1.658035 | 20.181182 [0]
getattr : 15.350072 [0] | 20.817017 -- 1.712302 | 26.283962 [0]

Key: , | -- __getattribute__- | , __getattribute__- , | -- __getattribute__- | , __getattribute__-
( ):
test_dict_lookup_true_parent_ia