ConfigParser and Unicode

In Python, there is a very handy module for saving and reading ini-like configuration files, called ConfigParser .

I had a problem with its use, associated with saving Unicode strings to a file. In some hardly perceptible cases (for example, this manifested itself when I was running an application under Windows XP), when reading or writing such parameters, a string conversion error appears.

I did not manage to find ready-made solutions on the Internet, although there are quite a few questions about “how to make it always work” - they usually respond in the spirit of “ask the author of the module to fix it”.
')
I want to offer my solution for those who use Python 2.X - it is quite simple and helps to solve this problem.



First, you need to inherit the RawConfigParser class by overriding the write () method — namely, replacing all str () calls with unicode () calls:

Copy Source | Copy HTML class UnicodeConfigParser ( ConfigParser .RawConfigParser): def __init__ (self, *args, **kwargs): ConfigParser .RawConfigParser. __init__ (self, *args, **kwargs) def write (self, fp): """Fixed for Unicode output""" if self ._defaults: fp. write ( "[%s]\n" % DEFAULTSECT) for (key, value) in self ._defaults.items(): fp. write ( "%s = %s\n" % (key, unicode (value).replace( '\n' , '\n\t' ))) fp. write ( "\n" ) for section in self ._sections: fp. write ( "[%s]\n" % section) for (key, value) in self ._sections[section].items(): if key != "__name__" : fp. write ( "%s = %s\n" % (key, unicode (value).replace( '\n' , '\n\t' ))) fp. write ( "\n" ) # This function is needed to override default lower-case conversion # of the parameter's names. They will be saved 'as is'. def optionxform (self, strOut): return strOut

1. Copy Source | Copy HTML class UnicodeConfigParser ( ConfigParser .RawConfigParser): def __init__ (self, *args, **kwargs): ConfigParser .RawConfigParser. __init__ (self, *args, **kwargs) def write (self, fp): """Fixed for Unicode output""" if self ._defaults: fp. write ( "[%s]\n" % DEFAULTSECT) for (key, value) in self ._defaults.items(): fp. write ( "%s = %s\n" % (key, unicode (value).replace( '\n' , '\n\t' ))) fp. write ( "\n" ) for section in self ._sections: fp. write ( "[%s]\n" % section) for (key, value) in self ._sections[section].items(): if key != "__name__" : fp. write ( "%s = %s\n" % (key, unicode (value).replace( '\n' , '\n\t' ))) fp. write ( "\n" ) # This function is needed to override default lower-case conversion # of the parameter's names. They will be saved 'as is'. def optionxform (self, strOut): return strOut
3. Copy Source | Copy HTML class UnicodeConfigParser ( ConfigParser .RawConfigParser): def __init__ (self, *args, **kwargs): ConfigParser .RawConfigParser. __init__ (self, *args, **kwargs) def write (self, fp): """Fixed for Unicode output""" if self ._defaults: fp. write ( "[%s]\n" % DEFAULTSECT) for (key, value) in self ._defaults.items(): fp. write ( "%s = %s\n" % (key, unicode (value).replace( '\n' , '\n\t' ))) fp. write ( "\n" ) for section in self ._sections: fp. write ( "[%s]\n" % section) for (key, value) in self ._sections[section].items(): if key != "__name__" : fp. write ( "%s = %s\n" % (key, unicode (value).replace( '\n' , '\n\t' ))) fp. write ( "\n" ) # This function is needed to override default lower-case conversion # of the parameter's names. They will be saved 'as is'. def optionxform (self, strOut): return strOut
4. Copy Source | Copy HTML class UnicodeConfigParser ( ConfigParser .RawConfigParser): def __init__ (self, *args, **kwargs): ConfigParser .RawConfigParser. __init__ (self, *args, **kwargs) def write (self, fp): """Fixed for Unicode output""" if self ._defaults: fp. write ( "[%s]\n" % DEFAULTSECT) for (key, value) in self ._defaults.items(): fp. write ( "%s = %s\n" % (key, unicode (value).replace( '\n' , '\n\t' ))) fp. write ( "\n" ) for section in self ._sections: fp. write ( "[%s]\n" % section) for (key, value) in self ._sections[section].items(): if key != "__name__" : fp. write ( "%s = %s\n" % (key, unicode (value).replace( '\n' , '\n\t' ))) fp. write ( "\n" ) # This function is needed to override default lower-case conversion # of the parameter's names. They will be saved 'as is'. def optionxform (self, strOut): return strOut
6. Copy Source | Copy HTML class UnicodeConfigParser ( ConfigParser .RawConfigParser): def __init__ (self, *args, **kwargs): ConfigParser .RawConfigParser. __init__ (self, *args, **kwargs) def write (self, fp): """Fixed for Unicode output""" if self ._defaults: fp. write ( "[%s]\n" % DEFAULTSECT) for (key, value) in self ._defaults.items(): fp. write ( "%s = %s\n" % (key, unicode (value).replace( '\n' , '\n\t' ))) fp. write ( "\n" ) for section in self ._sections: fp. write ( "[%s]\n" % section) for (key, value) in self ._sections[section].items(): if key != "__name__" : fp. write ( "%s = %s\n" % (key, unicode (value).replace( '\n' , '\n\t' ))) fp. write ( "\n" ) # This function is needed to override default lower-case conversion # of the parameter's names. They will be saved 'as is'. def optionxform (self, strOut): return strOut
7. Copy Source | Copy HTML class UnicodeConfigParser ( ConfigParser .RawConfigParser): def __init__ (self, *args, **kwargs): ConfigParser .RawConfigParser. __init__ (self, *args, **kwargs) def write (self, fp): """Fixed for Unicode output""" if self ._defaults: fp. write ( "[%s]\n" % DEFAULTSECT) for (key, value) in self ._defaults.items(): fp. write ( "%s = %s\n" % (key, unicode (value).replace( '\n' , '\n\t' ))) fp. write ( "\n" ) for section in self ._sections: fp. write ( "[%s]\n" % section) for (key, value) in self ._sections[section].items(): if key != "__name__" : fp. write ( "%s = %s\n" % (key, unicode (value).replace( '\n' , '\n\t' ))) fp. write ( "\n" ) # This function is needed to override default lower-case conversion # of the parameter's names. They will be saved 'as is'. def optionxform (self, strOut): return strOut
8. Copy Source | Copy HTML class UnicodeConfigParser ( ConfigParser .RawConfigParser): def __init__ (self, *args, **kwargs): ConfigParser .RawConfigParser. __init__ (self, *args, **kwargs) def write (self, fp): """Fixed for Unicode output""" if self ._defaults: fp. write ( "[%s]\n" % DEFAULTSECT) for (key, value) in self ._defaults.items(): fp. write ( "%s = %s\n" % (key, unicode (value).replace( '\n' , '\n\t' ))) fp. write ( "\n" ) for section in self ._sections: fp. write ( "[%s]\n" % section) for (key, value) in self ._sections[section].items(): if key != "__name__" : fp. write ( "%s = %s\n" % (key, unicode (value).replace( '\n' , '\n\t' ))) fp. write ( "\n" ) # This function is needed to override default lower-case conversion # of the parameter's names. They will be saved 'as is'. def optionxform (self, strOut): return strOut
9. Copy Source | Copy HTML class UnicodeConfigParser ( ConfigParser .RawConfigParser): def __init__ (self, *args, **kwargs): ConfigParser .RawConfigParser. __init__ (self, *args, **kwargs) def write (self, fp): """Fixed for Unicode output""" if self ._defaults: fp. write ( "[%s]\n" % DEFAULTSECT) for (key, value) in self ._defaults.items(): fp. write ( "%s = %s\n" % (key, unicode (value).replace( '\n' , '\n\t' ))) fp. write ( "\n" ) for section in self ._sections: fp. write ( "[%s]\n" % section) for (key, value) in self ._sections[section].items(): if key != "__name__" : fp. write ( "%s = %s\n" % (key, unicode (value).replace( '\n' , '\n\t' ))) fp. write ( "\n" ) # This function is needed to override default lower-case conversion # of the parameter's names. They will be saved 'as is'. def optionxform (self, strOut): return strOut
10. Copy Source | Copy HTML class UnicodeConfigParser ( ConfigParser .RawConfigParser): def __init__ (self, *args, **kwargs): ConfigParser .RawConfigParser. __init__ (self, *args, **kwargs) def write (self, fp): """Fixed for Unicode output""" if self ._defaults: fp. write ( "[%s]\n" % DEFAULTSECT) for (key, value) in self ._defaults.items(): fp. write ( "%s = %s\n" % (key, unicode (value).replace( '\n' , '\n\t' ))) fp. write ( "\n" ) for section in self ._sections: fp. write ( "[%s]\n" % section) for (key, value) in self ._sections[section].items(): if key != "__name__" : fp. write ( "%s = %s\n" % (key, unicode (value).replace( '\n' , '\n\t' ))) fp. write ( "\n" ) # This function is needed to override default lower-case conversion # of the parameter's names. They will be saved 'as is'. def optionxform (self, strOut): return strOut
11. Copy Source | Copy HTML class UnicodeConfigParser ( ConfigParser .RawConfigParser): def __init__ (self, *args, **kwargs): ConfigParser .RawConfigParser. __init__ (self, *args, **kwargs) def write (self, fp): """Fixed for Unicode output""" if self ._defaults: fp. write ( "[%s]\n" % DEFAULTSECT) for (key, value) in self ._defaults.items(): fp. write ( "%s = %s\n" % (key, unicode (value).replace( '\n' , '\n\t' ))) fp. write ( "\n" ) for section in self ._sections: fp. write ( "[%s]\n" % section) for (key, value) in self ._sections[section].items(): if key != "__name__" : fp. write ( "%s = %s\n" % (key, unicode (value).replace( '\n' , '\n\t' ))) fp. write ( "\n" ) # This function is needed to override default lower-case conversion # of the parameter's names. They will be saved 'as is'. def optionxform (self, strOut): return strOut
12. Copy Source | Copy HTML class UnicodeConfigParser ( ConfigParser .RawConfigParser): def __init__ (self, *args, **kwargs): ConfigParser .RawConfigParser. __init__ (self, *args, **kwargs) def write (self, fp): """Fixed for Unicode output""" if self ._defaults: fp. write ( "[%s]\n" % DEFAULTSECT) for (key, value) in self ._defaults.items(): fp. write ( "%s = %s\n" % (key, unicode (value).replace( '\n' , '\n\t' ))) fp. write ( "\n" ) for section in self ._sections: fp. write ( "[%s]\n" % section) for (key, value) in self ._sections[section].items(): if key != "__name__" : fp. write ( "%s = %s\n" % (key, unicode (value).replace( '\n' , '\n\t' ))) fp. write ( "\n" ) # This function is needed to override default lower-case conversion # of the parameter's names. They will be saved 'as is'. def optionxform (self, strOut): return strOut
13. Copy Source | Copy HTML class UnicodeConfigParser ( ConfigParser .RawConfigParser): def __init__ (self, *args, **kwargs): ConfigParser .RawConfigParser. __init__ (self, *args, **kwargs) def write (self, fp): """Fixed for Unicode output""" if self ._defaults: fp. write ( "[%s]\n" % DEFAULTSECT) for (key, value) in self ._defaults.items(): fp. write ( "%s = %s\n" % (key, unicode (value).replace( '\n' , '\n\t' ))) fp. write ( "\n" ) for section in self ._sections: fp. write ( "[%s]\n" % section) for (key, value) in self ._sections[section].items(): if key != "__name__" : fp. write ( "%s = %s\n" % (key, unicode (value).replace( '\n' , '\n\t' ))) fp. write ( "\n" ) # This function is needed to override default lower-case conversion # of the parameter's names. They will be saved 'as is'. def optionxform (self, strOut): return strOut
14. Copy Source | Copy HTML class UnicodeConfigParser ( ConfigParser .RawConfigParser): def __init__ (self, *args, **kwargs): ConfigParser .RawConfigParser. __init__ (self, *args, **kwargs) def write (self, fp): """Fixed for Unicode output""" if self ._defaults: fp. write ( "[%s]\n" % DEFAULTSECT) for (key, value) in self ._defaults.items(): fp. write ( "%s = %s\n" % (key, unicode (value).replace( '\n' , '\n\t' ))) fp. write ( "\n" ) for section in self ._sections: fp. write ( "[%s]\n" % section) for (key, value) in self ._sections[section].items(): if key != "__name__" : fp. write ( "%s = %s\n" % (key, unicode (value).replace( '\n' , '\n\t' ))) fp. write ( "\n" ) # This function is needed to override default lower-case conversion # of the parameter's names. They will be saved 'as is'. def optionxform (self, strOut): return strOut
15. Copy Source | Copy HTML class UnicodeConfigParser ( ConfigParser .RawConfigParser): def __init__ (self, *args, **kwargs): ConfigParser .RawConfigParser. __init__ (self, *args, **kwargs) def write (self, fp): """Fixed for Unicode output""" if self ._defaults: fp. write ( "[%s]\n" % DEFAULTSECT) for (key, value) in self ._defaults.items(): fp. write ( "%s = %s\n" % (key, unicode (value).replace( '\n' , '\n\t' ))) fp. write ( "\n" ) for section in self ._sections: fp. write ( "[%s]\n" % section) for (key, value) in self ._sections[section].items(): if key != "__name__" : fp. write ( "%s = %s\n" % (key, unicode (value).replace( '\n' , '\n\t' ))) fp. write ( "\n" ) # This function is needed to override default lower-case conversion # of the parameter's names. They will be saved 'as is'. def optionxform (self, strOut): return strOut
16. Copy Source | Copy HTML class UnicodeConfigParser ( ConfigParser .RawConfigParser): def __init__ (self, *args, **kwargs): ConfigParser .RawConfigParser. __init__ (self, *args, **kwargs) def write (self, fp): """Fixed for Unicode output""" if self ._defaults: fp. write ( "[%s]\n" % DEFAULTSECT) for (key, value) in self ._defaults.items(): fp. write ( "%s = %s\n" % (key, unicode (value).replace( '\n' , '\n\t' ))) fp. write ( "\n" ) for section in self ._sections: fp. write ( "[%s]\n" % section) for (key, value) in self ._sections[section].items(): if key != "__name__" : fp. write ( "%s = %s\n" % (key, unicode (value).replace( '\n' , '\n\t' ))) fp. write ( "\n" ) # This function is needed to override default lower-case conversion # of the parameter's names. They will be saved 'as is'. def optionxform (self, strOut): return strOut
17. Copy Source | Copy HTML class UnicodeConfigParser ( ConfigParser .RawConfigParser): def __init__ (self, *args, **kwargs): ConfigParser .RawConfigParser. __init__ (self, *args, **kwargs) def write (self, fp): """Fixed for Unicode output""" if self ._defaults: fp. write ( "[%s]\n" % DEFAULTSECT) for (key, value) in self ._defaults.items(): fp. write ( "%s = %s\n" % (key, unicode (value).replace( '\n' , '\n\t' ))) fp. write ( "\n" ) for section in self ._sections: fp. write ( "[%s]\n" % section) for (key, value) in self ._sections[section].items(): if key != "__name__" : fp. write ( "%s = %s\n" % (key, unicode (value).replace( '\n' , '\n\t' ))) fp. write ( "\n" ) # This function is needed to override default lower-case conversion # of the parameter's names. They will be saved 'as is'. def optionxform (self, strOut): return strOut
18. Copy Source | Copy HTML class UnicodeConfigParser ( ConfigParser .RawConfigParser): def __init__ (self, *args, **kwargs): ConfigParser .RawConfigParser. __init__ (self, *args, **kwargs) def write (self, fp): """Fixed for Unicode output""" if self ._defaults: fp. write ( "[%s]\n" % DEFAULTSECT) for (key, value) in self ._defaults.items(): fp. write ( "%s = %s\n" % (key, unicode (value).replace( '\n' , '\n\t' ))) fp. write ( "\n" ) for section in self ._sections: fp. write ( "[%s]\n" % section) for (key, value) in self ._sections[section].items(): if key != "__name__" : fp. write ( "%s = %s\n" % (key, unicode (value).replace( '\n' , '\n\t' ))) fp. write ( "\n" ) # This function is needed to override default lower-case conversion # of the parameter's names. They will be saved 'as is'. def optionxform (self, strOut): return strOut
19. Copy Source | Copy HTML class UnicodeConfigParser ( ConfigParser .RawConfigParser): def __init__ (self, *args, **kwargs): ConfigParser .RawConfigParser. __init__ (self, *args, **kwargs) def write (self, fp): """Fixed for Unicode output""" if self ._defaults: fp. write ( "[%s]\n" % DEFAULTSECT) for (key, value) in self ._defaults.items(): fp. write ( "%s = %s\n" % (key, unicode (value).replace( '\n' , '\n\t' ))) fp. write ( "\n" ) for section in self ._sections: fp. write ( "[%s]\n" % section) for (key, value) in self ._sections[section].items(): if key != "__name__" : fp. write ( "%s = %s\n" % (key, unicode (value).replace( '\n' , '\n\t' ))) fp. write ( "\n" ) # This function is needed to override default lower-case conversion # of the parameter's names. They will be saved 'as is'. def optionxform (self, strOut): return strOut
21. Copy Source | Copy HTML class UnicodeConfigParser ( ConfigParser .RawConfigParser): def __init__ (self, *args, **kwargs): ConfigParser .RawConfigParser. __init__ (self, *args, **kwargs) def write (self, fp): """Fixed for Unicode output""" if self ._defaults: fp. write ( "[%s]\n" % DEFAULTSECT) for (key, value) in self ._defaults.items(): fp. write ( "%s = %s\n" % (key, unicode (value).replace( '\n' , '\n\t' ))) fp. write ( "\n" ) for section in self ._sections: fp. write ( "[%s]\n" % section) for (key, value) in self ._sections[section].items(): if key != "__name__" : fp. write ( "%s = %s\n" % (key, unicode (value).replace( '\n' , '\n\t' ))) fp. write ( "\n" ) # This function is needed to override default lower-case conversion # of the parameter's names. They will be saved 'as is'. def optionxform (self, strOut): return strOut
22. Copy Source | Copy HTML class UnicodeConfigParser ( ConfigParser .RawConfigParser): def __init__ (self, *args, **kwargs): ConfigParser .RawConfigParser. __init__ (self, *args, **kwargs) def write (self, fp): """Fixed for Unicode output""" if self ._defaults: fp. write ( "[%s]\n" % DEFAULTSECT) for (key, value) in self ._defaults.items(): fp. write ( "%s = %s\n" % (key, unicode (value).replace( '\n' , '\n\t' ))) fp. write ( "\n" ) for section in self ._sections: fp. write ( "[%s]\n" % section) for (key, value) in self ._sections[section].items(): if key != "__name__" : fp. write ( "%s = %s\n" % (key, unicode (value).replace( '\n' , '\n\t' ))) fp. write ( "\n" ) # This function is needed to override default lower-case conversion # of the parameter's names. They will be saved 'as is'. def optionxform (self, strOut): return strOut
23. Copy Source | Copy HTML class UnicodeConfigParser ( ConfigParser .RawConfigParser): def __init__ (self, *args, **kwargs): ConfigParser .RawConfigParser. __init__ (self, *args, **kwargs) def write (self, fp): """Fixed for Unicode output""" if self ._defaults: fp. write ( "[%s]\n" % DEFAULTSECT) for (key, value) in self ._defaults.items(): fp. write ( "%s = %s\n" % (key, unicode (value).replace( '\n' , '\n\t' ))) fp. write ( "\n" ) for section in self ._sections: fp. write ( "[%s]\n" % section) for (key, value) in self ._sections[section].items(): if key != "__name__" : fp. write ( "%s = %s\n" % (key, unicode (value).replace( '\n' , '\n\t' ))) fp. write ( "\n" ) # This function is needed to override default lower-case conversion # of the parameter's names. They will be saved 'as is'. def optionxform (self, strOut): return strOut
24. Copy Source | Copy HTML class UnicodeConfigParser ( ConfigParser .RawConfigParser): def __init__ (self, *args, **kwargs): ConfigParser .RawConfigParser. __init__ (self, *args, **kwargs) def write (self, fp): """Fixed for Unicode output""" if self ._defaults: fp. write ( "[%s]\n" % DEFAULTSECT) for (key, value) in self ._defaults.items(): fp. write ( "%s = %s\n" % (key, unicode (value).replace( '\n' , '\n\t' ))) fp. write ( "\n" ) for section in self ._sections: fp. write ( "[%s]\n" % section) for (key, value) in self ._sections[section].items(): if key != "__name__" : fp. write ( "%s = %s\n" % (key, unicode (value).replace( '\n' , '\n\t' ))) fp. write ( "\n" ) # This function is needed to override default lower-case conversion # of the parameter's names. They will be saved 'as is'. def optionxform (self, strOut): return strOut

Secondly, the writing and reading of the configuration file should be done with a wrapper for open () from the codecs module, which needs to specify utf-8 as the encoding. In the case of a download, this can be done by using not read () for reading, but readfp () :

Copy Source | Copy HTML

1. import codecs
3. # Saving
5. confFile = codecs . open ( 'myConfig.ini' , 'w' , 'utf-8' )
6. config = UnicodeConfigParser ()
7. # ...
8. config.write (confFile)
9. confFile.close ()
11. # Loading
13. config = UnicodeConfigParser ()
14. config.readfp ( codecs . open ( 'myConfig.ini' , "r" , "utf-8" ))

I hope someone will come in handy. If you have a more beautiful and successful solution, I will be glad to hear it.