Story of one optimization
It will be about unity-launcher-editor - the editor of elements and context menu (quicklists) of the Unity panel for Ubuntu. The editor is written using the python + gtk bundle, and it tolerably tolerates the responsibilities. Annoying startup time: it takes an inadmissibly long time until the main window appears.
You can blame Python, Unity or developers, and you can try to figure out what's the matter, the project is open source. During the "study" made notes, which formed the basis of this note. Curious please under the cat.
Look under the hood using the standard profiler:
Result:
Wow! 50 seconds to launch a small application.
')
Apparently, something amiss is happening in the
A quick scan of the code confirms the guess that the
Let's start with
Apparently in the panel there are 24 elements for each of which
The first thing that catches your eye is the line:
The image-missing icon is one for all, so there’s no need to look for it every time. Let's make it an attribute of a class. In the
and replace the corresponding line in
Let's see how the launch time has changed:
Result2:
The result is somewhat unexpected, so I rechecked (with and without changes): consistently minus 35-40 seconds.
9 seconds is much less annoying, but nonetheless annoying.
Let's try to improve.
IconManager is created as a separate object each time (and always without arguments), we will make it an attribute of the class.
Add to
All occurrences of
Checking:
Result3:
The window appears almost instantly. The total time is almost two seconds, but it also includes closing the window with the mouse. It is better to focus on the line
At this you can stop.
Small changes led to a decrease in the launch time of the application from 46 seconds to 0.3 (for
Changes sent to developers.
Open source is not perfect, but everyone can improve it by spending quite a bit of their time.
UPD:
Two issues remained unclear:
The indirect culprit was the object
The IconManager constructor expands the list of paths in which the icon search takes place. All changes are saved for the duration of the application, although Gtk.IconTheme.get_default () is not a Singleton. In my case, each IconManager initialization increases the list by 67 items, which leads to a change in its size by about 300 times (from 10 to 2985).
By changing a few lines in the constructor, you can reduce the start time to 0.8 seconds (without previous optimizations) and 0.6 (with optimizations). Previous changes have become "cosmetic".
Bug created , patch sent .
You can blame Python, Unity or developers, and you can try to figure out what's the matter, the project is open source. During the "study" made notes, which formed the basis of this note. Curious please under the cat.
Step 1
Look under the hood using the standard profiler:
python -m cProfile -s cumulative uleResult:
211267 function calls (203946 primitive calls) in 49.801 seconds Ordered by: cumulative time ncalls tottime percall cumtime percall filename:lineno(function) 1 0.002 0.002 49.801 49.801 ule:19(<module>) 1 0.000 0.000 49.415 49.415 ule:29(main) 4598/4584 0.007 0.000 49.224 0.011 types.py:46(function) 4599/4585 49.208 0.011 49.218 0.011 {method 'invoke' of 'gi.CallableInfo' objects} 1 0.001 0.001 46.013 46.013 app.py:47(__init__) 1 0.001 0.001 45.945 45.945 app.py:230(__populate) 24 0.007 0.000 45.936 1.914 app.py:211(new_from_file) 45 0.002 0.000 45.454 1.010 iconmanager.py:93(get_icon) 45 0.014 0.000 0.408 0.009 iconmanager.py:29(__init__) 1 0.000 0.000 0.343 0.343 iconmanager.py:71(get_icon_with_file) 1 0.000 0.000 0.343 0.343 iconmanager.py:60(get_icon_with_name) ................................................................................ Wow! 50 seconds to launch a small application.
')
Step 2
Apparently, something amiss is happening in the
app.py module:app.py:230(__populate) - called once, app.py:230(__populate) 46 seconds to complete a callapp.py:211(new_from_file) - called 24 times, approximately 2 seconds for each calliconmanager.py:93(get_icon) - called 45 times, approximately 1 second per callA quick scan of the code confirms the guess that the
__populate function calls new_form_file , which calls get_icon .Let's start with
__populate : def __populate(self): self.launcher_view.clear_list(); self.unity = self.gsettings.get_value('favorites') log.info(self.unity) for menu_item in self.unity: self.new_from_file(menu_item, False) self.launcher_view.connect('selection-changed', self.__launcher_view_row_change) #make first row active self.launcher_view.set_selected_iter(0) Apparently in the panel there are 24 elements for each of which
new_from_file(menu_item, False) is called def new_from_file(self, filename, selected=True): try: file_path = normalize_path(filename, True) obj = DesktopParser(file_path) sname = obj.get('Name',locale=LOCALE) desc = obj.get('Comment',locale=LOCALE) icon = obj.get('Icon') pix = IconManager().get_icon(ThemedIcon('image-missing'),32) if icon: if icon.rfind('.') != -1: pix = IconManager().get_icon(FileIcon(File(icon)),32) else: pix = IconManager().get_icon(ThemedIcon(icon),32) data = (pix, '%s' % sname, obj, sname.upper(), file_path) return self.launcher_view.add_row(data,selected) except: return None The first thing that catches your eye is the line:
pix = IconManager().get_icon(ThemedIcon('image-missing'),32) The image-missing icon is one for all, so there’s no need to look for it every time. Let's make it an attribute of a class. In the
__init__ method, before calling __populate , __populate add: self.pix_missing = IconManager().get_icon(ThemedIcon('image-missing'), 32) and replace the corresponding line in
new_from_file with pix = self.pix_missingLet's see how the launch time has changed:
python -m cProfile -s cumulative uleResult2:
194247 function calls (186926 primitive calls) in 8.863 seconds Ordered by: cumulative time ncalls tottime percall cumtime percall filename:lineno(function) 1 0.007 0.007 8.864 8.864 ule:19(<module>) 1 0.000 0.000 8.363 8.363 ule:29(main) 2988/2974 0.008 0.000 8.189 0.003 types.py:46(function) 2989/2975 8.176 0.003 8.182 0.003 {method 'invoke' of 'gi.CallableInfo' objects} 1 0.001 0.001 6.817 6.817 app.py:48(__init__) 1 0.000 0.000 6.704 6.704 app.py:244(__populate) 24 0.003 0.000 6.697 0.279 app.py:224(new_from_file) 22 0.001 0.000 6.489 0.295 iconmanager.py:93(get_icon) 1 0.004 0.004 0.248 0.248 app.py:18(<module>) The result is somewhat unexpected, so I rechecked (with and without changes): consistently minus 35-40 seconds.
9 seconds is much less annoying, but nonetheless annoying.
Let's try to improve.
Step 3
IconManager is created as a separate object each time (and always without arguments), we will make it an attribute of the class.
Add to
__init__ : self.icon_manager = IconManager() All occurrences of
IconManager() replaced by self.icon_manager .Checking:
python -m cProfile -s cumulative uleResult3:
178980 function calls (171659 primitive calls) in 1.949 seconds Ordered by: cumulative time ncalls tottime percall cumtime percall filename:lineno(function) 1 0.001 0.001 1.949 1.949 ule:19(<module>) 1 0.000 0.000 1.449 1.449 ule:29(main) 1560/1546 0.004 0.000 1.314 0.001 types.py:46(function) 1561/1547 1.304 0.001 1.310 0.001 {method 'invoke' of 'gi.CallableInfo' objects} 1 0.001 0.001 0.285 0.285 app.py:48(__init__) 1 0.004 0.004 0.252 0.252 app.py:18(<module>) 206/45 0.005 0.000 0.191 0.004 {__import__} 9/5 0.000 0.000 0.187 0.037 importer.py:56(load_module) 9/5 0.000 0.000 0.186 0.037 module.py:239(_load) 1 0.002 0.002 0.174 0.174 pkg_resources.py:14(<module>) 1 0.000 0.000 0.159 0.159 app.py:245(__populate) 2617/377 0.011 0.000 0.159 0.000 {map} 24 0.003 0.000 0.153 0.006 app.py:225(new_from_file) 1 0.004 0.004 0.146 0.146 Gtk.py:22(<module>) 159/132 0.012 0.000 0.132 0.001 module.py:111(__getattr__) 1 0.000 0.000 0.107 0.107 pkg_resources.py:698(subscribe) 102 0.000 0.000 0.107 0.001 pkg_resources.py:2835(<lambda>) 102 0.001 0.000 0.106 0.001 pkg_resources.py:2256(activate) 22 0.001 0.000 0.098 0.004 iconmanager.py:93(get_icon) The window appears almost instantly. The total time is almost two seconds, but it also includes closing the window with the mouse. It is better to focus on the line
1 0.001 0.001 0.285 0.285 app.py:48(__init__) : the execution time changed from 6.8 seconds in the previous measurement to 0.3 in the last one.At this you can stop.
Total
Small changes led to a decrease in the launch time of the application from 46 seconds to 0.3 (for
app.py:48(__init__) ).Changes sent to developers.
Open source is not perfect, but everyone can improve it by spending quite a bit of their time.
Changes in the form of a diff file
=== modified file 'unitylaunchereditor/dialogs/app.py' --- unitylaunchereditor/dialogs/app.py 2011-11-28 17:12:07 +0000 +++ unitylaunchereditor/dialogs/app.py 2013-02-26 16:47:52 +0000 @@ -71,6 +71,8 @@ #bottom window buttons self.__create_bottom_box(main_box) + self.icon_manager = IconManager() + self.pix_missing = self.icon_manager.get_icon(ThemedIcon('image-missing'),32) self.__populate() self.connect('delete-event', Gtk.main_quit) @@ -198,12 +200,12 @@ sname = info['Name'] desc = info['Comment'] icon = info['Icon'] - pix = IconManager().get_icon(ThemedIcon('image-missing'),32) + pix = self.pix_missing if icon: if icon.rfind('.') != -1: - pix = IconManager().get_icon(FileIcon(File(icon)),32) + pix = self.icon_manager.get_icon(FileIcon(File(icon)),32) else: - pix = IconManager().get_icon(ThemedIcon(icon),32) + pix = self.icon_manager.get_icon(ThemedIcon(icon),32) data = (pix, '%s' % sname, obj, sname.upper(), file_path) return self.launcher_view.add_row(data,True) @@ -215,12 +217,12 @@ sname = obj.get('Name',locale=LOCALE) desc = obj.get('Comment',locale=LOCALE) icon = obj.get('Icon') - pix = IconManager().get_icon(ThemedIcon('image-missing'),32) + pix = self.pix_missing if icon: if icon.rfind('.') != -1: - pix = IconManager().get_icon(FileIcon(File(icon)),32) + pix = self.icon_manager.get_icon(FileIcon(File(icon)),32) else: - pix = IconManager().get_icon(ThemedIcon(icon),32) + pix = self.icon_manager.get_icon(ThemedIcon(icon),32) data = (pix, '%s' % sname, obj, sname.upper(), file_path) return self.launcher_view.add_row(data,selected) @@ -325,12 +327,12 @@ self.launcher_view.set_value(obj['Name'], self.launcher_view.COLUMN_NAME) self.launcher_view.set_value(obj['Name'].upper(), self.launcher_view.COLUMN_NAME_UPPER) icon = obj['Icon'] - pix = IconManager().get_icon(ThemedIcon('image-missing'),32) + pix = self.pix_missing if icon: if icon.rfind('.') != -1: - pix = IconManager().get_icon(FileIcon(File(icon)),32) + pix = self.icon_manager.get_icon(FileIcon(File(icon)),32) else: - pix = IconManager().get_icon(ThemedIcon(icon),32) + pix = self.icon_manager.get_icon(ThemedIcon(icon),32) self.launcher_view.set_value(pix, self.launcher_view.COLUMN_ICON) if button_type == TOOLBUTTON_REMOVE: UPD:
And I did not notice the elephant
Two issues remained unclear:
- too long start (50 seconds) for two dozen menu items;
- unexpected decrease in load time (5 times) after the first optimization.
The indirect culprit was the object
self.icontheme = Gtk.IconTheme.get_default() .The IconManager constructor expands the list of paths in which the icon search takes place. All changes are saved for the duration of the application, although Gtk.IconTheme.get_default () is not a Singleton. In my case, each IconManager initialization increases the list by 67 items, which leads to a change in its size by about 300 times (from 10 to 2985).
IconManager class constructor
def __init__(self): self.icontheme = Gtk.IconTheme.get_default() # add the humanity icon theme to the iconpath, as not all icon # themes contain all the icons we need # this *shouldn't* lead to any performance regressions path = '/usr/share/icons/Humanity' if exists(path): for subpath in listdir(path): subpath = join(path, subpath) if isdir(subpath): for subsubpath in listdir(subpath): subsubpath = join(subpath, subsubpath) if isdir(subsubpath): self.icontheme.append_search_path(subsubpath) By changing a few lines in the constructor, you can reduce the start time to 0.8 seconds (without previous optimizations) and 0.6 (with optimizations). Previous changes have become "cosmetic".
Changes in the form of a diff file
=== modified file 'unitylaunchereditor/core/iconmanager.py' --- unitylaunchereditor/core/iconmanager.py 2011-11-26 14:36:43 +0000 +++ unitylaunchereditor/core/iconmanager.py 2013-02-28 19:13:42 +0000 @@ -28,6 +28,7 @@ class IconManager: def __init__(self): self.icontheme = Gtk.IconTheme.get_default() + search_paths = self.icontheme.get_search_path() # add the humanity icon theme to the iconpath, as not all icon # themes contain all the icons we need # this *shouldn't* lead to any performance regressions @@ -38,7 +39,7 @@ if isdir(subpath): for subsubpath in listdir(subpath): subsubpath = join(subpath, subsubpath) - if isdir(subsubpath): + if isdir(subsubpath) and subsubpath not in search_paths: self.icontheme.append_search_path(subsubpath) def get_icon_with_type(self,filepath, size=24): Bug created , patch sent .
Source: https://habr.com/ru/post/171005/
All Articles