Lazy webapp loading

In the standard webapp framework there is a lot of useful and a little extra. But there is one feature in this nice creation of Google - all the modules used are loaded when the application starts. This is convenient for development and debugging, but it poses one problem: the instance has a rather long start for a rather large application. As a result, the user has to wait for the entire application to load, even if only a small part of the code is needed to display the page he needs.

Today we will “teach” the webapp not to download too much.
The solution to the problem is quite simple, so first there will be a code, and only then a description and explanation. The lazy-loading code for the main application module looks like this:

from google.appengine.ext.webapp import RequestHandler, WSGIApplication, util import os, sys dev_server = os.environ.get('SERVER_SOFTWARE').lower().startswith('dev') class lazy_loader(object): def __init__(self, fullname): self.modulename, sep, self.objname = fullname.rpartition('.') if dev_server: self() def __call__(self): try: return self.obj() except AttributeError: if not sys.modules.has_key(self.modulename): __import__(self.modulename, globals(), locals(), [], 0) self.obj = sys.modules.get(self.modulename).__dict__.get(self.objname) return self.obj() application = WSGIApplication([(route[0], lazy_loader(route[1])) for route in [ (r'/', 'handlers.MainPage'), (r'/search/', 'handlers.Search'), (r'/_ah/queue/sendmail', 'mail.SendMail'), (r'/_ah/queue/HeavyTask', 'tasks.HeavyTask'), ]], debug=os.environ.get('CURRENT_VERSION_ID').lower().startswith('dev')) def main(): util.run_wsgi_app(application) if __name__ == '__main__': main() 

Lazy loader.

lazy_loader is the main thing in this example. This is a small interlayer object whose main task is to initialize as quickly as possible. An example was taken from here , after which the code was slightly modified.
')
Actually, during initialization, this object only saves the name of the query processor class. The if dev_server: self.__call__() only makes it easier to debug on the development server and you can do without it. The definition of the development server is outside the scope of the function to increase performance.

When you call the handler lazy_loader immediately passes the call. The construction with catching AttributeError chosen because the self.obj object should be present in most cases (if the name of the object is correct, then it will be absent only on the first call). If you instead check self.obj on every call, the application will perform unnecessary operations; a bit, but still ... For the "lost" AttributeError (if such suddenly appear in the handler's code) you can not worry - they will still pop up on the second call of the handler.

Loading handler.

Perhaps the most interesting thing happens when the handler is first called: a call to the missing self.obj calls AttributeError , during the processing of which the module is imported and the handler is received. Standard python expressions from package import module can only import modules whose names are known at the design stage, so you can’t do without the powerful built-in function __import__ . We will use it in the simplest way - first import the module, and then get it from the sys.modules dictionary.

Since each imported module is added to sys.modules , it is possible to check whether the necessary module has already been imported. In this example, the handlers module will be imported only once during the lifetime of the instance.

Webapp configuration

In order for the configuration of routes to be more visual and less cumbersome, filtering lists is used , which adds lazy_loader to all list items (for example, (r'/', 'handlers.MainPage') replaced by (r'/', lazy_loader('handlers.MainPage')) .

And another handy little thing that I decided not to remove from this example is the debug configuration of the webapp , depending on the version of the application specified in app.yaml . Debug information will be issued only if the application version starts with “dev” (for example, develop , dev-0-1-123 , dev-B1-37-2 , etc.)