Optimization of presentation templates in the Codeigniter Framework using AST transformations

Recently, I worked with the portal, the attendance of about 100 thousand people per month written on Codeigniter. All anything, but any page of this portal was given by the server for at least 3 seconds. At the same time, the hardware was no longer where to expand, and we will not talk about the architecture of the application. I needed to find a solution that would help shorten the response time of the application with the smallest code changes.

Prehistory

Codeigniter is a great framework for web applications, no doubt. It is easy, flexible, and very easy to learn.

But there are a few problems. One of which is the lack of a handler for views. As a template engine, pure php is used (with small inserts of Codeigniter).

Many will say that this is not a problem, but the advantage - the lack of preprocessing before outputting to the page can significantly reduce the response time from the application, especially if the template engine is also written in php and not in the form of an extension.

In fact, a big plus of template engines is that they can compile templates and cache them on disk for later processing. That is, if the templates do not change often, when using a template engine, we get at least one plus - convenience. If there are a lot of templates, then another caching will be added here. I do not know how other developers, but I prefer to use the template engine when possible.

Problem

When you use Codeigniter for a small project, then most likely no problems with the templates will be noticeable. But when your project grows to hundreds of templates - you will suffer from a slow layout of templates.

So it was in my case - the number of template files connected when the page was loaded reached 50 (information from the built-in function get_included_files ).

The page I chose for the experience is as follows and is the most loaded on the site:

The page displays a list of 30 items - restaurants and various kinds of information about them, each of which, in turn, is composed of + - 35 templates. Since php is used as a template engine and there is nothing more then there is no caching there. As a result, we need to build about 900 templates.

Before working with templates, I was able, with the help of minimal code optimization, to reduce the page output time by 1 second (30%) to + -2 seconds:

 Loading Time: Base Classes 0.0274 Controller Execution Time 1.9403 Total Execution Time 1.9687 

It was still too much

Decision

It is clear that the layout of about 900 templates is expensive, especially in php. Therefore, it was necessary to “glue” all these templates into one, in order not to do it every time a page is requested.

The use of a ready-made template engine like twig or smarty disappeared immediately, since all the controllers would have to be rewritten, and there are a lot of templates.

At that time I was already a little familiar with AST trees. Templates represented something like this:

 ... <div class="brand-block"> <?php $this->load->view('payment_block', array('brand' => $brand); ?> <?php $this->load->view('minimal_block', array('brand' => $brand)); ?> <?php $this->load->view('deliverytime_block', array('brand' => $brand)); ?> <?php if (!$edit): ?> <?php $this->load->view('deliveryprice_block', array('criteria' =>$criteria); ?> <?php endif; ?> </div> ... 

Design

 $this->load->view(string $templatePath,array $params) 

do an "include" with passing additional parameters $params
The essence of the task was to replace all such calls with the contents of the templates themselves and passing inline parameters to them. Recursively

Interesting, I thought, and took up the tools which found just one: Nikic PHP-Parser . This is a very powerful tool that allows you to do all sorts of manipulations on the abstract syntax tree of your code and then save the modified tree back to php code. And all this can be done in php itself - the parser does not have any dependencies on c-extensions and can work on php 5.2+.

Implementation

PHP-Parser provides convenient tools for working with AST: NodeVisitor and NodeTraverser interfaces with which we will build our optimizer.

The main thing is to find all the calls to the view method on the property of the load class and understand what template should be loaded. This can be done using NodeVisitor. We are interested in his leaveNode(Node $node) method which will be called when NodeTraverser will "leave" from the node of the AST tree:

 class MyNodeVisitor extends NodeVisitorAbstract { public function leaveNode(Node $node) { //    -      if ($node instanceof Node\Expr\MethodCall) { // ,       if ($node->name == 'view') { //          //     Codeigniter'a,      //    :) //   ,       . //   -  ,    //       ,        if ($node->args[0]->value instanceof \PhpParser\Node\Scalar\String_) { //    ,       $code = md5(mt_rand(0, 7219832) . microtime(true)); $node->name = 'to_be_changed_' . $code; $params = null; //  ,      `inline` if (count($node->args) > 1) { if ($node->args[1]->value instanceof Node\Expr\Array_) { $params = new Node\Expr\Array_($node->args[1]->value->items, [ 'kind' => Node\Expr\Array_::KIND_SHORT, ]); } else { if ($node->args[1]->value->name != 'this') { $params = $node->args[1]->value; } } } //  ,       //        $this->nodesToSubstitute[] = new TemplateReference($this->nodeIndex, $node->args[0]->value->value, $params, $code); } } ... 

Thus, we will be able to select all the elements that should be replaced. You can also make a replacement for any other elements: explicit require, include, etc.

Do not forget that the replacement must be done recursively deep into. To do this, you need to make a wrapper over PHP-Parser where exactly and will be replaced with the insides of the template:

 class CodeigniterTemplateOptimizer { private $optimizedFiles = []; private $parser; private $traverser; private $prettyPrinter; private $factory; private $myVisitor; private $templatesFolder = ''; public function __construct(string $templatesFolder) { $this->parser = (new ParserFactory)->create(ParserFactory::PREFER_PHP5); $this->traverser = new MyNodeTraverser(); $this->prettyPrinter = new PrettyPrinter\Standard(); $this->factory = new BuilderFactory(); $this->templatesFolder = $templatesFolder; $this->myVisitor = new MyNodeVisitor(); $this->traverser->addVisitor($this->myVisitor); } public function optimizeTemplate(string $relativePath, $depth = 0, $keepOptimizing = true) { if (substr($relativePath, -4, 4) !== '.php') { $relativePath .= '.php'; } if (!isset($this->optimizedFiles[$relativePath])) { $templatePath = $this->templatesFolder . $relativePath; if (file_exists($templatePath)) { $templateOffset = 0; $notOptimized = file_get_contents($templatePath); //    AST $stmts = $this->parser->parse($notOptimized); if ($keepOptimizing) { $this->myVisitor->clean(); $this->traverser->setCurrentWorkingFile($relativePath); //     AST $stmts = $this->traverser->traverse($stmts); //       MyNodeVisitor $inlineTemplateReference = $this->myVisitor->getNodesToSubstitute(); ++$depth; $stmsBefore = count($stmts); foreach ($inlineTemplateReference as $ref) { //   -     $nestedTemplateStatements = $this->optimizeTemplate($ref->relativePath, $depth); $subtempalteLength = count($nestedTemplateStatements); $insertOffset = $ref->nodeIndex + $templateOffset; $pp = new PrettyPrinter\Standard(); //     `inline`:    `extract` if ($ref->paramsNodes) { array_unshift($nestedTemplateStatements, new Node\Expr\FuncCall(new Node\Name('extract'), [$ref->paramsNodes])); } //    ,      if (get_class($stmts[$insertOffset]) === 'PhpParser\Node\Expr\MethodCall' && ($stmts[$insertOffset]->name === "to_be_changed_" . $ref->code)) { //   ""    AST //    if(1),       $stmts[$insertOffset] = new Node\Stmt\If_(new Node\Scalar\LNumber(1), [ 'stmts' => $nestedTemplateStatements ]); } else { //     ,    ast } } } //    "" . //         $this->optimizedFiles[$relativePath] = $stmts; } else { throw new Exception("File not exists `" . $templatePath . "` when optimizing templates"); } } //    return $this->optimizedFiles[$relativePath]; } public function writeToFile(string $filePath, $nodes) { $code = $this->prettyPrinter->prettyPrintFile($nodes); // create directories in a path if they not exists if (!is_dir(dirname($filePath))) { mkdir(dirname($filePath), 0755, true); } // write to file file_put_contents($filePath, $code); } } 

That's it, run the optimizer:

  //      -    $optimizer = new CodeigniterTemplateOptimizer('./views/'); //      $optimizer->writeToFile($to, $optimizer->optimizeTemplate($from)); 

With DirectoryIterator you can build a script in two minutes that will optimize the entire template folder.

Conclusions and results

After replacing the templates with optimized ones, I managed to reduce more than 1s of the execution time, the results of the Codeigniter profiler:

 Loading Time: Base Classes 0.0229 Controller Execution Time 0.7975 Total Execution Time 0.8215 

With the help of template optimization, I was able to reduce more time than when optimizing php code. The cost of optimizing templates is incomparable with changing many lines of code. Also, optimization of templates in no way changes the behavior of the application (well, this is simply “gluing”), which is a very positive fact.

The code snippets cited in the article have been adapted to provide general points that will help you understand and do not pretend to work.