Python transpiler chain → 11l → C ++ [to speed up Python code and more]

Replacing "slices" \ slices on ranges \ ranges

s[-1] s[-2] s[:-1] s[1:] s[:1:] s[1:2] s[::2] s[3:10:2] s[3:10:] 

 s.last s[(len)-2] s[0..<(len)-1] s[1..] s[0..<1] s[1..<2] s[(0..).step(2)] s[(3..<10).step(2)] s[3..<10] 

1. When it is required, for example, to get the previous character by s[i-1] , but with i = 0 such / this record instead of an error will silently return the last character of the string [ and I encountered this error in practice - commit ] .
2. The expression s[i:] after i = s.find(":") will work incorrectly when the character is not found in the string [ instead of the "part of the string starting from the first character : and then" the last character of the string will be taken ] (and , to return -1 with the find() function in Python, I also consider incorrectly [ should return null / None [ and if -1 is required, then it should be written explicitly: i = s.find(":") ?? -1 ] ] ).
3. Writing s[-n:] to get the last n characters of the string will work incorrectly with n = 0.

Chains of comparison operators

List inclusion (list comprehension)

1. In all places where I met Python's list comprehension, you can easily get away with the functions `filter ()` and `map ()`.

    dirs[:] = [d for d in dirs if d[0] != '.' and d != exclude_dir] dirs[:] = filter(lambda d: d[0] != '.' and d != exclude_dir, dirs) '[' + ', '.join(python_types_to_11l[ty] for ty in self.type_args) + ']' '[' + ', '.join(map(lambda ty: python_types_to_11l[ty], self.type_args)) + ']' # Nested list comprehension: matrix = [ [1, 2, 3, 4], [5, 6, 7, 8], [9, 10, 11, 12], ] [[row[i] for row in matrix] for i in range(4)] list(map(lambda i: list(map(lambda row: row[i], matrix)), range(4))) 

2. `filter ()` and `map ()` in 11l look prettier than in Python

    dirs[:] = filter(lambda d: d[0] != '.' and d != exclude_dir, dirs) dirs = dirs.filter(d -> d[0] != '.' & d != @exclude_dir) '[' + ', '.join(map(lambda ty: python_types_to_11l[ty], self.type_args)) + ']' '['(.type_args.map(ty -> :python_types_to_11l[ty]).join(', '))']' outfile.write("\n".join(x[1] for x in fileslist if x[0])) outfile.write("\n".join(map(lambda x: x[1], filter(lambda x: x[0], fileslist)))) outfile.write(fileslist.filter(x -> x[0]).map(x -> x[1]).join("\n")) 

   and therefore the need for list comprehensions in 11l actually disappears [the replacement of list comprehension with filter() and / or map() is performed during the conversion of the Python code to 11l automatically ] .

Convert the if-elif-else chain to switch

 ch = instr[i] if ch == "[": nesting_level += 1 elif ch == "]": nesting_level -= 1 if nesting_level == 0: break elif ch == "'": ending_tags.append(''') # '' elif ch == "'": assert(ending_tags.pop() == ''') 

 switch instr[i] '[' nesting_level++ ']' if --nesting_level == 0 loop.break "'" ending_tags.append("'") // '' "'" assert(ending_tags.pop() == "'") 

 switch (instr[i]) { case u'[': nesting_level++; break; case u']': if (--nesting_level == 0) goto break_; break; case u''': ending_tags.append(u"'"_S); break; // '' case u''': assert(ending_tags.pop() == u'''); break; } 

Converting small dictionaries to native code

 tag = {'*':'b', '_':'u', '-':'s', '~':'i'}[prev_char()] 

 var tag = switch prev_char() {'*' {'b'}; '_' {'u'}; '-' {'s'}; '~' {'i'}} 

 auto tag = [&](const auto &a){return a == u'*' ? u'b'_C : a == u'_' ? u'u'_C : a == u'-' ? u's'_C : a == u'~' ? u'i'_C : throw KeyError(a);}(prev_char()); 

 rn = {'I': 1, 'V': 5, 'X': 10, 'L': 50, ...} 

 var rn = ['I' = 1, 'V' = 5, 'X' = 10, 'L' = 50, ...] 

 auto rn = create_dict(dict_of(u'I'_C, 1)(u'V'_C, 5)(u'X'_C, 10)(u'L'_C, 50)...); 

Capture \ Ca pture external variables

 writepos = 0 def write_to_pos(pos, npos): nonlocal writepos outfile.write(...) writepos = npos 

 var writepos = 0 fn write_to_pos(pos, npos) @outfile.write(...) @writepos = npos 

 auto writepos = 0; auto write_to_pos = [..., &outfile, &writepos](const auto &pos, const auto &npos) { outfile.write(...); writepos = npos; }; 

Global variables

 break_label_index = -1 ... def parse(tokens, source_): global source, tokeni, token, scope source = source_ tokeni = -1 token = None break_label_index = -1 scope = Scope(None) ... 

 var break_label_index = -1 ... fn parse(tokens, source_) :source = source_ :tokeni = -1 :token = null break_label_index = -1 :scope = Scope(null) ... 

Index / number of the current container item

 items = ['A', 'B', 'C'] for index, item in enumerate(items): print(str(index) + ' = ' + item) 

 var items = ['A', 'B', 'C'] loop(item) items print(loop.index' = 'item) 

Performance

And now let's add to the diagram the implementation generated by the Python transpiler → 11l → C ++:

Execution time [ 488Kb file conversion time ] was 868 ms for CPython and 38 ms for generated C ++ code [ this time includes full-featured [ i.e. not just working with data in RAM ] launching the program by the operating system and all the input / output [ reading the original file [ .pq ] and saving the new file [ .html ] to the disk ] ] .

I also wanted to try Shed Skin , but it does not support local functions.
Numba also failed to use (gives the error 'Use of unknown opcode LOAD_BUILD_CLASS').
Here is the archive with the program used for performance comparison [ for Windows ] (installed Python 3.6 or higher and the following Python packages are required: pywin32, cython).