How I reinvented dictionaries in Python

In our Django-application it was necessary to develop a report (calculation) of bonuses.
The report should have a nested structure summarizing the users, divisions and the entire company. Schematically, its logic can be represented:

print total for department in departments: print department.total for user in department.users: print user.total for row in user.rows: print row.data 

This report had two complicating points:

1. In the role of "row" could be different models (and be interleaved), which does not allow to use iterators on Query Sets.
2. Time to build a report. Data collection takes significant time (several seconds). Data in the report may vary. Speaking of cleanliness, this is not a static report, but a tool for monitoring and adjusting the accrued bonuses in the form of a report. But the data does not change very often, say for every 100 views there will be one change, after which you need to rebuild the report. Those. data can be cached.

The structure of nested dictionaries perfectly solves both problems: you can put all the required scalars in them (numbers, strings, dates), serialize and add to the cache.
')
The data structure for the report has become (simplified):

 { 'total': { 'income': 1234, 'bonus': 123, 'expense': 1234, 'penalty': 123 }, 'departments': { '{dept_id}': { 'department': { 'title': 'Mega Department' } 'total': { 'income': 1234, 'bonus': 123, 'expense': 1234, 'penalty': 123 }, 'users': { '{user_id}': { 'user': { 'name': 'John Smith' }, 'total': { 'income': 1234, 'bonus': 123, 'expense': 1234, 'penalty': 123 }, 'rows': { '{sale_id}': { //   'type': 'sale' 'base_income': 1234, 'bonus': 123, 'comment': 'some description' }, '{expense_id}': { //   !!! 'type': 'expense' 'expense': 1234, 'penalty': 123, 'comment': 'some description' }, ... } }, ... } }, ... } } 

And here I faced the problem that filling out such a structure from dictionaries is not as convenient as I wanted. Checking dictionaries for the presence of keys or using setdefatult (key, {}) turns the code into an unreadable mess.

This structure is somewhat similar to XML. And I would like to use something similar to how XPath expressions are constructed to address the nodes of an XML tree:

 /departments/{dept_id}/users/{user_id}/rows/{row_id}/base_income 

or in Python, something like:

 data.departments.{dept_id}.users.{user_id}.rows.{row_id}.base_income 

Taking into account that {dept_id} and other others {id} are integers, I allowed myself to use square brackets: [].

 data.departments[{dept_id}].users[{user_id}].rows[{row_id}].base_income 

Actually, I needed a class that would behave mostly like a dictionary, but at the same time:

1. access to attributes could be done without square brackets
2. Missing attributes were automatically created.

This is how ElasticDict appeared .

Eventually

The data preparation code looks like this:

 data = ElasticDict() for sale in Sale.objects.filter(...).prefetch_related(...): data.departments[sale.user.department.pk].users[sale.user.pk].rows[sale.pk] = {'base_income': sale.amount, 'bonus': sale.calc_bonus()} #    ,     for expense in Expense.objects.filter(...).prefetch_related(...): data.departments[sale.user.department.pk].users[sale.user.pk].rows[expense.pk].base_expense = expense.amount data.departments[sale.user.department.pk].users[sale.user.pk].rows[expense.pk].penalty = expense.calc_penalty() 

The code in the template is:

 {{ data.total }} {% for dept_id, department in data.departments.items %} {{ department.total }} {% for user_id, user in department.users.items %} {{ user.total }} {% for row_id, row in user.rows.items %}: {{ row.data }} {% endfor %} {% endfor %} {% endfor %} 

Conclusion

It should be noted that ElasticDict () is a subclass of the usual dict () a, i.e. it has everything that is available in a regular dictionary. At that moment, when you need to "fix" the structure (again, we want to get the KeyErrors when accessing non-existent keys), the ElasticDict instance can be exported to the usual dict (). A recursive traversal of ElasticDict () is done, where all instances of this class are replaced with regular dictionaries. There is also an inverse transformation - we input the dictionary to the input, we get ElasticDict at the output, also with a recursive walk.

Comments / suggestions are welcome!

UPDATE from the English-speaking get-together suggested that there is already an analogue to addict . I think that those who voted "I need" should switch to it, as to a more stable (proven).