Personal management system development

Motivation

Modern technology development is moving towards predicting human behavior based on tracking the entire array of its activities in virtual and physical reality (big data). There is a positive point in this, because if you see where a person is going, you can show him the shortest and most effective way. But, according to the law of unity and struggle of opposites, the following threat arises (for marketers, on the contrary, a perspective), as I see it - this is an opportunity not to predict behavior, but to form predictable behavior.

If you are interested in what an individual can do about it - welcome under cat.

Of course, if we look even wider, then society itself, using culture and other social tools, forms the predictable behavior of the individual, but this is too broad, because Culture gives an incomparably greater level of freedom of judgment, unlike a focused informational tape, so I’m narrowing back.

So, what can make a separate customer to complicate the task of forming their own predictable behavior? For me, the answer lies in the following phrase - in order to learn to manage something, you must learn to measure it.
')
But what can a person measure at himself and how to analyze it? The decision was invented for a long time - this is keeping a personal diary (its / my little data) and the subsequent analysis of recorded events.

But when keeping a diary, the following difficulty arises, at least for me, it is almost impossible to effectively categorize events \ objects (I’m vicious about object-oriented thinking). For example, I would like to separately consider tasks, thoughts, questions, mistakes, etc., to see the connections between them, to get an analyst. It should be noted that some examples of effective solutions to this problem can be found in numerous books on time management. But I tried to make my bike.

So, what properties can a perfect diary have?

• Data acquisition and their fast (in the limit, automatic) classification, according to different types of objects, such as thoughts, tasks, reminders, contacts, sleep, notes, etc. Maintain connections between objects;
• Download data from external sources (social networks, bank accounts, physical activity trackers);
• Automatic construction of analytics, for different slices of information;
• Issuing tips on current actions (high priority task, birthdays, etc., etc.);
• Ability to customize algorithms for individual processing of objects;
• The system should provide a choice of what information is available for publication in the external network, and what remains personal (the implementation should ensure integrity in this regard), data in the external network are also classified and put into components;
• In the limit, any information that a person encounters, and any information that is generated by this person should be classified and fit into such a diary.

And now, having finished with dreams, we return from Plato's ideal to our sinful and, having passed through many curved mirrors, and not only mirrors, we get the realization of the concept:

Task

Initially, I set myself the following goals:

1. Creation of any kinds of objects without development;
2. The output of identical objects on a separate page with the display in a tabular form, as well as with the ability to go to a separate object and display / maintain as a card
3. The possibility of establishing links between objects;
4. Setting statuses and categories for objects;
5. Doing analytics \ reminder \ prediction.

results

Happened:

• Creation of new objects;
• Creation of status schemes linked to any kinds of objects;
• It is possible to create links between objects, the links are displayed in a separate tab. In essence, links solve the problem of inheritance and are links between any objects;
• Hyperlinks to objects - it is possible to set a direct link to the object;
• For tasks it is possible to create separate folders for ease of processing.

Problems:

• No analytics;
• No cascading deletion of child objects;
• There is no possibility of finding objects;
• There is no possibility to process several objects on one page;
• The complexity of creating new types of objects (the description of the object takes about half an hour, even for me as an author)
• More or less clear architecture turned into a set of hacks

Decision

Disclaimer! The author is not a professional programmer and expert in the field of development in python \ js, but just an amateur, so ideas on the proper organization of the architecture and the technologies used are welcome and I will be very grateful for these tips.

Github code

Tools used

Backend:

• Python 3.2.5 - Server Part
• SQLite 3.5.1 - Database

Frontend:

• Bootstrap 3.3.6 - Data Mapping
• JQuery 2.2.0 - Logic
• TinyMCE 4.3.2 - Text Editor. Additionally, the syntaxhighlighter plugin was used to format the code.
• Datatables 1.10.10 - Display data in tabular form.

Solution Architecture

General architecture description

1. In Python, a service is written that spins on localhost and handles requests from html pages.
2. Objects are html-pages with a specific markup, in which the names of the fields are indicated, correspond to the fields in the database (no ORM solutions are used, the processing is done by your bicycle)
3. Communication with the server occurs through json messages.
4. All objects from the server are loaded at the initial page load, in the future only the delta is exchanged.
5. There are built-in objects, these are statuses and relationships, the remaining objects are described separately (see below)

Unfortunately, as the functionality grows and the desire to quickly get a complete solution, the architecture began to creep away and failed to fully encapsulate objects, so in practice there are different hacks and moments of penetration of the encapsulation, but in general the scheme is relevant.

Frontend:

• Main.js - Entry point for frontend event handling. Initial data is initialized, events are handled from the html page, and other objects are invoked for operation.
• datatype.js - Internal conversion of data into the required format for processing, depending on the scenario.
• card.js - Processing an object card, displaying card data for editing, viewing, links, saving data.
• main_view.js - Controls the display of the card \ table on the page
• myajax.js - Wrapper for ajax request
• mydatatables.js - Wrapper for working with datatables.net.
• mytinymce.js - TinyMCE Wrapper
• % ObjectName% - Parameters for each type of object.
• all_relations.js - All possible types of objects available when displaying relationships.
• object_state.js - Class for processing the view of an object
• rel.js - Handling relationships between objects.
• requester.js - Convert data to a format for transfer to the server.
• settings.js - Constants.

Backend:

1. server.py - Request processing.
2. json2sql.py - Converting a json query from a frontend to an sql query for a database
3. dbserver.py - Execute sql query.
4. settings.py - Options.
5. object2db.py - Parameters of object types on the server.
6. actionfile.py - Creating directories for tasks.

Database schema:

• % object_name% - A separate table for each object created.
• sys_status - Possible statuses grouped by status schemes.
• Sys_rel2obj_items - Data on relationships between objects
• Sys_rel_object2object - Adjusting the correspondence of relationships between objects
• object_type - Type Description
• Sys_rel - Relationship Description

Examples of JSON messages:

• Get statuses by object - {“object”: “status”, “action”: ”select”, “items”: [{“id”: 1, ”msg”: {“fields”: [{“column”: ” id ”,“ op ”:”> = ”,“ value ”:” 0 ″}]}}]}
• Get relationships by object - {“object”: “rel_items”, “action”: ”select”, “items”: [{“id”: 1, ”msg”: {“fields”: [{“column”: ” obj1_type ”,“ op ”:” = ”,“ value ”:” project ”}, {“ column ”:” id1 ″, “op”: ”=”, “value”: ”22 ″}]}}]}
• Get possible object relationships - {“object”: “sys_rel_object2object”, “action”: ”select”, “items”: [{“id”: 1, ”msg”: {“fields”: [{“column”: ” id ”,“ op ”:”> = ”,“ value ”:” 0 ″}]}}]}
• Get all objects - {“object”: “project”, “action”: ”select”, “items”: [{“id”: 1, ”msg”: {“fields”: [{“column”: ”id ”,“ Op ”:”> = ”,“ value ”:” 0 ″}]}}]}
• Create an object - {“object”: ”project”, “action”: ”insert”, “items”: [{“id”: 0, ”msg”: {“fields”: {“title”: ”123 ″, "Note": "123"}}}]}
• Edit object - {“object”: ”project”, ”action”: ”update”, ”items”: [{“id”: 0, “msg”: {“fields_set”: {“title”: ”1234 ″, "Note": "123"}, "fields_where": {"id": "22 ″}}}]}

Where:

• Object - the type of object (for example, project, task, etc.)
• Action - type of action (create, delete, change, delete, get)
• Items - within one message, you can request several items, for example, to do several insert.
• Msg - message within one package
• Fields - fields for this object

Some of the objects created are:

• Project - project management.
• Task - task management
• Question - keeping questions
• Think - keeping thoughts, ideas, etc.
• Timing - time control ()
• Contact - maintaining contacts
• Diary - keeping a personal diary
• Snippet - snippet

Algorithm for creating a new object

1. Create a table in the database
2. Create file% object_name% .html
3. Create file% object_name% .js
4. Describe the object in the object2db.py file
5. If you need to add a status, fill in the data in the tables (sys_status)
6. If you want to create new relationships, fill in the data in the tables (sys_rel_object2object)

I did not describe the creation in more detail (and did not record the video, although initially the idea was), because It turned out quite a long and tedious process. I did not do the constructor, because for the concept, it turned out to be redundant, but this is the first thing I will begin when developing version 2.0, see below.

Future plans for version 2.0

The ideal is to work with any objects within one window, a minimum of actions with the mouse. Maximum translate to description on the command line.

Ideas:

• Object Editor. - Creating objects, without the need to manually create html-pages, the presence of a separate js-file for the object and a separate configuration in python.
• Creating statuses.
• Creating \ setting relationships.
• Display on a single page of multiple objects. - At the moment, the architecture is built in such a way that only one type of object can be displayed on one page (not counting references to other objects).
• Command line - Command line for reducing the number of actions to perform operations.
• For example, being on the “Project” object in the command line, “Task” is typed and the “Task” object is automatically created with reference to this project. Since currently you need to go to the tab
• Encryption. - Add database encryption.
• Analytics - The possibility of basic analytics on objects, with the display of the count by status, importance, etc.
• Adding an object dates (dates) - a separate object that can be added to any object with the ability to add stocks
• Activities - Ability to practice certain actions
• Check for duplication - When you create an instance of an object, check for duplication
• Required fields - Definition of required fields.

Conclusion

I tried to write an article as short as possible so that you could quickly understand the basic idea and description of the implementation, without drowning in too much information.

I use this system, about 3 months, simultaneously correcting errors, the flight is equal.

Thank you for your time.