On an approach to building a database at the initial stage of work on a web project

Here we consider an approach that can facilitate the commencement of work on an Internet project, the requirements for which may be defined, but some, possibly significant, details are not disclosed. I believe that in general, all startups and projects at the beginning can rarely be described fully enough to make it possible to build the required database architecture and competent code. At the initial stage, designing a database structure based on the idea of ​​using a NoSQL-like approach as a draft for a project saves effort and money in the face of constantly changing requirements. This approach allows you to get quick results and build the entire logical scheme of the project. This approach gives a gain in speed and simplifies the matter at the most turbulent initial stage, if we have to redraw the architecture quite often and deeply. At the same time, the transition to a classical, more or less normalized base is fairly painless.



Suppose that there is some rather crude idea, on the basis of which you need to build some Internet product. Where to start? Or if the idea is crude, then nothing should be done in terms of implementation? However, this is not always possible. Rather, it is difficult for a developer to understand whether current requirements are final or tomorrow everything will change in the most significant way. And we, being firmly convinced that the framework has been defined, proceed to work. If everything goes well, the result will be good and the project code will be quite beautiful (with accuracy to the programmer’s skills). However, things don't always go well. And then, in general, the project moves like this:

1. Based on the idea of ​​the designed base - as it later turns out to be a first approximation. For each information entity, a separate table. For each characteristic a separate field.
2. Built some of the project on the basis of the resulting database. Implemented logic, stretched layout.
3. Consultation with the customer (or "internal customer"). Then it turns out that everything needs to be changed. Since the information that we had two days ago was outdated, it turned out features of which we did not even suspect. This situation is very common, because few people take the trouble to build a complete process scheme.
4. Change some part of the database schema - in the simplest case, you will need to add / remove fields. Following this, change all SQL queries. And the third necessary step in this case is testing in new conditions.

And then the cycle repeats from point 2 . I think that there are too many cases here and, moreover, rather painstaking ones. Based on what I saw, looking at myself and studying the experience of colleagues in the shop, it is clear that at some point the developer gets tired and further redesigning the system leads to non-optimal solutions.

On the one hand, the absence of a clear (for the programmer) picture of the process is bad and it would seem that in such a situation one does not have to wait for good results. On the other hand, this is a completely vital situation, which can be explained not only by the customer’s features, but also by objective reasons. Indeed, having on hand at least to some extent a finished and lively product, where logic is implemented and interconnections are built, immediately those issues that were not and could not be thought out in advance immediately emerge.



For myself, I have found an excellent way out, which allows minimizing labor costs at a stage until we know enough some key details to implement. And they can be clarified only after receiving some working model. The essence of the approach is that we store all informational entities in separate tables, but without separating the fields. In the form of a JSON array. This allows you to store objects of arbitrarily great complexity.

')

This approach allows you not to touch the database when changing / adding fields and not to waste time changing and debugging queries in the new conditions. We are left with only a change in pure logic and validation. In the future, when moving from storing JSON-packed data to a normalized structure, we need to change almost only the requests, and the logic and checks remain unchanged.



Why do you need to switch from NoSQL storage type to regular tables? For me, this issue is resolved easily: as long as I can’t call myself a MongoDB expert or something like that, it’s better for commercial development to use proven tools. Although, if you have a choice and if we don’t have a team of analysts in the project (who write SELECT * to our database) and our database will not use third-party services, then all storage can be transferred to NoSQL rails.



So, by designing the entire structure in the style of JSON lines for starters, we save time on debugging and much earlier we can get some complete results, having which all the necessary issues for building a sufficiently complete and reasonable architecture will be solved.



That's what I have to say.