Your design sucks

… but it normal. Any design sucks. And always will suck.

If you don't believe me, let's explain ...

No project is experiencing a meeting with the implementation

When you start to realize what you have designed, you inevitably come across such things in reality that do not correspond to your initial expectations.
')
The data that you expected as mandatory in the response of the external service may be missing (or be invalid). The expected uniqueness may not be completely unique in practice (even in sha1, collisions sometimes occur). Processes that were supposed to be reliable will fall much more often than you expected.

This is normal.

In some cases, you can simply lock in, throw an exception, or fall in any way loud. In other cases it is necessary to weaken the requirements of the system. Or add an additional filter layer, which will “clean up” and transfer the correct input data to the system.

Missing data can be made optional or replaced by default.

Incorrect data can be considered as missing, or write them "as is" and add an additional validated version, which is present only if the original is valid.

Limiting the uniqueness can cancel, or merge data, or select one of the versions.

You can reapply an unreliable process, or make its results optional (and possibly reapply the next time it is needed), or explicitly handle failures.

Whether to consider the violation of your assumptions as an error, whether to add an intermediate layer or correct the concept of the system to better match reality, it is necessary to decide in each case separately. It is always a decision about a fair compromise: is the processing of a mess in the outside world worth the complexity of your system?

No project survives meeting real users.

Your conceptual model may be elegant and correct, but if it does not fit into the model in the heads of your users, you will have dissatisfied users. Or no users at all.

People will search by unexpected parameters (“yes, I know that I can search by client_id, but it’s much easier to see from which phone they call me and look at it”).

People will have superstitious habits - expect that functions like “extinguish everything and rebuild again” will be used much more often than expected, because users will perceive it as “turn off and on” and react accordingly.

People will use your system for things for which it was never designed, but with a superficial glance it might seem that it was - and they will scream and complain that they cannot send a ten-megabyte pdf sms.

When you are faced with performance issues such as data retrieval, you either restrict the fields by which you can search (possibly generating a flurry of complaints), or add indices, or reorganize data, or implement an external search engine.

When you encounter superstitions, you either restrict access to improperly used functions (again complaints), or change the interface to bring the user to a function that he really should have used, or add clever logic to define situations when rather simple actions.

If you discover unanticipated use cases, you either declare them incorrect (complaints ... and, possibly, workarounds that will add problems of one of the other two types to you), or implement their support, despite the fact that it was not planned initially, or integrate with the other. service that provides this functionality.

Once again: these are all compromises; and it becomes apparent that there are three main options for action.

Three Typical Answers

Ignore what includes and “throw an error if assumptions are violated”, and “no, this is not in a bad mood” - that is, refuse to acknowledge the problem relating to your model. This option should be used sparingly, but when the basic principles of your system are at risk, it should be seriously considered.

Adapt that covers “loosen restrictions” and “add indexes” —that is, to tweak the design so that the problematic scenario would be processed well. This is the best default option for scenarios that are “almost suitable”, that is, when there are small differences between “what is” and “what is required”.

Move outside , which includes intermediate layers between your system and the outside world, or between the system and user expectations — that is, adding something outside the kernel of the system, but inside the system as a whole, as users see it. This option is best for scenarios that are more orthogonal to the basic principles of your system than opposed to them.

No project is experiencing a meeting with the future.

Even with all of the above, only two things can be said for sure: at least part of your project sucks; and six months later, you realize that this part is more than you think now.

We learn while we develop our systems; we learn when we watch users of our systems; and we definitely learn when the system crashes and someone calls in panic at three in the morning.

And what to do with all this? The best idea I have managed to find so far is quite simple: design assuming that everything goes wrong, and try to do to fix the consequences of these “wrong” things as easily as possible.

In fact, all the basic principles of good programming can be viewed as consequences of this approach.

Security programming implies that when the outside world reveals a discrepancy between your project and reality, the system reports an error - which means that you can correct the error and live on, and you do not need to repair the corrupted data that resulted from the under-processing of the error ( or: meticulous checks ensure that everything that the system is not adapted to will be clearly visible).

Building the simplest construction that can only work means not to fall into hypothetical assumptions, which means not only protection against the fact that you predict the future, but it’s also the fact that you have just the least. Less code means less bugs, less architectural elements means less conceptual errors (or: YAGNI is a preparation for adaptation).

Sharing responsibility means that implementation details do not “leak” to the outside world, because when the outside world does not rely on a specific implementation, and you suddenly realize that your implementation is crap, you can redo or even replace it and not worry about redoing the code out this part of the system (or: compartmentalization is the primary externalization within an aggregating system).

Total

Your design sucks.

And always will suck.

If your project does not seem to suck right now - this is because you are missing something.

If you are sure that you are not missing anything, you have forgotten that you cannot see the future.

Design under the assumption that later you will definitely, absolutely exactly, be redesigned.

And yet, remember that the new option will still suck - perhaps just less, or at least in another way.

And most importantly: do not worry too much about it. The “hand-face” sound is not a recognition of incompetence, it is a noise made by a problem turning from insoluble to trivial.

Personally, I like to realize that I was an idiot, because it means that I'm going to do better than it was.

So it's normal that your design sucks.

Continue to settle. Continue to settle a little less. Continue to settle in a different way. Keep studying.

And remember - to release something slop is 100% better than not releasing anything at all.

Successful hacking.

About the author: Matt Trout is a Perl developer, co-founder and technical engineer of Shadowcat Systems Limited (consulting in software development), by ORM DBIx :: Class, one of the mentaners of the Catalyst web framework, the author of many modules on CPAN.