Being a contemporary full-developer

I live on the periphery of the technological get-together. And on the periphery in a geographical sense. This means that:

• I have never been to professional conferences. Just because this opportunity has never been given.
• I have never bought training courses: it’s strange for me to pay for what you can learn on your own internet or on books. It is tempting, of course, to gain concentrated knowledge, supported by the implementation of practical tasks, to get a certificate. But I never had the money or the time for that.
• I am an adherent of digital asceticism: not by my own will, but as a result of eternal restructuring in our country. In short periods of financial stability, I update the electronics on a residual basis. And now I am writing this text on Core2Duo a decade ago. Still waiting for the next stable plateau on my income curve.
• Three times in my life I bought games. In the 90s. These were cassettes for the outgoing Spectrum. I look at the purchase of software with surprise: when everyone used pirated software, I also used it without understanding the essence of the issue. And then, at the beginning of the two thousandths, I completely switched to Linux and there was nothing to buy.
• Surprisingly, I have never lost my mobile phone, from the very beginning of the emergence of GSM-communication. And they never broke and were not repaired. All three pieces are neatly on the shelf, and the fourth, a smartphone, is in your pocket. He is five years old. I don’t have any experience buying software on Google Play, I don’t even know how to do it. My wife knows, but I don't.
• I beg providers to blacklist me and not to call me with offers to buy online or a television package. Because I do not have a TV, but for the Internet, the minimum speed is enough for me. I ask you not to tell me about the new mobile tariffs, because I choose the cheapest one available.
• I have never seen live English-speaking foreigners. And never spoke with foreigners in a voice. I have never been given such an opportunity in my bearish corner, although it would be interesting. I am slowly pumping my English, but I only need it to read literature.

Probably enough. Who am I, and what am I doing in Habrahabr? The fact is that I ... I am a permanent survivalist. In other words, a modern full-stack developer. Oh, I imagine how profile programmers frowned now! Fullstak ... What can I tell them? Guys, I would love to pump skills in one direction. I would be glad, like you, to plunge into the subject and become a highly specialized guru. But, unfortunately, the reality is that in the regions a person at a remote place has to grab any job, just not to go anike or not become a taxi driver.

What languages ​​did I have to study and give out finished things? If you make a sample on a time scale, you get the following list:

Yes, I myself am afraid of this hodgepodge. What an enchanting throwing? Is that right? Web - strength analysis - inventory accounting - geymbling ... Nuclear power stations, cartography, telephony. Author, are you serious? Absolute! Life will squeeze - still not so raskoryachishsya.

Edge of transition

To get a feel for the transition from project to project, you can look at the two lowest lines in the table above. For the project from the penultimate line "Cartography and navigation, mobile development for Android" we will manage only links.

• You can read here: Annual Report for 2018 ;
• Watch the video here: Industrial development on Qt and QML ;
• For aesthetic critics, there is even a section on project economics .

And for completeness, a couple of publications:

• Year after dismissal from Rosatom
• Completion of high-precision navigation tests on agricultural machinery

And about the bottom line of the table "Web development and telephony" I will discuss later in this text.

So, a promising mobile application project for Android for custom navigation equipment is abruptly closed. Just yesterday, the developer wrote the backend code in C ++ and the front in QML, bolted the native code to Java through JNI, and today he frantically must search for a C ++ developer job at a remote place. The modern C ++ market is such that finding an employer with C ++ in a remote place is a great success. All employers want to see a C ++ programmer in the office. Month in the search - a complete zero. It's time to retrain, good background allows.

Definition

A full-stack developer (a member of the family of specialists around) is a mythical character, the subject of an employer's desire, who dreams of optimizing software production to a team of one person. A full-developer has magical abilities: it has a bottomless memory, for it knows all modern languages ​​and technologies; a global conceptual apparatus is integrated into the brain, surpassing the organization of the thinking process of Vladimir Lenin, Albert Einstein and Leonardo da Vinci; systemic thinking of such a specialist is able to debug anything directly in the brain, without the use of debugging tools. Unpretentious, eats sunlight.

Transition to a new task

Suddenly, I get his arar , and takes me to my team. He has several tasks hanging on it, and on trial I implement a prototype web service. As planned, the web service notifies users about various events via various communication channels, including telephone calls. As a result, the site appeared QrCall.org , about which it has already been written on Habrahabr: Go there - I do not know where .

Looking ahead, I’ll write the deadlines right away: it took two weeks to discuss and write the terms of reference, it took a month and a half to create the first implementation and output to production.

To make it clear, by now this web service looks like this (developer version):

So, we need to jump from C ++ to PHP7 and the stack of tools corresponding to modern realities. There is no time for the buildup (C) Putin. Stroustrup, Shildt, Gottshling, Salter with Kleper are placed on the far shelf. In parallel with the writing of TK, I remember that it was taken in PHP7. The request "what's new in PHP7" gives several articles on Habrahabr and in programmer blogs. Yeah, namespaces and imports, a new kind of ternary operator and any syntactic sugar, scalar types and their ilk, anonymous classes, refinement of closures, generators ... Most of them know everything. Once again I note for myself that PHP is a heavenly delight compared to the harsh advantages.

Selection of tools

Telephony

Most worried about the future implementation of telephony. Will I master? Immediately it is clear that you will have to work with SIP, but how? There is a memory in the brain that once was played with some console SIP client, and could even dial a phone number and make a call. To solve this problem will be enough. In extreme cases, you will have to be confused with Asterisk. I called a familiar signalman, describing the essence of the problem, please remind what console client I could feel. The verdict is straightforward - this is Linphone and its linphonec console. But dialing a number in the console is not enough. We still have to play a sound file in the virtual tube. I install Linphone, I go to its console, I look at the possibilities. So, there is an opportunity to switch from a sound deyvays to a file. It's good. And in the console there is a play command that launches a sound file for playback. In principle, nothing more is needed.

Although no, there is still the problem of parallel calls. I discuss it with itsar , he says that the call alerts are rare, line them up just in the queue. Only limit the duration. Oh well. If so, then there are no more questions on the phone.

PHP framework and package manager

Next, you need to decide on which framework to do the project, and at the same time you need to sort out the package manager Composer. I used to look closely at Composer and put Yii2 without components through it, because everything that was needed was already included in Yii2. Well, whatever framework framework I choose, Composer will still be needed. I read how to install it. I put works.

Next question on the framework. I find out that in 2019, Yii2 is no longer relevant, and Yii3 is stuck in some kind of intermediate state. What remains? For Zend and Symfony, I have not yet matured, so there are practically no options - only Laravel . I read the documentation, I watch the tutorials, I order a book of the Russian-speaking author (it turned out to be very sensible, what I need to start). After the ancient Codeigniter and irrelevant Yii Laravel framework is understood easily, you can immediately see how a programmer's idea advanced in the design of web applications. Everything that was already dreamed of is already implemented, run-in and overgrown with standard approaches. Yes, the project is planned not loaded, so I can afford some Nubian curvature in implementation.

I put Laravel "by default", suggesting Composer to decide for itself which version of Laravel is currently relevant. He puts 5.5. Well, let it be this version, it is more run-in than 5.8, then it will be easier to solve problems. We do not pursue innovations.

NPM

Some Laravel components, for example, the system for assembling and minifying CSS Mix files (add-on over Webpack), require the Node.Js JavaScript server runtime and the npm package manager written in JavaScript to work. The Debian Linux Stable that I use already has the npm package. However, it is a rather ancient version, and is not suitable for Laravel 5.5 infrastructure. I figure out how to install from third-party sources, find deb.nodesource.com, and install from it. Hmm, strangely, in the same package, npm is put together with Node.Js. This is not a Unix way at all, but oh well. The main thing that works.

Layout

The idea of ​​the QrCall.org project is to call the user through a QR code . And this means that visitors will enter the site from mobile devices, using the cameras of which this QR code will be scanned. At the same time, user registration, setting up alerts and printing of QR codes will most likely be done from desktop computers. So, without adaptive layout can not do.

Immediately notice the generation of mobile / desktop content on the server by analyzing UserAgent. This is not our approach for 2019. Here the CSS framework Bootstrap will definitely help us. In general, the layout of web applications is a separate, gigantic, big topic that a separate specialist should deal with. For me, web development is nothing more complicated than picking with layout. I realized a long time ago that I have a cretinism of layout. I spend a tremendous amount of time to make the next regular indent, or to align several elements. But we don’t have the resources for the typesetter, so you have to do what you can, it’s desirable that the result be even and beautiful.

The question is: which version of Bootstrap to use? 3 or 4? It turns out that Bootstrap comes bundled with Laravel 5.5 right away, and this is version 3.x. There is no time to sort out how to redo the environment on Bootstrap 4, so I’m leaving version 3. In the end, there are hundreds of thousands of websites on the Internet that use Bootstrap 3, which means it's a decent technology to use.

The most interesting thing is that as a result it turned out to make an adaptive layout not only for the “open” part of the site, but also for the personal account.

Here is the page in the desktop version:

And here she is in the mobile view:

Development

How will the framework be used? I have this approach: to maximize the use of all ready-made fremverk components, but with one condition: if there is a good understanding of how this component works. As one person, who already knew Zen Laravel, said, "This is not about the fact that the documentation is written in a non-Russian language, but about the fact that even in native English for the framework, it is not always revealing." Therefore, I believe that if you can not quickly deal with a component or method, it is better to do something simpler using your own methods than writing code that is poorly comprehensible to yourself.

What I'm talking about? The Laravel framework is a large framework with many implemented abstractions and with its approach to the structure of the code. There are simple things in it that have been successfully used for a long time both in Laravel and in other frameworks. There are complex but understandable things, for example, the implementation of queues (which will have to be used for telephony). And there are really complex fundamental things, which cannot be penetrated with a swoop. For example, it is a bunch of Service-container + Service provider + Facade. So far, I have understood how to make a purely mechanical service provider, place it in the service container and fasten the facade to all of this. But for what it is necessary to do it - I have not realized yet. It seems like this approach reduces the code, you can refer to the abstraction and its methods in a static style, without using the keyword new (dubious advantage). And the use of facades makes it easy to organize automated testing of a web application, and as a side effect of all this convenience, using a service provider automates dependency injection. In general, while there is no understanding, it is easiest for me to get along with the usual helper classes, which I do.

Technology

So, to summarize the above, I have the following, quite traditional technology stack:

• OS: Linux (I choose Debian Stable)
• Language: PHP7
• Framework: Laravel
• Front End Whistles: Bootstrap 3 + Vue.js
• Database: MariaDB (also known as MySQL)
• Web server: NGinx
• Telephony: SIP + linphonec / linphonecsh
• VCS: Git

Yes, not the most fashionable and advanced stack. But we need to do business, and not to practice in wrapping the environment in Docker-containers and not to promote the idea of ​​JS-only development.

Turn

Queues are a very specific subject area, and in large teams, a specially trained specialist is usually responsible for the queues, ensuring the work of queues on hundreds of servers. In our case, hundreds of servers are not observed, so you need to use the queue as simply as possible and in the case. Therefore, I decided to use the MySQL relational database engine already used in the project as a repository of tasks. There was some temptation to make one's mind, for example, based on Redis, but there was simply no time to deal with this NoSQL database.

During development, a request for a task appeared that could not be directly solved by means of a queue on MySQL, but which would be solved using Redis. The problem was that after successfully completing the task, the task is deleted from the queue, and there is no way to check whether, for example, a specific task was performed in the last 10 minutes? When using the Redis repository, this could be implemented via Rate Limitting, but when using MySQL, this is not possible. Therefore it was necessary to implement a similar functionality simply on the basis of analytics of the action log. Fortunately, the action log is an indispensable part of our small information system.

Paid services

When placing a site on the Internet, you must pledge to pay for various services. Hosting and domain name are always almost obligatory payments. The hosting was cheap, paid for 2Gb RAM, at the rate of 1Gb for the database, the rest - for the operating system and the execution of scripts. In the tariff there were 2 microprocessor cores, although I think that one of them would have done an excellent job with our loads. 20Gb disk space is more than enough for our project. The deployment process required a compilation of linphonec , because on my developer’s desktop there was an older Debian Linux than in the ready-made virtual machine image provided by the hoster, and the standard repository package contained an ancient version of this program with several unpleasant “features”. And now, on the compilation of linphonec, 2Gb was not enough for me. The magic of templates in C ++ relieves the memory as if not in itself, so I had to set up a swap, after which the build was successfully completed.

SIP telephony has never been free either, but there are several large IP telephony operators in Russia that compete with each other and offer very low-cost tariffs. The only incomprehensible point was that when ordering the tariff, the operator convinced that the tariffing would be per second, but in fact it turned out to be per minute. But this must be dealt with separately.

In our modern realities, it is impossible to count on the fact that there will be free Email-sending services. And in our case, email distribution is needed to send various alerts. In my experience, all attempts to organize the sending of many letters via Yandex.Mail or via Google.Mail only lead to the fact that the mail servers of the receiving party, after three or four letters, mark the messages as spam. That is, problems arise already at the debugging stage, not to mention production. So I had to be confused with the Mailgun service, through which letters are delivered quickly and without problems. With Mailgun, only one thing is not clear: in some articles they write that they give you to send 10,000 emails monthly for free. And on the Mailgun site itself, it is somehow slippery that I understand 10,000 emails from the moment of registration. In any case, the service has not yet overcome this limit, so you just have to watch.

There are some difficulties with setting up Mailgun. First you need to be confused, and make the correct settings in your account, linking it to your domain name. After that, you can make a test email from the site using any suitable command or script, for example via SMTP. And in Laravel there is even a ready-made connection driver to Mailgun, thanks to which you don’t even need to configure the mail subsystem to send emails at the operating system level.

Fairvol

Fairvol I always adjust simply and primitively, with just three rules. The first rule for INPUT permits the passage of packets for established connections. The second rule allows connections to certain ports. The main thing is not to make a mistake, and not to forget about the port for SSH, but it will be oh. The third rule is the DROP of all other INPUT chain packets. Of course, options are possible in this scheme, but this is the basis.

Since SIP telephony is used in the system, in order for the kernel to correctly see the SIP protocol pacts for the established connections, and not to chop them off, you need to have the nf_conntrack_sip kernel module loaded . Still, if the host is hidden behind NAT, it is recommended to include the nf_nat_sip module, but in my case it was not required, it works and so.

Backup

Now the server is running a self-written script that is run-in on other projects. He receives a list of commands and a list of directories. The list of commands is the commands that need to be executed before backup, for example, MySQL dump commands. A list of directories - these are directories, the contents of which must be reserved. The result is a zip-archive that folds into a specific directory, and the script ensures that such archives do not add up more than a certain number. If more, older archive files are deleted. The archive directory is synchronized with another host via rsync.

I understand that this approach is solid crutches, so I plan to move to backups via borg . But for the time being it is tuned in such a way as it is easier and more convenient for me.

Depla

For deployment, I use the GIT version control system and specially written scripts, divided into stages, with one central launch script. The code being developed is pushed to the central repository server. At the moment when you need to update the site, run scripts that run part of the commands from root (stop and start all sorts of services), and part of the commands from the web server (get the code via git , launch the artisan utility). To do this, the settings in / etc / sudoers are made and the permissions for the script files are set so that they can be executed by a specific user, but cannot be changed by any other third-party user.

Due to the fact that Laravel has a migration system, no third-party utilities are required to update the database structure and populate tables with primary data. In fact, with the Deploy, only the code is transferred, and this is enough. The system has already gone through several regular updates, so far the flight is normal.

Hey, comrade! - will tell me. And where is your continuous integration? And I will answer: fear God! The project is not so extensive as to bother with CI systems. If the project starts to bring dividends, then we will put together a team of programmers, and solemnly hoist the CI system on top of everything else, and then everything will be feng shui.

Flaws

We are all smart in hindsight. It's time to think whether strategic miscalculations were made? Yes! Could something be done better? Of course yes! Below I will list the mistakes and shortcomings of this project. Some things are easy to fix; some are time consuming.

A major miscalculation was that I hoped for the default installation of Laravel, and used Bootstrap 3 instead of Bootstrap 4 as a CSS framework. Still, Bootstrap 4 stepped far ahead of Bootstrap 3, and in addition to the flex layout there appeared a lot of those things that Bootstrap 3 just did not have, and that had to be done by hand. Now you can crawl on Bootstrap 4, but the layout will definitely go, but with my typeset cretinism it will take a long time to straighten it.

All modern sites that work with user authentication are required to work on the HTTPS protocol. Unfortunately, my hands have not reached this stage yet, and the stage is very important. I already have experience in translating websites from HTTP to HTTPS, I even wrote a memo for myself Configuring Let’s Encrypt HTTPS certificates on the NGinx web server , but I need time to do this.

I repent, but I still have not set up a rotation of logs. In our information system, three non-standard logs are created, and they need to be processed so that the disk space is not devoured. It seems like a simple thing, but you need to remember how it is done, pick up the logrotate , watch for a few days how the logs are created.

Testing. Yes, I suffer a lot from the fact that I do not have time to create a testing system. I do not write unit tests, since the development is mostly related to the creation of the user interface and the mapping of user actions in the interface to the database. By and large, I need to organize a functional testing system. Unfortunately, there is simply no time for this, and individual testers are not provided. I know that Laravel has helpers specifically for functional testing, and if the project develops, I will try to close this technological debt. Judging by the description, it is quite simple to make a block of tests that at least just checked the response codes of the pages.

Total

Many have heard about such a thing as "continuing education" or "lifelong education." In fact, all my life I just do this. , , , , : , , , ( , , ).

? "". ? -, , . , , , , . , .

, , " , , " , . . ++ , . : , " ", , , , , . -. - , . , , . - : , - , , , .

— , - , , , . , proof-of-concept, , , , . , — , .

, - . , , . , - , , -. , , , ...

? : -, , , , , .