The miracle happened. Released "untied" version of the Arduino Mega Server

The revolution about which the Bolsheviks spoke for so long took place. Now you can take a microSD memory card, burn AMS distribution files onto it, and your Arduino turn into a small (or large, how to see) miracle. You no longer need "crutches" with the support of a third-party server, the Arduino Mega Server has become completely autonomous and completely "usable" in a single mode. And this opens up very interesting prospects for all of us.

There was a lot of talk about the fact that this is impossible and that the limitations of the microcontroller and the network card will not allow this, there have been many discussions and experiments, but now all this is over, we managed to solve all the problems and overcome all obstacles and still make it work like this how to.

In the first article about the Arduino Mega Server, I described the project in detail, talked about the technology and outlined approximate prospects for the development and use of AMS, in this article I will talk about how we managed to achieve autonomous server operation and what opportunities are already open to us.
')

Battle for autonomy

Autonomy of work is a fundamental moment. If you are “tied” to a third-party server, then this drastically limits the possibilities of using your projects. Judge for yourself that a small controller works, it needs support in the form of installation and configuration of a large MajorDoMo server. Therefore, all the forces were thrown to solve this problem, but ... it was not there. On the way to bring AMS to autonomous navigation, there were many unexpected and formidable obstacles.

The first obstacle is the low speed of reading files from a microSD card. In the first article, I mentioned that the bottleneck in this system is the speed of the memory card, so it turned out to be not quite so. The problem is not in the microSD card, but in the algorithm of the function

File.read(); 

It performs byte reads from the file and thus slows down the whole process. The average read speed in this mode is 25 kilobytes per second. The output was a switch to block reading with a buffer size from 32 to 256 bytes. The optimal resource / performance ratio was 128 bytes, and this increased the speed of reading from the memory card 10 (!) Times, to about 260 kilobytes per second. And this, as you understand, is a completely different calico. The read speed of 25 kilobytes per second simply put an end to the whole project because, in principle, it is impossible to transfer files in such a reasonable time with such speed.

 File.read(buff, MAX_BUFFER_SIZE); 

We successfully overcame the first obstacle, but when I measured the speed of uploading files to the network, I was in deep shock: 4.1 kilobytes per second. That is, all our efforts to speed up the reading of data from the disk broke against the wall of the tortoise rate of return of this data to the network.

 client.write(data); 

You have probably already guessed what the problem is and how it can be solved. That's right, replace the byte output to the network by block. What was done and gave a speed increase of 14 (!) Times up to about 57 kilobytes per second. And here our server became similar to the real one, it finally began to work, and not crawl with a snail's speed. By the way, version 0.11 works at a speed of 4.1 kilobytes per second and only the tandem mode saves it.

 client.write(buff, size); 

It was very cool, but as it turned out, it was still too early to relax, the real difficulties had not yet begun. The next epic problem was the delay in the responses of the server, which was devoted to an entire article (do not miss the fascinating reading). In short, it turned out that the standard Ethernet library, supplied with the Arduino development environment for many years, contains a genetic defect that makes it incompetent. It does not work correctly with requests and simply "hangs" non-single requests. In the meantime, the standard Arduino Ethernet library has been fixed.

Reducing the delay of server responses, in combination with the speed of reading files from a memory card and data transfer to the network, raised to a normal level, made the Arduino Mega Server a “serious” and truly workable system.

Iron wall

And here we rested in the circumstances of force majeure called the chip W5100 (I am aware of more advanced options, but this is a slightly different story). Its four hardware sockets impose very severe restrictions on networking with it. You can’t just take and download a webpage with a dozen links to CSS, JavaScript and HTML files through it. Each link generates a new request and a new connection and ... when there are more than four of them, then problems start with a delay in response. And in order for all this to work, you need to organize network interaction taking into account this feature of the W5100 chip.



To solve this problem, a kind of server-side programming language (macros) was introduced, which allows you to collect pages on the server side and, most importantly, allows you to manipulate the assembly of pages from sketches on Arduino and from the download pages themselves. This gives complete freedom and you become like a magician or a wizard and can do anything you want with the output pages.



For example, on the fly, change the style, design and functionality of the displayed pages. One can imagine this in the form of a certain three-dimensional matrix: one axis is the functional of the site, the second axis is design (visual presentation), the third axis is the topology of the site. In other words, one site turns into many sites, each with its own design, its own functionality and its topology. And the work of many sites provides a single core.

And all this happens on the eight-bit Arduino microcontroller.

beauty

Since we got our hands on such a powerful tool as AMS Matrix Engine, it would be a sin not to use it, and as a demonstration three additional sites were created to which you can switch with one click on the go (even reloading the AMS page will be done carefully). By the way, it was possible to make not three, but thirty three sites or one hundred thirty three sites (visually - topologically - functional presentation of information) - your Arduino will easily maintain them (not at the same time, of course, but we don’t need it).



Default (geek mod, you'll like it)



Home (calm design and “quiet” features)



Modern ("modern" design and functionality to taste)



Ampere (branding for your company, online store, school or laboratory)



I will explain, because not everyone could understand what was written above. You have a powerful microcontroller and dozens of pins and serving more than fifty sensors. Among them are temperature, current, security and others. Each group of sensors forms a logical cluster, for example, protection, climate, the health of your favorite cactus, etc. So, for each logical cluster, AMS Matrix Engine allows you to create a separate site with a separate design and functionality. It will not even occur to the user that all this wealth works on a “dead” and nondescript microcontroller gathering dust in the corner.

Perspectives of use

Your projects . Take the controller, fill it with AMS firmware and insert a microSD memory card with files from the distribution kit. It's all. The system will start and you can work with it. If you are not satisfied with the functionality, then you simply change and append everything you need.

Sellers Arduino iron . Instead of selling bare metal, you can sell ready-made configurations and (boxed) solutions for customers. The benefit is obvious - the threshold of entry decreases and the market expands. The client buys a set for his task and uses it. For this, he does not need either a soldering iron or programming knowledge: he works with the equipment, as with a regular site. The configuration is assembled as from Lego cubes without a soldering iron .



Branding Your company, your online store, your startup, your laboratory, in case you work with Arduino-iron, can supply your products with a branded AMS version with the functionality you need.

The creators of training sets . From AMS you can make an excellent training set. The controller itself will contain documentation, sketches, examples, ready-made experiments, etc. In combination with sensors that are connected with one click (without a soldering iron) and wireless sensors, this will be a fascinating set that “deactivates” any child for a long time.

Schools and training centers . On the basis of AMS, you can conduct training on the whole range of modern technologies: microcontrollers, robotics, Smart Home, programming, web technologies, website building, interface design, etc., etc.

Alternatives

I often hear that it is easier to take “Malinka” with more advanced network capabilities, but what about purely microcontroller functions? Mega has 16 analog pins and 54 digital ones (not counting other advantages). How do I (or you) connect the 14-channel voltage and current control unit to the Malinka? And where should I connect dozens of sensors and actuators on the Malinka? And here we get a powerful microcontroller and luxurious control over the network in one bottle.

Are there any disadvantages?

A conditional disadvantage of technology can be called a high threshold of entry (not for use, everything is just fine here, but for self-modification and addition). You need to know a lot of modern technology and be quite skilled user. As well as the early stage of the project itself and the lack of quality documentation.

But if you are not afraid of such difficulties, then you will not find a better simulator for mastering the whole range of modern technologies.

Latest current version

You can download the latest version from the official site of the Arduino Mega Server project. And ask any questions - on the forum . For those interested: a typical page loading time in a single mode is four seconds, sometimes slightly less, sometimes slightly more, but on average about four seconds. In tandem mode - about twice as fast. The scope for optimization is huge, but in general it is quite possible to use it.

Supplement . The channel on Youtube is open and here is the promo video for the Arduino Mega Server, which demonstrates how to work with a real system.