GWT - we pry into the transmitted data

GWT is a great framework. I am a Java developer, and I have worked with thin clients using JSP, JSF and GWT. There is nothing special to say about JSP, the technology is almost extinct now, but JSF had to cook a couple of years on two projects, and the impressions, to put it mildly, are not the best: the JSTL jumble, HTML, JavaScript and other things give untold “pleasure” that comes to to ecstasy in the moments of analysis of the incomprehensible behavior of some complex page. Yes, in the examples everything looks neat and simple, but the real life is not the same, and the JSF-pages of the project are even medium-sized and, it seems, with a competent, leisurely approach to design, using templates, it still begins to “smack”, especially readability. In GWT, everything is quite neat, because we write in our native Java language, albeit in a truncated version, but the fact that there is more than enough.

If we say that huge JavaScript files are loaded on the client in GWT, then my opinion is this. In GWT, when you first log in, the client loads the logic, albeit a lot, but only once. After loading between the client and the server only the pure data of business logic go. Specifically, I have the first download of 3MB, then AJAX-requests / answers, it really depends on what data volumes you have. In JSF, the data and the description of the interface in which they are displayed go to any page in the browser. In most cases, the size of the interface description (html-a) is many times larger than the size of the data itself. To show 1KB of data on the client (several lines of the table, for example), you will have to send 15-30 kB of html to the browser (we assume that the images and scripts are cached). But the data on the page is usually much more: headers, menus, various blocks, etc. In fact, a page fairly rich in data and functionality usually weighs at least 100–200 kB. GWT, for displaying the same page, will take 1 kB of data from the server and that's it. If users are permanent and for a long time are on the site, then the loss of the primary 3MB is compensated in half an hour. With this in mind, GWT is ideal for the implementation of jobs, but for ordinary sites it may not be so good.

In any case, this is just an introduction in order to “plunge” into the topic under discussion, and the article is not about technology comparison, but about how to peek at the data between the browser and the server can help solve problems and optimize client-server interaction. At the time of writing, we have already used a very much outdated GWT 1.5, and for the transfer of the DTO is just as old and no longer supported by the Gilead library. I have plans to upgrade to the latest version of GWT and reject Gilead in favor of the GWT request RequestFactory . Regardless of technology, the tools and debugging methods are the same. I used FireFox with the FireBug plugin installed.
')

How we found a memory leak

Once we decided to update the technologies used - GWT, Gilead, JBoss, Hibernate. Replaced the library, deployed the server, recompiled the project. Launched, tested, and decided to go into commercial operation. Updated server, waiting. The first reviews work faster. An hour later, everything started to “hang up”, after 15 minutes “got a stake”, and a little later the server fell through OutOfMemory. We started frantically trying to figure out the problem, restarting the server every hour at that time. Again, the article is not quite about this, it ended up that the leak could not be found, and we rolled back to the old version. For a long time I tried to analyze the dumps, do load testing, but did not find the source of the problem.

A month later, the tester began to complain that one of the journals in the System opened for a long time (in turn, the users complained about this). I did not reproduce this problem, I asked him to install FireFox and FireBug and see what was happening there. It turned out that when a journal was opened, a package about 4 megabytes in size was loaded onto the client (!!!), in order to display a table of a dozen entries. The source of the problem was immediately found - on the server, when selecting data for display, there was no limit on the number of records read from the database, and the entire contents of the database table was loaded on the client. In my test database of the developer, there were fifty records in the table, and the tester had a fresh dump of the industrial database with thousands of records in this table. Fix the problem - add one line of code.

I think that the reason for the "fall" of the server when updating technology was in this magazine. Why did old technologies cope? Apparently, data caching was less used in them (indirectly confirmed by reviews that new technologies worked faster). Unfortunately, the file hit so hard on our users (and of course on us, developers and implementers) that the next attempt to update JBoss was completed only a year later, but there are no official plans for GWT and Gilead. This is how a forgotten line of code can ruin grandiose plans, steal hundreds of man hours of work, and add gray hair to many people.

How to save network traffic and memory

Do not be lazy, and spend a couple of hours looking at the FireBug that is transmitted to the server and back, you can find interesting things. A specific example: the client (browser) periodically requests the presence of notifications from the server. In fact, it is necessary to transmit to the server information about who requested, in response to receive a list. We have a User object stored on the client that contains a lot of information about it, including various list data. So, the method requesting data from the server was implemented simply - the entire User object is transferred. Many other methods are implemented in the same way. Having looked under debugging, I saw that the size serialized in line and transferred to the server of the User is 15kB. But on the server in the user's session cache there is a copy of this object, and in order to get the data requested by the method, the user ID is generally sufficient. If we replace the User object with its ID in the method, the request size can be reduced from 15kB to 50-100 bytes, or even call the method without parameters, and take user data from the current session.

It seemed that for a modern network infrastructure of 15kB? Dust! But we should not forget that there are many requests and the user is not alone in the system. In addition, in addition to network resources, the server consumes processor and memory resources in order to convert the resulting 15kB into a Java User object, from which the ID will be pulled out proudly for the request, and everything else will be left to the garbage collector.

How I did load testing

When the server began to fall when we updated technologies, one of the attempts to find the source of the problem was to conduct load testing. A quick search and browsing of free utilities and software libraries did not help in creating a testing environment, so I decided to try writing something simple myself. All that is sent to the GWT server is HTTP requests, which means you can try emulating them with standard Java language tools. Only here what to transfer in requests? Again, FireBug came to the rescue. We go to the client side, perform any actions, in the FireBug we collect the contents of POST requests. The self-written utility for load testing turned out to be quite small and quite functional. A certain User object performs a repeating set of actions, i.e. sending HTTP requests. Several dozens / hundreds of such objects are launched in different threads and give a decent load on the server. Right now I'm testing under load the System to check after switching to JBoss 7: processor resources are used by 90% of the server, 500MB of memory is consumed during the night, when entering the client part there is a noticeable “inhibition”, which means that the load is good!

Conclusion

All of the above applies not only to GWT, but also to any thin client: see under debugging what goes between the server and the client. This not only can give an indication of bottlenecks / bottlenecks, but also help in better understanding the essence of the technologies used.