100 thousand visits per day (debriefing and new experiment)

In my last article about the speed of working with GAE data, a graphic counter of impressions was embedded. Everyone could see the value of the counter and consumed CPU resources. As I already said, the counter was quite “heavy”: the load it creates is equivalent to displaying 1000 records from a database on a page without using caching.

The experiment with the counter turned out to be very useful, and its results are somewhat unexpected for me (different from requests from a single IP address). I want to share the results of the experiment and set up a new experiment, taking into account the errors that have been passed. By the way, the source code of the new graphical counter is available for all and is given in the article.


')
Query schedule



Maximum 7 requests per second (so many open the main page of Habr in the evening). In the daytime, I think this figure rises to 10 requests per second.

CPU consumption graph



The graph captures such an unpleasant moment as a lack of activity. At this point, the article was on the main page, but the pictures were not displayed, since the free resource limit was over. In the morning the situation was corrected.

Resource consumption for the first day



There is no resource consumption on the graph until 10 am MCK (since at 10 o'clock the resource counters are reset). Till 10 o'clock about the same amount was spent. In total, about 150 thousand requests and 500 MB of traffic (some requests for a static image, but they are much smaller).

Number of running instances

Unfortunately, by mistake I deleted the screen with the running instances. With 6 visitors per second, 16 instances were launched (this is the maximum I recorded).

Inhibited requests ???



And here is the most interesting. After a while, a problem arose: approximately every 5th request started to be executed 5-10 times longer than the average (on average, the counter was loaded for 0.5 seconds). Initially, even with many visits, this was not observed. The situation has not changed until now, when there were very few visits.

How to explain these errors? Options:

1. Warm up request. No, since not enough time has passed for the instance to unload (for example, on this chart, the time between 3 erroneous requests is less than a minute).
2. Simultaneous modification of the record in the repository with the same key. No, since it took more than 10 seconds between requests.
3. The application has been limited as ineffective.

Remains the last. This is despite the fact that, on average, the request was executed in less than 1 second (about 0.5 seconds). The algorithm for determining the inefficiency of applications has not been disclosed, so it is quite difficult to say something definite ...

Honestly, for me these mistakes remain a mystery.

Solution to the problem

If the problem is really that the application has been limited as ineffective, then the problem will not recur with the new counter code (org.toyz.litetext was previously used), which runs about 7 times faster. The formation of the picture is made very simple, but quite effectively (the code is shown below - you can use it on your website).

Second edition:

public final class DynamicImage
{
//
private static final String FOLDER = "resources" ;

private static Image backgroundImage;

private static final Map< String , Image> imageTable = new Hashtable < String , Image>();

static
{
try
{
backgroundImage = makeImage( "appengine.png" );

imageTable.put( "1" ,
makeImage( "1.png" ));
imageTable.put( "2" ,
makeImage( "2.png" ));
imageTable.put( "3" ,
makeImage( "3.png" ));
imageTable.put( "4" ,
makeImage( "4.png" ));
imageTable.put( "5" ,
makeImage( "5.png" ));
imageTable.put( "6" ,
makeImage( "6.png" ));
imageTable.put( "7" ,
makeImage( "7.png" ));
imageTable.put( "8" ,
makeImage( "8.png" ));
imageTable.put( "9" ,
makeImage( "9.png" ));
imageTable.put( "0" ,
makeImage( "0.png" ));

imageTable.put( "(" ,
makeImage( "leftBracket.png" ));
imageTable.put( ")" ,
makeImage( "rightBracket.png" ));
}
catch (IOException exception)
{
throw new RuntimeException(exception);
}
}

public Image drawText( String text,
Anchor anchor,
long backgroundColor)
{
if ( null == text)
throw new ArgumentNullException( "text" );

if ( null == anchor)
throw new ArgumentNullException( "anchor" );

Collection<Composite> compositeCollection = new ArrayList <Composite>();

// background
compositeCollection.add(ImagesServiceFactory.makeComposite(backgroundImage,
0,
0,
1f,
Anchor.TOP_LEFT));

int xOffset = 0;

for ( int pos = 0; pos < text.length(); pos++)
{
String symbol = Character.toString(text.charAt(pos));

if (!imageTable.containsKey(symbol))
continue ;

Image image = imageTable. get (symbol);

compositeCollection.add(ImagesServiceFactory.makeComposite(image,
xOffset,
0,
1f,
anchor));

xOffset += image.getWidth();
}

return ImagesServiceFactory.getImagesService().composite(compositeCollection,
backgroundImage.getWidth(),
backgroundImage.getHeight(),
backgroundColor);
}

private static Image makeImage( String imageName)
throws IOException
{
InputStream inputStream = null ;
ByteArrayOutputStream outputStream = null ;

try
{
String filePath = FOLDER + File .separatorChar + imageName;
inputStream = DynamicImage. class .getResourceAsStream(filePath);

if ( null == inputStream)
throw new IllegalStateException( "filePath=" + filePath);

outputStream = new ByteArrayOutputStream();

int length;
byte [] buffer = new byte [1024];

while ((length = inputStream.read(buffer,
0,
buffer.length)) > 0)
{
outputStream.write(buffer,
0,
length);
}

outputStream.flush();

return ImagesServiceFactory.makeImage(outputStream.toByteArray());
}
finally
{
if ( null != inputStream)
inputStream.close();

if ( null != outputStream)
outputStream.close();
}
}
}

* This source code was highlighted with Source Code Highlighter .

I ask the community to participate in the experiment. The results will definitely report in this article.

The results of the experiment made me happy! For half a day the counter was opened by about 100 thousand people (as expected). At the same time, 1 GB of traffic was spent (well, at least the picture was reduced, otherwise it would not fit into the 3 GB limit):



The maximum number of running instances was 9 pcs. (at 8 requests per second):



And requests at this moment were executed without delay. Here is an excerpt from the log:



At the same time, each request performed a record in the database and checking the uniqueness of the IP address. It turned out for FREE! Enough free resources!

Conclusion

With proper use, GAE is able to withstand a tremendous burden. Google really did what many dreamed about: paying only for actually used resources (all honest: if you don’t use it, you don’t need to pay anything). It remains for us to only create services that would attract such a number of visitors.