Simple things with difficult AppEngine

I thought to do in the toy , about which I wrote earlier, a simple thing - to calculate what place a person occupies in the overall rating:

As I wrote, the game uses AppEngine for various statistics. I will tell tackles about optimizations that had to be applied for this simple feature.

It is obvious that to calculate the place in the rating on the fly is quite a resource-intensive task - you need to sort all users each time and with an impressive amount of them, the page will start to slow down. So I decided to denormalize this value and added the appropriate field to the user class:

class Player(db.Model): name = db.StringProperty() scores = db.IntegerProperty() #  #... rank = db.IntegerProperty() # <--   

Each time a player gains or loses points, his place is recalculated as follows. Suppose a table of players looks like this:

Name	Glasses	A place
...
User 1	123	50
User 2	121	51
User 3	111	52
User 4	105	53
User 5	100	54
User 6	99	55
...

')
Now, if User 5 scores 21 points and has a total of 121, I recount the places of all players with points between 100 and 121. Before the conversion:

Name	Glasses	A place
...
User 1	123	50
User 5	121	54
User 2	121	51
User 3	111	52
User 4	105	53
User 6	99	55
...

After recalculation:

Name	Glasses	A place
...
User 1	123	50
User 5	121	51
User 2	121	52
User 3	111	53
User 4	105	54
User 6	99	55
...

To my surprise, such a simple algorithm gave a high load on AppEngine (CPU goes off scale):

I was surprised, and decided that the problem is in more than 30 fields in the Player class, and there is no update without selecting all of these fields in AppEngine. I then decided to disengage the rank field from the Player class and make a separate “plate” for counting places.

 class Player(db.Model): #...  30-  ... rank = db.ReferenceProperty(reference_class=PlayerRank) class PlayerRank(db.Model): score = db.IntegerProperty() rank = db.IntegerProperty() 

Recalculation now occurred only for the PlayerRank class with two fields. Well, of course this helped a lot with something, but the result is not satisfactory:

Obviously, AppEngine who spends all its resources recalculating a place in the ranking is a bad AppEngine. The problem turned out to be too many operations. For example, if a player has 1 rating point, and another 1,000 players have 2 points, then by typing this player just a couple of points you have to recalculate all 1000 other players who now need to lower the place. I had to optimize further as follows.
Keep the space occupied not for the player, but for the number of points. Those. if 1000 players have 2 points, then they all will have one place (say the 3000th).

Glasses	A place
...
2	3000
one	3001
...

Thus, it is necessary to recount not a thousand player records, but only two.

 class Player(db.Model): #...  30-  ... def rank(self): return ScoreRank.all().filter('score =', self.score).get().rank class ScoreRank(db.Model): score = db.IntegerProperty() rank = db.IntegerProperty() count = db.IntegerProperty() 

The ScoreRank.count property is added to control how many players have a given number of points. If this count becomes 0, the ScoreRank entry for the given number of points is deleted.
AppEngine responded:

Conclusion

On the one hand, AppEngine is forcing the developer to write intricate algorithms, in which the need would have disappeared, use the developer's relational database and, say, LAMP. On the other hand, the algorithms thus turn out fast, the pages fly, in other approaches these pages might be a bottleneck and brakes (think brake online stores on php + mysql). From the third, AppEngne quotas are surprising. 10 thousand requests to select an object with 30 fields and update it threw out the application for a free quota. The understanding comes that AppEngine is expensive , contrary to popular opinion.

Link to the toy: www.vkubiki.ru