How to cut down a feature and not shoot yourself in the foot

The main goal of the projects is to make money. The project I was working on was not an exception.

I am the developer of Wheels Market Wheels and today's post will be devoted to how we differentiated the prices of paid services on our “classified”.

Our company develops 3 products, each for 3 platforms - web, android and ios. Users can apply various paid services to their ads, for example, a paid extension of the ad's lifetime or placing hot offers in a block.

When I was attracted to this project, in my head, even before the discussion began, the thought kept that for differentiated prices?

Differentiated price - the price of the formation of which depends on the characteristics of the ad (region, brand, model, year, etc.).

The team had the task to increase the average check. It was decided to “gash” the feature, which contains a functional about which further and will be discussed. The meaning of the feature was that through the admin panel we will be able to change the price of any paid service, relying on different parameters.

At the time we started development, we already had microservice written in Go. Sites and applications communicated with him through the client, sending the ad object to a POST request, and then, receiving in return the prices for any paid services, rendered them to the user.

In the process of studying microservice, it turned out that the prices given to the user were hardcoded, that is, the price for placing an ad in hot offers was described by the “hot” variable: 300, and the result of the answer looked like this:

{ status: "ok", data: { color-red: 45, hot: 300, paid-auto-re: 5, re: 0, s: 90, unset-auto-re: 0, up: 200 } } 

It was decided to do an important refactoring and get rid of hardcode in favor of a method that would give a differentiated price for the service.

We have divided the development process into several stages:

1. The choice of data storage format.
2. Development of the admin panel for price regulation.
3. Differential price return method.

Choosing a storage format

Initially, according to the statement of work, the product manager wanted to be able to set the price for a paid service from the administrative panel and this was one of the mostly important tasks. Before me there was a question in what format to store the data.

I decided to fill the database with “rules”, that is, if the category of the ad is “Auto” and the region is “Almaty”, then we apply the following price for the ad. It remains to deal with the database.

The first thing that came to mind was the MySQL database where the table with the rules for prices would be stored. For example:

Id	catId	regionId	coeff	serviceName
one	13	14	1.4	hot

According to the TOR, it was necessary to set the price based on the region and category. But, knowing how all these “Wishlist” work, I thought that the price would need to be changed not only for the category and region, but also for a certain car model or for some other reason.

In general, MySQL was no longer an option, and the choice fell on MongoDB, which would provide us with the broadest possibilities of dynamic scaling and could work with data arrays, without which the rules would be useless.

In principle, at the time of development, all our ads were already stored in MongoDB. Feeding the rules to regulate prices there was also easier. We stopped at this

Admin panel development for price regulation

The admin panel is not a difficult thing, it should have had the standard CRUD functionality, that is, adding, editing, deleting and displaying these rules in an easy-to-read form.

We wrote this whole thing on phalcon, since this admin panel was only part of the main admin panel for a site running phalcon. We also wrote a functional in the API that validated and saved our rules to the MongoDB collection.

The ad json object looked like this:

')
As we see here, there are some levels of nesting, that is, you need not only to save data with nesting, but also to do a search for this nesting in MongoDB. To validate the data, we introduced a small collection, where we stored possible fields for use, so as not to store everything that the user sent to the API.

Differential Price Return Functionality

The last stage of our development was the stage of writing code that could work with all these rules and would return a response with a differentiated price to the request.

Like before:

Like now:

And where is the refactoring and where are the differentiated prices? But.

These hard-coded variables in the code in the end remained as default values.

The algorithm for returning prices used to work like this:

1. Formation of an array with prices for services.
2. Exception handling by type for this section. If the service is free, then give 0.
3. Return of the result.

Now everything works as well, except that before returning the answer to the user, now there is a campaign in the method of price differentiation.

The first approach to solving the problem was as follows.

At the time of the formation of the array with the prices for each of its elements, that is, services, there was an additional campaign in MongoDB.

The “getPriceForService” method returned the price and took the following arguments:

1. Ad object
2. Service Name
3. Price before processing

Remember, I wrote above that there is a collection with possible rules for regulating prices, and they are needed here.

In order not to pass through each of the fields of an object, only a selection of possible rules was made. On the next screen, we see the process of forming the request in MongoDB.

 //      func getPriceForService(advert *Advert, serviceName string, basePrice int) (result int) { var mongoResult map[string]interface{} query := bson.M{} query["serviceName"] = serviceName query["$and"] = []bson.M{} //     for _, data := range getUsableValuesForPrice() { var value, err = advert.GetValueString(data.Rule) stringVal, err := getStringFromMixed(value) if err == nil { list := []string{"rules.", data.Rule} var str bytes.Buffer for _, l := range list { str.WriteString(l) } if err == nil { query["$and"] = append(query["$and"].([]bson.M), bson.M{"$or": []bson.M{bson.M{str.String(): stringVal}, bson.M{str.String(): nil}}}) } else { checkErr(err) } } } //     err := mongo.GetSession(). DB(mongo.GetDbName()). C("COLLECTION_NAME"). Find(query). Sort("-priority"). One(&mongoResult) if err == nil { stringResult, err := getStringFromMixed(mongoResult["coeff"]) checkErr(err) coeff, err := strconv.ParseFloat(stringResult, 64) if err == nil { //     return int(math.Ceil(coeff * float64(basePrice))) } else { checkErr(err) } } return basePrice } 

In the end, we received a request that could only fulfill and calculate the new price for the service, using the received coefficient.

However, after the release of this code in production, our MongoDB died due to the fact that with a single request to microservice it gives results for all services, and I call the method when forming an array for each element. That is, I increased the load on MongoDB by 7-8 times, and by the sweat of my face I began to rewrite my code.

Information note: To work with MongoDB, we used mgo , which allows you to easily build queries to the database .

That evening, having studied the code anew, I decided to call this functionality at the very last moment, that is, before giving the results to the client. I will knock on the same method, just a little rewritten. The rewritten method began to accept not the name of the service anymore, but the list of services with prices ready for return.

 //     func getPriceForServices(advert *Advert, serviceList Services) (result Services) { var mongoResult []map[string]interface{} var services []string for key, _ := range serviceList { services = append(services, key) } query := bson.M{} query["serviceName"] = bson.M{"$in": services} query["$and"] = []bson.M{} //     for _, data := range getUsableValuesForPrice() { var value, err = advert.GetValueString(data.Rule) stringVal, err := getStringFromMixed(value) if err == nil { list := []string{"rules.", data.Rule} var str bytes.Buffer for _, l := range list { str.WriteString(l) } if err == nil { query["$and"] = append(query["$and"].([]bson.M), bson.M{"$or": []bson.M{bson.M{str.String(): stringVal}, bson.M{str.String(): nil}}}) } else { checkErr(err) } } } //     err := mongo.GetSession(). DB(mongo.GetDbName()). C("Collection_name"). Find(query). Select(bson.M{"serviceName": 1, "coeff": 1}). Sort("priority"). All(&mongoResult) checkErr(err) //   ,  for _, element := range mongoResult { coeff, err := getStringFromMixed(element["coeff"]) checkErr(err) intCoeff, error := strconv.ParseFloat(coeff, 64) checkErr(error) serviceName, err := getStringFromMixed(element["serviceName"]) if val, ok := serviceList[serviceName]; ok { price := int(math.Ceil(intCoeff * float64(val))) serviceList[serviceName] = price } } return serviceList } 

As before, after receiving the request and completing it, we get the data with the rules for each service and the coefficient by which the old price should be multiplied.

This approach eliminated all unnecessary trips to MongoDB, thus we stopped loading our base and received differentiated prices :) (profit).