Sell Side Platform
Good afternoon, dear reader! Continuing the series of articles on the implementation of the RTB stack by our company, I suggest that you familiarize yourself with the implementation of our SSP - VOX Ad Exchange .
The load here is not as big as, for example, in our DSP , and the main processing time of the request is spent waiting for responses from connected DSPs. It is written in the form of an asynchronous ASP.NET MVC application.
So, let's begin to figure out how the processing of requests.
Request for advertising begins to form on the client side. Here, using javascript, we try to collect as much information as possible about the visitor of the site where our script is installed. Basically this is technical information: whether the flash player is included in the browser, the size of the browser window, the screen size, we get the address of the page and the referrer to the page from which the user came and so on. Further information is collected in the JSON string, the request for advertising is encoded and sent. Now the back-end gets down to business.
')
Having received a request for advertising, we also receive from the database all the information we know about this visitor (history of his interests, advertising views and clicks). According to the user-agent, we get data about the browser used, its version and visitor's operating system, and by its ip-address we determine its current location. We send all the information collected about the user to our DMP for its further analysis.
Having collected data about the user, we proceed to the site. We receive information about the cost of advertising space, the CTR of the site, the categories of content, the presence of prohibited content, etc. On the basis of these data, the minimum threshold price of the auction will be formed. Having finished with the sites, we proceed to the longest SSP task, this is polling the connected DSPs. Based on the data obtained above in the manner described, a request is made to the DSP. Next, a parallel polling of the DSP takes place, and the response of each of them is set aside 120 ms, which includes the time for delivering the request and receiving the response (ping to the servers on which the DSP is located). So the location of the DSP also plays an important role. Having received the answer from all DSPs, it remains only to sort them by the size of the bet and take the answer from the largest one. The answer is in the form of JSON, to show advertising, we return to the front-end.
The frontend remains only a little - to display the code of the received ad. Since there is no way to keep track of the quality of the DSP code and its compatibility with sites, our code for displaying ads on the page creates a separate iframe, loads the code into it, and this is what we put on the page. This is where the ad request processing ends.
Let's now figure out what difficulties we had to face during the implementation of SSP, and how we dealt with them. One of the main problems during the implementation of SSP was the clogging of the IIS pool. The reason for this was the expectation of a response from the DSP for 100 milliseconds or more. While the SSP was waiting for a response from the DSP, the remaining requests were waiting for the SSP to process the current one. And the solution to this problem was the use of asynchronous controllers, and here async / await modifiers came in handy. Also, it would be superfluous to mention that the information about the site and the DSP is cached in the memory of the application and is updated every minute. The next main place to pay attention is the DSP survey. Initially, the question arose how to parallelize the DSP poll. During the development of SSP, several solutions to this task were tested, such as: Parallel.ForEach, Task.WaitAll and Task.WhenAll. No particular differences in performance were found; each method correctly performed its task, but due to the transition to the asynchronous model, the choice fell on the latter option. The request itself to the DSP is done via HttpWebRequest. The most important thing that should be done when working with HttpWebRequest is to enable KeepAlive support, because requests to the DSP are constantly following each other, and constant breaks and connection setup will noticeably slow down the receipt of the response. I will give an example of the resulting class for polling DSP:
The client part of our SSP. This is the personal account of the web publisher (site), which contains statistics on all its advertising spaces. VOX is a self-service platform that allows you to create both banner advertising spaces (the 5 most popular sizes in RTB) and Rich Media banners (fullscreen). At each advertising space, you can hang a stub (HTML or Javascript), as well as add a counter display third-party system.
Here, perhaps, we will finish the review of the work of our SSP.
The load here is not as big as, for example, in our DSP , and the main processing time of the request is spent waiting for responses from connected DSPs. It is written in the form of an asynchronous ASP.NET MVC application.
So, let's begin to figure out how the processing of requests.
Request for advertising begins to form on the client side. Here, using javascript, we try to collect as much information as possible about the visitor of the site where our script is installed. Basically this is technical information: whether the flash player is included in the browser, the size of the browser window, the screen size, we get the address of the page and the referrer to the page from which the user came and so on. Further information is collected in the JSON string, the request for advertising is encoded and sent. Now the back-end gets down to business.
')
Having received a request for advertising, we also receive from the database all the information we know about this visitor (history of his interests, advertising views and clicks). According to the user-agent, we get data about the browser used, its version and visitor's operating system, and by its ip-address we determine its current location. We send all the information collected about the user to our DMP for its further analysis.
Having collected data about the user, we proceed to the site. We receive information about the cost of advertising space, the CTR of the site, the categories of content, the presence of prohibited content, etc. On the basis of these data, the minimum threshold price of the auction will be formed. Having finished with the sites, we proceed to the longest SSP task, this is polling the connected DSPs. Based on the data obtained above in the manner described, a request is made to the DSP. Next, a parallel polling of the DSP takes place, and the response of each of them is set aside 120 ms, which includes the time for delivering the request and receiving the response (ping to the servers on which the DSP is located). So the location of the DSP also plays an important role. Having received the answer from all DSPs, it remains only to sort them by the size of the bet and take the answer from the largest one. The answer is in the form of JSON, to show advertising, we return to the front-end.
The frontend remains only a little - to display the code of the received ad. Since there is no way to keep track of the quality of the DSP code and its compatibility with sites, our code for displaying ads on the page creates a separate iframe, loads the code into it, and this is what we put on the page. This is where the ad request processing ends.
Let's now figure out what difficulties we had to face during the implementation of SSP, and how we dealt with them. One of the main problems during the implementation of SSP was the clogging of the IIS pool. The reason for this was the expectation of a response from the DSP for 100 milliseconds or more. While the SSP was waiting for a response from the DSP, the remaining requests were waiting for the SSP to process the current one. And the solution to this problem was the use of asynchronous controllers, and here async / await modifiers came in handy. Also, it would be superfluous to mention that the information about the site and the DSP is cached in the memory of the application and is updated every minute. The next main place to pay attention is the DSP survey. Initially, the question arose how to parallelize the DSP poll. During the development of SSP, several solutions to this task were tested, such as: Parallel.ForEach, Task.WaitAll and Task.WhenAll. No particular differences in performance were found; each method correctly performed its task, but due to the transition to the asynchronous model, the choice fell on the latter option. The request itself to the DSP is done via HttpWebRequest. The most important thing that should be done when working with HttpWebRequest is to enable KeepAlive support, because requests to the DSP are constantly following each other, and constant breaks and connection setup will noticeably slow down the receipt of the response. I will give an example of the resulting class for polling DSP:
Code
public class DspRequests { private const string _method = "POST"; private const string _contentType = "application/json"; private static List<string> _headers = new List<string> { "x-openrtb-version: 2.0" }; private List<string> _dspUrls { get; set; } public List<string> DspUrls { get { returns _dspUrls; } set { _dspUrls = value; } } public async Task<List<string>> PollDspAsync(string request) { List<string> responses = null; var tasks = _dspUrls.DspRecords.Select(el => PollAsync(el, request)); responses = (await Task.WhenAll(tasks)).Where(el => el != null).ToList(); return responses; } private async Task<string> PollAsync(string url, string bidRequest) { string response = null; HttpWebRequest httpWebRequest = (HttpWebRequest)WebRequest.Create(url); httpWebRequest.Method = _method; httpWebRequest.ContentType = _contentType; httpWebRequest.KeepAlive = true; foreach (string header in _headers) httpWebRequest.Headers.Add(header); httpWebRequest.Timeout = 120; using (StreamWriter streamWriter = new StreamWriter(await httpWebRequest.GetRequestStreamAsync())) { streamWriter.Write(bidRequest); } using (HttpWebResponse webResponse = (HttpWebResponse) await httpWebRequest.GetResponseAsync()) { if (webResponse.StatusCode == HttpStatusCode.OK) { using (StreamReader streamReader = new StreamReader(webResponse.GetResponseStream())) { response = streamReader.ReadToEnd(); } } } return response; } } The client part of our SSP. This is the personal account of the web publisher (site), which contains statistics on all its advertising spaces. VOX is a self-service platform that allows you to create both banner advertising spaces (the 5 most popular sizes in RTB) and Rich Media banners (fullscreen). At each advertising space, you can hang a stub (HTML or Javascript), as well as add a counter display third-party system.
Here, perhaps, we will finish the review of the work of our SSP.
Source: https://habr.com/ru/post/264015/
All Articles