Five ways to speed up Facebook API requests in practice
It's no secret to anyone that HTTP requests to external servers are most often the bottleneck of web applications. Thus, the load time of the API request data is much longer than the time required to execute most of the most complex web application scripts.
While working with the Facebook API, I have accumulated several recipes for query optimization: how to increase the speed of scripts, reduce their number and resource intensity.
')
The methods outlined in this article only work with the Facebook API. But I do not exclude that they may be applicable in other services that provide API.
You probably have an idea for a web application that works with user data and its social graph Facebook. And you have already opened the documentation more than once and even managed to write a couple of scripts. Let this be a “birthday present” application, and the first thing you should do is get a list of the user's friends and their birth dates.
It's simple, you think:
Since your script runs on the server side, you should get all the data before they are output to the user. And here you are faced with the first and most important problem: one request to the API takes 1-2 seconds. So, just to get a list of all friends you need 2-4 seconds (depending on the number of friends).
Let's try to calculate how long the script that displays the birthdays of 10 friends will run.
It took me 13 seconds for this! Will your user wait so long? I do not think. And if the number of friends is measured in hundreds, or even a couple of thousand ...
Let's start the optimization.
When you request user data (the User object in the Graph API), Facebook defaults to you all the fields to which you have access. You can get rid of redundant information by specifying which fields you need to return. To do this, the request uses the fields parameter, in which the required fields are listed separated by commas. For example, for our case it will be a query: {user_id}? Fields = id, name, birthday
Requests with the fields parameter are faster because they return only the necessary data. They are faster transmitted over the network, as they are smaller. They are also processed faster and use less memory.
Although this method of optimization in this program has almost no effect on the total execution time, but it is this that should never be forgotten. Read only the required fields for each request!
Using the ids parameter you can select several objects that you want to receive. At the same time, you can use the fields parameter to limit only the required fields of objects (objects must be of the same type). For example, the query ? Ids = 4.501012028 will allow two users to receive open data at once;)
The number of objects that you can specify, separated by commas, is limited only by the maximum long URL. Do not forget that the size of the response to the request also increases. Therefore, it is reasonable to limit the number of objects in a single request. For example, here is the answer, if you use the first and second methods.
To use this method in our example, you need to rewrite the script.
Now, instead of 10 requests, we use only one, respectively, the script has decreased to 4-5 seconds. But we still need to get the data of all friends, not just 10.
Pagination is just a splitting of information into “pages”. Using the limit parameter, we only received information about 10 friends. To get the same information about the others, there is an offset parameter. For example, the first ten friends: me / friends? Limit = 10 & offset = 0 , the second ten friends: me / friends? Limit = 10 & offset = 10 , etc.
You can optimize script performance as follows:
Now, to display information, you do not need to wait until all parts of requests are processed. In this case, for the first batch, you will already receive a sufficient amount of data in one request.
Let us turn to more serious ways, which allow to receive tens of times more data in one request (and maybe all at once).
FQL gives you the ability to use the SQL-style interface for querying data. With it, you can more easily implement some queries that are not available through the Graph API. The query form is as follows: SELECT [fields] FROM [table] WHERE [conditions] . But FQL has many limitations (if you compare it with SQL). For example, in FROM only one table can be used. But you can use subqueries. In FQL, you can use logical operators, ORDER BY and LIMIT constructs, and some other operators.
To get the name and date of birth of the user, you need to make an FQL query to the user table. For example, in the Graph API Explorer this will be the query fql? Q = SELECT uid, name, birthday_date FROM user WHERE uid = me () . And in order to get a list of id friends, use the query to the friend friend table? Q = SELECT uid2 FROM friend WHERE uid1 = me () .
Let's try to merge two queries into one using an embedded query. It's simple: fql? Q = SELECT uid, name, birthday_date FROM user WHERE uid IN (SELECT uid2 FROM friend WHERE uid1 = me ()) . Instead of dozens of requests, we received all the information we need in just one!
There are many ways to optimize FQL queries. Examine all the tables in the documentation. But FQL is not the top of query optimization.
Imagine that you need to get a list of the user's latest posts, his news feed, all unread notifications and new messages. And all this, and more, needs to be done quickly. You can, of course, parallelize queries by making them asynchronous. But this is not our way.
Using the Graph API, you can process the data of a single query, even if it is an FQL query. Batch Request allows you to send to the server in one request - a pack of several different requests. The only limitation is up to 20 requests in one batch call. Requests can be GET, POST and DELETE.
For example, here is such a “package” of requests:
Batch requests must be sent using the POST method, so cURL is involved in the code above. It is also easier to work with possible errors.
All uses of Batch Requests are not described in this article. You can learn more about them in the documentation .
While working with the Facebook API, I have accumulated several recipes for query optimization: how to increase the speed of scripts, reduce their number and resource intensity.
')
The methods outlined in this article only work with the Facebook API. But I do not exclude that they may be applicable in other services that provide API.
Introduction
You probably have an idea for a web application that works with user data and its social graph Facebook. And you have already opened the documentation more than once and even managed to write a couple of scripts. Let this be a “birthday present” application, and the first thing you should do is get a list of the user's friends and their birth dates.
It's simple, you think:
- I authorize the user (I get the user id and access_token , which I use in further requests);
- I get a list of his friends (on exit I have an array with id and name of friends);
- for each id, I make a request for user data (in order to get birthday data, during authorization you need to request friends_birthday permission).
Since your script runs on the server side, you should get all the data before they are output to the user. And here you are faced with the first and most important problem: one request to the API takes 1-2 seconds. So, just to get a list of all friends you need 2-4 seconds (depending on the number of friends).
Let's try to calculate how long the script that displays the birthdays of 10 friends will run.
<?php $start = microtime(true); $app_id = "YOUR_APP_ID"; $app_secret = "YOUR_APP_SECRET"; $my_url = "YOUR_URL"; session_start(); $code = $_REQUEST["code"]; if(empty($code)) { $_SESSION['state'] = md5(uniqid(rand(), TRUE)); //CSRF protection $dialog_url = 'https://www.facebook.com/dialog/oauth?client_id=' . $app_id . '&redirect_uri=' . urlencode($my_url) . '&scope=user_birthday,friends_birthday&state=' . $_SESSION['state']; echo("<script> top.location.href='" . $dialog_url . "'</script>"); } if($_REQUEST['state'] == $_SESSION['state']) { $token_url = 'https://graph.facebook.com/oauth/access_token?' . 'client_id=' . $app_id . '&redirect_uri=' . urlencode($my_url) . '&client_secret=' . $app_secret . '&code=' . $code; $response = file_get_contents($token_url); $params = null; parse_str($response, $params); $graph_url = "https://graph.facebook.com/me?access_token=" . $params['access_token']; $user = json_decode(file_get_contents($graph_url)); $uid = $user->id; // $graph_url = 'https://graph.facebook.com/'.$uid.'/friends?limit=10&access_token=' . $params['access_token']; $friends = json_decode(file_get_contents($graph_url)); $time = microtime(true) - $start; printf(' %.4F .<br/>', $time); $n = sizeof($friends->data); for ($i = 0; $i < $n; $i++) { $graph_url = 'https://graph.facebook.com/' . $friends->data[$i]->id . '?access_token=' . $params['access_token']; $friend_data = file_get_contents($graph_url); // json //$friend = json_decode($friend_data); //echo($i.' '.$friend->name.' - '.$friend->birthday); } } else { echo("The state does not match. You may be a victim of CSRF."); } $time = microtime(true) - $start; printf(' %.4F .', $time); It took me 13 seconds for this! Will your user wait so long? I do not think. And if the number of friends is measured in hundreds, or even a couple of thousand ...
Let's start the optimization.
Method 1: We read only the required fields
When you request user data (the User object in the Graph API), Facebook defaults to you all the fields to which you have access. You can get rid of redundant information by specifying which fields you need to return. To do this, the request uses the fields parameter, in which the required fields are listed separated by commas. For example, for our case it will be a query: {user_id}? Fields = id, name, birthday
- Enable permissions in the Graph API Explorer by clicking Get Access Token and checking all the check boxes in User Data Permissions and Friends Data Permissions .
- Type me in the field after https: graph.facebook.com/
- Compare previous output with me? Fields = id, name, birthday
Requests with the fields parameter are faster because they return only the necessary data. They are faster transmitted over the network, as they are smaller. They are also processed faster and use less memory.
Although this method of optimization in this program has almost no effect on the total execution time, but it is this that should never be forgotten. Read only the required fields for each request!
Method 2: We request the data of several objects in one request
Using the ids parameter you can select several objects that you want to receive. At the same time, you can use the fields parameter to limit only the required fields of objects (objects must be of the same type). For example, the query ? Ids = 4.501012028 will allow two users to receive open data at once;)
The number of objects that you can specify, separated by commas, is limited only by the maximum long URL. Do not forget that the size of the response to the request also increases. Therefore, it is reasonable to limit the number of objects in a single request. For example, here is the answer, if you use the first and second methods.
To use this method in our example, you need to rewrite the script.
$ n = sizeof ($ friends-> data);
$ graph_url = 'https://graph.facebook.com/?ids=';
for ($ i = 0; $ i <$ n; $ i ++) {
$ graph_url. = $ friends-> data [$ i] -> id. ',';
}
$ graph_url = substr ($ graph_url, 0, -1);
$ graph_url. = '& access_token ='. $ params ['access_token'];
$ friend_data = file_get_contents ($ graph_url);
// decode json response and output data Now, instead of 10 requests, we use only one, respectively, the script has decreased to 4-5 seconds. But we still need to get the data of all friends, not just 10.
Method 3: Use filtering and pagination
Pagination is just a splitting of information into “pages”. Using the limit parameter, we only received information about 10 friends. To get the same information about the others, there is an offset parameter. For example, the first ten friends: me / friends? Limit = 10 & offset = 0 , the second ten friends: me / friends? Limit = 10 & offset = 10 , etc.
You can optimize script performance as follows:
- Write the client part with Ajax requests to the server.
- Get the data of the first 10 friends and display them.
- While the data is displayed on the screen, load the next “portion” of data.
Now, to display information, you do not need to wait until all parts of requests are processed. In this case, for the first batch, you will already receive a sufficient amount of data in one request.
Let us turn to more serious ways, which allow to receive tens of times more data in one request (and maybe all at once).
Method 4: Build FQL Queries (Facebook Query Language)
FQL gives you the ability to use the SQL-style interface for querying data. With it, you can more easily implement some queries that are not available through the Graph API. The query form is as follows: SELECT [fields] FROM [table] WHERE [conditions] . But FQL has many limitations (if you compare it with SQL). For example, in FROM only one table can be used. But you can use subqueries. In FQL, you can use logical operators, ORDER BY and LIMIT constructs, and some other operators.
To get the name and date of birth of the user, you need to make an FQL query to the user table. For example, in the Graph API Explorer this will be the query fql? Q = SELECT uid, name, birthday_date FROM user WHERE uid = me () . And in order to get a list of id friends, use the query to the friend friend table? Q = SELECT uid2 FROM friend WHERE uid1 = me () .
Let's try to merge two queries into one using an embedded query. It's simple: fql? Q = SELECT uid, name, birthday_date FROM user WHERE uid IN (SELECT uid2 FROM friend WHERE uid1 = me ()) . Instead of dozens of requests, we received all the information we need in just one!
// get a list of the user's friends and their birthdays $ graph_url = 'https://graph.facebook.com/fql?q=SELECT uid, name, birthday_date FROM user WHERE uid IN (SELECT uid2 FROM friend WHERE uid1 ='. $ uid. ') & access_token ='. $ params ['access_token']; $ frnds = file_get_contents ($ graph_url); // decode json response and output data
There are many ways to optimize FQL queries. Examine all the tables in the documentation. But FQL is not the top of query optimization.
Imagine that you need to get a list of the user's latest posts, his news feed, all unread notifications and new messages. And all this, and more, needs to be done quickly. You can, of course, parallelize queries by making them asynchronous. But this is not our way.
Method 5: Batch Requests
Using the Graph API, you can process the data of a single query, even if it is an FQL query. Batch Request allows you to send to the server in one request - a pack of several different requests. The only limitation is up to 20 requests in one batch call. Requests can be GET, POST and DELETE.
For example, here is such a “package” of requests:
- Send new status with link.
- Get information about the latest status (new is not considered).
- Like the last status and write a comment.
$ batched_request = '[{"method": "POST", "relative_url": "me / feed", "body": "message = Soon new post & link = http: //habrahabr.ru/"}}';
$ batched_request. = ', {"method": "GET", "name": "get-post", "relative_url": "me / feed? limit = 1"}';
$ batched_request. = ', {"method": "POST", "relative_url": "{result = get-post: $. data.0.id} / likes"}';
$ batched_request. = ', {"method": "POST", "relative_url": "{result = get-post: $. data.0.id} / comments", "body": "message = New automatic comment" } ';
$ batched_request. = ']';
$ url = 'https://graph.facebook.com/';
$ param = array ();
$ param ['access_token'] = $ params ['access_token'];
$ param ['batch'] = $ batched_request;
$ ch = curl_init ();
curl_setopt ($ ch, CURLOPT_URL, $ url);
curl_setopt ($ ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt ($ ch, CURLOPT_POSTFIELDS, $ param);
$ ret = curl_exec ($ ch);
// response processing
if ($ ret == 'false') echo '= false =';
curl_close ($ ch);
Batch requests must be sent using the POST method, so cURL is involved in the code above. It is also easier to work with possible errors.
All uses of Batch Requests are not described in this article. You can learn more about them in the documentation .
Source: https://habr.com/ru/post/132794/
All Articles