Optimizing Facebook Graph API with Real-Time Updates
Applications for Facebook can have various functions: for example, often the application will have enough information obtained through the API while the user is working with the application. But what to do if your application should work with the most “fresh” user data, even if they have not opened it for more than a month?
There are two ways to receive data not only while the user is working with the application:
Under the cat, you will learn a virtual example of how the use of real-time updates helps to reduce the number of requests to the API by more than 100 times per day in some situations. We will write a subscription script for updates and check its work, having received data on changes in objects from Facebook itself.
The beginning of the article contains a lot of theory, which is similar to the Russian translation of the Facebook page. If it is easier for you to read in English - you can skip everything in the second method to the section Example of Real-Time Updates . But for some reason, I failed to set up this functionality after the first reading. I hope this article will help you quickly understand this.
The functionality of some applications may be associated with the need to obtain data on a schedule. For example, you collect statistics of the number of likes of pages from a particular directory. To do this, you receive data at intervals of once per hour. This is the correct example of use, otherwise you simply will not get this data. But often this method is also used for objects, according to which updates can be received not “according to the schedule” but real-time from Facebook itself.
')
Of the serious shortcomings of the first method are the following:
Imagine that you made an application whose functionality is tied to the user's friends, and it is very important for you to have the latest information. Let's say your application has been installed by 1000 users. And let an acceptable time of caching data about friends is 1 hour. As a result, every day you will send 24,000 requests to ensure that your database of user’s friends is up to date! And you have only 1000 users, and we can assume that there will be no more than a couple of hundred changes in friends for everyone per day (well, this is a guess based on personal experience).
To solve the shortcomings of the first method, Facebook made Real-Time Updates for its Graph API.
Real-time update support allows your app to subscribe to changes in Facebook data. Your application receives updates from Facebook and saves the data, instead of polling Facebook servers “on schedule”. Caching data and using this API can improve the reliability of the application and reduce its load time. Whenever a change happens to which you are subscribed, Facebook makes an HTTP POST request to the update processing script with a list of changes. The server will send you a notification of the change within a couple of minutes after its appearance.
At this point, you can subscribe to the following types of objects:
* Not all properties and associations of the User object you can subscribe. For example, you can access links: feed, friends, activities, interests, music, books, movies, television, likes, checkins. But you still cannot subscribe to changes in these links: home, tagged, posts, photos, albums, videos, groups, notes, events, inbox, outbox, updates, accounts
In order to set up a subscription for updates, you need:
By sending a request to
The request must include the access_token of the application, which can be obtained using your application ID (app-id) and application secret key (app-secret). Just send an HTTP GET
An application can have only one subscription for each of the possible types of objects. If you add an object to which you already have a subscription, then the existing subscription will be replaced with a new one. To add or change a subscription, you must send a POST request containing the fields:
Getting a list of all the objects as well as their fields to which you subscribe
Making a GET request to subscribe URL
To delete the entire subscription to all objects, send a DELETE request. If you want to remove a subscription to a specific object, specify the object parameter (for example, object = user).
Facebook servers make an HTTP GET request to your callback_url when you are trying to add or modify a subscription. After a successful subscription, Facebook servers will send change notifications using HTTP POST to the same URL.
Before the subscription is finally added or modified, Facebook servers will make an HTTP GET to your callback_url with the following parameters:
The update processing script should first check the value of hub.verify_token - the “secret code” that you sent to Facebook, and then return the hub.challenge parameter back. This was done specifically to protect the script as a means of carrying out DDoS attacks on the server.
Note to developers: In PHP, the dot is automatically converted to underscore in the parameters. Therefore, you should refer to
When the data changes and there is a valid subscription, Facebook servers make an HTTP POST request for the callback_url that you asked. The content type of the request will be application / json; the body is a JSON-encoded string containing information about one or more changes. An example can be seen below in the application code.
It should be noted that the line will not contain data on the current value. Therefore, you will need to make a simple API request to get them (due to security policy). You can download the necessary data immediately so that the user, having returned to the application, does not experience complications with an increase in download time.
Facebook aggregates the changes and sends them in batches every 5 seconds, or if the number of changes that have occurred has exceeded 1000. In the event that the change notification was not delivered to you, Facebook will try again, and then a little later, reducing the frequency of requests in the next 24 hours.
Let's apply the theory in practice. To do this, you will need a hosting, PHP script, which will be located on your domain at the address specified by you callback_url. You also need an application in Facebook. You can use the existing one, or create a new one on the developer’s application page .
Save the script for callback_url
After that we will use the new tool for developers of Facebook - Graph API Explorer .
Select a POST request to add a subscription and fill in all the fields. To get access_token (applications), open the link in the browser by filling in the app-id and app-secret data of your application.
The path to the script can be facebook / callback.php for example. It is important that you take the app-secret from the application settings, and you invent the secret code in verify_token yourself!
So, if after clicking Submit in the gray box below, nothing happened, then you still added a subscription to the friends of the user object's friends, and now as soon as any user of your application adds or deletes one of the friends, Facebook will send you a callback_url . In my example, I added a subscription to the permissions offline_access object.
If your application has been installed, do not worry. If not, too. The sequence of actions is the same.
First you need to add the offline_access permission. If the application has not been installed, it will happen automatically. My application is not installed. I will try to call the authorization window through the browser line. Open the link by replacing the app-id with the id of your application, and replacing the site-url with http: // <your domain>:
After successful authorization Facebook redirects you to http: // <your domain> /? Code = ...
According to the logic that is contained in the script — when processing an incoming notification, we simply add the transferred JSON-encoded string to the text data.txt file, which will be in the same directory as the script. Check if this file has been created and what information it contains.
My added information looks like this:
As you can see, I installed the application and allowed offline_access permission. By uid, you can define a user whose data has changed, and by time, the time of addition.
You can go to the application settings on Facebook, open your application and remove the offline_access resolution. In the data.txt file you will see that you have again been notified. You can also add or remove someone from your friends - you will see that these notifications you subscribe to will be sent to callback_url and processed by your script.
Now you know about the great Facebook feature Real-Time Updates, which can send you information about changes in user data and permissions. Unfortunately, for some reason, when the user deletes the application, the data has not changed. Therefore, as if the new method did not save time and resources - after all, when implementing certain tasks (for example, sending a mailing list to all users of the application), it is worthwhile to bypass the usual API of all users in your database, checking whether your application is installed .
There are two ways to receive data not only while the user is working with the application:
- Get permission from the user offline_access (save the “eternal” user access_token) and get the necessary data “according to the schedule” (pull the cron script).
- Write a script that will receive all data changes from Facebook, set up and subscribe to updates via Real-Time Updates.
Under the cat, you will learn a virtual example of how the use of real-time updates helps to reduce the number of requests to the API by more than 100 times per day in some situations. We will write a subscription script for updates and check its work, having received data on changes in objects from Facebook itself.
The beginning of the article contains a lot of theory, which is similar to the Russian translation of the Facebook page. If it is easier for you to read in English - you can skip everything in the second method to the section Example of Real-Time Updates . But for some reason, I failed to set up this functionality after the first reading. I hope this article will help you quickly understand this.
First way
The functionality of some applications may be associated with the need to obtain data on a schedule. For example, you collect statistics of the number of likes of pages from a particular directory. To do this, you receive data at intervals of once per hour. This is the correct example of use, otherwise you simply will not get this data. But often this method is also used for objects, according to which updates can be received not “according to the schedule” but real-time from Facebook itself.
')
Of the serious shortcomings of the first method are the following:
- The less frequently the data is checked, the less reliable it is.
- High resource costs with frequent updates.
- Users are reluctant to allow offline_access - access to their data at any time.
Imagine that you made an application whose functionality is tied to the user's friends, and it is very important for you to have the latest information. Let's say your application has been installed by 1000 users. And let an acceptable time of caching data about friends is 1 hour. As a result, every day you will send 24,000 requests to ensure that your database of user’s friends is up to date! And you have only 1000 users, and we can assume that there will be no more than a couple of hundred changes in friends for everyone per day (well, this is a guess based on personal experience).
To solve the shortcomings of the first method, Facebook made Real-Time Updates for its Graph API.
Second way
Real-time update support allows your app to subscribe to changes in Facebook data. Your application receives updates from Facebook and saves the data, instead of polling Facebook servers “on schedule”. Caching data and using this API can improve the reliability of the application and reduce its load time. Whenever a change happens to which you are subscribed, Facebook makes an HTTP POST request to the update processing script with a list of changes. The server will send you a notification of the change within a couple of minutes after its appearance.
Objects
At this point, you can subscribe to the following types of objects:
- user (user) - receive notifications about specific fields and user connections with the corresponding Graph API nodes. *
- permissions — Receive notifications when a user changes permissions for your application.
- page (page) - receive notifications when the pages that have added (installed) your application change their general properties.
* Not all properties and associations of the User object you can subscribe. For example, you can access links: feed, friends, activities, interests, music, books, movies, television, likes, checkins. But you still cannot subscribe to changes in these links: home, tagged, posts, photos, albums, videos, groups, notes, events, inbox, outbox, updates, accounts
Subscription
In order to set up a subscription for updates, you need:
- Set up a script (URL) that will receive both HTTP GET (for verification of the subscription) and POST (for changing data) requests from Facebook.
- Make a POST request to the Graph API
https://graph.facebook.com/<app-id>/subscriptions
to subscribe, and be ready to process a verification request.
By sending a request to
https://graph.facebook.com/ <app-id> / subscriptions? access_token = ...You can do three things, depending on which HTTP POST, GET, or DELETE request you send:
- Add and edit subscriptions (POST).
- Get a list of all the objects as well as their subscribed fields (GET)
- Delete Subscription (DELETE)
The request must include the access_token of the application, which can be obtained using your application ID (app-id) and application secret key (app-secret). Just send an HTTP GET
https://graph.facebook.com/oauth/access_token?client_id=<app-id>&client_secret=<app-secret>&grant_type=client_credentialsand the answer will come: access_token = ...
Add and edit subscriptions
So, you have read above where you need to send a request to add a subscription. Do not rush to immediately fulfill these requests - the structure of the presentation of the material is not the same as the sequence of necessary steps. This was done specifically to study the material consistently. But in the example, you will be the first to perform queries that are described in the last part before the example.
An application can have only one subscription for each of the possible types of objects. If you add an object to which you already have a subscription, then the existing subscription will be replaced with a new one. To add or change a subscription, you must send a POST request containing the fields:
- object - the type of the object: user, permissions or page. You will monitor all objects of this type, for example, all users of your application.
- fields - A list (items separated by commas) of the properties and links of the selected object. For example, to follow changes in the name, picture, friends of the user, as well as News Feed links, you must specify the value “name, picture, friends, feed”
- callback_url - the URL to which Facebook will send the changes to which the application is subscribed.
- verify_token is the “secret code” that you specify in order to verify the authenticity of the request. It will be passed to the script that handles the subscription.
Getting a list of all the objects as well as their fields to which you subscribe
Making a GET request to subscribe URL
https://graph.facebook.com/ <app-id> / subscriptions? access_token = ...You will receive a response with JSON-encoded content that lists your subscriptions. I subscribed to the friends connection of the user object, and also to offline_access in permissions. Here is the answer:
{
"data": [
{
"object": "user",
"callback_url": "http://millione.tv/< Path to the script>",
"fields": [
"friends"
],
"active": true
},
{
"object": "permissions",
"callback_url": "http://millione.tv/< Path to the script>",
"fields": [
"offline_access"
],
"active": true
}
]
}
Deleting a subscription
To delete the entire subscription to all objects, send a DELETE request. If you want to remove a subscription to a specific object, specify the object parameter (for example, object = user).
Your Callback Server
Facebook servers make an HTTP GET request to your callback_url when you are trying to add or modify a subscription. After a successful subscription, Facebook servers will send change notifications using HTTP POST to the same URL.
Subscription Verification
Before the subscription is finally added or modified, Facebook servers will make an HTTP GET to your callback_url with the following parameters:
- hub.mode - The string 'subscribe' is passed in this parameter.
- hub.challenge - Random string.
- hub.verify_token is the “secret code” that you set in order to verify the authenticity of the request that you sent to Facebook.
The update processing script should first check the value of hub.verify_token - the “secret code” that you sent to Facebook, and then return the hub.challenge parameter back. This was done specifically to protect the script as a means of carrying out DDoS attacks on the server.
Note to developers: In PHP, the dot is automatically converted to underscore in the parameters. Therefore, you should refer to
$_GET["hub_mode"] , $_GET["hub_challenge"] and $_GET["hub_verify_token"] .Change Notifications
When the data changes and there is a valid subscription, Facebook servers make an HTTP POST request for the callback_url that you asked. The content type of the request will be application / json; the body is a JSON-encoded string containing information about one or more changes. An example can be seen below in the application code.
It should be noted that the line will not contain data on the current value. Therefore, you will need to make a simple API request to get them (due to security policy). You can download the necessary data immediately so that the user, having returned to the application, does not experience complications with an increase in download time.
Facebook aggregates the changes and sends them in batches every 5 seconds, or if the number of changes that have occurred has exceeded 1000. In the event that the change notification was not delivered to you, Facebook will try again, and then a little later, reducing the frequency of requests in the next 24 hours.
Example of Real-Time Updates
Let's apply the theory in practice. To do this, you will need a hosting, PHP script, which will be located on your domain at the address specified by you callback_url. You also need an application in Facebook. You can use the existing one, or create a new one on the developer’s application page .
1. Appendix
2. The update processing script
Save the script for callback_url
<? php
/ * Note the request must complete within 15 seconds.
Otherwise Facebook server will consider it a timeout and
resend the push notification again. * /
define ( 'VERIFY_TOKEN' , '<secret_code>' ) ;
$ method = $ _SERVER [ 'REQUEST_METHOD' ] ;
if ( $ method == 'GET' && $ _GET [ 'hub_mode' ] == 'subscribe'
&& $ _GET [ 'hub_verify_token' ] == VERIFY_TOKEN ) {
echo $ _GET [ 'hub_challenge' ] ;
} else
if ( $ method == 'POST' ) {
$ data = file_get_contents ( "php: // input" ) ;
$ fh = fopen ( 'data.txt' , 'a' ) or die ( 'open file' ) ;
if ( - 1 == fwrite ( $ fh , $ data ) ) { die ( 'write data' ) ; }
fclose ( $ fh ) or die ( 'close file' ) ;
}
?>
3. Add a subscription
After that we will use the new tool for developers of Facebook - Graph API Explorer .
Select a POST request to add a subscription and fill in all the fields. To get access_token (applications), open the link in the browser by filling in the app-id and app-secret data of your application.
https://graph.facebook.com/oauth/access_token?client_id=<app-id>&client_secret=<app-secret>&grant_type=client_credentials
The path to the script can be facebook / callback.php for example. It is important that you take the app-secret from the application settings, and you invent the secret code in verify_token yourself!
So, if after clicking Submit in the gray box below, nothing happened, then you still added a subscription to the friends of the user object's friends, and now as soon as any user of your application adds or deletes one of the friends, Facebook will send you a callback_url . In my example, I added a subscription to the permissions offline_access object.
4. Create user activity
If your application has been installed, do not worry. If not, too. The sequence of actions is the same.
First you need to add the offline_access permission. If the application has not been installed, it will happen automatically. My application is not installed. I will try to call the authorization window through the browser line. Open the link by replacing the app-id with the id of your application, and replacing the site-url with http: // <your domain>:
https://www.facebook.com/dialog/oauth?client_id=<app-id>&redirect_uri=<site-url>&scope=offline_access
After successful authorization Facebook redirects you to http: // <your domain> /? Code = ...
According to the logic that is contained in the script — when processing an incoming notification, we simply add the transferred JSON-encoded string to the text data.txt file, which will be in the same directory as the script. Check if this file has been created and what information it contains.
My added information looks like this:
{"object":"permissions","entry":[{"uid":"100001828786121","id":"100001828786121","time":1310669052,"changed_fields":["connected"]},{"uid":"100001828786121","id":"100001828786121","time":1310669052,"changed_fields":["offline_access"]}]}As you can see, I installed the application and allowed offline_access permission. By uid, you can define a user whose data has changed, and by time, the time of addition.
5. Actions on deletion
You can go to the application settings on Facebook, open your application and remove the offline_access resolution. In the data.txt file you will see that you have again been notified. You can also add or remove someone from your friends - you will see that these notifications you subscribe to will be sent to callback_url and processed by your script.
findings
Now you know about the great Facebook feature Real-Time Updates, which can send you information about changes in user data and permissions. Unfortunately, for some reason, when the user deletes the application, the data has not changed. Therefore, as if the new method did not save time and resources - after all, when implementing certain tasks (for example, sending a mailing list to all users of the application), it is worthwhile to bypass the usual API of all users in your database, checking whether your application is installed .
Source: https://habr.com/ru/post/124195/
All Articles