Windows Azure Mobile Services custom API on the example of a whiteboard
In this article, I want to share the experience of creating a simple bike skramboard with a backend in the cloud based on Windows Azure Mobile Services and Azure Cloud Storage.
It is necessary to immediately warn picky readers that the main priority was simplicity and time minimization. In the end, this important artifact of agile development is just a white board with pieces of paper, therefore, in the virtual analog there will be only basic functionality.
Windows Azure platform was not chosen by chance. Despite the fact that Azure, like any other cloud platform, is not cheap, in our scenario, when a small agile team will use the board, this decision will cost almost nothing.
The article is more about what you can get from Windows Azure, especially if you already have a subscription.
All source code is available in the open source project at https://bitbucket.org/savamura/scrumboard .
Look at the UI without backend here .
')
The main part of the application is the client code. It was his writing that took the most time (in fact, it took the most time to write an article). Despite this, I will not describe the implementation in too much detail, relying on clear code. The client will need HTML5 Boilerplate , jQuery , jQuery UI , knockout.js , underscore.js , a little Backbone.js . Mix all together in MVVM salad, add salt to taste.
View (View) is one html page with two buttons (new sticker, save board), a dialog with the properties of the sticker and, in fact, the field where these stickers are “pasted”. Above is a strip with phases under which you can place a sticker.
The data model is extremely simple - it is the coordinates of the sticker, the inscription, additional information, as well as a link to the task in your favorite task management system. The coordinates of the sticker are changed by dragging it, and the associated data via the dialog box. Drag and drop stickers and dialog boxes are effortlessly obtained using jQuery.UI. Using knockout.js allows you to separate the presentation logic from the elements of this representation. In the entire application, there are three ViewModel: BoardViewModel, StickerViewModel and StickerPropertiesDialogViewModel.
We will return to the client a little later. First, create a backend mobile service.
Windows Azure Mobile Services is a specialized cloud hosting for node.js applications. In addition to the REST API, Push notifications to the main mobile platforms are available, authentication using identity providers (Google, Facebook, Twitter, Microsoft), launching scheduled tasks. You can create your application directly in the web console, and you can configure the synchronization of the application with the git repository. Let's try both.
First, create a new Mobile Service in the Azure web console. You must select a unique name, region, as well as the option “Create a free 20 MB SQL database”. You cannot immediately abandon the database, but you can delete it after it is created, or not use it. The fact is that out of the box there are two types of REST API available: the base CRUD API for managing data in SQL tables and the Custom API in which you can do anything. It is a little strange that the SQL database is so annoyingly proposed for use in the cloud service (although if you compare its cost with the cost of no-SQL storage, then everything becomes clear). We could, of course, store data in the SQL database, and, most likely, 20 megabytes would be enough for our board, but it seems to me that in the cloud service it is better to use cloud storage, besides, it is perfect for our task. That is why we will write a Custom API in which we will save and load data into Azure Table Storage.
So, go to the API tab and create a new one. Set the name “board” and leave the default access settings for HTTP requests: “Anybody with the Application Key”. Later these values ​​can be changed. Clicking on the newly created API will open the online code editor with the Hello World application. On this, we will probably finish our acquaintance with the online editor.
To be able to access the API from the local machine during debugging, you need to remember to write the cross-origin resource sharing (CORS) rule in the “configure” tab. You can specify localhost or simply *, thereby allowing access to the API from anywhere.
Azure Mobile Services allows you to deploy a web application using a private git repository. Let's try to take advantage of this cool feature, besides, it will give us an easy way to edit the code in your favorite IDE.
Go to the “Dashboard” tab and in the “Quick glance” column click on “Setup source control”. At the time of this writing, this option was in a preview state.
You will be asked to come up with a username and password for the git repository, after which Azure thinks a bit and in the “Configure” tab you will see the path to the private repository and the secret URL, the synchronization code of the repository with the running application, but I synchronized automatically. Now you can clone the created repository locally using your favorite git client, for example, SourceTree. Also clone bitbucket.org/savamura/scrumboard.git . Inside you will find the service folder, which needs to be synchronized with the one that was obtained as a result of the cloning of the Azure repository. Before you change the changes back to the cloud, let's deal with the server code.
There are four subfolders in the service folder: api with a custom API code, a scheduler with a code of tasks running according to a schedule, shared may contain common code, npm modules. The table folder is intended for scripts that modify data in the corresponding SQL table.
Each js script inside the api folder should have a paired json file with the same name. It contains access settings for our API.
Board.js is a node.js module that is used by the express web engine inside Azure Mobile Services.
Our application will require express, azure, and underscore modules:
Since we will store the data in Table Storage, we need to prepare it. Return to the Azure web-based management console, create a separate Cloud Storage and copy the Access Key. To access the repository, we use the TableService client written by Microsoft.
The table in which the stickers will be stored can be created both in the web console and from the code. Our application is not too heavy, and for illustration I have included the code that checks the existence of the table, but it is easier to prepare it in advance so as not to complicate the processing of requests.
Our API will handle only two types of requests: get (will return the entire board) and post (save it).
Inside the methods, you can use console.log tracing. Logs are available in the web console. Not very convenient, but at least you can debug it.
Since the application runs in node.js, all code is asynchronous. You need to be careful and do not forget to complete the request (request.respond or response.send). Full documentation is available on MSDN . First we check the existence of the table and create it using the createTableIfNotExists method. The method passes the name of the table and the callback function.
If the table exists, save the data:
Please note that a batch is used to save these stickers. It must be remembered that the batches have some limitations, for example, the PartitionKey should be the same for all entries, and the number of entries should not exceed 100. In this example, there is no splitting of all entries into batchies of 100 pieces.
In order to save a record to a table, the object hierarchy must be linearized. In other words, unlike, for example, from MongoDB, the tree object structure cannot be saved. Linearization occurs on the client. As the PartitionKey, the board identifier (guid) is used, which is now reserved in the client, but in the general case can be anything. The RowKey uses stickerId (guid), which is also created on the client.
In order to extract data from a table, it is enough to correctly form a query:
Here, in fact, the entire base server code. Of course, there must also be validation of the input data, error handling and access checking. Also, it would not hurt to implement the removal of stickers and protection against xss-attacks. All these problems we address in the next versions. The main thing is to provide basic functionality.
We only need to bind the client and server code.
The code that makes server-side calls is encapsulated in the StorageClient hotel wrapper class and, of course, can be replaced. There are only two methods: load the board and save the board. Information about all the stickers is immediately linearized, serialized and transmitted to the server. Of course, it would be possible to transfer only the changed data, but this will complicate the code somewhat.
To make custom API remote calls, you can use the WindowsAzure.MobileServicesClient client provided by Microsoft. It will be available at your_custom_api.azure-mobile.net/client/MobileServices.Web-1.0.3.min.js (version may vary). The link to it can also be copied from the Azure web console, if you click on the cloud icon with a lightning bolt next to “Dashboard”, then select the HTML / JavaScript platform and Connect an existing HTML app. There you can also get an API access key (below the manage keys button is the application key). The key can be re-created, while the old one will stop working.
StorageClient Constructor:
Our custom API is called board, so to save the board data, you need to do the following:
azureClient.invokeApi returns a promise, to complete which, you can subscribe.
The method of loading data looks the same, however, we additionally pass in it methods that will need to be called upon completion of the download.
We assume that we have debugged our application locally. It's time to embark him. As you may have guessed, hosting static content is best in Azure Blob Storage. To do this, you need a container that you can create, for example, via the Azure web console. During creation, specify the level of access to Public Blob, which will mean that access to all blobs in the container will be public, but the metadata of the container itself will remain private.
Now you need to upload files with client code to the cloud. The console utility AzCopy, which can be downloaded from the Microsoft Download Center, is perfect for this task. Click Download and select WindowsAzureStorageTools.msi. After installation, copy all the files you need to upload to the cloud in a separate folder.
Execute
AzCopy <project folder> <container path> / destKey: <access key> / S
If immediately after this you try to open your application in the browser (index.html), you will find that the browser opens a dialog box for selecting the path to save the file. This is due to the blob's content type, which by default is set to application / octet-stream.
This needs to be fixed by writing a small console application in C #. In it, we will go over all the blobs in the container and set them to the standard Content Type depending on the extension. Don't forget to plug in the nuget Windows Azure Storage package.
Everything, after executing this utility with the correct parameters, you can open the application through a browser.
On this, perhaps, it was possible to finish, but we missed one important detail: access control. Now anyone with a link to the application can see the tasks you need to complete in order to take over the world. In truth, your plans are most likely not interesting to anyone, and if you choose guid as the container name, then you need not worry. However, anything can happen, and if you want to control the situation, you will have to make a little more effort.
Open the Azure web console, find your mobile app and go to the “Identity” tab. Let's say that everyone in the team has a twitter account, so we will register our board as a web application there: https://dev.twitter.com/apps/new . After filling in all the fields, you will receive the coveted Consumer Key and Consumer Secret, which must be copied to the appropriate fields in the “Identity” tab. In addition, it is very important to specify the main URL of your mobile service as the Callback URL in the twitter settings of the application, as well as put a check mark in front of “Allow this application to be used.” I did not understand this immediately, so I had to suffer.
Now it is enough, in the http application, after creating the MobileServiceClient client and before sending the request for receiving the board data, call
The process is described in detail in this lesson .
In the web application code, at the beginning of each of the custom API methods, you need to check the userId.
So we got to the end of this article, but not to the end of the topic. Last week, a new batch of updates to the Azure cloud service came out, including the emergence of support for CORS in Cloud Storage. This means that by rewriting the storageClient client code and setting the CORS rules as necessary, you can save the data in Table Storage directly from the browser. In this case, you do not even need to write the server part in the form of the Azure Mobile Service, since its main function is to save the data to a table. Of course, we will lose the ability to authenticate users, but we will get a working application consisting only of the browser client code. Habrastia has already appeared on this topic.
It is necessary to immediately warn picky readers that the main priority was simplicity and time minimization. In the end, this important artifact of agile development is just a white board with pieces of paper, therefore, in the virtual analog there will be only basic functionality.
Windows Azure platform was not chosen by chance. Despite the fact that Azure, like any other cloud platform, is not cheap, in our scenario, when a small agile team will use the board, this decision will cost almost nothing.
The article is more about what you can get from Windows Azure, especially if you already have a subscription.
All source code is available in the open source project at https://bitbucket.org/savamura/scrumboard .
Look at the UI without backend here .
')
Client part
The main part of the application is the client code. It was his writing that took the most time (in fact, it took the most time to write an article). Despite this, I will not describe the implementation in too much detail, relying on clear code. The client will need HTML5 Boilerplate , jQuery , jQuery UI , knockout.js , underscore.js , a little Backbone.js . Mix all together in MVVM salad, add salt to taste.
View (View) is one html page with two buttons (new sticker, save board), a dialog with the properties of the sticker and, in fact, the field where these stickers are “pasted”. Above is a strip with phases under which you can place a sticker.
The data model is extremely simple - it is the coordinates of the sticker, the inscription, additional information, as well as a link to the task in your favorite task management system. The coordinates of the sticker are changed by dragging it, and the associated data via the dialog box. Drag and drop stickers and dialog boxes are effortlessly obtained using jQuery.UI. Using knockout.js allows you to separate the presentation logic from the elements of this representation. In the entire application, there are three ViewModel: BoardViewModel, StickerViewModel and StickerPropertiesDialogViewModel.
We will return to the client a little later. First, create a backend mobile service.
Azure Mobile Service and Custom API
Windows Azure Mobile Services is a specialized cloud hosting for node.js applications. In addition to the REST API, Push notifications to the main mobile platforms are available, authentication using identity providers (Google, Facebook, Twitter, Microsoft), launching scheduled tasks. You can create your application directly in the web console, and you can configure the synchronization of the application with the git repository. Let's try both.
First, create a new Mobile Service in the Azure web console. You must select a unique name, region, as well as the option “Create a free 20 MB SQL database”. You cannot immediately abandon the database, but you can delete it after it is created, or not use it. The fact is that out of the box there are two types of REST API available: the base CRUD API for managing data in SQL tables and the Custom API in which you can do anything. It is a little strange that the SQL database is so annoyingly proposed for use in the cloud service (although if you compare its cost with the cost of no-SQL storage, then everything becomes clear). We could, of course, store data in the SQL database, and, most likely, 20 megabytes would be enough for our board, but it seems to me that in the cloud service it is better to use cloud storage, besides, it is perfect for our task. That is why we will write a Custom API in which we will save and load data into Azure Table Storage.
So, go to the API tab and create a new one. Set the name “board” and leave the default access settings for HTTP requests: “Anybody with the Application Key”. Later these values ​​can be changed. Clicking on the newly created API will open the online code editor with the Hello World application. On this, we will probably finish our acquaintance with the online editor.
To be able to access the API from the local machine during debugging, you need to remember to write the cross-origin resource sharing (CORS) rule in the “configure” tab. You can specify localhost or simply *, thereby allowing access to the API from anywhere.
Deploy to the cloud using git
Azure Mobile Services allows you to deploy a web application using a private git repository. Let's try to take advantage of this cool feature, besides, it will give us an easy way to edit the code in your favorite IDE.
Go to the “Dashboard” tab and in the “Quick glance” column click on “Setup source control”. At the time of this writing, this option was in a preview state.
You will be asked to come up with a username and password for the git repository, after which Azure thinks a bit and in the “Configure” tab you will see the path to the private repository and the secret URL, the synchronization code of the repository with the running application, but I synchronized automatically. Now you can clone the created repository locally using your favorite git client, for example, SourceTree. Also clone bitbucket.org/savamura/scrumboard.git . Inside you will find the service folder, which needs to be synchronized with the one that was obtained as a result of the cloning of the Azure repository. Before you change the changes back to the cloud, let's deal with the server code.
Server side code
There are four subfolders in the service folder: api with a custom API code, a scheduler with a code of tasks running according to a schedule, shared may contain common code, npm modules. The table folder is intended for scripts that modify data in the corresponding SQL table.
Each js script inside the api folder should have a paired json file with the same name. It contains access settings for our API.
Board.js is a node.js module that is used by the express web engine inside Azure Mobile Services.
Our application will require express, azure, and underscore modules:
var express = require("express"); var azure = require("azure"); var _ = require("underscore"); Since we will store the data in Table Storage, we need to prepare it. Return to the Azure web-based management console, create a separate Cloud Storage and copy the Access Key. To access the repository, we use the TableService client written by Microsoft.
function createTableService() { return azure.createTableService( "your_storage_acount", "your_storage_account_key"); } The table in which the stickers will be stored can be created both in the web console and from the code. Our application is not too heavy, and for illustration I have included the code that checks the existence of the table, but it is easier to prepare it in advance so as not to complicate the processing of requests.
Our API will handle only two types of requests: get (will return the entire board) and post (save it).
Inside the methods, you can use console.log tracing. Logs are available in the web console. Not very convenient, but at least you can debug it.
Since the application runs in node.js, all code is asynchronous. You need to be careful and do not forget to complete the request (request.respond or response.send). Full documentation is available on MSDN . First we check the existence of the table and create it using the createTableIfNotExists method. The method passes the name of the table and the callback function.
exports.post = function(request, response) { console.log(request.body); var tableService = createTableService(); tableService.createTableIfNotExists("Stickers", function(error) { if (!error) { console.log("Table Stickers created or existed"); saveBoard(tableService, request.body, response); } else { console.error(error); response.send( statusCodes.INTERNAL_SERVER_ERROR, { message: "Table creation failed. " + error }); } }); }; If the table exists, save the data:
function saveBoard(tableService, boardModel, response) { tableService.beginBatch(); _.each( boardModel.stickers, function(sticker) { sticker.PartitionKey = boardModel.boardId; sticker.RowKey = sticker.id; tableService.insertOrReplaceEntity(stickersTableName, sticker); }); tableService.commitBatch( function(error, operationResponses, batchResponse) { var batchErrors = ""; _.each( operationResponses, function(opresp) { if (opresp.error) batchErrors += opresp.error }); console.log("Stickers saved. Error: " + error + " " + batchErrors); console.log(operationResponses); console.log(batchResponse); if (error || batchErrors !== "") { response.send( statusCodes.INTERNAL_SERVER_ERROR, { message : error + " " + batchErrors }); } else { response.send(statusCodes.OK, { message : 'Batch commited!' }); } }); } Please note that a batch is used to save these stickers. It must be remembered that the batches have some limitations, for example, the PartitionKey should be the same for all entries, and the number of entries should not exceed 100. In this example, there is no splitting of all entries into batchies of 100 pieces.
In order to save a record to a table, the object hierarchy must be linearized. In other words, unlike, for example, from MongoDB, the tree object structure cannot be saved. Linearization occurs on the client. As the PartitionKey, the board identifier (guid) is used, which is now reserved in the client, but in the general case can be anything. The RowKey uses stickerId (guid), which is also created on the client.
In order to extract data from a table, it is enough to correctly form a query:
exports.get = function(request, response) { console.log(request); var boardId = request.query.boardId; if (!boardId) { response.send(statusCodes.BAD_REQUEST, { message : 'boardId is undefined.' }); } var query = azure.TableQuery.select() .from(stickersTableName) .where('PartitionKey eq ?', boardId); var tableService = createTableService(); tableService.queryEntities( query, function(error, entities) { if(!error) { response.send(statusCodes.OK, entities); } else { response.send(statusCodes.INTERNAL_SERVER_ERROR, error); } }); }; Here, in fact, the entire base server code. Of course, there must also be validation of the input data, error handling and access checking. Also, it would not hurt to implement the removal of stickers and protection against xss-attacks. All these problems we address in the next versions. The main thing is to provide basic functionality.
We only need to bind the client and server code.
MobileServicesClient
The code that makes server-side calls is encapsulated in the StorageClient hotel wrapper class and, of course, can be replaced. There are only two methods: load the board and save the board. Information about all the stickers is immediately linearized, serialized and transmitted to the server. Of course, it would be possible to transfer only the changed data, but this will complicate the code somewhat.
To make custom API remote calls, you can use the WindowsAzure.MobileServicesClient client provided by Microsoft. It will be available at your_custom_api.azure-mobile.net/client/MobileServices.Web-1.0.3.min.js (version may vary). The link to it can also be copied from the Azure web console, if you click on the cloud icon with a lightning bolt next to “Dashboard”, then select the HTML / JavaScript platform and Connect an existing HTML app. There you can also get an API access key (below the manage keys button is the application key). The key can be re-created, while the old one will stop working.
StorageClient Constructor:
var StorageClient = function() { this.azureClient = new WindowsAzure.MobileServiceClient( 'https://your_custom_api.azure-mobile.net/', 'your_application_key_goes_here'); }; Our custom API is called board, so to save the board data, you need to do the following:
StorageClient.prototype.saveBoard = function(data) { this.azureClient .invokeApi( "board", { method: "POST", body: data }) .done( function(response) { console.log(response); $.jGrowl("Board saved!"); }, function(error) { var xhr = error.request; $.jGrowl("Error calling azure API " + xhr.status + " " + xhr.responseText); }); }; azureClient.invokeApi returns a promise, to complete which, you can subscribe.
The method of loading data looks the same, however, we additionally pass in it methods that will need to be called upon completion of the download.
StorageClient.prototype.loadBoard = function(boardId, onBoardLoaded, onBoardLoadFailed) { var query = this.azureClient .invokeApi( "board", { method: "GET", parameters: { boardId: boardId } }) .done( function(xmlHttpRequest) { console.log(xmlHttpRequest); onBoardLoaded(xmlHttpRequest); }, function(error) { var xhr = error.request; onBoardLoadFailed(xhr); }); }; Client Deploy in Blob Storage
We assume that we have debugged our application locally. It's time to embark him. As you may have guessed, hosting static content is best in Azure Blob Storage. To do this, you need a container that you can create, for example, via the Azure web console. During creation, specify the level of access to Public Blob, which will mean that access to all blobs in the container will be public, but the metadata of the container itself will remain private.
Now you need to upload files with client code to the cloud. The console utility AzCopy, which can be downloaded from the Microsoft Download Center, is perfect for this task. Click Download and select WindowsAzureStorageTools.msi. After installation, copy all the files you need to upload to the cloud in a separate folder.
Execute
AzCopy <project folder> <container path> / destKey: <access key> / S
If immediately after this you try to open your application in the browser (index.html), you will find that the browser opens a dialog box for selecting the path to save the file. This is due to the blob's content type, which by default is set to application / octet-stream.
This needs to be fixed by writing a small console application in C #. In it, we will go over all the blobs in the container and set them to the standard Content Type depending on the extension. Don't forget to plug in the nuget Windows Azure Storage package.
namespace BlobMetadataChanger { class Program { static void Main(string[] args) { var connectionString = args[0]; var containerName = args[1]; var storageAccount = CloudStorageAccount.Parse(connectionString); var blobClient = storageAccount.CreateCloudBlobClient(); var container = blobClient.GetContainerReference(containerName); var blobs = container.ListBlobs(useFlatBlobListing: true).Cast<CloudBlockBlob>(); foreach (var blob in blobs) { UpdateBlobContentTypeIfExtensionMatch(blob, ".html", "text/html; charset=UTF-8"); UpdateBlobContentTypeIfExtensionMatch(blob, ".css", "text/css; charset=UTF-8"); UpdateBlobContentTypeIfExtensionMatch(blob, ".js", "application/javascript"); UpdateBlobContentTypeIfExtensionMatch(blob, ".png", "image/png"); UpdateBlobContentTypeIfExtensionMatch(blob, ".gif", "image/gif"); UpdateBlobContentTypeIfExtensionMatch(blob, ".jpg", "image/jpg"); } } static void UpdateBlobContentTypeIfExtensionMatch( CloudBlockBlob blob, string extension, string contentType) { if (blob.Name.EndsWith(extension)) { blob.Properties.ContentType = contentType; blob.SetProperties(); } } } } Everything, after executing this utility with the correct parameters, you can open the application through a browser.
Access control
On this, perhaps, it was possible to finish, but we missed one important detail: access control. Now anyone with a link to the application can see the tasks you need to complete in order to take over the world. In truth, your plans are most likely not interesting to anyone, and if you choose guid as the container name, then you need not worry. However, anything can happen, and if you want to control the situation, you will have to make a little more effort.
Open the Azure web console, find your mobile app and go to the “Identity” tab. Let's say that everyone in the team has a twitter account, so we will register our board as a web application there: https://dev.twitter.com/apps/new . After filling in all the fields, you will receive the coveted Consumer Key and Consumer Secret, which must be copied to the appropriate fields in the “Identity” tab. In addition, it is very important to specify the main URL of your mobile service as the Callback URL in the twitter settings of the application, as well as put a check mark in front of “Allow this application to be used.” I did not understand this immediately, so I had to suffer.
Now it is enough, in the http application, after creating the MobileServiceClient client and before sending the request for receiving the board data, call
azureClient.login("twitter").then(loadBoard, onError); The process is described in detail in this lesson .
In the web application code, at the beginning of each of the custom API methods, you need to check the userId.
function CheckUser(request) { var allowedUsers = [ “Twitter:1111111”, “Twitter:22222222” ]; return _.find( allowedUsers, function(allowedUser) { request.user.userId == allowedUser }) !== undefined; } Conclusion
So we got to the end of this article, but not to the end of the topic. Last week, a new batch of updates to the Azure cloud service came out, including the emergence of support for CORS in Cloud Storage. This means that by rewriting the storageClient client code and setting the CORS rules as necessary, you can save the data in Table Storage directly from the browser. In this case, you do not even need to write the server part in the form of the Azure Mobile Service, since its main function is to save the data to a table. Of course, we will lose the ability to authenticate users, but we will get a working application consisting only of the browser client code. Habrastia has already appeared on this topic.
Source: https://habr.com/ru/post/204394/
All Articles