Video Streaming with Azure and .NET

The project Jisp, the development of which is engaged in the company WaveAccess, there was the task of downloading and playing video. In general, nothing unusual, but well-known problems are associated with it, such as the need for preprocessing, as well as the need to support the possibility of streaming in the required quality and format. The video should play successfully on the main platforms, namely Android, iOS and desktop (web). Read more about solving problems under the cut!




To solve this problem, we used the technology of Azure Media Services. On the server side .Net platform is used. In addition to the standard requirements associated with video streaming, there were also their own requirements for the business logic of the application. Users can upload videos and attach them to a specific geolocation. Full video should be available only in the specified area. In other cases, other users can lose only a short fragment given by the author of the video. In addition, you need to trim the video to square proportions.

Access setting

Before moving on to uploading videos, you need to configure the vault and access to it. To do this, you need to prepare 2 accounts: Media service account and Azure storage Account.

Media service account

He is responsible for storing the metadata video. When creating this type of account, you need to specify the following set of parameters:

• Resource group - used to group resources in Azure with the ability to configure access to the entire group
• Location - the territorial location of the data center in which the resource will be created
• Storage account - used to save assets

Storage account

This type of account is designed to store files, in this case, he is directly responsible for storing the video.

Authorization

Once created accounts, you must configure the authorization. Azure media services use authorization through the Azure Active Directory. There are 2 authorization options:

• User
• Service principal

The first approach makes sense if we do not have an intermediate service that would take over the authorization. For example, if we are dealing with a desktop application that interacts directly with Azure Media Services. In this case, we will ask the user for the username and password for Azure Active Directory.

User Account Authorization


The second option is more convenient if we have our own service, which is responsible for authorizing users in the application. For example, if we have a mobile or web application with our back-end service. In this case, AMS access keys are stored on the server, and user authorization rests with the business logic of the application.

Authorization using the Service Principal


In the case of Jisp, the second option is selected.

Having prepared accounts and authorization, we can begin integration with the .Net SDK for AMS. The main object responsible for working with AMS entities is CloudMediaContext. Below is the code that is responsible for its initialization:

var azureKey = new AzureAdClientSymmetricKey(clientId, clientKey); var credentials = new AzureAdTokenCredentials(tenant, azureKey, AzureEnvironments.AzureCloudEnvironment); var tokenProvider = new AzureAdTokenProvider(credentials); var context = new CloudMediaContext(new Uri(url), tokenProvider); 

Assets

After we initialize CloudMediaContext, you can proceed to download the video. For this, AMS uses asset entities. This is a resource that directly contains a video stream, audio, thumbnail images, and other files related to video. First, you need to create an asset-resource itself.

 var inputAsset = context.Assets.Create(assetName, storageName, AssetCreationOptions.None); 

After that, add the source video file to the asset:

 var assetFile = inputAsset.AssetFiles.Create(fileName); 

Next, you need to convert it for adaptive streaming:

 var job = context.Jobs.CreateWithSingleTask("Media Encoder Standard", "Adaptive Streaming", inputAsset, "Adaptive Bitrate MP4", AssetCreationOptions.None); job.Submit(); job = job.StartExecutionProgressTask( j => { Console.WriteLine("Job state: {0}", j.State); Console.WriteLine("Job progress: {0:0.##}%", j.GetOverallProgress()); }, CancellationToken.None).Result; Console.WriteLine("Transcoding job finished."); var outputAsset = job.OutputMediaAssets[0]; 

After the video is ready, we publish it with the help of Locator:

 context.Locators.Create(LocatorType.OnDemandOrigin, outputAsset, AccessPermissions.Read, TimeSpan.FromDays(365)); 

Now we can play our video.

 var videoUri = outputAsset.GetMpegDashUri(); 

Filters

For video modification, Azure Media Services offers a filter mechanism. It is possible to specify the quality of audio or video, the resolution used by the codec, select a specific interval.

There are 2 types of filters:

• Local Such filters can only be applied to the video for which they were created. Accordingly, for each video you need to create a separate filter.
• Global. Such filters can be applied to any video of this account. They are especially relevant when you need to support streaming video in various formats for different devices.

The following is an example of creating a filter that limits video playback to a specified interval:

 var filterName = $"{length}seconds"; outputAsset.AssetFilters.Create(filterName, new PresentationTimeRange(start: 0, end: Convert.ToUInt64(10000000 * length)), new List<FilterTrackSelectStatement>()); 

To apply the created filters, you need to modify the streaming url as follows.

Source url:
testendpoint-testaccount.streaming.mediaservices.windows.net/fecebb23-46f6-490d-8b70-203e86b0df58/BigBuckBunny.ism/Manifest (format = mpd-time-csf)

url after adding 10seconds filter:
testendpoint-testaccount.streaming.mediaservices.windows.net/fecebb23-46f6-490d-8b70-203e86b0df58/BigBuckBunny.ism/Manifest (format = mpd-time-csf, filter = 10seconds)

If we need to apply multiple filters, the url will be as follows:
testendpoint-testaccount.streaming.mediaservices.windows.net/fecebb23-46f6-490d-8b70-203e86b0df58/BigBuckBunny.ism/Manifest (format = mpd-time-csf, filter = 10seconds; square)

Scaling

Like any cloud service, AMS greatly simplifies the scaling process.
If we talk about video preprocessing, AMS provides the following options:

1. Vertical scaling, in this case, increasing the speed of video processing. There are 3 tariffs with x1, x2 and x4 performance, respectively. Therefore, if the time of preparation of the video is not critical, you can choose the minimum rate. If it is necessary to reduce the processing time to a minimum, it makes sense to use x4.
2. Horizontal scaling, i.e. the ability to simultaneously process multiple videos. This is easily achieved by setting the required number of Media processing units.

If we talk about streaming video, AMS supports automatic bandwidth changes as needed. There is also a division into standard and premium rates. The first is designed for bandwidth up to 600Mbps. The second is 200 Mbps per streaming unit.

Speed

For example, take the video quality 720p, length 30 seconds, 1 minute and 5 minutes, and compare the preprocessing time. The results were as follows:

Rate	30 sec (4.4 MB)	1 minute (8.1 MB)	5 minutes (44.9 MB)
x1	1:32 (49 Kb / s)	2:14 (62 Kb / s)	8:38 (89 Kb / s)
x2	1:07 (68 Kb / s)	1:31 (91 Kb / s)	5:07 (150 Kb / s)
x4	0:33 (137 Kb / s)	0:45 (185 Kb / s)	1:49 (422 Kb / s)

For small videos, the gain in preprocessing performance is not as significant, especially for the x2 tariff. But as the situation increases, the situation changes for the better, and tariffs x2 and x4 give a corresponding increase in processing speed.

Underwater rocks

Azure Media Services, like any technology, imposes its limitations.

The first difficulty we face is the delay between the moment the video is loaded and the moment it is ready for streaming. The ideal option is to be able to stream immediately after downloading. In principle, AMS makes it possible to get a link to a raw video. But the preparation of asset files was still longer than simply loading videos into Azure Storage bypassing AMS. In this regard, we decided to first upload the original video to Azure Storage bypassing AMS and give it directly for the preprocessing period.

The next difficulty was related to the change in the proportion of the video. In our case, AMS incorrectly handled non-standard proportions (1: 1). Although the video itself became square, the metadata showed a different ratio, and the thumbnail image was distorted. In this regard, this task had to be implemented using ffmpeg.

There was also a certain difficulty with the indication of the fragment being played. By default, the duration was a multiple of 6 seconds. Using the extended configuration, it was possible to reduce the step to two seconds, which was acceptable.

As for the rest, Azure Media Services does an excellent job with its task, providing a ready-made solution for streaming video to many clients.

about the author

The WaveAccess team creates technically sophisticated high-load fault-tolerant software for companies all over the world. Alexander Azarov , senior vice president of software development at WaveAccess, comments on:

Complex at first glance, the problem can be solved relatively simple methods. It is important not only to learn new tools, but also to perfect the knowledge of familiar technologies.

Company blog