Application Development with NexStreaming NexPlayer SDK
Developing a mobile player application is not easy. It is necessary to provide support for many formats of video, audio, subtitles. We'll have to implement the playback of files that are stored locally and on the Internet. If we recall that the result of all these works will be performed in an environment whose resources should be treated carefully, we will have to tackle optimization. How to approach the creation of high-quality, fast and economical player for Android? NexPlayer SDK will help to answer this question.
NexStreaming is a global company that develops software for mobile devices. NexStreaming is headquartered in Seoul, Korea, with branches located in Spain, the United States, Japan and China. NexPlayer SDK is its most popular product. This is a software development kit for creating players. It is used in its mobile applications by some very well-known companies that stream video. The player is compatible with all modern DRM-technologies. In addition, its basic functionality can be expanded. For example, add tools to improve the sound, support for advertising inserts, functions of collecting statistical information about the audience.
NexPlayer SDK from NexStreaming provides the means to decode and play video and audio data. Application developers can use the SDK to quickly create their own products. NexPlayer SDK is reliable, stable, compatible with international standards. From this material, you will learn how to create a player designed for the x86 architecture using the NexPlayer SDK.
')
NexPlayer SDK is optimized for the x86 platform, that is, it is fully supported by devices built on this platform. Here are the main features of the SDK:
First you need to download the SDK from the NexStreaming website, filling out a small form and waiting for an e-mail with the data to enter the closed section of the site, where the materials are located. In particular, among the materials available for download, you can find a demo application in an .apk file that can be immediately installed on an Android device, a sample project that can be imported into the IDE, the SDK itself as a set of libraries, and its documentation . In order to decide which libraries you need, you should read the documentation and look at the examples. Developing an application using the NexPlayer SDK is quite simple and straightforward, and the presence of examples contributes to its rapid development. It takes about an hour to create a working application that uses the NexPlayer SDK.
Integration of the NexPlayer SDK into an Android application designed for the x86 platform consists in copying libraries and source code from the SDK to your project. By following a few simple steps, described below, you will equip the application with basic components that will allow it to work perfectly on devices built on the basis of components from Intel. So, the preparatory stage is as follows:
The set of copied libraries depends on the needs of your application. In order to understand exactly which libraries you need, you should look at the demo and use the recommendations given below.
After copying, NexPlayer SDK will detect changes and will be able to use libraries to organize efficient work with Intel resources. As soon as all the libraries are in the specified directories, the SDK will take over switching between ARM and x86 versions of the application. If you want to update the NexPlayer SDK libraries in an existing project, just overwrite their files.
NexPlayer SDK includes a large number of libraries, including those that are responsible for supporting DRM. They are located in the app / assets / x86 folder. The required set of libraries includes the player engine (engine), decoders (decoders) and rendering layer (rendering layer). Here is a list of them:
Some of the library names contain the abbreviation "ics". It indicates that the library is designed for Android Ice Cream Sandwich. The abbreviation "jb" says that the library is designed for Android Jelly Bean. If your application is designed to support certain versions of Android, libraries for unsupported versions can be removed.
Here are the libraries responsible for supporting codecs:
The following are the libraries whose task is to support subtitles:
In order to reduce the size of the application, include in the project only those libraries that you need.
Libraries from the app / libs / x86 folder need to be downloaded. This is done in the initManager () method of the corresponding class. For example, to use the “NexHDSample”, you need to load the appropriate x86 library in the aforementioned method of the NexHDManager class (app / src / NexHDManager.java) with the following code:
System.loadLibrary ("NexHTTPDownloaderSample_jni");
When working with the NexPlayer SDK, there are two ways to display video. This is the use of NexVideoRenderer and OpenGL based output subsystems. It is recommended to use NexVideoRenderer. The point is that this subsystem provides a level of abstraction on complex operations of managing surfaces and video output tasks. It selects the most appropriate way to output video based on the hardware of the device and the operating system installed on it.
In order to use NexVideoRenderer, you need to do the following:
The NexVideoRenderer class code can be found among SDK usage examples at NexPlayerSample / src / com / nexstreaming / nexplayerengine / NexVideoRenderer.java
HTTP Live Streaming technology allows you to work with several audio and video streams. The SDK supports this feature; its setMediaStream () API allows you to select streams for playback. This can be done from the user interface while playing materials. There are the following options for working with video and sound:
Below are the test results, during which the load on the system created by the NexPlayerDemoApp application at startup, simple, playing a local MP4 file and playing streaming video is estimated. The application is developed using the NexPlayer SDK. The VTune analyzer for Android was used for the analysis, as well as the Intel SoC Watch. The tests were carried out on a tablet on which Android 4.4.2 was installed. It is equipped with a 1.6 GHz Intel Atom Z3740 processor and Intel HD Graphics (Gen 7) graphics accelerator. The device is connected to the Internet via Wi-Fi.
Measurement reference point for a device built on the Intel Atom Z3740 processor
NexPlayerDemoApp application in idle state
The more time the Intel Atom Z3740 processor spends in the C0 state when the application is idle, the higher the power consumption. In the case of NexPlayerDemoApp, the use of the processor is very close to the reference point of measurement, as a result - the inaction of the application does not create an unnecessary load on the system.
NexPlayerDemoApp application at startup
These data are also very close to the reference point of measurement.
When playing video, the x86 version of the NexPlayerDemoApp application uses, on average, 33% of processor power. When analyzing this task, no anomalies were identified, and when it is executed, a uniform load is created on the processor.
The load on the processor created by the x86 NexPlayerDemoApp application when playing local video
When playing streaming video, the x86 version of the NexPlayerDemoApp application uses, on average, 25% processor power. When analyzing this task, no anomalies were identified, and when it is executed, a uniform load is created on the processor.
The load on the processor generated by the x86 NexPlayerDemoApp application when playing streaming video
The power consumption of the x86 version of the NexPlayerDemoApp application was approximately 3.8 watts. Most of the time during the measurements, streaming video was played, there was a short period of inactivity. The rest of the time was spent playing back the local video file.
Find out how the demo application built using the NexPlayer SDK works on Android 5.0 Lollipop. To test use the tablet Asus Fonepad 8 based on the CPU Intel Atom Z3530. The device is connected to the Internet via Wi-Fi.
Application main screen
On the main screen of the application you can find several buttons:
During the test, the application coped with its functions. In particular, videos from various sources were played at the correct speed and without delays.
Local video playback
With the help of the NexPlayer SDK, you can quickly and easily develop applications whose main task is video playback. In addition, the tests prove the efficiency and low power consumption of such applications on devices built on the x86 platform.
Preliminary Information
NexStreaming is a global company that develops software for mobile devices. NexStreaming is headquartered in Seoul, Korea, with branches located in Spain, the United States, Japan and China. NexPlayer SDK is its most popular product. This is a software development kit for creating players. It is used in its mobile applications by some very well-known companies that stream video. The player is compatible with all modern DRM-technologies. In addition, its basic functionality can be expanded. For example, add tools to improve the sound, support for advertising inserts, functions of collecting statistical information about the audience.
NexPlayer SDK from NexStreaming provides the means to decode and play video and audio data. Application developers can use the SDK to quickly create their own products. NexPlayer SDK is reliable, stable, compatible with international standards. From this material, you will learn how to create a player designed for the x86 architecture using the NexPlayer SDK.
')
Main features
NexPlayer SDK is optimized for the x86 platform, that is, it is fully supported by devices built on this platform. Here are the main features of the SDK:
- Android 1.6 or higher support
- Playing files in mp4, 3gp, avi, asf, piff formats
- Support for streaming data technologies HTTP Live Streaming version 5.0 and 3GPP Progressive Download. Supports AES128 and HTTPS protocols, h.264, AAC, AAC +, eAAC + codecs. The SDK allows you to work with both software and hardware codecs.
- Subtitle support in .smi, .srt, .sub, 3GPP timed text, TTML (PIFF / CFF only), CEA 608 and CEA 708 formats. Supports Web Video Text Tracks technology (WebVTT).
How to create a player for the x86 platform using the NexPlayer SDK
First you need to download the SDK from the NexStreaming website, filling out a small form and waiting for an e-mail with the data to enter the closed section of the site, where the materials are located. In particular, among the materials available for download, you can find a demo application in an .apk file that can be immediately installed on an Android device, a sample project that can be imported into the IDE, the SDK itself as a set of libraries, and its documentation . In order to decide which libraries you need, you should read the documentation and look at the examples. Developing an application using the NexPlayer SDK is quite simple and straightforward, and the presence of examples contributes to its rapid development. It takes about an hour to create a working application that uses the NexPlayer SDK.
Integration of the NexPlayer SDK into an Android application designed for the x86 platform consists in copying libraries and source code from the SDK to your project. By following a few simple steps, described below, you will equip the application with basic components that will allow it to work perfectly on devices built on the basis of components from Intel. So, the preparatory stage is as follows:
- Copy the necessary libraries from the SDK / libs folder to the assets / x86 folder of your own project
- Copy the necessary libraries from the SDK / libs folder to the libs / x86 folder of the project
- Copy java-files from SDK / src to src / com / nexstreaming / nexplayerengine project folder
The set of copied libraries depends on the needs of your application. In order to understand exactly which libraries you need, you should look at the demo and use the recommendations given below.
After copying, NexPlayer SDK will detect changes and will be able to use libraries to organize efficient work with Intel resources. As soon as all the libraries are in the specified directories, the SDK will take over switching between ARM and x86 versions of the application. If you want to update the NexPlayer SDK libraries in an existing project, just overwrite their files.
NexPlayer SDK includes a large number of libraries, including those that are responsible for supporting DRM. They are located in the app / assets / x86 folder. The required set of libraries includes the player engine (engine), decoders (decoders) and rendering layer (rendering layer). Here is a list of them:
- libnexplayerengine.so
- libnexalfactory.so
- libnexadaptation_layer_for_dlsdk.so
- libnexralbody_audio.so
- libnexralbody_video_opengl.so
- libnexral_nw_ics.so
- libnexral_nw_jb.so
- libnexcal_oc_ics.so
- libnexcal_oc_jb.so
- libnexcralbody_mc_jb.so
- libnexcal_in_aac_x86.so
- libnexcal_in_mp3_x86.so
- libnexcal_in_amr_x86.so
Some of the library names contain the abbreviation "ics". It indicates that the library is designed for Android Ice Cream Sandwich. The abbreviation "jb" says that the library is designed for Android Jelly Bean. If your application is designed to support certain versions of Android, libraries for unsupported versions can be removed.
Here are the libraries responsible for supporting codecs:
- libnexcal_h364_x86.so - H.264 video support library
- libnexcal_aac_x86.so - sound library for AAC, AAC-Plus and HE-AAC formats
- libnexcal_mp3_x86.so - sound library for MP2 and MP3 formats
The following are the libraries whose task is to support subtitles:
- libnexcal_3gpp_x86.so - support for 3GPP format timed text
- libnexcal_closedcaption_x86.so - support for CEA 608 and CEA 708 formats
- libnexcal_ttml_x86.so - TTML (CFF) format support
- libnexcal_webvtt_x86.so - support for the WebVTT format
In order to reduce the size of the application, include in the project only those libraries that you need.
Libraries from the app / libs / x86 folder need to be downloaded. This is done in the initManager () method of the corresponding class. For example, to use the “NexHDSample”, you need to load the appropriate x86 library in the aforementioned method of the NexHDManager class (app / src / NexHDManager.java) with the following code:
System.loadLibrary ("NexHTTPDownloaderSample_jni");
How to display video on the screen of an application designed for the x86 platform
When working with the NexPlayer SDK, there are two ways to display video. This is the use of NexVideoRenderer and OpenGL based output subsystems. It is recommended to use NexVideoRenderer. The point is that this subsystem provides a level of abstraction on complex operations of managing surfaces and video output tasks. It selects the most appropriate way to output video based on the hardware of the device and the operating system installed on it.
In order to use NexVideoRenderer, you need to do the following:
- Pass the context (android.context.Context) to the constructor.
- Install event listeners (NexPlayer.IListener and NexPlayer.IVideoRendererListener).
- Create an instance of an object of the NexPlayer class.
- Make the necessary training NexPlayer (use the methods NexPlayer.setNexAlFactory and NexPlayer.init).
- Call the init method of the NexVideoRenderer class object (NexVideoRenderer.init).
- Add an instance of NexVideoRenderer as an interface element to the layout.
The NexVideoRenderer class code can be found among SDK usage examples at NexPlayerSample / src / com / nexstreaming / nexplayerengine / NexVideoRenderer.java
Streaming broadcast
HTTP Live Streaming technology allows you to work with several audio and video streams. The SDK supports this feature; its setMediaStream () API allows you to select streams for playback. This can be done from the user interface while playing materials. There are the following options for working with video and sound:
- Use a playlist with various soundtrack options. Video and sound with this approach can be selected independently.
- Use a playlist with various video options. Each video clip being played contains both sound and video, but the user can select alternative video options. For example, it may be material shot by a camera from different angles or from different points.
- The combination of lists of available audio tracks and video. This is a mixture of the two options described above. The main video stream provides video tracks in different bitrates, but reproduced with the same sound. Additional audio tracks are also available, for example, in different languages.
Analysis of performance and power consumption x86-applications NexPlayerDemoApp
Below are the test results, during which the load on the system created by the NexPlayerDemoApp application at startup, simple, playing a local MP4 file and playing streaming video is estimated. The application is developed using the NexPlayer SDK. The VTune analyzer for Android was used for the analysis, as well as the Intel SoC Watch. The tests were carried out on a tablet on which Android 4.4.2 was installed. It is equipped with a 1.6 GHz Intel Atom Z3740 processor and Intel HD Graphics (Gen 7) graphics accelerator. The device is connected to the Internet via Wi-Fi.
Measurement reference point for a device built on the Intel Atom Z3740 processor
| Batch processor status | % of time spent in the specified state, core # 0 | % of time spent in the specified state, core number 1 | % of time spent in the specified state, core number 2 | % of time spent in the specified state, core number 3 | % of the time the graphics core |
| C0 | 4.00 | 2.50 | 0.40 | 0.40 | 0.30 |
| C6 | 95.00 | 95.10 | 98.30 | 98.10 | 99.50 |
NexPlayerDemoApp application in idle state
| Batch processor status | % of time spent in the specified state, core # 0 | % of time spent in the specified state, core number 1 | % of time spent in the specified state, core number 2 | % of time spent in the specified state, core number 3 | % of the time the graphics core |
| C0 | 4.80 | 3.70 | 1.2 | 0.90 | 0.00 |
| C6 | 93.50 | 94.10 | 93.40 | 93.90 | 100.00 |
The more time the Intel Atom Z3740 processor spends in the C0 state when the application is idle, the higher the power consumption. In the case of NexPlayerDemoApp, the use of the processor is very close to the reference point of measurement, as a result - the inaction of the application does not create an unnecessary load on the system.
NexPlayerDemoApp application at startup
| Batch processor status | % of time spent in the specified state, core # 0 | % of time spent in the specified state, core number 1 | % of time spent in the specified state, core number 2 | % of time spent in the specified state, core number 3 | % of the time the graphics core |
| C0 | 4.80 | 3.70 | 1.2 | 0.90 | 0.90 |
| C6 | 93.50 | 94.10 | 93.40 | 93.90 | 99.00 |
These data are also very close to the reference point of measurement.
Playing local video
When playing video, the x86 version of the NexPlayerDemoApp application uses, on average, 33% of processor power. When analyzing this task, no anomalies were identified, and when it is executed, a uniform load is created on the processor.
The load on the processor created by the x86 NexPlayerDemoApp application when playing local video
Streaming video playback
When playing streaming video, the x86 version of the NexPlayerDemoApp application uses, on average, 25% processor power. When analyzing this task, no anomalies were identified, and when it is executed, a uniform load is created on the processor.
The load on the processor generated by the x86 NexPlayerDemoApp application when playing streaming video
power usage
The power consumption of the x86 version of the NexPlayerDemoApp application was approximately 3.8 watts. Most of the time during the measurements, streaming video was played, there was a short period of inactivity. The rest of the time was spent playing back the local video file.
NexPlayerDemoApp app on Android 5.0 Lollipop
Find out how the demo application built using the NexPlayer SDK works on Android 5.0 Lollipop. To test use the tablet Asus Fonepad 8 based on the CPU Intel Atom Z3530. The device is connected to the Internet via Wi-Fi.
Application main screen
On the main screen of the application you can find several buttons:
- History of views (History).
- Open URL - when you click on this button, a pop-up window appears to enter a link to the streaming video.
- Watch sample streams - clicking on this button opens a screen where you can select one of the offered videos for viewing.
- Play local files - after clicking on this button, the program shows a list of device folders that contain video files.
During the test, the application coped with its functions. In particular, videos from various sources were played at the correct speed and without delays.
Local video playback
Results
With the help of the NexPlayer SDK, you can quickly and easily develop applications whose main task is video playback. In addition, the tests prove the efficiency and low power consumption of such applications on devices built on the x86 platform.
Source: https://habr.com/ru/post/264103/
All Articles