How Yahoo moved from Flash to HTML5 to video

Adobe Flash was once the de facto standard in the world of web media, but over time the industry turned away from it for security and performance reasons. Requiring users to install a plugin for video playback is also a bad practice. As a result, we turn to HTML5 for the video.

Development in the field of video playback on HTML5 is still in its infancy , and browsers initially supported these features in the most primitive way. Only recently has support been expanded to include adaptive streaming. Adaptive streaming has two main advantages:

• Adaptive Bitrate (ABR): The algorithm determines the user channel bandwidth, processor power, player size, etc. in real time and adjusts the video settings.
• Resizable buffer size: a feature that allows us to control the time it takes to start playback.

These features allowed the video streaming industry to move from Flash to HTML5 and JavaScript.

Our Yahoo video player uses HTML5 in all modern browsers. In this post, we describe our path to the realization of these opportunities, describe the problems we have encountered, and describe the opportunities that we see.

First steps towards HTML5

We took the first steps towards HTML5 in October 2015, when we organized the global live broadcast of the NFL game for the first time. For this event, we rolled out a "clean" HTML5-player on Safari. It was based on native HTML5 support in the browser for HTTP Live Streaming (HLS) . As part of this event, we have developed special features to enable different types of render depending on the client’s browser (Flash support, device configuration, operating system, browser, etc.).

Architectural solutions

To achieve HTML5 support in all browsers, we needed to re-design the streaming in our player. We faced a choice, and the decision could affect the entire Yahoo business and user experience.

The first, and perhaps most important, point is which streaming protocol to support. The choice was between HLS and DASH , they both support HTTP streaming streaming. However, in order to keep the stack simple enough, and the development is fast enough, we decided to support HLS. For iOS, we would have to support HLS anyway, and as the standard evolves, at some point Media Source Extensions (MSE) can start working with HLS. MSE is a recent development of the HTML5 standard that allows you to dynamically generate media streams for playback via tag .

The next question is to build by yourself, buy ready-made or use an open source HTML5 player. Yahoo is not the only one who needs an HTML5 player. There are also several open source projects. To use them is to save time at the beginning. However, analysis, including real-world testing by other players in the market, showed that existing players do not provide enough of the quality, performance and scalability that we expect from Yahoo Video Player.

And finally, our existing video player, which supports Flash, was already mature and proven. We needed to decide whether to port design and logic from Flash to JavaScript, or build and design a player from scratch. We chose the latter. This made it possible to achieve some architectural goals, including a level of extensibility that allowed DASH to be maintained at later stages. This solution allowed us to avoid problems and shortcomings associated with the previous design.

As you will see below, all these decisions have brought us many benefits.

Zoom into the future

Armed with these solutions, we began developing a player that would free us from Flash. The project was given the code name "Zoom", this is the main enemy of the superhero Flash from the DC Comics universe.

The media pipeline for HLS streaming looked like this:



Figure 1. Media pipeline for HTML5

The player acts as a demultiplexer for the incoming transport stream ( MPEG-TS ). It splits the stream into audio and video, which are then packaged in the fragmented MP4 format, which is received by the MSE layer in the browser.

When we designed the new HTML5 player, we had several goals. It should be:

• Modular: each component must be developed separately and tested independently
• Expandable: a new player must be able to support new features (for example, DASH) without having to redesign the player.
• Stateless: we can use components (like ABR) in different instances of the player on the same page or in the same application.

Figure 2 below shows the high-level architecture of the new HTML5 MSE player.

Figure 2. Yahoo's HTML5 player architecture

• Framework Services provides common features such as the HTTP downloader (for downloading videos), Web Workers (for multithreading), and the channel bandwidth determiner.
• Stream & Media Services includes services that process media at different stages of the pipeline. This includes loading the transport stream, splitting the stream and packing to MP4, which will be played using MSE.
• Streaming Controller is the component that controls the streaming of video content. It also communicates with the ABR engine to determine the desired bitrate.
• Playback Controller is responsible for playing videos using different modules. It stores a state machine with different playback states. It also provides an API for play, pause, rewind, and so on.

Problems

First, we moved from a single framework (Adobe Flash), which provided the same environment in all browsers, to several frameworks (MSE, XHR , Web Workers, HTML5 Media Elements) on different platforms and browsers (Chrome, Firefox, IE, Edge , Safari, etc.), each of which added its own troubles to the system.

The second problem was advertising. Video playback was turned to HTML5, but most advertisers in the video world continued to use Flash. Therefore, we had to find a way to show ads on Flash, and the video at the same time play through HTML5. We made our player use several render components, each of which supported their own rendering technique (Flash, HTML5, etc.). This allowed us to maintain optimal conditions for the user and not lose revenue.

The third problem is how to improve user involvement, a key metric for successful video consumption. We wanted users to get involved and interact with the video all the time, and that it didn’t require any action on their part, such as a click or additional page load. At the same time, we did not want the pages to have consistent playback. Therefore, we have included "playlists" in the first-class API in the video player itself. Now we can set a curated list of videos that are highly context sensitive and personalized for a specific user.

Figure 3 below shows the high-level architecture of the Yahoo video player.

Figure 3. Yahoo video player architecture

• The controller is responsible for switching the render components and provides access to various playback functions.
• Ads Controller manages and roars video ads.
• Playlist Manager manages video playlists and provides access to playlist functions.

Performance (rebuffering and startup time) is the main driver for user engagement. Performance changes are always associated with obstacles.

Audio / video demultiplexing and MP4 packaging are expensive for processor operations. If you perform these operations in the main browser UI thread, this will affect the responsiveness of the user interface of the page and the player. Fortunately, browsers have web workers that allow multi-threading, but using them means that you need to transfer messages between threads.

From our experience, in Firefox, using a worker for splitting the stream and MP4 packaging was 20% more efficient (compared to the version without a worker). On the other hand, we found that the additional load associated with the transfer of messages between threads is particularly noticeable in IE and Edge, and this leads to an increase in re-buffering. To solve these problems, we made the following changes:

• Run handler units inside the workers
• Minimize messages between threads

Effective use of web workers for media transformation has given our player a performance advantage over other players. This is a 10% -20% improvement for the processor and a 30% improvement in re-buffering.

Opportunities

Despite the fact that we faced a lot of problems, designing our player gave us the opportunity to add features that were not available in the past player.

Since we added the ability to switch render components and advertise on Flash, and the content on HTML5, we were able to preload the video. That is, we can start downloading the next video in the playlist even before it starts playing. For example, when the ad is loaded and started to play, we can preload the content in the background. The transition from advertising to video is obtained as in the TV.

We also improved the algorithm that determines the bandwidth of the channel. It was originally based solely on content download time. We added to it the information received from the API, for example TTFB (Time To First Byte - time to the first byte).

We also added a feature that allows you to switch the bitrate at the level of key frames. This helped us respond better to unexpected changes in network conditions.

results

We started rolling out a new HTML5 player in Google Chrome, and over time, added support for other browsers. Now the new player works in all modern browsers. Figure 4 below shows the number of video plays, depending on the render component used. Today we use HTML5-rendering for approximately 70% of the traffic. This number will grow with the introduction of the player to other parts of the Yahoo network. The most visible platform that does not support MSE is IE in Windows 7. Flash continues to be used there.

Figure 4. Views and render method

As for such an important metric as a re-buffering indicator, the HTML5 player is approximately at the same level or slightly better than Flash (Fig. 5).

Figure 5. Rebuffering Rate - Flash vs. HTML5

HTML5 is better when it comes to starting time after the user has clicked on "PLAY". Figure 6 shows the delay between clicking on “PLAY” and rendering the first frame of the video (Click To Play Latency) for Flash and for HTML5 players.

Figure 6. Click To Play Latency - Flash vs. HTML5

The advantages of the HTML5 player are clearly seen on these charts: faster loading, better performance, and so on.

What's next

Adaptive streaming on the Internet is developing rapidly. The industry is focused on optimizing playback in the context of a single player, while at Yahoo we are also working on streaming optimization of several videos on one page. We are also working to ensure that MSE-HTML5-players come to the mobile web world.

Apple recently announced support for fragmented MP4 as a transport stream in HLS. This solution fits well with our decision to work with HLS. This gives us three advantages:

• Simplify the player, as MP4 is natively compatible with MSE.
• Improved player performance due to avoiding expensive processor-related operations for splitting the stream and assembling into MP4.
• Reducing the amount of information transmitted and improving the quality and start time.

We continue to focus on improving modern streaming technologies on the Internet, and we are hiring new people! Write letters to amitnj@yahoo-inc.com , and we will discuss career opportunities in our team.