HTML5 Audio is a state of affairs. Part 2

(An article by the frontend and media specialist Mark Boas (Mark Boas) dated May 8, 2012. Translation of the final part. The beginning made it clear that you have to sweat before you master the whole kitchen, but this is still developing ... Do not expect short recipes.)



Content of the first part :

■ MIME Types

● Server part

● Client part

■ Know the type of audio in advance .canPlayType (probably)

■ Current codec support in browsers

■ Container Formats and File Extensions

■ We have the <audio> tag and are not afraid to use it!

■ Buffering, Search and Playback Intervals

● Buffered attribute

● TimeRanges object

● Attributes seeking and seekable

● Attributes seeking and seekable

Search in the context of a media file is peering forward or backward into a media file. This usually happens when the full file buffering is not complete. The seeking attribute is used to indicate that a " seeked " event has occurred. true means that part of the file has not yet been loaded.

It's funny : all modern browsers, except Safari / Windows, have the seekable attribute, meaning that the seeking event will never happen so that the user can go directly to the desired part of the program. The seeked event does not appear, which annoys some who believe that " seeking == true " should always be ahead of the seeked event.

Recall that the buffered property says which parts were loaded, which is often used as an indicator of readiness for playback. If the browser supports it, then it may make sense to use the seekable attribute to determine which part of the file is loaded.

')

seekable returns a TimeRanges object - intervals that are ready for immediate playback. It uses a technology known as “byte-range-requests”, which allows you to request some of the data not strictly at the end of HTTP requests, but earlier, while there are still requests. In other words, we don’t have to have all the data to start playing media. Examples:

var isSeeking = audio.seeking; //     ""? //   seekable, ,    "" var isSeekable = audio.seekable && audio.seekable.length > 0; //   ,       var seekableEnd = audio.seekable.end(); 

Note : spacing can be misleading. audio.seekable.end () actually indicates the end point of the last interval, not the end point of the entire media file. In practice, however, this is sufficient, since the browser either gives a series of requests or not. If it turns out that there were no requests, audio.seekable will be equivalent to audio.buffered , which will give the correct indication of the end of the intervals in the file.

Notice that the playback plots in the " seekable " state are different from those in the " buffered " state. They cannot be both at the same time.

Funny : In practice, only one TimeRanges object is created in the seekable property. In browsers that support interval requests, the TimeRanges object extends from zero to the end of the playback time; in non- supporting, TimeRanges grows from zero to the end as it loads.

Buffering and data retrieval may provide useful information, but it would be too easy if not for a few moments.



1. Preloading is not supported in moderately old browsers (Opera 10 / Firefox 3.6).

2. The buffered attribute is not always supported (Blackberry PlayBook).

3. Not all HTML5 browsers support byte-range-search, for example, Safari / Windows. In this case, the search for downloaded content remains, as Flash does.



If you want to give users the best possible solution, do the in-test.

● Note about preloading

I mentioned the preload attribute in a previous article (eng.) . It takes 3 possible values:

1. none - does not preload anything. Wait for the play event before loading anything.

2. metadata - preloads only metadata. Gets the beginning and end of the file through the interval request and determines the duration of the playback.

3. auto - preloads the entire file. Receives the beginning and end of the file, and then proceeds again to download from the beginning of the file.

Funny : Mobile Safari ignores the preload attribute, which is always " none " in it.

When it comes to preloading, remember that this is a request or a wish to the browser about how you want to see the preloading process. Browsers are not required to perform it, so they can work in different ways.

● Successfully played

It should be noted that there is a property played , indicating whether the intervals were completely reproduced.

 var played = audio.played; //   TimeRanges 

Note : if we sum up all the audio.played intervals, we get a fraction of the audio that was listened to, which can be useful for collecting metrics.

■ Media events

The audio and video interfaces API is based on a complete set of events. The full list is available in the WhatWG version of the specification .



Installing media event handlers makes it easy to track changes to the state of controls. For example, this is useful for updating the playback time display at each timeupdate event.



A short list of frequently used events:

Event	Description
durationchange	The duration attribute has been updated.
ended	Playback stopped at the end
pause	Playback was stopped (note the absence of a stop event)
play	File started playing
timeupdate	The current playback position has changed (usually every 250 ms)
volumechange	Value has changed

Tip : Android (at least 2.3) does not always raise the event ended . Well, let's say, he sometimes creates one for decency, but in general, you have to create your own event, capturing the pause event when reaching the end of the media file and comparing audio.currentTime with audio.duration . Or you can dry the pause event and make sure that audio.currentTime is zero. To be absolutely sure, you have to do both.

Some other events:

Event	Description
canplay	The file can be played, but it may need to pause while it is loading.
canplaythrough	With the current download rate, it is assumed that the file can be played from start to finish without interruption.
progress	Browser shows playback status (usually every 250 ms)

Again, HTML5 Media Event Inspector behaves friendly.



You can test browser support using areweplayingyet.org . It is worth looking at the code and tests to understand the mechanism of what is happening.

■ Streaming playback

Streaming audio is a general requirement for media kontorolu. Until recently, Adobe flash technologies dominated this area. Proprietary server technologies are well known. One of the leaders is Adobe's Real Time Messaging Protocol ( RTMP ). However, modern browsers only support streaming on the HTTP protocol, and something like flash requires RTMP streams.



Now browsers, if supported, are only SHOUTcast and Icecast servers that play audio over HTTP. SHOUTcast is proprietary and plays only MP3 and AAC files, while Icecast is open and supports Ogg Vorbis, MP3 and AAC.

Flow	AAC	MP3	Ogg vorbis	Built-in support
Icecast	Yes	Yes	Yes	Yes
SHOUTCast	Yes	Yes	Not	Yes
RTMP	Yes	Yes	Not	No (Flash required)

■ Evolution of specifications (or “Ti Divi! Won’t ruin!”)

As audio specifications evolve, there are several inconsistencies in browser implementations that you should keep in mind. They usually appear in older browsers.

● Load method

Until recently, the load method informed the browser about a media file change and made the control change. Now he resets the item and starts fetching and loading a new resource from scratch.



Thus, for old browsers like Firefox 3.6, in order to change the source of information, you must not only change the value in the method, but also execute the commands:

 var audio = new Audio(); audio.setAttribute("src","mynew.mp3"); audio.load(); //     

● When will browsers return to official documentation?

For cross-browser solutions, it is useful to know which browsers support the W3C documentation and to what extent and how it is deviated from it.

● Autoplay and volume

iOS and Blackberry ignore the autoplay attribute and media element volume changes, and Android supports autoplay , but does not have a volume element.



In fact, autorun is disabled on Blackberry and IOS, since both systems require user permission to "kick" them. This is an unfortunate side effect: any audio playback suffers from the necessary interaction delay.



To smooth out these inconveniences, you can get acquainted with the excellent article of Dr. Remy Sharp about fixing audio sprites in iOS (English) .

● Simultaneous playback of several audio tags

Particularly annoying to the developers of everything (especially games), except for the simplest applications, the inability of some browsers (again we see a couple: iOS and BlackBerry) play several tracks at the same time.

● Dependence on OS

Tip : despite the limited use of this method by developers, you can install other codecs not built into the OS, such as Ogg Vorbis, and browsers that use the codecs of the operating system should start supporting them.

Safari (5+) relies on Quicktime settings . This is rarely a problem when using Windows.



Internet Explorer (9+) uses operating system codecs. Since it only works on Windows, fortunately, this is almost always the case.

■ What's new?

For web audio, some new features and a new API are being refined, so let's look at the near future.

● Change pitch

Note a couple of future functions: playbackRate and defaultPlaybackRate . They change the speed and direction of playback — to fast forward and rewind functions, or perhaps allow users to adjust the listening speed to listen to more podcasts per day.



● audio.playbackRate returns 1 at normal speed and acts as a speed multiplier. For example, setting the playbackRate 2 will double the speed; if you set it to -1, the game will play in the opposite direction.

● audio.defaultPlaybackRate — default media speed after a pause, restart, or an event of this kind.

● Media fragments

If we want to refer to a part of a media file now, we most often have to download at least a part of the file that is not really needed. W3C's proposals for media fragments were created to resolve this issue and others , for example, searching the video area or simply related lyrics (audio tags, captions) tracks. For audio, we can tell by the URI parameters which parts we want to load.

■ Enhanced audio API: future sound in browsers

When it came to advanced audio, Mozilla first introduced the initial experimental Audio Data API for Firefox. This should help work with audio in JavaScript and create a platform for low-level development.



To keep up, Google released the Web Audio API for Chrome and launched it as a W3C proposal. The Web Audio API is a high-level implementation that takes an approach to audio objects as audio nodes and provides many useful functions.



Both implementations reflect the desire of developers to do more with audio — create, manage, analyze, and mix audio on the fly. In fact, we can one day be able to do in the browser what we can do now only in native OS applications.



So, do we want to work with advanced audio in the browser? Well, two separate implementations are already there. If you want to write API-based applications, you need to do third-party cross-libraries to abstract from the differences.



A fairly new Mozilla MediaStream processing API has also emerged , which uses a different approach with the ability to manage audio and video streams at both high and low levels.



In fact, using modern APIs, using the speed of modern JavaScript engines, we can even write decoders for unsupported codecs. Currently we have JSMad MP3 decoder and lossless decoder Alac.js. Apparently, FLAC and AAC in the stream. The hope is that in the future we will not have to worry about the support of formats in the browser.



For those interested in modern Internet audio, I wrote about the differences in approaches and different libraries for cross-browser solutions in the article “ HTML5 Audio API: how low can we go? ” (English) .



The good news is that the interested parties agree, there is communication in the W3C working group on audio . It seems that everything is going to implement the ideas of a single audio API , which is rapidly approaching publication as a web standard.

■ Conclusion

Although the browser implementations of the current HTML5 audio specification are improving, there are still a few problems preventing the creation of a full-fledged cross-browser solution. You need to know the limitations of browsers and platforms such as iOS. Problems hopefully disappear.



There is good news from the front of “advanced audio”: a new standard is being created , we are close to obtaining a single specification for the work of browser creators. At the same time, there are JavaScript libraries that can help bridge the gap between existing implementations.



There are also signs that we can see the end of dependencies of various browsers on codecs. Opera Mobile already supports MP3 formats where the OS supports it, and it looks like Mozilla can afford it too . If this becomes possible for mobile browsers, desktop browsers are likely to follow suit.



Much is changing and much has been achieved. Irregularities of implementations are smoothed out, standards are being adopted, and we see the basis of very interesting new technologies.



The future of online audio looks brilliant!

Additional Fiction (English)

• “Maybe Maybe Not”: the state of HTML5 Audio
• HTML5 Audio Status
• Mastering the HTML5 <audio> tag
• Disclosing the power of HTML5 <audio>
• Everything you wanted to know about HTML5 video and audio