Apple encodes video using JPEG, JSON and

^{File fragment unlock_001.jpg}

Apple has a lot of experience in developing web technologies, but now it’s in an absurd situation. If you go to the page with a description of the design innovations of the iPhone , then you can see the strange thing: the video with the iPhone unlocking animation is encoded with several unusual JPEG files.

The question arises: how does it work? And why resort to such tricks if you can use the standard <video> tag.

The explanation is simple. It turns out that the <video> tag on the iPhone only works in full-screen mode, so Apple cannot use it to insert videos on the page. In addition, Apple uses only H.264 video, and not all browsers support it, which is unacceptable for a promotion page.
')
Previously, Apple solved this problem by creating a JPEG for each frame of animation, resulting in a two-second animation took 5 MB. Now they have implemented the optimization: the iPhone 5 animation with a volume of just one megabyte consists of five files:

www.apple.com/iphone/design/images/unlock/unlock_manifest.json
www.apple.com/iphone/design/images/unlock/unlock_keyframe.jpg
www.apple.com/iphone/design/images/unlock/unlock_001.jpg
www.apple.com/iphone/design/images/unlock/unlock_002.jpg
www.apple.com/iphone/design/images/unlock/unlock_endframe.jpg

The operation algorithm is described in the ac_flow.js file and is based on the fact that only part of the image is updated to save traffic. The unlock_001.jpg and unlock_002.jpg contain these updated image fragments. The unlock_manifest.json file contains coordinates to position the parts. To prevent the JPEG artifacts of individual fragments from spoiling the big picture, Apple uses blocks of 8x8 pixels in size, as does the JPEG codec. The above files unlock_001.jpg and unlock_002.jpg are actually used as a continuous stream of 8x8 blocks. Instructions for stream processing are specified separately in base64 format, which has to be decoded:



Each instruction consists of five bytes, the first three bytes indicate the coordinates of the target on canvas, and the last two bytes indicate how many blocks to place there. For example, an AAxAC instruction means that you need to take two blocks (AC) and place them at position 49 (AAx).

With such a video assembly algorithm, there is no possibility of re-using blocks, but the manifest itself turns out to be quite laconic.

via dbloom