Audio/Video Updates in Chrome 70

François Beaufort
François Beaufort
GitHub

Watch video using Picture-in-Picture

Picture-in-Picture (PiP) allows users to watch videos in a floating window (always on top of other windows) so they can keep an eye on what they’re watching while interacting with other sites, or applications. With the new Picture-in-Picture Web API, you can initiate and control Picture-in-Picture for videos on your website.

Read our article to learn all about it.

AV1 decoder

AV1 is a next generation codec developed by the Alliance for Open Media. AV1 improves compression efficiency by greater than 30% over the current state-of-the-art video codec, VP9. Chrome 70 adds an AV1 decoder to Chrome Desktop x86-64 based on the official bitstream specification. At this time, support is limited to “Main” profile 0 and does not include encoding capabilities. The supported container is MP4 (ISO-BMFF) (see From raw video to web ready for a brief explanation of containers).

To try AV1:

  • Go to the YouTube TestTube page.
  • Select "Prefer AV1 for SD" or "Always Prefer AV1" to get the desired AV1 resolution. Note that at higher resolutions, AV1 is more likely to experience playback performance issues on some devices.
  • Try playing YouTube clips from the AV1 Beta Launch Playlist.
  • Confirm the codec av01 in "Stats for nerds".
Stats for nerds featuring AV1 in YouTube.
Figure 1. Stats for nerds featuring AV1 in YouTube.

Support for codec and container switching in MSE

Chrome is adding support for improved cross-codec or cross-bytestream transitions in Media Source Extensions playback using a new changeType() method on SourceBuffer. It allows the type of media bytes appended to the SourceBuffer to be changed afterwards.

The current version of MSE supports adaptive playback of media; however adaptation requires that any media appended to a SourceBuffer must conform to the MIME type provided when initially creating the SourceBuffer via MediaSource.addSourceBuffer(type). Codecs from that type and any previously parsed initialization segments must remain the same throughout. This means the website has to take explicit steps to accomplish codec or bytestream switching (by using multiple media elements or SourceBuffer tracks and switching among those), increasing application complexity and user-visible latency. (Such transitions require the web app to take synchronous action on the renderer main thread). This transition latency impairs the smoothness of media playback across transitions.

With its new changeType() method, a SourceBuffer can buffer and support playback across different bytestream formats and codecs. This new method retains previously buffered media, modulo future MSE coded frame eviction or removal, and leverages the splicing and buffering logic in the existing MSE coded frame processing algorithm.

Here's how to use the changeType() method:

    const sourceBuffer = myMediaSource.addSourceBuffer('video/webm; codecs="opus, vp09.00.10.08"');
    sourceBuffer.appendBuffer(someWebmOpusVP9Data);

    // Later on...
    if ('changeType' in sourceBuffer) {
      // Change source buffer type and append new data.
      sourceBuffer.changeType('video/mp4; codecs="mp4a.40.5, avc1.4d001e"');
      sourceBuffer.appendBuffer(someMp4AacAvcData);
    }

As expected, if the passed type is not supported by the browser, this method throws a NotSupportedError exception.

Check out the sample to play with cross-codec and cross-bytestream buffering and playback of an audio element.

Intent to Ship | Chromestatus Tracker | Chromium Bug

Opus in MP4 for MSE

The open and highly versatile audio codec Opus has been supported in the <audio> and <video> elements since Chrome 33. Opus in ISO-BMFF support (aka Opus in MP4) was added after. And now Opus in MP4 is available in Chrome 70 for Media Source Extensions.

Here's how you can detect if Opus in MP4 is supported for MSE:

if (MediaSource.isTypeSupported('audio/mp4; codecs="opus"')) {
  // TODO: Fetch data and feed it to a media source.
}

If you want to see a full example, check out our official sample.

Due to lack of tools to mux Opus in MP4 with correct end trimming and preskip values, if such precision is important to you, you'll need to use SourceBuffer.appendWindow{Start,End} and SourceBuffer.timestampOffset in Chrome to obtain sample-accurate playback.

Intent to Ship | Chromestatus Tracker | Chromium Bug

Allow protected content playback by default on Android

In Chrome 70 for Android, the default value of the “protected content” site setting changes from “Ask first” to “Allowed”, lowering the friction associated with playback of such media. This change is possible, in part, because of additional steps taken to clear media licenses alongside cookies and site data, ensuring that media licenses are not used by sites to track users who have cleared browsing data.

Protected content setting in Android.
Figure 2. Protected content setting in Android.