IBC2022 Tech Papers: Exploring the benefits of Dynamic Resolution Encoding and support in DVB standards

Abstract

In 2015, Netflix introduced Per-Title Encoding Optimisation, defining the best resolution for a title at a given bitrate. Since then, Netflix has improved its algorithms to make the technique more dynamic through scenes and has deployed it for VOD applications to enhance bandwidth use and video quality. Thanks to recent developments in AI, Dynamic Resolution Encoding (DRE) can now be applied to live applications with very limited extra CPU consumption. This paper will explain why DRE is not a feature limited to new codecs, such as VVC or AV1 and can also be applied to legacy codecs, like AVC or HEVC, for streaming applications. It will present concrete results of experiments that apply DRE on top of the AVC, HEVC or VVC codec for broadcast (TS) and broadband (DASH/HLS), examining how DRE can leverage the compression power of such a codec to reduce HD or UHD 4K bandwidth, improve video quality, and reduce power consumption. In addition, the paper will report on the interoperability results with DVB-T2 UHD TVs as well as with DVB DASH players. It will conclude with the latest development of DVB standards to support DRE.

Introduction

Video compression experts all know that when bandwidth is reduced, a good trade-off to preserve quality and limit visible compression artifacts is to reduce the resolution. Of course, the best resolution for a given bitrate highly depends on the video content. In December 2015, Netflix popularised the concept of variable resolution encoding, in its blog “Per-Title Encoding Optimisation”. At the time, the best resolution was selected for each VOD content. In the following years, Netflix improved the concept with Dynamic Resolution Selection, applied for each scene. As Netflix’s market is VOD, this selection can be made offline, but a viewing-based selection would be much too time consuming. To mitigate this issue, Netflix developed an objective video quality (VQ) metric, called VMAF, to help in the resolution selection process automation. More recently, Fraunhofer FOKUS made use of AI to build optimised encoding ladders per scene, leading to significant bandwidth and storage savings.

Harmonic has implemented a similar concept on live content with a very dynamic selection, applied for each video delivery segment ofa few seconds duration. The first section explains how the Dynamic Resolution Encoding can work in a live workflow. The second section presents the possible use cases and why its application is not limited to new codecs. The third section presents concrete results of experiments that apply DRE on top of the AVC,HEVC or VVC codec for broadcast (TS) and broadband (DASHor HLS). The next section goes through the results of interoperability evaluation with DVB-T2 UHD TVs as well as with DVB DASH players. The last paragraph concludes with the latest development of DVB standards to support DRE.