IBC2023 Tech Papers: A Real-time 8K-60 fps-HDR VVC/H.266 Software Encoder for Next-generation Live Applications

Abstract

VVC/H.266 is the newest video coding standard designed to significantly improve compression efficiency over HEVC/H.265 and provide efficient coding for a broader range of video formats. Live VVC encoding will facilitate the adoption of next-generation applications such as 8K terrestrial broadcasting or 8K adaptive bitrate streaming. This paper aims to provide an analysis of the capabilities of VVC for live applications by describing a real-time 8K VVC software encoder. The encoder has been extensively optimised with advanced mode decision and partitioning algorithms, SIMD instructions, and a scalable multithreading architecture. The proposed encoder achieves 24% bitrate reduction at the same quality, over a highly optimised real-time HEVC encoder that was deployed during the Tokyo 2020 Olympics games. By combining all optimizations and Intel’s 4th generation Xeon® Scalable Processors the new VVC encoder reaches the performance for 8K 60 fps, 10-bit, HDR live encoding at broadcast-grade quality.

Introduction

The 8K-UHD TV format has been designed to provide a stronger sensation of realness and, at the same time, a much more immersive experience in which the user is fully absorbed by the audiovisual content, Sugawara and Masaoka (1). Recent studies have also proven that 8K brings in additional quality benefits, such as the perception of a higher depth of field and 3-dimensionally, resulting in an increased sense of realness and immersion, Park et al. (2), Masaoka et al. (3).

Data storage, transmission, and playback of uncompressed 8K video are complex and expensive. Uncompressed live 8Kp60 (7680x4320px, 10-bit, at 60 fps) video results in a data rate of 48 Gbps. To achieve practical distribution data rates, advanced video coding is needed. The main challenge of an 8K video encoder is, therefore, to compress the video signal in a very high quality to ensure the expected Quality of Experience (QoE) and, at the same time, deliver a relatively low bitrate so that the encoded bitstream can be transmitted over practical distribution networks, for example over the open Internet. Moreover, for live media applications, the encoder must operate in real time and with low latency.

8K live broadcasting and streaming has been tested and deployed in multiple environments and applications. Some highlights include: the 8K satellite channel launched by NHK in Japan, Hara et al. (4); the live 8K OTT streaming pilot over the Internet of the Tokyo 2020 Olympic Games, Maglitta and Velhal (5), and the 8K VR live streaming service of the Beijing 2022 Winter Olympic Games, Koenen et al. (6). In all these cases the High Efficiency Video Coding (HEVC)/H.265 codec has been used for final distribution. NHK published in 2015 the first recommendation for 8K live services to produce broadcast-grade quality and defined a bitrate of 85 Mbps using HEVC, Sugito et al (7). Although lower bitrates have been enabled by more recent HEVC encoders, HEVC is reaching a point of saturation for real-time conditions, which means that adding more computation will result in marginal compression gains.

To further reduce the bandwidth beyond the HEVC capabilities, the latest video coding standard, called Versatile Video Coding (VVC)/H.266, is needed, Bross et al. (8). VVC includes features to provide better compression efficiency for ultra-high-resolution content with extended colour gamut and higher bit-depth support, which make it more suitable for encoding 8K 10-bit HDR video compared to HEVC. VVC also includes coding tools for more efficient Adaptive Bitrate (ABR) streaming and scalable encoding.

Optimised real-time VVC encoders will be essential in the near future to facilitate the deployment of next generation 8K live applications such as 8K terrestrial broadcasting or 8K ABR streaming. Although some 8K VVC live encoders have already been announced, KDDI (9) and Spin Digital (10), these only reach the performance required for live 8K at maximum 30 fps. The first real-time 8Kp60 VVC encoding implementations are also expected to be CPU-based, running on the latest-generation CPU architectures. However, to the best of our knowledge, none has yet been released or demonstrated.