IBC2023 Tech Papers: Optimizing the Virtual Reality experience using a 360° view of client and network data

Abstract

Virtual Reality (VR), and other forms of Extended Reality (XR), are beginning to drive deep immersive experiences for premium entertainment and sports content. In many cases, data is streamed directly to a headset, requiring the underlying network to accommodate constant throughput of over 25Mbps to support UHD 4K and over 50Mbps for UHD 8K. However, these more demanding video requirements are at odds with networks that are already congested and highly volatile. Granted, networks today have a glut of capacity on both wireline and mobile 5G networks, but available capacity along a delivery path may not be equitably shared across users, and can also drastically change over time and thereby stand in the way of smooth, high-throughput delivery.

Introduction

Virtual and Mixed Reality are beginning to drive deep immersive experiences for premium entertainment including sports content. These new formats have unlocked additional ways to consume rich 360° and VR content by streaming it directly to a headset, allowing the complete virtual reconstruction of a venue experience. Users may select any seat in the house and control what they want to see, and how they want to see it. Mixed Reality further enhances this experience by overlaying supplementary data, statistics, deep insights, and dynamic computer-generated objects that are contextual to the content stream. MR adds another layer of information, one that augments the action on the field, enhances a player’s profile or provides additional information about the venue itself. It also allows content creators to connect the traditional streaming experience with the immersive on-line gaming culture.

COVID-19 also drove massive change as sports and entertainment transitioned from social activities to a virtual singularity. Leagues and venues began experimenting with ways to engage viewers in a more immersive way both at home, and in-person. Although the methods and technology have been evolving the last few years (i.e., minting NFTs during games, real-time stat overlays, multi-viewers), the constant is now that viewers expect an additional interactive dynamic engagement, and an event alone is no longer enough.