IBC2023 Tech Papers: AI Image Analysis in Era of Short-time Viewing

Abstract

In the era when short videos are preferred, broadcasting stations have been enhancing momentums to distribute summary videos of broadcast content on social networking services (SNS). Therefore, we have developed automatic generation systems for news and programme summary videos. Using a video summarisation artificial intelligence (AI) that has learned the image composition and camerawork typical of important scenes, it is possible to automatically generate summary videos with a high quality close to videos edited by actual programme production staff. These systems have been on trial/practical use in various NHK broadcasting stations. The generated summary videos are posted daily on SNS. Furthermore, considering a programme website is also important content that could boost viewer contact rates, we developed a support system for programme website creation using an AI to extract thumbnails automatically. Using thumbnail candidate images extracted automatically by AI that has learned the unique features of a programme’s representative images, you can create programme websites with minimal effort. These technologies can streamline the production of Internet-content such as summary videos and programme websites. Moreover, they will greatly boost Internet deployments of various broadcasting programmes.

Introduction

With the consumption of short video clips on the Internet increasing, broadcasters are attempting to boost user contact with their content by producing summary videos and distributing them on the Internet. Editorial operations to create summary videos require certain specialities and high work costs. For content that is updated daily, such as news programmes, automation of the summary video production process is especially desirable. Therefore, we are developing automatic video summarisation technologies.

In this paper, we introduce a system that automates the production of news summary videos. Our automatic video summarisation technology has the following features. In news footage, the importance of shots is strongly correlated with image composition and camerawork, such as zooming in on important people, panning to show items of interest in detail, and special shooting angles for buildings involved in incidents. We trained an artificial intelligence (AI) system to learn picture-making such as the image composition and camerawork typical of summary videos produced by skilled editors. Moreover, our technology used the similarity of keywords with an anchor’s introduction speech to evaluate the importance of each video segment.

This system summarises 15–30-minute news programmes into about 1–2 minutes, with about 20 summary videos being produced and distributed each day. Summary videos that used to take a skilled editor more than one hour to produce can now be generated about 10–20 minutes after the end of the broadcast programme, making it possible to produce and distribute summary videos without losing the immediacy of the news.

With regard to common programmes not including an anchor’s introduction, we developed a practical system to generate summary videos automatically. As with the news video summarisation system, the AI on this system learned picture-making typical of important scenes using summary videos created by professional video production staff. This system has also been on trial use in a number of NHK regional broadcast stations to post summary videos on social networking services (SNS).

Along with summary videos, attractive thumbnails on programme websites can boost the number of accesses to broadcast contents. An attractive thumbnail contains an eye-catching image that is representative of the video. We developed a support system for programme website creation using AI to select attractive thumbnails from videos. This AI was generated by learning the composition of good thumbnails and their suitability as representatives of the video.

The summary videos and programme websites could lead users to broadcast content, so such technologies are expected to provide a bridge between the Internet and broadcasting.