IBC2022 Tech Papers: Providing immersive sign language education program utilising android TV platform and AI technologies

Abstract

More than 90% of deaf children are born into hearing families. Unless the family learns sign language together, deaf children easily become isolated and may receive inadequate home education. This is the first social barrier deaf children must face.

To solve the above social problem, EQ4ALL designed a sign language education application for Google’s Android TV platform in partnership with KT skylife, leading IP converged direct to home (DTH) broadcaster.

To provide animmersive education experience for both hearing and deaf family members, a web cam connected to a set-top-box is used to capture user signing with machine vision technology while a remote controller’s microphone is used for voice-based dictionary.

The project shows that pay TV operators can use their set-top-box platform for environmental, social, and governance (ESG) activities.

Introduction

Excluding age-related deaf and hard of hearing people as most of them are not registered, sign language is primary language for deaf and hard of hearing (DHH) people. According to a 2017 survey by the Korean Ministry of Culture, 65.9% of registered DHH people in Korea use Korean sign language as their primary language while 69.5% of them have difficulty using Korean.

There are 410 thousand registered DHH people in Korea while 466 million DHH globally. According to the World Health Organization (WHO), this will increase to 9 million DHH people by the year 2050.

DHH people suffer from inequal opportunities throughout their whole life. 

Among these 5 areas of inequality, we focused on family communication as this is the first barrier for deaf children. More than 90% of deaf children are born into hearing families unless the whole family learn sign language, the deaf child will be in an isolated environment and receive an inadequate home education.

It was shown that the age of first language acquisition strongly affects second language. It means that the earlier deaf children learn sign language, the more chance they are becoming proficient in verbal language (Rachel I Mayberry, 2007).

An analysisof SAT-10 reading comprehension results showed that sign language fluent deaf students, mostly early sign language learners, had significantly higher scores than students without sign language proficiency (Iva Hrastinski& Ronnie B. Wilbur, 2016) which means early adoption of sign language leads to better academic achievement.

To solve the social problem above, we designed sign language education application for the Android TV platform.