aA
Kaunas University of Technology (KTU), Lithuania students created a mobile app that allows translating sign language characters. The app is also an alternative way of text typing.
Priklausomybė nuo išmaniųjų telefonų
Priklausomybė nuo išmaniųjų telefonų
© Shutterstock

Sign language is expressed through movements and position of hands and other body parts, the University writes. Although hearing people often think that sign language is international, it has different grammar and lexicon in every language. Today’s market lacks systems that could recognise not only the sign alphabet but also other signs characteristic to the sign language.

The first Lithuanian mobile app for sign language translation developed by Lukas Gužauskas, a graduate of the Kaunas University of Technology (KTU) Faculty of Informatics, and Laurynas Bičkauskas, a master’s student at KTU, aims to fill this gap. The app is also an alternative way of text typing.

According to the World Health Organisation, over 5% of the world’s population – around 466 million people – have a hearing disability. It is estimated that by 2050 this number will reach more than 900 million.

A smartphone for an easier communication

The app created by the KTU students allows learning the international sign language alphabet. It has a list of sign language imagery that consists of 29 different letters of the international sign alphabet.

“We plan four basic functions available to the app user: a preview of the international sign language alphabet; a real-time alphabet letter prediction from camera view, recording text on the screen; and human language translation – from audio to on-screen text,” explains Gužauskas, a graduate of the Software Systems study programme.

While using the app, communication between a sign language user and a hearing person will no longer require a piece of paper. The smartphone camera scans the gestures of a person with a hearing disability shown in their native language, and the other person is speaking. The app delivers both ways in a textual format.

Lukas Gužauskas
Lukas Gužauskas
© Organizatorių nuotr.

According to Gužauskas, the app has several settings. The first is the hand detection setting. The user can toggle between left- and right-handed usage. The other is the speed and accuracy of the prediction.

According to the creators of the app, there are open source software based sign language translator versions in the market, however, all of them are designed for computer use. The aim of the team was to design the first app for a readily accessible device, a smartphone.

A challenge turned into an advantage

The communication with a person having a hearing disability can be challenging if one doesn’t know sign language. It is hard to understand for a hearing person because the sign language has a different sentence structure and grammar.

“Problems are common because hearing people often don’t understand what the deaf person writes and vice versa. The sign language cannot be translated literally. Because of the hearing impediment, it can become hard for a deaf person to write a grammatically correct sentence, and hearing readers can get the message wrong. I have noticed that live communication with the hearing can often be too difficult for a deaf person”, – says Gužauskas, who has the hearing disability and knows the problem first-hand.

The student says that deaf people usually have to use the services of the sign language interpreters, but they need to be booked in advance.

Mantas Lukoševičius, a researcher at the KTU Faculty of Informatics and the supervisor of the final project of the students, says that in the process, he had also encountered the communication barrier.

“They have turned a challenge into an advantage: knowing the problem well they created a tool for everyone. It’s great to work with students like Laurynas and Lukas. They went all the way from the idea to the final result. The students have identified and refined the problem they are solving and collected the necessary data – the images of the sign language alphabet. They have also done experiments with modern artificial intelligence methods that best recognise those images, and successfully transferred the trained neural network model into a smartphone”, says Lukoševicius.

DELFI EN
It is forbidden to copy the text of this publication without a written permission from DELFI.
|Populiariausi straipsniai ir video

The Roop gets place in final in next year Eurovision selection

Lithuania will hold a national Eurovision selection next year, and this year's winner The Roop, a...

Sociologist Audronė Telešienė: stay connected and choose reliable information sources

After the World Health Organization (WTO) declared the COVID-19 virus outbreak a pandemic,...

New research suggests climate change could reduce lifespan among hundreds of species

Researchers from Queen’s University Belfast and Tel Aviv University in Israel have carried out one...

Hundreds of fans ask for refund of tickets to Zalgiris' Euroleague game

Several hundred basketball fans have asked for refunds of their tickets to the upcoming Friday game...

Vilnius is 7th most accessible city in Europe

Vilnius came in 7th place beating many other European tourist hotspots.

Top news

President signs into law tougher penalties for breaking quarantine rules

Lithuanian President Gitanas Nauseda has signed into law a legislative package introducing tougher...

811 coronavirus cases in Lithuania

On Sunday morning, there were 811 COVID-19 cases in Lithuania. 10 deaths related to the novel...

Lithuania's Devold says could make 100,000 medical coveralls by late summer

Norwegian-owned Lithuanian textile manufacturer Devold says it could producer around 100,000...

10th coronavirus death reported

Another patient infected with COVID-19 died in Lithuania on Saturday, bringing the country's death...

Lithuania no longer allows cars from Russia, Belarus to enter

A ban for passenger cars to enter Lithuania from Belarus and Russia have come into force on...

|Maža didelių žinių kaina