SMART WITH SIGN LANGUAGE MOBILE APPLICATION

AFIFAH BINTI AWANG ABDUL RAHMAN

Bachelor of Information Technology (Informatics Media) with Honours

Faculty of Informatics and Computing

Universiti Sultan Zainal Abidin, Terengganu, Malaysia

2018

ii  

DECLARATION

I hereby declare that this report is based on my original work except for quotations and

citations, which have been duly acknowledged. I also declare that it has not been previously

or concurrently submitted for any other degree at Universiti Sultan Zainal Abidin or other

institutions.

_________________________________

Name : Afifah bt Awang Abdul Rahman

Date : ………………………………....

   

iii  

CONFIRMATION

This project report entitled Smart with Sign Language Mobile Application was prepared

and submitted by Afifah Binti Awang Abdul Rahman, matric number BTDL16043817 and

in my point of view, this project fulfils a condition to be awarded a Bachelor of Information

Technology (Informatics Media) with Honours in Universiti Sultan Zainal Abidin

(UniSZA).

___________________________________

Supervisor : Mr Mohd Sufian bin Mat Deris

Date : ………………………..……..

iv  

DEDICATION

In the name of Allah, I dedicate this project to Him, the Most Gracious and the Most

Merciful, all praise is only for Him. He has been the source of my strength throughout this

project. Never forget to my very kind and helpful supervisor, Mr Mohd Sufian bin Mat

Deris for the valuable ideas, time, support, advice and guidance given through the

development until completion part of the project. I also dedicate this work to my beloved

family that supported and encouraged me all the way and make sure that I give it all to

finish what I have started. Next, I want to thank all my friends that willing to help me finish

this project. Lastly, a thousand thank you to everyone who directly or indirectly involved

in the process of making this project and documentation.

v  

ABSTRACT

The deaf community is considered a minority in Malaysia. Majority of people are

not familiar sign language used by deaf community as they are not exposed to this at an

early age. This limits their ability to interact with majority of the population, which may

cause problems especially when emergency arises. Smart with Sign Language mobile

application is a platform for our targeted users to be able to learn about sign language

interactively. This application uses Android-based system where user can download easily

using their mobile phone. The user interface contains 2D animation which is very fun for

our targeted users from the age of 10 to 17 years old. As a result, users can learn sign

language while enjoying games and quizzes provided inside the app. Hence, learning sign

language using a fun technique is very effective to attract children of all ages.

vi  

ABSTRAK

Komuniti bisu pekak boleh dikatakan sebagai golongan minoriti di Malaysia.

Kebanyakan masyarakat tidak tahu akan Bahasa isyarat yang digunakan oleh golongan

bisu pekak adalah kerana mereka tidak didedahkan tentang perkara ini seawal usia kanak-

kanak. Hal ini telah membataskan kebolehan perhubungan dengan masyarakat lain

dimana akan menimbulkan masalah yang besar terutamanya apabila berlaku kecemasan.

Aplikasi mudah alih Smart with Sign Language dijadikan sebagai satu platfom untuk

pengguna agar dapat belajar tentang Bahasa isyarat secara interaktif. Aplikasi ini

menggunakan sistem dasar Android dimana pengguna boleh muat turun dengan mudah

menggunakan telefon pintar. Paparan antaramuka pengguna menggunakan konsep

animasi 2D yang mana ia sangat menarik untuk sasaran pengguna bermula umur 10

hingga 17 tahun. Sebagai hasilnya, pengguna boleh belajar Bahasa isyarat sambil boleh

bermain permainan dan kuiz yang disediakan didalam aplikasi tersebut. Maka, belajar

Bahasa isyarat menggunakan teknik yang menarik adalah salah satu cara yang berkesan

untuk menarik perhatian kanak-kanak dipelbagai peringkat umur.

vii  

CONTENTS

 DECLARATION  ii 

CONFIRMATION  iii 

DEDICATION  iv 

ABSTRACT  v 

ABSTRAK  vi 

CONTENTS  vii 

LIST OF TABLES  ix 

LIST OF FIGURES  x 

LIST OF ABBREVIATIONS  xi 

LIST OF APPENDICES  xii 

 

CHAPTER 1 INTRODUCTION  1 

1.1  Background  1 

1.2  Problem Statement  3 

1.3  Objectives  4 

1.4  Scope  4 

1.5  Milestones and Activities  5 

1.6  Limitation of Work  6 

1.7  Expected outcomes  6 

 

CHAPTER II LITERATURE REVIEW  7 

2.1  Introduction  7 

2.1.1  Sign language  7 

2.1.2  Mobile Application  8 

2.1.3  Smart with Sign Language Application  8 

2.2  Related Application  9 

2.2.1  Existing Application (Learn to Sign)  9 

2.2.2  Existing Application (Eddy: Digital Learning of Sign Language)  11 

2.2.3  Existing Application (MyBim)  12 

2.2.4  Existing Application (V2M)  14 

2.3  Summary  15 

viii  

CHAPTER III METHODOLOGY  16 

3.1   ADDIE Model  16 

2.1  Analysis Phase  17 

3.3   Design Phase  19 

3.3.1  Storyboard & users’ interface  21 

3.4  Development Phase  26 

3.4.1  Tools  27 

3.4.2  Review  27 

3.4.3  Testing  27 

3.5  Implementation Phase  28 

3.6  Evaluation Phase  28 

3.6.1  Formative Evaluation  28 

3.6.2  Summative Evaluation  29 

3.7  Evaluation as The Final ADDIE Stage  30 

 

CHAPTER IV  31 

 

REFERENCES  32 

 

ix  

LIST OF TABLES  

 

Table 1.1 Gantt Chart 5 

x  

LIST OF FIGURES  

Figure 2.1 Learn to Sign 10 

Figure 2.2 Eddy Application 12 

Figure 2.3. MyBim Application 13 

Figure 2.4 V2M 15 

Figure 3.1. A diagram of ADDIE model components 17 

Figure 3.2. Lesson Plan 20 

Figure 3.3. Navigation Map 20 

xi  

LIST OF ABBREVIATIONS

MSL Malaysian Sign Language

xii  

LIST OF APPENDICES

APPENDIX TITLE PAGE

1  

CHAPTER 1  

 

INTRODUCTION

1.1 Background

As time pass by, we cannot deny that smartphone is becoming the most important

thing in our life. It is used mostly anywhere and at any time in our daily routine whether

it’s for communication, entertainment, work or other purposes. With this pattern of

behavior in using mobile application, we can see clearly that mobile application can go far

into the future.

The type of people who like to think over the border and beyond will take this

opportunity and will strive to make it as their income by developing useful mobile

application for users. One of the useful applications we can see nowadays is a sign language

application which helps people a lot in communicating among each other. There are tons

of these application in the Google Play Store available for user to download with multiple

functionalities.

For most of the time, common problem that people has been facing is their

communication barrier with deaf people. This is crucial because deaf people look like

2  

ordinary people except, they cannot speak nor hear. There will be scenarios where they

needed help with something basic like asking for directions, but we cannot help because

the lack of knowledge in sign language. Knowing the basics in sign language proves to be

crucial in a bigger scenario such as in a life-threatening emergency where there is one

normal person and one deaf person who happen to know how to speak sign language only.

Imagine if we are in that situation whether we are the one who needed help or vice versa.

It is almost impossible for them to understand each other in such intense situation.

Smart in Sign Language Mobile Application is no doubt the solution to the

problem above. The idea of interactive 2D animation tutor and user-friendly education

platform will definitely attract users to learn more about sign language. It is very efficient

and easy to carry around for the fact that it is in our mobile phone, so users can learn

basically anywhere as long as they have their phone.

 

   

3  

1.2 Problem Statement

It is undeniable that there is plenty of sign language application specifically

designed to help users learn this language. But most of them lack in interface design and

the knowledge delivery itself is unclear. Even though it is useful and functional, but this

kind of application is unpractical anymore because it cannot effectively help users fully

understand sign language. The main purpose of problem statement in this section is to focus

on the solution to the problems.

There are currently three main problems. Firstly, most people in Malaysia does

not understand sign language. This is probably because of the individual have no interest

in learning sign language due to the mindset that sign language is not important. Secondly,

they do not know how to reply to the conversation in sign language. Most of the schools

and institutions in Malaysia do not have sign language subject in their syllabus which leads

to decreasing the number of people who know about sign language. Thirdly, not many

people are interested in learning sign language with the existing application especially

teenagers and young kids. It is especially difficult to expose sign language to our kids at

young age without an interesting method of learning them. Hence, only have a few

percentages of Malaysian can speak and understand sign language.

4  

1.3 Objectives

‐ To design an improved version of Malaysian sign language application for much

effective learning.

‐ To develop a kid’s and teenager’s friendly application as a platform for them to

learn the basics of Malaysian Sign Language.

‐ To test the functionality and effectiveness of Smart with Sign Language Mobile

Application for users to use and learn for easier communication among normal

person and hearing-impaired person.

 

1.4 Scope

The scopes of this study are divided into:

i) Admin. The role of an admin is to login the application to make changes,

updates and maintenance.

ii) User. This application is targeted for users at the range age of 10 years old

until 17 years old. Users can use this mobile application to learn the basics

of MSL. Users also be able to play games and quizzes provided in the

application to test their knowledge of what they have learned previously.

iii) Android Based. This application will be developed for android operating

system’s mobile devices.

 

 

5  

1.5 Milestones and Activities

Table 1.1 Gantt Chart

Week 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

Topic Discussion and Determination

Project Title Proposal

Proposal Writing ‐ Introduction

Proposal Writing ‐ Literature

Review

Proposal Progress ‐ Presentation &

Evaluation

Discussion & Correction Proposal & Proposed Solution Methodology

Proposed Solution Methodology (continued)

Proof of Concept

Drafting Report of Proposal

Submit Draft of Report to Supervisor

Seminar Presentation

Correction Report

Final Report Submission

6  

1.6 Limitation of Work  

‐ Users can easily learn and improve their MSL skills from their mobile devices.

‐ The application will provide variety of words like objects, color, numbers and

other basic words as well.

‐ In further level, users can start learning about phrases and basic sentences in

daily communication

 

1.7 Expected outcomes  

‐ Become a platform for kids and teenagers to learn about Malaysian sign

language wherever they go.

‐ Making kids and teenagers more interested in learning MSL because of our fun

interface.

 

   

7  

CHAPTER II

LITERATURE REVIEW

 

2.1 Introduction  

This section focuses on the work or research that whether it is directly or indirectly

related to the project itself. It contains many information that gives such benefit to the

process of development of the project which makes it very important.

 

2.1.1 Sign language  

Sign language is a visual language which uses hands and arms gesture, facial

expression and body language to communicate. This language is used by hearing and

speech impaired person, and also by normal people which maybe have family members or

friends that is needed to communicate in this language. Sign language developed and

existed remarkably in many types of language worldwide. Each country has its own native

sign language, and perhaps some have more than one language. Some that we familiar with

are American Sign Language, British Sign Language, European Sign Language and many

others. Each has its own differences and uniqueness. But this project will be focusing

mainly in Malaysian Sign Language which not many people internationally knows about it

yet.

8  

2.1.2 Mobile Application  

Mobile application (mobile apps) is usually refers to an app, that is a software

specifically designed for mobile devices such as smart phone or tablet computer1. It is

similar to services that users can accessed on PCs. Apps are generally small and most of

them have limited function. A mobile application is also known as an app, web app, online

app, iPhone app or smartphone app.

 

2.1.3 Smart with Sign Language Application

This application act as a platform for users to learn interactively about Malaysian

Sign Language which is developed and designed specifically for young kids and teenagers

from the age of 10 to 17 years old by making the interface and content itself much

interesting than the ones that already existed. This is to make sure that this circle of people

will get attracted by this apps so that they can learn MSL effectively.

   

                                                            1 https://www.techopedia.com/definition/2953/mobile‐application‐mobile‐app 

9  

 

2.2 Related Application  

2.2.1 Existing Application (Learn to Sign)

‘Communication is key’ a phrase known worldwide, it is how people exchange

ideas, knowledge, feelings, thoughts and much more. Communication between people

comes in all different forms: verbal, visual or electronic to name a few. For many choosing

which form of communication, they wish to communicate in is an option. However, when

someone suffers from hearing loss, be it since birth, an accident or due to old age, they tend

to lose the option to choose. Depending on where you live in the world, the support and

care that is available to those who are deaf can vary greatly. For example, deaf people in

the developed areas of Europe, North America and Oceania have greater opportunities

through language accessibility, interpreting services and disability rights.

Furthermore, to developed countries, when the beliefs around deafness are that of

acceptance and protection deaf individuals could feel part of the community and have equal

rights as hearing individuals. Some deaf children say “they were raised on equal footing

with their hearing siblings…”2. Whereas on the other end of the spectrum, the negative

connotation, deaf varies from being a form of punishment from God, a bad omen, familial

shame, act of fate and/or burdensome and can lead to neglecting, abusing, isolating and

even abandoning the deaf individual. Because of that, the Learn to Sign development team

have developed an app which will assist hearing and deaf individuals in communicating

                                                            2  http://www.deafchildworldwide.info/ 

10  

with each other while raising awareness on deafness to change the attitudes of many

countries due to varying beliefs around deafness. They see that deaf children always get

abuse from the community or being in total silence because they have no one to

communicate with. Miscommunication between deaf people and normal people because

zero knowledge in communicating in sign language.

Overall, this educational mobile application helps the hearing individuals as well

as deaf individuals to learn basic signs and phrases through means of a game. The

usefulness and value of this application was tested amongst basic users and experts in Sign

Language, whereby each iteration improved on the next based on the feedback and results.

 

 

Figure 2.1 Learn to Sign

 

11  

2.2.2 Existing Application (Eddy: Digital Learning of Sign Language)

Eddy Digital Learning of Sign Language is a mobile application that helps users

to learn basic Malaysian Sign Language (MSL) in more interactive way and on the go. Plus,

this app contains exciting gaming scenes, 3 dimensional animated MSL dictionary and

MSL spelling features to make it easy for everyone to learn. This application is on-demand,

trusted and integrated source for learning Malaysian Sign Language that hopes to bridge

the communication gap.

Users can easily learn MSL using gaming method which proven to be very

effective way of learning. Users can choose from multiple categories on which category to

learn first. All the animation and interface are very attractive an interesting. The quizzes

are also fun because they included the sound and movement where for users especially kids

seem to be very interested in those sorts of thing.

As a conclusion, Eddy Digital Learning of Sign Language is an interactive and fun

application to learn sign language for beginners. But there are quite a few glitches in the

game. They keep repeating the same emotion over and over. The size of the application

itself is also ridiculously big for a learning mobile application. Other than that, this app is

a great mobile app to use.

12  

 

Figure 2.2 Eddy Application  

 

2.2.3 Existing Application (MyBim)

MyBim is an educational mobile application that promote academic knowledge

about deaf community and the Malaysian Sign Language as the official sign language in

Malaysia. Their job is to contribute the voice and ideas in language and linguistic issues

and to carry out public advocacy. MyBim development team also organize activities and

functions to build public knowledge on the deaf community and the Malaysian Sign

Language. Their mobile application is basically same with other sign language application

which is contains several categories of sign language learning. Users can watch videos

provided to see how the movement of the certain words in sign language. They also have

extra features that is latest news, gallery, map and enquiry section.

13  

To summarize, majority of users says that it is a good application but there is some

comment that wanted this application to be offline. Furthermore, they also need to improve

on the user interface and make it more appealing. Other than that, this is application is very

useful for individuals trying to simply learn the basics of Malaysian Sign Language on their

mobile devices.

Figure 2.3. MyBim Application

14  

2.2.4 Existing Application (V2M)

V2M is an application which is very helpful for the users to communicate among

each other in real time using this platform. Their team developed a mobile application that

uses disrupted speech of Deaf-mutes for the purposes of social connectivity. This is because

all the existing mobile application uses sign language as the only option for communication

with them.

This team developed an Android based application named as vocalizer to mute

(V2M) which act as an interpreter and encourages two-way communication between Deaf-

Mute and normal person. Normal to Deaf-Mute person communicate using the module

which takes text or spoken message of a normal person as an input and output a 3D avatar

that performs sign language for a Deaf-mute person. Meanwhile for Deaf-Mute person to

communicate with normal person, the application uses disrupted speech of a Deaf-Mute

person and converted into recognizable speech format by using speech recognition system.

15  

 

Figure 2.4 V2M

 

 

2.3 Summary

As discussed above, all existing applications have their own strength and abilities,

but there are still some weaknesses can be seen. If there are a little tweak and changes made

to the existing application, it can improve a lot of things and functions by developing new

application.

   

16  

CHAPTER III

METHODOLOGY

3.1 ADDIE Model    

ADDIE model is one of the most common used models in the instructional design

field as a guide to produce an effective design. This model is an approach that helps

instructional designer or content developer in developing an instructional product. In fact,

the elements made through the ADDIE model can be used in any online or face-to-face

environment. Hence, ADDIE is used to develop Smart with Sign Language Mobile

Application. The acronym ADDIE stands for important components in the process of

creating an instructional design. These components are Analysis, Design, Development,

Implementation, and Evaluation. Each phase in an ADDIE model is related to and interacts

with each other. Figure 3.1 below shows the scope of this model in diagram form.

   

17  

Figure 3.1. A diagram of ADDIE model components

 

 

2.1 Analysis Phase

Analysis stage is the most important phase in this process. When instructional

designers do the analysis phase before creating, developing, and implementing a plan, they

will save a lot of courses, effort, and time. In order to carry out the analysis phase for the

Smart with Sign Language Mobile Application, we looked into a few things such as the

learners (where they are at, their skills and needs, etc.), develop an instructional analysis

(to provide the necessary steps of learning sign language and present opportunities to learn

and use new information in an instruction), create instructional goals (aimed at specifying

18  

the end desired result), and analysis of the learning objectives (how to measure the

attainment of goals).

This mobile application project focuses on children from the age of 10 to 17 years

old as the target users. We need to know the needs and problems of the users prior to

building the application. We need to know the users’ initial knowledge base of sign

language, so we can plan the application development on the basis of how much they

already know what they need to learn.

Based on my surveys and interviews with respondents, I can conclude that most

people do not have the basic knowledge of sign language. Even for those who have friends

and family that is deaf-mute, they only communicate with their own way, not using the sign

language. Even in the deaf-mute community, not all of them is able to learn and

communicate using sign language. This is part of the problem that hinders their friends and

family to communicate with them as well.

As for this sign language mobile application, it will be presented in English and

no internet connection is required for this application. In the end result of our project, we

aim to enable both normal and deaf-mute people to learn sign language effectively

starting from the basic alphabets, phrases and daily sentences through 2D animation that

will demonstrate the gestures. 

 

19  

3.3 Design Phase

The design phase is the next step in ADDIE model. This phase is when the work

flow, program animation design, user interface and the overall application development are

initiated. In the lesson plan, users will first go through all the alphabets, followed by basic

words and finally phrases and daily sentences (Figure 3.2). There is also a game section

provided for users to practice their knowledge (Figure 3.3). This sign language mobile

application is mostly developed in Adobe Flash CS6 where the 2D cartoon avatar is created

and animated.

Figure 3.2. Lesson Plan

HOME 

ALPHABET BASIC WORDS  PHRASES  DAILY SENTENCES 

COLOUR 

OBJECTS 

ACTIONS 

FEELINGS 

20  

 

Figure 3.3. Navigation Map

  One of the features that will be applied is by making the camera view

available in two perspective that is from the front and from the side so that users can see

exactly how the hand movement should be like. The repeat button is also available for the

users to click on as many times as needed to repeat the sequence until they memorize it.

21  

3.3.1 Storyboard & users’ interface  

Loading page when opening application

Pop-up will appear to ask for name to display in score later.

   

22  

This is the main menu interface

When clicked on “LEARN” button

When clicked on “ALPHABETS” button in LEARN menu

When clicked on “A” option in ALPHABETS section

23  

When clicked on “BASIC WORDS” button in LEARN menu

When clicked on “PHRASES” button in LEARN menu

When clicked on “DAILY SENTENCES” button in LEARN menu

When clicked in one of the “DAILY SENTENCES” options

24  

When clicked on “QUIZZES” button in Main Menu

When clicked on “SPELL ME” option in QUIZZES menu

When clicked on “GUESS WHAT?” option in QUIZZES menu

Pop-up that will appear every time user complete any of the game stage

25  

When clicked on “FINGER SEEK” option in QUIZZES menu

Pop-up that will appear every time user

complete this game stage

When clicked on “Display Score” button in QUIZZES menu

When clicked on “CREDITS” button in Main Menu

26  

This pop-up will appear when clicked the exit

button on top right corner

3.4 Development Phase

This phase depends on the first two phases, which are the analysis and the design

phase. If these phases are executed correctly, the development will be easier. The third

phase is where the developers create and assemble the content assets that were created

before. The prototype is developed at this stage, followed by developing course materials,

review and pilot session.

27  

3.4.1 Tools  

E-learning Tools (i.e., Flash, Media software)

Graphic Tools (i.e., Photoshop, Illustrator)

Server Tools (i.e., Flash Media Server, Android AIR 3.0)

 

3.4.2 Review

Review is a critical factor in the development phase. This is to make sure that there

is no misspelled words, incomplete sentences, and inappropriate images. All the links and

buttons are also checked to ensure that they can function properly. These items can be a

distracter in the learning process and can impact the integrity of this mobile application.

3.4.3 Testing

This sign language mobile application will be tested by random users to get instant

feedback so that it can be perfected through time. One of the main reasons is to make sure

the objectives are achieved.

28  

3.5 Implementation Phase

Once the Development stage is completed, we will proceed to the Implementation

phase. In this phase, the materials created during the development are introduced to the

target audience and the learning process starts. Users acquire knowledge autonomously by

going through the sign language lesson plan in the mobile application platform and trying

to understand the material with no outside help. Data gathering during the Analysis stage

really helps in defining what should be added in these circumstances.

3.6 Evaluation Phase

The final stage of the ADDIE framework is Evaluation. It consists of two parts:

formative and summative. Formative evaluation is presented in each stage of the ADDIE

process. Summative evaluation consists of tests designed for domain specific criterion-

related referenced items and providing opportunities for feedback from the users.

Revisions can be made as necessary.

3.6.1 Formative Evaluation

Formative evaluation runs parallel to the learning process and is meant to evaluate

the quality of the learning materials in sign language mobile application and their reception

29  

by the users. We need to make sure this type of learning platform is fun and engaging so

that the users will achieve the learning objectives as well as to get feedback from them.

3.6.2 Summative Evaluation

The main goal of summative evaluation is to prove the effectiveness of this mobile

application. For that, we use the Donald Kirkpatrick evaluation model, which has long ago

become the standard for evaluating.

Level 1: Reaction

How users reacted to this sign language mobile application. All the data will

be collected from questionnaire afterward containing a few statements about

the course that users need to rate. These questionnaires are usually called

“Smile sheets”.

Level 2: Learning

On this level we test the knowledge and skills acquired during the learning

process.

Level 3: Behavior

According to Donald Kirkpatrick, this evaluation level is the hardest to

implement. It involves analyzing the changes in the user’s behavior as a

result of participating in the learning process and understand how well and

how often the acquired knowledge and skills are being practiced outside of

that environment.

30  

Level 4: Result

The fourth level deals with analyzing the financial results of the learning

process. Namely, whether the delivered results matched up to the goals that

had been set. Other factors that can be considered include increase in

productivity and improvement in quality of the application.

3.7 Evaluation as The Final ADDIE Stage

Even though evaluation is the final stage of the ADDIE methodology, it should be

considered not as a conclusion of a long process, but as a starting point for the next iteration

of the ADDIE cycle.

31  

CHAPTER IV

IMPLEMENTATION

32  

REFERENCES  

[1] Aldoobie, N. (2015). ADDIE Model. American International Journal of

Contemporary Research, 68-72.

[2] Kanwal Yousaf, Z. M. (2018). A Novel Technique for Speech Recognition and Visualization Based Mobile Application to Support Two-Way Communication between Deaf-Mute and Normal Peoples. 1-5.

[3] Salomons, J. L. (n.d.). A Mobile Language Learning Application : Learn To Sign. 6-23.