© COPYRIG

HT UPM

UNIVERSITI PUTRA MALAYSIA

FACIAL FEATURE EXTRACTION BASED ON IMPROVED HARRIS CORNER DETECTION ALGORITHM

ELHAAM BAGHERIAN

FSKTM 2011 9

© COPYRIG

HT UPM

FACIAL FEATURE EXTRACTION BASED ONIMPROVED

HARRIS CORNER DETECTION ALGORITHM

ELHAAM BAGHERIAN

MASTER OF SCIENCEUNIVERSITI PUTRA MALAYSIA

2011

© COPYRIG

HT UPM

FACIAL FEATURE EXTRACTION BASED ON IMPROVED

HARRIS CORNER DETECTION ALGORITHM

By

ELHAAM BAGHERIAN

Thesis Submitted to the School of Graduate Studies,Universiti Putra Malaysia, in Fulfilment of the Requirements

For the Degree of Master of Science

July 2011

© COPYRIG

HT UPM

DEDICATION

To whom made me think

© COPYRIG

HT UPM

i

Abstract of thesis presented to the Senate of Universiti Putra Malaysia in fulfilment of therequirement for the degree of Master of Science

FACIAL FEATURE EXTRACTION BASED ON IMPROVEDHARRIS CORNER DETECTION ALGORITHM

By

ELHAAM BAGHERIAN

July 2011

Chairman: Assoc. Prof. Rahmita Wirza O.K. Rahmat, PhD

Faculty: Computer Science and Information Technology

The extraction of facial feature points has become an important issue in many

applications, such as face recognition, face expression recognition and face detection.

Segmenting the facial features’ points in an image is the first important step for human

face recognition, identification and verification. Problems occur in different face

orientations and poses, and under varied lighting conditions, covering and facial

expressions.

A method of facial feature extraction and corner detection is presented in this study to

unravel these problems. The proposed technique has been developed to extract the facial

features from a colored image, captured by the webcam under normal lighting condition.

In order to precisely extract the facial features such as eyes, mouth and nostrils, some

preprocessing steps are employed once the image is captured. Some of these steps are also

used during the corner detection phase.

Experiments are conducted with a number of images from the frontal, near frontal, up and

down views of the head and from different expressions such as happy, sad, surprised and

© COPYRIG

HT UPM

ii

neutral. This technique is evaluated on two different standard databases, BioID and

George Tech. These two databases consist of 1520 images and 710 images respectively.

Each of these databases includes images with different orientations and expressions,

occlusions and lighting conditions.

This technique is also tested using five different webcams; with different levels of

resolution and quality and web camera specifications, in order to maintain the accuracy of

the technique. The performance of the technique is judged by its accuracy on each of the

features like nose, eyes and mouth. After validations and verifications are made which are

based on the defined performance parameter, it can be observed that the proposed

technique is more accurate and precise.

© COPYRIG

HT UPM

iii

Abstrak tesis yang dikemukakan kepada Senat Universiti Putra Malaysia sebagaimemenuhi keperluan untuk ijazah Master Sains

Oleh

ELHAAM BAGHERIAN

Julai 2011

Pengerusi: Prof. Madya Rahmita Wirza O.K. Rahmat

Fakulti: Sains Komputer dan Teknologi Maklumat

Pengekstrakan titik ciri wajah telah menjadi isu penting dalam banyak aplikasi, seperti

pengecaman wajah, pengecaman ekspresi muka dan pengesanan wajah. Mensegmenkan

titik ciri wajah dalam imej adalah langkah pertama yang penting untuk pengecaman,

pengenalan dan pengesahan wajah manusia. Masalah-masalah umumnya berlaku pada

kepelbagaian orientasi muka dengan kedudukan di bawah keadaan pencahayaan yang

berbeza juga wajah terlindung dengan kepelbagaian riak wajah.

Satu kaedah pengekstrakan ciri wajah dan pengesanan sudut dipersembahkan dalam kajian

ini untuk mengungkap masalah tersebut. Teknik yang dicadangkan ini dibangunkan untuk

mengekstrak ciri wajah pada gambar berwarna, yang ditangkap menggunakan webcam di

bawah keadaan pencahayaan yang normal. Untuk mengekstrak ciri wajah seperti mata,

mulut dan hidung secara tepat, beberapa langkah Prapemprosesan dijalankan selepas foto

diambil. Langkah-langkah ini juga digunakan semasa fasa pengesanan sudut.

© COPYRIG

HT UPM

iv

Eksperimen dijalankan dengan beberapa imej dari hadapan, berhampiran pandangan

hadapan, atas dan bawah kepala pada riak wajah yang berbeza seperti seronok, sedih,

terkejut dan neutral. Teknik yang dicadangkan ini, dinilai berdasarkan kepada dua

pangkalan data piawai yang berbeza, iaitu BioID dan GEORTECH. Kedua-dua pangkalan

data ini masing-masing terdiri daripada 1520 dan 710 gambar. Setiap pangkalan data ini

mempunyai gambar dengan orientasi dan riak wajah yang berbeza, wajah terlindung dan

kondisi pencahayaan yang berbeza.

Teknik ini juga diuji dengan lima webcam yang berbeza; dengan pelbagai ciri seperti

resolusi dan kualiti untuk mengesahkan bahawa dengan webcam yang berbeza spesifikasi,

ketepatan teknik dapat dipertahankan. Prestasi teknik ini dinilai daripada ketepatan setiap

ciri yang ditemui seperti hidung, mata dan mulut. Setelah disahkan dan dibuktikan

berdasarkan pada parameter prestasi yang telah ditetapkan, dapat dilihat bahawa teknik

yang dicadangkan lebih tepat dan jitu.

© COPYRIG

HT UPM

v

ACKNOWLEDGEMENTS

I would like to thank my supervisor, Associate Prof.Dr.Rahmita Wirza O.K. Rahmat for

her valuable comments and advice through the course of this research. Her encouragement

and professional review helped this thesis and other technical papers to be further

improved.

My further gratitude goes to Dr Nur Izura Udzir for her great help and technical advice.

Also, my eternal gratitude is owed to my family who has been supportive in everything

especially to my mother, Maryam, for her never ending love and support.

I also want to thank all my second family members in Malaysia, including all my friends,

for providing me with great friendship and experience in my academic and social life.

© COPYRIG

HT UPM

vi

I certify that a Thesis Examination Committee has met on 1st of July 2011 to conduct thefinal examination of Elhaam Bagherian on her thesis entitled “FACIAL FEATUREEXTRACTION BASED ON IMPROVED HARRIS CORNER DETECTIONALGORITHM” in accordance with Universities and University College Act 1971 andthe Constitution of the Universiti Putra Malaysia [P.U.(A) 106] 15 March 1998. Thecommittee recommends that the student be awarded the Master of Science.

Members of the Thesis Examination Committee were as follows:

Mohamed Othman, PhDProfessorFaculty of Computer Science and Information TechnologyUniversiti Putra Malaysia(Chairperson)

Hajah Fatimah binti Dato Ahmad, PhDProfessorFaculty of Computer Science and Information TechnologyUniversiti Putra Malaysia(Internal Examiner)

Shyamala a/p C Doraisamy, PhDFaculty of Computer Science and Information TechnologyUniversiti Putra Malaysia(Internal Examiner)

Aini Hussain, PhDProfessorDepartment of Electrial Engineering, Electronics and SystemsUniversiti kebangsaan Malaysia(External Examiner)

NORITAH OMAR, PhDAssociate Professor and Deputy DeanSchool Of Graduate StudiesUniversiti Putra Malaysia

Date: 1 July 2011

© COPYRIG

HT UPM

vii

DECLARATION

I declare that this thesis is my original work except for quotations and citations whichhave been duly acknowledged. I also declare that it has not been previously, and it is notconcurrently, submitted for any other degree at Universiti Putra Malaysia or at any otherinstitutions.

_____________________ELHAAM BAGHERIAN

Date: 1 July 2011

© COPYRIG

HT UPM

viii

TABLE OF CONTENTS

PageABSTRACT iABSTRAK iiiACKNOWLEDGMENTS vAPPROVAL viDECLARATION viiLIST OF TABLES xLIST OF FIGURES xiiLIST OF ABBREVIATIONS xv

CHAPTER

1 INTRODUCTION1.1 Background ....................................................................................................... 11.2 Problem Statement ............................................................................................ 5

1.2.1 Size and Orientation..............................................................................51.2.2 Lighting Conditions ..............................................................................61.2.3 Occlusions .............................................................................................71.2.4 Facial Expression ..................................................................................8

1.3 Research Objectives and Scope ........................................................................ 91.4 Research Contributions ..................................................................................... 91.5 Thesis Organisation........................................................................................... 9

2 LITERATURE REVIEW2.1 Introduction ..................................................................................................... 112.2 Main Approaches ............................................................................................ 122.3 Face Recognition Techniques ......................................................................... 13

2.3.1 Neural Networks .................................................................................152.3.2 Geometrical Feature Matching............................................................152.3.3 Graph Matching ..................................................................................172.3.4 Eigenface.............................................................................................182.3.5 Fisherface ............................................................................................20

2.4 Facial Feature Extraction ................................................................................ 222.5 Techniques of Facial Feature Extraction......................................................... 27

2.5.1 Geometry-Based..................................................................................272.5.2 Template-Based...................................................................................282.5.3 Color Segmentation Technique...........................................................282.5.4 Appearance-Based Approach..............................................................292.5.5 Hybrid Approach.................................................................................292.5.6 Real-time Approach ............................................................................31

2.6 Advantages and Disadvantages of Previous Works........................................ 512.7 Conclusion ...................................................................................................... 55

3 METHODOLOGY3.1 Introduction ..................................................................................................... 563.2 Research Methodology.................................................................................... 56

© COPYRIG

HT UPM

ix

3.3 Pre-Processing................................................................................................. 623.4 Detecting Corners............................................................................................ 643.5 Summary ......................................................................................................... 76

4 RESULTS AND DISCUSSIONS4.1 Introduction ..................................................................................................... 774.2 Experimental Evaluation................................................................................. 774.3 Standard Databases ......................................................................................... 80

4.3.1 BioID Database ...................................................................................804.3.2 Georgia Tech Database .......................................................................814.3.3 Evaluation by Databases .....................................................................814.3.4 GT Database........................................................................................85

4.4 Web cameras ................................................................................................... 884.4.1 Analyze the Results of Testing by Five Different Web Cameras .....101

4.5 Different Lighting Conditions.......................................................................... 1024.5.1 Analysis of the Results under Different Lighting Conditions

4.6 Summary ....................................................................................................... 109

5 CONCLUSION AND FUTURE WORK5.1 Conclusion 1105.2 Future work 112

REFERENCES ............................................................................................................... 114BIODATA OF STUDENT.............................................................................................. 121LIST OF PUBLICATIONS/AWARDS......................................................................... 122