developing english language test papers: what … · guru bahasa inggeris dan faktor-faktor yang...
TRANSCRIPT
DEVELOPING ENGLISH LANGUAGE TEST PAPERS:
WHAT TEACHERS PRACTISE
LEE JUN
FAKULTI PENDIDIKAN
UNIVERSITI TEKNOLOGI MALAYSIA
UNIVERSITI TEKNOLOGI MALAYSIA
CATATAN: * Potong yang tidak berkenaan ** Jika tesis ini SULIT atau TERHAD, sila lampirkan surat daripda pihak berkuasa/organisasi
berkenaan dengan menyatakan sekali sebab dan tempoh tesis ini perlu dikelaskan sebagai SULIT atau TERHAD
♦ Tesis dimaksudkan sebagai tesis bagi Ijazah Doktor Falsafah dan Sarjana secara penyelidikan, atau disertasi bagi pengajian secara kerja kursus dan penyelidikan, atau Laporan Projek Sarjana Muda (PSM)
BORANG PENGESAHAN STATUS TESIS ♦ JUDUL: DEVELOPING ENGLISH LANGUAGE TEST PAPERS: WHAT
TEACHERS PRACTISE
SESI PENGAJIAN: 2005/2006 Saya ____________________________LEE JUN_________________________________
(HURUF BESAR) mengaku membenarkan tesis (PSM/Sarjana/Doktor Falsafah)* ini disimpan di Perpustakaan Universiti Teknologi Malaysia dengan syarat-syarat kegunaan seperti berikut; 1. Tesis adalah hakmilik Universiti Teknologi Malaysia. 2. Perpustakaan Universiti Teknologi Malaysia dibenarkan membuat salinan untuk tujuan
pengajian sahaja. 3. Perpustakaan dibenarkan membuat salinan tesis ini sebagai bahan pertukaran antara institusi
pengajian tinggi. 4. ** Sila tandakan (√)
SULIT (Mengandungi maklumat yang berdarjah keselamatan atau
kepentingan Malaysia seperti yang termaktub di dalam AKTA RAHSIA RASMI 1972) TERHAD (Mengandungi maklumat TERHAD yang telah ditentukan oleh
organisasi/badan di mana penyelidikan dijalankan)
TIDAK TERHAD
Disahkan oleh _______________________________ _____________________________
(TANDATANGAN PENULIS) (TANDATANGAN PENYELIA) Alamat Tetap: 35A, KLINIK KESIHATAN SPAOH, DR ABDUL HALIM ABDUL RAOF 95600 SPAOH, BETONG, SARAWAK Nama Penyelia Tarikh: _________________________ Tarikh: _______________________
√
DEVELOPING ENGLISH LANGUAGE TEST PAPER:
WHAT TEACHERS PRACTISE
LEE JUN
Tesis ini dikemukakan sebagai memenuhi
syarat penganugerahan Ijazah Sarjana Muda
Sains serta Pendidikan (TESL)
Fakulti Pendidikan
Universiti Teknologi Malaysia
APRIL, 2006
“Saya akui bahawa saya telah membaca karya ini dan pada pandangan
saya karya ini adalah memadai dari segi skop dan kualiti untuk tujuan
Penganugerahan Ijazah Sarjana Muda
Sains Serta Pendidikan (TESL)”
Tandatangan Penyelia :…………………………
Nama Penyelia : DR. ABDUL HALIM ABDUL RAOF
Tarikh :…………………………
iii
“Saya akui karya ini adalah hasil kerja saya sendiri kecuali nukilan dan ringkasan
yang tiap-tiap satunya telah saya jelaskan sumbernya.”
Tandatangan : ……………………………….
Nama : LEE JUN
Tarikh :……………………………….
iv
Dad and Mum…
It’s for you!
A product of your five years sowing
v
ACKNOWLEDGEMENT
First and foremost, I would like to thank my supervisor, Dr. Abdul Halim Abdul
Raof for his patience and kindness in supervising me throughout the whole research
project. It has been a fruitful and unforgettable journey which has improved my attitude
towards life. He is a very caring supervisor and I feel honoured to work under his
supervision. Without his tireless efforts and on-going support, constructive advice as
well as guidance, my research would not have been completed successfully.
I would also like to thank some teachers who had willingly became my subjects
and provided me with useful information. They are teachers in Sekolah Menengah
Kebangsaan Tun Habab, Sekolah Menengah Kebangsaan Spaoh, and schools around
Skudai, Johor.
Special thanks is also given to my examiners, Dr. Zaidah Zainal and Puan
Norzarina Sulong; Project Sarjana Muda Coordinators, PM Dr. Shaharom Nordin, Puan
Zanariah bt Mohd Salleh, and Mr. Baharim Kassim.
I would like to extend my heartfelt gratitude to my family and all my friends who
on their advice and endless support.
Thank you to all.
vi
ABSTRACT
Developing test papers is a must in the teaching career. In developing test
papers, there are a number of factors to be considered. At the same time, teachers have
their own methods and practices that they follow, some of which assist them while
others hinder the process. This study was conducted to find out the practices in
developing test papers among English language teachers and the factors that influenced
such practices. The study was carried out with 29 TESL teachers who taught English
language in lower secondary forms from five different schools in Skudai, Johor. Data
was collected via questionnaire. The findings show that there were a wide range of
practices in developing test papers among teachers. Of all the practices, revising PMR
past year questions was the most preferable practice among teachers. Holding group
discussion was a common practice for English option and experienced English option
teachers. And, the practice of adapting resources was more dominant among Non
English option teachers especially experienced Non English option teachers. It is also
found that the main factors that influenced the practices of developing test papers were
students, practical issues and syllabus. This study has provided an overview of the
testing and evaluation practices of English language teachers in the Malaysian context.
Overall, teachers must be careful in considering the factors which influenced their
practices and at the same time be creative and professional in choosing the appropriate
practices of test development to ensure the effectiveness of measurement for decision
making.
vii
ABSTRAK
Penghasilan kertas ujian adalah sesuatu yang wajib dalam karier sebagai seorang
guru. Dalam proses penghasilan kertas ujian, terdapat pelbagai faktor yang perlu
dipertimbangkan. Pada masa yang sama, guru-guru mempunyai kaedah dan amalan
tersendiri, ada yang membantu manakala ada yang menghalang proses penghasilan.
Kajian ini dijalankan bagi mengenal pasti amalan penghasilan kertas ujian di kalangan
guru Bahasa Inggeris dan faktor-faktor yang mempengaruhi amalan-amalan tersebut.
Kajian ini melibatkan 29 orang guru Bahasa Inggeris sebagai Bahasa Kedua yang
mengajar di peringkat menengah rendah dari lima buah sekolah yang berlainan di
Skudai, Johor. Data dikumpulkan melalui soal-selidik. Dapatan kajian menunjukkan
terdapat pelbagai cara pembinaan kertas ujian di kalangan guru. Antara amalan-amalan
penghasilan kertas ujian, mengadaptasikan soalan PMR tahun-tahun lepas adalah amalan
yang paling digemari di kalangan guru. Mengadakan perbincangan adalah satu amalan
yang biasa dilakukan oleh guru opsyen Bahasa Inggeris dan guru opsyen yang
berpengalaman. Selain itu, amalan mengubahsuai sumber adalah lebih dominan di
kalangan guru bukan opsyen terutamanya guru bukan opsyen yang berpengalaman.
Kajian juga mendapati faktor yang mempengaruhi amalan penghasilan kertas ujian
termasuklah pelajar, isu praktikal dan silibus. Secara keseluruhan, kajian ini memberi
satu gambaran keseluruhan amalan guru-guru Bahasa Inggeris dalam penghasilan kertas
ujian di dalam konteks Malaysia. Guru seharusnya cermat dalam mempertimbangkan
faktor-faktor yang mempengaruhi mereka dan pada masa yang sama bersikap
professional serta kreatif dalam memilih amalan yang bersesuaian untuk penghasilan
kertas ujian bagi memastikan keberkesanan pengukuran dalam membuat keputusan.
viii
CONTENTS
CHAPTER SUBJECT PAGE
DEDICATION
ACKNOWLEDGEMENT
ABSTRACT
ABSTRAK
CONTENTS
LIST OF TABLES
LIST OF SYMBOLS
LIST OF APPENDICES
iv
v
vi
vii
viii
xiii
xv
xvi
SECTION ONE
INTRODUCTION
ONE INTRODUCTION 1
1.1 Background of the Study
1.2 Statement of Problem
1.3 Objectives of the Study
1.4 Research Questions
1
3
3
4
ix
1.5 Significance of the Study
1.6 Scope of the Study
1.7 Definition of Terms
1.7.1 Practices
1.7.2 Test Papers
1.7.3 Developing
1.8 Conclusion
4
5
6
6
7
7
7
SECTION TWO
LITERATURE REVIEW
TWO LITERATURE REVIEW 8
2.1 Practices in Developing English Language Test
Papers
2.1.1 Developing Own Questions
2.1.2 Adopting Materials
2.1.3 Adapting Materials
2.1.4 Working with Item Bank
2.2 Factors that Influenced Teachers’ Practices in
Developing English Language Test Papers
2.2.1 Test Paper
2.2.1.1 Format of a Test Paper
2.2.1.2 Marking System
2.2.1.3 Quality of a Test Paper
2.2.2 Teachers
2.2.3 Students
2.2.4 Availability of Resources
8
9
9
11
12
13
14
14
15
15
16
18
19
x
2.2.5 Practical Issues
2.2.6 Syllabus
2.2.7 Other Factors
2.2.7.1 Bloom’s Taxonomy
2.2.7.2 Feedback from Parents
2.2.7.3 Feedback from Colleagues
2.2.7.4 School
2.3 Penilaian Menengah Rendah (PMR) English
Language Test Paper
2.4 Conclusion
20
22
23
23
23
24
24
25
27
SECTION THREE
METHODOLOGY
THREE METHODOLOGY 28
3.1 Research Design
3.2 Participants of the Study
3.3 Data Collection Instrument
3.4 Data Collection Procedures
3.5 Data Analysis Procedures
3.6 Pilot Studies
3.7 Conclusion
28
29
29
30
31
31
33
xi
SECTION FOUR
DATA ANALYSIS AND FINDINGS
FOUR DATA ANALYSIS AND FINDINGS 34
4.1 Section A: Demographic Information
4.1.1 Gender
4.1.2 Highest Education Level
4.1.3 English Option vs. Non English Option
4.2 Section B: Practices in Developing Test Papers
4.2.1 English Option vs. Non English Option
4.2.2 Experienced Teachers
4.2.3 Section B: Summary of Findings
4.3 Section C: Factors that Influenced the Practices
of Test Papers Development
4.3.1 English Option vs. Non English Option
4.3.2 Experienced Teachers
4.3.3 Section C: Summary of Findings
4.4 Section D: Main Concerns, Difficulties and
Advice
4.4.1 Main Concerns
4.4.2 Difficulties
4.4.3 Advice
4.5 Conclusion
35
35
35
36
38
41
44
46
46
49
50
53
54
54
56
57
58
xii
SECTION FIVE
CONCLUSIONS
FIVE CONCLUSIONS 60
5.1 Summary of the Study
5.2 Summary of Main Findings
5.3 Implications of the Study
5.4 Limitations of the Study
5.5 Future Research
5.6 Conclusion
60
61
62
63
65
66
REFERENCES
APPENDICES
67-71
72-95
xiii
LIST OF TABLES
NO. OF TABLE TOPIC PAGE
2.1 Latest Format of the PMR English Language 2005 Paper
4.1 Gender of the Participants
4.2 Highest Education Level of the Participants
4.3 English Option Teachers vs. Non English Option Teachers
4.4 Details of English Option Teachers vs. Non English Option
Teachers
4.5 Overall Rankings of Practices of Developing Test Papers
among Teachers
4.6 Practices of Developing Test Papers: English Option vs. Non
English Option
4.7 Experienced Teachers: Overall Rankings of Practices of
Developing Test Papers
4.8 Experienced English Option Teachers: Rankings of Practices
of Developing Test Papers
4.9 Experienced Non English Option Teachers: Rankings of
Practices of Developing Test Papers
4.10 Factors that Influenced in the Practices of Developing Test
Papers
4.11 Factors that Influenced in the Practices of Developing Test
Papers by Category
26
35
36
36
37
39
42
44
45
45
47
48
xiv
4.12 Factors that Influenced in the Practices of Developing Test
Papers: English Option vs. Non English Option Teachers
4.13 Factors that Influenced Experienced Teachers in the Practices
of Developing Test Papers
4.14 Factors that Influenced Experienced English Option Teachers
in the Practices of Developing Test Papers
4.15 Factors that Influenced Experienced Non English Option
Teachers in the Practices of Developing Test Papers
4.16 Main Concerns in Developing Test Papers
4.17 Difficulties faced in Developing Test Papers
4.18 Advice in Developing Test Papers
49
51
52
52
55
56
57
xv
LIST OF SYMBOLS
ESL - English as a Second Language
KBSM - Kurikulum Bersepadu Sekolah Menengah
KBSR - Kurikulum Bersatu Sekolah Rendah
MES - Malaysia Examination Syndicate
PMR - Penilaian Menengah Rendah
STPM - Sijil Tinggi Pelajaran Malaysia
TEFL - Teaching English as a Foreign Language
TESL - Teaching English as a Second Language
UPSR - Ujian Penilaian Sekolah Rendah
xvi
LIST OF APPENDICES
APPENDIX TOPIC PAGE
A Questionnaire 73
B Data Analysis (Section A) 78
C Data Analysis (Section B) – Part 1 79
D Data Analysis (Section B) – Part 2 82
E Data Analysis (Section C) – Part 1 85
F Data Analysis (Section C) – Part 2 86
G Approval Letter from BPPDP, Kementerian
Pelajaran Malaysia 94
CHAPTER ONE
INTRODUCTION
A test paper is an instrument used in a classroom to gather information about
teaching and learning process. As a teacher, developing test papers is an integral part
of teaching. Teachers normally develop test papers to assess the progress of
students, effectiveness of teaching, and achievement of objectives. When developing
test papers become a routine to teachers, it is believed that there are certain practices
applied by teachers in developing test papers. Thus, this study was carried out to
identify the practices in developing test papers among teachers of English as a
Second Language (ESL). This study also intended to find out the factors that
influenced teachers’ practices in developing test papers.
1.1 Background of the Study
Developing test papers has become a necessity in any teaching career. Many
ESL teachers found themselves involved in testing from the very beginning of their
career. They often need to develop test papers to gather information about the
teaching and learning process or to prepare students for public examination.
2
Preparing test papers is a challenging task for teachers. And this task
becomes more challenging when teachers need to construct test papers for the
purpose of assessing ESL students. It requires knowledge of test construction and
experience in developing test papers (Brown, 1996). For this reason, pre-service or
teacher training courses include a language-testing component in their syllabus
(Wharton, 1998). There are also a wide range of materials to guide teachers in
enhancing their skills and knowledge in developing test papers. Besides that, rich
experience is an advantage in developing test papers. This is because to become an
expert in developing test papers, continuous involvement and hand-on experiences
are the keys.
Oosterhof (1996) highlights that a test paper must be fully developed in
advance, and developing test papers is not a spontaneous process. Ory and Ryan
(1993) also address that a significant amount of time and effort are needed in
developing fair and accurate test papers. It is hard work and time consuming. But,
sad to notice that after a test paper is developed, criticisms are aimed at the test
papers (Sanders, 1995) such as ambiguous questions, content of the test does not
correspond to materials covered, inconsistency in scoring, etc (Oosterhof, 1996).
Considering the need of developing test papers, knowledge in testing and
evaluation, experience in constructing test papers and the long process in developing
test papers, teachers form their own practices of developing high quality test papers.
While developing a test paper, there are numerous issues that may influence the
kinds of choices that teachers must make. All these issues must be considered
simultaneously in practices of developing test papers. This is because the issues
would have some significance toward the quality of test papers.
3
1.2 Statement of Problem
Language testing courses in the syllabus of teacher training colleges often
focus on the theoretical aspect of developing test papers. The same goes to language
books available. The knowledge from the syllabus and books are effective enough to
guide teachers in developing test papers. The emphasis of syllabus and books are the
steps in the development of test papers, interpreting scores, validity and reliability of
test papers, types of questions, and the dos and don’ts in developing test papers.
However, there is a lack in terms of practices teachers should follow in developing
test papers. In a survey of the literature on testing and construction of items, it is
found that there is a limited reference which addressed the issue of practices in
developing test. Looking at the lack of related studies; this study is to cater the need
of investigating the practices in developing test among teachers in order to provide
guidelines for teachers in their practices of developing test papers.
Developing test papers is a process not done in isolation. Therefore, there are
many information that can be gathered from a detail analysis of a test paper
developed by a teacher. One of the information is the influences in developing test
paper practices. This is because practices in developing test papers are greatly
influenced by a few factors. Identified factors that influenced teachers provide some
practical reasons of certain practices in developing test. Thus, investigating the
practices and factors that influenced the practices are necessary in developing test
papers.
1.3 Objectives of the Study
The objectives of this study were to find out:
4
1.3.1 the practices used by English Language teachers in developing
English Language test papers, and
1.3.2 the factors that influenced teachers in their practices of developing
English language test papers.
1.4 Research Questions
Based on the objectives of this study, the following are the research questions
which serve as the main focus of this research:
1. What are the practices of English language teachers in developing
English language test papers?
2. What are the factors that influenced teachers in having such practices?
1.5 Significance of the Study
This study which looked into English Language teachers’ practices in
developing test papers is useful and beneficial. First of all, from the findings, it
provides an overview scenario of language testing practices in schools. Tuckman
(1988) mentions that test papers designed by teachers have a high degree of validity.
So, information from this study can serve as a platform for the Malaysian
Examination Syndicate (MES) to apply the practices which teachers used in order to
set more valid and reliable test papers for standardized public examinations. Thus,
this study puts some pressure on professional test developers and examination board
to improve their practices in tests development (Hughes, 2003).