16/05/2009

1

1

Prof Dr Shahrin MohammadFakulti Kejuruteraan Awam

Mula berkhidmat 1984 – Fakulti Kejuruteraan AwamKetua Makmal Komputer 1988 – 1990Ketua Jabatan Struktur dan Bahan 1997 – 1998P T k l i M kl t 1998 2001Pengurus Teknologi Maklumat 1998 - 2001Timbalan Dekan (Akademik) 2001-2005Wakil Pengurusan ISO 9001:2000 2001-2005 Pengarah Kualiti Akademik UTM 2005-2007

Certified ISO 9001:2000 Lead Auditor, Facilitator/Trainer ISO 9001:200, Internal AuditorEngineering Accreditation Council Panel of TrainersLAN and MOHE facilitator on MQF Credit System

2

LAN and MOHE facilitator on MQF Credit SystemFacilitator/Trainer on Outcome based Approach for Engineering AccreditationFacilitator/Trainer on Quality Assurance for Academic ProgrammeFacilitator/Trainer on Academic Auditing

16/05/2009

2

Change?

Survey by Harvard Univ.Survey by Harvard Univ.

•• ....largelarge measuremeasure ofof satisfactionsatisfaction withwith technicaltechnicalcompetencecompetence ofof thethe graduatinggraduating studentsstudents……cancan crunchcrunch numbers,numbers, analyseanalyse marketsmarkets…….. ButButwhenwhen itit comescomes toto humanhuman--relationrelation skills,skills,HarvardHarvard isis steppingstepping upup itsits efforteffort…… ThatThat

3

pp gpp g ppseemsseems wherewhere improvementsimprovements areare neededneeded....OralOral andand writtenwritten communication,communication, teamwork,teamwork,andand otherother humanhuman skillsskills……

Anything new ?

■ Facts on CQI■ Why CQI?■ How to do it?■ How to do it?

16/05/2009

3

5

What is CQI ?

■ The culture, strategies and methods necessary for continual improvement in meeting and exceeding customers' expectations. (www.european-quality.co.uk/index.php)

■ A management approach to improving and maintaining quality that emphasizes internally driven and relatively continuous assessments of potential causes of quality defects, followed by

ti i d ith t idi d i lit laction aimed either at avoiding decrease in quality or else correcting it an early stage.www.qaproject.org/methods/resglossary.html

16/05/2009

4

Continuous Quality Improvement

The systematic pursuit of excellence and satisfaction of the needs of stakeholders, in a dynamic and

competitive environmentcompetitive environment

Facts on CQI

■ exists only when it is done systematically■ is a dynamic behavior of an organization■ is a dynamic behavior of an organization■ does NOT result from external factors■ can NOT be achieved in isolation■ exists when the continuous pursuit of excellence

motivates and guides the philosophies, planning, policies and processes of the organizationF f CQI i d ti i NOT th i l■ Focus of CQI in education is NOT on the curriculum

■ Foundation of CQI is Assessment

16/05/2009

5

Facts on CQI

■ Quality improvement comes from within an institution■ Continuous improvement requires integration of■ Continuous improvement requires integration of

defined objectives, performance metrics, & regular assessment

■ Continuous improvement is cyclical. Assessment of performance is the baseline for future assessment

■ Essential to synchronize educational objectives, mission of IHL and needs of stakeholders to achievemission of IHL and needs of stakeholders to achieve CQI

■ CQI can be proven through well documented report and data.

Facts on CQI

■ Success is achieved through meeting the needs of those we serve.those we serve.

■ Most problems are found in processes. CQI does not seek to blame, but rather to improve processes.

■ It is possible to achieve continual improvement through small, incremental changes using the scientific method.

■ Continuous improvement is most effective when it■ Continuous improvement is most effective when it becomes a natural part of the way everyday work is done.

16/05/2009

6

The BIG Picture

Facts on CQI - 10 principles of CQI

1. Define Quality vs “I know it when I see it”• Each quality outcome should have a specific definition

2. Customer Orientation vs Internal Focus• Each quality indicator is determined by its relationship to improving outcomes

3. Work Process Focus vs End Product Focus• Each quality outcome is designed to be measured according to the extent to which

work processes result in ongoing improvement over time.4. Us Partnerships vs We and They

• Each quality outcome can be achieved only through collaboration.5. Proactive vs Reactive

• Each quality outcome is achieved through planned and deliberate intervention over time, as opposed to end product corrective action.

16/05/2009

7

cont…6. 100% Quality Attitude vs That’s good enough

• Each quality outcome is attained by an approach that works to achieve the best result7. Management by Facts vs Management by Intuition

• Each quality outcome is attained through successive evaluation of data and • Each quality outcome is attained through successive evaluation of data and adjustment accordingly.

8. Engagement and Empowerment vs “Just follow the plan, man”• Each quality outcome is implemented by collective engagement in designing

successful processes. 9. Quality belongs to everyone vs “We have a quality dept”

• Each quality outcome is implemented through the interactive participation of all components of the system.

10. Continuous means continuous vs “we measure when its over”• Each quality outcome is implemented through the iterative processes of PDCA..

14

16/05/2009

8

3 domains of educational goals

(cognitive, psychomotor and

affective)

Taxonomy level

1

The three main stages in the Teaching and Learning processes

What do you want

How can you best help the

LearningOutcomes

)

Planning stage

Implementation stage

1

2

ythe students to be

able to do?students to achieve it?

15Assessment stage

3

How will you know that they

have achieved

it?

4How do

you close the loop? How can

you improve?

QA: Quality Assurance Cycle

Approval

Institutional and self study report

by JPT

3 – 5years

Monitoring

Internal QA

Academic Auditing(visit and report)

16

Accreditationvisit and report

By “EAC-BEM”“MQA”

Program Audit

Institutional Audit

Self Accrediting Institutions (SAI)

16/05/2009

9

Leadership, Governance & Administration (8)

Mission, Vision, Educational Objectives &

Monitoring & Program O

LDER

NEE

DS

STIT

UTI

ON

AL

ISSI

ON

ON

SECTION 2 of MQA Code of Practice in Perspective

Educational Resources (6)

Educational Objectives & Learning Outcomes (1)

Academic Staff (5)

Curriculum Design (2a)

Delivery (2b)INPUT (STUDENTS)

Selection (4a)

Review (7)

Student Assessments

(3)

OUTPUT (GRADUATES)

STAK

EHO

AND

INS M

OM

R

S KEH

OLD

ER’S

SAT

ISFA

CTI

O

17

Support Services (4b)

Continual Quality Improvements (9)

FEED

BAC

KS

FRO

STAK

EHO

LDER

STAK

STAKEH

OLD

ER’S N

EAN

D IN

STITUTIO

NM

ISSION

LEADERSHIP, GOVERNANCE and QMS(C5)

EAC-BEM ACCREDITATION PRACTICES IN A NUTSHELL

PEO and PO DEVELOPMENT (S3 S4)

PROGRAMME REVIEW FOR STEED

S N

AL FES

FACILITIES (C4)

DEVELOPMENT (S3,S4)

ACADEMIC STAFF (C3)

CURRICULUM DESIGN (C1)

TEACHING AND LEARNING ACTIVITIES (C2,C3)

STUDENTS (C2)

CQI(S4,C4,C5)

PO ASSESSMENT

(S3)

TAKEH

OLD

ER’S SATISFA

C

OUTPUT (GRADUATES)

INPUT

18

EEDB

ACK

S FRO

M

STAKEH

OLD

ER

SUPPORTING UNIT AND STAFF(C2,C3)

CTIO

N

INPUT

16/05/2009

10

EAC - Requirements

…….The process of continual quality improvement shall be demonstrated to be effective with full accountability from the academic staff and to a lesser extent, the supporting staff….

(Clause 7 5 2)(Clause 7.5.2)

Summary Material (Self Assessment) Document - EAC

Describe how and provide evidences that the lt bt i d b thi i di l tiresults obtained by this periodic evaluation are

being used to improve the effectiveness of the programme. (Cl 8.2.2)

Describe the process by which the assessment results are applied to further develop and improve the programme. (Cl 8.2.3)

16/05/2009

11

Summary Material (Self Assessment) Document - EAC

Describe the process for ensuring continual quality improvement of the programme. Provide evidence on

f ff ffthe participation of academic staff, supporting staff and students in the continual quality improvement process. (Cl 8.8(iii))

Provide all relevant documents and evidences related to Programme Objectives and Programme Outcomes (one copy) as follows:-…(iii) all evidences related to CQI of the programme.

22

16/05/2009

12

• Who are your stakeholders?• What are the services/outcomes you produce?• Are your objectives/outcomes understood by your constituencies?

A Guide for Continuous Improvement (taken from the EAC Training Module)

• Are your objectives/outcomes understood by your constituencies?• What services, facilities, and policies must be present if you are to

satisfy your stakeholders? • Does your institutional leadership understand your needs?• What steps do you perform to produce your services/outcomes? • How do you measure your results? • How do you use these measurements to continually improve the y y p

services you provide?• Are you achieving your objectives/oucomes? Are you improving?• Are your constituencies/stakeholders satisfied?

• Form a team that has knowledge of the system needing improvement.

Core Steps in Continuous Improvement

• Define a clear aim.• Understand the needs of the people who are served by

the system.• Identify and define measures of success.• Brainstorm potential change strategies for producing

improvement.p• Plan, collect, and use data for facilitating effective

decision making.• Apply the scientific method to test and refine changes.

16/05/2009

13

Setting PObj’s

Setting PO’s

Course Mapping

Program Specification

Course Outlines

Prepare performance

criteria

Input from stakeholders

Course Notesand E-

learning materials

Course/teaching

Planning /Developm

ent/ C

uriculum D

esign Stage

TnL Policies and Code of

Practices

Plan made available to students

Incorporating Generic Skills

in T & L

Problem based Learning

Collaborative or Corporative Learning

E-Learning Traditional Approach

Various methods/tools of Assessment and Evaluation

E l ti b L t f h E l ti b St d t d th F lt ll t i

Implem

entation/Deliver/

Teaching Stage

25

Evaluation by Lecturers for each course Evaluation by Students and the Faculty on overall competencies

Technical Competencies Generic Skills Technical Competencies Generic Skills

Develop Assessment Base on Blooms Taxonomy

Develop Performance criteria for each skill

Entry Self AssessmentExit Self Assessment

Peer-evaluationsLearning PortfoliosSurveys/FeedbacksExamination ResultsExternal examiners

External advisor

Concentrate on quizzes, tests and examinations

Group subjects that addressed the generic skills

Forms or records used in the evaluation

Assessment, review

and evaluation stage

P-Obj and Programme Outcomes

• Curriculum Design• Provide resources• Determine Performance

Criteria and Level• Teaching Activities to

STAKEH

OLD

ER’S N

EAN

D IN

STITUTIO

NA

MISSIO

N

3

1. CQI for assessment strategies

2. CQI for curriculum de i e o e

FES

achieve Course Outcomes

Assessment Process/ToolsCollect Data

EDS

AL

1

2design, resources, criteria and teaching activities

3. CQI for P-Obj and PO4. CQI for out-of-class

activities

26

EEDB

ACK

S FRO

M

STAKEH

OLD

ERS

4 Complying EAC-BEM CQI Requirement using UTM CQI Model

Analyse and Report DataDetermine the Gap

16/05/2009

14

• Identify program objectives (P-obj)

• Identify program outcomes

CQI Cycle For P-Objectives and P-Outcomes• Communicate

the results• Identify best

practices• Identify new

project (PO)• Map P-Obj with PO• Map CO with PO• Specify Curriculum content, TnL

approaches, Performance Criteria and level

• Develop assessment plan

PLAN

DOCHECK

ACTION

• Analyse the data • Share results

C lt

project• Make necessary

changes to the plan

• Carry out TnL approaches at course• Compare results with what has been plan using CQI tools

• Review the gap• What can be

learn and need to be monitored

• Carry out TnL approaches at course level

• Implement out-of-class activities• Assess the students learning• External examiners visit• Carry out stakeholders survey• Involvement of external advisors• Collect appropriate data• Carry out Academic Auditing

• Map CO with PO• Specify Course content, TnL

approaches, Performance

CQI Cycle For Course Outcomes Level• Communicate

the results• Identify best

practices• Identify new

project Criteria and level• Develop course assessment

plan

PLAN

DOCHECK

ACTION

• Analyse the data • Share results

C lt

project• Make necessary

changes to the plan

• Carry out TnL approaches at course level

• Assess the students learning• Collect appropriate data

• Compare results with what has been plan using CQI tools

• Review the gap• What can be

learn and need to be monitored

16/05/2009

15

(taken from ABET Training module)(taken from ABET Training module)

(taken from ABET Training module)

16/05/2009

16

Evaluation & Assessment Cycles“A 2-loop Process”

D i O

Determine educationalobjecti es

Determine HowOutcomes will be

Achieved

Determine OutcomesRequired to Achieve

Objectives

objectives

Assess Outcomes/Evaluate

Objectives

Establish Indicators for Outcomes to Lead

to Achievement of Objectives

Determine HowOutcomes will be

Assessed

Input fromConstituencies

Formal InstructionStudent Activities

Evaluation & Assessment Cycles“A 2-loop Process”

D i O

Determine educationalobjecti es

Determine HowOutcomes will be

Achieved

Determine OutcomesRequired to Achieve

Objectives

objectives

Assess Outcomes/Evaluate

Objectives

Establish Indicators for Outcomes to Lead

to Achievement of Objectives

Determine HowOutcomes will be

Assessed

Input fromConstituencies

Formal InstructionStudent Activities

(taken from EAC-BEM Training module)

16/05/2009

17

I wish to thank EACI wish to thank EAC BEM andBEM andI wish to thank EACI wish to thank EAC BEM andBEM andI wish to thank EACI wish to thank EAC--BEM and BEM and everyone who contribute to this everyone who contribute to this material. Special thanks are due material. Special thanks are due

toto

I wish to thank EACI wish to thank EAC--BEM and BEM and everyone who contribute to this everyone who contribute to this material. Special thanks are due material. Special thanks are due

totoAbdul Abdul WahabWahab, , AzlanAzlan, , JailaniJailani, , MazlanMazlan, , MegatMegat JohariJohari,,

MohdMohd. . SalehSaleh JaafarJaafar and Wan and Wan HamidonHamidon

34