16th Annual Conference

Evaluating Teacher and Program Effectiveness November 22-23, 2012

From the Conference Chairs Dear CTELT 2012 Participants, We would like to welcome you to the 16th Annual Current Trends in English Language Testing Conference. The TESOL Arabia Testing, Assessment and Evaluation SIG and the organizing committee from the Dubai Chapter have worked especially hard to make this CTELT Conference a success. We are pleased to be able to feature four prominent testers and teacher educators as our keynote speakers: Dr Deena Boraie from American University in Cairo in Egypt; Peter Davidson from Zayed University in the UAE; Dr Amanda Howard from British University in Dubai in the UAE and Dr Phil Quirke from Madinat Zayed and Ruwais Higher Colleges of Technology in the UAE. Each will share their expertise and insights about the importance of teacher and program effectiveness. We would like to express our gratitude to our hosting institution, HCT Dubai, Men's Campus. Thanks also go to our sponsors: TESOL Arabia and CORE Education and Consulting Solutions. A special vote of thanks goes to our presenters who provide the important content of this conference. Without them this event would not have been possible. Enjoy the conference! Regards, Dr Christine Coombe & Peter Davidson CTELT Conference Chairs TAE SIG Co-chairs

CTELT 16 Conference: Thursday, November 22, 2012

PLACE/TIME 8:30 9:00 9:00 10:00 10:00 11:00 Plenary L115 Room: L115 Room: L217 REGISTRATION & COFFEE: Room: Center of Engineering Excellence (COEE 103 A & B)

Plenary L115
Concurrent Sessions

Outside L115 Measuring Teacher and Program Effectiveness Effectively Deena Boraie, The American University of Cairo The Good, the Bad and the Ugly in Teacher Evaluation Peter Davidson, Zayed University Evaluating Program Effectiveness: The HCT iPad Project Matt Robby & Christina Gitsaki Using Eportfolios in Ongoing Assessment Jonathan Turner Suhair Al-Alami Testing in the Communicative Classroom: Points to Consider

11:00 12:00

12:00 1:30 1:30 2:30 Plenary L115 Concurrent Sessions

LUNCH: DMC Cafeteria

Teacher Effect: Teacher Effectiveness Phil Quirke,Madinat Zayed & Ruwais Colleges, Higher Colleges of Technology Testing Local Rachel Lange Discussion Session: Developing a Research Agenda for the TESOL Arabia Testing, Assessment and Evaluation SIG Christine Coombe, Beth Wiens & Deena Boraie

2:30 3:30

CTELT 16 Conference: Friday, November 23, 2012

PLACE/TIME 8:30 - 9:00 9:00 10:00 Plenary L115 Room: L115 Room: L217 Room: Center of Engineering Excellence (COEE 103 A&B)

REGISTRATION & COFFEE: OUTSIDE L115

Evaluating Teacher Effectiveness: The Way Forward Amanda Howard, British University in Dubai

10:00 - 11:00 Concurrent Sessions

COFFEE BREAK & NETWORKING

11:00 - 12:00

What Can the CEPA Reconceptualizing L2 Teacher Exam Tell Us About Education Programs: Is There the Effectiveness of a Secret Mix? Foundations Programs Aymen Elsheikh in the UAE? Ryan Gjovig & Paul Jaquith

Personalizing Language Needs: The Benefits of Individual Diagnostic Feedback in Tertiary English Medium Education Mick King

Concurrent Sessions 12:00- 1:00

Measuring Teacher and Program Effectiveness Effectively Deena Boraie

Faculty Perceptions of Student Evaluations of Teaching Edith Flahive

Understanding Test Reliability and Validity: A Guide to Better Improving Your Assessment Paul Dessoir

Plenary Sessions
Thursday, November 22nd, 9:00 - 10:00 a.m. L115

Measuring Teacher and Program Effectiveness Dr. Deena Boraie, The American University of Cairo, Egypt The impact of any individual teacher or program on students learning can never be truly isolated. Therefore, the question of how effective a particular teacher or program is inherently difficult to answer because not everything can be translated into the classroom and student learning directly. In this plenary, the presenter will examine different aspects of the effectiveness of the current systems of measuring effectiveness of teachers and programs and will suggest a way forward balancing between the need to demonstrate effectiveness and the cost and complexity of doing so effectively.

Dr. Deena Boraie is the Associate Dean for Instructional Affairs at the School of Continuing Education at the American University in Cairo. She currently serves as President-Elect of the TESOL International Association and is a member of the editorial board of Language Testing, an international peer-reviewed journal.

Thursday, November 22nd, 10:00-11:00 a.m. L115

The Good, the Bad, and the Ugly in Teacher Evaluation Peter Davidson, Zayed University, Dubai, UAE

Teacher evaluation is a contentious issue. Because teaching is so multi-dimensional, it is difficult to evaluate teaching but not impossible. In this talk we will look at the characteristics of an effective teacher, discuss some of the problems with current evaluation tools, and look at some recent developments in Teacher Evaluation.

Peter Davidson teaches at Zayed University in Dubai, having previously taught in New Zealand, Japan, the UK and Turkey. He recently co-edited The Fundamentals of Language Assessment (2009), and The Cambridge Guide to Second Language Assessment (2012). He is particularly interested in language testing, faculty evaluation, and 21st century education.

Thursday, November 22nd, 1:30-2:30 p.m. L115

Teacher Effect: Teacher Effectiveness Dr. Phil Quirke, Madinat Zayed & Ruwais Higher Colleges of Technology, UAE In this presentation, Phil explores the positive effects teachers have on learners, learning and the learning environment. Drawing on a variety of examples from his own experience as a teacher and teacher educator Phil questions the term "Teacher Effectiveness" whilst suggesting there are key traits to be seen in what have been called "master practitioners". Involving the audience throughout, this plenary aims to trigger reflection, enquiry and hopefully not a little discussion.

Phil Quirke is Director of the Madinat Zayed & Ruwais Colleges, Higher Colleges of Technology in the UAE. He has been in ELT Leadership positions for fifteen years and considers his key role as a catalyst for creating an environment where learning and teaching can creatively thrive. He has published on areas as diverse as face, action research, appraisal and journalling. He runs a support site for teachers doing further studies at the postgraduate level: http://www.philseflsupport.com. For this presentation he has drawn heavily on his books, An Exploration of Teacher Knowledge and Reflective Writing (edited with Jill Burton, Joy Peyton and Carla Reichmann), which is available free online at http://www.tesl-ej.org/wordpress/books/.

Friday, November 23rd, 9:10-10:00 a.m. L115

Evaluating Teacher Effectiveness: The Way Forward Dr Amanda Howard, British University in Dubai, UAE

Assessment is a key component of education at all levels, and teacher evaluation is an important part of this. Stakeholders need to know that the interests of their learners are being served by those who manage the learning environment so that effective teaching and learning can take place, and specific mechanisms are required in order to provide such information. Teachers can also benefit from the assessment process, as it can reinforce belief in pedagogic practices and personal abilities, and provide them with an opportunity to interact with other staff on a professional level. However, many of the evaluation practices currently employed to appraise teachers in their workplaces are axiomatic, as their origin is historically, rather than practically, based: they do not necessarily involve all the partners in typical classroom interaction, or relate to professional development. This session will use research data to investigate some of the many factors that have an impact on teacher evaluation, such as the Observers Paradox (Labov 1972), the timing of assessment, and the identity of the assessor. The main purpose in doing this is to identify those which could be considered to be most relevant in terms of reflecting an individuals regular pedagogic practice in the classroom, whilst providing a clear link to ongoing professional development.

Amanda Howard PhD is an Assistant Professor and the TESOL coordinator in the Faculty of Education at the British University in Dubai, responsible for modules and thesis supervision at both EdD and MEd levels. She has many years of experience working as a teacher, teacher trainer and lecturer in the Middle East, and as a lecturer for several Russell Group universities in the UK. Her research interests include teacher education and professional development, observation and feedback, discourse analysis, and teaching Young Learners. amanda.howard@buid.ac.ae

Concurrent Sessions Thursday, November 22nd 11:00-12:00 p.m. Evaluating Program Effectiveness: The HCT iPad Project Dr. Matt Robby & Dr. Christina Gitsaki, Higher Colleges of Technology Room L115 Educational program evaluation is critical to determining the impact of the program on learning and teaching. This session will review the iPad project at the Higher Colleges of Technology and present a comprehensive plan for the evaluation of the project and its impact on English teaching and learning. Preliminary results from the qualitative data collection will be discussed. Dr. Matthew A. Robby is the Chair of Applied Research for the Higher Colleges of Technology. His expertise includes the evaluation of programs, learning innovations, and promoting change and effectiveness. Dr. Christina Gitsaki is the Associate Dean of English at the Higher Colleges of Technology. She has published widely in the area of language teaching and educational technologies.

Using Eportfolios in Ongoing Assessment Jonathan Turner, HCT Western Region, UAE Room L217 In this presentation we will show how eportfolios can be built and used as ongoing assessment tools. With the range of artifacts available for inclusion in an eportfolio, we will show how all the language skills can be evidenced in an eportfolio, giving a more rounded view of student progress. Jonathan Turner is a teacher and teacher trainer (CELTA and DELTA) with HCT Western Region. He has an MSC in TESOL and is completing an MA in Online and Distance Learning.

Testing in the Communicative Classroom: Points to Consider Suhair Al-Alami, Al-Ghurair University, UAE Room: Center of Engineering Excellence 103 A & B What are the factors that inhibit EFL students progress within the area of acquiring communicative competence? In the presenters opinion, a variety of factors contribute, the most prominent of which being the evaluation system. The paper, therefore, seeks to pinpoint a number of points delineating effective language assessment in the communicative classroom. Suhair Al-Alami holds a PhD in Applied Linguistics from Ain Shams University, Egypt, and another PhD in Applied Linguistics from Aston University, UK. Dr Al-Alami works at AlGhurair University, Dubai.

Concurrent Sessions Thursday, November 22nd 2:30-3:30 p.m.

Testing Local Rachel Lange, UAE Ministry of Higher Education and Scientific Research, UAE Room: L115 This presentation focuses on the phenomenon of test localization. It will look at the advantages and disadvantages of producing a local solution for placement or evaluation rather than using an international commercial exam. The Common Educational Proficiency Assessment (CEPA) will serve as a case study in test localization, and the presenter will look at how this test responds to factors in the UAE educational environment and adopts standards and best practices in order to remain as effective as a commercially-produced test.

Rachel Lange holds a BFA from Alfred University and an MA in Applied Linguistics from the University of Pittsburgh. She currently supervises the development of CEPA assessment materials for the UAE Ministry of Higher Education and Scientific research and consults on materials development projects for language teaching, learning, and assessment.

Developing a Research Agenda for the TESOL Arabia Testing, Assessment & Evaluation SIG Dr Christine Coombe, HCT Dubai, Mens Campus Dr Deena Boraie, The American University of Cairo, Egypt Beth Wiens, Zayed University Room: L217 Join this discussion group if you are interested in helping conceptualize and draft the 2012-13 Research Agenda of the TESOL Arabia Testing, Assessment and Evaluation SIG.

Dr Christine Coombe has a Ph.D in Foreign/Second Language Education from The Ohio State University. She is currently on the English faculty of HCT Dubai, Mens Campus. Christine is co-editor or co-author of numerous publications on language and assessment including: Assessment Practices (2003, TESOL Publications), Practical Guide to Assessing English Language Learners (2007, University of Michigan Press), Evaluating Teacher Effectiveness in EF/SL Contexts (2007, UMP), and The Cambridge Guide to Second Language Assessment (2012, Cambridge University Press). She served as TESOL President (2011-2012). Dr. Deena Boraie is the Associate Dean for Instructional Affairs at the School of Continuing Education at the American University in Cairo. She currently serves as President-Elect of the TESOL International Association and is a member of the editorial board of Language Testing, an international peer-reviewed journal. Beth Wiens teaches in the Academic Bridge Program at Zayed University in Dubai, United Arab Emirates. She is a past president of TESOL Arabia and has chaired the TESOL Arabia Conference several times, most recently in 2012. Her interests include assessment, curriculum and materials development, and integrating technology into the classroom.

Concurrent Sessions Friday, November 23rd 11:00 a.m. - 12:00 p.m. What can the CEPA exam tell us about the effectiveness of foundations programs in the UAE? Ryan Gjovig, Ministry of Higher Education & Scientific Research, UAE Paul Jaquith, Higher Colleges of Technology, UAE Room: L115 In the 2011-2012 academic year, the CEPA-English exam was used by the HCT foundations program to track student performance and progress. More than 5000 students were assessed during this period, including 1750 who had three separate measurements of their English ability at three points in time (High school spring semester, January 2012, and June 2012). This presentation will discuss the findings of the project, and implications for foundations programs in the UAE. Ryan Gjovig is the Head of the CEPA department at the Ministry of Higher Education & Scientific Research. He holds a BA in Communications from Harding University, and an MA in TESOL/TEFL from the University of Birmingham, England. Paul Jaquith is the Associate Academic Dean of Foundations for the Higher Colleges of Technology. He holds an MA in TESOL from Columbia University and is a doctoral candidate in Language Testing at Roehampton University in London.

Reconceptualizing L2 Teacher Education Programs: Is There a Secret Mix? Dr Aymen Elsheikh, Qatar University, Qatar Room: L217 What are the components needed in a teacher education program in order to help in preparing effective teachers? This presentation will attempt to answer this question by highlighting the literature on the reconceptualization of teacher knowledge as well as drawing on the presenters recent empirical study in an EFL context. Dr. Aymen Elsheikh is a lecturer at Qatar University. He is also TESOL Sudans Vice President. His research interests include teacher education and the socio-political and cultural contexts of ELT.

Personalizing language needs: The benefits of individual diagnostic feedback in tertiary English medium education Mick King, Middlesex University Dubai, UAE Room: Center of Engineering Excellence 103 A & B Tertiary level English as a medium of instruction programmes often assume language proficiency despite low language entry requirements, so although students struggle in assessments, little feedback is given on language errors. This action research project employed individual diagnostic feedback to reduce students errors, thereby raising self-confidence ahead of ensuing assessments. In Mick Kings 25-year TESOL career he has published and presented on various topics and currently combines teaching MA TESOL and Education with EdD study at Exeter University, UK.

Concurrent Sessions Friday, November 23rd 12:00-1:00 p.m. Measuring Teacher and Program Effectiveness Effectively Dr Deena Boraie, American University of Cairo, Egypt Room: L115 The impact of any individual teacher or program on students learning can never be truly isolated. Therefore, the question of how effective a particular teacher or program is inherently difficult to answer because not everything can be translated into the classroom and student learning directly. In this plenary, the presenter will examine different aspects of the effectiveness of the current systems of measuring effectiveness of teachers and programs and will suggest a way forward balancing between the need to demonstrate effectiveness and the cost and complexity of doing so effectively. Dr. Deena Boraie is the Associate Dean for Instructional Affairs at the School of Continuing Education at the American University in Cairo She currently serves as President-Elect of the TESOL International Association and is a member of the editorial board of Language Testing, an international peer-reviewed journal.

Faculty Perceptions of Student Evaluations of Teaching Edith Flahive, Abu Dhabi Mens College, UAE Room: L217 This presentation will examine the effectiveness of one component of the performance appraisal system used at a tertiary college in the UAE; Student Evaluations of Teaching (SETs), from the perspective of faculty members. Implications for learning and teaching will be considered, and recommendations made for training students in the use of evaluation ratings systems.

Edith Flahive (B.Ed, RSA DELTA, MA TESOL, MBA Educational Leadership and Management), teaches English at Abu Dhabi Mens College. Her research interests include learner autonomy, classroom methodologies, and educational leadership.

Understanding Test Reliability and Validity: A Guide to Better Improving your Assessment Paul Dessoir Room: Center of Engineering Excellence COEE 103 A & B Reliability and validity are very important concepts in assessment. However, many teachers have a vague understanding of what they are and why they are important. This presentation will review the two concepts, emphasize their importance and go over practical ways to help teachers better improve the reliability and validity of their assessments. Paul Dessoir has been teaching for over 16 years in both Asia and the Gulf. His interests are mainly focused on assessment, including both alternative and traditional assessment.