Balagtas, Ppango, Reyes, Ubiña, Atweh Rubric For Internationalization

DEVELOPMENT OF A RUBRIC ON THE INTERNATIONALIZATION OF TEACHER EDUCATION INSTITUTIONS IN THE PHILIPPINES AND BEYOND
Marilyn Ubia-Balagtas Bill Atweh Marla C.Papango Zenaida Q. Reyes Marilou M. Ubia
DEVELOPMENT OF A RUBRIC ON THE INTERNATIONALIZATION OF TEACHER EDUCATION INSTITUTIONS(TEIs) IN THE PHILIPPINES AND BEYOND
Outline Rationale
Research Problems Conceptual Framework Significance of the Study Methodologies
Results
Conclusion Recommendations
Rationale of the Study

This study was conceived to have a valid and reliable rubric that gauges the level of internationalization of Teacher Education Institutions (TEIs) in the Philippines and even beyond its borders for them to have a more objective basis in improving their programs and processes for global competitiveness.
Research Problems
1. What are the dimensions and indicators of an internationalized teacher education institution?
2. How valid is the rubric in determining the level of internationalization of teacher education institutions as viewed by experts?
Research Problems (contd.)

3. What is the intra-rater and inter-rater reliability of the rubric in determining the level of internationalization of sampled TEIs?
Conceptual Framework
Figure 1 shows all the components of internationalization based on PNUs areas of commitment as NCTE and the dimensions of internationalization by Padama et. al (2010), which could also be the areas of concern of TEIs in the Philippines and other nations.
Rubric for Internationalization
Significance of the Study

1. The rubric is a user-friendly instrument that will help TEIs in the Philippines gauge their level of internationalization.
Significance of the Study

2. The rubric developed in this study is a more practical and efficient way in determining the areas for improvement in the programs and operations of TEIs for them to reach a desired level of internationalization.
Methodology
Design - descriptive-developmental research
Descriptive research. Describes the process in developing the rubric on the internationalization of TEIs. Developmental research. The outcome of this research is a valid and reliable rubric that can be used by all TEIs in the Philippines and even beyond to gauge the level of internationalization of their institutions.
Data Gathering Procedure

Planning stage - conceptualization of the research proposal by the team of researchers Development stage - designing of the rubric, its components and indicators. Evaluation stage - validation of the rubric and how its reliability was established after it underwent a series of revisions. Finalization stage - final writing of the research report until it is ready for dissemination and utilization.
Participants
Of the eight (8) validators of the rubric, 6 were based in the Philippines and 2 in Australia. They were purposively chosen based on the following criteria: 1. Participants should have been trained in education in other foreign institutions. 2. They have been in two or more countries. 3. They are educators in a teacher training institution. 4. They should be at least a masters degree holder in education.
Instrument
Validators Instrument
a 4-point rating scale that aims to solicit the degree of acceptability of the contents and indicators of an internationalized institution based on the knowledge and experience of the validators
Data Analysis
considered all the ratings given by experts considered every suggestion in the revision of the instrument computed the mean of the ratings given by the validators interpreted mean of ratings acted on the mean of ratings recommended deletion of item/s based on merit
Analysis and Action Taken on the Ratings of the Validators

Mean of Ratings
4.00
Interpretation
Acceptable without any revision
Action Taken
No action taken; the indicator was retained without any revision
3 3.99
Acceptable with minor revision

Acceptable with major revision Not acceptable
The indicator has been modified to a little extent based on the suggestion of any or all of the validators
The indicator has been modified to a great extent based on the suggestion of any or all of the validators The indicator was changed or totally deleted as per suggestion of any one or all of the validators.
2 -2.99
1.0 to 1.99
Results and Discussion

Domains, Dimensions, and Indicators of an Internationalized Teacher Education Institution
Domain
1. Knowledge Creation &Application 2. Quality and Excellence
Dimension
Curriculum and Instruction Research Collaboration Academic Standards and Quality
Mobility and Exchanges for Students and Teachers International and Intercultural Understanding/Networking Cooperation and Development Assistance International Students Recruitment Facilities and Support System Diverse Income Generation
No. of Indicators
10 + 6+ 7+
5+ 5+ 6+ 3+ 7+ 3+
3. Culture of Sharing and Service
4. Growth, Efficiency and Accountability

Research Problem # 1: Domains, Dimensions, and Indicators of an Internationalized Teacher Education Institution
Example: Domain: Knowledge Creation and Application Dimension 1: Curriculum and Instruction
Level of Internationalization
Indicator
Final Rating 1
Internationalized to a little extent
4
Means for Verification
Internationalized to Very High extent
3
Internationalized to High extent
2
Internationalized to some extent
0
Not at all Internationalized
The institution has courses on foreign languages to understand the people of other countries. Examples English Spanish Japanese Korean French Chinese German Others (pls. specify)
Curriculum Guide/ Program Curriculum/C ourse Prospectus
The institution led at least 4 research activities a year done in collaboration with educational institutions or industries in other countries .
The institution led only 3 research activities a year done in collaboration with educational institutions or industries in other countries.
The institution led only 1 research activity a year done in collaboration with educational institutions or industries in other countries.
The institution led NO research activity a year done in collaboration with any educational institution or industry in other countries.
Results and Discussion Research Problem # 1:

Each indicator was described in its level of internationalization in terms of its quality and quantity of evidence. It is presented in the form below: Domain: Knowledge Creation and Application Dimension 1: Curriculum and Instruction
Indicator
4
3
Internationalize d to High extent
2
Internationalize d to some extent
1
Internationalize d to a little extent
0
Not at all Internationalize d
Final Rating
The institution has courses on foreign languages to understand the people of other countries. Examples English Spanish Japanese Korean French Chinese German
4 Internationalized to a very high extent

Curriculum /Course Prospectus
Curriculu m Guide/ Program
3 Internationalized to a high extent

Indicator
4
3
2
1
0
Final Rating
The institution has courses on foreign languages to understand the people of other countries. Examples English Spanish Japanese Korean French Chinese German Others (pls.
4 Internationalized to a very high extent

3 Internationalized to a high extent

Indicator
4
3
2
1
0
Final Rating
2 Internationalized to some extent

1 Internationalized to a little extent

Indicator
4
3
2
1
0
Final Rating

Not all Internationalized

Research Problem # 1:
Each indicator was described in its level of internationalization in terms of its quality and quantity of evidence. It is presented in the form below:
Example:
Domain: Knowledge Creation and Application Dimension 2: Research Collaboration
Indicator
Internationalize d to Very High extent
The institution leads research activities done in collaboration with educational institutions or industries in other
Memorandum of Agreement or Understan ding Completion Report
The institution led at least 4 research activities a year done in collaboration with educational institutions or industries in other
1. The institution leads research activities done Level of Internationalization in collaboration with 0 3 2 1 educational institutions or industries in other countries.
Internationalize d to High extent Internationalize d to some extent Internationalize d to a little extent
Final Rating
The institution led only 3 research activities a year done in collaboration with educational institutions or industries in other
The institution led only 1 research activity a year done in collaboration with educational institutions or industries in other

Example:
Domain: Knowledge Creation and Application Dimension 2: Research Collaboration Level of Internationalization
Indicator
4
Memorandum of Agreement or Understanding Completion Report

Internationalize d to some extent Internationalize d to a little extent
Final Rating

Example:
Indicator
4
The institution led at 3 2 1 0 least 4 research activities a year done in collaboration with educational institutions or industries in other countries.
Internationalize d to some extent Internationalize d to a little extent Not at all Internationalize d
Final Rating

The institution led only 3 research Dimension 2: Research Collaboration Final Level of Internationalization activities a year Rating 4 3 2 0 Indicator done in1 collaboration with educational institutions or industries in other countries.
Example:
Domain: Knowledge Creation and Application

Example:
Indicator
4
3
2
1
0
Final Rating

Example:
Indicator
4
3
2
1
0
Final Rating

Example:
Indicator
4
3
2
1
0
Final Rating
The institution led NO research activity a year done in collaboration with educational institutions or industries in other countries.
Research Problem # 1: Domain/ Dimensions
Indicators
Domain 1: Knowledge Creation and Application
The institution:
1.1 has an Inclusive Curriculum 1.2 has courses on foreign languages to understand the people of other countries. 1.3 offers courses that may serve as a venue for understanding and appreciation of the culture of other countries. 1.4 has customized programs that are responsive to the demands of different sectors, agencies or organizations here and abroad. 1.5 has programs delivered in different modalities that could allow students from 1. other countries to take courses at their own time and place . Curriculum 1.6 has information and communication technologies that could facilitate and efficient and effective learning. Instruction 1.7 has its own laboratory to test theories or theorize from experiences. 1.8 provides the students special learning experiences where they could appreciate the culture of other tribal groups/classes or races. 1.9 has created a joint curriculum or course program with foreign institutions/universities. 1.10 updates regularly the syllabus of course offerings to integrate the new trends and address pressing issues around the globe that have implications to education. 1.11 Others, not captured by the list
Domain/ Dimensions
Indicators
Domain 1: Knowledge Creation and Application The institution:
2.
Research Collaboration
2.1 leads research activities done in collaboration with educational institutions or industries in other countries. 2.2 has faculty, students or administrative staff who participate in international studies as researchers. 2.3 has faculty, students or administrative staff who participate in international studies as respondents. 2.4 conducts studies on international comparative education to understand differences between and among nations. 2.5 organizes international conferences where research studies are presented or disseminated. 2.6 contributes to the growing body of knowledge recognized abroad.
Domain/ Dimensions Domain 2: Quality and Excellence
Indicators
The institution: 3.1 is level three accredited by nationally known accrediting body 3.2 benchmarks its curricular or extra-curricular activities with national international standards . 3.3 has become the benchmark of other institutions. 3.4 has faculty and administrators who have high profile (i.e. recognized for their expertise here and abroad). Academic Standards 3.5 has faculty, administrators and staff with special trainings or and Quality exposures abroad.(e.g . conferences) 3.6 is recognized in international society (e.g. cited in international publications as a good institution for learning). 3.7 has faculty and administrators who are recipients of scholarships, fellowships or grants abroad. 3.8 Others, not captured by the list 3.
Domain/ Dimensions
Indicators
Domain 3: Culture of Sharing and Service The institution:
4.1 has students studying in other institutions abroad for some of their courses delivered online or face-to-face. 4. Mobility 4.2 has foreign students studying in the institution taught online or faceand Exchanges to-face. for Students 4.3 has faculty members who taught abroad for faculty exchange or and served as consultants abroad. Teachers 4.4 has faculty from foreign institutions who are teaching some courses for the students of the institution. 4.5 has visiting professors from other countries that do academic works in the institution. 4.6 Others, not captured by the list
Domain/ Dimensions
Indicators
Domain 3: Culture of Sharing and Service The institution: 5.1 has Memorandum of Agreements or Memorandum of Understandings with institutions from other countries for any international or intercultural understanding/networking. 5.2 has twinning programs with foreign institutions. 5.3 participates in international assemblies or activities for the promotion of ones culture. 5.4 organizes both curricular and/or extra-curricular multicultural activities. 5.5 and its faculty, students or administrative staff are active members of international organizations. 5.6 has a Center for Multicultural Education/ Foreign Students Organization that serves as venue for the understanding of the culture of different nations 5.7 has accredited foreign students organizations. 5.8 Others, not captured by the list
5.
International and Intercultural Understandi ng/ Networking
Domain/ Dimensions
Indicators
Domain 3: Culture of Sharing and Service

The institution:
6.1 is engaged in international academic networks/alliance, consortia, or links with other universities and colleges in other countries. 6.2 is engaged in formulating policies to achieve academic, scientific, economic, technological or cultural ties with other colleges/ universities abroad or NGOs 6. on education ( e.g. ASEAN,APEC,NGO) Cooperation 6.3 receives funding for the promotion of cultural understanding / and international learning/ecumenical or inter-religious activities. Development 6.4 has international development projects commissioned by international Assistance agencies ( World Bank, UNESCO, ADB, -Aid, AFAP, etc.) 6.5 opens its programs to foreign students from less developed countries that have satisfied the entry requirements. 6.6 offers scholarships/grants for foreign students who come from less developed countries. 6.7. Others, not captured by the list
Domain/ Dimensions
Indicators
Domain 4: Growth, Efficiency, and Accountability The institution:

7.
International Students Recruitment
7. 1 has educational programs, course offerings, syllabi and policies for admission and retention of students that are available online. 7.2 has responsive computerized or online system in recruiting, screening, enrolling international students. 7.3 has a responsive system to address inquiries of prospective foreign students.
7.4 Others, not captured by the list
Domain/ Dimensions
Indicators
Domain 4: Growth, Efficiency, and Accountability The institution:

8.1 has classrooms equipped with the state-of-the-art technologies 8.2 has its own library with the convergence of complete and updated materials/facilities both print and non-print. 8. Facilities 8.3 has sufficient number of licensed fulltime librarians in all its sections. and Support 8.4 has a dormitory for local and foreign students with complete amenities. System 8.5 has provision for guidance and counseling of foreign students. 8.6 provides medical and dental support for its local and foreign faculty members, staff, and students . 8.7 has amenities/support system for its foreign students, faculty, and visitors
8.8 Others, not captured by the list
Domain/ Dimensions
Indicators
Domain 4: Growth, Efficiency, and Accountability The institution: 9.1 has generates alternative sources of income like offering educational programs or creating publications patronized by international 9. Diverse institutions and industries. Income 9.2 has income generating projects (IGP) in partnership with foreign Generation agencies or institutions. 9.3 has a specific budget coming from its partner educational institutions from other countries. 9.4 Others, not captured by the list
VALIDATION RESULTS OF THE RUBRIC

No. Domain/ Dimension
No. of Indicators
Action Taken
Domain 1. Knowledge
Creation and Application 10 All indicators were revised to a little extent as per suggestion of the validators Three (3) had minor revisions as per suggestion of the validators while the other three (3) did not have revisions.
Curriculum and Instruction Research Collaboration
2
Domain
2. Quality and Excellence Academic Standards and Quality 7 All indicators were accepted without revision

No. of Indicators
Action Taken
Domain 3: Culture of Sharing and Service Mobility and Exchanges for Students and Teachers International and Intercultural Understanding/ Networking Cooperation and Development Assistance One (1) was modified and another one (1) was deleted for it can be encompassed by another indicator under the same domain and dimension as per suggestion of the validators. The other three (3) indicators did not have any revisions. Two (2) indicators were modified as per suggestion of the validators while the other three (3) were not revised at all.
5 -1
6 +1
One (1) was modified as per suggestion of the validators while five (5) were not revised at all. Then one (1) was added as a result of the reclassification of an indicator in domain 9.

No. of Indicators
Action Taken
Domain 4. Growth, Efficiency, and Accountability
International Students Recruitment Facilities and Support System
7+2
Diversity Income Generation
3-1
All indicators were accepted without any revision Two (2) more indicators were added based on the suggestions of the validator to split into two statements two indicators, which were complex in their scope. Six (6) indicators were revised as per suggestion of the validators. One (1) was revised and reclassified under domain 6 as per suggestion of the validators
Reliability of the Rubric

Summary of Intra-Rater Reliability Coefficients
Raters Rater 1 Rater 2 Rater 3 Rater 4 Rater 5 TEI in the Philippines TEI in Australia
Rater 1 0.88 -
Rater 2 0.77 -
Rater 3 0.67 -
Rater 4 0.71
Rater 5 0.62
The coefficients that range from 0.62 to 0.88 , which are all significant at 0.01 level of confidence indicate that the rubric has moderate to high intra-rater reliability.

Even if the raters rated the same institution twice using the same rubric for an interval of one week, reliability was still not that high (if target is at least 0.85 reliability coefficient for standardized instrument for all its users) . This could be due to the absence of the actual data or documents for verification when the tryout of the rubric was done . Although the four of the raters were personally engaged in the accreditation of their institution at the time that they rated also its level of internationalization, the raters still need to have the actual listing of data obtained from the suggested means for verification in order to arrive at a more reliable rating of their institution.
Reliability of the Rubric Summary of Inter-Rater Reliability Correlations

Raters Rater 1 Rater 2 Rater 3 Rater 4
Sampled TEI in the Philippines Rater 1 Rater 2 Rater 3 Rater 4

0.52 0.69 0.62 0.52 0.76 0.53 0.69 0.76 0.55 0.62 0.53 0.55 -
The table shows that the inter-rater reliability coefficients for the rubric range from 0.52 to 0.76 indicating moderate to high inter-rater reliability.

Although the raters were asked to rate the level of internationalization of the whole University right after they went through the accreditation of their respective programs, which is a process that also requires perusal of actual documents, their differences in their ratings to some indicators of internationalization may just be limited to their knowledge of the programs and processes in their respective college and not really of their knowledge of the entire University.

Further analysis was done to know how the four (4) raters judged the level of internationalization of their institutions.
Descriptive Statistics of the Ratings of the Five Raters to the Level of Internationalization of Sampled TEIs
Sampled Institution
Statistics/ Interpretation Mean Standard Deviation
Over-all Level of Internationalization
Philippines
Rater 1 1.8868 1.21940 2 Rater 2 2.0000 1.30089 2 Rater 3 2.0943 1.49697 2 Rater 4 2.4717 1.17020 2
Australia
Rater 5 3.57 0.67 4
Internationalized to a Very high extent
Interpretation
Internationalized to Some Extent

Descriptive Statistics of the Ratings of the Four Raters to the Level of Internationalization of a Sampled TEI in the Philippines Sources of Sum of Mean Squares Square df F Sig. Variation
Between Groups Within Groups Total
8.642
2.881 1.694
0.169
Not significant at 0.05 level
353.660 362.302
208 211
1.700
The result reveals that the difference in the ratings of the four raters to the level of internationalization of their institution is not significant, which means that they rated their institution consistently as internationalized to some extent (Level 2 out of 4 levels of internationalization).
Conclusions
1. The results reveal that the researchers who were the ones who developed the rubric with reference to the framework of Padama et.al (2010) came up with acceptable descriptions of each level of internationalization. Their own exposures to TEIs in other countries have enabled them to capture the qualities of a worldclass TEI as viewed by the validators who accepted almost all the descriptions they provided in the rubric.
Conclusions
2. The test of validity of the rubric in internationalizing TEIs show a good start considering its novelty. Validators who have been exposed to many TEIs in the world and who have travelled to different countries as a learner, observer, and a consultant confirmed the acceptability of the indicators of internationalization and the descriptions for each level of internationalization.
Conclusions
3. The rubric, considering its novelty, has moderate to high intra-rater and inter-rater reliability which is a good start for the instrument in gauging the level of internationalization of any TEI in the country and even beyond its borders.
Recommendations
1. Since the purpose of this development study is to have a standardized rubric that
encompasses all elements of an internationalized TEI not only as viewed in the Philippines but also beyond its borders, further validation of experts from other countries particularly from normal schools at least in the Asia-Pacific region is recommended.
Recommendations
2. Follow up study is also needed to test its criterion-related validity, where the results of the level of internationalization of the institution using the rubric could be compared to the ratings of the institution using another instrument, for example, the instrument used by the accreditors of institutions with programs in teacher education.
Recommendations
3. The rubric should have follow-up tryouts using actual perusal of documents for verification from other normal schools at least in the Asia-Pacific region to further test its reliability until it gets at least 0.85 reliability coefficients from all users so that it could really claim validity and reliability in gauging the level of internationalization of TEIs not only in the Philippines but also beyond its borders.
Thank you to all our validators and those who helped us prepare this paper! A special gratitude is extended to Dr. Edith Padama, Dr. Twila G. Punsalan, and Prof. Jean B. Borlagdan for their inputs to this research.
Copyright 2011 by Balagtas, M.U. et.al this September 2011. Anyone interested to refer to this material or to its instrument may ask permission from the authors or cite them as a source.
Contact Numbers: mobile: 09209532926 or email at mubalagtas@yahoo.com

Balagtas, Ppango, Reyes, Ubiña, Atweh Rubric For Internationalization

Transféré par

Informations du document

Description originale:

Titre original

Copyright

Formats disponibles

Partager ce document

Partager ou intégrer le document

Options de partage

Avez-vous trouvé ce document utile ?

Ce contenu est-il inapproprié ?

Droits d'auteur :

Formats disponibles

Balagtas, Ppango, Reyes, Ubiña, Atweh Rubric For Internationalization

Transféré par

Droits d'auteur :

Formats disponibles

DEVELOPMENT OF A RUBRIC ON THE INTERNATIONALIZATION OF TEACHER EDUCATION INSTITUTIONS IN THE PHILIPPINES AND BEYOND

Rationale of the Study

Research Problems (contd.)

Rubric for Internationalization

Significance of the Study

Significance of the Study

Data Gathering Procedure

Analysis and Action Taken on the Ratings of the Validators

Acceptable with minor revision

Results and Discussion

3. Culture of Sharing and Service

4. Growth, Efficiency and Accountability

Results and Discussion

Curriculum Guide/ Program Curriculum/C ourse Prospectus

Results and Discussion Research Problem # 1:

4 Internationalized to a very high extent

Curriculu m Guide/ Program

3 Internationalized to a high extent

Results and Discussion Research Problem # 1:

4 Internationalized to a very high extent

Curriculu m Guide/ Program

3 Internationalized to a high extent

Results and Discussion Research Problem # 1:

2 Internationalized to some extent

Curriculu m Guide/ Program

1 Internationalized to a little extent

Results and Discussion Research Problem # 1:

Curriculu m Guide/ Program

Not all Internationalized

Results and Discussion

Domain: Knowledge Creation and Application Dimension 2: Research Collaboration

Means for Verification

Internationalize d to Very High extent

Memorandum of Agreement or Understan ding Completion Report

Not at all Internationalize d

Results and Discussion

Means for Verification

Internationalize d to High extent

Memorandum of Agreement or Understan ding Completion Report

Memorandum of Agreement or Understanding Completion Report

Not at all Internationalize d

Results and Discussion

Internationalize d to High extent

Memorandum of Agreement or Understan ding Completion Report

Results and Discussion

Domain: Knowledge Creation and Application

Means for Verification

Internationalize d to Very High extent

Internationalize d to High extent

Internationalize d to some extent

Internationalize d to a little extent

Not at all Internationalize d

Memorandum of Agreement or Understan ding Completion Report

Results and Discussion

Results and Discussion

Results and Discussion

Research Problem # 1: Domain/ Dimensions

Domain 1: Knowledge Creation and Application

Domain 1: Knowledge Creation and Application The institution:

Domain 3: Culture of Sharing and Service The institution:

Domain 3: Culture of Sharing and Service

Domain 4: Growth, Efficiency, and Accountability The institution: