Using a mixed methods research design to deconstruct the nature of low academic performance in primary schools facing exceptional

challenge within the Republic of Trinidad and Tobago Jerome De Lisle School of Education, University of the West Indies, St. Augustine Abstract MacBeath et al. (2005) defined schools facing exceptionally challenging circumstances as institutions confronted by complexity resulting from higher levels of poverty, disadvantage, and turbulence among student, parent and staff, thereby threatening school performance. In the Trinidad and Tobago primary school system, this definition was operationalized by locating poorly performing schools located in disadvantaged areas that also experienced higher numbers of economically disadvantaged students. Overall, the study employed an explanatory sequential mixed methods research design, with a quantitative Phase I project and a mixed methods Phase II project. The Phase I project was designed to identify and characterize schools facing exceptional challenge and Phase II was a multi-site case study of three selected institutions, two single-sex and urban, one co-education and rural. The Phase II project used an embedded qualitative-dominant mixed methods design, with multiplemethods. However, qualitative themes were given priority. Metainferences were of three kinds: unique to one method, corroborative, or contradictory. It was not possible to resolve all contradictory findings, pointing to findings providing different lenses on the phenomenon. Overall, the evidence suggested that mechanisms and processes associated with exceptional challenge as operationalized in Trinidad and Tobago were complex and context dependent. Sustainable school improvement would require not just leadership training but also integrated services and interlaced interventions targeting the multiple levels and multiple deficits. Key words Mixed Methods Schools facing challenge Poverty concentration School effectiveness and improvement National Assessments of Educational Achievement

Using a mixed methods research design to deconstruct the nature of low academic performance in primary schools facing exceptional challenge within the Republic of Trinidad and Tobago Rationalizing the concept of challenging context in the South Schools confront a variety of external contexts, some more challenging than others (Harris et al., 2006; Levin, 2006; Michalak, 2009). These different contexts impinge on the very nature and purpose of schooling, the pathway to successful school improvement, essential processes such as leadership and teaching-learning, and the attainability of national targets set for school improvement (MacBeath et al., 2005; Jacobson, 2008). For schools within countries of the developing world, data from international assessments suggest that variations in contexts are much higher (Mullis, Martin, Kennedy, & Foy, 2007). These variations are attributable to uneven national development, the absence of policies fostering equity, and the insufficiency of essential human and physical resources. There is a large body of work on the relationship between contexts of poverty and the underperformance of schools in the United States, Canada, and the UK (Ylimaki, Jacobson & Drysdale, 2007). However, the issue has not been extensively studied within small developing states. Such a study can address the transferability of key school effectiveness constructs contexts (Harber & Muthukrishna, 2000). Trinidad and Tobago provides an especially interesting case for investigating the nature of challenging contexts. Although now classified as a high-income country, the legacy of unequal development has persisted. This inequity has been identified as a notable problem in the English speaking Caribbean and in Trinidad and Tobago (World Bank, 1993; World Bank, 1995). Poverty concentration in Trinidad and Tobago schools arises from both geographic location and the operation of the education market (De Lisle et al., 2009). At the primary school level, there are various choice options in the education market, including private, denominational or government institutions. However, private schools are not necessarily the preferred choice among the socially advantaged because public schools can be equally high performing. The great majority of public primary schools are, in fact, managed by the different denominations (MacKenzie, 1991). The existence of this complex education market has a downside with many low performing primary schools being vilified and rejected. The Ministry of Education has recently labelled some schools as underperforming based on the number of low performing students in the high stakes Secondary Entrance Examination. Several labels have been used to describe schools performing below expectations within accountability systems. However, not all descriptors give adequate attention to the issue of context. One approach might be to label such schools as ineffective or underperforming (Van de Grift and Houtveen, 2006; Houvteen et al., 2007). Muijs et al (2004) focused more on context in the label schools in socioeconomically disadvantaged areas. The label high poverty-low performing (HP-LP) is perhaps one of the more useful because it captures both comparative underperformance and context. High poverty contexts might refer to a variety of challenging contexts, with multilingual (Pretorius & Currin, 2010) and urban dimensions (Jacobson, 2008). Ainscow, Muijis and West (2006) used the term schools facing challenge to describe schools confronted by location, history, and pupil and parental attitudes. MacBeath et al. (2005) added an important dimension of schools at the extreme end of the continuum of contexts, with exceptionally challenging defined as conditions of poverty, disadvantage, and turbulence among student, parent and staff that is significantly higher than that of other schools.

Evidence and context within the Anglophone Caribbean Policy formulation in Caribbean education systems has traditionally suffered from the lack of high-quality, contextualized data. This lack of indigenous knowledge includes evidence from both empirical data and qualitative case studies. Crossley (2008) recommended the use of qualitative approaches to gather contextualized evidence within small states. However, given the possibility of significant variations across geographic location, case studies have limited utility. High quality evidence in the Caribbean requires both generalizable data and information to elucidate the black box of operations. One approach might be to use multi-site case studies and multistage purposeful random sampling (Onwuegzubie & Collins, 2007). However, prior information on the characteristics of institutions and communities must come from a large-scale quantitative study (Teddlie & Yu, 2007). In the past decade, large-scale achievement data have only been available from public examinations in the Anglophone Caribbean (World Bank, 1993). When used in evaluating system performance, however, this type of data is limited because of the strong shadow education market (Mizala, Pilar & Urquiola, 2007). Public examinations also do not provide standards-referenced data to answer the question, how good is good enough? Only in the last decade, have monitoring assessments been introduced into some Anglophone Caribbean islands (UNESCO, 2008). Trinidad and Tobago installed a new national assessment system in 2004 designed to better to monitor achievement standards. From 2005, both norm-referenced and standards-referenced data on education districts, schools and students were provided. The context and design of the study To operationalize exceptional challenge among primary schools in Trinidad and Tobago, a measure of school performance was developed using the numbers of students at different achievement levels in Language and Mathematics national tests administered at Standards 1 and 3. Also obtained was a measure of community economic and social disadvantage called the Basic Needs Index and the percentage of students experiencing economic disadvantage in the school. The socioeconomic composition of the schools was estimated from the percentage of the school population entitled to free school meals. Schools in the category of exceptional challenge scored below 240 on the API, reported a free school meals index of above 90, and are situated in communities with a BNI of below 50. The overall purpose of the study was to identify schools facing exceptional challenge and to deconstruct the nature of underperformance in these schools. Deconstruct means to dismantle and analyze antecedents and consequences. Thus, the study was designed to gather deep insight into the nature of low achievement within schools facing exceptional challenge and to elucidate factors contributing to low school underperformance. The three research questions guiding this study were: (1) What are the locations and institutional characteristics of schools facing exceptional challenge? (QUAN) (2) What is the nature of key processes and systems, such as teaching-learning, leadership, and the parent-school-family interface? (QUAL/MIXED METHODS) (3) How do the contextual factors impinge on organizational efficiency and academic achievement? (MIXED METHODS) Design of the study The overall mixed methods design was sequential explanatory, with quantitative and qualitative data collected in two consecutive phases within the study (Ivankova, Creswell & Stick, 2006). The design of the overall programme is illustrated in Figure 1. Using the framework of Morse & Niehaus (2009), the Phase I and II components were separate
3

research projects within an overall mixed methods research programme. Although Phase I was a mono-method quantitative study, Phase II was a qualitatively driven embedded mixed methods study of underachievement at each school site. The theoretical thrust for the overall programme was deductive. The theoretical drive of the Phase I project was deductive and the Phase II mixed methods study inductive. The projects were sequential, with priority given to the quantitative Phase I. Overall Mixed Methods Programme Design: Sequential Explanatory. Theoretical Thrust: Quantitative
PHASE I Project: Quantitative Study of secondary databases for all 477 primary schools in Trinidad and Tobago Phase II Project: Mixed Methods Qualitatively-Driven Embedded Study of three school sites designed to elucidate nature of underachievement in different challenging contexts

To identify schools facing exceptional challenge

Integrated analysis of nature and factors involved in challenge

Figure 1: Overall Mixed Methods Programme Design Rationale for the mixed methods research design The rationale for the mixed methods design in the overall programme and in the Phase II project was explored by considering the different purposes of mixing identified by Greene, Caracelli, and Graham (1989). As shown in Table 1, the five purposes are triangulation, complementarity, development, initiation and expansion. The complementary framework proposed by Bryman (2006) extends and complements this framework. In the overall programme, the main purposes were complementarity and development, with the qualitative component in Phase II designed to enhance and illustrate the nature of processes within the schools identified as exceptionally challenging. The qualitative component in the Phase II project sought new frameworks while extending the breadth and range of inquiry. Sampling strategy The overall sampling strategy was sequential mixed method with multilevel sampling used for Phase II (Teddlie & Yu, 2007). In Phase I, Census data 1 was obtained for all 477 primary schools in Trinidad and Tobago. The criteria used to select the schools facing exceptional challenge were (1) 90% of the student population receiving free school meals and (2) and API of under 150. A short list of six schools was developed and visits were made to each school. The final list of three schools in Phase II was chosen based on criteria that captured the variations for composition (coeducational/single-sex) and geographic location (urban-rural). The schools came from two educational districts, which accounted for 74 of the 159 schools classified as under academic watch using the API rubric. All three schools were enrolled in the Governments Performance Enhancement Project for low performing schools. The two urban schools (School A-Female and School B-Male) had populations just over 100 (118 in both schools), but in the rural school, which catered for pupils from a small very isolated village, the school population was just 41. In Phase II, probability samples were obtained for some surveys and purposive samples used for interviews, observations and some surveys.
4

Table 2: Rationale and purposes for mixed method design used to investigate challenging schools in Trinidad and Tobago
Primary Purpose of Mixing
Greene, Caracelli, & Graham (1989)

1. Complementarity -sought elaboration, enhancement, illustration, and clarification of the results from one method with the results from the other method. 2. Development sought to use the results from one method to help develop or inform the other method.

Programme/ Project Overall Programme MM

Additional Purposes
Bryman (2006)

Overall Programme MM

3. Initiation sought the discovery of paradox and contradiction, new perspectives of frameworks, the recasting of questions or results from one method with questions or results from the other method 4. Expansion -sought to extend the breadth and range of inquiry by using different methods for different inquiry components. 5. Triangulation -sought convergence, corroboration, and correspondence of results from the different methods.

Phase II MM

Completeness- A more comprehensive account of the area of inquiry was provided. Process- quan provided an account of structures but qual provides sense of process Different RQs- quan and qual each answered different research questions Explanation qual used to help explain findings generated by the other Sampling quan used to facilitate the sampling of respondents or cases. Context - qual provided contextual understanding coupled with generalizable, externally valid findings in quan. Diversity of views quan uncovered relationships between variables and qual revealed meanings among research participants Illustration - Used qual to illustrate quan findings. Unexpected results - quan or qual generates surprisingly results only explained by other. Enhancement Findings of quan or qual augmented by gathering data using qual or qual. Different RQs- quan and qual reach answered different research questions

Phase II MM

Phase II MM

Instrumentation In Phase II semi-structured interview schedules were prepared for individual and focus group interviews with leaders, teachers and parents. Both structured and unstructured observation methods were used. For teaching and learning, the Instructional Practices Inventory was used to assess the nature of teaching and learning (Painter & Valentine, 1996). Video and photographs from two of the three sites supplemented field notes, which contained descriptions of teaching-learning events. Parents and teachers in the focus group were required to complete survey questionnaires. Teachers completed questionnaires on collective teacher efficacy and group organizational citizenship behaviour. Questionnaires were also administered on student engagement to both students and teachers of all target classrooms. The student engagement questionnaires included items from existing instruments on student engagement (OECD, 2003).

Table 2: Distribution of schools facing exceptional challenge in the eight education districts of Trinidad and Tobago Educational Administrative Regions UrbanExceptionally Ownership Multigrade Status District rural Challenging Context No. % Govt Denom Full Partial Multigrade POS & Environs City of Port of Spain Urban 5 13.9 1 4 4 1 0 Diego Martin Suburban 1 5.3 0 1 1 0 0 San Juan /Laventille Suburban 3 7.3 2 1 2 1 0 Victoria City San Fernando Urban 4 14.3 1 3 1 2 1 Princes Town Rural 4 3.0 0 4 1 2 1 Penal/Debe Rural 1 3.6 0 1 0 1 0 St. George East Arima Borough Urban 0 0.0 0 0 0 0 0 Tunapuna/Piarco Suburban 3 5.7 1 2 0 2 1 Caroni Chaguanas Borough Urban 1 4.0 0 1 1 0 0 Couva/Tabaquite/Talparo Rural 5 8.5 1 4 1 4 0 St. Patrick Point Fortin Borough Urban 2 10.0 2 0 2 0 0 Siparia Rural 2 6.1 0 2 2 0 0 North Eastern Sangre Grande Rural 7 17.1 2 5 0 3 4 South Eastern Mayaro/Rio Claro Rural 3 14.3 0 3 1 1 1 Tobago Tobago -5 15.2 2 3 1 3 1 46 9.6 12 34 17 20 9

Classified by Ministry as Low Performing 5 1 3 2 3 0 0 2 1 3 1 1 5 0 0 27

Procedures and Analyses A merged database was created for all primary schools in the country. This database was analyzed using SPSS 13.0. Means and standard deviations were reported along with a comparison of means from schools classified as exceptionally challenging and normal schools. The principals of the chosen schools were contacted and a formal meeting held with staff. The field assistants then accessed the schools and gathered data twice a week for two months. For the data gathered in Phase II, priority was given to the qualitative data. Survey data from the purposive samples (focus group interviews) were included but reported under the dominant qualitative themes (Tashakorri & Teddlie, 1998; Morse & Niehaus, 2009) Findings (1) What are the locations and institutional characteristics of schools facing exceptional challenge? Forty six schools were classified as experiencing exceptionally challenging external contexts. Of these schools, only 27 were classified as underachieving by the Ministry of Education. Of the schools facing exceptional challenge, 12 were government-owned and 34 were denominational. The majority of schools (29) were either multi-grade or partially multigrade without a full complement of staff for every class. For the 46 schools classified as both facing challenge and performing poorly, the majority were in the urban areas of Port of Spain and Environs (9), Victoria (9), in Northern Eastern (7). Table 3: Characteristics of schools facing exceptional challenge compared
Exceptionally Rest of Challenge Schools % receiving free lunch 101.27 67.64 Mean Academic Performance Index 192.50 281.30 Mean % below 30% 28.77 14.19 Mean Eleven Plus % score (2001-2004) 42.50 54.65 % meeting & exceeding standards in Language (Std. 3) 25.81 52.27 % meeting & exceeding standards in Maths (Std. 3) 13.72 35.87 Years of teacher experience 15.94 9.63 % Teachers with A-Levels 25.00 24.16 % Teachers with basic training (teachers college) 79.50 86.52 st % Teachers with 1 Year University (Cert. Ed.) 9.63 11.05 % Teachers with Degrees 6.60 5.56 Student/teacher ratio 11.90 16.69 School size 107.48 280.58 Effect Size Benchmarks = Small (0.01); Medium (0.06); and Large (0.14) School Characteristics pvalue .000 .000 .000 .000 .000 .000 .007 .699 .001 .426 .345 .000 .000 Effect size .154 .221 .105 .090 .119 .082 .015 .000 .023 .001 .002 .072 .067

1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13.

Table 3 compares 13 characteristics of 46 for both school types. Of the 13 variables, 10 showed significant differences. No differences were found for teachers with A-Levels and teachers with partial or full degrees. Schools facing exceptional challenge performed more poorly in national assessments (Mean API=192.50; P-value=.000, Eta Squared=.221), the mean SEA score (Mean=42.50; P-value=.000; Eta Squared=.090), and had larger number of students below the 30% remedial score (P-value=28.77; P-value=.000; Eta Squared=.105). The greatest deficiency appeared to be in Language, with the difference between the mean percentage meeting and exceeding standards in exceptionally challenged schools (Mean=25.81 cf Mean=52.27) much lower. The effect size for the difference in Language (Eta Squared=.119) was larger than that of Mathematics (Eta Squared=.082). The schools facing exceptional challenge were generally smaller (Mean=107.5; P-value=.000; Eta
7

Squared=.067) with smaller class size ratios (Mean= 11.90; P-value=.000; Eta Squared =.067). (2) What is the nature of key processes and systems, such as teaching-learning, leadership and organization, and the parent-school-family interface? The parent-school-family-interface Location The dominant theme at all sites was the influence and nature of the schools context. Schools A (Boys) and B (Girls) were in a disenfranchised urban community in the capital city of Port of Spain and school C was located in a disadvantaged community located within an isolated rural agricultural area. Although the economic situations in the communities were varied, the majority of families in these schools worked in low paying menial jobs. All twenty-one families interviewed in the focus group reported manual father occupations, such as labourer, gardener, and handyman. The majority of mothers were homemakers or employed in low-level jobs such as cleaning. Nine of the twenty-one fathers had either no education or primary school as the highest level. Based on the survey data, family sizes were relatively small, but larger in the rural school. In the urban school, however, children might have several stepsiblings in other homes. Despite the proximity of the communities to major towns, the school and communities remained isolated and this was often reflected in the students lack of experiences. Nine of the twenty-one focus group families never or occasionally carried their children on visits to major cities or towns. The urban boys school was unique in drawing students from a wide range of economically disadvantaged communities in the Port of Spain and Diego Martin areas. In the case of the rural school, the community was isolated because of the quality of roads and distance from the main traffic ways. This made it expensive for transport in and out of the area and with large family sizes; it was often prohibitive for all siblings to go to the better quality schools outside the village. Views of Parents and Teachers compared Parents generally had neutral or negative views of the institutions. The majority of parents in the rural school focus group believed that standards in the schools had declined considerably as one parent expressed:
But now the standard of the school [has declined]. There [was a good] principal in the school at the time [and after him], [they] break down everything in the school. I would say [that] because I know. I passed by all my other schoolteachers before [there] was a total breakdown of the school. Up to now, the school cant catch itself and I [am] sure [that] a lot of parents who [are here, if they] want to speak the truth -- they would say the same thing.

Despite their low educational level, parents were aware of specific teaching-learning deficits, including inattention to language and language development, as discussed within a parent focus group:
Parent 1: I wish . . . [the] children . . . here [had] learn[t] to read and that is a big problem [for] me, that is a [very] big problem. Parent 2: You know the [other] children-they [are] coming home [everyday] with a paragraph [to] read. Well, they dont have that here at all. I dont know if the other class does . . . it but I never [saw] my daughter [bring anything home to read]. She doesnt bring [her] reading book [home] and she [does not] write. She [is in] std 5. I mean you have to [read at that stage]. 8

Parent 3: Standard 5 [does not get] reading or spellings. [They] right to say [that]. Well we have nothing too] in the Standard 4 because my daughter, she [is] in Standard 4 [and] she could read well and she could spell [well] because [has to] help the other one[s], and so you [can] understand [that] they need more spelling and reading.

The parents concerns centred especially upon the examination classes of Standards 4 and 5, which prepared students for the entrance into secondary school. Perhaps, parents were conscious that poor language skills at this stage could severely hinder the students chances at success, both in the examination and in the secondary school. Parents and teachers had different perspectives on homework and reinforcement, with teachers blaming the parents as noted by the following teacher:
Well, I can only speak about this culture. Its what Im dealing with right now. Theres a total and complete lack of parental support. You will see it and hear of it as you go along in the other classes. I dont give home work. I have stopped it because its an exercise in frustration . . . so whatever recapping we do, [we do] in the class. [They] come in the morning and we try and work [and] they would retain [the information] to a certain point. But the reinforcement at home [is not there]. There is no support network at home. There are different issues [like] the television on all day, they next to a bar, they didnt have any electricity.

The principal of the rural school acknowledged this lack of connection between school and community; indeed, he regarded it as the greatest challenge for further progress of the schools, admitting:
Well, I think one of the greatest challenges would be that people in the community have lost confidence in the school. I think thats our biggest challenge and so therefore what you find happening is that people are taking their children and sending them to schools outside of [the area] as in [nearby location]. I think another challenge would be that our school has not been producing pupils that go to secondary school and do an entire five years there because we have a high secondary school dropout rate.

Overall low expectations for the students from school personnel stemmed in part from the realization that the school had not been the first choice for many parents. This, teachers felt, influenced the attitude of both parent and child to school work. As one of the leaders in the urban school pointed out:
You know parents who are not interested in work send their children here. There are those who are interested [and] try to get in[to] the better schools first. And they might come here as a last resort and then when they go to these good schools, now they would give them some attention like when they fail there.

These opposing views pointed to a lack of trust between community, parents, and teachers. Even though the urban girls school had established a less harmful reputation, generally all three schools maintained a distrustful view of parents. Thus, at the urban schools, staff often bemoaned the lack of parental involvement and even blamed parents for much of the situation at hand, as one school leader in the urban school expressed:
Education Leader: Well, in the community, its more or less the attitude of the parents. . . and the interest . . . They dont push the children school wise and theyre not too interested in the childrens schoolwork. As such, they give no assistance to the children in 9

[their] schoolwork at home. And even when we have meetings, class meetings, when you [would] expect, well every parent [to] come you might get about a two thirds for the most. Moderator: [Is that] in every class? School Leader: Yes, in most classes, yes Moderator: So the [level of] parental involvement, [dont] teachers have no control over that School Leader: No.

The childs experience Children in the rural community reported having fewer books at home than children in the urban community, with no child reporting more than 100 books in the rural community. Comparing the urban schools, pupils from the girls school had more books at home, with 29.3% having 100 or more books compared with 16.7% for the boys. However, some books were well below the reading age of the children. No student in the rural school had a computer at home and 61.9% of the urban girls school pupils and 53.5% of the urban boys school pupils reported the same. This finding is important because 14 of the 21 focus group parents said they rarely accessed the library in their area or visited areas of cultural interest such as museums. It is just as well that children in the focus group claimed that they had little help with their schoolwork at home. Overall, 26 of the 99 students felt like outsiders in the school, but at this age group, the majority of students had a relatively positive view of the school. However, 12 of the 14 students in the rural school claimed to feel bored. The same number felt that their parents wanted them to go to another school, reflecting the communitys lack of trust. Feelings that their parents wanted another school were also strong in the urban boys school but almost absent in the female urban school (B). Parental expectations were overwhelmingly positive in the rural school but in the urban school, some students reported low parental expectations. Such low expectations were more common in Mathematics, Social Studies, and Science than in Reading and Writing. Teachers and Teaching-Learning Qualifications None of the teachers was from the schools community. In school A, all 4 teachers in the focus group possessed a Teachers Diploma, with 6 to 10 years experience. Two teachers had undergraduate degrees. In school B, six teachers participated in the survey. Three had between 21-25 years experience, one had 11 to 15 years, and two had 6-10 years. However, only one had an undergraduate degree. For school C, three of the six teachers had additional academic qualification at the lower level and two had undergraduate degrees. Four of the six had professional training. Four of the six teachers had been teaching for at least 15 years. Thus, contrary to the findings in the secondary data, teachers in these schools were relatively well trained and experienced. The critical issues of preparation and attendance The most critical issues in the teaching-learning environment were student attendance and attitude towards schoolwork. Daily attendance by students was never acceptable in most schools, as noted by the principal of the urban boys school:
On the roll we have 114 sometimes you have people leaving, sometimes you have people coming the average attendance a day is lets say 85 out of that 114 so I dont know how you would really rate that thats 85/114 that what probably about 80% somewhere round there between 70 and 80% 10

In the rural school, teachers observed that some students only came for the school meals. It might also be that low attendance was part of the school and community culture, although one teacher felt that this was beginning to change:
There was a time where on a Friday you had nobody in school at all. Now that has started to change, they start coming to school. It have (sic) some of them who would come to school like once for the week, twice for the week, [but] all that has started to change [as we] try to incorporate more of the community into the school and the school in the community

In school A, the attendance of students was much lower in Standard 1, but increasing in Standards 3, and 5. In all 3 schools, teachers believed that few students were not well prepared. Of the 33 students directly assessed in Standards 1, 3 and 5, as many as15 did not turn in work on time and 14 did not complete work with 100 percent efficiency. Although 24 of the 33 students were eager to learn and 22 were confident about their ability to participate, 19 of the 33 did not persist when given difficult tasks. For school B, students in the 3 classes surveyed were rarely considered withdrawn or not engaged. However, preparation for school and attendance were significant problems in Standard 1. School B provided school level data on student attendance for the year 2007. The figures varied from 58% to 84%. The lowest periods of attendance were in the months of July (58%), December (67.6%) and September (77%). School C reported high engagement and positive behaviours in the four classes sampled. Teachers reported more positively on the learning behaviours of students in School C, with 18 of 25 regarded as completing and turning work in on time and 20 of 25 as eager to learn. The students of school C were also more likely to persist on difficult tasks (19 of 25). Still, attendance was very poor at the infants level. Although class sizes were small, observation suggested that there was (frequently) limited one-to-one interaction in classrooms. More often than not, instruction was pitched at the entire class. Even when teachers used instructional aids, teaching was never authentic and rarely constructivist, with off-task behaviours very high. Observational data on four classes in School B indicated that students were frequently disengaged, engaged on worksheets without teacher assistance, or engaged in seatwork with some teacher assistance. None of the observed classes in this school reported students as engaged in instruction on authentic project work or in active conversations as they constructed knowledge. Leadership and organization A lack of vision In all three schools, leadership was lassiez-faire, with little focus on instructional improvement and student success. Surprisingly, none of the principals possessed a Bachelors degree in Educational Administration, the current minimum qualification for primary school principals in Trinidad and Tobago. In schools B and C, the principals had some leadership experience. Leadership by the principal of the rural school was very lax, even negligent fearing that teachers would transfer out of the school, a shad happened in the past. He readily admitted:
And another challenge we have [here] is to try to keep the teachers happy here so that they would want to remain and the children would benefit from that kind of stability. I think those are the challenges that I that I would consider major challenges.

However, turnover of staff was a challenge for leadership in both urban schools and especially in school A, high turnover had impacted strong on leadership influence.
11

Although school leaders claimed to lack support from the schools community, for the urban schools, there were several city businesses involved in resourcing and funding student development programmes. The rural school was formally enrolled in the Caribbean-wide US funded school improvement exercise. Although resources were abundant on paper, the principals wanted greater psychological support and presence from the Ministry of Education and the Denominational Board. In a sense, then, the leaders and staff felt isolated and abandoned, powerless to effect change, as the principal of one of the urban schools described:
Physical resources in terms of charts and books and things like that [are] fairly accessible but in terms of human resources, [that is a] major problem. As I say, we need guidance counsellors here because you would cringe to hear some of the stories from some of these children here. And again I feel they just dont want to come to [this area] because they hear [where it is situated]. When we inquired [of other nearby schools], they had counsellors. And we are right around the corner. We [are] in walking distance and nobody is assigned to this school to deal with those problems.

On observation, however, principals were more inclined to engage in social and bureaucratic activities as their administrative duty, but generally did little monitoring or evaluating. In part, this was because of the turbulence of the school environment and the need to provide psychological support to teachers. Observation of the principals daily tasks suggested that little was done to protect teaching and learning time. Indeed, daily activity in the school was often disjointed and unfocused. Largely, this reflected the leaders philosophies, which rarely focused upon teaching-learning. Instead, principals saw themselves as social engineers, keeping harmony and peace and thereby fostering stability. One principal described what she understood her leadership to be:
Well, I guess thats what I see my role as: being [a] motivator both to students and staff [and] being able to relate to staff and students and parents. Being able to resolve disputes amicably again children and staff [and] being a role model again for both staff and students . . . thats it in a nut shell.

Classes in all three schools were very small, most well below 20 students and there was only one of each class in the school, but none was multi-grade. All schools had sharply declining rolls, even School B, which reported the highest API score. The annual data for this institution showed that the school population had declined consistently each year from 140 in 2002-2003 to 108 in 2006-2007. The decline in roll was also apparent at the other sites suggesting that parents were not inclined to enrol their children in these failing institutions. The early grades were poorly attended by both students and teachers and most of the efforts were focused upon the upper school with the Eleven Plus examination. Field assistants observed that teaching time was an issue in all schools and especially for the rural school, where classroom teaching started late and ended early. (3) How do the contextual factors impinge on organizational efficiency and academic achievement? The relationship between the different variables identified at the three study sites are explored in Figure 1. As shown, the dominant theme was the impact of the economically and socially disadvantaged school community. In the rural school and in the girls school, the schools community was the actual physical community but in the urban boys school, the school community was drawn from the immediate environment and several high poverty communities in the area. The nature of the community had a direct impact upon both the

12

governance and leadership of the school. Principals philosophies of leadership and leadership in practice were all aimed at managing the turbulent environment.
ECONOMICALLY & SOCIALLY DISADVANTAGED SCHOOL COMMUNITY High Poverty Many Social Problems Ministry of Education Parental Involvement Low Parental Education Intervention programmes Denominational Education Board

LEADERSHIP IN THE SCHOOL Not Focused upon

Low Supply of high quality experienced teachers

TEACHERS-------------TEACHING/LEARNING-----------STUDENTS

LOW ACHIEVEMENT

LOW PERFORMING SCHOOL

Figure 1: Model of recurring processes, interfaces, and sub-systems in schools facing exceptional challenge in Trinidad and Tobago Thus, poverty, poor nutrition, and social problems dominated all aspects of school life. Teachers believed that they lacked training in dealing with social issues and special needs. Thus, arguably, teachers perceptions of leadership and governance were also influenced by the intensity and frequency of the problems that occurred in this environment. In the urban schools, the interviews indicated that teachers were very reluctant to accept postings in this school. In a sense, then, the schools did the best with what was available to them. There were multiple interventions in all three schools, with homework centres, adult reading programmes, and early school reading programmes in the rural school; however, these interventions were not co-ordinated and therefore failed to impact on the problems. Discussion and Implications The methodology and findings in this study were inextricably linked, with the use of multiple methods and methodologies in Phase II, a key strategy in capturing the phenomenon of (under)achievement as a complex issue in the social system (Rogers, 2008; Xu, 2009). Such an approach supports the qualitative philosophy of holism (Morse & Chung, 2003). One might even argue for a tradition of using mixed methods to gain added insight into issues like
13

poverty and vulnerability within social systems (Lucke et al., 2001; Shepard et al., 2002). Arguably, elements and antecedents of achievement are found across multiple levels and subsystems. Consequently, the data collection strategy required information from multiple actors (Goldenberg, Gallimore, & Reese, 2005). Table 4: Findings and meta-inferences in the challenging school study
Relationship Between Findings 1) Corroboration/ Expansion/ Enhancement Description of meta-inference 1. Low performing urban and rural schools face different challenges. The free school meal and basic needs quantitative indices accurately capture the socioeconomic status of the school population. Social and economic disadvantage operated through several mechanisms: limited family resources, lack of access to experiences, and higher absenteeism. The alienating response of the school magnified these deficiencies. The quality of teachers was not always related to poor teaching (QUAL) although underperformance was correlated with teacher quality in the QUAN. There was a moderate correlation between resources and school performance in the QUAN. However, observation and interviews confirmed that schools were well supplied with teaching resources, although they were not always used in the classroom (observation QUAL). Teachers reported high levels of collective teacher efficacy. However, in the observational and interview QUAL data, low expectations/efficacy was evident, but these beliefs were deflected, rationalized or justified. Lack of parental concern was cited as a problem by most teachers in the site survey and interviews, but interviews with parents suggested that they were very concerned about the schools underperformance, but had limited access to opportunities for resolution. The observation QUAL data revealed that inauthentic, low- quality classroom teaching was a major factor in low school performance. Implications for Research/Policy Making 1. School improvement plans must consider the context and location of the school. Greater use might be made of demographic and other indices in planning support and targeting aid for schools. Integrated education, health, and social services are required along with a retraining of staff to serve low-income communities.

2.

2.

3.

3.

2) Conflicting/ Contradictory

1.

1.

Teacher training programmes might focus on reorientation of teachers rather than simply providing skills. Simply providing more resources might not be enough to improve schools, integrated services and interventions, teacher reorientation and on-site training should be necessary.

2.

2.

3.

3.

4.

4.

Quantitative measures of individual and collective teacher efficacy do not always capture low expectations or belief systems of local teachers. This critical variable should be addressed in teacher training and school reform. Parents must be given a greater say in school and community arrangements for enhancing performance. Integrated parental education and outreach programmes should be set up.

3) Unique to methodology

1.

1.

2.

3.

The interview QUAL data revealed that poor nutrition and student absenteeism were significant factors in limited learning opportunity. The QUAN data revealed that some schools performed adequately despite high levels of social and economic disadvantage.

2.

3.

Teacher training should be integrated with the provision of resources and focus on the nature of teachinglearning and low engagement among children from low-income communities. The school feeding programme might be extended by the inclusion of breakfast for schools with high social and economic disadvantage. Best practice might be disseminated and shared, allowing schools to learn from or emulate each other.

14

Although the use of multiple qualitative methods and multiple methodologies provided complex and rich data, some information was contradictory. Contradictory data might be regarded as a challenge to integrating or merging (Plano-Clark et al., 2010). There were several apparent contradictions in the data within Phase II (between the qualitative and quantitative and between the qualitative methods) and between Phase I and Phase II quantitative and qualitative). It was not always possible to resolve these differences as suggested in Creswell and Plano-Clark (2010). It might be that contradictory findings provide multiple perspectives on a complex issue and useful for policy-making. There were, in fact, three kinds of relationships between the findings from different methods and methodologies. As shown in Table 3, findings might corroborate or expand as in triangulation. Some findings were conflicting, however, because they represented a different perspective or provided a clearer view of an issue and some findings were unique to the particular methodology. For example, although the quantitative data suggested that there were fewer trained teachers in schools facing exceptional challenge, in reality the situation was often more complex than that. The staff of the rural school included several highly qualified teachers although this did not seem to alter their approach to teaching in this school. Table 4 also summarises the policy-making implications derived from the different types of meta-inferences. Although all the schools were involved in various kinds of individual interventions, these different programmes were neither focused nor explicitly linked together. Linking can achieve reinforcement across and within the school and community. For example, teaching resources might be linked to specific training on constructivist teaching. Likewise, the early childhood interventions might be linked to the adult literacy classes in the community. Additional interventions and services are required, including integrated services to deal with the economically and socially disadvantaged communities. Programmes and policies to address student absenteeism must also be established. Although several international studies have focused on the leadership component in improving schools in challenging contexts (Jacobson, 2008), it seems more useful in the context of Trinidad and Tobago to ensure coordinated efforts by the education districts and other support services external to the school in effecting school improvement in these contexts. This relates to the level of autonomy in school sites, the complex nature of the contextual variables, and the quality of personnel available for leadership. Even if committed informed leadership were available, turnaround will prove quite a difficult task for these schools. Fostering collective responsibility and better integration with the community therefore emerge as critical goals. References Ainscow, M., Muijs, D., & West, M. (2006). Using collaboration as a strategy for improving schools in complex and challenging circumstances: What makes the difference? National College for School Leadership. Retrieved from
http://networkedlearning.ncsl.org.uk/knowledge-base/research-papers/using-collaboration-asa-strategy-for-improving-schools-in-complex-and-challenging-circumstances.pdf

Bryman, A. (2006) Integrating quantitative and qualitative research: how is it done? Qualitative Research 6(1), 97-113 Creswell, J. W., & Plano Clark, V.L. (2010). Designing and conducting mixed methods Research (2nd Edition). Thousand Oaks, CA: Sage. Crossley, M. (2008). The advancement of educational research in small states. Comparative Education, 44(2), 247-254. De Lisle, J., Keller, C., Jules, V., & Smith, P. (2009). When choosing might mean losing: A Mixed method study of secondary school choice in the Republic of Trinidad and
15

Tobago. Caribbean Curriculum, 16(1), 131-176. De Lisle, J., Smith, P., & Jules, V. (2010). Evaluating the geography of gendered achievement using large-scale assessment data from the primary school system of the Republic of Trinidad and Tobago. International Journal of Educational Development, 30(4), 405-417. Goldenberg, C., Gallimore, R., & Reese, L. (2005). Using mixed methods to explore Latino children's literacy development. In T. Weisner (Ed.), Discovering successful pathways in childrens development: new methods in the study of childhood and family life (pp. 21-46). Chicago: University of Chicago Press. Government of Trinidad and Tobago (2006). Vision 2020 operational plan 2007-2010. Port of Spain: Ministry of Planning. Greene, J. C., Caracelli, V.J., & Graham, W.F. (1989). Toward a conceptual framework for mixed-method evaluation designs. Educational Evaluation and Policy Analysis 11 (3), 255-274. Harber, C., & Muthukrishna, N. (2000). School effectiveness and school improvement in context: The case of South Africa. School Effectiveness and School Improvement, 11(3), 421-434. Harris, A., Muijs, D., Chapman. C., Russ, J., & Stoll, L. (2006). Improving schools in challenging contexts: Exploring the possible. School Effectiveness and School Improvement 17(4), 409-425. Hemming, P. J. (2008). Mixing qualitative research methods in childrens geographies. Area, 40 (2), 152-162. Houtveen, T., van de Grift, W., Kuijpers, J., Boot, M., Groot, F., & Kooijman, H. (2007). Improving underperforming schools. Journal of Education for Students Placed At Risk, 12(4), 36-381. Ivankova, N. V., Creswell, J. W., & Stick, S. L. (2006). Using mixed methods sequential explanatory design: From theory to practice. Field Methods, 18(1), 3-20. Jacobson, S. (2008). Leadership for success in high poverty elementary schools. Journal of Educational Leadership, Policy and Practice, 23(1), 3-19. Kairi Consultants. (2007). Analysis of the Trinidad and Tobago survey of living conditions. Port of Spain, Trinidad: Ministry of Social Development. Levin, B. (2006). Schools in challenging circumstances: A reflection on what we know and what we need to know. School Effectiveness and School Improvement , 17(4) 399407. Lucke, J., Donald, A. M., Dower, J. A. F., & Raphael, B. (2001) Considerations in the design of a mixed-method cluster evaluation of a community programme for 'at-risk' young people. Evaluation, 7(1), 110-113. MacBeath, J., Gray, J., Cullen, J., Cunningham, H., Ebbutt, D., Frost, D., Steward, S. & Swaffield, S. (2005). Schools facing exceptionally challenging circumstances: Final report. Cambridge: University of Cambridge Faculty of Education. MacKenzie, C. G . (1991). Denominational primary schooling: The case of Trinidad and Tobago. International Review of Education, 38(2), 211-226. Michalak, J. M. (2009). Making a difference in challenging urban schools: Successful principals. European Educational Research Journal, 8(3), 386-396. Mizala, A., Pilar, R., & Urquiola, M. (2007). Socioeconomic status or noise? Tradeoffs in the generation of school quality information. Journal of Development Economics, 84, 7175. Moe, C. (2009, November 14). Ministry moves to assist underperforming schools. Trinidad Guardian Newspaper. Retrieved from

16

http://guardian.co.tt/news/general/2009/11/14/ministry-moves-assistunderperforming-schools. Morse, J.M., & Chung, S.E. (2003). Toward holism: The significance of methodological ` pluralism. International Journal of Qualitative Methods, 2(3). Retrieved October 15, 2003 from http://www.ualberta.ca/~iiqm/backissues/2_3final/html/morsechung.html Morse, J.M., & Niehaus, L. (2009). Mixed method design: Principles and procedures. Walnut Creek, CA: Left Coast Press. Muijs, D., Harris, A., Chapman, C., Stoll, L., & Russ, J. (2004). Improving schools in socioeconomically disadvantaged areas: A review of research evidence. School Effectiveness and School Improvement,15(2), 149175. Mullis, I. V. S., Martin, M. O., Kennedy, A. M., & Foy, P. (2007). PIRLS 2006 international report: IEAs Progress in International Reading Literacy Study in primary schools in 40 countries. Chestnut Hill, MA: TIMSS & PIRLS International Study Center, Boston College. Organisation for Economic Co-operation and Development (OECD). (2003). Student engagement at school: A sense of belonging and participation. Results from PISA 2000. Retrieved from http://www.pisa.oecd.org/dataoecd/42/35/33689437.pdf. Onwuegbuzie, A. J., & Collins, K. M. T. (2007). A typology of mixed methods sampling designs in social science research. The Qualitative Report, 12(2), 281-316. Painter, B., & Valentine, J. (1996). Instructional practices inventory. Columbia, MO: Missouri Center for School Improvement, University of Missouri. Plano Clark, V. L., Garrett, A. L., & Leslie-Pelecky, D. L. (2010). Applying three strategies for integrating quantitative and qualitative databases in a mixed methods study of a nontraditional graduate education program. Field Methods, 22(2), 154-174. Pretorius, E.J., & Currin, S. (2010). Do the rich get richer and the poor poorer? The effects of an intervention programme on reading in the home and school language in a high poverty multilingual context. International Journal of Educational Development, 30(1), 67-76. Rogers, P. J. (2008). Using programme theory for complicated and complex programmes. Evaluation: The International Journal of Theory, Research & Practice, 14(1), 29-48. Shephard, M. P., Orsi, A. J., Mahon, M. M., & Carroll, R. M. (2002). Mixed-method research with vulnerable families. Journal of Family Nursing, 8, 334-352. Tashakkori, A., & Teddlie, C. (1998). Mixed methodology: Combining qualitative and quantitative approaches. Thousand Oaks, CA: Sage. Teddlie, C., & Yu, F. (2007). Mixed methods sampling: A typology with examples. Journal of Mixed Methods Research 1(1), 77-100. UNESCO. (2008). Education for all global monitoring report 2008: Education for All by 2015. Will we make it? Paris: Author. Van de Grift, W., & Houtveen, T. (2006). Underperformance in primary schools. School Effectiveness and School Improvement, 17, 119. World Bank (1993). Caribbean region: access, quality, and efficiency in education. Washington, DC.: Author World Bank (1995). Trinidad and Tobago: Poverty and unemployment in an oil based economy. Report No. 14382-TR. Country Department III. Country Operations II Latin America and the Caribbean Region. Washington, US: Author. World Bank (1995). Trinidad and Tobago: Poverty and unemployment in an oil-based economy. (Report No. 14382-TR.Washington, DC: Author. Xu, S. (2009). Inquiry from the outside and the inside: A mixed method study of university

17

underachievement in China. International Journal of Adolescence and Youth, 15/12(155-173) Ylimaki, R. M., Jacobson, S. L., & Drysdale, L. (2007). Making a difference in challenging, high-poverty schools: Successful principals in the USA, England, and Australia. School Effectiveness & School Improvement,18(4), 361-381.

18