Running head: METHODOLOGICAL CRITIQUE

Methodological Critique
Bassam Adlouni
#56135098
March 08, 2013
ETEC 500

METHODOLOGICAL CRITIQUE

Methodological Critique
There is an ongoing debate among researchers and theorists about the qualitative and
quantitative educational approach of research since the mid of 19th century until recently (Neglas,
1999). Neglas (1999) argue that some issues between these two methods are related to their complex
nature, especially philosophical and theoretical backgrounds. In this article, I have two examples of
qualitative and quantitative studies, showing the summary of each study and comparing them in
terms of essential research process criterias. These criterias are: research problems, selection of
participants, data collection and analysis, procedures and instruments, reporting of literature, and
conclusions.
Quantitative Research Example: Can Instructional and Emotional Support in the First-Grade
Classroom Make a Difference for Children at Risk of School Failure? Bridget K. Hamre and
Robert C. Pianta
Topic and purpose
Hamre and Pianta (2005) conducted a national prospective study on 910 children to find the
effect of the supportive classroom environment on children with high risk of school failure. This
study is an extension of the work of Morrison and Coroner, and Rimm-Kaufman et al (Hamre &
Pianta, 2005, p. 36). They hypothesised that instructional and emotional support of teachers to high
risk students would decrease their school failure.
Procedures used to gather data and data analysis
The actual sample was way higher at the beginning of the study but researchers have
considered many factors in selecting their target group, such as, healthy conditions of the babies, and
randomization of the sample, ensuring economical, educational, and ethnic diversity (Hamre &

METHODOLOGICAL CRITIQUE

Pianta, 2005, p. 37). Hamre and Pianta (2005) adopted the recommendations of Rutter and Maughan
(2002) in using multiple methodologies to conduct their natural experience study (p.36) which
looks different than the scientific study because there is no manipulation of at least one variable
(Gay, Mills, & Airasian, 2011, chapter 10). These methodologies are, longitudinal data to measure
change within individual, natural experiment to disarticulate the variables, and quantified
measurement to study the causative factor (p. 36-37). These 3 approaches are well identified in the
study; it started with the main categories, functional and demographic indicators. Thereafter, they
disarticulated these main categories into subcategories or variables like, sustained attention,
externalizing behaviours, social skills and academic, and the indicator of mothers with less than
4-year college degree (Hamre & Pianta, 2005). However, this study failed to put other persuading
factors to reflect the demographic indicators. The longitudinal data was applied by measuring the
changes that accompanied the children on different stages from the ages of 54 months to
kindergarten to first grade. Furthermore, the observation was limited to one day or few hours on an
important variable like classroom process which I also argue that it affects the validity of the study.
The third approach used in this study, as a quantifying study, was used to measure these variables by
different systems like COS-1, Woodcock-Johnson composite, CPT, TRF, and SSRS. All of these
systems were used by giving numeric values to subcategorized indicators to quantify the data for
easier comparisons. An example is the Likert type format to assess the student-teacher relationship
indicator of 28 subcategories (p.39). This quantitative research used different methodologies in
collecting data and assessing the variables. In addition, researchers used ANCOVA to conduct the
data analysis which is a statistical method to compare two or more dependent variables and see the
differences and the relation between them (Gay, et.al, 2011, p. 277).
Conclusion

METHODOLOGICAL CRITIQUE

The design of the methodology used in this study is relatively rich and comprehensive; it is
based on good literature reviews including important sources of good authenticity. However, I find
that the sample taken does not properly fit that outcome of the study. In other words, studying the
effect of the teachers instructional support or motivation on students could be done on a later stage
of life and there is no concrete reason to start collecting the sample from birth. I also find that if
researchers have controlled the sample, by enhancing the instructional support in one group, it would
give more validity to the study. Randomization was present to control any extraneous variables but
this study lacked the treatment that should give enough validity to the study (Gay et. al, 2011,
chapter 10).
Qualitative Research Example: Developing Teacher Epistemological Sophistication About
Multicultural Curriculum: A Case Study. Christine Sleeter.
Topic and purpose
This is a qualitative study, the researcher used the case study of one novice teacher in a
Multicultural Curriculum Design course to examine how this teacher begin to think more complexly
to extract an insight for the teacher education pedagogy, the reflection of beliefs, and assumptions
represented by that novice teacher.
Procedures used to gather data and data analysis
The researcher began studying the general epistemological beliefs which is based on the
knowledge with its 3 dimensions, certainty, source, and structure (Sleeter, 2012, p.51). Sleeter used
a single case study of Ann to work on and to focus on the complexity and the uniqueness of
individual cases. The data collected included, papers and journals written by Ann, notes of the
researchers observation on Ann, and recorded interviews with her. The researcher used a rubric tool

METHODOLOGICAL CRITIQUE

for reflection and analysis of all the data collected. On a later stage, the researcher had developed
main categories to be considered as the dimensions of his/her assessment of thinking complexly
about multicultural curriculum that include: task definition, perspective taking, self-reflexivity,
and the locus of decision making (Sleeter, 2012, p.52). All these themes were based on 3 levels of
assessments: a novice, developing, and accomplishment.
To assess Anns reflection on her beliefs and assumptions, Sleeter (2012) used 4 themes to
work on: analyzing epistemological and ideological assumptions in documents, reading from
multiple ideological perspectives, engaging in personal interactions, and engaging in reflective
writing and developing a curriculum unit (p.53).
Conclusion
The researcher noticed that Ann progressed from the stage of seeing the curriculum design as
a rigid assigned authority to a more constructivist status that allows the mixture of her and students
beliefs and experience to expand the learning process. The researcher concluded that it is important
for teachers to observe their own backgrounds and beliefs, and to agitate their students thinking,
and leading them to be more engaged in discussions and in seeking knowledge.
My Comparisons between these two studies
I found that the research problem in the quantitative study is clearer and narrower because it
has definite variables such as instructional support and school failure. The goal is set and definite
beside the research question is known, what is the effect of instructional support on the students
failure. While in the qualitative study, the goal could be changed depending on the observation
during or after the study being done. The sample study in the quantitative, as known, is larger than
the qualitative one. It deals more with variables and numbers. More numbers mean more validity to

METHODOLOGICAL CRITIQUE

the study as it is believed. While in the qualitative one it is difficult to follow a large sample due to
the effort needed plus the unique characteristics of each case. In the qualitative study there is a need
for taking notes, observations, interviews, and to live the experience of the sample, while in the
quantitative study, the researcher has many tools to assess the variables such as questionnaires,
website, TRF, SSRS and so many more.
In data collection, it is the better to have one observer in the qualitative study to keep the
consistency within the analyses and the notes taken. This will lead to a good conclusion at the end of
the study, because the observer lives the experience. While in the quantitative one, there might be a
researchers team to collect data, or there might be a third party, or, others to analyse the data with a
different research perspectives. Regarding procedure and instruments, the quantitative study has
many complex instruments to use based on scientific tools while it depends on the researchers
experience in qualitative research. Reporting of Literature, in quantitative study is a comprehensive
procedure taking into consideration all the studies available on that topic with many citations and
resources. In qualitative study, literature review is not that comprehensive because of the unique
characteristics of the study and the uncertainty of the outcome. This will later give us a reason about
the importance of the hypothesis and its vital role in quantitative studies. I also noticed that the
qualitative review focuses more on theories and on general concepts, the researcher would start
extracting the main themes to work on to support findings, the researchers analyses, and discussion
after conducting the study. Reporting of conclusions in quantitative research should reflect the
findings concluded from analysing of all numbers. It is tested according to the scientific tools, while
in the qualitative studies; it is the researchers point of view and his/her ability to convince the reader
about how correct the finding and the analyses are. The last thing to say is that in triangulation
method, quantitative and qualitative research may drive more authenticity and validity to the study.

METHODOLOGICAL CRITIQUE

My proposal research
I used to believe in quantitative studies and its authenticity more than the qualitative one.
However, in this course and after reading many samples about quantitative researches, I become
aware that not all quantitative studies are well-intentioned because of the many flaws that the
researchers have fallen into. I argue that no matter what the sample is, it is more about efficiency
research has conducted, and I have to be selective and being a good researcher of the research. I
now realize that scientific and quantitative research is complex and need many tools to assess like the
validity and reliability tools which I find it difficult for me to attain at this point. The difficulty in
quantitative research might not come from the process that is well defined, but rather, from the
connection between variables and the manipulation or the treatment that one group has to receive to
assess how the independent variables like the example of, instructional methods or emotional
support the student receive. I find that the already mentioned factors are the hub of any quantitative
and scientific research including the assessment of validity and reliability. While in the qualitative
method, I would not take generalization into account which is the characteristic of quantitative study,
because I may adhere to the specific problem that is related to my study purpose that needs solution
specific to it. Besides, I may not consider a validity test that needs more experience to conduct and
can be applied mainly on quantitative methods, but I will consider the authenticity as my tool to
assess the proposed qualitative research (Niaz, 2009). In qualitative study, the aim is, to gather an
'authentic' understanding of people's experiences (Seale, 1997, p.379).
The next plan
I would like to propose a qualitative study of one student, I am currently mentoring in our
department, and keeping in mind the basic research question in how would a new student utilize
technology to consolidate the learning experience achieved in the operating room. I am already

METHODOLOGICAL CRITIQUE

living the experience with the student; however, at the end or during the experience, I might change
my topic to reflect the theme I am going to construct. I might also find that there would be more
significant issues needed to be addressed related to education and learning.
Methods
I am planning to use my direct observation, the students statements, notes, emails. Taking
into consideration that open-ended questions are the most effective route to gather needed
observations (Seale, 1997, p.379).

METHODOLOGICAL CRITIQUE

References
Gay, L. R., Mills, G. E., & Airasian, P. W. (2011). Educational research: Competencies for analysis
and applications. Upper Saddle River, N.J: Merrill/Pearson
Hamre, B. K. and Pianta, R. C. (2005). Can Instructional and Emotional Support in the First-Grade
Classroom Make a Difference for Children at Risk of School Failure? In L. R. Gay, G. E.
Mills, & P. W. Airasian, Educational research: Competencies for analysis and application
(10th ed.) (pp. 949967). Upper Saddle River, NJ: Merrill.
Niaz, M. (2009). Qualitative methodology and its pitfalls in educational research. Quality &
Quantity, 43(4), 535-551.
Niglas, K. (n.d.). Retrieved 2013, from http://www.leeds.ac.uk/:
http://www.leeds.ac.uk/educol/documents/00001487.htm
Seale, C. (1997). Ensuring rigour in qualitative research. The European Journal of Public Health,
7(4), 379-384.
Sleeter, C. (2012). Developing teacher epistemological sophistication about multicultural curriculum:
A case study. In L. R. Gay, G. E. Mills, & P. W. Airasian, Educational research:
Competencies for analysis and application (10th ed.) (pp. 55-59). Upper Saddle River, NJ:
Merrill.