RM Synthesis Flashcards
(10 cards)
What are different types of significance in research
According to Thompson (2002) there are three kinds of significance in research:
Statistical (non-random variation)
Practical (non-trivial)
Clinical (meaningful improvements)
Need to measure effect sizes
Rossi, Lipsey, & Henry (2019) argue that practical significance is of importance to evaluation stakeholders who must decide whether a program is worth the investment.
Examples of mixed methods research studies
Johnston et al. (2018) Anti-bullying intervention.
Al Nashmi (2012): analyzed online discussion forums in middle eastern countries. They used content coding and they quantitized the themes and used descriptive statistics to identify the most common themes. They also used a Chi-squared test to identify significant correlations between themes and different countries. Government was a common theme in all countries.
Kerrigan (2014) conducted a MM study to determine the factors that influence data driven decision making at colleges. Used convergent parallel design. The QUANT involved surveying faculty and staff about their data use habits. The QUAL involved interviewing about data use. Analysis: She analyzed the data separately then she TRANSFORMED the qual data by assigning a numerical value for the extent of data use. FINDINGS: She found variation in the breadth and depth of data use at colleges. The amount of available technology was not a good predictor of data use. Human capitol (skills) was a good predictor. She used the QUAL findings to understand better the factors. The amount of collaboration and leadership support explained the breadth and depth of data use.
Sibinga (2014)
Buck et al. (2009) conducted a MM sequential explanatory design to examine the attitudes towards science of adolescent AA girls. They surveyed attitudes, such as the value of science and the girls confidence. They then used the results to interview girls who fit 4 different profiles, such as high value but low confidence. Conclusion: teach science as a process of inquiry.
Tashakkori and Teddlie (2006) Three advantages of mixed methods research: Inferences, RQs, Diversity
Sampling in research
Petus-Davis et al (2011) describes different kinds of sampling in research:
Probability sampling, uses randomization and counting techniques to approximate the population (simple random, stratified)
Non-probability sampling: Does not use randomization (convenience sampling, purposive sampling)
Lochmiller and Lester (2017)
Ethics in Research
Lecompt and Goetz (1982) argue that the main goal of qualitative ethnographic research should be to accurately represent the perspective and experience of the participants. We should strive for a “faithful and accurate” rendition of their lives. Constructs need to be meaningful to participants
Wink (2012) argues that critical pedagogy involves incorporating participant voices into the research
Banks (2016) and Brinkman and Kavale (2015) both argue that qualitative research is not value neutral. There are power imbalances that can lead to oppression of minority participants.
Banks (2016) argues for “Strong objectivity” in which researchers acknowledge their biases and act to limit impact. Minority scholars (like Woodson) can provide a counternarrative to harmful majority research like the benevolent slave owner thesis.
Brinkman and Kavale (2015) describe power imbalances and the need for confrontational interviewing.
Milner (2007) argues that failure to consider culture can lead to unforseen harmful consequences. We need to examine our own background and ask how it influences our research: 1. Research self 2. Research self in relation to others 3. Include participant voices 4. Provide historical context for racism
Improvement Science
Lemire, et al. (2017) define Improvement Science as a change process that seeks to design, test, implement and scale up changes for systemic improvement informed by subject matter experts. It involves interplay between INDUCTIVE and DEDUCTIVE logic.
Lewis (2015) says that IS involves three key questions: What are we seeking to improve? How will we know when we have reach our goal? What change can help us reach this goal?
Bryk et al (2015) has 6 principles of I.S.
- Focus on the problem and the user
- Focus on variation in performance (variation in key processes can produce undesirable outcomes, so create systematic protocols to reduce variation)
- See the system that produces the outcomes (current and solution system models and adaptive integration)
- We cannot improve at scale what we can’t measure
- Disciplined inquiry: 3 questions and PDSA cycles
- Use networked improvement communities
Booth et al (2015) study of using NIC to address failure in algebra for underrepresented minority students. Algebra by example.
Defining Mixed Methods Research
Johnson, Onwuegbuzie, & Turner (2007) define MM as the third research paradigm that involves synthesis of moth QUANT and QUAL research methods in order to achieve both breadth and depth of understanding and corroboration.
Johnson, Onwuegbuzie, & Turner (2007): QUANT can help generalize QUAL and QUAL can help interpret and validate QUANT
Johnson, Onwuegbuzie, & Turner (2007): MM is built on foundation of methods triangulation and multiple operationalization
CPC (2018): MM involves: QQ, integration, research design, theoretical framing
CPC (2018): Only use MM for certain research problems that require it (e.g. understand problem before making a survey)
CPC (2018) and Johnson & Onwuegbuzie (2004):
ADVANTAGES of MM: harness strengths and offset weaknesses, more evidence for problem, answers more RQs, new insights, it’s practical
WEAKNESSES: researcher skill, time and money, not always prefered
CPC (2018): Researchers use the worldview that fits their study and can use multiple world views. They recommend pragmatism because it enables pluralism
Mertens (2018): Mixed methods research is best suited to complex social issues, wicked problems.
Worldview, paradigm, Lens, Theoretical Framework, Conceptual Framework,
CPC (2018): Worldview is the beliefs and assumptions about knowledge. Paradigm is specific to a community. Postpositivism, constructivism, pragmatism, transformative, dialectical pluralism
Mezirow (1997) frame of reference=structures of assumptions that frame our understanding of the world. Habits of mind are related habitual ways of thinking. We change frames of reference through critical reflection and dialogue.
Creswell and Miller (2000): The Researcher’s choice of paradigm and lens will inform the type of validity strategies. For participant lens+constructivist=prolonged engagement
THEORETICAL FRAMEWORK:
Lester (2005) argues that the TF comes from established theory that helps guide further research. It helps create RQs and interpret findings
Grant & Osanloo (2014) the TF is like the elevation view blueprint
CONCEPTUAL FRAMEWORK:
Maxwell (2013) argues that a CF is a collection of ideas from prior research and their relationships. It’s like a spot-light that shows you what to look for, but it also can be limiting.
Grant and Osanloo (2014) the CF is like the floor plan view blueprint.
Construct validity
Lochmiller and Lester
Refers to the degree to which a test measures what it intends to measure
Construct Validity
LandL
Refers to the congruence between an abstract co Celt and the way you measure the concept.
Validity
The degree to which inferences that you make about your data are accurate
