Chapter 2 - Research Methodology Flashcards
(47 cards)
What are the three main goals of science? What types of research studies can be used to do each?
Description, predication and explanation.
For description and prediction, you can use observational and correlational studies, which only observe certain trends but cannot come up with any causal theories because you cannot control outer factors. Experimental gives the researcher maximal control and allows them to study cause and effect relationships in more depth.
A theory is…
A hypothesis is…
A theory is an explanation or model of how or why a phenomenon works. This can only be determined by experimental methods, however predictions can be made with descriptive studies based on a trend.
A hypothesis is a prediction that is narrower than the theory and is testable.
What is the pattern for the scientific method? What is Occam’s razor law of parsimony?
What makes a good theory?
What is the role of a theory?
Theory —> hypothesis —> research —> support or refute
If support, then strengthen or revise the theory to be more specific. If refute, then discard or revise the theory to try to take out confounding variables.
This is a cyclical process that includes constant revision and evolves over time.
Occam’s law of parsimony says that when two theories explain the same phenomenon, the simpler theory is preferred.
A good theory evolves over time and is able to be falsified. It is supported by a lot of data and is testable. The more the replications, the more reliable the theory until it becomes a law. Also, all theories have some assumptions, but the less assumptions and hence possible confounding variables, the more parsimonious and reliable it is!
Too many assumptions = very low predictability.
A theory integrates unrelated facts and principles into a coherent whole.
Give a brief overview of the systematic way to test a hypothesis, with 7 steps:
What do reports include (4 parts)
1) Frame a research questions that is basic and general and able to be tested with data.
2) Conduct a literature review to ensure your information is novel and to understand what information already has been collected.
3) form the hypothesis based on previous research that is more specific and will guide your studies
4) Design the study in one of the three methods: between group study, within group study, or mixed. We also need to operationally define the variables and topics being studied because they may be abstract or vague. This tells other researchers exactly what variables should eb used and how they can be measured to accurately replicate the study.
5) Conduct the study and collect data.
6) Analyze data: Describe it, what is the average and what are the SEMs, what conclusions can be drawn, are there any confounding variables? Did the results happen by chance — SEM overlap.
7) Report the results and restart the process changing factors and controlling for others that might have affected the results.
To present reseracrh:
Poster sessions, reports presented at conferences, full reports in peer-reviewed scientific journals, pre-prints before anything is confirmed.
These reports include:
1) Background and significance
2) Full methodology for how the question was studied, and report the hypothesis and what was believed before.
3) Complete results of the statistical analysis, don’t leave out any null results,
4) Discussion of what the results mean in relation to past evidence and what you believed in your hypothesis.
Why is replication essential?
Results could have been due to chance or there could have been other confounding variables actually causing the observed pattern. If it doesn’t replicate, then the original study was a false positive, due to confounding variables or some random results. We want to increase accuracy so that results aren’t constantly changing.
What are four questionable research practices that produce false results?
Small samples: Using samples like this can amplify random error due to chance. Because the smaller the population, the less that sample represents the overall population, and hence the less the results can be generalized. Random chance can influence a smaller sample more accurate results.
HARKing: HARK stands for “hypothesizing after the results are known”. This skews the effectiveness fo the results because researchers can ignore data that did not support their hypothesis and amplify data that did. This makes the relationship look much stronger than it actually is. It also leads people to think a study was designed to support a certain hypothesis when in reality that might have been a factor they didn’t even think about that caused the results and they are highlighting it afterward. So you must be explicit about what the predicted results were before.
P-hacking: This is when statistical tests are run over and over again until a relationship is found. We can’t just select the most optimal set of data for your work because this is not reliable.
Underreporting null results: Null means we found that there was no relationship between those two factors. If this is not reported, then the strength of their relationship will appear much stronger then it actually is in reality, which skews the results for viewers. Makes the strength of evidence for the hypothesis look more than it is.
What is something that can be done to make results more reliable once many studies studying similar factors are done?
Meta-analysis. This is when we analyze multiple analysis’s of different trials that were already conducted. This allows us to compare different studies which look at similar things, and hence we can have a much larger data base which allows us to have more accurate conclusions from this data. In this analysis, we look at how those studies were designed and the more reliable studies are weighed more heavily.
Once you’ve defined a hypothesis, you have to address the research method to be used. There are three types (briefly describe them):
Descriptive research: Observing behaviour without manipulation to describe that behaviour objectively and systematically, but we cannot determine any causal relationships from this. We can sometimes predict when and how those behaviours might occur, but we can’t explain because there are too many confounding variables that could affect the results that we cannot control for.
Correlational Studies: Examine how variables are related in the real world without any attempt to alter them or conclude that one variable causes another — again there are too many things that cannot be controlled for that could be effecting results, and we cannot determine the direction of the correlation or the strength using just correlational. So this is essentially descriptive research but between two different phenomenons.
Experimental studies: Allows us to control variables to tailor an experiment that details how two variables are related, to what strength and what direction is the relationship. We can manipulate one variable and measure the effect this has on another variable. To check for CAUSATION, we must use experimental tests to rule out other factors that could create that relationship.
What are the three types of descriptive research methods? Which one requires the most care in interpreting the results and why?
1) Case study: This is an intense observation of a very unique phenomenon — atypical person or organization. Because this is so unique, it does not happen common so you cannot collect data from many people. hence these results cannot really be generalized across the population, however relationships can be predicted based on this. This requires the most care in interpreting the results because the results will likely not generalize beyond the one person being studied.
2) Observational studies: There are two main techniques for this:
1. Participant observation: Where the researcher is present in the situation and immerses themselves within the experiment by talking to and asking the subjects questions, so they can manipulate the study partly by asking specific questions or putting out different options to chose from, but they can’t manipulate the subjects directly. They can only watch how the people react.
2. Naturalistic observation: Observer is passive and separated form the situation, and makes no attempt to alter behaviour.
To do this you must specifically define terms using an operational definition which is essentially for replication. Also behaviour should be coded to make it easy to compare between people. This is often done to study animal behaviour or interactions in social settings without any manipulation or set up.
3) Self reports and interviews:
Ask people about thoughts and feelings which is more interactive but can effect data depending on where and how this data is being collected. These responses are usually converted into numeric form, making them easy to analyze.
What are correlational studies? How do you display correlational studies? What is the correlation coefficient?
correlational studies examine how variables are related in the real world without any attempt to manipulate them. This can get confusing with experimental but if there are no interactions with the participants other then analyzing the data, and nothing is manipulated — only collected based on different traits. So although interviews may ask certain questions people are randomly divided and no one is manipulated. We cannot describe causal relationships this way.
To display correlational studies we use a scatterplot, which plots one variable against another and the closer the dots are to forming a line the more strong the relationship. just because one variable is on the y-axis that doesn’t mean it is the responding, this is jut describing the relationship. Again we don’t know the direction here, like which variable causes which because we cannot control for any variables to solidify this.
The correlation coefficient is a numerical measure to describe the strength of the relationship between two variables and what direction one goes as the other goers up or down.
Greater magnitude = smaller scatter of value = stronger correlation (so closer to 1 or -1 means stronger). this means the slope is 1 and so they follow closely the same pattern.
Negative: This means that as one variable goes up the other goes down and vice versa. The slope is less then zero so from left to right the graph should go down.
Positive: This means the variable move in the same direction — increase and decrease together. They will still have a positive slope even if they both decrease at the same time. But again we cannot assume one causes the other:
Ex) Heart attack rates are negatively correlated with exersize. So if you increase exersize, does that mean you decrease heart attacks? Or if you are at more risk for heart attacks, then you are more willing to do exersize? Sometimes someone can be at more risk for heart attacks which motivates them to do exersize and this flips the relationship. Clearly nothing can be determined from this correlational data.
What are complications that prevent causal conclusions? (2) overall, the more general the correlation, the more …..
Directionality problem: How do we find out the direction of the relationship between two variables, or in other words which one effects the other? We have to manipulate one variable in order to do this and we can’t with descriptive/observational studies. Does happiness cause wealth or wealth cause happiness? We would have to manipulate wealth and see if it affects happiness and vice versa.
Third variable problem: Since we cannot control for all variables in these studies, there may be a three variable C which is changing A and B, instead of A causing B to do something.
Ex) People who are highly sensitive towards rewards in daily lives would be more likely to use e-cigarettes and will seek approval from peers, and this is something effecting both variables.
The more general the correlation, the more variables that can affect the relationship, so using correlational research we cannot conclude that one variable causes another!
Why are correlational studies used if they don’t provide certainty?
They are used for ethical reasons, we cannot manipulate someone to drink and drive. But we can study patterns and once we collect a large data set we can make conclusions.
What is the main benefit of an experimental method? How can you tell when this method is being used?
This method allows us to control variables, ruling out confounding variables and measuring the effect of a manipulated variable on the desired variable. This helps us to draw a causation conclusion.
What is an independent variable? There are ________ __________ of independent variables.
An independent variable is the variable the researcher manipulates across different test subjects, and is the only thing changing if all other variables are controlled effectively. This allows results to be focused on one simple cause and effect relationship. There are different levels of independent variables, which can be applied to different groups to determine to what extent that relationship goes.
What is the dependent variable?
The dependent variable is the one variable that should be changing, and because everything else is controlled we assume this is in response to the manipulated variable.
What is random assignment? Why is it used?
Random assignment is when you have a sample, and you randomly chose who from the sample will have the manipulated variable applied on them, and who will just be the control. or in other words, you are deciding who is the experimental group and who is the placebo. This allows everyone in the population to have an equal chance at being assigned to each, and although due to random chance variances in background and values may effect the results, overall in a large enough sample size this allows results to be as close to realistic as possible. With a large enough sample size and using random assignment, the averages will be realistic.
So, groups will be slightly different based on chance, but these differences should balance out if the sample size is large enough! If you don’t randomly assign people, then you can’t be sure if the differences in the results are due to the way the groups were selected.
What is the between groups design for the experimental method? What is its drawback?
For this design, if there are 20 data points in each condition (the experimental and control groups), how many people participated in the study?
The between groups design is when each treatment consists of a different group of people. So one group is assigned the manipulated variable, the other is assigned the control, and the results of the dependent variable in each group are compared. The issue with this method is that differences could be due to random differences within the groups of people. These are confounding variables that can skew the results.
For this design, if there are 20 data points in both the control and experimental groups, then there are 40 people total because each group consists of 20 DIFFERENT people.
What is the within-subject design and what is another name for it? What is the drawback for this method? How is that drawback regulated?
For this design, if there are 20 data points in each condition (the experimental and control groups), how many people participated in the study?
The within-subject design occurs when the same group of people receives both treatments and the results with the dependent variable are compared that way. This is better then the between groups design because any random variances between the groups that may be skewing the results are accounted for because they would be present in both treatments. So by looking at the difference between the groups — as long as measurements are not taken too far apart — we can be sure that this is due to the independent variable, as this is the only thing changing between two groups.
The drawback for this method is that when participants repeat the task, they will have more experience the second time and hence this could influence performance. However, this can be regulated by randomly varying the order of the treatments, so that any skewed results balance out when the average is taken.
For this design, if there are 20 data points for each group, then that means there are 20 people participating. Because each group consists of the same people, we are just comparing before and after the treatment!
What is a confound and how does it affect causality? What type of error does this lead to?
A confound is anything that affects the dependent variable and that is not accounted for in the independent variable. It unintentionally varies between the study’s experimental outcomes, and because we assume the changes in the DV are due to the IV in experiments, this can directly skew the results. Essentially, this is an “undetected third variable”. Controlling for confounds is essential when trying to articulate causality, because it allows the researcher to eliminate alternative explanations. If a confound is present, it could produce a causality which is not accurate based on the information that the researcher has. The more confounds that can be eliminated, the more confident you can be that the IV caused the DV to change.
This leads to systematic error which cannot be combatted with a larger sample size.
What is the group called that researchers ultimately want to know about? How do they realistically study this group?
The group the researchers want to know about is the population, but obviously it is not possible to do this. So using the resources provided, researchers will use as large a sample size as they can get from the population, to try and produce the most accurate average. So the sample is the people selected from the population for use in that study.
What is random sampling? What is a convenience sample? Why is truly random sampling hard to obtain?
Random sampling: This is where you randomly choose a group of people from the population to be studied — without offering any motives, because this can skew that randomality.
A convenience sample is when people choose whoever is easiest to study for the sample, which can be extremely biased. Therefore, we cannot generalize the results as well because it was only for a specific group of people.
Truly random sampling is very hard to obtain because you can’t really chose from people all across the world. Instead its easiest to chose people in your own city or even neighborhood, based on price. The issue is, belief systems and languages don’t easily transfer between cultures and so it is hard to incorporate a lot of diversity within these studies.
Can random assignment be used for descriptive, experimental and correlational? What about random sampling?
Random assignment can only be used for experimental because this is where the researcher is producing different groups to assign certain variables to. They can’t do this in the other two studies because they are simply observing. Random sampling however can be used for any kind of study, because this is how you chose the group you are going to study or manipulate since you can’t ever use the whole population.
What are the three main pillars in ethics for research, and what does each mean?
1) Respect for persons: People must maintain autonomy when deciding to participate as well as throughout the study. They need to know what is happening — not too much if it will affect the study — so that they can give informed consent. CONFIDENTIALITY AND ANONYMITY are also part of this.
2) Beneficence: Minimize risks as much as possible. Risks can be larger for studies with a high chance of a large success rate. But for small success chances, risk must be small.
3) Justice: Fairness in the distribution of costs and benefits of research across a population. You cant just give one specific ethnicity the trial but then provide the results to everyone to benefit from. It should be randomly tested on the population.
What is the mixed design?
For this design, if there are 20 data points in each condition, how many people participated in the study?
The mixed design is when you use within-subjects and between-subjects design. So you use two different groups, one is the control group and one is the manipulated group. You measure both before and after, and see what the change in their variable is. This way any differences in the groups are not going to effect the result because we are just looking at changes!
For mixed design, if there were 20 data points in each condition, (so 20 for one group and control, 20 for the other group and control, 20 for one group and experimental, 20 for the other and experimental — there are 4 conditions), then there would have been 40 participants because there are two groups with 20 in each.
