Tests Flashcards
(25 cards)
If a test of difference is required what is the difference between independent and related data?
Ind design involves a comparison of two sets of scores from different participants e.g male and female comparisons.
Rel designs involve a comparison of two sets of scores from the same participants e.g repeated measures or matched pairs
What is a difference and what is a correlation?
A difference tests whether the scores in one set of scores is higher or lower than those in another set of scores.
A correlation is when you are concerned with whether there is a relationship or association between two sets of scores.
What is nominal data?
Data collected in categories i.e whether one conformed or not. Scores cannot be put in order of scale.
What is an objective account?
An objective accounts is one that is impartial and ideally could be accepted by any subject as it does not draw on any assumptions or prejudice of subjects.
What is ordinal data?
What is interval data?
What is ration data?
- Scales that are made up by psychologists e.g scores of different ppts can be put in order but no more i.e ratings of aggression as opposed to size of eyes
- Where gaps or intervals between points on the scale can be seen to be identical e.g. someone who took 10 seconds to complete a task genuinely took twice as long as somebody else whereas those who are rated 10 on attractiveness are not necessarily 5 x more attractive than someone who scores 5.
- Ratio data is interval data but with an absolute zero i.e. there can be no negative scores e.g. height as opposed to change in weight where a negative score is possible.
what is the main threat to objectivity?
- people are prone to a variety of cognitive biases. Most people interpret events in biased ways, basing ideas on prior consumption. Different people interpret the world differently- one persons terrorist is another persons freedom fighter. If there is room for interpretation, this then opens up room for biases to come into play, there is subjectivity of interpretation of the same events.
1) List some cognitive bias and then a solution in the scientific process.
1) 1) We have a tendency to see causality where it does not exist: we are naïve judges of cause and effect - if we feel a correlation, we automatically identify a link in cause and effect.
- We can solve this through empirical experiments that provide empirical data .
2) Some behaviour is ambiguous and open to interpretation e.g. how aggressive someone has been, so different observers may make different judgments.
- We can operationalize the evidence and standardize the procedure and get different people to do the same experiment and see if they get the same results.
3) When deciding who goes in each condition in an experiment, researchers might put those who think they will score higher in the condition they want to do better.
- We can use a computer programme and randomly allocate participants.
4) We display hindsight bias: when we see results we think that they are ‘just obvious’ and can make sense of them by making up a theory that explains them.
We make predictions in advance based on a theory that we have worked out before knowing the results
What are the two senses of the term “replicability”?
- The replicability of the procedure
- The replicability of the results
what is meant by replicabilty of procedure and how is it achieved?
Studies should be replicable in the first sense so that other can repeat the research. This enables them to check if results were fluke. It is achieved in a report by including all details of the procedure. This is ensured in a report by including all details of the procedure in the Method section e.g. standardised procedure
What is meant by replicability of results?
If a study Is repeated results will be the same. Results should be replicable in the second sense otherwise we would tend to think that the results are a fluke due to sampling bias.
- If one researcher publishes a statistically significant result it is possible that this is a ‘one off’ finding i.e. a type 1 error. However, if that finding is replicated, this reduces the likelihood of the original finding being due to such an error. In psychology, lab experiments are generally higher in replicability cause the research situation is tightly controlled making it easier for subsequent researchers to repeat the study in precisely the same way.
What is the difference between replicability and triangulation?
When another finding replicates the same EFFECT, we refer to this as triangulation. When research is replicated the exact same procedure is carried out. The difference between relocation and triangulation is often a matter of degree rather than a difference of kind.
What are the 5 Features of Science?
1) Theory construction
2) Hypothesis Testing
3) Use of empirical methods
4) objectivity - “An objective account is one which attempts to capture the nature of the object studied in a way that does not depend on any features of the particular subject who studies it.
An objective account is… an impartial, one which could ideally be accepted by any subject, because it does not draw on any assumptions, prejudices, or values of particular subjects.”
Disinterestedness (another term for objectivity) admonishes (encourages) investigators to proceed objectively, putting aside personal biases and prejudices
5) replicability
what is theory construction?
In psychology we construct theories to explain why humans behave the way they do. A theory is an explanation of why things happen the way they do. Usually there is a known phenomena which we want to explain and theorists try to figure out a plausible theory. Before testing the theory it is useful to check that the impression they have is valid - sometimes there is research that suggests a particular theory. e.g. Asch noticed that the functioning of society seems to depend on the degree of conformity and constructed two theories that this could be due to informational or normative social influence.
What is hypothesis testing?
What does Popper say?
-Theories generate hypotheses; these should be precise and testable. Ideally, a theory will lead to hypothesis that, if supported, can only be explained by that theory e.g. The genetic theory of schizophrenia predicts that identical twins will be more similar than non-identical twins therefore we would expect higher concordance rates in MZ twins. if this occurs, the genetic theory will be supported.
Popper argued that genuine science involved the falsification of hypotheses. We can never be 100% certain that a theory is true, but we can show a theory to be false. For example, the claim that ‘All swans are white’ is falsifiable (and was falsified when black swans were discovered) but is never verifiable as true (because the possibility of a black swan always exists).
This leads to the following claim: science should try to advance knowledge by testing theories and eliminating those which are falsified by evidence. A theory that has been tested but remains unfalsified is not proven to be true, but is the most likely candidate for being true. Risky predictions, i.e., ones that are only likely to be true if that specific theory is true, are better tests of a theory. e.g. Popper was largely unimpressed with Freud’s theories as he argued they were largely unfalsifiable and whenever a finding emerged that contradicted the theory, Freudians could reinterpret the evidence as supporting the theory!
-For Popper, then, hypotheses should be improbable. This does not mean ‘unlikely to be true’ but ‘unlikely to be true unless the theory that leads to them is true’.
For example, if researchers found that separated identical twins had 100% concordance rates for schizophrenia (i.e., if one twin had it, so did the other, in all cases) this would be a very improbable finding unless the genetic theory of schizophrenia was true.
-Scientific progress:
We should expect scientific progress to be slow and gradual, with hypotheses successively rejected or modified as evidence accumulates. Falsification can be conclusive, so theories should not be revived in their original form. Surviving theories should become more subtle and specific in their claims.
What is the use of empirical methods?
The centrality of systematic collection of evidence using objective and replicable methods of research should be obvious by now. The scientific revolutions of the 16th century onwards put data collected through the senses at the centre of attempts to acquire knowledge. Appeal to non- empirical sources (e.g., the Bible or other holy book, authority, divine revelation, etc) were rejected and the only source of genuine knowledge has been identified as that which comes through the senses.
Since then scientists (including psychologists) have developed a set of tools (experimental methods, statistical procedures, etc) that allow the information gathered through experience to be collected systematically, i.e., in objective and replicable ways.
This also leads to the rejection of non- systematic, though empirical, ways of acquiring information. Common sense and everyday observation are not sufficient to obtain knowledge, as these are subject to errors and biases. They may suffice for everyday purposes, but science requires more systematic ways of gathering information.
what are the 5 ways that we can validate new knowledge?
1) replication
2) Triangulation
3) Cross-cultural research
4) review
5) Meta-analysis
Validating New Knowledge…
Triangulation?
This is having more than one source of data of different types of data. Multiple sources of data improves the likelihood that findings are genuine. It is useful to collect using different… methods….measures…. researchers….participants.
Validating New Knowledge…
Cross-cultural research?
this allows knowledge to be validated in different contexts.
Validating New Knowledge…
Reviews?
This involves summarising the results from research on the topic, allowing researchers to identify overall trends in the findings. They usually take the form of a narrative.
Validating New Knowledge…
what are the two problems with Reviews?
Cherry-picking - Only including studies that support the authors viewpoint.
Statistical analysis often not very systematic - at best overall counts of positive/neutral/negative findings were made, which ignores the size of differences and the sample sizes of research.
Validating New Knowledge…
Meta-analysis?
More sophisticated form of review
1) Less likely to involve cherry picking cause - methodological criteria can be used to decide if a study should be included e.g whether certain controls were used. This enables others to replicate the study and check the results. Nowadays searches of the literature are conducted using standardised search terms on computerised data bases, so there is no excuse for not doing a thorough search.
2) results from studies are combined in a systematic way - often merged to give one overall score, known as the effect size which expresses the difference between two sets of scores in a standardised way. This is like a weighted average - often significant effect might emerge from a meta - analysis that did not emerge from individual studies I.e in small studies a result might not be significant but combining the studies allows a significant effect to be identified.
Two sample test…
Level of data Test of diff Tests of corr
Independent Related
design design
Nominal data: Chi-sqaured Sign test xxxxxxxxxx
Ordinal, interval: Mann-Whitney Wilcoxon Spearmans
Ratio: T-test T-test Pearsons
Validating New Knowledge…
Meta-analysis - the file drawer problem?
If researchers do not find a significant result in their research they may; 1) not bother to submit their research for publication or 2) be less likely to get it published if they do submit it. This could lead to a misleading pattern of results in the literature, even in meta-analyses. E.g if trials of drug therapies are only published if they work, then thr drugs could be a lot less effective than claimed - if at all.
Example of the file drawer problem - a group of academics published paper in New England journal of medicine which listen all the trials on SSRIs that hadnt been formally registered on the FDA. 37 studies ssdd assessed by the FDA as positive and with one exception every one got properly written up and published. Meanwhile, 22 studies that had negative or iffy results were simply not published at all. 11 were written up and published in a way that described them as having positive outcomes.
PEER REVIEWS
1) What is the purpose of peer review?
2) what are the stages?
Peer review is the process of subjecting a piece of research to independent scrutiny by other psychologists working in a similar field who consider the research in terms of its validity, significance and originality.
1) The point of peer review process is designed to ensure that good quality research that increases the knowledge base of psychology is published. To be accepted as credible research must be published in academic journals for which there is a peer review process.
2) stage 1 - by experts from the journal - they decide if research is worth publishing based on methodological and other criteria. In an ideal word, blind reviewing processes would be used; the reviewer would be unaware of the identity of the researcher and or the reviewer would review the study methodologically I.e without looking at the results. Research is then either rejected, accepted or provisionally accepted.
Stage 2 - by wider academic I.e those who conduct research in the same field, can respond in a variety of ways. community - when research is published - challenge findings, develop or amend theories, cites in other papers. One simple way to judge thr worth of a piece of research is simply by counting citations - how often others include reference to it in their papers
