Year 2 Research methods Flashcards
1
Q
What is research methods?
A
The processes by which information or data is collected usually for the purpose of testing a hypothesis and/ or a theory.
2
Q
What is a correlation?
A
A mathematical technique in which a researcher investigates as association between two variables (co-variables).
3
Q
What is a correlation coefficient?
A
A number between -1 and +1 that represents the direction and strength of a relationship between co-variables.
4
Q
What are correlations plotted on?
A
Correlations are plotted on a scattergram where each axis represents one of the variables investigated.
5
Q
What is a +1 correlation?
A
A +1 correlation is a perfect positive correlation.
6
Q
What is a -1 correlation?
A
A -1 correlation is a perfect negative correlation.
7
Q
Explain what the correlation coefficient means.
A
The closer the coefficient is to +1 or -1, the stronger the relationship between the co-variables is; the closer to zero, the weaker the relationship is.
However, certain coefficients that appear to indicate weak correlations can still be statistically significant - it depends on the size of the data set.
8
Q
What does ‘r’ mean when talking about correlations?
A
The letter ‘r’ stands for correlation coefficient.
9
Q
What is descriptive statistics?
A
Descriptive statistics refer to things like graphs, tables and summary statistics (such as measures of central tendency and measures of dispersion). They are used to identify trends and analyse sets of data.
10
Q
What is inferential statistics?
A
Inferential statistics refers to the use of statistical tests which tell psychologists whether the differences or relationships they have found are statistically significant or not. This helps decide which hypothesis to accept and which to reject. A correlation coefficient is calculated using a statistical test an, as such, is an inferential statistic.
11
Q
What is a case study?
A
An in-depth investigation, description and analysis of a single individual, group, institution or event.
They often involve analysis of unusual individuals or events .
12
Q
How do you conduct a case study?
A
Conducting a case study usually involves the production of qualitative data. Researchers will construct a case history of the individual concerned and they may be subject to experimental or psychological testing to assess what they are (or are not) capable of, and this may produce quantitative data.
Case studies tend to take place over a long period of time (longitudinal) and may involve gathering additional data from family and friends of the individual as well as the person themselves.
13
Q
What is content analysis?
A
A research technique that enables the indirect study of behaviour by examining communications that people produce (in texts, emails, TV, film and other media).
This is also turning qualitative data into quantitative data.
14
Q
Explain content analysis.
A
Content analysis is a type of observational research in which people are studied indirectly via the communications they have produced. The aim is to summarise and describe this communication in a systematic way so overall conclusions can be drawn.
15
Q
What is coding in content analysis?
A
Coding is the initial stage of content analysis. It is the stage in which the communication to be studied is analysed by identifying each instance of the chosen categories.
It is essentially categorising information into meaningful units.
16
Q
What is thematic analysis in content analysis?
A
Thematic analysis is an inductive and qualitative approach to analysis that involves identifying implicit or explicit ideas within the data. Themes will often emerge once the data has been coded.
17
Q
What is a theme in content analysis?
A
A theme in content analysis refers to any idea, explicit or implicit, that is recurrent.
Once a theme has been identified it can then be developed into broader categories.
18
Q
What happens once a theme has been identified in content analysis?
A
Once the researcher is satisfied that the themes they have developed cover most aspects of the data they are analysing, they may collect a new set of data to test the validity of the themes and categories.
Assuming these explain the new data adequately, the researcher will write up the final report, typically using direct quotes from the data to illustrate each theme.
19
Q
What are the strengths of case studies?
A
- Case studies are able to offer rich, detailed insights that may shed light on very unusual and atypical forms of behaviour. This may be preferred to the more superficial forms of data that might be collected from an experiment or questionnaire.
- Case studies may contribute to our understanding of ‘normal’ functioning. E.g. the case of HM demonstrated ‘normal’ memory processing - the existence of separate stores in STM and LTM.
- Case studies may generate hypotheses for future study and one solitary, contradictory instance may lead to the revision of an entire theory.
20
Q
What are the limitations of case studies?
A
- Generalisation of findings is obviously an issue when dealing with such small sample sizes. Also, the information that makes the final report is based on the subjective selection and interpretation of the researcher.
- Personal accounts from participants or their family and friends may be prone to inaccuracy and memory decay, especially if childhood stories are being told. This means that the evidence from case studies begins to look low in validity.
21
Q
What are the strengths of content analysis?
A
- Content analysis is useful because it can get around (circumnavigate) many of the ethical issues normally associated with psychological research. Normally all the evidence needed all ready exists and so there is no issues with obtaining permission.
- Communication of sensitive nature such as a conversation by text is high in external validity.
- Content analysis is flexible as it can produce both qualitative and quantitative data depending on the aims of the research.
22
Q
What are the limitations of content analysis?
A
- People tend to be studied indirectly as part of content analysis so the communication they produce is usually analysed outside of the context within which is occurred. There is danger that the researcher may attribute opinions and motivations to the speaker or writer that were not intended originally.
- Content analysis may suffer from a lack of objectivity, especially when more descriptive forms of thematic analysis are employed.
23
Q
Describe Matthews et al’s study?
A
Matthews et al analysed 1,200 instances of graffiti gathered from toilet walls in US bars. Graffiti was coded according to a number of distinct categories: sexual references, socio-political, entertainment, physical presence, love/romance and scatological (an interest in excrement and excretion).
Graffiti was also classified in terms of whether it was interactive (a response to other graffiti) or independent.
They found that males composed significantly more sexual and physical presence graffiti, whilst females authored more romantic and interactive graffiti.
24
Q
How do you conduct a content analysis?
A
- Sampling method – Time or event sampling?
- Recording data – Should the data be transcribed or recorded? Should the data be collected by an individual researcher or within a team?
- Analysing and representing data – How should the material be categorised or coded in order to summarise it? Should the number of times something is mentioned be calculated (quantitative analysis) or described using themes (qualitative analysis)?
25
What is reliability?
Reliability is a measure of consistency, it refers to how consistent the findings from an investigation or measuring device are. In general terms, if a particular measurement can be repeated then that measurement is described as being reliable.
26
What are the two ways to assess reliability?
Test-retest
| Inter-observer reliability
27
What is test-retest reliability?
A method of assessing the reliability of a questionnaire or psychological test by assessing the same person on two separate occasions. This shows to what extent the test (or other measure) produces the same answers (to show if it is consistent or reliable).
There must be sufficient time between test and retest to ensure that the participant cannot recall their answers to the questions to a survey but not so long that their attitudes, opinions or abilities may have changed.
28
How do you test-retest using a questionnaire or test?
The two sets of scores would be correlated to make sure they are similar. If the correlation turns out to be significant (and positive) then the reliability of the measuring instrument is assumed to be good.
29
What is inter-observer reliability?
The extent to which there is agreement between two or more observers involved in observations of a behaviour. There is an issue relevant to observational research that one researcher's interpretation of events may differ widely from someone else's - introducing subjectivity, bias and unreliability into the data collection process. Inter-observer reliability is measured by correlating the observations of two or more observers.
30
What is the formula for reliability / inter-observer reliability?
A general rule is that if (total number of agreements) / (total number of observations) > +0.80, the data has high inter-observer reliability.
31
How is inter-observer reliability established?
This may involve a small-scale trial run (a pilot study) of the observation in order to check that observers are applying behavioural categories in the same way, or it may be reported at the end of a study to show that the data collected was reliable.
Observers need to watch the same event, or sequence of events, but record their data independently.
The data collected by the two observers should be correlated to assess its reliability.
*When using this with content analysis or interviews it is referred to as inter-rater reliability and inter-interviewer reliability*.
32
How do you improve reliability with questionnaires?
The reliability of questionnaires over time should be measured using the test-retest method. Comparing two sets of data should produce a correlation that exceeds +0.80.
A questionnaire that produces low test - retest reliability may require some of the items to be 'deselected' or rewritten.
33
How do you improve reliability with interviews?
For interviews the best way to ensure reliability is to use the same interviewer each time. If this is not possible or practical, all interviewers must be properly trained so.
E.g. An interviewer should not ask questions that are too leading or ambiguous. This is more easily avoided in structured interviews where the interviewer's behaviour is more controlled by the fixed questions.
Interviews that are unstructured and more 'free-flowing' are less likely to be reliable.
34
Why are experiments reliable?
| What can affect reliability?
Lab experiments are often described as being 'reliable' because the researcher can exert strict control over many aspects of the procedure.
One thing that may affect the reliability of a finding is if participants were tested under slightly different conditions each time they were tested.
35
How do you improve reliability with observations?
The reliability of observations can be improved by making sure that behavioural categories have been properly operationalised, and that they are measurable and self-evident.
Categories should not overlap and all possible behaviours should be covered on the checklist.
If categories are not operationalised well, or are overlapping or absent, different observers have to make their own judgements of what to record where and may well end up with differing and inconsistent records.
36
Define validity.
The extent to which an observed effect is genuine.
37
What is validity?
Validity refers to whether a test, observation, experiment... produces a result that is legitimate. It is whether the observed effect is genuine and represents what is actually 'out there' in the real world.
38
What is internal validity?
Whether the researcher has managed to measure what they intended to measure.
39
What is external validity?
The extent to which findings can be generalised beyond the research setting in which they were found
40
Can a test be reliable but not valid?
It is possible for studies and measures to produce reliable data that is not valid.
E.g. a broken set of scales may give a consistent reading of someone's weight which is an inaccurate reading. Therefore, the scales are reliable but the weight recorded is not true so the measurement lacks validity.
41
What does internal validity refer to?
Internal validity refers to whether the effects observed in an experiment are due to the manipulation of the IV and not some other factor.
One threat to internal validity is if participants respond to demand characteristics and act in a way they think is expected.
42
What is ecological validity?
Ecological validity concerns generalising findings from one setting to other settings. It is a form of external validity that measures the extent to which findings from a research study can be generalised to other settings and situations.
It is not about how natural a setting is but how much a task is like everyday life.
43
What is temporal validity?
Temporal validity is the issue of whether findings from a particular study, or concepts within a particular theory, hold true over time. It is a form of external validity.
44
What is face validity?
A basic form of validity that measures whether something, a test or scale, etc, appears to on the face of it measure what it is supposed to measure.
45
What is concurrent validity?
The extent to which a psychological measure relates to an existing similar measure.
A test would have concurrent validity if the results obtained are very close to, or match, those obtained on another recognised and well-established test.
46
How do you improve validity on experimental research?
In experimental research, validity is improved in many ways.
- Using control groups means that a researcher is better able to assess whether changes in the dependent variable were due to the effect of the IV.
- Standardising procedures minimises the impact of participant reactivity and investigator effects on the validity of the outcome.
- Single blind and double blind procedures also reduce the impact of participant reactivity and investigator effects.
47
How do you improve validity on questionnaires?
Many questionnaires and tests incorporate a lie scale within the questions in order to assess the consistency of a respondent's response and to control for the effects of social desirability bias.
Validity may be further enhanced by assuring respondents that all data submitted will remain anonymous.
48
How do you improve validity on observations?
Observational research may produce findings that have high ecological validity as there may be minimal intervention by the researcher, especially in covert observations.
Behavioural categories that are too broad, overlapping or ambiguous may have a negative impact on the validity of the data collected. Therefore, behavioural categories need to be identified to increase validity.
49
Does qualitative or quantitative data have higher ecological validity?
Qualitative methods of research usually have higher ecological validity than quantitative, less interpretative methods of research. This is because the depth and detail provided by qualitative methods is better able to reflect the participant's reality.
50
What is interpretive validity?
The extent to which the researcher's interpretation of events matches those of their participants.
51
What is triangulation?
A way to further enhance validity, it is the use of a number of different sources as evidence.
52
What are the levels of measurement?
Nominal data
Ordinal data
Interval data
53
What are statistical tests?
Tests that are used to determine whether a significant difference or correlation exists and consequently whether the null hypothesis should be rejected or accepted.
54
What is the criteria for the test chi-squared?
A test for association (difference or correlation)
Nominal data
Independent groups
55
What is the criteria for the test Mann-Whitney?
A test for a difference
Ordinal data
Independent groups
56
What is the criteria for the test Pearson's R?
PARAMETRIC
Test for correlation
Interval data
Design is not important as we are testing for a correlation.
57
What is the criteria for the test related t-test?
PARAMETRIC
Test for difference
Interval data
Repeated measures
58
What is the criteria for the sign test?
Test for difference
Repeated measures
Nominal data
59
What is the criteria for the test Spearman's Rho
```
Test for correlation
Ordinal data (it can be used with interval data)
Type of design is not important as we are testing for a correlation.
```
60
What is the criteria for the unrelated t-test?
PARAMETRIC
Test for difference
Interval data
Independent groups
61
What is the criteria for the test Wilcoxon?
Test for difference
Ordinal data
Repeated measures
62
What is nominal data?
Data represented in the form of categories.
| It is discrete data (only one item can appear in one of the categories).
63
What is ordinal data?
Ordinal data is ordered in some way.
It doesn't have equal intervals between each unit and it lacks precision because it is based on subjective opinion rather than objective measures.
64
What is interval data?
Interval data has a standardised scale that includes units of equal, precisely defined size.
The data is more detailed and the most precise and sophisticated form of data.
65
How do the levels of measurement relate to the measures of central tendency and measures of dispersion.
Nominal data = Mode = N/A
Ordinal data = Median = Range
Interval data = Mean = Standard deviation
66
What is probability?
A measure of the likelihood that a particular event will occur where 0 indicates statistical impossibility and 1 statistical certainty.
67
Define significance.
A statistical term that tells us how sure we are that a difference or correlation exists.
A significant result means that the researcher can reject the null hypothesis.
68
What is the critical value?
When testing a hypothesis, the critical value is the numerical boundary or cut-off point between acceptance and rejection of the null hypothesis.
69
What is a type 1 error?
The incorrect rejection of a true null hypothesis (a false positive).
It is when the critical value has been too lenient, 0.10.
When the null hypothesis is rejected and the alternative hypothesis is accepted, when it should have been the other way round.
70
What is a type 2 error?
The failure to reject a false null hypothesis (a false negative).
It is when the critical value has been too stringent, 0.025.
When the null hypothesis is accepted but it should have been the alternative hypothesis.
71
What is the null hypothesis?
A hypothesis that states that there is 'no difference' between the conditions.
72
What does a statistical test determine?
A statistical test determines which hypothesis is true and thus whether to accept or reject the null hypothesis.
73
What is the significance level?
The point at which the researcher can claim to have discovered a significant difference or correlation within the data.
(The point at which the researcher can reject the null hypothesis and accept the alternative hypothesis).
74
What is the usual level of significance in psychology?
0.05 or 5%.
| p< or equal to 0.05.
75
What does 'p' stand for?
Probability.
76
What does p<0.05 mean?
That the probability that the observed effect (the result) occurred by chance is equal to or less than 5%.
This means that even though the result is significant, there is still up to 5% probability that the observed effect occurred by chance.
77
What can't probability be 100%?
Psychologists can never be 100% certain about a particular result as they have not tested all members of the population under all possible circumstances.
78
What is the calculated value?
The number result once the test has been calculated.
79
How do you check for statistical significance?
To check for statistical significance, the calculated value must be compared with a critical value.
80
How does the critical value have to compare with the calculated value for a test to be significant?
The tests with a letter R in their name require the calculated value to be equal to or more than the critical value.
The other tests require the calculated value to be equal to or less than the critical value.
81
What are the criteria for knowing what critical value to use?
- One-tailed or two-tailed test.
- The number of participants in the study (N value).
- The level of significance (P value).
82
When may a more stringent level of significance be used?
A more stringent level of significance may be used (0.01) in studies where there may be a human cost - such as drug trials or trials that for practical reasons cannot be repeated in the future.
In all research, if there is a large difference between the calculated and critical values, the researcher will check more stringent levels, as the lower the p value is, the more statistically significant the result.
83
Why is p 0.05?
Because the 5% level of significance balances the risk of making a type 1 or type 2 error.
84
What is the criteria for parametric tests?
Interval data
Normally distributed sample
Homogeneity of variance (spread of scores will be similar)
85
Name the parametric tests.
Unrelated T test
Related T test
Pearson's R
86
Define abstract.
The key details of the research report. It is the first section in a journal article and is a short summary around 150-200 words that includes all the major elements: aims and hypotheses, method/procedures, results and conclusions.
When researching a particular topic, psychologists will often read lots of abstracts in order to identify those investigations that are worthy of further examination.
87
Define introduction.
The introduction is a literature review of the general area of investigation detailing relevant theories, concepts and studies that are related to the current study.
The research review should follow a logical progression - beginning broadly and gradually becoming more specific until the aims and hypotheses are presented.
88
Define method.
A description of what the researcher did, including design, sample, apparatus/materials, procedure and ethics.
The method should include sufficient detail so that other researchers are able to precisely replicate the study if they wish.
89
What are the sections of method?
```
Design
Sample
Apparatus/materials
Procedure
Ethics
```
90
What is design in 'method'?
The design is clearly stated and reasons or justification should be given for the choice.
independent groups
naturalistic observation
91
What is sample in 'method'?
Information related to the people involved in the study :
how many were there?
biological/demographic information (as long as it doesn't compromise anonymity)
sampling method
target population
92
What is apparatus/materials in 'method'?
Detail of any assessment instruments used and other relevant materials.
93
What is procedure in 'method'?
A list of everything that happened in the investigation from beginning to end.
This includes a verbatim record of everything that was said to participants (briefing, standardised instructions and debriefing).
94
What is ethics in 'method'?
An explanation of how these were addressed within the study.
95
Define results.
The results section should summarise the key findings from the investigation. This is likely to feature descriptive statistics (tables, graphs and charts, measures of central tendency and measures of dispersion).
Inferential statistics should include reference to the choice of statistical test, calculated and critical values, the level of significance and the final outcome (what hypothesis was accepted and what hypothesis was rejected).
Any raw data that was collected and any calculations appear in an appendix rather that the main body of the report.
If the researcher has used qualitative methods of research, the results/findings are likely to involve analysis of themes and/or categories.
96
Define discussion.
A consideration of what the results of a research study tell us in terms of psychological theory.
97
What happens in the discussion part of a scientific report?
The researcher will summarise the results or findings in verbal, rather than statistical form.
These should be discussed in the context of the evidence presented in the introduction and other research that may be considered relevant.
98
What should the researcher be mindful of in the discussion part of a scientific report?
The researcher should be mindful of the limitations of the present investigation and discuss these as part of this section. This may include reference to aspects of the method, or the sample for instance, and some suggestions of how these limitations might be addressed in a future study.
99
What is the last section of the discussion part of a scientific report?
Finally, the wider implications of the research are considered. This may include real-world applications of what has been discovered and what contribution the investigation has made to the existing knowledge-base within the field.
100
Define references.
| Sections of a scientific report
List of sources that are referred to or quoted in the article, and their full details.
Full details of any source material that the researcher drew upon or cited in the report.
101
What format do book references take?
Book references take the following format: author, date, title of book (in italics), place of publication, publisher.
The name of the journal and title of the book appear in italics as does the journal volume and issue number. For a journal article the last information is the page number.
102
What is a paradigm?
A set of shared assumptions and agreed methods within a scientific discipline.
103
What is a paradigm shift?
The result of a scientific revolution: a significant change in the dominant unifying theory within a scientific discipline.
104
Define objectivity.
When all sources of personal bias are minimised so as not to distort or influence the research process.
105
What is the empirical method ?
Scientific approaches that are based on the gathering of evidence through direct observation and experience.
106
What is replicability?
The extent to which scientific procedures and findings can be repeated by other researchers.
107
What is falsifiability?
The principle that a theory cannot be considered scientific unless it admits the possibility of being proven untrue (false).
108
What did Thomas Kuhn suggest?
Philosopher Thomas Kuhn suggested that what distinguishes scientific disciplines from non-scientific disciplines is a shared set of assumptions and methods - a paradigm.
Kuhn suggested that social sciences (including psychology) lack a universally accepted paradigm and are probably best seen as 'pre-science' as distinct from natural sciences such as biology or physics.
109
What are natural sciences characterised by?
Natural sciences are characterised by having a number of principles at their core.
110
Why is Psychology a pre-science?
Psychology is marked by too much internal disagreement and has too many conflicting approaches to qualify as a science and therefore is a pre-science.
111
What did Kuhn say about paradigm shifts and psychology?
According to Kuhn, progress within an established science occurs when there is a scientific revolution. A handful of researchers begin to question the accepted paradigm, this critique begins to gather popularity and pace, and eventually a paradigm shift occurs when there is too much contradictory evidence to ignore.
Kuhn cited the change from a Newtonian paradigm in physics towards Einstein's theory of relativity as an example of a paradigm shift.
112
What is a theory?
A theory is a set of general laws or principles that have the ability to explain particular events or behaviours.
A theory proposes a simple and economical principle that appears to reflect reality. It provides understanding by explaining regularities in behaviour.
It should also be possible to make clear and precise predictions on the basis of a theory.
This is the role of hypothesis testing.
113
How does theory construction occur?
Theory construction occurs through gathering evidence via direct observation.
114
What is an essential component of a theory?
An essential component of a theory is that it can be scientifically tested.
Theories should suggest a number of possible hypotheses that can be tested using systematic and objective methods to determine whether the hypothesis will be supported or refuted.
The theory can then be strengthened or it may need to be revised or revisited.
115
What is deduction in terms of theories?
The process of deriving new hypotheses from an existing theory is known as deduction.
116
Who was Karl Popper and what did he argue?
Another philosopher of science whose work appeared around the same time as Thomas Kuhn was Karl Popper (1934) who argued that the key criterion of a scientific theory is its falsifiability.
117
What did Karl Popper say about falsifiability?
Popper suggested that genuine scientific theories should hold themselves up for hypothesis testing and the possibility of being proven false.
He believed that even when a scientific principle had been successfully and repeatedly tested, it was not necessarily true. Instead it had simply not been proven false yet. This became known as the theory of falsification.
118
What is the theory of falsification?
That even when a scientific principle had been successfully and repeatedly tested, it was not necessarily true. Instead it had simply not been proven false yet.
119
What is the opposite of good science?
Popper drew a clear line between good science, in which theories are constantly challenged, and what he called 'pseudosciences' which couldn't be falsified.
120
What theories become the strongest type of theories?
Those theories that survive most attempts to falsify them become the strongest - not because they are necessarily true - but because, despite the best efforts of researchers, they have not been proved false.
This is why psychologists avoid using phrases such as 'this proves' in favour of 'this supports' or 'this seems to suggest' and why an alternative hypothesis must always be accompanied by a null hypothesis.
121
What is Karl Popper's falsifiability claim called?
Popper's hypothetico-deductive method.
122
What does replicability mean?
If a scientific theory is to be replicable and trusted, the findings from it must be shown to be repeatable across a number of different contexts and circumstances.
123
What does replication determine in findings from a research study?
Replication has an important role in determining the validity of a finding.
Replication also has a role of determining the reliability of the method used in a study.
124
How is replication used to assess the validity of a finding?
Replication is used to assess the validity of a finding by repeating a study, as Popper suggests, over a number of different contexts and circumstances then we can see the extent to which the findings can be generalised.
125
What is needed for replicability to become possible?
In order for replicability to become possible, it is vital that psychologists report their investigations with as much precision and rigour as possible, so other researchers can seek to verify their work and verify the findings that have established.
126
How much objectivity should scientific researchers strive for?
Scientific researchers must strive to maintain objectivity (as much as possible) as part of their investigations. They must keep a critical distance during research and must not allow their personal opinions or biases to sway the data they collect or influence the behaviour of participants they are studying.
127
What experiments are associated with the greatest level of control?
As a general rule, those methods in psychology that are associated with the greatest level of control - such as lab experiments - tend to be the most objective.
128
What is the basis of the empirical method?
Objectivity is the basis of the empirical method.
129
What does the empirical methods emphasise?
The word empiricism is derived from the Greek for 'experience' and empirical methods emphasise the importance of data collection based on direct, sensory experience.
The experimental method and the observational method are good examples of the empirical method in psychology.
130
What did empiricist John Locke say?
Early empiricists such as John Locke saw knowledge as determined only by experience and sensory perception.
Thus, a theory cannot claim to be scientific unless it has been empirically tested and verified.
131
What is empiricism in psychology?
Empiricism (founded by John Locke) states that the only source of knowledge comes through our senses – e.g. sight, hearing etc. ... The idea that knowledge should be gained through experience, i.e. empirically, turned into a method of inquiry that used careful observation and experiments to gather facts and evidence.
132
How do you assess reliability?
Test-retest
| Inter-observer reliability
133
How is reliability measured?
Reliability is measured using a correlation analysis.
In test-retest and inter-observer reliability, the two sets of scores are correlated. The correlation coefficient should exceed +0.8 for reliability.
