Contemporary Research Flashcards
(50 cards)
1
Q
Quality criteria (3 most important)
A
- Reliability
- Validity
- Objectivity
2
Q
Reliability: definition
A
whatever the features measure in a trustworthy and robust way
–> measurement errors
3
Q
Validity: definition
A
whatever method measures the design features
–> measure what it aims to measure
4
Q
Objectivity: definition
A
whatever the result has been obtained independently of the researcher/research context
5
Q
Quality criteria: Additional criteria
A
- Transparency
- Geniuses
6
Q
Not reliable/ not valid
A
far from the centre/ point are far away from each other
7
Q
Reliable but not valid
A
far from the centre/point are close
–> Reliability of the procedures is high, but the wrong thing is measured
8
Q
Not reliable but valid
A
point gathering around the centre
–> reliability of the method is low
9
Q
Only between a reliable and valid method
A
A perfect balance between reliability and validity
10
Q
Reliability – are the measurements
trustworthy?
A
- Independence of the measurement from the point in time
- Extent of accuracy and stability of measurement results when the measurement has been performed repeatedly
11
Q
Reliability: Tested through
A
- Test-retest method: measurement can be repeat
- Split-half method: measurement can be split into several separate measurements
- Parallel-test method: different measurements can be developed that measure the same issue
- Internal consistency (Cronbach’s alpha): individual parts are trained in the same direction
12
Q
Cronbach’s alpha: 0,7
A
consider sufficient
13
Q
Cronbach’s alpha: 0,9 or more
A
–> consider good
14
Q
Validity – are the conclusions sound?
A
- Accuracy of data collection
- The extent to which the measuring instrument actually measures the things it is supposed to measure
15
Q
Validity: different types?
A
- Criterion validity : involves the use of a criterion in which a comparison is made
- Construct validity: involves a comparison with other variables that measure the same or something similar
- Content validity : present when the measurement of the construct fully captures it’s content in all its aspects
16
Q
Objectivity – Is the study independent?
A
refers to the independence of the
results from the persons involved in the research
17
Q
Objectivity – Different types:
A
- Objectivity of implementation
- Objectivity of analysis: The data should be analysed by different researchers
–> if the study has many degrees of freedom - Objectivity of interpretation: empirical analysis is reviewed by different researchers
–> only if these researchers answer the research question in the same way then the objectivity is given
18
Q
Replication crisis
A
methodological crisis that spreads through different disciplines
–> increasing doubt about the publication of methodological studies
19
Q
Replicability
A
The ability of independent researchers to obtain the same results using the same methods and data as the original study.
20
Q
Reproducibility
A
The ability to obtain consistent results by independently recreating the study, including collecting new data but following the same methods
21
Q
Causes of the replication crisis
A
- Questionable research practices
- Publication bias
- Low power
- Academic incentives
22
Q
- Questionable research practices
A
- p -hacking
- HARKing
- Optional stopping
- Outcome switching:
23
Q
p -hacking
A
Performing multiple analyses, selectively reporting significant results
24
Q
HARKing
A
Hypothesizing after the results are known
25
Optional stopping
Stop data collection as soon as results become significant
26
Outcome switching
Measuring multiple dependent variables (outcomes), reporting only those with significant results
27
Questionable research practices : Consequence
Probability of type 1 error enormously increased: : The null hypothesis is rejected even though it is true
28
Publication bias
- Studies with “exciting,” “new,” and “surprising” results have a better
chance of being published
- Often, depending on the discipline, only significant results are published
- File drawer problem (number of “missing” non-significant, unpublished
results)
29
Consequence for meta-analyses (Publication bias)
No reliable estimate of the true effect
30
Low power
- Statistical power: Probability of finding an effect if it exists in the population
- Dependent on:
--> the magnitude of the effect (e.g., d, r, eta2)
--> the size of the sample
--> the significance level (e.g., 5%)
31
Low power: consequence
Many not informative studies = waste of resources
32
Academic incentives
- Successful scientific careers usually require a large number of publications
- Magazines are looking for impactful articles with “exciting” and “surprising” results
- Replication of past results is not rewarded
- Careful review of submissions is not adequately rewarded
- Current academic structures favor questionable research practices
- Individual demands on academic careers collide with collective
demands on sound science
33
Academic incentives : Consequence
Research that imitates science but is not
reproducible
34
Definition of Open Science
is transparent and accessible knowledge that is shared and (further) developed through collaborative networks.
35
How to increase transparency
- pre registration study : tell the reviewer in advance what you aim to do
- Registration report : wrote your methods but you don’t run the study
36
Open Science: characteristic
- Open access and transparency in experimental methodology, observation and collection of data
- Public availability and reusability of research data
- Uses web-based tools to engage in scientific collaborations
- Serves quality assurance in research
37
Preregistration
Planning and registration before conducting the study
- Research question
- Hypotheses
- Characteristics of the sample
- Measurements
- Analysis
--> Prevents p-hacking for the most
part
38
Registered report
Writing & peer review of the research project before it is actually carried out
* Research question & literature
* Hypothesis(s)
* Experimental procedure & analysis pipeline
* Sampling plan
* Pilot data (if available)
--> Minimizing publication bias and
HARKing
39
Open Science Framework (OSF)
is a tool that promotes open, centralized
workflows
It covers different aspects of the research process:
- Developing the research idea
- Designing a study
- Storing and analyzing the collected data
- Writing and publishing reports or papers
--> Enables registration of research projects and joint collaboration
40
Open Data as part of Open Science
Open data is data that is openly accessible, usable, editable, and shared
by anyone for any purpose, including
commercial
41
New technologies in research
- Observation is a fundamental method of data collection. It consists of gaining experience in the non-communicative process with the help of all perceptual aids.
1. Virtual Reality
2. Eye-Tracking
3. Emotion Recognition
4. Galvanic Skin Response
42
Virtual Reality
provides a realistic and controlled
experimental environment
Impressions in VR --> Measurement reaction
- Real-time data collection
- Increased reliability and validity, as it is not based on human memory
- Easy manipulation of the experiment environment
43
Eye-Tracking
Eye tracking captures the eye movement of consumers
Recording of eye movements while viewing point-of-sales activities, advertising, website, usability tests, etc.
1. Evaluation of the length of stay of the gazes (“fixation”)
2. Sequence of movements (“saccades”)
3. Pupil size (“excitement”)
44
Pupil Labs
Leading company in the field of eye tracking hardware and software that builds a community of researchers
45
Face recognition
can recognize people in photos,
while Emotion Recognition recognizes a person's emotions
46
Top use cases for face recognition
1. Security & law enforcement
* Issuing identity documents
* Border checks
* Find missing children and disoriented adults
2. Health
* Track a patient's use of medication more
accurately
* Detect genetic diseases
* Support pain management procedures
3. Banking & retail
* Fully mobile user-friendly onboarding
experience
47
Galvanic Skin Response
- Belongs to the generic term of electrodermal activity (EDA)
- Refers to changes in sweat gland activity
- Measures the intensity of our emotional state, also known as emotional arousal
- Does not measure the nature of the emotion
- Both positive and negative emotions trigger changes in sweat gland
activity
48
Behavioral Lab-HSG
fully equipped laboratory with an infrastructure for capturing a wide range of implicit and explicit metrics
- Facial Expression Analysis
- Mouse-Tracking
- Vocal Feature Analysis
- Elektroenzephalographie (EEG)
49
Dos: Bachelor thesis
* Choose a topic that really interests you
* Consult list of available professors
* Look for open topics at institutes
* Do more thorough research before you start
writing (1st structure, 2nd writing)
* Allow enough time
* Ask people to proofread
* Share tips and learnings with friends
50
Don’ts: bachelor Thesis
* Allow too little time, work full-time/complete an internship on the side
* Panic yourself by considering the bachelor thesis as a non-insurmountable hurdle
* Dragging out the bachelor thesis and postponing the deadline
* Start writing right away without
research/structure/plan
