Van Dongen & Sikorski (2021) - reading assignment Flashcards
(19 cards)
What are normative conceptions of objectivity?
normative: what scientists should do to be objective
value-free objectivity as an example of a normative notion, that follows the
value free ideal: researchers should not use values to justify claims.
What are descriptive conceptions of objectivity?
work of Douglas in which Douglas maps out different uses of the word “objectivity” and then analyzes
these different uses to show that they do not share a core. another example:
the work of Koskinen, who also analyses different uses of the term ‘objectivity’ (“when do we call something objective”) and tries to unify these different meanings into one account. In both
cases, it is descriptive because they analyze how the term objectivity is used to analyze its meaning
What Van Dongen and Sikorski consider the problem with existing normative conceptions of objectivity?
The ones they know of (such as the value-free conception) are not testable and practicable by
scientists. We cannot actually evaluate whether people have used their non-epistemic values. So
there is no way to check the objectivity or know as a scientist what to do to be objective.
What is via negative approach to objectivity?
The via-negative approach defines objectivity by the absence of certain factors (as opposed to a
positive definition which defines what factors should be present).
More specifically, in the case of objectivity, the factors that need to be absent are “factors that have
been empirically or methodologically identified as making scientific practice susceptible to the
actions and decisions of scientist which can inadvertently or intentionally influence research results.”
What is the example of Bem’s Feeling the Future experiments supposed to demonstrate?
it shows how researchers can misuse researcher degrees of freedom to get a
certain effect. In this example the suspicion is that Bem took the freedom to only continue the studies
“that showed promise”
What are two categories of mis-using researchers degrees of freedom?
Van Dongen & Sikorski distinguish between the degrees of freedom that researchers have before and
during research (design, data collection, measurements) and the degrees of freedom that researchers
have after their observations are in (data management, analysis specification and result reporting)
What are examples of degrees of freedom which researchers have before and during research?
researchers can choose different measurement instruments, they can choose how to draw a sample from the population they are interested in, whether they
want to do an experiment or a quasi-experiment or an observational study etc
What are examples of degrees of freedom which researchers have after data collection?
researchers can choose how they deal with missing data (e.g., delete the whole participant, or impute data), what kind of statistical assumptions to make, what
statistical test to do, what inference criteria to use (p-value or bayes factor) etc.
Objectivity is property of….
practice!
scientific practice should be resilient to actions that can influence its outcome
How Van Dogen and Sikorski objectivity can be empirically tested?
You can do a checklist to see whether things are done to make the practice more resilient against factors that can influence its outcome. For example, you can check whether the study
is pre-registered and see what decisions were already fixed (e.g., if that preregistration
includes a specific method for dealing with missing data and you see in the final report that that is indeed what they did, then you know the researcher did not have “degrees of freedom” anymore in how to deal with missing data)
What are 7 limitation points discussed by van Dongen and Sikorski?
- incompleteness
- ritualization
- restricted
- scientist-independent problems
- exploratory research
- too demanding
- objective research does not guarantee true results
incompleteness
not reaching complete definition of objectivity, but rather number of currently identified necessary conditions
ritualization
designing “ritual” to rull out questionable research practices
restricted
restriction to quantifiable or countable research
scientist independent problem
sometimes source of negative influence is independent of scientist
external restrictions regarding measurement instruments, samples, treatments etc
exploratory research
argument that most scientific breakthroughs were serendipitious discoveries
too demanding
very strict conceptualization, most scientific papers dont fullfill it
objective research do not guarantee true/trustworthy results
results can be untrue due to false positive or fault of measurement instrument
In the section ‘implications and applications’ they explain how their conception applies to practices in psychological research, such as the research practices that resulted in the paper of Bem 2011. What is the main conclusion?
The practices that resulted in Bem’s article are not objective: they did not preregister the study, and
they did not do a blind analysis or multiverse analysis. As such they allowed themselves many
degrees of freedom that they could have misused and as such the practice is not resilient against such
factors.
