TEST #1 (notes) Flashcards Preview

RESEARCH METHODS TEXTBOOK > TEST #1 (notes) > Flashcards

Flashcards in TEST #1 (notes) Deck (97):

Goals of science



• The scientific method

a commitment to test knowledge
through observation and (if possible)


What is a theory?

“A set of explanatory principles used
to make sense of and integrate a
range of empirical findings.”
– Account for Multiple facts
– Generate predictions about novel situations
– Provide a guide for research, narrowing the
questions asked



Explains, describes a lot of information. explains a lot of things and allows to make predictions. We base our beliefs on evidence. Provides a guide for research narrowing the questions asked. Our theories will be about specific predictions.


Good theories are

–Precise predictions
– Falsifiable (falsification or refutation)
–Productive theory
–Parsimonious (Parsimony) theory
– Not simple re-descriptions of
phenomena theory


Falsifiable (falsification or refutation) theory

a good theory can be proven wrong. The more specific the theory is the better.If the theory is vague it will always be right which is bad.


Productive theory

Can make many predictions.


Parsimonious theory

idea that we prefer simple explanations over complicated. all theories are equal but we prefer more simple. however if complex explains something better we prefer complex.


Not simple re-descriptions of
phenomena theory

When phenomena is a re-description in other words.



Use theories to develop the hypothesis. Movement from general to more specific.



evaluate a theory based on research results. from specific to more general, broad. Induction is working truths, not absolute truths. Theories that we have now are being revised. – Not prove/disprove, but support/fail to
support. “Theory never becomes fact; instead,
theory serves to explain fact.”


History of psychological science in
relation to physical sciences

History of psychological science in
relation to physical sciences
• Privileging theory and hypothesistesting
in psychological science
• Psychological science focuses on
hypothesis-testing, for phenomena
that might not be prevalent or robust
Psychology don't value description.


Description is foundation of science

Psychology doesn't view observation and prediction as important. It is focused on hypotheses.


hypotheses in psychology

are very valued now. Psychological science focuses on
hypothesis-testing, for phenomena
that might not be prevalent or robust


• Natural sciences both:

– Describe phenomena
– Create and test theories to explain
• “Before we inquire into origins and
functional relations, it is necessary to know
the thing we are trying to explain.”


Examples of descriptive research:

– Tip-of-the-tongue effect (you know the word but cannot recall it, description)
–Amnesia (pure description)
– Milgram’s obedience studies
– Development

Once you have enough observation and description then you make theories.



never been studied empirically


What questions can science answer?

–Empirical Questions


–Empirical Questions

• Empirical method, “based on observing the world”
• Not based on “argument, opinion, gut feeling, or
appeals to common sense or logic.”
– Many good questions are not empirical
questions. Things that you can observe are different.
– Turning interesting questions into
empirical questions


Examples of non empirical questions

"Does God exist?", "What is the best pie?"


Examples of empirical question

"What is the best selling pie"


– Operationalizations

what is the procedure, process, we need to capture what we need to study.
Concepts -> Operations to be
performed (Procedures)
–Precision for replicability
– One Concept ->
Many operationalizations.
For ex. we want to study aggression. Then for the operationalization we say "for ex. instead of punching we use aggressiveness in essay writing."


Applied research

this research is relevant to a real world phenomenon. For example people texting and driving.


Basic research

something that is not obvious for a real world question. For example, cognitive processes. It has a value.


Laboratory research

takes place in the lab.you have more control. we want to control the situation..


Field research

takes place in a real world. no control of situation. Takes place in a real world.


–Advantages Laboratory versus Field

Field: takes place in real world.
Lab: you have more control.
No research is perfect, it depends of situation.


Disadvantages Laboratory versus Field

Field: no one wants to interactively prticipate with you
Lab: people must participate, but less natural.


– Relation? Relation to Basic & Applied of Laboratory versus Field?

Field is more likely to be applied. Lab is more likely to be basic however can also be applied.


Laboratory versus Field findings

– Findings are correlated between lab and field
(r = .71; Mitchell, 2012)
-But, 14% of effects changed sign from
lab to field (30 of 215). (effects change signs on the lab they found one thing on the field completely opposite)
–Substantial variability depending on subfield
of psychology
• Studies reporting small effects
• Studies on gender differences (more likely to occur)



Collects data in form of numbers. For ex verbal lables (super hungry - 1 , not hungry -7 still quantitative). Still works with numbers. fMRI is also quantitative. As well as other neuro images, reaction times.



words, photos, childhood movies. Non-numerical data. Might ask to rate photos to turn to quantitative.


Advantages of quantitative and qualitative

Qualitative: gives more detail, information,
Quantitative: more ojective


Disadvantages of quantitative and qualitative

Quantitative: less information and details.
Qualitative: less objective, more probability for bias when turn to quantitative, a lot of effort for info anaysis



– “discovering something while looking for
something else entirely.”
– Happy or beneficial events that develop
by chance.
• Replication and extension (working on details might help)
• Past research (own and others)



Quantitative. Taking numbers of people from research and re-analyzing the data.


Scientific Literature

• Generate study idea/question
• Design study (e.g., Measurement)
• Collect data (e.g., Sampling)
• Analyze data (e.g., Statistics)
• Publish paper
– Empirical studies, Reviews, Meta-analyses,


Publishing Science

• Write Paper
– Authorship order
(in psychology order of names by biggest contribution, last name on the paper in neuroscience is superviser)
• Submit Paper
(gets rejected ,might be from desk review or from reviewer)
• Resubmit Paper (repeat)
• Revise Paper / Resubmit Paper
• Publish Paper


Scientific Journals

• Scholarly journals
• Peer-reviewed
– Scholarly, peer-reviewed articles from
reputable journals
Not all scholary articles are peer reviewed but all peer reiewed articles are scholarly.
• Professional societies (APA, APS) they post this journals
• Journal “Impact Factor”
– An index of how often articles in this journal
are cited by other researchers (only select articles are being indexed or archived)


Finding Scientific Articles

• PsycInfo
– Index or database of articles
• What words to use? (search terms)
– Too broad (add/change words)
– Too narrow (remove/change words)
– Thesaurus
–Ask Librarian
• Narrowing your search
– Date
– Methodology
• Finding similar articles
– Important Keywords
– Cited by
– Cited article search (Web of Science)
–Articles cited often are influential
• Library Resources
– Guide for Psychology
– Using PsycInfo
–Searching for cited references
– Guide to APA style


Study Design

• Theory
• Hypothesis / Research Question
• Study Design



population of your interest. changes depending on interest. you can only study population if it's small enough.


How we get information?

we ask, measure, etc.



sample of population smaller, easier to study. Needs to be representative of population.


Probability sampling

probability of people in a sample to get into population.you study. Better external validity.


Random sampling

simplest form of probability sampling. The probability is known in random sampling. Equal chances of entering the sample. Sample must capture all aspects of population. Do they look like a big group as much as possible?


Naturalistic Observation

• Systematic observation for descriptive
purposes, to understand and predict
• Who to observe?
(normal people in real life. you need a coding schema, may have poor details, superficial measurement)
–Few (participant observation)
(detailed, rich data and participant observation)
If you want to learn about biker gang you need to join it.
• How to measure?
– Many/Few behaviors
(for example link between aggression and environment)
– Operationalization
(what do we code as aggression? how do we operationalize the behaviour)


Types of naturalistic measurement

Measuring without influence
–Participant reactivity
(when you see someone watching your behaviour changes)
–Hide in plain sight
–Use video
• Archival research
(Instagram, email, forum)
• Behavioral traces


Measurement =

true score + measurement error



the same answer every time you replicate the experiment. car without an engine can be reliable.
–Same/similar results on repeated
–Low reliability and measurement error
(if there is a lot of measurement error it means something else is contributing. you don't know how to interpret scores. different answer every time is not reliable)



the measure of the ideas we are interested in. car without an engine can't be valid.
–Measuring what’s intended


Sources of measurement error

– Problems with tool
(might be not accurate)
– Variation in what’s being measured due to
context, etc.


Split-half Questionnaires

One half gives similar scores than the other half. One half should predict what you measure on the second part of the item.


Test-Retest Questionnaires

You are expected to give similar responses for the second time to the one that you answered for the first time.


–Internal reliability (Cronbach’s alpha)

Need to make sure you measure the same thing, one construct, one idea.


• Construct validity

"I am measuring what I want to measure" is "the degree to which a test measures what it claims, or purports, to be measuring."
Construct Validity refers to the ability of a measurement tool (e.g., a survey, test, etc) to actually measure the psychological concept being studied. In other words, does it properly measure what it's supposed to measure? For example, if we want to know our height we would use a tape measure and not a bathroom scale because all height measurements are expressed in inches and not in pounds.


Content validity

–Construct coverage
I need to measure all parts that you want to measure
Content validity is an important research methodology term that refers to how well a test measures the behavior for which it is intended. For example, let's say your teacher gives you a psychology test on the psychological principles of sleep. The purpose of this test is to measure your knowledge or mastery of the psychological priniciples of sleep, right? If the test does indeed measure this, then it is said to have content validity -- it measures what it is supposed to measure.


Face validity

This is a very basic form of validity in which you determine if a measure appears (on the face of it) to measure what it is supposed to measure. In other words, does the measure "appear" to measure what it is supposed to measure? For example, if you were going to measure anxiety, does your measure appear to actually measure anxiety? If so, it has face validity. Obviously this is not a test you should use to determine if a measure should be used, but more of a first step in determining validity. Many say that it is not valid.


Criterion validity

We present the evidence to convince the reader that we are reliable. criterion or concrete validity is the extent to which a measure is related to an outcome.easures how well one measure predicts an outcome for another measure. A test has this type of validity if it is useful for predicting performance or behavior in another situation (past, present, or future).


Cluster sampling

Cluster sampling refers to a type of sampling method . With cluster sampling, the researcher divides the population into separate groups, called clusters. Then, a simple random sample of clusters is selected from the population. The researcher conducts his analysis on data from the sampled clusters.


Stratified sampling

Stratified sampling is a probability sampling method and a form of random sampling in which the population is divided into two or more groups (strata) according to one or more common attributes.
Stratified random sampling intends to guarantee that the sample represents specific sub-groups or strata. Accordingly, application of stratified sampling method involves dividing population into different subgroups (strata) and selecting subjects from each strata in a proportionate manner. The table below illustrates simplistic example where sample group of 10 respondents are selected by dividing population into male and female strata in order to achieve equal representation of both genders in the sample group.


Jingle Fallacy

The “jingle fallacy” refers to the use of a single term to describe a multiplicity of quite different things. In this case, the phrase “non-cognitive skills” lumps together a vast range of skills, traits, strengths, or attributes: essentially, as the term implies, anything that is not cognitively based.


Jangle Fallacy

occurs when people use different terms to describe the same thing. This can get in the way of cross-disciplinary collaboration and the adoption of common measurements. The problem is often compounded by the different vocabularies of various disciplines.


How to avoid these fallacies?

Don’t confuse measures for
constructs. Look at the actual items. Do they look the same? Are they different?


Bad data

people press random data


Inattentive responding

“For this question, please select the
option for disagree.” People dont pay attention.
–54 missing responses (31.2%)
–6 other responses (3.4%)
–113 correct responses (65.3%)


How to get representative samples?

–Percentage of population
(vs. Sample Size)
(if population of interest is large you get small percentage although the sample will look big)
–Compare demographics
–Compare other variables
Think about representativeness
–Beware of self-selection and bias
Only certain type of people will answer certain questions.
• All influence possible interpretations. All influence how confident you feel.


Is probability sampling necessary?

• Probability sampling “not necessary”
for most psychological research. Still tells us about the world. Any person can stand in for another person.


Are samples randomly drawn?

• Don’t know probability of population
members entering sample
– Samples not randomly drawn
Our class does not represent general population in Canada. Not everybody ends up in university.


Convenience sampling

is one of the main types of non-probability sampling methods. A convenience sample is made up of people who are easy to reach. Consider the following example. A pollster interviews shoppers at a local mall.


URPP participants

easy to get, convenient, always around. easy to get data from, understand instructions.


Purposive sampling

purposive sample is a non-representative subset of some larger population, and is constructed to serve a very specific need or purpose. A researcher may have a specific group in mind, such as high level business executives. It may not be possible to specify the population -- they would not all be known, and access will be difficult. The researcher will attempt to zero in on the target group, interviewing whomever is available. "If you want only left handers"


Quota sampling

Sample that exactly represents population. 85% right handed in sample 15 % left handed.
The defining characteristic of a quota sample is that the researcher deliberately sets the proportions of levels or strata within the sample. This is generally done to insure the inclusion of a particular segment of the population. The proportions may or may not differ dramatically from the actual proportion in the population. The researcher sets a quota, independent of population characteristics.
Two of each species
Example: A researcher is interested in the attitudes of members of different religions towards the death penalty. In Iowa a random sample might miss Muslims (because there are not many in that state). To be sure of their inclusion, a researcher could set a quota of 3% Muslim for the sample. However, the sample will no longer be representative of the actual proportions in the population. This may limit generalizing to the state population. But the quota will guarantee that the views of Muslims are represented in the survey.


Snowball sampling

(also known as chain-referral sampling) is a non-probability (non-random) sampling method used when characteristics to be possessed by samples are rare and difficult to find. For example, if you are studying the level of customer satisfaction among elite Nirvana Bali Golf Club in Bali, you will find it increasingly difficult to find primary data sources unless a member is willing to provide you with contacts of other members.


Advantages and disadvantages of convenience sampling

Advantages: convenient, fast, cheap, easy, easy to get larger samples.
Disadvantages: lack of representativeness.


Can we generalize to population?

Yes, but to a limited population


WEIRD people

–Western, Educated, Industrialized, Rich,
and Democratic
–University undergraduates are outliers
–Moral reasoning, self concepts
–Visual perception, spatial reasoning
University students have a different percepton and spatial reasoning. Visual system might be different in villager and city person



500,000 people mainly from the US
and India
• Internet users (vs. Non-Internet users)
–Younger, overeducated, underemployed,
less religious, more liberal
–Less extraverted, more socially anxious
–Not representative of general population
• mTURK (Paolacci & Chandler, 2014)
• Provide data that are reliable, valid,
and produce results similar to
traditional participants
• Samples more diverse than University


What are we measuring?

What are we measuring?
How we are measuring?
fMRI, survays, look activity, ask
• Psychological phenomena versus
physical phenomena
psychology observes unobservable.
• Technology won’t help us. No tool can help observations.


Reaction times

behavioural measure.In cognitive psychology, reaction time (RT) is used to measure the amount of time that it takes an individual to process information (Luce). It is the duration of the interval between presentation of a stimulus (e.g., a word on a computer monitor) and the participant's response to the stimulus. Can be counted


Behavioural trace

evidence of behaviour



Good for studying phenomenon for long term effect that cannot be manipulated. a self-report is any test, measure, or survey that relies on the individual's own report of their symptoms, behaviors, beliefs, or attitudes. Self-report data is gathered typically from paper-and-pencil or electronic format, or sometimes through an interview.


Self report advantages/disadvantages

Adv: cost, ease of administration, interpretation
Disadv: if questions are emberrasing you might not find out the truth





Measurement error (extraneous variables)

– Multiple measures
• Measuring more than once


An open-ended question

is designed to encourage a full, meaningful answer using the subject's own knowledge and/or feelings. Allows variety


When you start new research...

Ask open ended questions. It's good not to limit responders.


Closed-ended questions

limit the answers of the respondents to response options provided on the questionnaire. Some examples of close ended questions are: Dichotomous or two-point questions (e.g. Yes or No, Unsatisfied or Satisfied) Multiple choice questions (e.g. A, B, C or D)


Neutral mid point

Is there a neutral mid point& in 7 point scale 4 is right in the middle. However, if you have 6 not 7, you dont have a midpoint. you cannot be neutral, you are not given that choice. If you feel that a person cannot be neutral this is a right choice.


7-point versus 5-point

Depends of the poll


– Anchors, Labels

What it means when you chooe a number? 5 could be strong and 1 weak. If there is an average number after computing everage lables which is consistent with lables of everyone than class thinks the same way


Reverse-coded items

Questionnaires that use a Likert scale (eg. strongly disagree, disagree, neutral, agree, strongly agree)
for answering questions often contain some items which are to be reverse scored. For example, in a
self-esteem questionnaire we may have some positively worded questions (eg. I take a positive
attitude toward myself), but also some negatively worded questions (eg. At times, I think I am no
good at all).


linguistic ambiguity

answers yes for all questions


Double-barreled question

“To what do degree do you agree with the
following statement:
I love to eat cake and ice-cream.” might like ice cream but not cake? two questions in one


Leading questions

want you to respond in a certain way.
“In light of the fact that spinach is a totally
disgusting vegetable that is bitter and
weird, to what degree would you be willing
to eat a cup of cooked spinach right at this


Creating good items

–Simple, complete sentences
–Avoid slang, abbreviations, jargon,
double negatives


Are self-report responses

It relies on common ground of language it is not complicated to interprete. We can easily rely on langage. Self report can be closed and open ended