Research methods Flashcards
(8 cards)
What Types of Measures Do Developmental Psychopathologists Use?
Psychopathology/psychological symptomsInterviews, rating scales, and observationPredictors, correlates, and consequences of psychopathology/psychological symptomsBehavioralRating scalesObservationsPhysiologicalHeart rate, skin conductanceNeuralEEG, FMRICognitiveTasks measuring memory and attention
Why Would We Even Want to Measure Psychopathology Anyway
ClinicalDiagnosisTreatment planningTreatment monitoring and progress ResearchWho develops psychopathology?What are the correlates of psychopathology?What happens to people who have psychopathology?*What treatments work for reducing symptoms of psychopathology?
Assessment
How do we do it?
*Interviews
*Rating scales
*Observations
Interviews
Three types:UnstructuredStructured*Semi-structured
Unstructured InterviewsClinician asks questions and arrives at diagnosisA lot of clinicians use this approach, and many rely on it entirelyLots of challenges w/ unstructured interviewsLess comprehensiveBiasesConfirmatory biasUse an availability heuristic where they base decisions on examples that come to mind easily*Combine information in idiosyncratic way
The garden of forking paths by Borges
Semi-Structured and Structured InterviewsInterviewer has a set of questions that are presented to the respondentSemi-structuredInterviewer has a lot of latitude in asking the questionsClinical judgment involved in determining when a symptom is presentStructuredQuestions are fixed and interviewer has very little flexibility*Can be administered by computer
Unstructured versus Structured and Semi-Structured InterviewsData suggest that structured and semi-structured are more reliable & valid than unstructuredStructured and semi-structured interviews can be used to measure psychopathology continuously by totaling up the number of symptoms reported
Disadvantages of Structured & Semi-structured InterviewsStructured and semi-structuredinterviews are the gold standard instruments in psychopathology researchNot as widely used in clinical practice (although this is changing)Why? FeasibilityLength*TrainingMy opinion –Diagnosis impacts treatment, services people get, and the course of their life. Better to get it right using a semi-structured interviewGet trained somewhere that values evidence-based practice
K-SADS (Kiddie Schedule of Affective Disorders & Schizophrenia)Good coverage across many sorts of disordersScreener tells you what to follow up onGives questions that correspond to DSM5 criteria, potential follow ups, and rating scaleCan ‘skip out’ if participants aren’t endorsing symptoms
Rating ScalesPeople knowledgeable about the child answer questions about behaviors and feelingsE.g., Children’s Depression Inventory(0) I am sad once in a while(1) I am sad many times(2) I am sad all the time(2) All bad things are my fault(1) Many bad things are my fault(0) Bad things are usually not my fault(0) I have fun in many things(1) I have fun in some things(2) Nothing is fun at all
Often used to measure psychopathology continuously (i.e., number of symptoms)Can be used to make a categorical decision Compared to structured/semi-structured interviews:Shorter (usually less than 20 minutes to complete)No interviewer
Assumption has been that they are less good than interviewsTrade-off between higher validity/reliability of interviews and feasibility of checklistsNot all data are consistent with this*Raises the possibility that clinicians could use shorter assessments
ObservationInterviews and rating scales rely on someone’s report of behaviorReporters might not know whether behavior is normative or clinically concerningObservation (can) provides access to the circumstances in which behavior occursTypes of ObservationsNaturalistic observationOccur in child’s natural environmentClassroom, homeStructured observationLaboratory-or clinic-based
Challenges Associated with Observational ApproachesFeasibilityExternal validity -extent to which findings will generalizePresence of an observer may change behaviorMay be difficult to see behaviors of interestLow-base rateE.g., physical aggression*Covert E.g., relational aggressionResearchers have developed very creative solutions to this
SummaryMore standardized approaches may yield better informationYet, clinicians often rely on unstructured approachesImportant to continue:Developing standardized approaches that are easy to use in clinical practiceDetermining when “more is better”Elevation on rating scale DOES NOT EQUAL diagnosis aloneClinicians use many different pieces of information to come to a determination
A ‘typical’ thoroughADHD assessmentIQ testing+Academic achievement testing (reading, writing, math)+ADHD rating scales from teachers, parents, and self-report (if old enough)+Semi-structured clinical interview (e.g., K-SADS) w/ parent and child (if old enough)=Determination of whether or notchild meets criteria for ADHD
Why do all this? Rule out learning disabilities and intellectual dev disorder as root cause of intattention+ hyperactivity in school
DIFFERENT INFORMANTS WHEN COLLECTING DATA ON CHILD PSYCHOPATHOLOGY
Use of Informants in AssessmentRating scales and interviews rely on someone’s report of symptomsParents, children, teacher
Disagreement among InformantsInformants often do not agreeCorrelations from .20 to .40Why?
1) Different PerspectivesRater-specific factors that lead to systematic differences in reportingSome evidence for bias in reports of some informantsLegitimate differences in the meaning of behaviors across settingsParents and teachers interact with children in very different contexts, and these contexts may change the interpretation of behaviorInformant discrepancies may tell us something real about children’s adaption in various settings
(2) Situation Specificity of Children’s BehaviorChildren’s behavior varies markedly across different situations and settingsDifferent demandsInter-rater differences may be capturing legitimate differences in children’s behavior across settings*Parents and teachers may be seeing different behaviors
Using Data from Multiple InformantsHow are data from multiple informants used to make a diagnosis?Combine ratings“Or” ruleSymptom is present if any informant says it is*“And” rule Symptom is present only if all informants agreeTwo different approaches
AndVersus OrRule for SymptomsDisorder with seven symptomsAny four symptoms meets criteria for a diagnosis
Takeaway: ‘And’ vs ‘Or’ ‘rule’Clinicians must use their judgement to determine whether to apply ‘and’ rule vs ‘or’ rule thinking to the child/adolescent walking in the doorThings to think about:Thespecificdisorderbeingscreened–somedisordersteachersandparentsmaybebetterorworsereporters (think external, observable behaviors vs. internal and cognitions and beliefs)Howreliable is each reporter? Do they have ample opportunity to observe target behaviour(s)?Howmuchinsightdoesthechildhaveintotheirownexperience?Etc.No hard and fast rules –judgement needed
Combining Informant ReportsBoth the and the or rules do not capture the fact that differences between informants are valuableDiagnosis identified by teacher may be different than one identified by a parentThink about maintaining ratings separatelyParent-reported, teacher-reported, combined (reported by both parents and teachers)
SummaryInformation about youth’s psychological symptoms can be obtained from many knowledgeable peopleYouth, parents, teachers, friends?Two informants will show limited agreementThose discrepancies are meaningful and may tell us something important about functioning in different contexts
Properties of Good Measures
What Types of Constructs do Developmental Psychopathologists Measure?Developmental psychopathologists measure many different constructsPsychopathology/psychological symptoms Interviews, rating scales, observatioPredictors, correlates, and consequences of psychopathology/psychological symptomsBehavioral Rating scalesObservations*Physiological Heart rate, skin conductance *Neural *EEG, FMR *Cognitive Tasks measuring memory and attention
Properties of Good MeasuresReliabilityConsistencyValidityAre we measuring what we think we are measuringReliability is a necessary condition for validityLess reliable = more error. At a certain point, your score is mostly made up of error, so it can’t be validly measuring what you want it to
ReliabilityInternal consistencyE.g., 10 items measuring ADHDIf those items are a reliable measure of the construct, people’s answers to those questions should be strongly positively relatedMy child is often on the goMy child cannot sit stillMy child is very fidgetyMy child can’t stay in her seat even when she is supposed to
Test-retest reliabilityDo we get similaranswer on different measurement occasions?E.g., my depression scores today should be relatively strongly positively related to my depression scores in 2 weeksBe careful! Some constructs should change over time, even over very short intervalsInter-rater reliabilityAgreement between two people judging whether something is present or occurringDiagnosisCan two clinicians agree that a child has ADHD?
other kinds: parallel-form reliability: the reliability coefficient obtained by two comparable sets of measures
split-half reliability: reflects the correlations between two halves of an instrument
Advanced Topic*: Measurement InvarianceGenerally –think of ‘fairness’ of a measurePeople in different groups with similar abilities should score similarly across items on a math testIf(just an example) boys and girls are = in inattention, we wouldn’t expect boys to score higher on a ‘I had trouble concentrating’ item on an ADHD screenerHowever, if boys and girls are not= in inattention, we wouldn’t have the same expectation
SummaryDevelopmental psychopathologists measure many different types of constructsCorrespondingly, they use many different types of measures*Good measures are reliable and valid
CORRELATIONAL STUDY DESIGN
Types of Correlational Research Designs in Developmental Psychopathology ResearchCross-SectionalRepeated measures designs:Longitudinal Design*Sequential Design
Cross-Sectional Design
Ex: in 2009, we compare 7 year old kids, 8 year old kids and 9 year old kids
BenefitsCompare cohorts of different ages to one another at a given timeRelativelycheap and practicalDrawbacksCan’t learn about how individual people change with ageAge effects are confounded with cohort effects
Longitudinal Design
Ex: we follow the same cohort in 2008 (age 7), 2009 (age 8) and 2010 (age 9)
BenefitsCan make within-subject comparisonsNo cohort effectsDrawbacksSubjects drop outMay be effects of repeated testingRequires foresight (and funding!)Time consuming*Age effects confounded with time of measurement effect
Sequential Design
(see image)
Benefit:Helps disentangle age effects from (1) cohort effects & (2) time of measurement effectsDrawback:Very time-consuming, complex, & expensive!
EXPERIMENTAL DESIGN
Evidence-Based Treatments –How 2 ClassifyWell-established treatments (from Chamblesset al. 1999):A large series (>= 9) of single-case study designs demonstrating efficacyORAt least 2 between group-design experimentsNote that those ^ criteria are more than 20 years oldThey have been critiqued*Shifting to systematic review of the literature followed by a committee reviewing the evidence (Tolinet al., 2015)
Single-Case Experimental DesignsExamine the effect of a treatment on a single child’s behaviorRepeated measures of behaviorReplication of treatment effectsA-B-A-B Reversal DesignsA –baseline of behaviorB –intervention phaseA –return to baseline (remove intervention)B –reintroduce intervention
AdvantagesInternal validityTemporal orderingA changes BDisadvantagesExternal validityCan be hard to interpret the findingsStable change*Ethics
Group-Based Designs –Randomized Control/Clinical Trial (RCT)A therapy experimentExperimental and control conditionsRandom assignmentInternal validityPowerful test of intervention efficacyPowerful test of theoryIf designed carefully, can let researcher establish causeResponsiveness: Changing A changes B
RCTsRandomly assigning participants to treatment and control groupsWhat about my intervention is causing improvement?Internal validity: Is it my intervention that is causing the change in outcomes?Construct validity: What about my intervention is causing the change in outcome?
RCTS: Type of Control group impacts conclusionsNo treatment control groupWait-list control groupPlacebo (in medication trial)Treatment as usual*Another, effective treatment
RCTS -DisadvantagesExternal validitySamplesDrop outNote that RCTs are looking at averagesEvenwithin the treatment group, some people will not have improvedAttrition Bias : the selective dropout of participants who systematically differ from from those who remain in the study.
*Vast majority of trials are focused on Efficacy *Few trials focused on Effectiveness and even fewer focused on Efficiency
Efficacy: does it work in clinical trials
Effectiveness: does it work in clinical practice
Efficiency: does it contribute to more efficient use of resources
Summary –Experimental DesignTheoretical expectation that A would cause BA and B have to be related empirically (correlation)Elimination of other possible causesCase-control design, prospective design, single-case designs, RCTTemporal ordering: A occurs before BProspective design, single-case designs, RCTResponsiveness: Changing A leads to change in BSingle-case designs, RC
When you read about research findings, always be thinking about these things:Internal validityAllows you to attribute differences to the variable of primary interestExtent to which an investigation rules out alternative explanations for the findingsE.g., Differences in outcome are due to depressionE.g., Differences in symptoms are due to my treatmentExternal validityDo the findings generalizeOther peopleOther measures*Other situations
PUTTING IT ALL TOGETHER: NOSOLOGY
Nosology -classification of diseaseIn developmental psychopathology, it is the organization of behavioral and emotional dysfunction into meaningful groupings
Dimensional versus Categorical Classification
CategoricalSomeone who has that disorder is fundamentally different than someone who does notDimensionalPresent in everyone to varying degree
Categorical ApproachMajor nosologicalframework in developmental psychopathology:Diagnostic and Statistical Manual of Mental Disorders (DSM) –5Outlines diagnoses and associated criteriaCategorical systemProfessional consensusMedical model Discrete disorders with separate causesAdvantages Synthesis of informationAids communicationDisadvantagesChildren (and adults) often do not fit into categories*Current categories proving inadequate for genetic and neuroscience research
Dimensional SystemsResearch Domain Criteria (RDoC)Rather than using diagnostic categories, move towards assessing key dimensionsNegative emotionalityTemper lossDimensional measurementAdvantagesAllows us to retain valuable informationProvides a measure of severityDisadvantagesWhich dimensions? Becomes very complicated very quickly*Is it too soon for RDoC?
SummaryPsychopathology can be conceptualized categorically or continuouslyDSM-5 is the major categorical frameworkContinuous frameworks are increasing in prominenceTwo approaches make fundamentally different assumptions about psychopatholog
