Flashcards in Ch. 12: Research Design & Statistics: A Foundation for Clinical Science Deck (95):
A philosophy of events and nature that values evidence more than opinions.
What scientists do as they practice science. The process of asking and answering questions. Methodological process.
The philosophical position that statements must be supported by experimental or observational evidence.
Means that events do not happen randomly or haphazardly; they are caused by other events.
An experiment-first-and-explain-later approach to research. The researcher starts by conducting a series of experiments, and then they propose a theory based on the results of those experiments.
An explain-first-and-verify-later approach to research. The investigator explains an event and then attempts to verify the explanation through the result of experiments.
A systematic body of information concerning a phenomenon, describing an event, explaining why the event occurs, and specifying how the theory can be verified. “X causes Y.”
A prediction stemming from a theory.
Hypothesis stating that two variables are not related.
Hypothesis stating that the two variables are related and that perhaps one is the cause of the other.
The result of systematic observation and, in some cases, experimentation.
Verbal descriptions of attributes of events.
Numerical descriptions of attributes of events.
The degree to which an instrument measures what it purports to measure.
The accuracy with which a test predicts future performance on a task.
The degree to which a new test correlates with an established test of known validity. A form of criterion-related validity.
The degree to which test scores are consistent with theoretical constructs or concepts. For example, a test of language development in children should meet the theoretical expectation that as children grow older, their language skills improve.
A measure of test validity based on a systematic examination of all test items to determine if they adequately sample the full range of the skill being tested and if they are relevant to measuring what the test purports to measure. Based on expert judgment.
Refers to the consistency with which the same event is measured repeatedly. Scores are reliable if they are consistent across repeated testing or measurement.
A number or index that indicates the relationship between two or more independent measures. Usually expressed through r. Expression of reliability. An r value of 0.00 indicates that there is no relationship between two measures. The highest possible positive value of r is 1.00. The lowest possible negative value of r is -1.00. The closer to 1.00, the greater the reliability.
Refers to consistency of measures when the same test is administered to the same people twice. When the two sets of scores are positively correlated, the stability of the scores over time is assumed.
Alternate-Form Reliability/Parallel Form Reliability
Based on the consistency of measures when two parallel forms of the same tests are administered to the same people. If both those forms are administered to a child and the scores are very similar, then the test has this form of reliability.
A measure of internal consistency of a test. Determined by showing that the responses to items on the first half of a test are correlated with responses given on the second half. Generally overestimates reliability because it does not measure stability of scores over time.
Refers to the extent to which two or more observers agree in measuring an event.
Refers to the extent to which the same observer repeatedly measures the same event consistently.
A means of establishing cause-effect relationships. Test if-then relationships. Involves a controlled condition in which an independent variable is manipulated to produce changes in a dependent variable.
Variable directly manipulated by the experimenter. Treatment is an independent variable.
Variable that is affected by the manipulation of the independent variable. All disorders are dependent variables. Must be defined very specifically so that they are measurable.
Variables other than independent variables that cause an effect on the dependent variable.
Group containing participants who do not receive treatment.
Pretest-Posttest Control Group Design
Experimental research design where there are two groups: an experimental group and a control group. This design helps evaluate the effects of a single treatment. The participants are randomly selected and assigned to groups. Each participant in each group undergoes a pretest and a posttest. Based on the logic that to assess the effects of an independent variable (treatment), the only difference between groups must be that variable.
Participants’ existing behaviors or skills measured before starting an experimental treatment or teaching program.
Measures of behaviors established after completing the treatment program.
Questions: “Which treatment is more effective?”
Research study design that helps establish cause-effect relations based on individual performances under different conditions of an experiment. Allow extended and intensive study of individual participants and do not involve comparisons based on group performances. Measure dependent variables continuously.
The basic single-subject experimental design. Useful in establishing treatment efficacy. The letters designate the different conditions of an experiment. The first A condition refers to baselines. The next B condition refers to treatment. The third A condition refers to treatment withdrawal.
An extension of the basic ABA single-subject experimental design. Useful in establishing treatment efficacy. The letters designate the different conditions of an experiment. The first A condition refers to baselines. The next B condition refers to treatment. The third A condition refers to treatment withdrawal. The fourth B condition refers to the reinstatement of treatment.
Research study design involving several participants who are taught one or more behaviors sequentially (in a staggered fashion) to show that only the behaviors of treated participants change; those of untreated participants do not change. This outcome, too, demonstrates that the treatment was effective. The researcher:
- Selects a target behavior to be taught to three or more participants
- Base-rates the target behaviors in all participants before treatment is applied
- Treats one participant while repeating the base rates on the untreated participants
- Treats the second participant while repeating the base rates on the untreated participants
- Alternates treatment and base rates until all participants are trained
Research study design involving a behavior being sequentially taught in different settings to demonstrate that the behavior changed only in a treated setting, and thus treatment was effective. The researcher:
- Base-rates a target behavior in three or more settings (e.g., hospital room, park, lobby)
- Teaches the behavior in one setting
- Repeats the base rates in the remaining untreated settings
- Teaches the behavior in another setting
- Continues to alternate base rates and teaching in different settings until the behavior is trained in all settings
Research study design involving several behaviors that are sequentially taught to show that only treated behaviors change, untreated behaviors show no changes, and thus the treatment was effective. The researcher:
- Selects three or more target behaviors
- Establishes base rates on those targets
- Trains the first behavior to a training criterion (e.g., 80% over three sessions)
- Repeats the base rates on the remaining untreated behaviors
- Trains the second behavior while repeating base rates on the remaining untreated behaviors
- Continue to alternate base rates and treatment until all the behaviors are trained
In this type of research, the researcher observes phenomena of interest and records his or her observations. The researcher does not want his or her presence to interfere with the natural phenomena that are being observed. Cannot lead to determination of cause-effect relationships. Involves a classification variable and a criterion variable. Can be more ethical in some situations (e.g., mothers who drink during pregnancy).
Form of descriptive research variable that is analogous to the independent variable in experimental research.
Form of descriptive research variable that is analogous to the dependent variable in experimental research.
Ex Post Facto Research
After-the-fact research. The investigator begins with the effect of independent variables that have occurred in the past. The investigator is making a retrospective search for causes of events. Therefore, this type of research is also called retrospective research or causal-comparative research. Researchers begin by defining the effect as it currently exists. They then look backward in an attempt to explain the cause. Can only suggest potential causes, as there is no experimentation to rule out extraneous variables.
Assess some characteristics of a group of people or a particular society. Attempt to discover how variables such as attitudes, opinions, or certain social practices are distributed in a population. The purpose is to generate a detailed inspection of the prevalence of phenomena in an environment by asking people as opposed to direct observation. Should be of a random sample. Questionnaires and interviews are the most common types.
The purpose of this kind of research is to measure the similarities and differences of groups of people with defined characteristics. Also known as standard group comparisons. E.g., could be used to compare people with/without head injuries on tests of attention and memory skills. A limitation is that similarities and differences found between groups of subjects might be due to variables other than the classification variable.
The purpose of this kind of research is to measure changes in subjects over time as they mature or get older. The presumed independent variable is maturation. Researchers choose this type of research when they tend to believe that age is the cause of changes seen in people, especially developing children. Often used to create developmental norms. Also known as normative research.
In this form of research, the same participants are studied over time. The investigator follows participants and observes the changes that occur within them as they get older. An advantage is that the investigator can directly observe changes in the behavior(s) of the same participants as they get older. Often have a small number of subjects, limiting the generizability.
Form of researcher where the investigator selects participants from various age levels and observe the behaviors or characteristics of the groups formed on the basis of age. Children of different age levels are observed simultaneously. Cheaper, faster, and more practical than longitudinal research.
In this procedure, the total age span to be studied is divided into several overlapping age spans. The subjects selected are those who are at the lower end of each age span, and they are followed until they reach the upper end of their age span.
In this form of research, the researcher investigates relationships or associations between variables. Never leads to cause-effect relationships being established. Correlation does not imply causation.
A statistical method of data analysis suggesting that to or more events are somehow associated or related. It suggests the direction (positive or negative) and the strength (high or low) of the relationship.
Form of research that involves observation and description of naturally occurring phenomena; thus, it is included under the aegis of descriptive research. It is not an experimental type of research. Most common method is for researchers to immerse themselves in the situation being studied. The researchers conduct detailed observations, taking notes, and recording video and audio. Qualitative form of research. Advantageous for detailed studies of clients. Time consuming, often expensive, and yields data that is difficult to quantify.
The degree to which data in a study reflect a true-cause effect relationship. Strongest when no confounding variable is present.
Threats to Internal Validity
Include instrumentation, history, statistical regression, maturation, attrition, testing, subject selection biases, and interaction of factors.
Refers to problems with such measuring devices as mechanical and electrical instruments, pen-and-paper instruments (e.g., questionnaires and tests), and human observers. May be reduced by judges. Criteria used by individual judges may become more or less stringent over the course of a study.
Includes the subjects’ life events that may be partially or totally responsible for changes recorded in the dependent variable after the independent variable is introduced. Events that occur, in addition to the experimental variable, between the first and subsequent measurements.
Refers to a behavior that foes from an extreme high or low point to an average level. E.g., many clients seek treatment when their problem is at its worst.
Refers to biological and other kinds of changes within participants themselves; such changes can have an effect on the dependent variable. The experimenter is not able to control those changes.
Refers to the problem of losing participants as the experiment progresses. Has an effect on the final results the investigator interprets. Also called mortality.
Refers to a change that occurs in a dependent variable simply because it has been measured more than once. The dependent variable is affected because of repeated measurement (e.g., the administration of pre- and posttests). In such cases, the investigator may incorrectly conclude that the treatment variable was responsible for the change recorded. Some behaviors change when repeatedly measured or tested.
Measures of behavior that change as a function of repeated testing.
Subject Selection Biases
Subjective factors that influence the selection of who participates in a study. It is best to use randomly selected and assigned groups.
Refers to generizability: to what settings, populations, treatment variables, and measurement variables the effect can be generalized. A matter of the extent to which the investigator can extend or generalize the study’s results to other subjects and situations. Threats to this limit generizability. Threats include the Hawthorne effect, multiple-treatment interference, and reactive or interactive effects of pretesting.
The extent to which a study’s results are affected by participants’ knowledge that they are taking part in an experiment or that they are being treated differently than usual.
Refers to the positive or negative effect of one treatment over another. This is likely when to or more experimental treatments are administered to the same participants.
Effect that occurs when treatment order influences treatment outcome.
Class I Evidence
Evidence based on a randomized group experimental design study, often referred to as a randomized clinical trial. This is the best evidence supporting a procedure. Evidence must come from at least one larger clinical trial with experimental and control groups.
Class II Evidence
Evidence based on well-designed studies that compare the performance of groups that are not randomly selected or assigned to different groups. Because of lack of randomization, the groups may or may not be equal to begin with. Therefore, there is no assurance that differences noted on posttests are due to treatment.
Class III Evidence
Evidence based on expert opinion and case studies. Case studies can claim improvement, but not effectiveness. Case studies do not include control groups. This is the weakest of the levels of evidence.
Level 1: Expert advocacy
Level used when there is no evidence supporting a treatment. The procedure is advocated by an expert. Such procedure is good only for research, not for routine clinical practice.
Level 2: Uncontrolled unreplicated evidence
Level used when a case study with no control group or such single-subject control conditions as baseline, withdrawal, reinstatement, or multiple baselines has shown the procedure to produce improvement. There is no assurance of effectiveness. The procedure is only acceptable only if one with better evidence is not available.
Level 3: Uncontrolled directly replicated evidence
Level used when a case study without controls has been replicated by the same investigator in another setting with different clients and has obtained the same or similar levels of improvement. While there is better evidence than in the previous two levels, still there is no assurance of effectiveness (we do not know if it is better than no treatment).
Level 4: Uncontrolled systematically replicated evidence
Level used with a case study has been replicated by another investigator in another setting with different clients and has obtained the same or similar levels of improvement reported by the original investigators. Although still uncontrolled, the evidence is getting stronger and is likely to show effectiveness in a controlled study.
Level 5: Controlled unreplicated evidence
The first level at which efficacy is substantiated for a treatment procedure. One of the group or single-subject designs is used to show that treatment is better than no treatment and that extraneous variables (such as maturation or parents’ work at home) are not responsible for the positive changes observed. Not only improvement, but effectiveness is demonstrated for the procedure.
Level 6: Controlled directly replicated evidence
Level used when the same investigator who demonstrated the effectiveness for the first time has replicated the study with new clients and has obtained similar results to document effectiveness. The technique is now known to reliably produce the effects, at least in the same setting.
Level 7: Controlled systematically replicated evidence
This is the highest level of evidence. The effectiveness of a treatment technique has been replicated by other investigators, in different settings, with different clients, to show that the technique will produce effects under varied conditions. A technique that reaches this level may be recommended for general practice.
Refers to the field of study involved with the art and science of data analysis. Purpose is to take large quantities of data and reduce them to manageable form and to aid researchers in making inferences.
The chance of something occurring.
A measure calculated from a sample.
A population value.
Refers to the dispersion or spread in a set of data. Common measures of this include the range, interquartile range, and standard deviation.
The difference between the highest and lowest scores in a distribution or set of scores. Can be deceptive.
“Cuts off the lowest and highest 25% of scores in a distribution, so the middle 50% of scores are left.
The interquartile range divided by 2.
The extent to which scores deviate from the mean or average score. Reflects the variability of all measures or score in a distribution. The larger this measure is, the more variable the scores. The smaller this measure is, the less variable the scores.
A distribution or set of scores is an index indicating the average or typical score for that distribution. Includes mean, median, and mode.
Measure of central tendency used to generate an average. Divide the total by the number of scores.
Measure of central tendency that is the score in the exact middle of the distribution. Divides the distribution into two parts.
Normal Distribution/Bell-Shaped Curve
Occurs when distributions are relatively symmetrical and the three measures of central tendency are generally the same.
Levels of Measurement
Represented by nominal, ordinal, interval, and ratio scales.
In this type of scale, a category is present (e.g., hypernasality) or absent (normal nasality). Items or observations are classified into named groupings or discrete categories that do not have a numerical relationship to one another. E.g., clinical diagnostic labels.
Numerical scale that can be arranged according to rank orders or levels. The numbers on the scale must be assigned in such a way that the intervals between them are equal with regard to the attribute being scaled.
Scale that has the same properties as an interval scale, but numerical values must be related to an absolute zero point.