Stats Flashcards
What is validity?
Validity refers to the extent to which something measures what it claims to measure.
What is reliability?
Reliability is the extent to which an experiment, test, or any measuring procedure yields the same result on repeated trials.
What is internal validity?
Examples?
Internal validity is the confidence that we can place in the cause and effect relationship in a study. It is the confidence that we have that the change in the independent variable caused the observed change in the dependent variable
Reliability of measurement instruments
Regression towards the mean (subjects selected based on extreme scores will tend to regress spontaneously towards the mean on subsequent tests)
Sampling
Experimental mortality (loss of participants over time may result in unequal characteristics in two groups)
Instrument obtrusiveness (the instrument should not affect the data collection e.g. poorly designed questionnaires)
Manipulation effectiveness (the independent variable must be manipulated enough so that the effect can be seen, ideally the degree of manipulation should be measured)
History (where two measurements of the dependent variable occur that are separated in time, there is the potential for various other influences to get introduced)
Maturation (people mature over time and this may in itself explain the change of a dependent variable)
Measurement sensitisation (the instrument may affect the way the subject see’s the world and so may bias future measures)
Measurement instrument learning (people may get used to the measurement instrument, a good example is the increasing performance on repeated use of the WAIS for estimation of IQ)
What is external validity?
Examples?
External validity is the degree to which the conclusions in a study would hold for other persons in other places and at other times, i.e. its ability to generalise.
Threats to external validity:
Representativeness of the sample
Reactive effects of setting (is the research setting artificial)
Effect of testing (if a pre-test was used in the study that will not be used in the real world this may affect outcomes)
Multiple treatment inference (this refers to study’s in which subject receive more than one treatment, the effects of multiple treatments may interact)
What is criterion validity?
Two examples?
Criterion validity concerns the comparison of tests. You may wish to compare a new test to see if it works as well as an old, accepted method. The correlation coefficient is used to test such comparisons
In concurrent validation, the predictor and criterion data are collected at or about the same time. An example could be testing a new, shorter test of intellectual functioning against a standard measure
In Predictive validation, the predictor scores are collected first and criterion data are collected at some later/future point. Here you want to know if the test predicts future outcomes. An example might be evaluating a new assessment method to select medical students. The test could be compared against the students performance at the end of year one to see if there is a correlation
What is construct validity?
Two examples?
The extent to which a test measures the construct it aims to
A test has convergent validity if it has a high correlation with another test that measures the same construct
A test’s divergent validity is demonstrated through a low correlation with a test that measures a different construct
What is the study power?
The power of a study is the probability of (correctly) rejecting the null hypothesis when it is false (i.e. it will not make a Type II error).
What is bias?
What is a confounder?
How do we address confounders in design stage?
Bias is a systematic error that can lead to conclusions that are incorrect.
A confounding factor is a variable that is associated with both the outcome and the exposure but has no causative role. A well known example is carrying matches and lung cancer. People who have lung cancer are more likely to smoke and therefore more likely to carry matches, but carrying matches does not cause lung cancer.
Confounding can be addressed in the design and analysis stage of a study. The main method of controlling confounding in the analysis phase is stratification analysis.
The main methods used in the design stage are listed below.
Matching (e.g. By age and gender)
Randomization
Restriction of participants (e.g. If watching TV is a known confounder then restrict participants to ones who don’t watch TV)
What are the two types of bias?
Selection bias - when selected sample is not a representative sample of reference population)
Information bias -(when gathered information about exposure, outcome or both is not correct and there was an error in measurement )
Give examples of selection bias
Loss to follow up bias: takes place in cohort studies when follow up cases are lost continuously. Lost cases may have something in common resulting in an unrepresentative sample.
Disease spectrum bias (aka case-mix bias): this can occur when a treatment is studied in more sever forms of a disease. Such results may then not apply to mild forms of the disease.
Self-selection bias (Volunteer bias or Referral bias): Those who volunteer may have shared characteristics resulting in a unrepresentative sample.
Participation bias (Non-response bias): Those who participate may have shared characteristics resulting in a unrepresentative sample.
Incidence-Prevalence bias (Survival bias, Neyman bias): Occurs in case-control studies and is attributed to selective survival among the prevalent cases (i.e. mild, clinically resolved, or fatal cases being excluded from the case group).
Exclusion bias: Occurs when certain patients are excluded for example if they are considered ineligible.
Publication or Dissemination bias: Many studies may not be published. This may be due to the fact that papers with positive results, and large sample sizes are more likely to get published.
Citation bias: Articles of high citation are easy to reach and have higher chance to be entered into a given study.
Berkson’s bias (aka admission rate bias): This is a type of bias resulting from case-control studies whereby cases and controls are selected from hospital settings. In such settings, cases can be unrepresentative of the general population and this can lead to confounding factors.
Give examples of information bias
Detection bias: This can occur when exposure can influence diagnosis. For example women taking an oral contraceptive will have more frequent cervical smears than women who are not on the pill and so are more likely to have cervical cancer diagnosed (if they actually have it). Thus, in a case-control study that compared women with cervical cancer and a control group, at least part of any higher pill consumption rates amongst the former group may be due to this effect.
Recall bias: In retrospective studies that participants should remember and determine their past exposure, it is likely to have cases and controls that do not act similarly in this regard. To better put, because of more reflection on reasons of disease it is likely to have cases that do recall and cite better the detailed conditions of their exposure than controls.
Lead Time bias: Lead time is the period between early detection of disease and the time of its usual clinical presentation. When evaluating the effectiveness of the early detection and treatment of a condition, the lead time must be subtracted from the overall survival time of screened patients to avoid lead time bias. Otherwise early detection merely increases the duration of the patients’ awareness of their disease without reducing their mortality or morbidity. Numerous cancer screening procedures were thought to improve survival until lead time bias was addressed.
Interviewer/Observer bias: Interviewer or observer knowledge about in-question hypothesis and disease or/and exposure can take effect on collection and registry of data.
Verification and work-up bias: This is a type of bias in which the results of a diagnostic test affect whether the gold standard procedure is used to verify the test result. It is more likely to occur when a preliminary diagnostic test is negative because many gold standard tests can be invasive, expensive, and carry a higher risk.
Hawthorn effect: This can occur when participants alter their usual behaviour due to their awareness that they are being studied.
Ecological fallacy: This can occur when conclusions about individuals are based only on analyses of group data.
Where is the mode, mean, and mean, with:
- normal distribution
- positively skewed data
- negatively skewed data
For normally distributed data - mode=median=mean For positively (right) skewed data - mean>median>mode For negatively (left) skewed data - mode>median>mean
What type of graph is a Graphical representation using Cartesian coordinates to display values for two variables for a set of data?
Scatter graph
What kind of outcome would you use in:
- Cohort
- Case Control
RR Used in cohort, cross-sectional and randomised control trials (not case-control)
OR Most commonly used in case-control studies, however they can also be used in cross-sectional and cohort study designs as well (with some modifications and/or assumptions).
How do you interpret likelihood ratios?
Likelihood ratio for a positive test result (LR+) = probability of a patient with a disease having a positive test divided by the probability of a patient without the disease having a positive test result
Likelihood ratio for a negative test result (LR-) = probability of an individual with disease having a negative test divided by the probability of an individual without disease having a negative test.
Generally speaking a LR+ of 10 or more is considered to significantly increase the probability of a disease (rule in disease).
Generally speaking, for patients who have a negative test, a LR- of more than 10 significantly increase the probability of disease (rule in disease) whilst a very low LR- (below 0.1) virtually rules out the chance that a person has the disease.
