Reliability and Validity Flashcards
(14 cards)
What is needed for a diagnostic system to be valid
If a diagnostic system is to be valid, it must also have high reliability!
The two are inextricably linked within science. If scientists cannot agree on who has schizophrenia (i.e. low reliability) then questions of what it actually is (i.e. questions of validity) becomes essentially meaningless
What is reliability?
consistency over time between the individuals who are using the system to rate patients
- if two therapists disagree on a diagnosis then theres low reliability
What is validity?
diagnostic system assesess what it claims to assess
- patients who are diagnosed actually have that mental disorder
What is co-morbidity?
occurrence of two illnesses together which confuse diagnosis and treatment
What is symptom overlap?
when two or more conditions share symptoms questioning the validity of the classification
What is a strength of the validity/reliability of diagnosis and classification of schizophrenia?
predictive validity has improved
What are limitations of the validity/reliability of diagnosis and classification of schizophrenia?
- low reliability
- lack of validity
Evaluate improved predictive validity as a strength of diagnosis and classification
P: Predictive validity
of the diagnosis of schizophrenia has improved.
E: For example, Mason et al (1997) tested the ability of four different classification systems of diagnosis to predict the outcome of the disorder (over a 13 yr period) in 99 schizophrenic patients.
E: Findings showed more modern classification systems had high predictive validity (idea that if diagnosis leads to successful treatment, then diagnosis is valid), especially if only
symptoms that had lasted six months were considered.
L:This suggests predictive validity has improved over time, as classification systems have been updated. However, Birchwood and Jackson argue that as 20% of schizos recover and never have another episode, but 10% are so affected they commit suicide, there is too much variety in the outcomes of schizophrenia for
predictive validity to be supported.
Evaluate lack of validity as a limitation of diagnosis and classification
P: A limitation of the diagnosis of schizophrenia is a lack of validity.
E: Criterion validity is a way of assessing if differing assessment systems arrive at the same diagnosis. This has been found to be poor in the diagnosis of Schizophrenia
E: For example Cheniaux study also demonstrates that schizophrenia is more likely to be diagnosed when using the ICD rather than the DSM
L: This suggests that schizophrenia is either over-diagnosed in ICD or under-diagnosed in
DSM. Either way, this is poor validity – a weakness of diagnosis.
Evaluate low reliability as a limitation of diagnosis and classification
P: A limitation of the diagnosis of schizophrenia is that it has low reliability.
E: There is evidence to suggest that the inter-rater reliability (the extent to which different assessors agree on their
assessments, in the case of diagnosis) in the
diagnosis of schizo is poor.
For example Cheniaux asked two psychiatrists to independently diagnose 100
patients using both DSM and ICD criteria. Inter-rater reliability was poor, with one
psychiatrist diagnosing 26 with schizophrenia according to DSM and 44 according to
ICD, and the other diagnosing 13 according to DSM and 24 according to ICD.
E: This inconsistency between mental health professionals indicates poor reliability in the
diagnosis of schizo because different professionals are not arriving at the same diagnosis for each patient.
L: However although reliability can be criticised, the diagnostic system does provide
practitioners with a common language, which allows for communication of research ideas and findings which will lead to a better understanding of the disorder and the
development of effective treatments.
What are limitations of diagnosis and classification
- co morbidity
- symptom overlap
- cultural bias
Evaluate co-morbidity as a limitation of diagnosis and classification
P: A limitation in the diagnosis of schizo is co morbidity.
E: There is research evidence to suggest that many patients with
schizophrenia often have another illness. Co-morbidity is the phenomenon that
two or more conditions occur together
E: For example, in a review Buckley et al concluded that
around half the patients with a diagnosis of schizophrenia have a
diagnosis of depression (50%) or substance abuse (47%). PTSD also occurs in 29% of schizo cases and OCD in 23% cases
L: This is a problem because it suggests that diagnostic tools cannot
tell the difference between schizophrenia and depression. In terms of classification it may be that very severe depression looks like
schizophrenia and vice versa and may benefit from being seen as a
single condition. This further presents a problem in terms of the
reliability and validity of diagnosis.
Evaluate symptom overlap as a limitation of diagnosis and classification
P: Symptom
overlap. When there is overlap between the symptoms of schizophrenia and other conditions.
E: For example, both schizophrenia and bipolar disorder involve
positive symptoms like delusions and negative symptoms like
avolition.
E: This again calls into question the validity of both the
classification and diagnosis of schizo. Under ICD a patient might be diagnosed as a schizophrenic.
L: However, many of the same patients would receive a diagnosis of bipolar disorder according to DSM criteria. This is
unsurprising given the overlap of symptoms, it even suggests that schizophrenia and bipolar disorder may not be two different conditions but one.
Evaluate culture bias as a limitation of diagnosis and classification
P: Culture bias
E: The diagnosis has a bias with regards to a diagnostic system (such as the DSM) means that certain people or groups or people are more likely to be classed as having a particular disorder than other types of people.
A westernised definition
of abnormality is reflected in diagnostic tools such as the DSM and ICD and thus they are inherently culturally biased.
These classification systems were developed in the USA and Europe, and as such may overemphasise a western concept of ideal mental health, ignoring the role of cultural factors.
E: African Americans and English people of Afro-Caribbean origin are several times
more likely than white people to be diagnosed with schizophrenia. Given that rates in Africa and the West Indies are not particularly high, this is almost certainly not due to genetic vulnerability. Instead, diagnosis seems to be set with
issue of culture bias. There may be several factors at work here. One issue is that positive symptoms such as hearing voices may be more acceptable in African cultures because of cultural beliefs in communication with ancestors, and
this people are more ready to acknowledge such experiences. When reported to a psychiatrist from a different cultural
tradition these experiences are likely to be seen as bizarre and irrational.
L: In addition, Escobar has pointed out that white psychiatrists may tend to over-interpret
symptoms and distrust the honesty of black people during diagnosis.
