In order to have base changes in practice on evidence we need to have what 2 things?
- A good range of available evidence
- High quality of available evidence
How do we find the evidence we need to decide whether we change our practice or not?
- Conduct literature searching as you would conduct research > Research questions THEN methods THEN results > Analysis of results THEN conclusion
- Ask the right question > What are you looking for exactly?
Once we have found the question, where do we ask it?
- Scope the databases so that you can adopt a logical and systematic plan for your search
- Adopt a flexible approach
What are our key sources of information?
- Journal data bases are searchable electronic storage of journal articles, conference proceedings and papers, reports, government and legal publications, patents, books
- A search will give you the citations
What is the difference between a database and a search engine?
- The search engine allows you to search the databases, but it is NOT a database and is only as good as the instructions you provide it
- It doesn’t have the information you need but allows you to find it
Why is google scholar frowned upon?
- Information not usually peer reviewed for accuracy
- Listing order is determined according to financial input
Lack of appropriate evidence can be due to one of two things…
- A small evidence base
- Issues with the search strategy
Name 3 common search problems
- Poor question
- Spelling mistakes
- Wrong key words
What is critical appraisal?
A systematic assessment of the quality of research evidence to determine whether it can inform practice.
What are the 3 aspects of appraisal?
- Trustworthiness
- Value
- Relevance
What are 4 key questions - trustworthiness?
- Is there any potential bias?
- Is the method reliable?
- Is the study valid?
- Do we accept the results?
What are 4 key questions - value?
- Are the results clinically important?
- What is the strength of the effect?
- How confident can we be in the results? (statistical significance)
- Can we actually use the results?
What are 4 key questions - relevance?
- Can the results be applied to the question?
- Is the cohort the same?
- Are these results actually useful to us?
- Has the research been conducted on the same kinds of people and conditions as what we want to apply it to? (external validity - generalisability)
What is reliability and what does it involve?
- If you do something repeatably you achieve the same answer/outcome
- It is about consistency across time (test-retest reliability), across items (internal consistency), across researchers (inter-rater reliability)
What is validity and what does it involve?
- Relates to how accurate the answer/outcome is
- The extent to which the scores from a measure represent the variable they are intended to
- Different types: face, external, internal, ecological
Critical appraisal demands the following skill set (6)
- High level thinking skills
- Knowledge of the subject area
- Understanding of health research methods and statistics
- Knowledge of the wider context
- The ability to identify relationships between ideas and concepts
- Developing judgements about study trustworthiness, value and relevance
What are the 4 appraisal tools?
- Jaded scale (Oxford quality scoring system)
- McMaster critical review guidelines
- CASP tools
- SIGN checklists
What does the the Jaded scale (Oxford quality scoring system) entail?
- A simplistic method of evaluating clinical trials
- Designed to assess the methodological quality of clinical trials
- Simple scoring method based on key threats to validity in the design of a clinical trial (randomisation, blinding, attrition)
- Limitations: Over simplistic, too much emphasis on blinding, not applicable to to other research designs
What does the the McMaster critical review guidelines entail?
- Detailed guidelines provided for different research designs in healthcare studies. No overt scoring method, but rather a set of questions to facilitate overall judgement of research quality
- General set of questions/prompts to get you thinking about the research question
What does the the CASP tools entail?
- 3 sections: Are the results valid? Are the results clinically important? Will the results help locally?
- 8 checklists covering different types of evidence: Trials, Reviews, Cohort studies, Case control studies, Qualitative studied, Economic evaluations, Diagnostic test studies, Clinical prediction rule
What are SIGN checklists?
A series of checklists for different types of evidence:
Systematic reviews and meta-analysis, Randomised controlled trials, Cohort studies, Case-control studies, Diagnostic studies, Economic studies, Considered judgement pro-forma
What is the hierarchy of evidence?
A way of ranking how useful different pieces of evidence are particularly in relation to evidence-based practice
Suggest 4 appropriate search terms to locate evidence answering the following research question: What effect does an increased dose have on intellectual development in children undergoing radiotherapy?
- Dose
- Paediatric
- Radiotherapy
- Intellectual
Suggest some real health or medical databases?
- Medline
