Research:

NHMRC levels of evidence:

I	Systematic review
II	Proper RCT
III-1	Pseudo RCT
III-2	Comparative study with concurrent control -Cohort study -Case-control study
III-3	Comparative study without concurrent control
IV	Case series
V	Case report

A	Excellent	· Many good trials · Guides practice
B	Good	· Couple of good trials · Guides practice in most situations
C	Satisfactory	· Couple of ok trials · Careful with application
D	Poor	· Low quality trials · Beware application

Type	Explanation	Mitigation
Hawthorne effect	Being studied = feel good	Control group
Sampling bias	Study participants are not reflective of the population	Appropriate inclusion and exclusion criteria
Selection	Intervention and control groups are not the same	Randomisation
Performance	Unequal care between groups	Identical care except for intervention vs control
Attrition	Unequal withdrawal between groups	Impossible to overcome
Reporting	Some findings reported, others not
Response	Questions encourage a certain type of answer	Control group
Recall	Patient reports symptoms differently depending upon allocation	Patient blinding
Detection	Intervention and control group assessed differently	Assessor blinding Pre-defined criteria
Observer	Data collector able to be subjective about the outcome	Assessor blinding Hard outcomes
Publication	Negative results don’t get published	Clinical trial registries

Definitions	· Narrative review: pick and choose according to your world view · Systematic review: combined systematic analysis of multiple studies · Meta-analysis: combined quantitative analysis of multiple studies
Stages	· Define inclusion and exclusion criteria · Search for trials · Assess heterogeneity · Perform meta-analysis · Assess whether result makes sense
Forest plot	= what’s the outcome? · X axis: odds ratio · Y axis: individual studies · Dot size: weight · Line width: confidence intervals · Diamond centre: point estimate · Diamond width: confidence interval
Funnel plot	= is there publication bias? · Large studies should cluster around the true effect · Publication bias is suspected if results cluster on one side of the plot
Pros	· Statistical power · Generalisability
Cons	· Whether to combine apples and oranges · How to combine apples and oranges
Interpretation	· Negative result -> probably accept · Positive result -> probably do a large RCT to confirm

Indications

· Long latency

· Rare outcome

· Unethical intervention

· Unethical to wait

Process

e.g. case-case control study

· Generate hypothesis: exposure -> outcome

· Find patients with exposure

· Find patients with no exposure

· Match them in as many ways as possible

· Calculate odds ratio

ANZCA version:

Introduction	· Appropriate question · Appropriate study design · Appropriate endpoint
Research methods	· Measurement techniques · Who, where, when, how many, how long for · Control group, randomization, blinding · Inclusion and exclusion criteria
Result	· Answers the question? · Statistically significant? · Presented clearly? · Missing data and why?
Discussion & conclusion	· Logical? · Biased?
Summary	· Quality? · Relevance? · Changes my practice?

Simple version:

Intro	Why did they bother? · Original? · Incremental? · Significant?
Method	Did they do it properly? · Right subjects? · Right intervention? · Right outcome of interest?
Result	What did they find? · What is the result? · Is it statistically significant? · Is it clinically significant? · If positive, why? (positive IRL, or confounder?) · If negative, why? (negative IRL, or underpowered?) · If fancy statistics, why? · If missing patients, why?
Implication	What does this mean? · Does this change what I think? (balance of evidence) · Does this change what I should do? (patient population) · Does this change what I could do? (feasibility)