Home > LIBRARY > Evidence Direct > EBM Toolbox > Critical appraisal

Evidence Direct: Critical appraisal

Critical appraisal looks at the way research has been conducted and asks are the results valid and reliable and how useful the results are. Below are some questions to ask once an item has been retrieved. You might also wish to see the Critical Appraisal approach used by the (Oxford) Centre for Evidence Medicine: http://www.cebm.net/cat_about.asp

 

Some initial questions to ask

  1. Have you searched the most appropriate databases/search engines, etc?

  2. Is the material from a known, reputable source?

  3. Has the material been evaluated in any way? If so, how and by whom?

  4. How up-to-date is the material?

  5. How does it compare with other quality material judged by the same criteria?

 

And that's only the start. You can also ask...

  • Did the trial address a clearly focused research question? Tip: a research question should be ‘focused’ in terms of: a) The population studied; b) the intervention given; c) the outcomes considered.

  • Did the authors use the right type of study? Tip: a) address the research question; b) have an appropriate study design.

  • Was the assignment of patients to treatments randomised? Tip: consider if this was done appropriately.

  • Were all of the patients who entered the trial properly accounted for at its conclusion? Tip: look for: a) the completion of follow-up; b) whether patients were analysed in the groups to which they were randomised.

  • Were patients, health workers and study personnel ‘blind’ to treatment? Tip: this is not always possible, but consider if it was possible – was every effort made to ensure ‘blinding’?

  • Were the groups similar at the start of the study? Tip: think about other factors that might effect the outcome such as age, sex, social class.

  • Aside from the experimental intervention, were the groups treated equally? Tip: for example, were they reviewed at the same time intervals.

  • How large was the treatment effect?

  • How precise was the estimate of the treatment effect? Tip: look for the confidence limits.

  • Can the results be applied to the local population? Tip: consider whether the patients covered by the trial are likely to be very different from your population.

  • Were all clinically important outcomes considered?

  • Are the benefits worth the harms and costs?

Note: the second part of the above adapted from Critical Appraisal Skills Programme (Bandolier).

 

What is the quality and validity of the evidence I've found?

There are now various grading systems for evidence, and much has been written about the numerous biases that can confound each type of scientific study in healthcare. For Levels of Evidence, go to Finding the Evidence/Levels of evidence on the menu.

The following - by grades - is a fairly simple way of evaluating evidence.

Grade I (strong) Evidence from well conducted randomised controlled trial, or review of results of well conducted randomised controlled trials.

Grade II (fairly strong) Evidence from well conducted prospective study, with a comparison group (i.e. a controlled trial but without randomisation, or a good observation study)

Grade III (weak) Evidence from retrospective study ie. looking back at past records.

Grade IV (weak) Evidence from cross-sectional study.

2. Critical Appraisal Tool

Try out CatMaker. This explains all about critical appraisal and even creates a critical appraisal for you!

 

3.Examples of Critically Appraised papers

a) By RMH clinicians (only available onsite at RMH):

For more RMH CATs click here (only available onsite at RMH).

b) A CAT

(click here) courtesy of the Paediatric Evidence Centre, Monash Institute of Health Services Research (author Tari Turner).

4. More Critical Appraisal resources

a) From Glasgow University - checklists for articles on:

b) EBM resources from University of Nottingham (includes audio):

c) NHMRC:

 d) "How to read a paper" from the BMJ archives (you will need an RMH Library membership number):

e) Other:

5. Critical Appraisal reading/reference materials

 

Search tip: choose the
service area you require,
then click on Go



Privacy Policy | Content Management System by Powerfront
Powered by Powerfront