Critical Appraisal Tools

Critical appraisal is integral to the process of Evidence Based Practice. Critical appraisal aims to identify potential threats to the validity of the research findings from the literature and provide consumers of research evidence the opportunity to make informed decisions about the quality of research evidence.

Critical appraisal can occur through a non-structured approach where you critically read the study as you read it, or through a structured approach through the use of a Critical Appraisal Tool (CAT). CAT’s are structured checklists that allow you to check the methodological quality of a study against a set of criteria. An advantage of using a CAT is that you can apply a level of consistency when reviewing a number of studies. However a potential disadvantage is that they may not ask about a potential source of bias that is important for the specific research questions being asked.

Below is a list of CATs, linked to the websites where they were developed.  We have also included some information about developing your own CATs.

The following tutorials provide some information on how to critically appraise the literature

Please choose a type of study 

1.     Randomised Controlled Trial (RCT)
2.     Cohort
3.    Diagnostic
4.     Case control Studies
5.     Economic Studies
6.     Other Quantitative
7.     Qualitative     
8.     Mixed Methods
9.     Systematic Reviews

Interested in improving your clinical research skills or need a refresher? iCAHE will be running virtual Professional Development courses in 2018 for Evidence Based Practice and Research Methods, free of charge.

See more here. 


Randomised Controlled Trial (RCT)

CASP: Randomised Controlled Trial Appraisal Tool

https://casp-uk.net/casp-tools-checklists/

Summary: Critical Appraisal Skills Program (CASP): RCT CAT is a methodological checklist which provides key criteria relevant to randomised controlled trials.

Authors: Public Health Resource Unit, NHS, England

PDF: CASP RCT CAT

PEDro Scale

https://www.pedro.org.au/

Summary: PEDro (Physiotherapy Evidence Database) Scale is an excellent webpage which provides access to a range of appraisal resources including a tutorial and appraisal tool.

Authors: The Centre of Evidence-Based Physiotherapy (CEBP), Sydney, Australia

PDF: PEDro Scale

The JADAD Score

http://onlinelibrary.wiley.com/doi/10.1002/9780470988343.app1/pdf

Summary: The Jadad scale assesses the quality of published clinical trials based methods relevant to random assignment, double blinding, and the flow of patients. There are 7 items in the scale, scored with a yes scoring 1 and a no scoring zero. The last 2 questions attract a negative score, which means that the range of possible scores is 0 (bad) to 5 (good).

Authors: Jadad AR, Moore RA, Carroll D. 

PDF: JADAD Scale

GATE CAT: Intervention RCT/Cohort Studies

https://www.fmhs.auckland.ac.nz/assets/fmhs/soph/epi/epiq/docs/GATE%20CAT%20Intervention%20Studies%20May%202014%20V8.docx

Summary: This CAT developed by the University of Auckland presents a comprehensive study review process focused on the 5 steps of Evidence Based Practice. Steps you through the process of asking, accessing, appraising (using the RAMboMAN tool), applying and auditing.

Authors: The University of Auckland, New Zealand

PDF: GATE CAT – Intervention RCT Studies

Scottish Intercollegiate Guidelines Network (SIGN) Methodology Checklist: Randomised Controlled Trials

http://www.sign.ac.uk/checklists-and-notes.html

Summary: This CAT developed by the Scottish Intercollegiate Guidelines Network (SIGN), scores the RCT over 10 questions and provides an overall assessment of the studies effort to reduce bias. Whislt developed to be used for the development of clinical guidelines they are excellent CATs for single study appraisals

Authors: Scottish Intercollegiate Guidelines Network

PDF: SIGN checklist RCT

Centre for Evidence Based Medicine, Oxford University

 https://www.cebm.net/2014/06/critical-appraisal/

Summary: This CAT presented by the CEBM, scores the RCT over 5 questions. Many of the questions are present in the CASP CAT

Authors: Centre for Evidence Based Medicine, Oxford University 

PDF: CEBM - RCT

JBI checklist for Randomised Controlled Trials

http://joannabriggs.org/research/critical-appraisal-tools.html

Summary: JBI Critical appraisal tools have been developed by the JBI and collaborators and approved by the JBI Scientific Committee following extensive peer review. Although designed for use in systematic reviews, JBI critical appraisal tools can also be used when creating Critically Appraised Topics in journal clubs and as an educational tool.

Authors: Joanna Briggs Institute, Adelaide, Australia

PDF: JBI checklist for RCT

Specialist Unit for Review Evidence (SURE) 2016: RCTs and Other Experimental Studies

http://www.cardiff.ac.uk/specialist-unit-for-review-evidence/resources/critical-appraisal-checklists

Summary: This CAT presents questions to assist with the critical appraisal of randomised controlled trials and other experimental studies. It has been adapted and updated from the former Health Evidence Bulletins Wales (HEBW) checklist (http://www.cardiff.ac.uk/insrv/libraries/sure/doc/Project%20Methodology%205.pdf) with reference to the NICE Public Health Methods Manual (2012) and previous versions of the Critical Appraisal Skills Programme (CASP) checklists, with reference to the CONSORT statement.

Authors: Cardiff University

PDF: SURE 2016 CAT for RCTs

RoB 2.0. A revised tool to assess risk of bias in randomized trials

http://www.bristol.ac.uk/population-health-sciences/centres/cresyda/barr/riskofbias/rob2-0/  

Summary: This CAT from the Centre for Research Synthesis and Decision Analysis, presents tools supported by guidance notes for different RCT designs.

Authors: Higgins et al. 

PDF: Individually-randomized, parallel-group trials - CAT                      Guidance sheet

         Cluster-randomized, parallel-group trials - CAT                             Guidance Sheet

         Individually-randomized, cross-over trials - CAT                            Guidance Sheet

CAT for RCTs: Roever and Oliveira (2015)

Summary: This CAT is based on a combination of other CATs

Authors: Roever & Oliveira

PDF: CAT for RCTs


Cohort Studies

CASP: Cohort Studies

https://casp-uk.net/casp-tools-checklists/

Summary: Critical Appraisal Skills Program (CASP): Cohort Studies is a methodological checklist which provides key criteria relevant to cohort studies.

Authors: Public Health Resource Unit, NHS, England

PDF: CASP Cohort

GATE CAT – Intervention Cohort Studies

https://www.fmhs.auckland.ac.nz/assets/fmhs/soph/epi/epiq/docs/GATE%20CAT%20Intervention%20Studies%20May%202014%20V8.docx

Summary: This CAT developed by the University of Auckland presents a comprehensive study review process focused on the 5 steps of Evidence Based Practice. Steps you through the process of asking, accessing, appraising (using the RAMboMAN tool), applying and auditing.

Authors: The University of Auckland, New Zealand

PDF: GATE CAT – Intervention  Studies

Scottish Intercollegiate Guidelines Network (SIGN) Methodology Checklist 3: Cohort studies 

http://www.sign.ac.uk/checklists-and-notes.html 

Summary: This CAT developed by the Scottish Intercollegiate Guidelines Network (SIGN), scores the cohort study over 10 questions and provides an overall assessment of the studies effort to reduce bias. Whilst developed to be used for the development of clinical guidelines they are excellent CATs for single study appraisals

Authors: Scottish Intercollegiate Guidelines Network

PDF: SIGN Checklist 3: Cohort studies   

JBI checklist for Cohort Studies

http://joannabriggs.org/research/critical-appraisal-tools.html   

Summary: JBI Critical appraisal tools have been developed by the JBI and collaborators and approved by the JBI Scientific Committee following extensive peer review. Although designed for use in systematic reviews, JBI critical appraisal tools can also be used when creating Critically Appraised Topics in journal clubs and as an educational tool.

Authors: Joanna Briggs Institute, Adelaide, Australia

PDF: JBI checklist for Cohort studies


Diagnostic

CASP Diagnostic CAT

https://casp-uk.net/casp-tools-checklists/

Summary: Critical Appraisal Skills Program (CASP): Cohort Studies is a methodological checklist which provides key criteria relevant to diagnostic studies.

Authors: Public Health Resource Unit, NHS, England

PDFCASP Diagnostic CAT

GATE CAT for Diagnostic Test Accuracy Studies:

https://www.fmhs.auckland.ac.nz/assets/fmhs/soph/epi/epiq/docs/GATE%20CAT%20Diagnostic%20Studies%20May%202014%202014%20V5.docx

Summary: This CAT developed by the University of Auckland presents a comprehensive study review process focused on the 5 steps of Evidence Based Practice. Steps you through the process of asking, accessing, appraising (using the RAMboMAN tool), applying and auditing.

Authors: The University of Auckland, New Zealand

PDF: GATE CAT for Diagnostic Test Accuracy Studies

Scottish Intercollegiate Guidelines Network (SIGN) Methodology Checklist 5: Diagnostic studies  

http://www.sign.ac.uk/checklists-and-notes.html   

Summary: This CAT developed by the Scottish Intercollegiate Guidelines Network (SIGN), scores the diagnostic study over 10 questions and provides an overall assessment of the studies effort to reduce bias. Whilst developed to be used for the development of clinical guidelines they are excellent CATs for single study appraisals

Authors: Scottish Intercollegiate Guidelines Network

PDF: SIGN Checklist 5: Diagnostic studies   

JBI checklist for Diagnostic Accuracy Studies

http://joannabriggs.org/research/critical-appraisal-tools.html    

Summary: JBI Critical appraisal tools have been developed by the JBI and collaborators and approved by the JBI Scientific Committee following extensive peer review. Although designed for use in systematic reviews, JBI critical appraisal tools can also be used when creating Critically Appraised Topics in journal clubs and as an educational tool.

Authors: Joanna Briggs Institute, Adelaide, Australia

PDF: JBI checklist for Diagnostic studies

CAT for an Article on Diagnosis or Screening

https://www.gla.ac.uk/media/media_64046_en.pdf

Summary: This 12 question CAT developed by the Dept. of General Practice, University of Glasgow can be used for diagnostic or screening studies, and is accompanied by a great jargon buster.

Authors: Dept. of General Practice, University of Glasgow

PDF: CAT for an Article on Diagnosis or Screening

CAT for Diagnostic Studies: Roever (2016) 

https://www.researchgate.net/publication/292612112_Critical_Appraisal_of_a_Diagnostic_Test_Study

Summary: This CAT is based on a combination of other CATs

PDF:  Roever 2016


Case Control Studies

CASP: Case Control Studies

https://casp-uk.net/casp-tools-checklists/

Summary: Critical Appraisal Skills Program (CASP): Cohort Studies is a methodological checklist which provides key criteria relevant to Case control studies.

Authors: Public Health Resource Unit, NHS, England

PDF: CASP: Case Control Studies

GATE CAT – Case control Studies

https://www.fmhs.auckland.ac.nz/assets/fmhs/soph/epi/epiq/docs/GATE%20CAT%20Case%20Control%20Studies%20May%202014%20V3.docx

Summary: This CAT developed by the University of Auckland presents a comprehensive study review process focused on the 5 steps of Evidence Based Practice. Steps you through the process of asking, accessing, appraising (using the RAMboMAN tool), applying and auditing.

Authors: The University of Auckland, New Zealand

PDF: GATE CAT – Case control Studies

Scottish Intercollegiate Guidelines Network (SIGN) Methodology Checklist 4: Case control studies  

http://www.sign.ac.uk/checklists-and-notes.html  

Summary: This CAT developed by the Scottish Intercollegiate Guidelines Network (SIGN), scores the case control study over 10 questions and provides an overall assessment of the studies effort to reduce bias. Whilst developed to be used for the development of clinical guidelines they are excellent CATs for single study appraisals

Authors: Scottish Intercollegiate Guidelines Network

PDF: SIGN Checklist 4: Case control studies  

JBI checklist for Case Control Studies

http://joannabriggs.org/research/critical-appraisal-tools.html     

Summary: JBI Critical appraisal tools have been developed by the JBI and collaborators and approved by the JBI Scientific Committee following extensive peer review. Although designed for use in systematic reviews, JBI critical appraisal tools can also be used when creating Critically Appraised Topics in journal clubs and as an educational tool.

Authors: Joanna Briggs Institute, Adelaide, Australia

PDF: JBI checklist for Case control studies

Centre for Evidence Based Medicine, Oxford University CAT for Case Control Studies

https://www.cebma.org/wp-content/uploads/Critical-Appraisal-Questions-for-a-Case-Control-Study.pdf

Summary: This CAT for Case control Studies has been developed by the Centre for Evidence Based Medicine, Oxford University, and has been adapted from Crombie, The Pocket Guide to Critical Appraisal; the critical appraisal approach used by the Oxford Centre for Evidence Medicine, checklists of the Dutch Cochrane Centre, BMJ editor’s checklists and the checklists of the EPPI Centre.

PDF:  CEBM CAT for a Case Control-Study


Economic Studies

CASP: Economic Evaluation studies

https://casp-uk.net/casp-tools-checklists/

Summary: Critical Appraisal Skills Program (CASP): Cohort Studies is a methodological checklist which provides key criteria relevant to Case control studies.

Authors: Public Health Resource Unit, NHS, England

PDF: CASP: Economic Evaluation studies

Scottish Intercollegiate Guidelines Network (SIGN) Methodology Checklist 6: Economic studies  

http://www.sign.ac.uk/checklists-and-notes.html   

Summary: This CAT developed by the Scottish Intercollegiate Guidelines Network (SIGN), scores the economic study over 10 questions and provides an overall assessment of the studies effort to reduce bias. Whilst developed to be used for the development of clinical guidelines they are excellent CATs for single study appraisals

Authors: Scottish Intercollegiate Guidelines Network

PDF: SIGN Checklist 6: Economic studies   

JBI checklist for Economic Evaluations

http://joannabriggs.org/research/critical-appraisal-tools.html      

Summary: JBI Critical appraisal tools have been developed by the JBI and collaborators and approved by the JBI Scientific Committee following extensive peer review. Although designed for use in systematic reviews, JBI critical appraisal tools can also be used when creating Critically Appraised Topics in journal clubs and as an educational tool.

Authors: Joanna Briggs Institute, Adelaide, Australia

PDF: JBI checklist for Economic Evaluations


Other Quantitative

McMaster Critical Review Form - Quantitative Studies

http://www.unisa.edu.au/Global/Health/Sansom/Documents/iCAHE/CATs/McMasters_Quantitative%

Summary: McMaster Critical Review Form for Qualitative studies contains a generic quantitative appraisal tool, accompanied by detailed guidelines for usage.

Authors: Occupational Therapy Evidence-Based Practice Research Group, McMaster University, Canada

PDF: McMaster Critical Review Form - Quantitative Studies

HCPRDU evaluation tool for quantitative studies

http://usir.salford.ac.uk/12969/

Summary: The Evaluation Tool for Quantitative Studies contains 51 questions in six sub-sections: study evaluative overview; study, setting and sample; ethics; group comparability and outcome measurement; policy and practice implications; and other comments.

Authors: Health Care Practice Research & Development Unit (HCPRDU), School of Nursing, University of Salford, UK CriSTal Checklist

PDF: HCPRDU evaluation tool for quantitative studies

READER Critical Appraisal Tool

http://www.ncbi.nlm.nih.gov/pmc/articles/PMC1238789/pdf/brjgenprac00035-0039.pdf

Summary: A tool used to aid critical reading by general practitioners which can also be used to CAT an article

Authors: Macauley D, Queens University, Belfast, Northern Ireland

PDF: READER CAT

GATE CAT – Risk Factor or Prognostic Studies 

https://www.fmhs.auckland.ac.nz/assets/fmhs/soph/epi/epiq/docs/GATE%20CAT%20Risk%20Factor%20Cohort%20Studies%20May%202014%20V3.docx

Summary: This CAT developed by the University of Auckland presents a comprehensive study review process focused on the 5 steps of Evidence Based Practice. Steps you through the process of asking, accessing, appraising (using the RAMboMAN tool), applying and auditing.

Authors: The University of Auckland, New Zealand

PDF: GATE CAT – Risk Factor or Prognostic Studies  

Critical Appraisal checklist for a study on educational intervention

https://www.gla.ac.uk/media/media_64040_en.pdf

Summary: This CAT developed through the University of Glasgow involves 13 questions that should be asked when reviewing a study involving educational interventions

Authors: Dept. of General Practice, University of Glasgow, UK

PDF: CAT for educational interventions

MINORS Tool

http://cobe.paginas.ufsc.br/files/2014/10/MINORS.pdf

Summary: MINORS is a valid instrument designed to assess the methodological quality of non-randomized surgical studies, whether comparative or non-comparative.

Authors: Slim et al, Department of General and Digestive Surgery, Hotel-Dieu, France

PDF: MINORS tool

The Single-Case Experimental Design (SCED) Scale

http://www.psycbite.com/docs/The_SCED_Scale.pdf

Summary: The  SCED scale was developed to assess the methodological quality of single-subject designs. It is a validated scale, that can also be used as a single-subject case study design checklist.

Authors: RL Tate, Mcdonald S, Perdices M, Togher L, Schultz R, Savage S.

PDF: SCED Scale

JBI Other Quantitative Checklists

http://joannabriggs.org/research/critical-appraisal-tools.html       

Summary: JBI Critical appraisal tools have been developed by the JBI and collaborators and approved by the JBI Scientific Committee following extensive peer review. Although designed for use in systematic reviews, JBI critical appraisal tools can also be used when creating Critically Appraised Topics in journal clubs and as an educational tool.

Authors: Joanna Briggs Institute, Adelaide, Australia

JBI checklist for Case Reports

PDF: JBI checklist for Case reports

JBI checklist for Case Series

PDF: JBI checklist for Case Series

JBI checklist for Prevalence Studies

PDF: JBI checklist for Prevalence Studies 

JBI checklist for Quasi experimental studies

PDF: JBI checklist for Quasi experimental studies

JBI checklist for analytical cross sectional studies

PDF: JBI checklist for analytical cross sectional studies


Qualitative

CASP: Qualitative Research

https://casp-uk.net/casp-tools-checklists/

Summary: Critical Appraisal Skills Program (CASP): Qualitative Research is a methodological checklist which provides key criteria relevant to qualitative research studies.

Authors: Public Health Resource Unit, NHS, England

PDF: CASP: Qualitative Research

McMaster Critical Review Form - Qualitative Studies

https://srs-mcmaster.ca/wp-content/uploads/2015/04/Critical-Review-Form-Qualitative-Studies-Version-2-English.doc

Summary: McMaster Critical Review Form for Qualitative studies contains a generic quantitative appraisal tool, accompanied by detailed guidelines for usage.

Authors: Occupational Therapy Evidence-Based Practice Research Group, McMaster University, Canada

PDF: McMaster Critical Review Form - Qualitative Studies

Critical Appraisal Checklist for Qualitative Research Studies

https://link.springer.com/article/10.1007/BF02820685

Summary: A checklist of 10 questions to help critically appraise qualitative research studies

Authors: Carla Treloar , Sharon Champness, Paul L. Simpson, Nick Higginbotham

PDF: Critical Appraisal Checklist for Qualitative Research Studies

JBI checklist for Qualitative Research

http://joannabriggs.org/research/critical-appraisal-tools.html

Summary: JBI Critical appraisal tools have been developed by the JBI and collaborators and approved by the JBI Scientific Committee following extensive peer review. Although designed for use in systematic reviews, JBI critical appraisal tools can also be used when creating Critically Appraised Topics in journal clubs and as an educational tool.

Authors: Joanna Briggs Institute, Adelaide, Australia

PDF: JBI checklist for Qualitative Research


Mixed Methods

Mixed Methods Appraisal Tool (MMAT) – Version 2011

http://www.nccmt.ca/knowledge-repositories/search/232%20(accessed%20May%202017)

Summary: National Collaborating Centre for Methods and Tools (2015). Appraising qualitative, quantitative and mixed methods studies included in mixed studies reviews: The MMAT. Hamilton, ON: McMaster University.   

AuthorsNational Collaborating Centre for Methods and Tools, McMaster University, Canada

PDF: Mixed Methods Appraisal Tool (MMAT)

Evaluation Tool for Mixed Methods Studies

http://usir.salford.ac.uk/13070/1/Evaluative_Tool_for_Mixed_Method_Studies.pdf

Summary: The evaluation tool for mixed studies allows appraisal of both the qualitative data collection and analysis component and the wider quantitative research design. It is applicable where the aim of the qualitative component is to draw out the informants understandings and perceptions.

Authors: Professor Andrew Long, School of Healthcare, University of Leeds

PDF: Evaluation Tool for Mixed Methods Studies

A Scoring system for Mixed Methods Research and Mixed Studies Reviews

https://www.sciencedirect.com/science/article/pii/S0020748909000145?via%3Dihub

Summary: A new form of literature review has emerged, Mixed Studies Review. These reviews include qualitative, quantitative and mixed methods studies. This scoring system assesses Qualitative, Quantitative experimental, Quantitative observational and Mixed Methods at the one time.

Authors: Pluye et al (2009) International Journal of Nursing Studies, 46: 529-46.

PDF: A scoring system for appraising mixed methods research, and concomitantly appraising qualitative, quantitative and mixed methods primary studies in Mixed Studies Reviews


Systematic Reviews

CASP: Systematic Reviews

https://casp-uk.net/casp-tools-checklists/

Summary: Critical Appraisal Skills Program (CASP): Systematic Reviews is a methodological checklist which provides key criteria relevant to systematic reviews.

Authors: Public Health Resource Unit, NHS, England

PDF: CASP: Systematic Reviews

AMSTAR

 

https://amstar.ca/Amstar_Checklist.php 

Summary: Assessment of Multiple Systematic Reviews (AMSTAR) is a 37-item assessment tool used to assess the methodological quality of systematic reviews.

Authors: EMGO Institute, The Netherlands

PDF: AMSTAR Printer Friendly

Scottish Intercollegiate Guidelines Network (SIGN) Methodology Checklist 1: Systematic Reviews

http://www.sign.ac.uk/checklists-and-notes.html     

Summary: This CAT developed by the Scottish Intercollegiate Guidelines Network (SIGN), scores the economic study over 10 questions and provides an overall assessment of the studies effort to reduce bias. Whilst developed to be used for the development of clinical guidelines they are excellent CATs for single study appraisals

Authors: Scottish Intercollegiate Guidelines Network

PDF:SIGN Checklist 1: Systematic Reviews   

JBI checklist for Systematic Reviews

http://joannabriggs.org/research/critical-appraisal-tools.html    

Summary: JBI Critical appraisal tools have been developed by the JBI and collaborators and approved by the JBI Scientific Committee following extensive peer review. Although designed for use in systematic reviews, JBI critical appraisal tools can also be used when creating Critically Appraised Topics in journal clubs and as an educational tool.

Authors: Joanna Briggs Institute, Adelaide, Australia

PDF:  JBI Checklist for Systematic Reviews

Centre for Evidence Based Medicine, Oxford University

https://www.cebm.net/2014/06/critical-appraisal/

Summary: This CAT presented by the CEBM, scores the SR over 5 questions. Many of the questions are present in the CASP CAT

Authors: Centre for Evidence Based Medicine, Oxford University 

PDF: CEBM SR

Areas of study and research

+ Click to minimise