EBP Step 2

Types of evidence
Hierarchy of Evidence
Setting the search strategy


Types of evidence

Is your question about treatment, exploration, aetiology, diagnosis or prognosis? The type of question will inform the type of research evidence you need to look for to address your clinical question

Secondary evidence or secondary research:

A type of research that involves the summary, collation, and synthesis of findings from existing or already completed primary studies. A systematic review, using either meta-analytic analysis or narrative synthesis, is the primary methodology in secondary research.

A systematic review summarises the results of available carefully designed studies in response to a research question. A high quality systematic review is considered to be the most reliable source of evidence to guide clinical practice.

Podcast: What is a systematic review?

Features of a systematic review include (Clarke 2011)

  1. Clear aims with pre-determined eligibility and relevance criteria for studies
  2. Transparent and reproducible methods
  3. Rigorous search designed to locate all eligible studies
  4. An assessment of the validity of the findings of the included studies
  5. Systematic presentation and synthesis of the included studies

Useful resources for the steps in developing, conducting and writing a systematic review:

Steps for writing a systematic review protocol

Steps for developing a search strategy

Critical appraisal

Data extraction

Quantitative extraction

  1. Data extraction for systematic research synthesis
  2. Data extraction and coding for systematic reviews

Qualitative extraction

  1. Noyes J & Lewin S. Chapter 5: Extracting qualitative evidence. In: Noyes J, Booth A, Hannes K, Harden A, Harris J, Lewin S, Lockwood C (editors), Supplementary Guidance for Inclusion of Qualitative Research in Cochrane Systematic Reviews of Interventions. Version 1 (updated August 2011). Cochrane Collaboration Qualitative Methods Group, 2011.

Data synthesis

Qualitative synthesis

Quantitative synthesis


Further reading

Dixon-Woods M, Agarwal S, Jones D, Young B, Sutton A. Synthesising qualitative and quantitative evidence: a review of possible methods. Journal of health services research & policy, 2005; 10(1): 45-53B.


Useful websites to find systematic reviews:

The Cochrane Library
The Joanna Briggs Institute
Centre for Reviews and Dissemination
The Campbell Collaboration
PubMed


Further reading

Hemingway P. What is a systematic review, 2nd ed. Evidence based med 2009.
Green S. Systematic reviews and meta-analysis. Singapore Med J 2005 Jun; 46(6):270-3.
Centre for cognitive ageing and cognitive epidemiology. Systematic reviews and meta-analyses: a step-by-step guide.


top

Primary evidence or primary research:

Any research that is conducted on human subjects or animals (or parts thereof), and which therefore requires ethical approval.

A beginner’s guide to ethics approval

National Statement on Ethical Conduct in Human Research

Observational research: Research in which there is no deliberate intervention, and subject choices (and their effects) are observed and measured.

Cross sectional: the observation of a defined population at a signal point in time or time interval. Exposure and outcome are determined simultaneously.

Case control: where characteristics of a known group of subjects with a specific disease (cases) are compared with matched controls. Matching is usually by key confounding variables.>

Cohort study: a group of subjects is recruited and their disease and exposure status is measured over time.


Further reading

Jepsen P, Johnsen SP, Gillman MW & Sørensen HT. Interpretation of observational studies. Heart 2004; 90:956–960.

Mann CJ. Observational research methods. Research design II: cohort, cross sectional, and case-control studies. Emerg Med 2003; 20:54-60.


top

Experimental research: A planned attempt to assess the effects of a prospectively-delivered intervention. Interventions can also be called treatment(s), or manipulation of features of subjects’ environments.

N=1 (Single Case Experimental Design) (SCED): where one individual is given, at least, one intervention, and the effect of that intervention is compared with pre-intervention (baseline) measures. Within-subject variability is established by the distribution of data of repeated measures taken from that individual.

Using Case Studies in Research

Dallery J, Cassidy RN, Raiff BR. Single-case experimental designs to evaluate novel technology-based health interventions. J Med Internet Res 2013; 15(2):e22.

Pre-post study: where the one group of subjects is given the same intervention, and this group of subjects acts as its own controls. Outcome measures are taken before (pre, baseline), the intervention/treatment is delivered, and the outcome measures are taken again (post-intervention).

Pretest-posttest designs explained

Quasi-experimental study: two groups of subjects are compared, usually in different time periods or under different circumstances. One group usually is the ‘control’ in which no intervention occurs (usual care) and the second group receives the intervention. This is an experimental study approach to negotiate around situations where having a concurrent control/intervention study in the one organisation may result in violation of the study protocol (for instance where control subjects may end up receiving the intervention). This can occur when the control and intervention are being delivered concurrently in one hospital in two wards (one control, one intervention) and the control ward staff start delivering the intervention (or vice versa).

Quasi-experimental study: designs, elements and examples

Randomised controlled trial: an experimental study conducted in a sample of subjects randomly selected from a known population. Intervention arms are randomly assigned.

Stolberg HO, Norman G, Trop I. Randomised controlled trials. Am J Roentgenol 2004; 183: 1539-1544.


Further reading

Knight KL. Study/Experimental/Research Design: much more than statistics. J Athl Training 2010; 45(1): 98-100.


Activity

iCAHE MS Excel training booklet - A short course in data handling and statistical function

Excel training dataset


top

Qualitative research: A method of inquiry which aims to gather an in-depth understanding of human behaviour and the reasons that govern such behaviour. The qualitative method investigates the why and how of decision making, not just what, where, when and how much. Hence, smaller but focused samples are more often needed than large samples.

Podcast: Qualitative Research: Points for beginners (11MB)

Qualitative researchers use different data collection techniques such as focus groups, interviews, ethnography and participant observation and document analysis.

Common qualitative research methodologies are:

Participatory Action Research (PAR): seeks to understand the world by trying to change it, collaboratively and reflectively. Aims to examine the political structures that disempower marginalised, deprived and oppressed groups of people to find ways in which these structures can be changed. It aims to transform “social realities”.

Baum F, MacDougall C, Smith D. Participatory action research. J Epidemiol Community Health. 2006 Oct; 60(10): 854–857.

Ethnography: is aimed at exploring cultural phenomena. It reflects the knowledge and the system of meanings in the lives of a cultural group. Ethnography is a means to represent graphically and in writing, the culture of a group of people. The typical ethnography is a holistic study and so includes a brief history, and an analysis of the terrain, the climate, and the habitat. It observes the world (the study) from the point of view of the subject (not the participant ethnographer) and records all observed behaviour.

Reeves S, Kuper A, Hodges BD. Qualitative research methodologies: ethnography. BMJ 2008:a1020

Grounded theory: is a research methodology that seeks to develop theory that is grounded in data systematically gathered and analysed. Rather than beginning with a hypothesis, the first step is data collection, through a variety of methods. From the data collected, the key points are marked with a series of codes, which are extracted from the text. The codes are grouped into similar concepts in order to make them more workable. From these concepts, categories are formed, which are the basis for the creation of a theory, or a reverse engineered hypothesis. This contradicts the traditional model of research, where the researcher chooses a theoretical framework, and only then applies this model to the phenomenon to be studied.

Noerager Stern P. Grounded theory methodology: Its uses and processes.

Holton JA. Grounded theory as a general research methodology.

Phenomenology: is the study of a phenomenon and aims to study everyday situations from the view point of the experiencing person. It describes the “subjective reality” of an event, as perceived by the study population.

Lester S. An introduction to phenomenological research, Stan Lester Developments, Taunton.


Further reading

Creswell JW, Hanson WE, Clark Plano VL, Morales, A. Qualitative research designs: selection and implementation. The Counseling Psychologist 2007; 35:236.

Patton MQ. Qualitative research & evaluation methods (3rd ed.) 2002. Thousand Oaks, CA: Sage Publications

Lingard L, Albert M, Levinson W. Grounded theory, mixed methods, and action research. BMJ 2008; 337 doi: http://dx.doi.org/10.1136/bmj.39602.690162.47


Mixed Methods research: The following articles describe what a mixed methods research is (and what it is not), when it should be used, how it is designed and the methodological challenges associated with conducting mixed methods investigation.

The Nature and Design of Mixed Methods Research

Steps in Conducting a Scholarly Mixed Methods Study

Mixed methods in Biomedical and Health Services Research

Major mixed methods designs:

Convergent parallel design (also known as Triangulation design): The purpose of the convergent design is to obtain different but complementary data on the same topic to best understand the research problem. This design is used when the researcher wants to triangulate the methods by directly comparing and contrasting quantitative statistical results with qualitative findings for corroboration and validation purposes.

Example of a study which utilised this approach:

Henwood BF, Katz ML, Gilmer TP. Aging in place within permanent supportive housing. Int J Geriatr Psychiatry 2015; 30(1): 80–87

Explanatory Sequential Design: This is a mixed methods design in which the researcher begins by conducting a quantitative phase and follows up on specific results with a second phase – qualitative phase. The qualitative phase is intended to explain the initial quantitative results in more depth. This design is most useful when the objective is to assess trends and relationships with quantitative data and also explain the mechanism or reasons behind the resultant trends.

Example of a study which utilised this approach:

Green LA, Lowery JC, Kowalski CP, Wyszewianski L. Impact of Institutional Review Board Practice Variation on Observational Health Services Research. Health Serv Res. 2006 Feb; 41(1): 214–230.

Exploratory Sequential Design: The primary purpose of this mixed methods design is to generalise qualitative findings based on a few individuals to a larger sample gathered during the second phase – quantitative phase. This design is particularly useful when the objective is to develop and test an instrument because one is not available or to identify important variables to study quantitatively when the variables are unknown. It is also appropriate when one wants to generalise qualitative results to different groups, to test aspects of an emergent theory or classification, or to explore a phenomenon in depth and measure the prevalence of its dimensions.

Example of a study which utilised this approach:

Yildirim C. Exploring the dimensions of nomophobia: Developing and validating a questionnaire using mixed methods research. Graduate Theses and Dissertations 2014. Paper 14005, Iowa State University

Embedded Design: This is a mixed methods approach where the researcher combines the collection and analysis of both quantitative and qualitative data within a traditional quantitative research design or qualitative research design. The premises of this design are that a single data set is not sufficient, that different questions need to be answered, and that each type of question requires different types of data.

Example of a study which utilised this approach:

Strachan JC. Facing choices: a mixed-methods approach to patients’ experience of care and discharge in an inpatient mental health unit. Health in Social Science Thesis Collection 2012. The University of Edinburgh.


Reference

Creswell J, Plano Clark V. Designing and conducting mixed methods research. 2nd edition, Sage Publications, California USA.


Further reading

The use of "mixing" procedure of mixed methods in health services research

Choosing a Mixed Methods Design

Hayes B, Bonner A, Douglas C. An introduction to mixed methods research for nephrology nurses. Renal Society of Australasia Journal 2013; 9(1): 8-14


Expert opinion:

This refers to the views of professionals who have expertise in a particular form of practice or field of inquiry, such as clinical practice or research methodology.
top


Hierarchy of Evidence

The hierarchy of evidence is a way of ranking the relative authority of various research designs based on the degree of bias within their methodologies. There is no standard hierarchy of evidence; however there is agreement on the position in the hierarchy of different types of research. Secondary research (systematic reviews, meta-analyses) rank at the top of the hierarchy because they include data from multiple primary studies. Primary researches rank next to secondary research and expert opinion is ranked lowest in the hierarchy.

NHMRC Hierarchy of Evidence


Further reading

Merlin T, Weston A, Tooher R. Extending an evidence hierarchy to include topics other than treatment: revising the Australian levels of evidence. BMC Med Res Methodology 2009; 9:34.


top


Setting the search strategy

Podcast: Developing a search strategy.

Example Question: What is the evidence for the effectiveness of Mediterranean diet in patients with type II diabetes?

P (population)

Middle aged man with type 2 diabetes

I (intervention)

Mediterranean diet

C (comparison)

Other types of diet

O (outcome)

Glycaemic level

Select relevant databases

The following databases are commonly used to find evidence-based practice related information:

MEDLINE: contains bibliographic and abstract coverage of biomedical literature including coverage in the areas of allied health, biological and physical sciences, humanities and information science as they relate to medicine and health care, communication disorders, population biology, and reproductive biology.

CINAHL: covers nursing, biomedicine, health sciences librarianship, alternative/complementary medicine, consumer health and several allied health disciplines. This database also offers access to health care books, nursing dissertations, selected conference proceedings, standards of practice, educational software, audiovisuals and book chapters.

PEDro: provides physiotherapists and others access to bibliographic details and abstracts of randomised controlled trials and systematic reviews in physiotherapy.

OTseeker: a database that contains abstracts of systematic reviews and randomised controlled trials relevant to occupational therapy. Trials have been critically appraised and rated to assist Occupational Therapists to evaluate their validity and interpretability. These ratings will help in judging the quality and usefulness of trials for informing clinical interventions.

Cochrane Library: covers all areas of health care; provides high quality evidence to inform people providing and receiving care, and those responsible for research, teaching, funding and administration at all levels. The library consists of 6 databases: Cochrane database of systematic reviews (CDSR), Database of abstracts of reviews of effects (DARE), Cochrane central registrar of controlled trials (CENTRAL), Cochrane methodology register (CMR), Health technology assessment database (HTA), and NHS economic evaluation database (NHSEED).

Google Scholar: Google Scholar is a free service for searching scholarly literature. It includes peer-reviewed papers, theses, books, preprints, abstracts and technical reports.

Choose key words from your question

To develop a search plan, identify the main concepts from your question (PICO or the other formats) and determine the relationships between these concepts. Break your topic into several parts to make it more manageable.

Example

Concept 1: Type 2 diabetes

Concept 2: Mediterranean diet

Concept 3: Glycaemic index

These concepts are the keywords you will use to search for research evidence. Remember to include similar words (synonyms) or alternative spellings, complete names or abbreviations, scientific or common names. In cases when a database has its own indexing language, or controlled vocabulary (e.g. medical subject headings (MeSH)), the search terms should include these index terms.

Example

Concept 1: type 2 diabetes, type II diabetes, Diabetes Mellitus, adult onset diabetes, maturity onset diabetes etc.

Concept 2: Mediterranean diet, diet, dietary pattern, Mediterranean, Mediterranean food, Mediterranean eating, Mediterranean nutrition etc.

Concept 3: Glycaemic index, Glycaemic level, blood glucose, blood sugar etc.

Think about how these keywords relate to one another, use Boolean operators (OR, AND) as appropriate. When you combine keywords using OR, your search will find items containing either one or both of your keywords. Therefore, use OR to make your search broader, and to combine words with similar meanings. On the other hand, when you combine keywords using AND, your search will only find items containing both of your keywords.

Most databases allow you to use a symbol to search for all words beginning with a particular base. These are called truncation symbols and may vary depending on the database software (?, *, $)

e.g. typing learn* in the Cochrane Library would find learn, learns, learner, learners, learning

Another symbol, usually called a Wildcard, may be used to search for alternative spellings or forms

e.g. typing organi?ation in the Cochrane Library would find both organisation and organizationwhile typing col?r would find both color or colour

For a step-by-step guide on developing a systematic search strategy see: A short course in searching the literature: A basic introduction to developing a search strategy, database hints, and executing a search

For a step by step guide in searching for evidence (PubMed example), refer to the following article:

Stillwell B, Fineout-Overholt E, Melnyk B, Williamson K . Searching for the evidence: strategies to help conduct a successful search. AJN 2010; 110(5): 41-47


<<Back to EBP Step 1

Go to EBP Step 3>>

EBP online home

top

Areas of study and research

+ Click to minimise