Jump to Content

 

Qualitative and quantitative research design

The research design tells the reader how the research was operationalised. It tells them:

In this section of your thesis or research proposal you explain and justify your:

In quantitative and experimental research the methods section will provide enough detail for the research to be replicated, and provide reassurance that you are using the best method to get the best (valid, generalisable) results.

In qualitative research the methods section will provide details of factors that might influence the collection and analysis of the data. For example, if you are an older female interviewing adolescent males about sexual practices, you are likely to have access to particular groups of males (they could self-select to those who have ‘acceptable’ practices) and receive particular kinds of responses to your questions. You need to reflect on how this will bias your research – it does not necessarily undermine it, but it will give you a particular perspective on the sexual activity of adolescent males. In order to be rigorous in your research activity you need to identify any potential biases, reflect on what these might mean for your research, and (if necessary or possible) build in measures to counteract their influence.

How much to write?

The amount of information provided when writing about research methods will differ depending on whether or not you are using a method that has stood the test of time and is well recognised in the field.

Using a standard method

If you are using a well recognized approach, then the important point is to let the reader know that you are using a standard method by referring to other research that has used the same method. Its past success then justifies your choice.

Using a modified method

If, however, you want to modify a method then you need to explain exactly what you have done and why. You will still need to base your choice on a standard method, explaining why you thought this was the best starting point for your research. You then explain why you need to modify it. Your reasons for modification will be different depending on your project, but they need to have a logical basis: your research may be using a different data source which involves modifying equipment or techniques; you might want to test a reaction at a different temperature, or use a different combination of materials than has previously been used; or you might want to find a better way of measuring attitudes or perceptions than is currently available. While there may be increased risks in using modifications of standard approaches, there is also the increased potential for creating new knowledge because the modified method becomes part of the research process. You could end up with a new method and new content knowledge.

The answer to the question of how much to write therefore depends on how much information is necessary for an informed reader to know why you made the choices you did and what they would need to know in order to replicate the study.

In most research proposals the framework or outline of the research design is stated. The following examples show how this might be done:

Statements on Research Design

Experimental: This is a randomized, parallel designed, concurrent control trial with subjects randomly allocated to either Group 1 (which receives ambulation and ROM exercises) or Group 2 (who receive strengthening exercises in addition to ambulation and ROM exercises).
Quantitative: An anonymous questionnaire containing 22 questions relating to the prevalence of use and knowledge of bovine colostrums will be distributed to local gymnasiums and sporting organizations in the Adelaide metropolitan area.
Qualitative: This study uses ethnographic techniques to represent, interrogate, juxtapose and construct the experience of pain for surgical clients.

You will of course need to elaborate on the information provided. In particular, you will need to justify why you chose this research design. To do this you can refer back to previous research that has been conducted on your topic and/or relate your choice to the appropriate theory of methods.

Methods

This section of your research writing provides detailed information about what you intend to do, and may cover the following:

Obviously you will not need to cover all of these aspects – this will depend upon your methodology. If you need to complete an ethics proposal before starting your research, it is practical to use the questions on the ethics proforma to guide the kinds and level of information required.

Writing about research design

All research writing has a section that provides a rationale for the method adopted. Writing about these aspects of your research differs according to the form of communication and the type of research that you are doing. Journal articles will often use a condensed version. Abstracts might only have a couple of sentences, while a thesis will provide much more detail and might even consist of one or two chapters. Writing about experimental methods is quite standardised and contains common elements, while writing about qualitative research is more dependent on the particular project. One of the main differences, however, is that writing about experimental methods focuses almost exclusively on the actual experiment, while writing about qualitative methods focuses more on factors affecting the research such as the researcher, the potential biases of the participants, and the strategies used to collate, collect and analyse the data.

It is therefore much easier to provide guidelines on how to write about experimental research than about other kinds of research. The following discussion contains information about the standard requirements for writing about experimental research. It will then consider the requirements for qualitative research, and provide an example to illustrate the differences.

Experimental or Hypothesis-testing Research

The methods section of a scientific report usually covers two main areas: materials and experimental rationale. Each of these sections should include enough detail so that your research can be duplicated by others and (hopefully) obtain similar results. The methods section will contain references to other people’s methods, but there would not be any results in this section.

Materials

The materials section includes information about subjects or participants in the study, equipment used or variables examined, interventions and treatments or chemicals used and their preparation.

Subjects

The choice of subjects for any study is a critical feature as poor choices could lead to false or misleading outcomes. For example, if you examined the role of soy supplements in reducing hot flushes in post menopausal women, then the subjects need to be post menopausal women. This may sound obvious, but you need to consider how menopause will be confirmed, for example by participants’ assessment or by setting predefined blood levels of leutenising and follicle stimulating hormones. Information regarding the inclusion and exclusion criteria for subjects must also be included and it needs to be explicit and justifiable. In the example used above, the types of issues to be addressed would include:

This section also provides information justifying the sample size; use a power analysis if possible. A power analysis gives the minimum number of subjects required to avoid a type II error (false negative). Unless you have this minimum sample size your study will not be worthwhile – even if you prove the hypothesis to be true, the study is unlikely to give significant results to support it.

Variables examined

The variables examined are the interventions made (independent variables) and the variables measured (dependent variables). For the example given above, the number and frequency of hot flushes over a defined time period would be the dependent variable and the soy tablets would be the independent variable. A description of each variable is required. A description of the dependent variables in the hypothetical study would outline how the hot flushes were measured. It could be either a descriptive account of the number and severity of episodes using a questionnaire, or a subjective measure of sweating using skin conductance recordings. If the latter was used, then details about the equipment, its location on the subject’s body, and how it operates must be included in the text. Information concerning whether skin conductance was measured 24hrs per day or nightly, how often, and at what intervals should also be included.

Intervention

The intervention is what you administered; it may be a drug treatment, cognitive behavioral therapy, occupational therapy, physiotherapy exercises or some type of surgical procedure. In the soy treatment for menopause example, the intervention would be soy tablets compared with a matching placebo tablet (also known as the control). In that case, the formulation of the tablets, dose, frequency and length of the intervention all need to be described in detail and, where appropriate, justified.

Rationale

This section of the report tells the reader how the research was conducted. It is a description of the study design and the procedures used. In some instances, the procedures can be followed more easily if they are presented in a flow chart or table as shown below.

Procedures - Study description for use of soy tablets in menopause trial.

Timeline

Intervention
Recruitment 240 subjects give informed consent
-2 to 0 weeks 2 week wash in phase with placebo
0 220 subjects randomized to one of 2 groups (Soy or placebo),
n=110 per group
Day 1 Soy 25 mg or placebo twice daily, for 12 weeks
measure sweating and temperature using conductance meter; symptom check list with questionnaire.
Week 6 measure sweating and temperature using conductance meter; symptom check list with questionnaire.
Week 12 measure sweating and temperature using conductance meter; symptom check list with questionnaire.
Week 13 Break randomization code, analyse data


When describing the procedures you must provide enough information, so that other researchers can repeat your study. If a well established methodology has been strictly followed then you do not need to include all of the details, but simply provide a reference to the original method so that others can refer to it. For example: Body temperature was measured by radiotelemetry using the method develop by Bloggs (1980). However, if you have adapted someone else’s method, then any modifications must be fully described and justified. For example: Body temperature was measured by radiotelemetry using a similar method to that developed by Bloggs (1980). In the present study the radiotransmitter was placed on the skin and not in the peritoneal cavity. This modification was considered necessary because skin temperature decreases as core temperature increases (Hall 2001). If a totally new method has been devised, for example, a method of measuring body temperature, then this has to be described, justified and validated. That is, we will need to provide evidence that our new method measures body temperature accurately and reliably by comparing it to another accredited method or instrument, for example: National Australian Testing Authority (NATA) certified thermometer.

For a double-blind, placebo-controlled, randomized clinical trial, such as the hypothetical soy trial in post-menopausal women, information regarding randomization procedures, masking of treatments or interventions, blinding of assessors, and expected treatment differences are also included in the methodology. Finally, the tests used for establishing statistical significance need to be described, including the name(s) of software packages used. The statistical tests used must be appropriate comparisons and not just those that favour the results. For example, if you wish to show the size of an effect of soy treatment on hot flushes then the correct comparison would be one between soy and placebo, giving the mean difference with confidence intervals. Other comparisons such as improvements from baseline may look impressive, but if people taking placebos also improved then these statistical analyses do not tell the reader anything meaningful.

This is the section where you justify what you did and your choice of methods. This justification is necessary because some methods are better than others. That is, good methods are valid and reliable.

Validity

Validity refers to the relevance and accuracy of what is measured. There are two broad types of validity: internal and external. Internal validity refers to whether the tests measure accurately what they were designed to measure. External validity refers to the extent to which your findings can be generalised to other people, or situations, or at other times (Trochim 2000).

Reliability

Reliability or precision refers to the ability to repeatedly get the same result with the same instrument, regardless of the assessor. This result may not necessarily reflect the true mean, but it is consistent.

In example A below, the measurement is reliable but the mean is not valid (accurate). In example B the mean is valid but unreliable, and in example C the mean is both reliable and valid.

Validity and reliability are the goals of any scientific research, but reliability is the limiting factor in determining validity.

Once you have determined that your methods are both reliable and valid then the rationale for your experimental procedures needs to be explained. For example, in the hypothetical soy study we would need to describe how skin conductance is a measure of sweating and why sweating is an indirect measure of hot flushes. Similarly, the choice of questionnaire used to evaluate menopausal symptoms would have to be justified. In scientific research, the usual practice is to use a questionnaire developed and validated by someone prominent in the field. However, if you modify established procedures in any way, then the modifications must be described and justified.

Multiple methods

In the field of Health Sciences, it is common practice to use more than one method to assess similar outcomes. For example, objective and subjective measures of signs and symptoms are often used to determine any benefit from a particular intervention. Objective measures (for example sweating, skin temperature, blood tests) are viewed as more rigorous evidence for the efficacy of an intervention and are considered less likely to be influenced by factors such as mood or a desire for success of the intervention on the participants’ part. However, subjective measures, such as the participants’ assessment of the intervention, provide different but equally important information. There is little value in a wonderfully effective drug for the treatment of mouth ulcers if the side effects are worse than the ulcers. Hence, different methods may yield different types of information or the same information from a different perspective (for example, physician versus subject’s assessment). When more than one method is used to assess the outcome of an intervention, we need to describe each method separately, along with the reason for its inclusion. Each separate method or instrument should provide different information and not merely duplicate – or worse contradict – the findings found using another method.

Limitations

Any methodological limitations of your study should be acknowledged in this section of the report. Most studies have limitations and it is better to acknowledge them rather than ignore them. Ignoring them invites the reader to conjure up their own limitations of the study design. Once we have identified any methodological limitations, you then need to describe the steps you took to minimize any possible adverse effects these limitations may have on the outcomes of the study.

Example

The following example is taken from a Masters thesis. Note the way that the headings take the reader through the process by which the subjects were selected – the inclusion and exclusion criteria; the ethical procedures that were met; how the blood was collected and what the rejection criteria was; the kinds of control samples prepared; the method of testing used; and how validity and reliability were established.

Writing About Materials and Methods – Experimental Research
Hodge, Sandra J. (n.d.) Studies of Immuno-regulation in Inflammatory Processes, thesis prepared for the Degree of Master of Applied Science, University of South Australia
Section headings from chapters 2 & 3

Chapter 2: Subjects
2.1 2.1 Ethical guidelines
2.2 2.2 In vitro investigation of leucocyte activation markers and cytokines in whole blood
2.3 2.3 Guidelines for inclusion in study of leucocyte markers of inflammation in infected neonates
2.4 2.4 Selection of infected infants for inclusion in study
2.5 2.5 Selection of specificity controls for inclusion in study
2.6 2.6 Obstetric parameters for investigation

Chapter 3: Methods
3.1 Collection specifications
3.2 3.2 Controls
3.2.1 3.2.1 Procedure controls
3.2.2 3.2.2 Isotype control
3.2.3 3.2.3 Autoflourescence control
3.3 3.3 Stimulation of whole blood
3.4 3.4 Staining of leucocyte activation markers and cytokines with monoclonal antibodies
3.5 3.5 Determination of apoptic changes using Annexin V staining
3.6 3.6 Comparison of apoptic changes in stimulated whole blood and PBMCs
3.7 3.7 Analysis of stained samples
3.8 3.8 Validation experiments:
3.8.1 3.8.1 Confirmation of specificity of intracellular and surface staining
3.8.2 3.8.2 Stability of leucocyte activation markers in stored blood
3.8.3 3.8.3 Expression of activation markers in cord and venous blood
3.9 3.9 Reference ranges
3.10 3.10 Statistical analyses

Writing up

Here are some hints for writing up the methods and materials sections of a research project:

Qualitative Research

Writing about qualitative research is less likely to follow standard conventions. This is because there are different perspectives on the research that will influence the way that a method is used. For example, the interview method is used by a range of theoretical and methodological approaches. Therefore, what is important is how you locate your research within the field. This means being clear about your epistemology, theoretical perspective and methodology as these will influence how you:

Validity

Qualitative research may not have the same concern with validity and reliability that experimental or hypothesis driven research does. If the research is positivist, then these concerns will still be discussed. If, however, it is post-positivist (for example, interpretive, inductive, ethnographic) then issues surrounding validity take on a different meaning. This is because this kind of research views knowledge (and the construction of knowledge) as perspectival (involves the views of the researcher as well as the researched). Validity is therefore reconceptualised and involves a researcher demonstrating that they have been rigorous and ethical in conducting the research. This means, when writing about methodology/ methods, you need to be:

Transparent about the research process – delineate all of the influences that led you to a particular understanding;

Self-reflexive about the inter-relationships between the research context (why you did the research), the methods used, the setting, the participants and yourself as a researcher. Some of the issues that you need to be self-reflexive about are:

Multiple methods

Qualitative research often adopts multiple methods. There are several combinations of methods, and only a couple will be covered here.

The use of multiple methods to understand the different perspectives is called ‘crystallisation’ – where the data is like a crystal: there are many sides through which to look into the crystal. For example, if you were doing an ethnographic study of a psychiatric ward you might use participant observation, interviews of both patients and staff and a policy (discourse) analysis of procedures. This will provide multiple perspectives on various aspects of the ward thereby providing the research with breadth and depth of data to strengthen the arguments and allow you to add new information to a body of knowledge.

Another form of multiple methods uses different approaches to reinforce and strengthen the findings. This is called ‘triangulation’ and can involve using different methods, different researchers, changing the site or time (for example: following up with the same people after 12 months). One way to triangulate would be to interview 20 patients about their experience of lower back pain treatment, then identify some key questions and themes and construct a questionnaire to be sent out to 5000 people about the same issue.

When writing about multiple methods it is important to explain why the combination was selected – what each method aimed to add to the research. It is not ethical to do research just for the sake of it, and the research participants are likely to take a dim view of having their time wasted.

Example

The following example of qualitative research uses multiple methods – discourse analysis, interviews and participant observation. Note how the writer has structured her methodology chapters to explore the influence of epistemological, theoretical and methodological issues on her research. This example is from the contents pages of a PhD thesis in Nursing. The headings indicate that the writer has been reflective and transparent in her explanation of how the research was constructed, how the data was collected, collated, analysed and written about, and in locating herself in the research.

Writing about Methodology and Methods – Qualitative Research
Price, K. 2000. Exploring what the doing does… A poststructural analysis of nurses’ subjectivity in relation to pain. Thesis prepared for a Doctor of Philosophy, University of South Australia

Chapter Two: A poststructural way of thinking
2.1 2.1 Generating possibilities 20
2.2 2.2 How do we think? 21
2.3 2.3 Literature and nurses’ actions related to pain as products of writing 26
2.4 2.4 Reading nurses’ actions – Jacques Derrida 29
2.5 2.5 Nurses’ actions as effects of power – Michel Foucault 37
2.6 2.6 Reading writes a new text 45
2.7 2.7 The possibility and impossibility of a poststructural analysis 48
2.8 2.8 Moving on 50

Chapter Three: Assembling texts for analysis
3.1 3.1 Collecting textual materials 51
3.2 3.2 Ethnocentrism and Being ethnocentric 52
3.3 3.3 Generation of textual materials 54
3.4 3.4 Constituting my role as researcher and the role of participant 58
3.4.1 3.4.1 Negotiating access to a specific location 65
3.4.2 3.4.2 Gaining ethical approval 68
3.4.3 3.4.3 Negotiating access to participants 70
3.5 3.5 Ethnographic techniques 79
3.5.1 3.5.1 Observation of nurses’ actions 79
3.5.2 3.5.2 Collection of documentation 84
3.5.3 3.5.3 Interviewing participants 88
3.6 3.6 Writing a new text – how? 93
Making the links 97

These two examples of writing about methodology/methods have touched on only a few of the numerous issues relating to this aspect of research communication. One of the best ways to find out about how this is done in the kinds of research that you are interested in, is to read research articles and theses on related topics or which use similar methodology/methods to those that you are considering.

References

This resource was developed by Dr Debra King.

 

top^