Principal methods for healthcare research

Introduction

Bulimia Nervosa (BN) is an eating disorder, which is characterised by repeated episodes of over eating and bingeing and by a set of compensatory behaviours intended to cause weight loss which includes vomiting, purging, fasting and excessive exercise.

The American Psychiatric society (A. P. A. DSM-IV, 2000) and the WorldHealthOrganisation ( World Health Organization. International Statistical Classification of Diseases, 10th Revision (ICD-10). Geneva, Switzerland: W. H. O., 1992), both suggest the followingdiagnosticcriteria for the disorder

1. A persistent preoccupation with eating and irresistible craving forfood.

2 . Repeated episodes of binging/overeating in which large quantities /amounts of food are consumed

3. Patients experiencing BN take excessive measures to compensate for the ‘ fattening‘ effects of food e. g. vomiting, purging and fasting.

The Beating Eating Disorder Association (BEAT). Formerly known as theEating DisordersAssociation, a voluntary sector group in the UK., as provided figures that up to 1. 15 million people in the UK, experience a significant problem with BN, and up to 90. 000 people are receiving treatment at any one time.

It is not therefore surprising that BN is a major and widespread problem. Consequently a number of treatments have been developed and trialled, based on focused research, and developing an evidence base. The National Institute of Clinical Excellence (NICE). Published guidance in 2004, (Eating disorders: Core interventions in the treatment and management of anorexia nervosa, bulimia nervosa and related eating disorders. London: National Institute for Health and Clinical Excellence (NICE) 2004). Which recommends Cognitive Behavioural therapy (CBT) as an efficacious treatment as well as Selective Serotonin Re-uptake Inhibitors (SSRI’s) such as Fluoxetine. Research in this area is an important on a number of counts. Firstly, on clinical intervention grounds so that the treatment offered for the disorder can be expanded and improved and secondly given that the Disorder was only recognised in 1979 ( Russell, G. 1979), and with a widespread epidemiology, sufferers experiences of the disorder are important in clinical practice to deepen understanding of sufferers experiences..

The two papers presented here both focus on the condition from different perspectives and ask different questions. The first paper is a qualitative study: Binge and purge Processes in Bulimia Nervosa: A Qualitative Investigation (Jeppson et al. 2003) using a grounded theory design focusing on the two core diagnostic behaviours of BN; binging and purging. They aimed to elucidate information on these processes and their function for Bulimic’s experience.

The second paper is a comparative intervention study, following a quantitative research design – a form of Randomised Control Trial (RCT) – Cognitive-Behavioural, Fluoxetine and Combined Treatment for Bulimia Nervosa: Short- and Long-term Results (Jacobi et al. 2002). Both papers illustrate their design paradigms and contain a number of flaws, which will be discussed in relation to their respective research methods.

Their Respective Research Design and philosophy

Jeppson et al(2003), deployed a qualitative research design, which followed a grounded theory approach (Glaser & Strauss, 1967). Grounded theory was originally developed by these social researchers, before the approach ‘ split’ into different camps, (Stebbins, R. A., 2001), the approach essentially emphasizes the ’generation of theory’ from data that is acquired in the actual process of conducting the research.

So rather than beginning with an hypothesis that needs to be tested that is the hallmark of the experimental quantitative research design, grounded theory begins the research process with an aim for exploration.

The quantitative paradigm illustrated by the Jacobi et al. (2002) paper, forms the basis of RCT’s and Therapy trials which are common in much of health care research, including Cognitive Behavioural Therapy (CBT) and treatment approaches/interventions to Bulimia Nervosa. At its philosophical ‘ core’ is the notion of empiricism (Owen, F. & Jones R, 1977), whereby quantitative properties and phenomena and their relationships are observed and studied. Thus the process of measurement is crucial to the paradigm and this affords the connection between ‘ empirical’ observations i. e did X effect Y andmathematics, through the expression of quantitative relationships

As measurement is crucial to the quantitative paradigm, the choice of Randomised Comparison Study is entirely appropriate for the Jacobi investigation, as it wants to compare the treatment effects of three different conditions treating the disorder.

Studies of this type are a form of scientific experiment, with RCT’s often referred to as the ‘ gold standard’ for intervention studies (Cartwright, A. 2007). They are often used for ‘ safety’ studies in drug trials and for testing the effectiveness of clinical interventions and services

The two research studies presented here, display different philosophical foundations, which shape their whole approach to research deign.

The Contrasting Research Hypothesizes

Jeppson et al (2003), developed an aim for their qualitative study based on previous research they had encountered on the BN binge-purge process that is seen to be central to the disorder. The study was not concerned with assessing the ‘ effects’ of interventions on binge and purge behaviour, but rather to try and understand what these behaviours meant and the functions they served for the patients they studied.

They considered in previous research that quantitative methods, had focused on self-monitoring of these episodes and ranking them to ‘ measure’ links to mental state e. g. Stress, and had deemed that it had not sufficiently led to an understanding of their function as experienced by BN patients themselves. They had an idea that these behaviours were crucial and played a part in assisting sufferers with their ‘ emotional regulation’, but were curious and wished to expand an understanding of their role.

They therefore set out with an ‘ aim’ for the research to find out what BN patients experiences of these two processes and the meanings /function they served for them, I believethis ‘ aim’ was entirely congruent with the qualitative method, and provided a question that RCTs looking at interventions had not and could not study.

The Jacobi et al., paper(2002), asked a different set of questions. They were interested in how CBT compared with the SSRI – Fluoxetine, and a combination of them both . They considered that both the psychological (Wilson, G. T., & Fairburn, C. G., 1993) and pharmacological interventions (Goldstein et al., 1995) had developed an evidence base for treating BN. They designed an ambitious study to investigate the question of the relative short and long-term effectiveness of the three interventions. They questioned whether the combined approach would be more beneficial in the long-term, as previous research studies investigating a combination approach, had experienced high drop-out rates due to the side effects of the of the SSRI, which may have influenced their results.

The study did not include a control or placebo condition, which would be the normal procedure for intervention type studies. The reason for this was that they considered previous research had demonstrated the effectiveness of CBT and Fluoxetine, and was not needed.

The study had four aims: To compare the effectiveness of CBT, Fluoxetine and there combination, to compare the short and long-term effects of these treatments. To inquire about the possible ‘ additive’ effect of combined intervention and to examine the differential effects of the treatments. These types of intervention question are suitable for a randomised study as the questions Jacobi and team are asking are essentially ones of intervention and differential effectiveness/measurement on a number of variables e. g. Binging and purging episodes, short and long term effects as ascertained by validated psychological questionnaires and clinicalinterview.

Approaches to sampling

It is generally considered that sample size is generally much smaller in qualitative studies than those recruited for quantitative studies such as RCT’s (Cresswell, J. W., 2003). As qualitative studies are generally smaller in size, criticisms have been leveled at this paradigm. Although the paradigm does not seek to study the ‘ total‘ population, issues surrounding diversity and variation within a population are often cited (Jones, 2007).

The Jeppson study used only eight participants The approach to sampling used in the study is important as the process of selecting participants is relevant to the aims of the

study. They used purposeful sampling to generate the data for their enquiry.

Purposeful sampling generally refers to the selection of participants who have experience or knowledge of the area under investigation (Procter et al, 2010) As all participants were selected with a specific diagnosis of BN (APA., DSM-IV, 1994) they would all currently have experiences of the processes under investigation.

They were recruited from a suburban area in Utah, USA., Descriptive information, including socio economic variation, inpatient and out patient status and educational information was given. This form of sampling used in the study is not random or based on statistical probability as in quantitative studies.

Sampling in quantitative research design, especially RCT’s, differs. This type of sampling is in essence the selection of of observable units, which can be measured (Maxim, P. S., 1999). As outlined in the introduction, BN is widespread in the UK (and other european countries). It is not possible in the Jacobi study to observe the effects of the treatment interventions with the whole population of people experiencing BN. Therefore quantitative studies tend to study a sample of the group or population which may represent the larger population.

The study recruited its patents by a mixture of media advert and direct referral to the Department ofPsychologyat the University of Hamburg. All underwent telephone screening, by a clinician not involved in the study (to assess inclusion criteria) and a semi structured psychiatric interview, and physical examination As selecting people for therapy trials is open to sampling errors, in that some people selected, may be ‘ exceptional’ or ‘ different’ from those of the larger ‘ Bulimic’ population. The method attempts minimize these possibilities, by careful inclusion/exclusion criteria and the random distribution of the participants into one of the three treatment conditions[1] The size of sample for the Jacobi study is more important than the qualitative study, to gain meaningful results.

Data Collection Methods and Data analysis

The Jeppson et al study (2003) study, collected its data from interviewing eight participants who agreed to take part in the study, by using a semi-structured interview format, lasting one hour. The semi-structured format differs from a more limited ‘ formalized’ set of questions, by nature of its flexibility. It allows for new questions to be added in response to what the interviewee says, inviting and generating further information, which the study utilised. The study used a ‘ framework’ and ‘ interview guide’, which is a collection or set of topics to guide the interview process (Lindlof &Taylor, 2002). The paper, makes reference to this, but does not provide specific details of what constituted the ‘ guide.

To enhance validity, ‘ member checks’ were conducted post interview, all participants received a 20 minute phone call, where findings were ‘ shared’; allowing corrective feedback on accuracy. Transcripts were also read by three mental health professionals

The study reported that ‘ redundancy’ or theoretical saturation occurred between interview 5-7. Grounded theory employs this framework, so sampling frame may be modified as a result of a process deemed, ‘ constant comparison analysis’ i. e, that data is determined by its recurrence and re-evaluation to assist with the ‘ theorizing process’ of the area under investigation. The type of process involved in the purposeful sampling used in Jeppson’s study, accepts that it is impossible to identify ahead of time, all categories of emergent data that the researcher needs to review. So its is purposely designed to pursue data collection categories to the point of concept or ‘ data saturation’, which occurs when the data is stable and the pattern(s) unlikely to change (Locke, 2001). They found this occurred during the interviewers 5-7, and added 1 further interview according to protocol. (Lincoln & Guba, 1985)

Once Jeppson collected the data, key points were assigned (or marked) to it, referred to as ‘ codes’. Concepts and categories were then then worked out. The categories that emerged are seen within this methodology as the basis of theory generation for the study. This process as been referred to by Glaser and Strauss (1967) as the reverse emergence ‘ hypothesis’. Thus, it was considered that to gain information on the participants binge and purging behaviours it was best to ask them about their experiences so as generate theory from their responses, rather than to come to the study with a pre-determined hypothesis or theory which needed testing.

As the researcher is part of the ‘ research process’ and data collection, ‘ reflexivity’, is a feature of the data collection process. This can be divided into ‘ personal reflexivity’ and ‘ epistemological reflexivity’. ‘ Personal reflexivity’ involves the researcher being aware and examining their own values, experiences etc. and how this may have shaped the research. It also involves thinking about how the research may have affected and possibly changed people and researchers. ‘ Epistemological reflexivity’ as such, asks how the research question is defined and conducted and how it may of limited what was foundby asking searching questions about the data and its findings (Willig, C. 2001). Jeppson notes, that he was ‘ aware’ of himself and the potential influence on the study.

Jacobi, collected its data by using a battery of validated Psychological Measures to assess outcomes from the three intervention conditions. They were administered at pre and post treatment, and at 3 months and one year follow up. Descriptive statistics[2] were given using the mean[3] and standard deviation[4] of patients with regard to age, marital status, andeducation. Frequency of binging and vomiting, onset of these behaviours , lowest BMI etc., no table provided.

Data analysis was conducted by using a number of inferential statistical tests, to ‘ measure’ and quantify a host of possible influences on the outcome results obtained form the measures of each condition.

Individual one-way analyses of variance ( ANOVAs) were used to compare data, i. e., demographics, clinical variables: frequency of binging etc pre and post. The ANOVA technique involves measuring the variance of the group means and comparing that to the variance predicted if all groups were randomly sampled from the same population i. e ANOVA, tests hypotheses about the mean(s) of a dependent variable(s) e. g frequency of binge eating across the different treatment groups, in relation to the independent variables i. e. group condition

To analyze the differential effects of the 3 treatments is was originally proposed to use uni and multivariate analyses of covariance[5], e. g . to study the covariance of say mood state with frequency of purging across the treatment conditions, to see if the amount of change between two or more variable maps changes in another variable, an estimate of correlation.[6]

Sample size, through attrition was assessed as to small to run theses tests, at one year follow up. The ANOVAs and MANOVA tests used in this study produce the F-statistic[7], which is used to calculate the P-value,[8] and significance. The choice of tests was appropriate[9], given 3 treatment conditions, and the comparative nature of the research question/hypotheses. A intention to treat analysis, conducted for drop-outs was in line with the research protocol for this study design.

Quality of studies

The two papers have approached the subject of BN from different research designs. It is important when considering research papers to assess their quality, inrespectof their chosen methods.

Jeppson’s paper, in a my opinion is a well designed study. The aim is suited to qualitative methodology. Assessing the quality of the study as proved challenging as its requires a different set of criteria, from quantitative research (table 1),

And Patton (2002), outlines a series of considerations to establish rigor for qualitative research (table 2 ). The use of one interviewer, sample containing severe bulimics (Anorexia?), may have influenced the themes and results Also some ‘ reporting issues‘ concerning method etc,, can all factors effecting quality.

Table1 . Critical evaluation of qualitative study

Table 2. Criteria to establish rigor in qualitative studies

Criteria to establish rigor in qualitative methods. Patton (2002) p. 552-558

Study: Jeppson et al (2003)

Binge and Purge Processes in Bulimia Nervosa: A Qualitative Study

My interpretation

(a)Persistentobservation(b) Peer debriefing

(c)Progressive subjectivity

(d) Member checks

(e)Triangulation

(f) Transferability

(g) Dependability

(h)Authenticity and fairness

(i) confirmabilityThemes became redundant before last interview -observationStudy does not mention that this procedure occurred.

Unclear in paper- could be implicit in methodology, not mentioned overtly

Yes conducted

Yes

Makes case in conclusion for clinical enrichment and therapeutic alliance

Process was of medium/high quality – audit conducted

Study was conducted and reported with ‘ fairness’ to process

Researcher is ‘ aware’ of potential influence – audit conducted

The Jacobi research aims were entirely suited to the quantitative randomised trial it adopted to look at the effect of the three treatment interventions and there effects in the short and long term . It, however, suffers from a number of serious flaws, undermining its validity and effects when assessed to by RCT quality criteria (Greenhalgh T., 2006).

Firstly, the study fails to provide a power calculation. For to detect differences between the three intervention groups using randomisation, it is considered that the study should be ‘ powered’ i. e contain enough patients to determine statistical accuracy. Normally 80% is considered the minimum standard of accuracy for clinical trails. (Ellis, Paul D, 2010), and leaves it open to type 2 errors.[10]

This was not assisted by the attrition rate, especially at follow up. It contributed to the study being significantly ‘ underpowered’, so much so that the results at one year follow-up being of poor quality, the results could be dismissed[11] (Table 3 provides a critical evaluation)

Table 3. Critical evaluation of quantitative study.

Question

Study:

Jacobi et al. (2002)

Cognitive-behavioural, Fluoxetine and Combined Treatment for Bulimia Nervosa: Short and long term results

Does the study address the focussed question? Yes
Were the patients randomly selected from a defined population? Patients were selected/recruited byadvertisementand referral to the Psychology dept. Of Hamburg University. All had met the diagnostic criteria for BN.
Was the assignment of patients to the intervention and control group randomised? There was no control group. Study compared effects of three different treatment conditions. Paper states they were randomised, but does not say how and by what method
Were the participants and observers both blinded? Not stated though this difficult to achieve within psychotherapy trials
Aside from the intervention were the groups treated equally? Yes
Was a follow up completedYes- at I year
Did the study have adequate power to see an effect if there was one? No, there was no power calculation provided though this is common in psychotherapy intervention trials/studies. Very high attrition rate – at follow up.
Were all the patients who entered the trial properly accounted for? Full disclosure is given about the dropouts, calculations were made that suggest that the results at follow up lack validity
Were the results of the trial presented in a manner that allowed for full examination? No: adequate data was presented on tests MANOVA/ANOVA. Ci’s not reported

Secondly, the randomization procedure which gives the randomized controlled trial its strength. The investigators did not provide details of the randomization methods its used for the study[12], although this may be attributable to poor reporting, its is a significant emission (Jadad, 1998). The paper whilst providing a range of data, omits to report confidence intervals (CI)[13] for the three conditions and its results, and their were no proposed effect sizes included the aims/hypotheses.

Discussion

Both studies addressed their research using different methods. Both were appropriate for the investigation, illustrating that methodology needs to be tailored to the aim/hypotheses of the question under investigation.

The Jeppson study was well designed, and managed to explore its research aim sufficiently, so some its findings had generalisability in treatment settings, and would benefit the CBT practitioner working with BN and aid ‘ therapeutic alliance’ by taking into account its findings. By increasing understanding of how the processes have functions for the individual, well beyond the ‘ behaviour’ itself.

Jacobi’s study, although ambitious with a good design in principle, was seriously flawed. The under-powering of the study, the absence of good reporting on a number of methods and data, leave the study flawed. Its hypothesis and aims were suited to the methodology, but its sample size and attrition rate made generalizability of the results difficult. It did provide some evidence (weak), that CBT is efficacious for the treatment of BN, which continues in clinical practice.

Word Count: 3489

References

American Psychiatric Association (1994). Diagnostic and statistical manual of mental disorders. Washington DC., American Psychiatric Association.

Cartwright, N. (2007). “ Are RCTs the Gold Standard?” Biosocieties(2): 11-20.

Cresswell, J. W. (2003). Research design: Qualitative, Quantitative and Mixed Methods Approaches. Thousand Oaks, CA., Sage.

Daly, J., & Lumley, J. (2002). “ Bias in Qualiative Research.” Australian and New Zealand Journal of Public Health26(4): 299-230.

Ellis, P. D. (2010). The Essential Guide to Effect Sizes: An Introduction to Statistical Power, Meta-Analysis and the Interpretation of Research Results. Cambridge, C. U. P.

Glaser, G. B., & Strauss, A. L. (1967). The Discovery of Grounded Theory. Chicago, Aldine Publishing Company.

Goldstein, D. J., Olmsted, M., Thompson, V. L., Potvin, j. H., & Rampey, A. H. (1995). “ Long term Fluoxitine treatment of bulimia nervosa.” British Journal of Psychiatry(166): 660-666.

Greenhalgh, T. (2006). How to read a paper: the basics of evidence based medicine. London., BMJ Publishing Group.

Jacobi, C., Dahme, B. & Dittmann, R. (2002). “ Cognitive-Behavioural, Fluoxitine and Combined Treatment for Bulimia Nervosa: Short- and Long-Term Results.” European Eating Disordrs Review(10): 179-198.

Jadad, A. R. (1998). Randomised controlled trials: a user’s guide. London, BMJ Books.

Jeppson, J. E., Scott Rchards, P., Mac Hardman, R. K. & Mac Granle, H. (2003). “ Binge and Purge processes in Bulimia Nervosa: A Qualitative Investigation.” Eating Disorders(11): 115-128.

Jones, R. (2007). “ Strength of evidence in qualitative research.” Journal of Clinical Epidemiology(60): 321-323.

Lehman, A. K., & Guba, E. G. (1985). Naturalistic inquirey. London., Sage.

Lindlof, T. R., & Taylor, B. C. (2002). QualitativeCommunicationResearch Methods. Thousand Oaks, CA., Sage Publications

Locke, K. (2001). Grounded Theory in Mangement Research. London, Sage Publications.

Maxim, P. S. (1999). Quantitaitive research methods in the social sciences. oxford., Oxford University Press.

NICE. (2004). Eating disorders: Core interventions in the treatment and management of anorexia nervosa, bulimia nervosa and related eating disorders. N. I. f. H. a. C. Excellence.

Owen, F., & Jones R, (1977). Statistics. London, Polytech Publishers.

Patton, M. Q. (2002). Qualitative Research & Evauation Methods. Thousand Oaks, CA., Sage Publications.

Procter, S., Allan, T., & Lacey, A. (2010). Sampling. The Research process inNursing. Eds Gerrish, & Lacey, A. Oxford, Wiley-Blackwell.

Ragin, C. C. (1994). Contructing Social Research: TheUnity and Diversity of Method, Pine Forge Press.

Russell, G. (1979). “ Bulimia nervosa: an ominous variant of anorexia nervosa.” Psychological Medicine9(3): 429-480.

Stebbins, R. A. (2001). Exploratory Research in the Social Sciences. Thousand Oaks, CA:, Sage Publications.

W. H. O. (1992). International Statistical Classification of Diseases, 10th Revision (ICD-10). Geneva, Switzerland: .

Willig, C. (2008). Introducing Qualitative Research in Psychology. Maidenhead, Open University Press.

Wilson, G. T., & Fairburn, C. G. (1993). “ Cognitive treatments for eating disorders.” Journal of Consulting and Clinical Psychology(61): 261-269.