Using examples from published research, critically consider that way that choice of study design might influence research findings and conclusions.
A study design is a fundamental component of any piece of research, providing a structure within which to implement a suitable methodology for the collection and analysis of information and data. The choice of study design employed in criminological research will inevitably influence the findings and conclusions drawn, which may subsequently be used to shape the direction of future policies and interventions within the sphere of criminal justice. This is particularly the case in relation to domestic violence, which remains a socially and politically contested area with regards to the extent to which it merits and requires criminalisation. I therefore intend, with reference to published research, to critically examine how choice of study design may yield different findings and contributions to contemporary understandings of domestic violence.
Before considering the various types of study design, it is necessary to provide a brief account of the nature and context of domestic violence. Academics highlight that it is a distinct phenomenon from other forms of conventional violence, often characterised by a series of ongoing violent episodes as opposed to one discrete event and a typical reluctance on the part of victims to recognise and voice their victimisation (Mirrlees-Black, 1999). Criminological research in this area has assumed heightened importance in recent years due to shifts in Government policy towards criminalisation[1]. It is therefore vital that any research in this area necessitates an appreciation of the contextual circumstances within which perpetrators and victims operate.
A number of study designs are available for the purposes of criminological research, the primary distinction existing between experimental and non-experimental designs; which include randomised control trials, quasi-experiments, cross-sectional and longitudinal designs as well as ethnographic studies. The choice of design will affect the strength of both the internal[2]and external[3]validity of the findings and conclusions. Bryman notes that the study design chosen reflects a decision about the priority being given to an array of dimensions. These include articulating causal connections, generalising the findings, understanding behaviour in a specific context and having a temporal awareness of social phenomena (2008: 35). In addition, certain designs may lend themselves more easily to quantitative or qualitative data, which may impact upon a researcher’s choice of design given the fundamentally diverse characteristics of both.[4]
The true experimental design is often referred to as the ‘ gold standard’ (Bryman, 2008: 38), despite being rarely employed in criminological research (Wesiburd, 2000: 181) due to its potential to yield findings with strong internal validity, particularly if randomised control trials are employed. The experimental design involves the manipulation of the independent variable in order to elicit variation in the dependent variable, a classic example being Bushman’s (1995) investigation into the impact of violent films on those pre-disposed to aggression. In randomly assigning subjects to control and experimental groups, he was able to control for rival causal factors, resulting in findings with strong internal validity. True experiments may be either laboratory experiments[5]or field experiments[6], although it can be argued that conclusions drawn from laboratory experiments may not have the same level of generalisation due to their artificial setting. However, it is argued that it is the theoretical principles that are being generalised, not the specific characteristics of the sample, setting or measure (Anderson and Bushman, 1997: 22).
The experimental design was utilised by Sherman (1984) in investigating the specific deterrent effects of arrest on perpetrators of domestic violence, whereby three types of police response (arrest, mediation ad separation) were randomly assigned to legally eligible cases. The dependent variable, in this case recidivism, was measured over the course of 6months following police intervention using both official data and victim reports. The study’s conclusion, that ‘ swift imposition of a sanction of temporary incarceration may deter male offenders in domestic violence cases’ (Sherman, 1984: 270), is often cited in support of mandatory arrest policies. However, given that experimental designs are notorious for their weaker external validity[7], it is questionable whether Sherman’s findings lend themselves to such a sweeping generalisation. Indeed, follow up studies in other US cities revealed more mixed results, with some concluding that arrest had negative effects for victims (Garner et al. 1995).
The findings and conclusions by Sherman have been attacked on a number of grounds. Dobash contends that the experiments were ‘ designed and carried out with little knowledge of existing and theoretical positions regarding violence against women’ (2000: 254). Indeed, it can be argued that the failure to appreciate the possibility that the reason the arrest of perpetrators resulted in lesser recidivism was not due to its deterrent effects but a result of the victim failing to call the police again or to report their victimisation to interviewers due to fear of a repeated retaliation by the perpetrator that might have followed the initial arrest. This is particularly since qualitative studies have revealed the extent to which victims’ reluctance to involve the police are a direct product of their fear that an arrest will lead to an escalation of the violence (Felson et al. 2002). In addition, Fagan highlights that the short follow-up period of 6months failed to take into account the ‘ episodic and cyclical patterns of family violence’ (1989: 382). It therefore appears that Sherman’s pre-occupation with experimental form, in this case a randomised control experiment, precluded a proper consideration of the nature and contextual characteristics of domestic violence.
A variation of the true experimental design is the quasi-experiment, commonly employed in the evaluation of criminal justice interventions due to the ethical difficulties associated with random assignment. Such experiments have been used to research the effect of rape law reforms on the number of rapes reported to the police.[8]However, due to the absence of random assignment, such a design may not yield findings that are considered as strong in terms of their internal validity due to the lack of control over intervening causal variables.
The ‘ Violent Men Study’ (Dobash, 2000) employed a non-equivalent control group design to evaluate the impact of different criminal justice sanctions on domestic violence, involving the ‘ matching’ of subjects within the different groups. The qualitative data revealed much higher rates of re-offending than the quantitative arrest records portrayed, leading the authors to remark that these stark differences ‘ cast grave doubt on the veracity of criminal justice statistics as indicators of the levels of violence or of the effectiveness of interventions’ (Dobash, 2000: 267). In addition, in contrast to the experimental study of arrest that merely speculated the reasons for reduced levels of recidivism following arrest (Sherman, 1984), the ‘ Violent Men Study’ was able to contribute to an understanding of both how and why certain interventions could lead to a cessation of violence (Dobash, 2000). Such findings were arguably facilitated by the fact that the study design was more attuned to the context and dynamics of domestic violence, such as the fact that it was longitudinal in nature, tracing victims’ experiences and perpetrators’ criminal involvement over a period of 12months at regular intervals and drawing a comparison between the two sets of data[9]. The advantages associated with this design have led to a replication of it in subsequent research.[10]
An example of a non-experimental study design is the cross-sectional, involving the collection of information and data relating to a designated phenomenon at a particular instance in time. A classic example is the investigation of the effects of visible public and social order on evolving crime rates in Chicago (Sampson and Raudenbush, 2001). However, the problems associated with such a design relate to both the internal validity and the ambiguity of the direction of causality, which may therefore significantly influence any conclusions drawn, as was the case in the Chicago study which could not establish whether the high crime rates resulted before or after a variation in social control (Bachman and Schutt, 2007: 154).
A study by Hoyle and Sanders (2000) of victims within Thames Valley police force areas utilised a cross-sectional non-experimental design to investigate victims’ perceptions of the value of various criminal justice interventions. The non-experimental dimension facilitated a more in-depth exploration of victims’ experiences and beliefs and a deeper contextual understanding of the nature and effects of domestic violence, with the authors highlighting that ‘ the more powerful the controlling behaviour of their partners, the less likely it is women will seek to involve the criminal justice system’ (2000: 21).
However, a significant drawback of the cross-sectional design is that it results in findings that inevitably fail to capture the recurring and ongoing nature of domestic violence. By contrast, a longitudinal design, involves the collection of information and data at several points in time, which can also resolve any ambiguity concerning the direction of causal influence. Thus, academics assert that ‘ the value of longitudinal data is so great that every effort should be made to develop longitudinal research designs when…appropriate’ (Bachman and Schutt, 2007: 155).
A study by Sullivan and Bybee (2004) interviewed 278 women, who has just left a domestic violence shelter, at various intervals for three years. By collecting and analysing data over a substantial period of time, the study was able to provide a greater understanding of the unique ‘ trajectory’ of domestic violence. However, the problem of attrition is particularly salient for longitudinal designs, with Bachman and Schutt highlighting that those in greatest need are often the most likely to drop-out (2007: 159). This has obvious implications for the validity of findings, since a drop-out of the most domestically victimised subjects could mask the extent and seriousness of domestic violence.
An interesting study design that is rarely employed, but could nevertheless provide a valuable insight into the phenomenon of domestic violence, relates to ethnography. The purpose of which is to look more in-depth at the mechanisms underlying a social process and to gain an insight into the interpretations and perceptions of actors (Crow and Semmens, 2008: 37). Such an approach was taken by Websdale, who provided a critical ethnography of how rural patriarchy shapes the perpetration and policing of domestic violence (Adelman, 2004: 135). Although such findings and conclusions cannot obviously be generalised, ethnographic designs serve to reveal important details about the experiences of particular sub-groups of the population, which may not be adequately reflected in the findings of more conventional study designs.
In conclusion, it is apparent that choice of study design in criminological research can have a marked influence on the findings and conclusions drawn. In relation to domestic violence, whilst it appears that true experiments yield results with the strongest internal validity, they nevertheless fail to account for the dynamic processes operating which trigger the variation in the dependent variable. The ‘ context-specific’ approach taken by other designs may have the potential to fill this gap by affording a greater emphasis on process as opposed to outcome. In addition, longitudinal designs are vital in capturing the intrinsic ‘ trajectory’ nature of domestic violence which is often ignored by cross-sectional designs. However, it is apparent that a deeper and proper understanding of domestic violence, in order to implement effective policies, necessitates the exploitation of the various strengths of both experimental and non-experimental study designs. To conclude otherwise would be to ignore the inherent nature of domestic violence and to relegate it to the status of any ordinary violent crime, from which it is clearly distinct.
Word Count: 1992
