1. No category

Tubex, H. (2013). Pitfalls of comparative (penological) research and how to overcome them. In K. Beyens,
J. Christiaens, B. Claes, S. De Ridder, H. Tournel & H. Tubex (Eds.), The Pains of Doing Criminological
Research. Brussel: VUBPress.
12 Pitfalls of comparative (penological)
research and how to overcome them
Hilde Tubex
Abstract
Over the last two decades, the widespread growth of prison populations has led to
various analyses examining possible explanations for differences in imprisonment rates.
In this contribution, we discuss the learning from the wealth of comparative penologic al
research available nowadays. Based on the literature and our own experiences with
comparative research, we discuss common pitfalls and try to offer advice on how to
address them. We illustrate the more methodological and theoretical discussion with
leading works in this area, demonstrating the major findings over the last decades. We
conclude that, regardless the various pains that one still has to face in doing comparative
research, significant progress has been made in contributing to answer one of the most
compelling penological questions, as how to best deal with crime in a society.
Introduction
One of the most compelling penological questions is how to best deal with crime in a society.
One way of setting out to answer that question is to examine practice and experience in a
variety of jurisdictions, analysing punishment and exploring the consequences of differing
levels of punitiveness. This research forms part of comparative criminology, an area that is
currently experiencing a revival demonstrated in numerous publications. However, the
current interest in comparative criminology might lead us to forget that it is actually a
relatively young discipline. While Downes (2011) rightly points out that criminology was
born comparative, with historical contributions from such people as John Howard and Alexis
de Tocqueville, this comparative focus seemed to have faded until the late twentieth century.
Downes sees several reasons for this abeyance, amongst which are the impact of positivism
Tubex, H. (2013). Pitfalls of comparative (penological) research and how to overcome them. In K. Beyens,
J. Christiaens, B. Claes, S. De Ridder, H. Tournel & H. Tubex (Eds.), The Pains of Doing Criminological
Research. Brussel: VUBPress.
which makes comparison meaningless as the reasons for criminal behaviour are to be found
in the biological or psychological state of the individual, and the methodological difficulties
that come with finding reliable and comparable data. The latter issue will be discussed
extensively later in this contribution.
Another important reason is the fact, which may be hard to imagine these days, that
crime and justice were not always ‘hot’ topics. For, after World War II and throughout the
sixties, crime rates were rather stable, the welfare state generous to everyone, and the
reigning criminological climate positive and hopeful, characterised by a strong belief in the
rehabilitative abilities of criminal justice interventions.
Downes and Hansen (2006) describe in their work how the optimistic belief in the
penal welfare state of the fifties and sixties dramatically declined in the following decades,
despite the growing investment in welfare provisions. For, after all, welfare provision was not
successful in dealing with crime, as crime rates were going up. It was argued that welfare
actually makes things worse, promoting the dependency of the ones who are subject to it,
resulting in the ‘survival of the unfittest’. Furthermore, welfare intervention was increasingly
seen as justifying and exacerbating discrimination and punishment.
The global economic crisis in the 1970s, resulting in an overall recession and causing
the growth of unemployment, was another attack on the belief in ‘penal welfarism’. This in
combination with rising crime rates and the growing risk of victimisation pushed crime and
punishment onto the political agenda (Garland, 2001; Lacey, 2008). This trend was
sometimes strengthened by other developments such as the rising success of extreme right
parties, as it was the case in Belgium (Snacken, 2007). As a result, and from the eighties
onwards, most industrialised countries experienced an increase in their imprisonment rate.
This evolution also attracted the attention of criminologists, who started looking beyond the
country borders to see what was happening abroad. Downes (2007) describes how his
attention was first drawn to the idea that there is an alternative to penal expansion in times of
increasing crime rates by Hans Tulkens, the then head of the prison department in the
Netherlands. This formed the basis of Downes’ pioneering comparative study on the
difference in imprisonment rates in the United Kingdom and the Netherlands (Downes,
1988).
Indeed, and as Pratt (2011) demonstrates, most Western countries experienced the
same post-war development, however, the ways this impacted on the welfare model and
penal policy were not the same everywhere. He illustrates the growing divergence between
Tubex, H. (2013). Pitfalls of comparative (penological) research and how to overcome them. In K. Beyens,
J. Christiaens, B. Claes, S. De Ridder, H. Tournel & H. Tubex (Eds.), The Pains of Doing Criminological
Research. Brussel: VUBPress.
the penal situation in Nordic countries (Finland, Norway and Sweden) and Anglo-Saxon
countries (England and New Zealand). The Anglo-Saxon countries shifted their welfare
investments to more coercive forms of governance: crime and insecurity became public
concerns and were picked up by the media, politicians became increasingly influenced by
‘penal populism’, resulting in penal excess. By contrast, welfare investments were less
challenged in the Nordic countries, levels of trust remained high and allowed for ‘a politics of
acceptance and inclusion’. It is this sustained belief in the potential of the welfare state that
forms one of the pillars of what Pratt describes as ‘Scandinavian penal exceptionalism’ (cf.
Infra).
David Garland’s ‘Culture of Control’ (2001) marked a milestone in the world of
comparative criminology. In this third book of his trilogy1 he describes the penal evolution in
the United States of America and the United Kingdom over the past 30 years. The content of
his book has been widely discussed and debated in several reviews2 and subsequent work
(Zedner, 2002; Tonry, 2007; Snacken, 2007; Lacey, 2008), but it has remained the
benchmark work of explanation of the recent evolution of prison populations.
Garland’s rather pessimistic analysis, based on merely US/UK data, has engendered a
boom of criminological analyses exploring this area. Some of these studies have been
conducted at a macro level3, looking for similar trends in several (comparable) countries,
while others focus on individual jurisdictions and differences between countries. The last set
of studies was strongly inspired by the predominance of Anglo-American publications, whose
approach gives primacy to trends in the United States of America and United Kingdom as
being most important and leading the future, and pays little attention to other countries
(Tonry, 2001). Various continental European, but also North American (including Canadian)
authors, pointed to the differences between Europe and the US/UK model. They also
emphasise that there are significant differences in the penal situation of countries within
Europe, and that even in the Anglo-Saxon family not all countries follow the US/UK path to
the same extent. All the above prompted several studies analysing the situation in individual
our neighbouring countries, identifying ‘various Cultures of Control’ (Nelken, 2009).
1
After Punishment and Welfare (1985) and Punishment and Modern Society (1990).
Cf. Punishment & Society, 2002: 253-261; British Journal of Criminology 2002: 228-261; Critical Review of
International Social and Political Philosophy, Vol. 7, No. 2 Summer 2004.
3 Macro level analyses start from the presumption that global transformations in the economic and social context
have impacted on the way we punish, leading to more punitive societies.
2
Tubex, H. (2013). Pitfalls of comparative (penological) research and how to overcome them. In K. Beyens,
J. Christiaens, B. Claes, S. De Ridder, H. Tournel & H. Tubex (Eds.), The Pains of Doing Criminological
Research. Brussel: VUBPress.
In what follows, we will discuss insights from the wealth of comparative penological research
available nowadays. We start from the acknowledgement that, despite well recognised
shortcomings and well known problems inherent in generating comparable data sets (cf.
Infra), the most commonly used single proxy measure for punitiveness is rates of
imprisonment, as it is the most practical available indicator. This chapter examines possible
explanations for differences in imprisonment rates, and while doing so, it leads us through
identified pitfalls in comparative research and offers advice on how to overcome them. The
more methodological and theoretical discussion (full text) will be illustrated with leading
works in this area (in boxes), demonstrating the major findings over the last decades.
Comparing like with like
The first basic rule of comparative research is to ensure that we compare like with like,
as explained by Nelken (20104), one of the leading scholars on the methodology of
comparative research. An initial check, if we are seeking to explain differences in
imprisonment rates between jurisdictions, is whether the difference is simply commensurate
with levels of crime. This seems quite evident, but the level and sort of criminality is often
overlooked in comparative studies, as demonstrated by Nelken (2011) in his critique of the
work of Cavadino and Dignan (2006). Although crime rates are important, it has repeatedly
and convincingly been demonstrated that crime rates are not the main driver of prison
populations. According to Tonry (2005: 8) ‘Punishment and crime have little to do with each
other.’ Various studies (Young and Brown, 1993; Beckett and Western, 2001; Lappi-Seppälä
2008; Cavadino and Dignan, 2011; Nelken, 2011) have brought together a body of evidence
showing that crime rates are actually going down, while prison rates are going up. It would be
far too easy to claim that this actually demonstrates that ‘prisons work’, as Nelken (2011: 12)
states “almost all criminologists agree that – even in the United States of America – only a
small part of the recent reduction in crime can be attributed to the increasing use of prison”.
According to Downes (2011), who illustrates this with the Dutch example, rising crime rates
might have an initial effect, as they can be the trigger for putting mechanisms in place that
lead to greater punitiveness which, once established, persists without further reference to
changes in crime rates.
What he explains as ‘the effort to hold constant in our comparison those factors... which would otherwise take
away the point of a given comparison’ (Nelken, 2010: 35).
4
Tubex, H. (2013). Pitfalls of comparative (penological) research and how to overcome them. In K. Beyens,
J. Christiaens, B. Claes, S. De Ridder, H. Tournel & H. Tubex (Eds.), The Pains of Doing Criminological
Research. Brussel: VUBPress.
We can conclude, therefore, that if we want to compare and explain the penal situation in
different countries, we have to make sure that the basic characteristics of these subjects, such
as crime rates and profiles, are comparable.
What to measure?
When it comes to comparing prison populations, the most frequently used measure is the
imprisonment rate: being the number of prisoners out of 100.000 inhabitants. But even using
this widely accepted definition, we are confronted with a set of difficulties. Since, while most
countries calculate the number of prisoners divided by all inhabitants, in Australia and
Canada for example, only adult inhabitants are counted. A further complication is that not all
Australian jurisdictions use the same age limit for the definition of adults5. Differences in
counting rules can have a significant impact. Van Dijk (2011) – well experienced in
comparative data as he initiated and supervised the International Crime Victims Surveys –
raised the point that the striking developments in the Netherlands, as represented in the
statistics of the Council of Europe, can partly be explained by differences in the way the
imprisonment rate is calculated. In the Netherlands, prison statistics include illegal
immigrants that have not committed any offences, and juveniles in institutions following civil
proceedings. If you deduct them, as most other countries do, the imprisonment rate in 2007
reduces from 113 to 72, which is, in comparison to other European countries, quite moderate.
In comparative penology, the key issue is to explain differences in penal policies, and
imprisonment rates are considered as an index of what is mainly referred to as punitiveness
(Tonry, 2007). This further complicates what we have to consider. A first point of discussion
is what imprisonment rates should best be related to. Should we consider imprisonment rates
in relation to overall crime statistics, or only to the most serious offences, like homicide,
which have a high clearance rate? How we resolve that question, will have a profound impact
on our understanding of punitiveness, as was demonstrated by Nelken (2009).
Furthermore, how well do imprisonment rates measure the intensity of prison use, as
the size of the prison population per 100.000 inhabitants (stock) is determined by both the
number of people entering the prison (flow) and the length of time they stay there. Some
5
In all states and territories the definition is 18 years and over, except for Queensland, where it is 17 years and
over, ABS 4517.0.
Tubex, H. (2013). Pitfalls of comparative (penological) research and how to overcome them. In K. Beyens,
J. Christiaens, B. Claes, S. De Ridder, H. Tournel & H. Tubex (Eds.), The Pains of Doing Criminological
Research. Brussel: VUBPress.
countries such as Switzerland and Scotland imprison a lot of people, but, on average, for
short periods of time, while others like Spain and Portugal have fewer admissions, but longer
sentences (Nelken, 2009). Which is more punitive, a country that imprisons many people for
short periods or one that imprisons fewer but for longer? Kommer (1994, 2004) demonstrates
in a European context that the ranking of the countries in the European Sourcebook would
look very different according to the indicator that is used to measure punitiveness.
In addition, we could question how well imprisonment rates measure sentence
severity and the intensity of social control. There are several possible interventions other than
imprisonment, such as monetary penalties, community based orders, home detention,
electronic monitoring. Further, some countries – like Norway and the Netherlands during the
eighties - stick to the principle of one person to each cell, and if no places are available and
no new prisons are build, the implementation of the sentence is postponed and people end up
on a waiting list, which can also bias prison numbers (Green, 2007; Downes and van
Swaaningen, 2007). Finally, it can be argued that prisons are not the only forms of custodial
exclusion, there are also mental health institutions, social care institutions, psychiatric
hospitals, asylums, etc.
Snacken (2010; 2012), moreover, argues that the overall picture of punitiveness is
much more diverse, including components as the retention / abolition of the death penalty,
prisoners’ rights, human rights and such initiatives as restorative and therapeutic justice.
Finally, imprisonment rates are only the outcome of a whole process, starting from the
legislative framework, public practice in reporting potentially criminal behaviour, police
activity, prosecution practices, sentencing and parole, each one with their own agendas and
impacts (Snacken, Beyens and Tubex, 1995). As Lacey (2008) states, the operation of the
criminal justice system itself, and its relationship with other institutions, plays an autonomous
role.
The lesson learned from the literature is that imprisonment rates definitely do not
paint the full picture and that an understanding of punitiveness is improved by
complementing them with other data such as information on the level and sort of offences,
the number of admissions, remand prisoners, the length of the sentence, and alternatives to
imprisonment. Even then, Nelken (2010) points out that we have to be critical of all official
sources, as they might be influenced by budget considerations and political influence.
Weatherburn (2011) illustrates this issue by comparing police recorded assault rates in
Victoria and New South Wales in 2009, arguing that the difference can be explained, to a
Tubex, H. (2013). Pitfalls of comparative (penological) research and how to overcome them. In K. Beyens,
J. Christiaens, B. Claes, S. De Ridder, H. Tournel & H. Tubex (Eds.), The Pains of Doing Criminological
Research. Brussel: VUBPress.
large extent, by different recording practices. Weatherburn’s analysis has been supported by
research conducted by the Australian Bureau of Statistics (2005). For these reasons, Van Dijk
(2010) argues that victimisation surveys should be the primary source when measuring crime.
But imprisonment rates also have positive value for researchers, such as their
availability and consistency, the existence of long-term series and the regular updating of
data. Further, it has been demonstrated in numerous studies that imprisonment rates correlate
quite well with other indicators of punitiveness, as – in the words of Van Swaaningen (2013)
- punitiveness is a socio-cultural construct with both quantitative and qualitative components.
It is for these reasons that they are, subject to the limitations outlined above, the most
frequently used proxy measure for punitiveness. Possible sources are the SPACE6 statistics,
the European Sourcebook7, the World Prison Population List (now at its 8th edition)8 and the
World Prison Brief9, both the latter provided by the International Centre for Prison Studies.
We can conclude that, in the absence of reliable alternatives, the use of imprisonment
rates remains a valuable tool for assessing punitiveness. In doing this, however, we should
bear in mind that it is an imperfect measure, which should always be accompanied by a
discussion of the problems inherent in its composition and the limitations that should be
placed on the generality of the conclusions that can be drawn from what it suggests.
What to compare?
After the discussion of the best measures to use to evaluate the penal situation, the next
question that arises is what is the best unit to select for comparison? Traditionally, countries
are compared, as they are mainly the level on which most data are collected and available.
However, from federations such as the US, Canada and Australia, we know that there are
significant differences within a country. In Australia, imprisonment rates are historically
lower in Victoria, fluctuating around a 100 per 100.000 adults, while in the Northern
Territory the rate is almost 800. The same is true for Canada, as the imprisonment rates in
their provinces vary from 104 to 212, traditionally being higher in the territories that have a
high Aboriginal population (Webster and Doob, 2011). Similar inter-state differences in the
Statistiques Penale Annuels du Conseil de l’Europe, initially collected in French, online available from 1997
onwards . http://www.coe.int/t/dghl/standardsetting/prisons/space_i_en.asp
7 http://www.europeansourcebook.org/sourcebook_start.htm
8 http://www.prisonstudies.org/publications/list/40-world-prison-population-list-8th-edition.html
9 http://www.prisonstudies.org/info/worldbrief/
6
Tubex, H. (2013). Pitfalls of comparative (penological) research and how to overcome them. In K. Beyens,
J. Christiaens, B. Claes, S. De Ridder, H. Tournel & H. Tubex (Eds.), The Pains of Doing Criminological
Research. Brussel: VUBPress.
US have been studied by Beckett and Western (2001), according to whom they are related to
the commitment to and investment in welfare.
Beckett and Western (2001) analysed the differences in imprisonment rates within the US.
They identify what they call ‘inclusionary’ and ‘exclusionary’ states. In ‘inclusionary’ states
criminal behaviour is perceived as a social problem to which the appropriate response is
investment in welfare. These states tend to have lower imprisonment rates than the
‘exclusionary’ states, which see crime as an individual responsibility and invest more in
punishment. Downes and Hansen (2006) tested this hypothesis on a larger sample of 18
OECD (Organisation for Economic Co-operation and Development) countries and find a
statistically significant negative relationship between the investment in welfare (measured in
percentages of Gross Domestic Product spent on welfare) and imprisonment rates. That is,
consistent with the American study, they found that increasing levels of national investment in
social welfare are related to relatively low prison populations. Both studies find this
relationship getting stronger over time.
But also in smaller counties such as Belgium and the United Kingdom penal trends are not
captured by nation borders. As Snacken (2007) demonstrates, the development of the victimoriented movement happened quite differently in the French and the Flemish part of Belgium.
For the United Kingdom, of the three major jurisdictions, two (England and Wales and
Scotland) have imprisonment rates of about 150/100.000 while the other (Northern Ireland)
has a rate of 99/100.000 (having risen from 54/100.000 in 2001), which also points to
differences in the penal policy within country borders.
The selection of national units is criticised by the so-called ‘globalists’ who start from
a broader perspective of more global and cross national developments, even claiming that the
days of comparative criminology are over. This globalised approach has been criticised by
several comparative criminologists. According to Nelken (2010), globalisation processes do
have their impact, but it did not all start with globalisation, and countries have always had an
influence on each other. For example, Finland decided after the fifties that there was no
reason why their imprisonment rate should be so much higher than their Scandinavian
neighbours and that they would implement a deliberate policy to bring their prison population
down (Lappi-Seppälä, 2000). This policy was effective and it makes Finland a prime example
Tubex, H. (2013). Pitfalls of comparative (penological) research and how to overcome them. In K. Beyens,
J. Christiaens, B. Claes, S. De Ridder, H. Tournel & H. Tubex (Eds.), The Pains of Doing Criminological
Research. Brussel: VUBPress.
of how the size of prison populations is not a matter of ‘fate’ but rather of ‘policy’ (Snacken,
Beyens and Tubex, 1995). Nation states still have their individuality, according to Nelken
(2010), and there might be even a start of de-globalisation.
Downes (2011) confirms that even the strong support for global trends still leaves
some exceptions to the rule, such as Canada and the Netherlands. The first would generally
be considered to be within the neo-liberal community – which is mainly characterised by
large prison populations – but the imprisonment rate in Canada has been moderate and stable
over the last 50 years (Doob and Webster, 2006; Webster and Doob, 2011). Also, in the
Netherlands, the imprisonment rates quintupled in a period that they could be described as a
strong welfare state (Downes, 2007; Downes and van Swaaningen, 2007). Therefore, there
will always be a need for a more detailed study of the local and the national. Downes (2011)
argues that there is not one positivistic formula to explain punitiveness, maybe the closest we
can get is the identification of risk and protective factors as developed by Tonry below.
Tonry has from the beginning argued that, regardless of globalisation, explanations for penal
policies remain remarkably local (Tonry, 2001), and he demonstrates in various articles that
not all countries go down the US path. Further, he emphasises that even the other so called
Anglo-Saxon countries such as the United Kingdom, Canada and Australia, only ‘halfheartedly’ follow the American example (Tonry, 2004; 2006). Throughout his articles, he
develops a set of ‘determinants of penal policies’ (2007) dividing them in risk and protective
factors. The risk of more punitive policies increases when the following characteristics are
present: conflictual political systems, elected judges and prosecutors, sensationalist
journalism, Anglo-Saxon political cultures, populist views on criminal justice policy, income
inequality, weaker social welfare systems, lower levels of trust and perceived legitimacy of
institutions. On the contrary, societies are better protected against punitiveness if they have
consensual political systems, nonpartisan judges and prosecutors, Francophonic political
cultures, and a criminal justice policy that is a matter for experts and professional
experience.
We can conclude that, while global trends have been identified, and undeniably have had an
impact across national borders, for each global model, there are counter-examples, illustrating
the need for more locally based research, explaining why this is the case and how the global
Tubex, H. (2013). Pitfalls of comparative (penological) research and how to overcome them. In K. Beyens,
J. Christiaens, B. Claes, S. De Ridder, H. Tournel & H. Tubex (Eds.), The Pains of Doing Criminological
Research. Brussel: VUBPress.
effects can be altered. We should also be cautious not to advance explanations of differences
in punitiveness viewed at a national level where more detailed data show wide intra-state
variation between more or less autonomous internal jurisdictions.
Limitations of comparative research
When comparing the penal situation in different countries, there are certain limitations that
are hard to overcome. A natural limitation is being able to access information that is only
available in foreign languages. However, with the growing domination of English, analyses
of non-English speaking countries have become more readily available. In the case of
Scandinavia we can even talk of a recent explosion of English literature (cf. Lappi-Seppälä,
2007; Lappi-Seppälä and Tonry, 2011; Pratt, 2007a; 2007b; 2011; Pratt and Eriksson, 2011;
Pratt and Eriksson, 2012; Ugelvik and Dullum, 2012). Other examples are from French
descent (De Maillard and Roché, 2004; Levy, 2007; Roché, 2007) and from Germany (Salle,
2003; Oberwittler and Höfer, 2005). The penological literature available on countries like
Spain and Portugal and on eastern and central European countries is still limited (e.g. Lévay,
2012) and we should be cautious when using translated information that unintended bias is
not introduced. Pratt (2007a; 2007b), for example, was criticised for the limitation in his
methodology in only relying on translated documentation, an issue he later remedied in
collaborating with a Swedish colleague (Pratt and Eriksson, 2011, 2012). But even then, the
cultural meaning of some words may become lost in translation (Pratt and Eriksson, 2012).
Nelken (2010) illustrates the same point with the Dutch word ‘gedogen’, which he translates
as ‘guided tolerance’, but that is different from the English meaning of ‘tolerance’.
David Nelken provides a valuable point of entry for Italy. His work on the analysis of
the Italian situation reveals that there are several cultural barriers and that, if you are not
familiar with the traditions and culture in a particular country, it might be difficult to
correctly interpret what is happening, or what is measured by international standards. He
illustrates this point in a critique of the analysis of Cavadino and Dignan (2006b) described
below.
Cavadino and Dignan (2006a; 2006b) select for their study twelve countries and divide them
into four groups based on Esping-Andersen’s typology of political economies., They
Tubex, H. (2013). Pitfalls of comparative (penological) research and how to overcome them. In K. Beyens,
J. Christiaens, B. Claes, S. De Ridder, H. Tournel & H. Tubex (Eds.), The Pains of Doing Criminological
Research. Brussel: VUBPress.
differentiate neo-liberal, conservative corporatist, and social democratic corporatist political
economies, and, to accommodate Japan in their analysis, add an oriental corporatist
political economy. They find that it is mainly the neo-liberal political economies in AngloSaxon countries that have high imprisonment rates. Neo-liberalism is characterised by the
free market principle, a minimal welfare state, material inequality, individualism and limited
social rights, resulting in social exclusion and marginalisation of those individuals who don’t
do well in the free market. In a later article (2011) the authors explore reasons for the
relationship between neo-liberalism and punitiveness. They see as the main explanations the
political culture and the interaction with political and state institutions that derive from the
different political economies, as well as the impact of public opinion and the media within
these political economies, which shape punishment.
Starting from the Italian example, Nelken (2009) points to alternative reasons explaining why
the imprisonment rate is relatively low, and that are not related to an assumed political
economy. The low imprisonment rate in Italy in 2008 was strongly related to an indulto
(collective pardon), releasing over a third of the prison inmates. To explain imprisonment
rates, some specific characteristics of the Italian criminal justice system need to be taken into
account, as there is the highly selective process of attrition throughout the criminal justice
system, which ensures that many prosecutions are started, but very few arrive at conclusion.
Therefore, Nelken supports the approach taken by Lacey (2008). She explains
differences in punitiveness, by building on the analysis of Cavadino and Dignan (2006b), and
relating the political economy of a country to their cultural and institutional structures.
Lacey (2008) in her work contrasts liberal market economies – which she claims are more
exclusionary - with more inclusive co-ordinated market economies. She explains this
describing the political structures of both models in combination with their electoral
arrangements and how they relate to the bureaucracy and judiciary. It is mainly liberal
market economies, like the United Kingdom, with a bi-partisan first-past-the-post electoral
system, that are more vulnerable to a politisation of crime and punishment issues. For, the
conflict model feeds more into penal populism, as the polarisation between parties triggers
the attention and impact of media reporting, while they rely less on informed expert opinion
and rational decision-making. Further, the economic structures in both models are different:
Tubex, H. (2013). Pitfalls of comparative (penological) research and how to overcome them. In K. Beyens,
J. Christiaens, B. Claes, S. De Ridder, H. Tournel & H. Tubex (Eds.), The Pains of Doing Criminological
Research. Brussel: VUBPress.
liberal market economies require a more flexible and active work force, gain less from
investment in weaker groups, with a higher risk of exclusion. Starting from a more positive
view than Garland (2001), she identifies ways and strategies how the co-ordinated market
economies can escape the threat of mass imprisonment and a ‘prisoners’ dilemma’.
The examples above demonstrate that for the purpose of comparative research, we should be
cautious in drawing conclusions about countries with which we are not familiar, and should
take steps to inform our analysis with good local understanding of the structures and
particularities of the countries to which we refer.
The lack of reliable sources
At this stage we probably arrive at the most frustrating point of doing comparative research:
finding comparable and reliable data. In the early days, when we started doing comparative
research (Tubex and Snacken, 1994, Tubex and Snacken; 1995; Snacken, Beyens and Tubex,
1995; Tubex and Snacken, 1998), data were mainly collected on a national, or even local
level, and were a mosaic of incompatible and inconsistent definitions and categorisations,
fragmented and punctured data sets with wide gaps, different sources constructing creative
variations on the same facts and numbers, etc. Comparative research was hard work, seeking,
sifting and sewing all possible sources together in a brave attempt to come to a coherent and
reliable reflection of what was going on. More than once we regretted the day we went down
the path of comparative research and wondered if it wouldn’t be easier to go there and count
the offenders ourselves.
Luckily enough, the world has moved on, and with the overall availability of websites
and internet, the access to comparative data has improved a lot. However, the pains of
comparative research are not over yet! In a recent check of the data available on (categories
of) prisoners in Belgium and its neighbouring countries, we still had to conclude that there is
a significant difference between the quality and the accessibility of government websites.
Unfortunately Belgium is not exactly a shining example. In previous publications we
repeatedly identified problems of availability and consistency of penological data in our
country (Tubex, 2000; Tubex, 2001; Beyens and Tubex, 2002; Tubex and Strypstein, 2004),
and according to more recent publications (Maes, 2010), this situation hasn’t changed.
Tubex, H. (2013). Pitfalls of comparative (penological) research and how to overcome them. In K. Beyens,
J. Christiaens, B. Claes, S. De Ridder, H. Tournel & H. Tubex (Eds.), The Pains of Doing Criminological
Research. Brussel: VUBPress.
Checking the internet, we conclude that what is currently available on the Belgian prison
population is indeed more attractively presented, but it is barely usable for scientific
purposes10.
On the international, European and global level there are various sources measuring
all possible components of life that can be used for comparative research. One example of
how these broad parameters are used to conduct macro level comparative research is the
study of Lappi-Seppälä described below.
In his study, Lappi-Seppälä (2008) explores seven factors that are possibly related to
imprisonment rates in a basic sample of 25 industrialised countries, and a larger sample of
99 countries ranked according to the UN Human Poverty Index. His quantitative analysis
builds on sources such as the Council of Europe Sourcebooks, the International Crime
Victims Survey, the EU International Crime Survey, the European Social Survey, the World
Values Survey, the Luxembourg Income study, the OECD, the European System of Social
Indicators and Eurostat, the UN Human Development Reports and the International
Monetary Fund data. He identifies the most important factors as being: public sentiment
(confidence in fellow citizens and government), strong welfare states, political structures and
legal cultures. He asserts that prison populations can be explained neither by focussing on
individual factors, nor by too general and complex theories. Explanations can be developed,
he argues, through analysis of detailed country specific characteristics such as those he
isolates.
The conclusion of Lappi-Seppälä that trust in the criminal justice system is correlating
with levels of punitiveness is challenged by Karstedt (2011). In her work, she tests the value
of his hypothesis for a broad cross national sample, including regions outside of Europe. Her
work is interesting as she combines a quantitative with a qualitative approach of punitiveness.
In one of her articles, Karstedt (2011) analyses the impact of democratic values on penal
punishment, as measured by imprisonment rates and prison conditions. Her cross-national
10
http://justitie.belgium.be/nl/binaries/Justitie%20in%20cijfers%202010_tcm265-138438.pdf
http://justitie.belgium.be/nl/binaries/JAARVERSLAG_NL_BD_tcm265-187836.pdf
http://www.moniteur.be/epi/doc/activiteitenverslag_2010_nl.pdf
Tubex, H. (2013). Pitfalls of comparative (penological) research and how to overcome them. In K. Beyens,
J. Christiaens, B. Claes, S. De Ridder, H. Tournel & H. Tubex (Eds.), The Pains of Doing Criminological
Research. Brussel: VUBPress.
comparative study includes 67 countries, and examines the impact of the values of
individualism and egalitarianism on punishment. The parameters used are imprisonment
rates deriving from the World Prison Population List, and prison conditions as described in
the Country Reports as issued by the US State Department, both for the period 1999-2005.
Her results demonstrate that neither individualism nor egalitarianism impact on the
imprisonment rate. In contrast, they do present a strong linear relationship with prison
conditions. This is, countries where these democratic values are strong do not imprison fewer
offenders, but they are treated better and they provide for better prison conditions. This effect
remains when the sample is limited to mature democracies, as measured by the Polity Index.
While the measures used in the examples above are rather broad brushes with which to paint,
they do allow analysis on a macro level. However, macro level research has also its
limitations and has been criticised because of their failure to explain diversity in punishment
and criminal policy making and for being inadequate to explain the conditions for change
(Snacken and Verfaillie, 2009).
We can conclude that, while two decades ago, the lack of sources was the main problem,
the dizzying wealth of possible tools now available might cause the same effect. A realistic
rule of thumb is still that all sources come with their weaknesses and inaccuracies, and that
their relativeness needs to be taken into account. Improved reliability and comparability are
becoming available but sources are still far from ideal.
Quantitative vs qualitative analysis
In addition to the more quantitative approaches of measuring punitiveness discussed above,
Pratt (2008a; 2008b) makes a strong case that not only the number of prisoners is relevant to
evaluate the penal situation in a given country, but also the conditions in which prisoners are
held. For him, it is part of the ‘Scandinavian penal exceptionalism thesis’, describing the
characteristics that have saved the Nordic countries from going down the path of penal excess
as other, particularly Anglo-Saxon countries.
In his award winning study, Pratt (2008a; 2008b) describes ‘Scandinavian exceptionalism’,
with relatively stable and low imprisonment rates, enabling them to have a prison system that
Tubex, H. (2013). Pitfalls of comparative (penological) research and how to overcome them. In K. Beyens,
J. Christiaens, B. Claes, S. De Ridder, H. Tournel & H. Tubex (Eds.), The Pains of Doing Criminological
Research. Brussel: VUBPress.
is based on normalisation, with permeable walls, with well trained and sufficient prison staff,
where prisoners are offered a good living standard and attractive work opportunities, and
are included in discussions about the management of the prison. He frames this development
in a historical context, including socio-political and religious factors, a demographically
homogeneous and mainly egalitarian society, with strong reliance on education and expert
knowledge, and a sustained belief in the welfare state. Recently, this ‘penal exceptionalism’
has been explicitly questioned by Scandinavian scholars having a critical look at their prison
system (Ugelvik and Dullum, 2012). Pratt acknowledges that there are signs of a rising
punitive approach, increasing politicisation and a decline of the role of expert opinion.
However, these changes should not be overstated and in a reaction to the critique, he argues
why the ‘penal exceptionalism’ claim still holds (Pratt and Eriksson, 2012).
Another example of qualitative analysis is the work of Green (2007, 2008), comparing the
reaction towards two child on child homicides, one in England and one in Norway. The first
case attracted unprecedented media attraction, the public opinion was calling for severe
intervention and politicians were going into this debate with a strong swing to repression and
punitiveness, while the opposite happened in the Norwegian case. Through an in depth
analysis of both situations Green describes and analyses the various factors that might explain
the difference in this reaction.
In his comparative study of two murders committed by children on children, one in the United
Kingdom in 1993 and another in Norway in 1994, Green (2007, 2008) identifies several
components that shaped the divergence in the reaction of the media and the political and
public responses to both cases. The way both families of the victims dealt with what happened
was important, as it influenced the way the media got involved and the level of
newsworthiness. There are also more cultural components in the conception of childhood and
to what extent they can be held responsible for their acts at a young age. But the bulk of the
explanatory strength is, according to Green, to be found in the relationship between media
culture and the reigning political system. It is mainly a majoritarian system as in Britain that
allows for a high media profile expressed in poorly informed populist rhetoric. This, in turn,
fed the moral panic of the public, distrusting the capability of the government to deal with
crime and calling for severe sentences. In contrast, the consensus democratic model in
Tubex, H. (2013). Pitfalls of comparative (penological) research and how to overcome them. In K. Beyens,
J. Christiaens, B. Claes, S. De Ridder, H. Tournel & H. Tubex (Eds.), The Pains of Doing Criminological
Research. Brussel: VUBPress.
Norway allows for a more stable political system that is less driven by media reporting and it
keeps the trust of citizens in government intact.
In conclusion, we can state that both quantitative and qualitative approaches have their value,
but in the light of what is said before about understanding what is happening in different
countries, it is important to include qualitative elements in the comparison.
Conclusion
It would be understandable if, after reading this chapter, one is tempted to conclude that one’s
future does not lie in comparative penological research: if a discipline founded in the opening
years of the 19th Century has been unable over 200 years to agree on what it means by the
punitiveness of a criminal justice system; if the most widely used measure it employs is
recognised as deficient; if the information available on the national level is more and more
blurred by intrastate, supranational and global trends; if the understanding of those data that
do exist is so beyond the grasp of all other than those who have lived in the culture of each
country they wish to study; and if governments have not found it necessary to generate series
of data adequate to allow them to be systematically investigated, the prospects of successful
research must appear remote.
This conclusion would be understandable, but it would be short-sighted. Comparative
penological research brings up a wide spectrum of contended and problematic issues that are
intrinsic to the whole area of criminology. It is because of this discipline and because it is not
possible to pursue it without engaging in the sorts of problems that this chapter has explored,
that those contended and problematic areas have been identified. While doing so, progress
has been made. The composition and evolution of prison populations is far less of a black box
than it was two decades ago. Still, not all mechanisms involved have been revealed, and how
they all relate to each other on various levels often remains unclear, which opens challenging
avenues for further research, but certain contours of a better understanding appear on the
horizon. There is the well documented impact of welfare states, the entwined cluster of
political systems and their relationship with legal, institutional and economical structures,
their correlation with a specific sort of media reporting and the extent to which this impacts
on the public opinion and the belief in the legitimacy of government. These factors do not
Tubex, H. (2013). Pitfalls of comparative (penological) research and how to overcome them. In K. Beyens,
J. Christiaens, B. Claes, S. De Ridder, H. Tournel & H. Tubex (Eds.), The Pains of Doing Criminological
Research. Brussel: VUBPress.
only have an influence on quantitative measures such as imprisonment rates, but also on the
qualitative conditions in which prisoners are held. Throughout the work done so far, it has
been demonstrated that mass imprisonment as we observe it in the US is not an inevitable
doom scenario, and alternative ways have been outlined: for Scandinavia, for Europe, but
also for Britain and other Anglo-Saxon countries. It is through the work of these scholars that
we can conclude that comparative penological research may be complex, and it definitely
requires passion for the subject. However, it is worthwhile and it provides for invaluable
research questions and opportunities. It is able to provide insights to some of the most
compelling penological questions as how to best deal with crime in a society.
References
Australian Bureau of Statistics (2005). Differences in Recorded Crime Statistics. Canberra:
Australian Bureau of Statistics.
Australian
Bureau
of
Statistics
(2011).
Prisoners
in
Australia
4517.0,
http://www.ausstats.abs.gov.au/ausstats/subscriber.nsf/0/7B05CD44A0E2FC8ACA25
795F000DBD0F/$File/45170_2011.pdf (Accessed 23 August 2012).
Becket, K. and Western, B. (2001). Governing social marginality: Welfare, incarceration and
the transformation of state policy. Punishment & Society, 3, 43-59.
Beyens, K. and Tubex, H. (2002). Gedetineerden geteld. In S. Snacken (Ed.), Strafrechtelijk
Beleid in Beweging (pp. 139-177). Brussel: VUB Press.
Cavadino, M. and Dignan, J. (2006a). Penal policy and political economy. Criminology and
Criminal Justice, 6, 435-456.
Cavadino, M. and Dignan, J. (with others) (2006b) Penal Systems: A Comparative Approach.
London: Sage publications.
Cavadino, M. and Dignan, J. (2011). Penal comparisons: puzzling situations. In A. Crawford
(Ed.), International and Comparative Criminal Justice and Urban Governance (pp.
193-213). Cambridge: Cambridge University Press.
De Mailard, J. and Roché, S. (2004). Crime and justice in France: Time trends, policies and
political debate. European Journal of Criminology, 1, 111-151.
Doob, A.N. and Webster, C.M. (2006). Countering punitiveness: Understanding stability in
Canada’s imprisonment rate. Law & Society Review, 40, 325-367.
Tubex, H. (2013). Pitfalls of comparative (penological) research and how to overcome them. In K. Beyens,
J. Christiaens, B. Claes, S. De Ridder, H. Tournel & H. Tubex (Eds.), The Pains of Doing Criminological
Research. Brussel: VUBPress.
Downes, D. (1988). Contrasts in Tolerance. Post-war Penal Policy in the Netherlands and
England and Wales. Oxford: Clarendon Press.
Downes, D. (2007). Visions of penal control in the Netherlands. Crime and Justice, A Review
of Research, 36, 217-296.
Downes, D. (2011). Comparative criminology, Globalization and the ‘Punitive turn’, in D.
Nelken (Ed.), Comparative Criminal Justice and Globalization (pp. 27-47). Farnham:
Ashgate.
Downes, D. and Hansen, K. (2006). Welfare and punishment in comparative perspective. In
S. Armstrong and L. McAra (Eds.), Perspectives on Punishment: The Contours of
Control (pp. 133-154). Oxford, New York: Oxford University Press.
Downes, D. and van Swaaningen, R. (2007). The road to dystopia? Changes in the penal
climate of the Netherlands. Crime and Justice, A Review of Research, 35, 31-71.
Federale
Overheidsdienst
Justitie.
Justitie
in
Cijfers
2011,
http://justitie.belgium.be/nl/binaries/Justitie%20in%20Cijfers%202011_tcm265157786.pdf (Accessed 23 August 2012).
Federale
Overheidsdienst
Justitie.
Jaarverslag
FOD
Justitie
2011,
http://justitie.belgium.be/nl/binaries/JAARVERSLAG_NL_BD_tcm265-187836.pdf
(Accessed 23 August 2012).
Federale
Overheidsdienst
Justitie,
Directoraat-generaal
Penitentiaire
Activiteitenverslag
Inrichtingen.
2011
http://justitie.belgium.be/nl/binaries/activiteitenverslag%202011%20DGEPI_tcm265178016.pdf (Accessed 23 August 2012).
Garland, D. (2001). The Culture of Control. Crime and Social Order in Contemporary
Society. Oxford, New York: Oxford University Press.
Green, D. (2007). Comparing penal cultures: Child-on child homicide in England and
Norway. Crime and Justice, A Review of Research, 36, 591-643.
Green, D. (2008). When Children Kill Children, Penal Populism and Political Culture.
Oxford: Oxford University Press.
Karstedt, S. (2011). Liberty, equality and justice: democratic culture and punishment, In A.
Crawford (Ed.), International and Comparative Criminal Justice and Urban
Governance (pp. 356-385). Cambridge: Cambridge University Press.
Kommer, M.M. (1994). Punitiveness in Europe – a comparison. European Journal on
Criminal Policy and Research, 2, 29-43.
Tubex, H. (2013). Pitfalls of comparative (penological) research and how to overcome them. In K. Beyens,
J. Christiaens, B. Claes, S. De Ridder, H. Tournel & H. Tubex (Eds.), The Pains of Doing Criminological
Research. Brussel: VUBPress.
Kommer, M.M. (2004). Punitiveness in Europe Revisited. ESC Newsletter, 3, 8-12.
Lacey, N. (2008). The Prisoners’ Dilemma: Political Economy and Punishment in
Contemporary Democracies. Cambridge: Cambridge University Press.
Lappi-Seppälä, T. (2000). The fall of the Finnish prison population. Journal of Scandinavian
Studies in Criminology and Crime Prevention,1, 27-40.
Lappi-Seppälä, T. (2007). Penal policy in Scandinavia. Crime and Justice, A Review of
Research, 35, 217-296.
Lappi-Seppälä, T. (2008). Trust, welfare, and political culture: Explaining differences in
national penal policies. Crime and Justice, A Review of Research, 37, 313-387.
Lappi-Seppälä, T. and Tonry, M. (2011). Crime, criminal justice and criminology in the
Nordic countries. Crime and Justice, A Review of Research, 40, 1-32.
Lévay, M. (2012). Human rights and penalization in Central and Eastern Europe: the case of
Hungary, In S. Snacken and E. Dumortier (Eds.), Resisting Punitiveness in Europe?
Welfare, Human Rights and Democracy (pp. 133-155). London and New York:
Routledge.
Lévy, R. (2007). Pardons and amnesties as policy instruments in contemporary France. Crime
and Justice, A Review of Research, 36, 551-590.
Maes, E. (2010). Evoluties in punitiviteit: lessen uit de justitiele statistieken. In I. Aertsen, K.
Beyens, T. Daems, E. Maes, (Eds.), Hoe Punitief is Belgie? (pp. 43-83). Antwerpen,
Apeldoorn: Maklu.
Nelken, D. (2009). Comparative criminal justice: Beyond ethnocentrism and relativism.
European Journal of Criminology, 6, 291-311.
Nelken, D. (2010). Comparative Criminal Justice: Making Sense of Difference. Compact
Criminology. London: Sage.
Nelken, D. (2011). Making sense of punitiveness: The 2008 Wiarda Inaugural Lecture. In D.
Nelken (Ed.), Comparative Criminal Justice and Globalization (pp. 11-25). Farnham:
Ashgate.
Oberwittler, D. and Höfer, S. (2005). Crime and justice in Germany: An analysis of recent
trends and research. European Journal of Criminology, 2, 465-508.
Pratt, J. (2008a). Scandinavian exceptionalism in an era of penal excess, Part I: the nature and
roots of Scandinavian exceptionalism. British Journal of Criminology, 48, 119-137.
Tubex, H. (2013). Pitfalls of comparative (penological) research and how to overcome them. In K. Beyens,
J. Christiaens, B. Claes, S. De Ridder, H. Tournel & H. Tubex (Eds.), The Pains of Doing Criminological
Research. Brussel: VUBPress.
Pratt, J. (2008b). Scandinavian exceptionalism in an era of penal excess, Part II: does
Scandinavian exceptionalism have a future? British Journal of Criminology, 48, 275292.
Pratt, J. (2011). Penal excess and penal exceptionalism: welfare and imprisonment in
Anglophone and Scandinavian societies, In A. Crawford (Ed.), International and
Comparative Criminal Justice and Urban Governance (pp. 193-213). Cambridge:
Cambridge University Press.
Pratt. J. and Eriksson, A. (2011). ‘Mr. Larsson is walking out again’. The origins and
development of Scandinavian prison systems. Australian and New Zealand Journal of
Criminology, 44, 7-23.
Pratt, J. and Eriksson, A. (2012). In defence of Scandinavian exceptionalism. In T. Ugelvik
and J. Dullum (Eds.), Penal Exceptionalism? Nordic prison policy and practice (pp.
235-260). London, New York: Routledge.
Roché, S. (2007). Criminal justice policy in France: Illusions of severity. Crime and Justice,
A Review of Research, 36, 471-550.
Salle, G. (2003) Situation(s) carcérale(s) en Allemagne. Prison et politique. Déviance et
Société, 27, 389-410.
Snacken, S. (2007). Penal policy and practices in Belgium. Crime and Justice, A Review of
Research, 36, 127-215.
Snacken, S. (2010). Resisting punitiveness in Europe. Theoretical Criminology, 14, 273-292.
Snacken, S. (2012). Conclusion: why and how to resist punitiveness in Europe, In S. Snacken
and E. Dumortier (Eds.), Resiting Punitiveness in Europe? Welfare, human rights and
democracy (pp. 133-155). London and New York: Routledge.
Snacken, S., Beyens, K. and Tubex, H. (1995). Changing prison populations in the Western
countries: fate or policy? European Journal of Crime, Criminal Law and Criminal
Justice, 1, 18-53.
Snacken, S. and Verfaillie, K. (2009). Media, public opinion and (criminological) research. In
H. Eisendrath and J.P. Van Bendegem (Eds.), It Takes Two to do Science (pp. 159182). Brussel: VUBPress.
Tonry, M. (2001). Symbol, substance and severity in Western penal policies. Punishment &
Society, 3, 517-536.
Tonry, M. (2004). Why aren’t German penal policies harsher and imprisonment rates higher?
German Law Journal, 5, 1187-1206.
Tubex, H. (2013). Pitfalls of comparative (penological) research and how to overcome them. In K. Beyens,
J. Christiaens, B. Claes, S. De Ridder, H. Tournel & H. Tubex (Eds.), The Pains of Doing Criminological
Research. Brussel: VUBPress.
Tonry, M. (2006). The prospects for institutionalization of restorative justice initiatives in
Western countries. In I. Aertsen, T. Daems and L. Robert (Eds.), Institutionalising
Restorative Justice (pp. 1-23). Collumpton, Devon and Portland: Willan Publishing
Press.
Tonry, M. (2007). Determinants of penal policies. Crime and Justice, A Review of Research,
36, 1-48.
Tubex,
H.
(2000).
Overzicht
van
de
penitentiaire
statistieken,
criminografische
basisinformatie. Panopticon, 21, 90-92.
Tubex, H. (2001). De zoektocht naar penitentiaire gegevens anno 2000, criminografische
basisinformatie. Panopticon, 22, 203-208.
Tubex, H. and Strypstein, J. (2004). Veel gedetineerden, weinig cijfers, criminografische
basisinformatie. Panopticon, 25, 112-120.
Tubex, H. and Snacken, S. (1994). Punitiviteit in Nederland, teloorgang van een voorbeeld?
Langgestraften en gevaarlijke gedetineerden in internationaal perspectief. In M.
Moerings (Ed.), Hoe Punitief is Nederland (pp. 331-347). Arnhem: Gouda Quint bv.
Tubex, H. and Snacken S. (1995). L'évolution des longues peines, aperçu international et
analyse des causes. Déviance et Société, 19, 103-126.
Tubex, H. and Snacken S. (1998). Straffen en behandelen: evolutie van de aanpak van
seksuele delinquentie. Panopticon, 19, 287-310.
Ugelvik, T. and Dullum, J. (2012) (Eds.), Penal Exceptionalism? Nordic Prison Policy and
Practice. London, New York: Routledge.
Van Dijk, J. (2010). The European crime falls: Security driven? European Society of
Criminology Newsletter, 5, 12-13.
Van Dijk, J. (2011). Trends in Dutch prisoner rates: regression to the mean or enduring
exception? In H. Kury and E. Shea (Eds.), Punitivity International Developments, Vol.
1: Punitiveness – a Global Phenomenon? (pp. 203-225). Brochum/Germany:
Universitätsverlag Dr. Brokmeyer.
Van Swaaningen, R. (2013). Reversing the punitive turn. In T. Daems, S. Snacken and D. van
Zyl Smith (Eds.) European penology? Oxford: Hart (in press).
Weatherburn, D. (2011). Uses and abuses of crime statistics. Crime and Justice Bulletin, 153,
November 2011.
Webster, C.M. and Doob, A.N. (2011), Explaining Canada’s imprisonment rate: the
inadequacy of simple explanations. In A. Crawford (Ed.), International and
Tubex, H. (2013). Pitfalls of comparative (penological) research and how to overcome them. In K. Beyens,
J. Christiaens, B. Claes, S. De Ridder, H. Tournel & H. Tubex (Eds.), The Pains of Doing Criminological
Research. Brussel: VUBPress.
Comparative Criminal justice and Urban Governance (pp. 304-330). Cambridge:
Cambridge University Press.
Young, W. and Brown, M. (1993) Cross-national comparisons of imprisonment. Crime and
Justice, A Review of Research, 17, 1-49.
Zedner, L. (2002). Dangers and Dystopias in penal theories. Oxford Journal of Legal studies,
22, 341-366.