(p.131) Appendix Methods of Systematic Review and MetaAnalysis
(p.131) Appendix Methods of Systematic Review and MetaAnalysis
Systematic Review
This book reports on systematic reviews of the effects on crime of the five major forms of public area surveillance: closedcircuit television (CCTV), improved street lighting, security guards, place managers, and defensible space.
Criteria for Inclusion of Evaluation Studies
In selecting evaluations for inclusion in each of the five systematic reviews, the following criteria were used.

a. The surveillance measure in question was the main focus of the intervention. For evaluations involving one or more interventions, only those in which the surveillance measure in question was the main intervention were included. The determination of the main intervention was based on the author identifying it as such or, if the author did not do this, the importance the report gave the primary intervention compared to any other interventions.

(p.132) b. There was an outcome measure of crime. The most relevant crime outcomes were violent and property crimes (including vehicle crimes).

c. The evaluation design was of high methodological quality, with the minimum design involving beforeandafter measures of crime in experimental and comparable control areas. In a few of the included studies, the comparability of the experimental and control areas was difficult to gauge or not as strong as the others. We were reluctant to exclude these studies unless they were clearly inadequate.

d. The total number of crimes in each area before the intervention was at least 20. The main measure of effect size was based on changes in numbers of crimes between the beforeandafter time periods. It was considered that a measure of change based on an N below 20 was potentially misleading. Also, any study with fewer than 20 crimes before would have insufficient statistical power to detect changes in crime. The criterion of 20 is probably too low, but we were reluctant to exclude studies unless their numbers were clearly inadequate.
Search Strategies
To locate studies meeting the above criteria, four search strategies were employed:

a. Searches of electronic bibliographic databases.

b. Searches of reviews of the literature on the effects on crime of the surveillance measure in question.

c. Searches of bibliographies of evaluation reports of applicable studies.

d. Contacts with leading researchers.
Both published and unpublished reports were considered in these searches. Furthermore, the searches were international in scope and were not limited to the English language.
CCTV
The following terms were used to search the databases for CCTV evaluations: closed circuit television, CCTV, cameras, social control, (p.133) surveillance, and formal surveillance. When applicable, “crime” was then added to each of these terms (e.g., CCTV and crime) to narrow the search parameters.
The search strategies (over two periods of time) resulted in the collection of a total of 44 evaluations of CCTV that met the inclusion criteria. Fortyeight other evaluations were obtained and analyzed but did not meet the inclusion criteria and were excluded. The majority of these evaluations were excluded because they did not use a control area. (This was also the case in the other reviews.) Two other evaluation reports were identified, but we were not successful in obtaining copies. Information on the excluded and unobtainable evaluations (for all five reviews) is available from the first author.
Improved Street Lighting
The following terms were used to search the databases for street lighting evaluations: street lighting, lighting, illumination, and natural surveillance. When applicable, “crime” was then added to each of these terms (e.g., street lighting and crime) to narrow the search parameters.
The search strategies (also over two periods of time) resulted in the collection of a total of 13 evaluations of improved street lighting that met the inclusion criteria. Twenty other evaluations were obtained and analyzed but did not meet the inclusion criteria and were excluded. Four other evaluation reports were identified, but we were not successful in obtaining copies.
Security Guards
The following terms were used to search the databases for security guard evaluations: security guards, private police, formal surveillance, and guardian. When applicable, “crime,” “surveillance,” “systematic review,” and “effectiveness” were added to each of these terms (e.g., security guards and crime) to narrow the search parameters.
The search strategies resulted in the collection of a total of five evaluations of security guards that met the inclusion criteria. Four other evaluations were obtained and analyzed but did not meet the inclusion criteria and were excluded.
(p.134) Place Managers
The following terms were used to search the databases for place manager evaluations: employee, place manager, guardian, conductor, attendant, park keeper, doorman, assistant, and occupational presence. When applicable, “crime,” “surveillance,” “supervision,” and “effectiveness” were then added to each of these terms (e.g., place managers and crime) to narrow the search parameters.
The search strategies resulted in the collection of only two evaluations of place managers that met the inclusion criteria. Five other evaluations were obtained and analyzed but did not meet the inclusion criteria and were excluded.
Defensible Space
The following terms were used to search the databases for defensible space evaluations: defensible space and CPTED (crime prevention through environmental design). When applicable, “crime,” “surveillance,” “road closures,” “barricades,” and “prevention” were then added to each of these terms (e.g., defensible space and crime) to narrow the search parameters. “Defensible space” and “CPTED” were also used together along with these other terms to narrow the search parameters.
The search strategies resulted in the collection of a total of five evaluations of defensible space that met the inclusion criteria. Four other evaluations were obtained and analyzed but did not meet the inclusion criteria and were excluded.
MetaAnalysis
To carry out a metaanalysis, a comparable measure of effect size and an estimate of its variance are needed in each program evaluation (Lipsey and Wilson, 2001; Wilson, 2001). In the case of both CCTV and street lighting evaluations, the measure of effect size had to be based on the number of crimes in the experimental and control areas before and after the intervention. This is because this was the only information that was regularly provided in these evaluations. Here, the odds ratio (OR) is used as the measure of effect size. For example, in the Doncaster city center CCTV (p.135) evaluation by David Skinns (1998), the odds of a crime after given a crime before in the control area were 2,002/1,780 or 1.12. The odds of a crime after given a crime before in the experimental area were 4,591/5,832 or 0.79. The OR, therefore, was 1.12/0.79 or 1.42.
The OR has a very simple and meaningful interpretation. It indicates the proportional change in crime in the control area compared with the experimental area. In this example, the OR of 1.42 indicates that crime increased by 42% in the control area compared with the experimental area. An OR of 1.42 could also indicate that crime decreased by 30% in the experimental area compared with the control area, because the change in the experimental area compared with the control area is the inverse of the OR, or 1/1.42 (0.70) here. The OR is calculated from the following table.
Before 
After 


Experimental 
a 
b 
Control 
c 
d 
Here, a, b, c, and d are numbers of crimes:
The variance of OR is calculated from the variance of LOR (the natural logarithm of OR). The usual calculation of this is as follows:
To produce a summary effect size in a metaanalysis, each effect size is weighted according to the inverse of the variance. This was another reason for choosing the OR, which has a known variance (Fleiss, 1981, pp. 61–67).
The estimate of the variance is based on the assumption that total numbers of crimes (a, b, c, d) have a Poisson distribution. Thirty years of mathematical models of criminal careers have been dominated by the assumption that crimes can be accurately modeled by a Poisson process (Piquero, Farrington, and Blumstein, 2003). However, the large number of changing extraneous factors that influence the number of crimes may cause overdispersion; that is, where the variance of the number of crimes VAR exceeds the number of crimes N. The equation
D increased linearly with N and was correlated .77 with N. The mean number of crimes in an area in the CCTV studies was about 760, suggesting that the mean value of D was about 2. For the lighting studies, the mean number of crimes in an area was about 445, suggesting that the mean value of D was about 1.6. However, for both cases this is an overestimate because the monthly variance is inflated by seasonal variations, which do not apply to N and VAR. Nevertheless, to obtain a conservative estimate, V(LOR) calculated from the usual formula was multiplied by D (estimated from the foregoing equation) in all cases. This adjustment corrects for overdispersion within studies but not for heterogeneity between studies. (For a more detailed discussion of the variance in the case of CCTV, see Farrington, Gill, Waples, and Argomaniz, 2007; for lighting, see Farrington and Welsh, 2004.)