Jump to ContentJump to Main Navigation
Critical Appraisal of Epidemiological Studies and Clinical Trials$

Mark Elwood

Print publication date: 2007

Print ISBN-13: 9780198529552

Published to Oxford Scholarship Online: September 2009

DOI: 10.1093/acprof:oso/9780198529552.001.0001

Show Summary Details
Page of

PRINTED FROM OXFORD SCHOLARSHIP ONLINE (www.oxfordscholarship.com). (c) Copyright Oxford University Press, 2017. All Rights Reserved. Under the terms of the licence agreement, an individual user may print out a PDF of a single chapter of a monograph in OSO for personal use (for details see http://www.oxfordscholarship.com/page/privacy-policy). Subscriber: null; date: 23 January 2017

(p.505) Appendix Methods of statistical analysis: formulae and worked examples

(p.505) Appendix Methods of statistical analysis: formulae and worked examples

Source:
Critical Appraisal of Epidemiological Studies and Clinical Trials
Publisher:
Oxford University Press

This appendix presents tables showing the calculations of measures of association and of statistical tests, for the main study designs discussed showing for each the formulae and worked examples. There are also reference tables for statistical issues.

Introduction

Table 1

Case–control designs, unmatched

p. 508

Table 2

Cohort designs with individual observation data

p. 512

Table 3

Cohort designs with person‐time data

p. 518

Table 4

Cohort or case–control designs with small numbers; exact test

p. 522

Table 5

Case–control studies with 1:1 matching

p. 526

Table 6

Case–control studies with fixed 1:m matching

p. 528

Table 7

Cohort at case–control designs: test for trend

p. 530

Table 8

Formulae for sample size determination

p. 534

Table 9

Constants for use in sample size formulae

p. 535

Table 10

Calculation of kappa to compare two sets of observations

p. 536

Table 11

Adjustment of results of a case–control study using values of sensitivity and specificity

p. 540

Table 12

Meta‐analysis: Mantel‐Haenszel and Peto methods

p. 542

Table 13

Meta‐analysis: confidence limits method

p. 548

Table 14

Meta‐analysis: DerSimonian‐Laird method

p. 550

Table 15

Table of probability (P) values and equivalent normal deviate (2) and χ2 values

p. 552

Table 16

Table showing functions in Microsoft Excel for relating P‐values, normal deviates, and χ2 values

p. 557

References

p. 558

(p.506) Introduction

These tables will enable the reader to analyse most studies where the factors involved are discrete rather than continuous variables. Of course, major analyses should not be done without consideration of alternative methods and discussion with colleagues, including those more skilled in statistical and epidemiological methods.

The formulae are primarily those derived from the work of Mantel and Haenszel. These involve approximations which are appropriate if the numbers of observations available are adequate: in a 2 × 2 table each expected value should be greater than 5. For the formulae used to produce summary risk estimates and test statistics after confounder control by stratification, it is the total number of observations which is relevant; the formulae can be applied even if some strata have few observations. The statistics used to test whether an odds ratio or attributable risk estimate is constant over several strata do depend upon having reasonable numbers of observations within each stratum.

The one exception in this Appendix to the limitation of small numbers is the ‘exact’ test for a 2 × 2 table presented in Table 4. This can be applied to simple tables with small numbers. For all the other designs considered, more precise formulae (‘exact’ tests) are also available, but these will usually give similar results except where the numbers of observations are small.

The sources of the formulae are given with each table. There are many texts which review these and related methods in detail, and discuss the situations in which more complex formulae may be appropriate, and provide information on them. (p.507)

(p.508)

Table 1. Case–control studies. Unmatched: formulae

1. Format of table

Appendix Methods of statistical analysis: formulae and worked examples

2. Risks

Appendix Methods of statistical analysis: formulae and worked examples

3. Statistical tests

Observed exposed cases = a

Expected value of a  = E = N1M1/T

Variance of a    = V = N1N0 M1M0/T2(T–1)

3A Chi–squared statistic, χ2, with 1 degree of freedom

Appendix Methods of statistical analysis: formulae and worked examples

3B χ statistic, or normal deviate = χ = Appendix Methods of statistical analysis: formulae and worked examples

3C Continuity correction: reduce absolute value of numerator by Appendix Methods of statistical analysis: formulae and worked examples before squaring; i.e. replace numerator by Appendix Methods of statistical analysis: formulae and worked examples, where Appendix Methods of statistical analysis: formulae and worked examples means the absolute value of (a – E), irrespective of being positive or negative.

4. Confidence limits (C.L.).

y% limits for logarithm of odds ratio = ln OR ± Zy(dev ln OR), where Zy = appropriate normal deviate (see Table 15) and dev ln OR = standard deviation of in OR

4A dev ln Appendix Methods of statistical analysis: formulae and worked examples

5. Stratified analysis over I subtables of above format, values ai etc.

5A Summary odds ratio = Appendix Methods of statistical analysis: formulae and worked examples

5B Summary χ2 statistic, 1 d.f. = Appendix Methods of statistical analysis: formulae and worked examples

Each term analogous to 3 above.

5C Continuity correction: reduce absolute value of numerator by Appendix Methods of statistical analysis: formulae and worked examples before squaring.

5D A formula for the variance of the logarithm of the summary odds ratio is given by Robins et al.[1]

For each stratum, these quantities are calculated. The subscripts have been omitted.

Appendix Methods of statistical analysis: formulae and worked examples

Then the variance of ln Mantel‐Haenszel summary odds ratio is:

Appendix Methods of statistical analysis: formulae and worked examples

and y% confidence limits for ln OR s

Appendix Methods of statistical analysis: formulae and worked examples

5E Test of homogeneity = Appendix Methods of statistical analysis: formulae and worked examples where variance of

ln ORi = square of standard deviations given in 4A above. Gives a χ2 on I – 1 degrees of freedom.

Sources:    3A, 5A, 5B   Mantel and Haenszel [2]

4A      Wolff [3]

5D      Robins et al. [1]

5E      Rothman and Greenland [4]

(p.509)

Table.1. Case–control studies. Unmatched: worked example

Data from the case–control study shown in Ex.6.24 and in Ex.7.9: association between cervical carcinoma and smoking.

1. Crude table:

Appendix Methods of statistical analysis: formulae and worked examples

2. Odds ratio = (130×198)/(45×87) = 6.57

3A, B Statistical test:

Appendix Methods of statistical analysis: formulae and worked examples

3C With continuity correction χ2 = 81.4

Appendix Methods of statistical analysis: formulae and worked examples

5. Stratified analysis:

Data are shown in Ex.7.9. Totals in the three subtables are 113, 211, 136 (p. 244).

Appendix Methods of statistical analysis: formulae and worked examples

5B Summary χ2: calculate for each subtable ai, Ei, Vi; and sum each of these.

Appendix Methods of statistical analysis: formulae and worked examples

Appendix Methods of statistical analysis: formulae and worked examples

5C. χ2 with continuity correction = 80.6

d.f. = 1 P <0.000001 (Appendix Table 15)

5D Using Robins formula, var ln ORMH = 0.0456,

and 95% confidence limits of ORMH = 4.12, 9.53.

5E Test of homogeneity: calculate for each table ORi,

and use formula 4 to obtain dev ln ORi

Appendix Methods of statistical analysis: formulae and worked examples

Hence test of homogeneity = sum of {(ln ORi – ln OR s)2/var ln ORi}

where var ln ORi = dev ln and OR s = 6.27

or, equivalently, test = sum of {(ln ORi/ln OR s)/dev ln ORi}2

Appendix Methods of statistical analysis: formulae and worked examples

d.f. = 2, P‐values from Appendix Table 16. The odds ratios vary significantly.

(p.510) (p.511)

(p.512)

Table 2. Cohort studies, count data: formulae

1. Format of table

Appendix Methods of statistical analysis: formulae and worked examples

2. Risks

Relative risk = r e/r 0 Odds ratio = Appendix Methods of statistical analysis: formulae and worked examples Attributable risk = r er 0

3. Statistical tests

Observed exposed cases = a

Expected value of a =E =N 1 M1/T

Variance of a = V = N 1 N 0 M 1 M 0/T 2(T–1)

3A Chi–squared statistic, χ2, with 1 degree of freedom = (a−E)2/v

Appendix Methods of statistical analysis: formulae and worked examples

3B χstatistic, or normal deviate = χ = Appendix Methods of statistical analysis: formulae and worked examples.

3C Continuity correction: reduce absolute value of numerator by Appendix Methods of statistical analysis: formulae and worked examples before squaring, i.e. replace numerator by Appendix Methods of statistical analysis: formulae and worked examples, Appendix Methods of statistical analysis: formulae and worked examples means the absolute value of (a–E), irrespective of being positive or negative.

4. Confidence limits (C.L.)

  1. (i) for logarithm of RR

    • = ln RR + Zy dev ln RR

  2. (ii) for logarithm of OR; analogous to RR, hence

    • = ln OR ± Zy dev ln RR

  3. (iii) for attributable risk

    • = AR ± Zy dev AR

where Zy = appropriate normal deviate (see Table 15) and dev = standard deviation

4A Formulae for standard deviation

Appendix Methods of statistical analysis: formulae and worked examples

5. Stratified analysis

5A Risks

Appendix Methods of statistical analysis: formulae and worked examples

5B Summary χ2 statistic on 1 d.f. = Appendix Methods of statistical analysis: formulae and worked examples

Each term analogous to 3 above.

5C Continuity correction: reduce absolute value of numerator by Appendix Methods of statistical analysis: formulae and worked examples before squaring.

5D

Tests of homogeneity; χ2 on I–1 degrees of freedom; variances are squares of standard deviations given in 4.

Appendix Methods of statistical analysis: formulae and worked examples

Sources: 3A, 5A, 5B Mantel and Haenszel [2]

4A   Rothman and Greenland [4], Wolff [3]

5D   Rothman and Greenland [4]

(p.513)

Table 2. Cohort studies, count data: worked example

Data from a prospective cohort study; association between maternal epilepsy in pregnancy and malformation in infants; Shapiro et al. [5]

(Ex. 3.2 showed another study in the same format).

1. Crude table:

Appendix Methods of statistical analysis: formulae and worked examples

2. Relative risk = 1.63 Odds ratio = 1.70 Attributable risk = 4.06%

3A Statistical test:

Appendix Methods of statistical analysis: formulae and worked examples

Appendix Methods of statistical analysis: formulae and worked examples

3C With continuity correction χ2 = 7.60.

4A Confidence limits:

for RR, dev ln RR = 0.168; limits = exp(ln RR ± 1.96 × dev ln RR)

=1.17, 2.27

for OR, dev ln OR = 0.188; limits = exp(ln OR ± 1.96 × dev ln OR)

=1.18, 2.46

for AR, dev AR = 1.758%; limits = AR ± 1.96 × dev AR

=0.61, 7.51%

5. Stratified analysis:

Analogous to Table 1. If the above data were one stratum of a stratified table, the weight w i for use in formula 5A(iii) would be 1/(dev AR i)2=1/(0.0176)2.

(p.514) (p.515) (p.516) (p.517)

(p.518)

Table 3. Cohort studies, person‐time data: formulae

1. Format of table

Appendix Methods of statistical analysis: formulae and worked examples

2. Risks

Relative risk = r e/r 0 Attributable risk = r er 0

3. Statistical tests

Observed exposed cases = a

Expected value of a = E = N 1 M1/T

Variance of a = V = N1N0M1/T2

3A Chi‐squared statistic, χ2 with 1 degree of freedom = (a − E)2/V

Appendix Methods of statistical analysis: formulae and worked examples

3B χ statistic, or normal deviate = χ = Appendix Methods of statistical analysis: formulae and worked examples

3C Continuity correction: reduce absolute value of numerator by Appendix Methods of statistical analysis: formulae and worked examples before squaring; i.e. replace numerator by Appendix Methods of statistical analysis: formulae and worked examples, Appendix Methods of statistical analysis: formulae and worked examples means the absolute value of (a–E), irrespective of being positive or negative.

4. Confidence limits (C.L.)

(i) for logarithm of RR  (ii) for attributable risk

= ln RR ± Zy dev ln RR   = AR ± Zy dev AR

where Zy = appropriate normal deviate (Table 15) and dev = standard deviation

4A Formulae for standard deviation

Appendix Methods of statistical analysis: formulae and worked examples

5. Stratified analysis over 1 subtables of above format, values, ai etc.

5A

Appendix Methods of statistical analysis: formulae and worked examples

5B Summary χ2 statistic = Appendix Methods of statistical analysis: formulae and worked examples

Each term analogous to 3 above.

5C Continuity correction: reduce absolute value of numerator by Appendix Methods of statistical analysis: formulae and worked examples before squaring.

5D Tests of homogeneity; χ2 on I–1 degrees of freedom; variances are squares of standard deviations given in 4.

Appendix Methods of statistical analysis: formulae and worked examples

Odds ratio: analogous to relative risk

Sources: 3A Mantel and Haenszel [2]

5D Rothman and Greenland [4]

(p.519)

Table 3. Cohort studies, person‐time data: worked example

Data from the prospective cohort study shown in Ex.6.7; association between exercise and coronary heart disease mortality.

1. Crude table:

Appendix Methods of statistical analysis: formulae and worked examples

2. Relative risk  = 81.85/23.83 = 3.43

Attributable risk = 81.85 − 23.83 = 58.02/10000 man‐years

3A Statistical test:

Appendix Methods of statistical analysis: formulae and worked examples

3B

Appendix Methods of statistical analysis: formulae and worked examples

3C With continuity correction χ2 = 100.45.

4A dev ln RR = 0.131   95% limits = exp{ln 3.43±(1.96×0.131)}

= 2.66, 4.44

dev = 4.60/10000  95% limits = 58.0 ± (1.96 × 4.60)

= 49.0, 67.0

5. Stratified analysis. Analogous to Appendix Table 1

(p.520) (p.521)

(p.522)

Table 4. Cohort (count data) or case–control, with small numbers; exact test for a fourfold table: formulae

1. Format of table

Appendix Methods of statistical analysis: formulae and worked examples

2. Risks

As in Table 2; the small numbers make no difference.

3. Statistical test

Probability of a particular table occurring = Appendix Methods of statistical analysis: formulae and worked examples

To calculate one‐sided probability of a or a more extreme value: if a is greater than expected, i.e. a > N 1 M 1/T, calculate quantity above for each value of a from observed value to the maximum, given when a = N 1 or a = M 1; sum these values. If a is less than expected, calculate for values of a down to zero. For two‐sided tests, the one‐sided value is usually doubled.

! represents a factorial; e.g. 5! is 5 × 4 × 3 × 2 × 1.

Sources: This test was developed independently in the 1930s by R.A. Fisher,

J.O. Irwin, and F. Yates.

Exact confidence limits and stratified analysis will not be presented:

see, for example, Breslow and Day [6] or Armitage et al. [7].

(p.523)

Table 4. Cohort (count data) or case–control, with small numbers; exact test for a fourfold table: worked example

Saral et al. [8] performed a double‐blind randomized trial of the drug acyclovir, compared with placebo, as prophylaxis against herpes simplex infection in 20 bone marrow transplant recipients who were seropositive for herpes simplex before randomization; the outcome was development of active herpes simplex infection.

1.

Table of results:

Appendix Methods of statistical analysis: formulae and worked examples

2. Risk: the relative risk and odds ratio are both zero.

3. Statistical test:

probability of this set of data, given the null hypothesis

Appendix Methods of statistical analysis: formulae and worked examples

This is the one‐sided P‐value; the two‐sided value of P = 0.003.

As the result obtained was an extreme one, only one calculation is necessary. Otherwise, further calculations are needed. For example, suppose there had been two infections in the acyclovir group compared with seven in the placebo. To assess significance, we add the probability of the observed table to the probability of more extreme results with the same marginal totals: the observed table is: Appendix Methods of statistical analysis: formulae and worked examples

More extreme results, given that a is less than its expected value, are a = 1 and a = 0 with the same marginal totals:

Appendix Methods of statistical analysis: formulae and worked examples

giving the final P‐value (one‐sided) of 0.032 + 0.0027 + 0.00006 = 0.035 or (two‐sided) = 0.07

(p.524) (p.525)

(p.526)

Table 5. Case–control, 1:1 matching: formulae

1. Format of table: numbers of pairs

Appendix Methods of statistical analysis: formulae and worked examples

2. Risks

Odds ratio = s/t

3A Statistical tests

Appendix Methods of statistical analysis: formulae and worked examples

3B χ statistic, or normal deviate = Appendix Methods of statistical analysis: formulae and worked examples.

3C Continuity correction: reduce absolute value of numerator by 1 before squaring; it becomes Appendix Methods of statistical analysis: formulae and worked examples.

4. Confidence limits

The derivation of confidence limits is complex. Approximate test‐based limits can be based on the statistic by the methods shown in Ex.7.7.

Sources: McNemar [9]. Stratified analysis of matched data is not frequently performed; matched multivariate models are more useful.

(p.527)

Table 5. Case–control, 1:1 matching: worked example

Data from the case–control study shown in Ex. 6.24 and Ex.7.9, association between nasal cancer and smoking.

1. Table:

Appendix Methods of statistical analysis: formulae and worked examples

2. Odds ratio = 30/7 = 4.29

3A, B χ2 statistic = (30−7)2/(30+7) = 14.30  Appendix Methods of statistical analysis: formulae and worked examples

3C With continuity correction, χ2 = 13.1.

4. 95% confidence limits for OR, test‐based = exp{ln 4.29 (1±1.95/3.78)}

= 2.0, 9.1

(p.528)

Table 6. Case–control studies, fixed 1:m matching: formulae

1. Format of table: numbers of pairs

Appendix Methods of statistical analysis: formulae and worked examples

2. Risks

Appendix Methods of statistical analysis: formulae and worked examples

3. Statistical tests

Appendix Methods of statistical analysis: formulae and worked examples

For the continuity correction, reduce the absolute value of the numerator by Appendix Methods of statistical analysis: formulae and worked examples (M + 1) before squaring.

4. Confidence limits

The derivation of confidence limits is complex. Approximate test‐based limits can be based on the χ statistic by the methods shown in Ex. 7.7.

Sources: 2,3 Based on Mantel and Haenszel [2]

For more complex matched designs, see Breslow and Day [6], pp.169–187.

(p.529)

Table 6. Case–control studied, fixed 1:m matching: worked example

Data from Collette et al. [10]; a 1:3 matched case–control study assessing the value of screening for breast cancer by physical examination and xeromammography. The cases were women from the defined population who had died from breast cancer; the controls were age‐matched women randomly selected from this population.

1. Table:

Appendix Methods of statistical analysis: formulae and worked examples

Appendix Methods of statistical analysis: formulae and worked examples

Appendix Methods of statistical analysis: formulae and worked examples

4. 95% confidence limits for OR (test‐based) = exp[ln 0.30 (1 ± 1.96/2.95)]

= 0.13, 0.67

(p.530)

Table 7. Cohort (count data) or case–control studies: test for trend: formulae

1. Format of table

Appendix Methods of statistical analysis: formulae and worked examples

2. Risks

Appendix Methods of statistical analysis: formulae and worked examples

3. Statistical tests

3A For each level k against the referent level: tests and confidence limits as in Appendix Table 1.

Heterogeneity χ2 for the table above, on k − 1 degrees of freedom

Appendix Methods of statistical analysis: formulae and worked examples

where Ek = expected value of ak = NkM 1/T.

This test assesses whether the odd ratios, or more directly the proportions of cases, in the various exposure levels are consistent with the overall value—it does not take into account the order of the levels of exposure.

3B Test for trend from regression of the values ak – Ek on the score xk; χ2 on 1 d.f.

Appendix Methods of statistical analysis: formulae and worked examples

For a continuity correction, if xk scores are one unit apart, replace

Appendix Methods of statistical analysis: formulae and worked examples

3C An approximate test of departure from the linear trend is given by the difference between the heterogeneity and the trend χ2 statistics, on k – 2 degrees of freedom: this tests the adequacy of the linear trend in describing the data.

4. Stratified analysis

The trend statistic above can be generalized to a stratified analysis over I subtables of the format above, but the formula is tedious for hand calculation; see Breslow and Day [6] pp.148–150.

The stratified χ2 statistic on 1 d.f. is

Appendix Methods of statistical analysis: formulae and worked examples

where Eki = NkiM 1i/Ti.

Sources: 3A: Armitage [12], Armitage et al. [7]

3B  Mantel [13]

(p.531)

Table 7. Cohort (count data) or case–control studies: test for trend: worked example

These are data from a case–control study relating the occurrence of twin births to maternal parity (the number of previous births the mother has had). The cases were births, the controls a sample of single births. From Elwood [11]

1. Table and elements of calculations:

Appendix Methods of statistical analysis: formulae and worked examples

2. Risks:

Given above: e.g. for parity 2 odds ratio = (454 × 1833)/ (853 × 716)

= 1.36

3A Global or heterogeneity χ2. For each level Ek = NkM 1/T

e.g. for parity 2, E 2 = 1307 × 2472 /7430 = 434.85

Appendix Methods of statistical analysis: formulae and worked examples

3B Test for trend:

Appendix Methods of statistical analysis: formulae and worked examples

3C Test for departure from linear trend:

χ2 = 91.2 − 88.2 = 3.0 d.f. = 2  P > 0.2 (Table 16).

(p.532) (p.533)

(p.534)

Table 8. Formulae for sample size determination: formulae

1. Unmatched studies, equal groups

Appendix Methods of statistical analysis: formulae and worked examples

2. Multiple controls per case

Given c controls per case: n = no. of cases or exposed subjects,

cn = no. of controls.

Appendix Methods of statistical analysis: formulae and worked examples

3. 1:1 matched studies

Appendix Methods of statistical analysis: formulae and worked examples

where H = OR/(1 = OR)  OR = odds ratio

and M = number of matched pairs.

Notation

Appendix Methods of statistical analysis: formulae and worked examples

(p.535)

Table 8. Formulae for sample size determination: worked example

See Chapter 7, pp 256–263

Table 9. Constants for use in sample size formulae

A. Table relating normal deviates to power and to significance level

Appendix Methods of statistical analysis: formulae and worked examples

B. Values of K = (Z α + Z β ) 2, for commonly used values of α and β

Appendix Methods of statistical analysis: formulae and worked examples

Normal deviates corresponding to frequently used values for significance levels (Z α) and for power (Z β); and table of K where K = (Z α + Z β ) 2. The value of Z β is the normal deviate corresponding to the one‐sided test for (1−power)

(p.536)

Table 10. Calculation of kappa to compare two sets of observations: formulae

1. Format of table. Shown for three categories.

pij = proportions of total subjects in cell i, j of the table

Appendix Methods of statistical analysis: formulae and worked examples

2. Calculation of kappa, κ

Appendix Methods of statistical analysis: formulae and worked examples

Kappa, κ

= (observed agreement − expected agreement)/ (1 − expected agreement)

= (OE)/(1 − E)

3. Standard error of kappa, SE(κ)

n = total number of subjects.

Let ui = pi. p.i(pi. = p.i)    ui uses only the marginal totals.

then

Appendix Methods of statistical analysis: formulae and worked examples

Test of significance of kappa compared with null value of 0 is given by z = κ/SE(κ), but this is not very useful.

4. Confidence limits for kappa

Using the Z‐values given in Table 9

two‐sided limits = K ± Z α/2.SE(κ)

Other formulae are available to test the significance of a difference in kappa from a pre‐specified value, and for weighted kappa.

5. Relationship of kappa to odds ratio (approximate)

Let OR 0 = observed odds ratio and OR T = true odds ratio

then OR T = (κ + OR 0 − 1)/κ

and OR 0 = κ (OR T − 1) + 1

Sources: 2,   Cohen [15]

3, 4  Fleiss et al. [16, 17]

5   Thompson and Walter [18]

For weighted kappa and further development see Fleiss et al. [17]

For discussion of applications to study results see Armstrong et al. [19]

(p.537)

Table 10. Calculation of kappa to compare two sets of observations: worked example

Data from Westerdahl et al. [14]; see also Ex. 5.6, p. 139.

1. Question: Have you ever been sunburned causing erythema and pain for a few days? If yes, how many times after the age of 19 years?

Appendix Methods of statistical analysis: formulae and worked examples

2. Calculation of kappa

Observed agreement = (0.096 + 0.527 + 0.125) = 0.747

Expected agreement by chance

= (0.165 × 0.152 + 0.637 × 0.656 + 0.197 × 0.192) = 0.481

Kappa, κ

= (observed agreement − expected agreement)/ (1 − expected agreement)

= (0.747 − 0.481)/(1 − 0.481) = 0.512

3. Standard error of kappa

Number of subjects, n = 593

U 1 = 0.00795  from 0.165 × 0.152 × (0.165 + 0.152)

U 2 = 0.54084

U 3 = 0.01478

Appendix Methods of statistical analysis: formulae and worked examples

Test of significance Appendix Methods of statistical analysis: formulae and worked examples (Appendix Table 15).

4. Confidence limits

95% two‐sided confidence limits = 0.512 ± 1.96 × 0.031 = 0.45, 0.57

5. Relationship of kappa to odds ratio

Suppose a study using this measure yields an odds ratio (observed) of 1.80, then estimated true odds ratio = (0.512 + 1.80 − 1)/0.512 = 2.56

(p.538) (p.539)

(p.540)

Table 11. Adjustment of results of a case–control study using values of sensitivity and specificity: formulae

1. Observed results of case–control study: proportions of cases and controls exposed:

Appendix Methods of statistical analysis: formulae and worked examples

2. Sensitivity and specificity values for exposure, for cases and for controls:

Appendix Methods of statistical analysis: formulae and worked examples

3. Hence estimated ‘true’ values:

Appendix Methods of statistical analysis: formulae and worked examples

(p.541)

Table 11. Adjustment of results of a case–control study using values of sensitivity and specificity: worked example

1. Observed results of case–control study (hypothetical data)

Appendix Methods of statistical analysis: formulae and worked examples

Results expressed as proportions exposed

Appendix Methods of statistical analysis: formulae and worked examples

2. Sensitivity and specificity of exposure assessment, in cases and in controls, from other sources

Appendix Methods of statistical analysis: formulae and worked examples

(p.542)

Table 12. Meta‐analysis: Mantel‐Haenszel and Peto methods: formulae

1. Format for each study is the same as given in Table 2

Appendix Methods of statistical analysis: formulae and worked examples

For each study i,

observed number of successes on new treatment = ai

expected number Appendix Methods of statistical analysis: formulae and worked examples

variance of ai

Appendix Methods of statistical analysis: formulae and worked examples

as in Appendix Table 2.

For meta‐analysis of a number of studies, a format of one line per study is useful.

2. Mantel–Haenszel method

Appendix Methods of statistical analysis: formulae and worked examples

2A Summary results: summary odds ratio Appendix Methods of statistical analysis: formulae and worked examples

2B χ statistic      Appendix Methods of statistical analysis: formulae and worked examples

2C Confidence limits for summary odds ratio

Precise limits given using the Robins et al. [1] formula for the variance of the ln summary odds ratio, given in Table 1, 5D

y% limits for summary OR = Appendix Methods of statistical analysis: formulae and worked examples

2D Test for heterogeneity

For each study calculate Qi = wi(ln ORi − ln OR s)2

where wi = weight = 1/variance of ln ORi, from Table 1.

ln ORi = ln OR for study i

ln OR s = ln summary OR

then Appendix Methods of statistical analysis: formulae and worked examples is distributed as a χ2 statistic on n − 1 degrees of freedom,

where n = number of studies.

3. Peto method

Appendix Methods of statistical analysis: formulae and worked examples

3A summary ln odds ratio = Appendix Methods of statistical analysis: formulae and worked examples

summary odds ratio = Appendix Methods of statistical analysis: formulae and worked examples

3B summary χ (normal deviate)= Appendix Methods of statistical analysis: formulae and worked examples

3C Confidence limits for summary odds ratio = Appendix Methods of statistical analysis: formulae and worked examples

where Z α is the normal deviate for the significance level α, from Table 9.

3D Test of heterogeneity

For each study Vi and aiEi have been defined as in 3 above, and (aiEi)2 is also calculated. The heterogeneity statistic Q is calculated as

Appendix Methods of statistical analysis: formulae and worked examples

Q is distributed as a χ2 statistic on n − 1 degrees of freedom, where n = number of studies.

As Appendix Methods of statistical analysis: formulae and worked examples the statistics produced by these two methods are the same. The odds ratio estimates are different.

Sources: Mantel and Haenszel [2]

Peto et al. [21]

Petitti [22]

(p.543)

Table 12. Meta‐analysis: Mantel–Haenszel and Peto methods: worked example

1. Format of table

Data from three randomized trials

assessing clomiphene in inducing ovulation [20]

Appendix Methods of statistical analysis: formulae and worked examples

2. Mantel–Haenszel method

Appendix Methods of statistical analysis: formulae and worked examples

2A Summary odds ratio: OR s = sum(ad/T)/sum(bc/T) =   8.93

Appendix Methods of statistical analysis: formulae and worked examples

2C Confidence limits for OR s ln ORs = 2.1892

variance of ln OR s from Robins et al. [1] formula: 0.1173

95% confidence limits using this variance   4.56, 17.47

2D Test of heterogeneity

Appendix Methods of statistical analysis: formulae and worked examples

Sum Qi is a χ2 on two degrees of freedom: clearly non‐significant

Peto method

Appendix Methods of statistical analysis: formulae and worked examples

3A Summary odds ratio

Appendix Methods of statistical analysis: formulae and worked examples

3B Normal deviate

Appendix Methods of statistical analysis: formulae and worked examples

3C Confidence limits for summary odds ratio

95% limits for or = 3.92, 11.85

3D Test for heterogeneity

Appendix Methods of statistical analysis: formulae and worked examples

(p.544) (p.545) (p.546) (p.547)

(p.548)

Table 13. Meta‐analysis: confidence limits method: formulae

1. Data required

The information required from each study is an appropriate measure of odds ratio, OR *, and its variance, V, which can be calculated if the lower and upper confidence limits ORL and ORu are given, as

Appendix Methods of statistical analysis: formulae and worked examples

where Z corresponds to the significance level: e.g. for 95% two‐sided confidence limits, Z = 1.96 (Table 9)

Then, for each study, the weight w is 1/V

Appendix Methods of statistical analysis: formulae and worked examples

2. Calculation of summary odds ratio

Summary ln Appendix Methods of statistical analysis: formulae and worked examples

hence, OR s = exp(ln OR s)

3. Confidence limits for the summary odds ratio

variance of ln Appendix Methods of statistical analysis: formulae and worked examples

95% confidence limits for ln Appendix Methods of statistical analysis: formulae and worked examples

95% confidence limits for Appendix Methods of statistical analysis: formulae and worked examples

4. Heterogeneity test

Heterogeneity statistic = Appendix Methods of statistical analysis: formulae and worked examples

which is a χ2 statistic on n = 1 degrees of freedom, where n = number of studies

Sources: Prentice and Thomas [23]

Greenland [24]

(*) This method is applicable to relative risk also, and (without the log transformation) to risk difference.

(p.549)

Table 13. Meta analysis: confidence limits method: worked example

1. Meta analysis of seven case–control studies relating sunburn to melanoma.

Appendix Methods of statistical analysis: formulae and worked examples

2. Summary ln OR s = 67.14/105.97 = 0.63 Summary OR s = exp(0.63) = 1.88

3. 95% CL for summary Appendix Methods of statistical analysis: formulae and worked examples = lower 1.56

upper 2.28

4. Test for heterogeneity: sum Qi = 29.47

Sum Q is a χ2 statistic on n − 1 = six degrees of freedom; from Table 16 P < 0.0001, so consideration has to be given to the results which are discordant, as shown by their Qi results.

I2 = (29.47−6)/29.47 = 0.80

= 80%

(p.550)

Table 14. Random effects model (DerSimonian–Laird) applied to confidence limits data

1. The random effects model of DerSimonian and Laird uses the same data as the fixed effects model shown in Table 13. For each study, the odds ratios ORi, and their variances vi are as in Table 13.

Weights wi = 1/vi are calculated. The fixed effects summary odds ratio OR s is calculated as before, and the quantities Qi = wi(ln ORi − ln OR s)2 and sum Qi = Q

2,3. For the random effects model, revised weights dwi are used where

Appendix Methods of statistical analysis: formulae and worked examples

where n is the number of studies.

D cannot be negative. If (n − 1) is greater than Q, then D = 0.

If there is little heterogeneity, Q will be small, so D will be small, and the DerSimonian and Laird weights will be similar to the fixed effects weights.

4. The percentage distribution of weights is shown, showing how the D–L method gives a more even weight distribution than the fixed effects method.

Source: DerSimonian and Laird [25]

(p.551)

Table 14. Random effects model (DerSimonian–Laird) applied to confidence limits data

1. Data as in Table 13: seven studies relating sunburn to melanoma

Appendix Methods of statistical analysis: formulae and worked examples

4. Weight distribution

Appendix Methods of statistical analysis: formulae and worked examples

(p.552)

Table 15. Table of probabilities (P‐values) and corresponding values of χ2 on 1 d.f., standardized normal deviate 2‐sided, and 1‐sided

P‐values

χ2

Normal deviate

Normal deviate

1 d.f.

2‐sided

1‐sided

0.000001

23.9

4.89

4.75

0.00001

19.51

4.42

4.27

0.0001

15.14

3.89

3.72

0.001

10.83

3.29

3.09

0.01

6.63

2.58

2.33

0.02

5.41

2.33

2.05

0.03

4.71

2.17

1.88

0.04

4.22

2.05

1.75

0.05

3.84

1.96

1.64

0.06

3.54

1.88

1.55

0.07

3.28

1.81

1.48

0.08

3.06

1.75

1.41

0.09

2.87

1.70

1.34

0.10

2.71

1.64

1.28

0.11

2.55

1.60

1.23

0.12

2.42

1.55

1.18

0.13

2.29

1.51

1.13

0.14

2.18

1.48

1.08

0.15

2.07

1.44

1.04

0.16

1.97

1.41

0.99

0.17

1.88

1.37

0.95

0.18

1.80

1.34

0.92

0.19

1.72

1.31

0.88

0.20

1.64

1.28

0.84

0.25

1.32

1.15

0.68

0.3

1.07

1.04

0.52

0.35

0.87

0.93

0.39

0.4

0.71

0.84

0.25

0.45

0.57

0.76

0.13

0.5

0.45

0.67

0.0

0.6

0.27

0.52

0.7

0.15

0.39

0.8

0.06

0.25

0.9

0.02

0.13

(p.553)

Table 15. Table of probabilities (P‐values) and corresponding values of χ2 on one degree freedom, standardized normal deviate two‐sided, and standardized normal deviate, one‐sided

The values tabulated

1. The chi‐square χ2 statistic on one degree of freedom. For any other number of degrees of freedom, use Table 16

The table gives the probability of the given χ2 or a larger value under the null hypothesis.

Appendix Methods of statistical analysis: formulae and worked examples

Although this is a one tail probability on a χ2 distribution, because the χ2 distribution is given by a normal deviate squared, this gives a two‐sided test; assessing a variable x 2 using a χ2 distribution is equivalent to assessing x using a normal distribution and a two‐sided test.

2. The standardized normal deviate (often called Z or chi) using a two‐sided test.

The probability given is that of the given value ± Z or a more extreme value of either sign, under the null hypothesis.

Appendix Methods of statistical analysis: formulae and worked examples

3. The standardized normal deviate (often called Z or chi), using a one‐sided test.

The probability given is that of the given value Z or a more extreme value of the same sign.

Appendix Methods of statistical analysis: formulae and worked examples

Relationships between these

1. For a given value of P, the corresponding χ2 (1) value is equal to the two‐sided standardized normal deviate squared.

e.g. for P = 0.05 χ2 = 3.84 Appendix Methods of statistical analysis: formulae and worked examples

2. For a given normal deviate, the two‐sided probability is twice the one‐sided.

e.g. for Z = 1.96, two‐sided probability = 0.05, one‐sided = 0.025

How to use the tables

1. Be sure you know which statistic you wish to use!

2. To find the value of a statistic corresponding to a given P‐value: look down the table to find the P‐value or the nearest to it; read off the corresponding statistic.

Examples: What χ2 (1) value corresponds to P = 0.05? Result: 3.84

What one‐sided deviate corresponds to P = 0.01? Result: 2.33

3. To find the P‐value corresponding to a given statistic:

find the correct column;

look down the table to find the given value or the nearst to it;

read off the corresponding P‐value.

Examples: What P‐value corresponds to χ2 (1) = 7.4? Result between 0.001 and 0.01; can be written 0.001 < P < 0.01

What P‐value corresponds to a one‐sided test giving a deviate of 1.04? Result: 0.15

(p.554) (p.555) (p.556)

(p.557)

Table 16. Functions in Microsoft Excel spreadsheets to convert between test statistics and probability (P) values

To get from:

to:

function

Example

Z statistic

P value, 2 sided

=2*(1‐normsdist(z))

Z stat

Z stat

1.96

3

P value

P value

0.049996

0.002700

Z stat

4.89

P value

0.000001

Z statistic

P value, 1 sided

=(1‐normsdist(z))

Z stat

Z stat

1.96

1.64

P value

P value

0.024998

0.050503

Z stat

4.75

P value

0.000001

P value, 2 sided

Z statistic

=abs(normsinv(P/2))

P value

P value

0.05

0.001

Z stat

Z stat

1.96

3.29

P value

0.1

Z stat

1.64

P value, 1 sided

Z statistic

=abs(normsinv(P))

P value

P value

0.05

0.1

Z stat

Z stat

1.64

1.28

P value

0.01

Z stat

2.33

Chi squared stat, df

P value

=chidist(chi, df)

Chi stat, df

Chi stat, df

Chi stat, df

3.84

10.3

38.932

1

3

21

P value

P value

P value

0.050044

0.016181

0.010000

P value, df

Chi sq

=chiinv(P, df)

P value, df

P value, df

P value, df

0.01

0.2

0.05

1

1

30

Chi sq

Chi sq

Chi sq

6.63

1.64

43.77

t test result, df

P value

=tdist(test, df, tails)

T test, df, tails

T test, df, tails

T test, df, tails

4

4

1.96

1

1

160

1

2

2

P value

P value

P value

0.077979

0.155958

0.051733

(p.558) References for Appendix

1. Robins JM, Breslow NE, Greenland S. Estimation of the Mantel–Haenszel variance consistent with both sparse-data and large-strata limiting models. Biometrics 1986; 42: 311–323.

2. Mantel N, Haenszel W. Statistical aspects of the analysis of data from retrospective studies of disease. J Natl Cancer Inst 1959; 22: 719–748.

3. Wolff B. On estimating the relationship between blood group and disease. Ann Hum Genet 1955; 19: 251–253.

4. Rothman KJ, Greenland S. Modern epidemiology Second ed. Philadelphia: Lippincott - Raven; 1998.

5. Shapiro S, Slone D, Hartz SC, et al. Anticonvulsants and parental epilepsy in the development of birth defects. Lancet 1976; i: 272–275

6. Breslow NE, Day NE. Statistical methods in cancer research. Volume 1: The analysis of case-control studies. Lyon: IARC Scientific Publication No 32; 1980.

7. Armitage P, Berry G, Matthews JNS. Statistical methods in medical research. 4 ed. Oxford: Blackwell Scientific; 2002.

8. Saral R, Burns WH, Laskin OL, Santos GW, Lietman PS. Acyclovir prophylaxis of herpes-simplex-virus infections: a randomized, double-blind, controlled trial in bone-marrow-transplant recipients. N. Engl. J Med 1981; 305, 63–67.

9. McNemar Q. Note on the sampling of the difference between corrected proportions or percentages. Psychometrika 1947; 12: 153–157.

10. Collette HJA, Day NE, Rombach JJ, de Waard F. Evaluation of screening for breast cancer in a non-randomised study (the DOM project) by means of a case-control study. Lancet 1984; 1: 1224–1226.

11. Elwood JM. Maternal and environmental factors affecting twin births in Canadian cities. Br J Obstet Gynaecol 1978; 85: 351–358.

12. Armitage P. Test for linear trend in proportions and frequencies. Biometrics 1955; 11: 375–386.

13. Mantel N. Chi-square tests with one degree of freedom; extensions of the Mantel-Haenszel procedure. Am Stat Assoc J 1963; 58: 690–700.

14. Westerdahl J, Anderson H, Olsson H, Ingvar C. Reproducibility of a self-administered questionnaire for assessment of melanoma risk. Int J Epidemiol 1996; 25(2): 245–251.

15. Cohen J. A coefficient of agreement for nominal scales. Educ. Psychol. Meas. 1960; 20: 27–46.

16. Fleiss JL, Cohen J, Everitt BS. Large sample standard errors of kappa and weighted kappa. Psychol Bull 1969; 72:323–327.

17. Fleiss JL, Levin B, Paik MC. Statistical methods for rates and proportions. 3 ed. New York: John Wiley & Sons; 2003.

18. Thompson WD, Walter SD. Variance and dissent. A reappraisal of the kappa coefficient. J Clin Epidemiol 1988; 41: 949–958.

19. Armstrong BK, White E, Saracci R. Principles of Exposure Measurement in Epidemiology. Oxford: Oxford University Press; 1992.

20. Hughes E, Collins, J., Vandekerckhove, P. Clomiphene citrate vs. placebo for ovaluation induction in oligo-amenorrhoeic women. Cochrane Database of Systematic Reviews 1995; 2 (CD-ROM). (This review has since been updated)

(p.559) 21. Peto R, Pike MC, Armitage P, Breslow NE, Cox DR, Howard SV et al. Design and analysis of randomized clinical trials requiring prolonged observation of each patient. II. analysis and examples. Br J Cancer 1977; 35(1):1–39.

22. Petitti DB. Meta-analysis, decision analysis, and cost-effectiveness analysis. Methods for quantitative synthesis in medicine Second ed. New York: Oxford University Press; 2000.

23. Prentice RL, Thomas DB. On the epidemiology of oral contraceptives and disease. In: Klein G, Weinhouse S, editors. Advances in Cancer Research Volume 49. 49 ed. Orlando: Academic Press; 1987. 285–401.

24. Greenland S. Quantitative methods in the review of epidemiologic literature. Epidemiol Rev 1987; 9:1–30.

25. DerSimonian R, Laird N. Meta-analysis in clinical trials. Control Clin Trials 1986; 7: 177–188. (p.560)