Jump to ContentJump to Main Navigation
Knowledge and its Limits$

Timothy Williamson

Print publication date: 2002

Print ISBN-13: 9780199256563

Published to Oxford Scholarship Online: November 2003

DOI: 10.1093/019925656X.001.0001

Show Summary Details
Page of

PRINTED FROM OXFORD SCHOLARSHIP ONLINE (www.oxfordscholarship.com). (c) Copyright Oxford University Press, 2017. All Rights Reserved. Under the terms of the licence agreement, an individual user may print out a PDF of a single chapter of a monograph in OSO for personal use (for details see http://www.oxfordscholarship.com/page/privacy-policy). Subscriber: null; date: 26 February 2017

(p.302) Appendix 1 Correlation Coefficients

(p.302) Appendix 1 Correlation Coefficients

Source:
Knowledge and its Limits
Publisher:
Oxford University Press

Given real‐valued random variables X and Y whose arguments are conditions (‘events’) in a suitable probability space, their correlation coefficient ρ[X,Y] is defined as Cov[X,Y]/(σ[X]σ[Y]); their covariance Cov[X,Y] is in turn defined as E[XY]–E[X]E[Y], where E[X] is the expectation of X; the standard deviation σ[X] is defined as √ E[(X–E[X])2] (e.g. Parzen 1960: 362). Covariance is not itself an adequate measure of correlation; for example, any random variable is perfectly correlated with itself, but Cov[X,X] = σ[X]2, which varies. One must calibrate Cov[X,Y] by dividing by σ[X]σ[Y].

All these notions can be relativized to a set of background conditions by conditionalizing the underlying probabilities on those conditions.

We must now define correlation coefficients for the conditions on which the probabilities are defined. Denote the probability of a condition C as P[C]. The indicator random variable X(C) is 1 if C obtains and 0 otherwise; thus E[X(C)] = P[C]. For any conditions C and D we define their correlation coefficient ρ[C,D] (with harmless ambiguity in ρ) as ρ[X(C),X(D)]. We will calculate an expression for ρ[C,D] in terms of probabilities, and then derive some elementary facts about it. The conditional probability P[C|D] is P[C ∧ D]/P[D] (0 < P[D]).

We assume that 0<P[C]<1, and similarly for D and E, otherwise the correlation coefficients for these states are not well‐defined.

Proposition 1. ρ[C,D] = (P[C ∧ D]–P[C]P[D])/√ (P[C](1–P[C])P[D](1–P[D])).

Proof. Cov [ X(C),X(D) ] = E [ X(C)X(D) ] E [ X(C) ] E [ X(D) ] = E [ X ( C D ) ] E [ X(C) ] E [ X(D) ] = P [ C D ] P [ C ] P [ D ] . Moreover , σ [ X(C) ] = E [ ( X(C) E [ X(C) ] ) 2 ] = ( E [ X(C) 2 ] ( E [ X(C) ] ) 2 ) by a standard calculation = ( E [ X(C) ] ( E [ X(C) ] ) 2 ) because X(C) { 0 , 1 } = ( P [ C ] ( 1 P [ C ] ) ) .

Proposition 2.

  1. (a) If P[C|D] > P[C] then ρ[C,D] > 0.

  2. (b) If P[C|D] = P[C] then ρ[C,D] = 0.

  3. (c) If P[C|D] < P[C] then ρ[C,D] < 0.

Proof. By Proposition 1, ρ[C,D] has the same sign as P[C ∧ D]–P[C]P[D] = (P[C|D]–P[C])P[D].▪

(p.303) Proposition 3.

  1. (a) If ρ[C,D]≥0 and ρ[C,E]≥0 then ρ[C,D]≤ ρ[C,E] iff

    ( P [ C | D ] P [ C ] ) ( P [ C ] P [ C | ~ D ] ) ( P [ C | E ] P [ C ] ) ( P [ C ] P [ C | ~ E ] )

  2. (b) If ρ[C,D]≤0 and ρ[C,E]≤0 then ρ[C,D]≤ ρ[C,E] iff

    ( P [ C | D ] P [ C ] ) ( P [ C ] P [ C | ~ D ] ) ( P [ C | E ] P [ C ] ) ( P [ C ] P [ C | ~ E ] )

Proof. (a) (P[C|D] P[C]P[D]) 2 = ( P [ C | D ] P [ D ] P [ C ] P [ D ] ) ( P [ C ] P [ C ~ D ] P [ C ] P [ D ] ) = ( P [ C | D ] P [ C ] ) P [ D ] ( P [ C ] ( 1 P [ D ] ) P [ C | ~ D ] P [ ~ D ] ) = ( P [ C | D ] P [ C ] ) ( P [ C ] P [ C | ~ D ] ) P [ D ] ( 1 P [ D ] ) .
Hence by Proposition 1,
ρ [ C,D ] 2 P [ C ] ( 1 P [ C ] ) = ( P [ C | D ] P [ C ] ) ( P [ C ] P [ C | ~ D ] ) .

But if ρ[C,D]≥0 and ρ[C,E]≥0 then

ρ [ C,D ] ρ [ C,E ] ρ [ C,D ] 2 P [ C ] ( 1 P [ C ] ) ρ [ C,E ] 2 P [ C ] ( 1 P [ C ] ) ( P [ C | D ] P [ C ] ) ( P [ C ] P [ C | ~ D ] ) ( P [ C | E ] P [ C ] ) ( P [ C ] P [ C | ~ E ] ) .

(b) is similar. ▪

Proposition 4. If P[C|D]≤P[C|E] and P[C|∼ D]≥P[C|∼ E] then ρ[C,D]≤ ρ[C,E].

Proof. Suppose that P[C|D]≤P[C|E] and P[C|∼ D]≥P[C|∼ E].

Case (i): P[C]≤P[C|D]. Then P[C]≤P[C|E], so by Proposition 2, ρ[C,D]≥0 and ρ[C,E]≥0. Hence by Proposition 3(a), ρ[C,D]≤ ρ[C,E] iff

( * ) ( P [ C | D ] P [ C ] ) ( P [ C ] P [ C | ~ D ] ) ( P [ C | E ] P [ C ] ) ( P [ C ] P [ C | ~ E ] ) .

But 0 ≤ P[C|D]–P[C] ≤ P[C|E]–P[C]. Moreover, since P[C] = P[D]P[C|D] + (1–P[D])P[C|∼ D], 0 ≤ P[C]–P[C|∼ D] ≤ P[C]–P[C|∼ E]. Hence (*) holds, so ρ[C,D] ≤ ρ[C,E].

Case (ii): P[C|D]<P[C]<P[C|E]. Then ρ[C,D]<0<ρ[C,E] by Proposition 2.

Case (iii): P[C|E]≤P[C]. Similar to case (i), using Proposition 3(b).

Proposition 5.

  1. (a) ρ[C,C] = 1

  2. (b) ρ[C,∼ C] = –1

  3. (c) –1 ≤ ρ[C,D] ≤ 1

  4. (d) ρ[C,D] = 1 iff P[C|D] = 1 and P[C|∼ D] = 0.

  5. (e) ρ[C,D] = –1 iff P[C|D] = 0 and P[C|∼ D] = 1.

Proof. (a) and (b) are simple consequences of Proposition 1.

(c) P[C|D]≤P[C|C] and P[C|∼ D] ≥ P[C|∼ C], so ρ[C,D]≤ ρ[C,C] = 1 by (a) and Proposition 4. Similarly, ρ[C,D]≥ ρ[C,∼ C] = –1.

(d) Suppose that ρ[C,D] = 1. Then ρ[C,C] ≤ ρ[C,D] by (a) and (c), and P[C|D] >P[C] by Proposition 2. Hence by Proposition 3

( P [ C | D ] P [ C ] ) ( P [ C ] P [ C | ~ D ] ) ( P [ C | C ] P [ C ] ) ( P [ C ] P [ C | ~ C ] ) = ( 1 P [ C ] ) P [ C ] .
(p.304) But P[C|D]–P[C]≤1–P[C] and P[C]–P[C|∼ D]≤P[C], so P[C|D]–P[C] = 1–P[C] and P[C]–P[C|∼ D] = P[C], so P[C|D] = 1 and P[C|∼ D] = 0. Conversely, if P[C|D] = 1 and P[C|∼ D] = 0 then P[D] = P[C ∧ D] = P[C] and the result follows by Proposition 1.

(e) is similar to (d). ▪