Jump to ContentJump to Main Navigation
Platform or Personality?The Role of Party Leaders in Elections$

Amanda Bittner

Print publication date: 2011

Print ISBN-13: 9780199595365

Published to Oxford Scholarship Online: May 2011

DOI: 10.1093/acprof:oso/9780199595365.001.0001

Show Summary Details
Page of

PRINTED FROM OXFORD SCHOLARSHIP ONLINE (www.oxfordscholarship.com). (c) Copyright Oxford University Press, 2017. All Rights Reserved. Under the terms of the licence agreement, an individual user may print out a PDF of a single chapter of a monograph in OSO for personal use (for details see http://www.oxfordscholarship.com/page/privacy-policy). Subscriber: null; date: 25 May 2017

Appendix A: Methodological Notes

Appendix A: Methodological Notes

Source:
Platform or Personality?
Publisher:
Oxford University Press

This project consists of three main sections, each reflecting a different facet of leader evaluations. The first examines the traits themselves in order to determine the extent to which traits can be grouped together in dimensions. The second section is geared towards understanding the origins of trait evaluations by identifying the factors that influence the way that voters perceive party leaders. Finally, the third section assesses the role that evaluations of leaders play in influencing vote choice. The project is also comparative—both across time and space—in order to fill the gaps in the literature and move towards a generalized understanding of the role of leaders in elections. The study therefore involves the pooling of election studies datasets, which are analysed simultaneously.

This appendix outlines the methodological challenges inherent in a cross-national and longitudinal study of this magnitude as well as the strategies employed to address them. This section begins with an overview of the methodology employed in the study as a whole. I then discuss the four main challenges encountered in this study. The first involves the difficulty in matching questions from a diverse set of election studies in order to code them in a common format for pooled analysis. Secondly, I address the related challenge of developing a measure of political sophistication across countries and years, where the same measures were not incorporated in all election studies. The third challenge discussed is an issue specific to the nature of this study: how do we compare the evaluations of different leaders from different countries in different years, from different parties? Fourth and finally, I address the issue of sampling weights, as one risks flooding a sample with the opinions of respondents from a single country or single election when running a pooled analysis.

Analysing Leader Evaluations on a Cross-National and Longitudinal Basis

To better understand the role of leaders in elections, a comparative analysis is necessary. I argue that in large part, the reason for the lack of agreement in the literature is the nature of the studies that have been conducted to date. Many of these studies have been based on single elections and, furthermore, have often examined distinct survey questions. I suggest that by looking at a common set of variables across countries and over time, we will gain more conclusive evidence about trait dimensionality, the origins of evaluations, and the impact of voters' perceptions of leaders.

There are thirty-five election studies from a total of seven countries with closed-ended traits questions.1 Concatenating these studies results in a dataset with over 186,000 respondents and over 400 variables, all coded in a similar format. In addition to trait evaluations, the dataset incorporates (p.146) variables such as party and leader thermometers, demographic variables, attitudes towards issues, vote choice, partisanship, media exposure, and political sophistication.

This appendix provides detail about the techniques employed for each part of the study addressed in greater detail within each individual chapters. This next section, then, will address only those issues that are common to the project as a whole, especially the challenges arising as a result of the comparative nature of the study.

Challenges Faced in the Study

While there were challenges that surfaced throughout the project—trying to find the model that provided the best fit, interpreting the data analyses, and so on—the major challenges that were common to the whole project were largely encountered during the early stages of the research during the creation of the dataset.

Coding Variables in a Common Format

The challenge presented by coding variables in a common format is inherent to any comparative analysis involving several, isolated election studies. Different questions are asked in different countries and different years, and in order to analyse issues of interest across these studies, we need to make judgements about the extent to which variables can be considered similar. The coding of the two main variable types (leaders' traits and issue attitudes) provided the greatest challenges in this study.

Appendix B provides a complete list of the original question wording of all variables included in the analyses, and as the list makes clear, there is indeed quite a variety in question wording. Such multiplicity is evident even within a single country. For example the Canadian Election Study of 1997, asked traits questions in the following format: ‘now we'd like to get your impressions of the party leaders. I would like you to tell me how well the following words fit each leader. What about Jean Charest. Does “arrogant” describe Jean Charest very well, fairly well, not very well, or not at all?’ In 2000, by contrast, the Canadian Election Study asked Canadians a similar question in a very different format: ‘Which party leader would you describe as arrogant?’ As is fairly obvious, these two questions are not the same, and trying to compare trait evaluations over time could consequently be a challenge, even if the trait itself is common to the studies, as ‘arrogant’ was in these two years. Preliminary investigation conducted elsewhere (Bittner 2007a) suggests that while question format does have an impact on voters' evaluations of party leaders, the impact is not so great as to preclude a comparative analysis.

All trait evaluations were recoded on a common 0–1 scale with evaluations separated by leader (with a value of ‘1’ reflecting the most positive evaluation of a given leader on the trait). Similar traits, regardless of original question wording, were given the same label, in order to permit large-scale analysis. To build the dataset, separate questions from separate studies that were considered to have the same root (as seen in Table 2.4) were coded under the same trait. For example, ‘strong leadership’ and ‘leadership’ were both recoded as ‘leadership’, even though the question wording was slightly different in the two studies. This approach/method made it possible to merge all of the separate studies together, in order to have many respondents evaluating leaders on the ‘same’ traits. This led to a total of 55 separate traits to be analysed, a process which is discussed in greater detail in Chapter 3.

The second main coding challenge emerged with respect to respondents' issue attitudes. Commonly included in most voting models, issue attitudes are an important component of this study, in particular when considering the factors that influence voters' evaluations of leaders. In order to determine the extent to which issue positions influence voters' perceptions of leaders' traits, it is necessary to identify and examine common issues across election studies. This presents a challenge, however, since there (p.147)

Table A.1 Major Election Issues

Country/year

Major election issue

Country/year

Major election issue

Canada

Britain

1968

Quebec separatism

1983

Economy

1984

Patronage and corruption

1987

Defence

1988

Free trade

1992

Healthcare

1993

Debt and deficit

1997

Healthcare

1997

Quebec

2001

Currency

2000

Quebec

Australia

2004

Party financing

1987

Economy/recession

2006

Party financing

1993

GST, economy

The United States

1996

Economy, recession

1972

Vietnam war

1998

GST

1976

Watergate, pardon of Nixon

2001

Immigration, asylum seekers

1980

Iran hostages

2004

Terrorism, defence spending

1984

Nuclear weapons, defence

New Zealand

1988

Willie Horton

1999

Law and order

1992

Economy

2002

Healthcare, genetic engineering

1996

Economy

Germany

2000

Moral decay of society, Clinton scandals

1980

Economy, recession, jobs

2004

Iraq, war on terror, defence

1987

Economy, recession, jobs

Sweden

1988

Large inefficient government

1991

Large inefficient government

was no one ‘issue’ that appeared in all election studies. In order to avoid the problem of identifying common issues across elections, I opted to gather opinion on issue dimensions, in the manner established by Kenneth Benoit and Michael Laver (2006) in their elite surveys on party policies and platforms. Their work locates parties on two main policy dimensions: taxes versus spending, and social liberalism. Happily, many of the surveys included in this study probe respondents for their attitudes about taxes versus social spending. On the dimension of social liberalism, where possible, this dimension was based on attitudes towards abortion. Where this was not possible, I made use of other variables, including attitudes towards immigration, attitudes towards family values, and attitudes towards treatment of violent criminals. This fits with Benoit and Laver's methodology as well. All variables in both the taxes versus spending dimension and the social liberalism dimension were coded on the same 0–1 scale, with ‘1’ reflecting more left-leaning/progressive attitudes and ‘0’ reflecting more right-leaning/conservative attitudes.

The final variable that I included in my analysis was created to reflect opinion on the main issue in each election. This variable was different from election study to election study, but all variables were coded in the same direction as the other two issues (0–1, with ‘1’ reflecting more left-leaning/progressive attitudes). Table A.1 lists the major issue of each election, and the exact question used from each election study can be found in Appendix B. Compiling such a list was a challenge in itself. Starting in 1992, the European Journal of Political Research began to publish a yearbook with information about elections that have taken place over the year. This was the main resource for all elections where it was available (Koole and Mair 1993, 1994; Katz and Koole 1997, 1999, 2002; Koole and Katz 1998, 2000, 2001; Katz 2003; Van Biezen and Katz 2005; Bale and Van Biezen 2007).

(p.148) For the remaining, pre-1992 elections, information about the important electoral issues was obtained from journal articles and books discussing specific elections (Williams and Wilson 1977; Irving and Paterson 1981, 1987; Clarke et al. 1984; Wilson 1985; Crewe and Harrop 1986, 1989; Arter 1989; Kinder et al. 1989; Johnston et al. 1992; Lockerbie 1992; Worlund 1992; Gant and Lyons 1993; Mendelsohn and Nadeau 1999; Blais et al. 2002). Finally, in two Canadian elections (1997 and 2000), a clear issue did not emerge in the campaign. For these two elections, I used attitudes towards a long-standing issue in the Canadian context, attitudes towards Quebec, as a substitute. Readers may question some of these choices, but I argue that by incorporating attitudes on the three issue dimensions (as imprecise as the measures may be), we are able to tap into voters' issue attitudes on a comparative level, and even maintain some contextual/time-specific information through the inclusion of the major issue variable.

Political Sophistication

Many of the questions that remain unanswered regarding the evaluation of party leaders relate to mechanisms—how do voters consider party leaders when making decisions at the ballot box? Is it an information shortcut, helping those who lack information about policies and platforms to come to a decision, or is it something else? By incorporating political sophistication into the analyses, it is possible to gain further insight into how these decisions are made.

The challenge of measuring this variable is compounded with the difficulty of concatenating separate election studies, as many measure political sophistication quite differently. Scholars suggest that the best measure of political sophistication is one that taps into factual knowledge about politics (Price and Zaller 1993; Delli Carpini and Keeter 1996). Unfortunately, not all of the studies included in this analysis incorporate questions of factual political knowledge into the survey. Table A.2 lists the type of measure used for each election study in the creation of the overall sophistication index.

Where possible, an index based on responses to factual questions was used, with one exception: the US election studies. Factual questions were not asked in all of the election studies, therefore to keep the measure consistent within the country, the interviewer rating (common to all studies in the National Election Study, NES) was utilized for the US election studies, including the 2000 Annenberg Election Study. After the knowledge index, it has been shown that measures of general political interest are the best indicators of political sophistication (Matthews 2006), and indeed, in the election studies that included more than one type of measure, the general interest measure was most highly correlated with factual knowledge indexes.2

Other scholars have pointed to the utility and validity of the interviewer rating (Zaller 1985; Bartels 1996), which, incidentally, is the preferred measure used by this study after political interest. Finally, when the interviewer rating was not available, an index was created based on media consumption in both television and print. Appendix B lists all of the variables included in all of the measures for each country, expanding on the information in Table A.2. All variables were recoded and combined to fit a 0–1 scale. These variables were then merged by pooling together all of the election studies.

In order to test the political sophistication measure, two main strategies were employed. First, I regressed the index on a series of dummy variables, one for each of the different sophistication measures that were combined to create the overall index. The reference category (constant) was the knowledge index. Table A.3 depicts the results of the analysis and illustrates that both the general interest measure and the interviewer rating did not differ substantially from the knowledge index. The media exposure variable was the most different from the other variables, and the coefficient suggests that the level of sophistication of the respondent was 0.21 points higher than the sophistication level of the reference group when the media variable was used as the sophistication measure. This measure was only used for two studies, however, and so the level of bias in the measure overall was minimal.

(p.149)

Table A.2 Political Sophistication: Measures Used to Construct Index

General interest

Knowledge index

Interviewer rating

Exposure to media

CA 1968

X

CA 1984

X

CA 1988

X

CA 1993

X

CA 1997

X

CA 2000

X

CA 2004

X

CA 2006

X

US 1972–2004

X

US 2000 (a)

X

UK 1983

X

UK 1987

X

UK 1992

X

UK 1997

X

UK 2001

X

AU 1987

X

AU 1993

X

AU 1996

X

AU 1998

X

AU 2001

X

AU 2004

X

NZ 1999

X

NZ 2002

X

GE 1980

X

GE 1987

X

SW 1988

X

SW 1991

X

Table A.3 Political Sophistication Index: Multivariate Analysis

General interest

0.004

(0.003)

Interviewer rating

0.016

(0.002)

Media exposure

0.210

(0.004)

Constant

0.540

(0.002)

Observations

156,301

R-squared

0.023

Ordinary least squares regression analysis.

(p.150)

Table A.4 Relationship between Demographics and Sophistication

University graduate

0.187

(0.004)

Income

0.064

(0.002)

Woman

−0.074

(0.003)

Non-White

−0.041

(0.004)

Homeowner

0.002

(0.004)

Age

0.02

(0.001)

Employed

0.017

(0.004)

Constant

0.339

(0.008)

Observations

40,835

R-squared

0.12

Ordinary least squares regression analysis.

The second main strategy used to test the validity of the measure was another regression analysis in which the sophistication measure was regressed on a series of socio-economic and demographic variables to determine the extent to which levels of sophistication related to common demographic variables in the ways we might expect. Table A.4 depicts the results of this analysis.

University graduates tended to have a higher level of political sophistication (coefficient of 0.187), as did those with higher incomes (coefficient of 0.064), those who were older (coefficient of 0.02), and those who were employed (coefficient of 0.017). Women tended to have lower levels of political sophistication (coefficient of −0.074) as did respondents of visible minority groups (with a coefficient of −0.041). All of these fit with our common perceptions of who in society is more politically sophisticated and informed (Popkin 1991; Sniderman et al. 1991; Bartels 1996; Delli Carpini and Keeter 1996). These results suggest that the political sophistication measure performs well, despite the methodological challenges involved with creating a common measure across this many election studies.

Cross-Party Leader Evaluations

Comparing evaluations of party leaders across countries comes with considerable methodological challenges in terms of grouping parties into ‘types’. Using each country's party label for cross-national analysis is not feasible, since, for example, German FDP does not exist in Canada, nor does the British Labour Party exist in the United States. However, there are commonalities among parties from different countries and scholars have expended considerable effort to develop cross-national comparisons between parties based on their policies and platforms. Some of the more prominent examples include expert surveys (Benoit and Laver 2006) and the Comparative Manifesto Project (Budge et al. 2001). See McDonald et al. (2007) for a review of these efforts. My analysis makes use of Benoit and Laver's extensive (2006) work, which maps parties along two-dimensional lines: stances on social liberalism and on taxes versus spending. By relying upon Benoit and Laver's placement of parties, I was able to (p.151)

Appendix A: Methodological Notes

Figure A.1 Categorization of Party Types, Based on Benoit and Laver's Party Policy in Modern Democracies (2006)

group similar parties together under common labels and could therefore create cross-national, longitudinal comparisons of leaders' traits.

I categorized parties according to their placement along these two issue dimensions and grouped parties with similar locations into the same category. Figure A.1 illustrates the placement of parties and the groups into which they were gathered based on an amalgamation of the individual country-based figures that Benoit and Laver include in their book. Party categories included ‘Conservative’ (including the Canadian Conservatives, the American Republicans, the British Conservatives, the German CDU and CSU, the Australian Liberals and the Swedish Moderate Party); ‘Centre-Left’ (including the Canadian Liberals, British Labour, New Zealand Labour, Australian Labor, the German SPD, and the US Democratic party); ‘Left’ (which includes the Canadian NDP, the Swedish Left Party, the New Zealand Alliance, the British LibDems, and the Australian Democrats); ‘Centre-Right’ (includes the Swedish Centre Party, the New Zealand National Party, New Zealand ACT, the Swedish Peoples Party, and the German FDP); and ‘Right’ (includes New Zealand First, Australian ONE, Australian Nationals, Swedish Christian, Swedish NDP, and the Canadian Reform Party).

There were two exceptions to the grouping of parties according to their locations on the two dimensions: Green parties (which includes the Swedish, Canadian, New Zealand, Australian, and German Green Parties) and Sectional parties (which includes the Canadian Bloc Quebecois, the Scottish National Party and the Welsh Plaid Cymru). While some may argue that these two party types could have been included in either Left or Centre-Left party categories, I felt that it was more logical to separate these parties into their own categories given their very specific focuses on either environmental issues (Green) or nationalist platforms (Sectional).

Some may object with the location of any of the parties along the two-dimensional lines: for example, it could be argued that the Canadian Green Party should be located further to the right on (p.152) the taxes versus spending dimension. While there may be some irregularities/inconsistencies in Benoit and Laver's conclusions, their model was relied upon as a whole because of their unified and systematic formulation of party locations. Furthermore, the placements of these parties are based upon elite surveys of more than simply a few individuals, suggesting a certain degree of rigour/authenticity/accuracy. While their results may not have been entirely accurate in all instances, they do provide a consistent and defensible way to conceive and organize party groups and therefore merit being used.

After having grouped parties into categories, the data were then recoded to ensure that trait evaluations were comparable along these lines. Thus, the trait, ‘leadership’ was labelled with the party category label, for example, ‘Conservative’. ‘Conservative leadership’ therefore included evaluations of the strength of leadership trait for leaders of the US Republican Party, the German CDU/CSU, the British Conservative Party, the Australian Liberal Party, the Canadian Conservative Party, and the Swedish Moderate Party. In total, this brought the total number of respondents evaluating this trait for Conservative leaders to 73,000.

By aggregating parties into these groups, a common nomenclature was developed that made it possible to analyse evaluations of these different parties' leaders en masse. Evaluating leaders from this many parties and this many countries would otherwise not have been possible.

Sampling Weights

The total number of respondents in each of the election studies varied from less than 2000 (e.g. in the NES) to approximately 50,000 (in the case of the Annenberg Election Study). The major risk that arises when attempting to conduct an analysis with pooled data is that the attitudes of respondents in a particular country or election could flood the sample and thus skew the results. In a sense, what we hope to achieve in pooling data is to mirror a random sample across countries and elections. Without any adjustments, however, some people have higher or lower probabilities of selection, with Americans in 2000 having a significantly higher probability of selection.3 In order to prevent this from happening and to instead ensure that the results of the analyses more accurately reflect the true relationships between variables, a system of sampling weights was created.

In order to determine sampling weights, I obtained each country's population in the year 1988. The year 1988 was chosen because it represents an approximate mid-point in the 1968–2006 time spread in the thirty-five election studies in this analysis. Furthermore, it is one of the two years in which the largest number of elections took place: three election studies were conducted this year, and three took place the year before. Population data were obtained from gapminder.org, a non-profit organization that works closely with both the United Nations and Google to promote the achievement of the UN Millennium Development Goals. This website relies heavily on accurate population data in order to perform graphical simulations on demand, and thus the data are reliable and dependable. The total population of the seven countries was calculated, as was each country's proportion of the total population. At the same time, the total number of observations in the dataset was obtained as was the proportion of each country's presence in the thirty-five election dataset.

Table A.5 lists all of the total number of respondents as well as the proportions of each country within the total. Weights were generated by dividing each country's proportion of the population in the real world by its proportion of the number of observations within the dataset. In order to ensure that any one dataset within a particular country did not flood the sample, each election study was weighted to ensure that it had an equal position within the country's total sample. These two sets of weights were multiplied by one another and the product was the weight assigned to each election study. In effect, (p.153)

Table A.5 Calculating Weights According to Total Population

A

B

C

D

E

Population

Proportion of total population

Sample in ES

Proportion of total ES sample

Weight (B/D)

NZ

3,331,000

0.007649962

11,755

0.063017326

0.121394582

AU

16,520,000

0.037939769

12,326

0.066078398

0.574162976

SW

8,469,000

0.019449873

7,626

0.040882189

0.475754206

GE

78,031,000

0.179205699

26,361

0.141318566

1.268097353

UK

57,160,000

0.13127344

22,749

0.121955011

1.076408736

CA

26,895,000

0.061766955

29,264

0.156881245

0.393717904

US

245,021,000

0.562714301

18,082

0.096935712

5.805025709

58,373

0.312931552

1.798202506

Total

435,427,000

1

186,536

1

* Population Numbers from the year 1988, since 1987/88 had the largest number of Election Studies, and since it represents the mid-point in the timespan of the election studies in the analysis (1968–2006).

** Data obtained from 〈http://gapminder.org〉, a non-profit organization that works closely with the UN and Google to promote achievement of United Nations Millennium Development Goals.

then, each individual's presence in the dataset more closely reflected a random sample than it did before I generated these sampling weights.4

By creating these sampling weights and incorporating them into the analyses conducted, we are able to ensure that no single election study (in particular Annenberg's survey in 2000 with over 50,000 respondents) swamps the dataset and the analyses, and that each respondent's weight within the larger dataset reflects the country's population in relation to the total population of all of the countries in the study.

The primary goal of this study is to update and unify a diverse and inconclusive literature on the evaluation of party leaders. In order to do this, it is imperative that a large, cross-national, and longitudinal analysis be conducted in contrast to bulk of research that has taken place to date, consisting of analyses of single elections or a few elections in a single country. While existing studies have provided us with important insight into the role of leader evaluations, many questions remain.

(p.154) This study combines data from thirty-five election studies across seven countries. As such, it involves a massive amount of data management, recoding, and troubleshooting in order to make the project workable. Many of the challenges that have arisen are inherent in any comparative project involving survey data where different questions are asked in different countries and different years. Some of the challenges, however, were specific to the issue of the comparative analysis of leader evaluations. This Appendix describes the challenges that were applicable to the study as a whole—other analysis-specific hurdles are addressed in the chapters in which those analyses are presented.

Notes:

(1) The complete list of studies includes: the Canadian Election Study (including 1968, 1984, 1988, 1993, 1997, 2000, 2004, and 2006), the National Election Study (from the United States, including 1972, 1976, 1980, 1984, 1988, 1992, 1996, 2000, and 2004), the National Annenberg Election Study (United States, 2000), the British Election Study (including 1983, 1987, 1992, 1997, and 2001), the Australian Election Study (including 1987, 1993, 1996, 1998, 2001, and 2004), the New Zealand Election Study (including 1999 and 2002), the German Election Study (including 1980 and 1987), and the Swedish Election Study (including 1988 and 1991).

(2) Results not shown.

(3) For a more detailed discussion, see Kish (1965), Kalton (1983), and Sudman (1983), who provide both theoretical and practical information about various aspects of sampling.

(4) This weighting scheme does weight the United States, non-parliamentary context heavier than the parliamentary context because of the size of the American population compared to the other countries. The risk, then, is that the US case is swamping the others, thereby skewing the analyses. Other weighting formulas were considered—for example, an alternative was to give each country equal weight, but then the US 2000 NAES would overwhelm the American population. If we were to give each election study equal weight, then those countries with a larger number of election studies (the United States and Canada) would dominate the sample. If we gave each country equal weight and then each study equal weight within the country, then there would still be some election studies weighted more heavily than others—namely, those countries with fewer election studies in the sample (Germany, Sweden, and New Zealand). Regardless of the weighting scheme chosen, tradeoffs are involved. To ensure that the results of the analyses were not dominated by the more weighty American and non-parliamentary context, they were all re-run without the American Election Studies included. The results were encouraging. The patterns described in the rest of this book continued to be present with the US case removed from the analysis. In many instances, the patterns observed were actually stronger without the opinions of American respondents. These results suggest, first, that the weighting scheme deployed does not result in an overstatement of the impact of party leaders (or in fact, making the evaluations of candidates skew our understanding of the evaluations of leaders). Second, the results also give further credence to the fact that evaluations of party leaders play a role in Parliamentary systems, where the role of leaders has not traditionally been thought to be as pronounced.