Jump to ContentJump to Main Navigation
Mapping Policy Preferences From TextsStatistical Solutions for Manifesto Analysts$
Users without a subscription are not able to see the full content.

Andrea Volkens, Judith Bara, Ian Budge, Michael D. McDonald, and Hans-Dieter Klingemann

Print publication date: 2013

Print ISBN-13: 9780199640041

Published to Oxford Scholarship Online: January 2014

DOI: 10.1093/acprof:oso/9780199640041.001.0001

Show Summary Details
Page of

PRINTED FROM OXFORD SCHOLARSHIP ONLINE (www.oxfordscholarship.com). (c) Copyright Oxford University Press, 2019. All Rights Reserved. An individual user may print out a PDF of a single chapter of a monograph in OSO for personal use. date: 18 November 2019

Coder Training: Key to Enhancing Reliability and Validity

Coder Training: Key to Enhancing Reliability and Validity

(p.169) 9 Coder Training: Key to Enhancing Reliability and Validity
Mapping Policy Preferences From Texts

Andrea Volkens

Judith Bara

Ian Budge

Michael D. McDonald

Robin Best

Simon Franzmann

Oxford University Press

Aside from document collection the major procedure involved in producing the policy estimates is coding of the ‘quasi-sentences’ in the documents into 57 policy categories. Here the concern is that different coders operating across different time periods and countries make the same coding decisions. Difficulties in comparing the coding units identified by different coders has led some outside evaluations to focus on inter-coder agreement on assignment of pre-set coding units, as used in CMP and MARPOR training tests. Properly simulated these show that coders exhibit moderately high reliability at this stage in their training. The simulation also yields the valuable insight that bad practices creep in after coding around 10 documents, so refresher training is required at that point. An independent check-coding experiment in which different coders of manifestos in four countries shows an overall correlation of 0.82 on RILE, confirming the reliability level identified from the final estimates in Chapter 6.

Keywords:   coder training, production coding, measuring coder reliability, enhancing reliability, stochastic variation, experimentation

Oxford Scholarship Online requires a subscription or purchase to access the full text of books within the service. Public users can however freely search the site and view the abstracts and keywords for each book and chapter.

Please, subscribe or login to access full text content.

If you think you should have access to this title, please contact your librarian.

To troubleshoot, please check our FAQs , and if you can't find the answer there, please contact us .