Jump to ContentJump to Main Navigation
How Matter MattersObjects, Artifacts, and Materiality in Organization Studies$

Paul R. Carlile, Davide Nicolini, Ann Langley, and Haridimos Tsoukas

Print publication date: 2013

Print ISBN-13: 9780199671533

Published to Oxford Scholarship Online: May 2013

DOI: 10.1093/acprof:oso/9780199671533.001.0001

Show Summary Details
Page of

PRINTED FROM OXFORD SCHOLARSHIP ONLINE (www.oxfordscholarship.com). (c) Copyright Oxford University Press, 2020. All Rights Reserved. An individual user may print out a PDF of a single chapter of a monograph in OSO for personal use. date: 14 July 2020

Knowledge Eclipse: Producing Sociomaterial Reconfigurations in the Hospitality Sector

Knowledge Eclipse: Producing Sociomaterial Reconfigurations in the Hospitality Sector

Chapter:
(p.119) 6 Knowledge Eclipse: Producing Sociomaterial Reconfigurations in the Hospitality Sector
Source:
How Matter Matters
Author(s):

Wanda J. Orlikowski

Susan V. Scott

Publisher:
Oxford University Press
DOI:10.1093/acprof:oso/9780199671533.003.0006

Abstract and Keywords

Drawing on a field study of the travel site TripAdvisor, the authors explore how online reviewing, rating, and ranking mechanisms are overshadowing traditional configurations of knowledge in the hospitality sector by redistributing resources, shifting practices and habitats, and redefining what counts, who counts, and how. The authors suggest that such sociomaterial reconfigurations offer important insights into the broader issues associated with the role of social media in knowledge practices, and the ways in which expert valuation schemes are being eclipsed by ones grounded in user-generated content. They maintain that these different valuation schemes entail different kinds of work, producing different valuations of the real, and enacting different (singular and multiple) realities. As such, these reconfigurations of valuation raise not just important epistemological issues but also critical questions of ontology and accountability.

Keywords:   social media, knowledge, valuation, ranking mechanisms, user-generated content, sociomaterial practice

Contemporary knowledge practices are currently undergoing a significant shift as the rise of social media and online user-generated content is spreading into ever more domains of everyday life. Most of us already routinely consult Wikipedia as a valued source of knowledge. Others turn to online forums or Twitter to obtain advice on technical difficulties. And product and service recommendations are now commonly sought on websites such as Amazon, Yelp, Netflix, Angie’s List, and TripAdvisor. Such sources of knowledge and advice—dependent as they are on the spontaneous contributions of large numbers of often-anonymous people distributed around the world— (p.120) raise questions about both the value of such knowledge, and its manner of valuation. We explore these questions in this chapter, focusing in particular on the hospitality sector and the recent rise in importance of user-generated knowledge about hotels through the TripAdvisor social media website. We argue that the reconfigurations of valuation exemplified by the case of TripAdvisor offer important insights into the broader issues associated with the role of social media in knowledge practices and the eclipsing of expert valuation schemes by ones grounded in user-generated content.

There has been a growing interest in the concept of value and in the processes of valuation (see Espeland and Stevens, 1998; Callon and Muniesa, 2005; Stark, 2009; Karpik, 2010; Hsu et al., 2012) and the role these play in structuring markets. This research has examined the influence of critics, standards, ratings, rankings, and judgment devices in guiding consumers’ choices and actions and shaping producers’ strategies, work practices, and pricing decisions. The rise of the internet and the recently burgeoning arena of social media have amplified the scale, scope, and reach of online valuation schemes. While the novelty and effect of such web-based amplification has been recognized—often referred to as “the wisdom of the crowds”—there are few detailed studies of the practices and technologies that are intricately involved in producing online valuation and their consequences for knowledge and outcomes (for a few exceptions, see David and Pinch, 2008; Jeacle and Carter, 2011).

Our research interest is in understanding the reviewing, rating, and ranking mechanisms of one such online valuation scheme—TripAdvisor—and how these are overshadowing traditional configurations of knowledge in the hospitality sector. We focus specifically here on how the practices, technologies, and outcomes of online forms of hotel valuation differ from more traditional forms of valuation, how these are reconfiguring the hospitality sector, and with what consequences.

6.1 Literature on Valuation

Valuation is critically entangled with practices of consumption and production, and as such is central to social reproduction and transformation (Willmott, 2010). Much has been written about the role of various mechanisms of valuation—prices, standards, benchmarks, brands, reviews—that allow products and services to be identified, assessed, and exchanged (Espeland and Stevens, 1998; Callon and Muniesa, 2005; Stark, 2009; Karpik, 2010; Willmott, 2010; Hsu et al., 2012). These mechanisms are particularly (p.121) significant in the case of goods and services that have uncertain market value and where multiple criteria and assessments of value are present. Karpik (2010) argues that because such goods and services cannot be easily evaluated through strictly calculative mechanisms, “judgment devices” are needed to guide people to make purchasing decisions under conditions of market uncertainty. Such devices are produced by multiple different actors such as producers, sellers, advertisers, media, and authorities and include brands, rankings (expert and popular), networks (personal and impersonal), and critics.

The role and influence of critics is central to Hsu et al.’s (2012) examination of the market for wines, a class of product for which “one cannot know the quality of a good until it has been purchased and consumed,” and thus where “consumers must rely on critics’ published reviews as proxies for hidden product quality” (Hsu et al., 2012: 83). Critics’ ratings also influence producers, “who must adjust resource allocations, divest business units, and adopt specific missions in order to conform to the standards espoused by these mediators, precisely because of their subsequent influence over consumers.” Focusing on pricing, their analysis suggests that the clearer the valuation scheme that is employed by critics, the stronger the relationship between producers’ prices and the quality ratings assigned by the critics. In shaping knowledge about quality, and thus the prices that are set and realized in practice, critics configure both producers and consumers and their practices of production and consumption.

In a series of studies examining the role and influence of ranking mechanisms on practices of production and consumption, Espeland and her colleagues (Espeland and Stevens, 1998; Espeland and Sauder, 2007; Sauder and Espeland, 2009) have shown how law school rankings produced by the U.S. News & World Report have shaped those schools over time. Drawing on detailed fieldwork, they find that the rankings normalize and structure the law schools as particular, standardized entities making them amenable over time to certain forms of manipulations and interventions. As Sauder and Espeland write:

By imposing a shared metric on law schools, rankings unite and objectify organizations, reinforcing their coherence as similar objects…. Rankings have become naturalized and internalized as a standard of comparison and success. In changing how law schools think about themselves and pressuring schools toward self-discipline, rankings are now deeply embedded within schools, directing attention, resources, and interventions. (Sauder and Espeland, 2009: 80)

Jeacle and Carter (2011) also examine ranking mechanisms, in their case the online hotel reviewing website, TripAdvisor. Drawing on Giddens’ (p.122) (1991) notion of abstract systems, they argue that TripAdvisor may be understood as an “expert system” which entails “inter alia the calculation of phenomena, the labelling of entities, the process of commensuration, and the ranking and ordering of entities” (Jeacle and Carter, 2011: 4). They see TripAdvisor as a “calculative regime,” exerting significant influence within the social structure of the travel industry (2011: 11). They argue that its power is located within the calculative practices constituting the Popularity Index algorithm, a ranking mechanism that orders hotels within a particular region. They note that this algorithm

provides the independent traveller with a clear and objective form of quantification from which to make hotel discriminations, and as such is invested with the objectivity and impartiality that are oft attributed to numbers…. [A] single number instantly labels the perceived quality of an establishment and that number is invested with credibility all the more so because it was constructed from the experiences and seemingly honest opinions of fellow travellers. Moreover, the Popularity Index places hotels in hierarchical relationships to one another—highlighting which hotels are ‘better’ or ‘worse’. As an expert system, the rankings convert the numerous individual ramblings on the site into hard and objective fact. (Jeacle and Carter, 2011: 9)

The importance of rating and ranking mechanisms, algorithms, and materiality more generally, in shaping valuation schemes and outcomes is strongly evident in Callon and Muniesa’s (2005) work on the calculative character of markets. They argue that valuation is a three-step process that is “distributed among human actors and material devices” (2005: 1245). They suggest that it proceeds as follows: first, the entities to be assessed have to be “detached”—moved, arranged, and ordered in a single “calculative space”; second, the entities have to be “taken into account,” associated and subjected to certain manipulations; and third, a new entity—the “result”—has be produced that corresponds to the manipulations performed in the calculative space and is able to circulate widely “without taking with it the whole calculative apparatus” (2005: 1231). For Callon and Muniesa (2005: 1232), such valuation “establishes a continuum between qualitative judgment and quantitative (or numeric) calculation,” a process they refer to as “qualculation.” The definition of qualculation is aimed at “collaps[ing] the distinction between the quantitative and the qualitative,” and indicating its inescapable materiality. Thus, qualculation is “impossible without material arrangements: paper and pencil; the benches in a court of law; a system for tallying arrivals and departures; a supermarket” (Callon and Law, 2005: 719).

Authors such as Haraway (1988), Suchman (2002a, 2002b, 2007), and Barad (2003, 2007) remind us that integral to the materiality of valuation (p.123) schemes foregrounded by Callon and Muniesa (2005) is the notion of location. This is a sense of positioning that is not confined to time and space but embraces the idea of accountability, specifically located accountabilities (Suchman, 2002a). The detachment upon which calculative apparatus depend can have the effect of de-realization, of producing valuations that are dissociated and disconnected from located practices and responsibilities. Whether they involve a laboratory, theory, technology, or metric, established valuation schemes provide distance from practicalities, but not lasting or universal absolution. In order to identify what constitutes value and for whom, we need to first acknowledge that we are analyzing a series of “partial, locatable, critical knowledges” (Haraway, 1988: 584). If knowledge is located and partial, then by implication there is no unmediated source of knowledge and no neutral process of knowledge production (Law and Urry, 2004).

Treating all knowledge as located and partial enables us to make two analytical moves. First, when framing valuation schemes from this perspective we readily identify and are receptive to the coexistence of multiple forms of knowledge. Second, we are sensitized to the particular agential cuts made by the qualculative apparatus in place (Barad, 2007); in other words, when organizing what counts as value we ask what gets included and what gets excluded from the process of knowledge production. This process of analysis draws attention to the differences that the agential cuts (inclusion/exclusion) enacted by particular valuation schemes make in practice. The agential cuts produced by the process of qualculation are performative, as Moser and Law (2006: 67) note:

Something is being made that was not there before. To put it differently, what counts as information is (and necessarily) being bounded in a new way. And it [is] this bounding, this simplification, this practical setting of limits, that renders qualculation possible. The decisions of qualculation determine what will perform as information and what will not. And this limit-setting is indeed practical: how the limits are done depends on the task at hand. What is included and excluded, likewise. That is what the bounding is about.

We turn now to a brief consideration of shifting historical conventions and practices in genres of valuation within the hospitality sector. We then compare two different forms of valuation currently operating within this sector—the standardized, institutionalized and expert-based valuation scheme produced by hotel grading agencies such as the Automobile Association, and the emergent, online, and user-based valuation scheme produced by social media review websites such as TripAdvisor.

(p.124) 6.2 Genres of Valuation within the Hospitality Sector

In examining the shifting role and nature of knowledge about hotels, we need to consider these within a historical context, and so we briefly trace the development of travel writing genres across a range of forms, norms, times, and places. This review indicates that travel writing has undergone a series of shifts in practices, technologies, and conventions of valuation over time, generating significant changes in the kind of travel knowledge produced, by whom, with what purpose, and with what outcomes. These changes entail different forms of valuation that have been used, and continue to be used, to assess and legitimate knowledge about hospitality in practice.

Writing about and for travel has been traced back to the peripli,1 ancient documents used by Phoenician, Greek, and Roman navigators that listed the ports, landmarks, and distances that vessels could expect to find along a coastline. Versions of these for journeys on land—such as the Roman itineraria—provided road maps with distances that identified the cities, villages, and landmarks that would be encountered on route. More elaborate documents were also evident, as in Pausanias’ Description of Greece, written in the second century, which offered first-hand descriptions of the art, sculpture, and architecture of ancient Greece. In later centuries, guides were used to help pilgrims on their religious journeys. One well-known example in this genre was the series of letters written in the fourth century by a Gallic woman, Egeria. Known as the Travels of Egeria, it recounts her pilgrimage to the Holy Land during the years 381–384, and offers detailed descriptions not just of her route but also of the holy sites, monuments, hospitality, and liturgical customs she experienced during her journey.

The pocket travel guide, more familiar to contemporary travelers, emerged in the nineteenth century. Early examples of this genre included Samuel Latham Mitchill’s 1807 guide to New York City, The Picture of New York,2 Gideon Minor Davison’s guidebook to Saratoga Springs, Niagara, Quebec, and Boston published in 1822 as The Fashionable Tour, and Mariana Starke’s advice to Britons traveling to France and Italy, published in 1824 with the title Information and Directions for Travellers on the Continent. Starke’s books, in particular, served as an early model for the modern guidebook, including practical information about passports, luggage, and the cost of food and accommodation in each city.3 She even devised an early rating system, using exclamation marks to indicate the value of particular sights. As she notes in the introduction of her book (Starke, 1826: v), “In the following pages, the Reader will find that several of these (p.125) works of Art are distinguished, according to their reputed merit, by one or more exclamation-points.”

Many other travel guides followed, drawing on the journals and travelogues of adventurers such as Juliette Star Dana, Isabella Bird, and John Lloyd Stephens, and offering specific details about first-hand observations and individual experiences. The genre of travel writing was further developed by publishers such as Ward & Lock in London, John Murray III in Scotland, and Karl Baedeker in Germany, who strove to distinguish subjective accounts of individuals’ travel from objective and factual guides to travel destinations. Their efforts served to produce two genres of travel writing—the personal travelogue and the impersonal travel guide. As more and more people began traveling abroad in the nineteenth and twentieth centuries, the latter guides grew in popularity, becoming widely recognized as arbiters of aesthetic and cultural judgment and indispensable to travelers venturing overseas. Gretton (1993) writing about the Murray Handbooks for Travellers quotes from an 1850 London Times review of the red-covered Handbooks:

Mr. Murray has succeeded in identifying his countrymen all the world over. Into every nook which an Englishman can penetrate he carries his Red Handbook. He trusts to his Murray as he would trust to his razor, because it is thoroughly English and reliable; and for his history, hotels, exchanges, scenery, for the clue to his route and his comfort by the way, the Red Handbook is his “guide, philosopher, and friend”.

In a similar vein, novelist E. M. Forster has his heroine in A Room with a View travel from England to Florence in 1908, taking along with her a Baedeker Guidebook to Italy so that she would know “what was beautiful, and what should be ignored” (Sattin, 2008). While Murray and Baedeker guides were primarily used to aid tourists abroad, they could also be used for more nefarious purposes, as Tisdall (2007) observes:

On a more sinister note, in 1942 Exeter, Bath, Norwich, York and Canterbury were all blitzed in what have become known as the “Baedeker Raids”—a reference to the words of the Nazi firebrand Baron Gustav von Sturm, who reputedly declared “We shall go all out to bomb every building in Britain marked with three stars in the Baedeker guide.”

Travel guides in the twentieth century expanded and grew to cover hundreds of different locations around the world. Well-known brands such as Fodor’s, Frommer’s, Rough Guides, Lonely Planet, and Let’s Go emerged and thrived. These organizations each have a distinctive and institutionalized “house style,” which directs the data gathering and writing activities of (p.126) the multiple individuals employed to produce and publish travel guides about particular destinations. While the format of the guides and the number of contributors have changed over time, the purpose of the guides remains markedly similar to those characterizing those of Starke and Baedeker—to provide advice and recommendations to travelers visiting unfamiliar places. As such, the knowledge in these guidebooks focuses on highlighting the specific landmarks and accommodations that are deemed worthy of endorsement and likely to produce positive experiences for the readers/travelers.

Emerging alongside these branded travel guides are guides that specialize in professional assessments and accreditations of different types of accommodation, produced by organizations such as Michelin, the RAC (Royal Automobile Club), and the AA (Automobile Association). These organizations employ full-time, trained inspectors who rate hotels according to various specifications and assign stars or letter grades to signify the quality and value of accommodation to be had in those hotels. The evaluation of hotels is based on extensive criteria that have been standardized in the hospitality industry over years and which involve routine and incognito inspections of hotels. These professionalized assessments are seen to offer the very best in objectified knowledge for prospective travelers, as noted in the descriptions of two such schemes—the AA4 and Michelin5 respectively:

Hotels or guest accommodation assessed by the AA are rated under a set of common quality standards agreed by the AA and the UK tourist authorities (VisitBritain, VisitScotland and VisitWales)…The common standards make types of accommodation easy to understand and give you confidence that establishments meet the standards you require.

A team of full-time professional restaurant and hotel inspectors anonymously evaluate establishments according to a rigorous set of criteria. Only the best restaurants and hotels are included in the guide, where every establishment is a trusted Michelin recommendation.

The most recent genre of travel writing is emerging within social media reviews—descriptions of and opinions about hotels and locales posted online by anonymous users of such websites as TripAdvisor, Yahoo!Travel, Expedia, Hotels.com, etc. Most of the content included in social media websites for the hospitality sector is provided by users of the website. Such content is growing in both size and influence. Studies suggest that social media reviews are having a substantial impact on the decision-making behavior of travelers planning and booking accommodation (Starkov and Price, 2007; Vermeulen and Seegers, 2009; Xiang and Gretzel, 2010). Almost (p.127) half of all consumers making travel purchases base their decisions on online content, using it to get ideas, narrow choices, and confirm their selections (Gretzel and Yoo, 2008). Most of these consumers report that user-generated content is more relevant, reliable, and enjoyable than the information provided by traditional travel service providers such as travel agents and guidebooks (Gretzel and Yoo, 2008). As a result, the once thriving market for printed travel guides is in retreat. Sales of travel guides globally have declined 10 percent every year since 2008 (Mesquita, 2011).

In this most recent genre of social media reviews, we see a return in content if not form to the individual recounting his or her first-hand travel experiences and observations. The content is personalized, subjective, and experiential—akin to the travelogues produced by early writers, which had been eclipsed by the standardized, institutionalized genres of modern travel guides. Unlike earlier genres, however, user-generated, online reviews include content that is both positive and negative (no longer just focusing on highlights and positive recommendations), and are produced anonymously (no longer attributable to a named individual or organization whose reputation and expertise could be identified, verified, and held to account).

6.3 Research Setting and Methods

In seeking to understand the valuation scheme provided by the TripAdvisor social media website, we elected to compare it to the traditional hotel valuation scheme produced by the UK-based Automobile Association (the AA). We chose the AA scheme as it is long-standing, widely regarded, highly professionalized, and particularly influential in practice. It thus offers a useful strategic contrast to the nascent and user-based valuation scheme of TripAdvisor.

We collected data from a number of research sites: the AA, TripAdvisor, various hotels affected by both valuation schemes, and the hospitality industry more generally. We conducted 55 interviews across these sites (see Table 6.1), ranging from 45 minutes to 2 hours. Nearly all interviews were recorded and transcribed. We also examined a large amount of documentation—reports, procedures, standards, trade press articles, blog discussions, newsletters, archival materials, and hundreds of reviews of multiple hotels—from the AA, TripAdvisor, hotels, and the hospitality industry. In addition, we attended a number of academic conferences and trade events focused on the hospitality sector, and participated as users and contributors on the TripAdvisor website. (p.128)

Table 6.1 Number and Type of Interviews

Number of interviews

Positions

Hoteliers

21

12 owners; 9 managers

AA employees

11

2 executives; 4 editors; 5 inspectors

TripAdvisor employees

14

3 executives; 5 directors; 6 managers

Industry professionals

9

British Hospitality Association

English Tourist Board

Institute of Hospitality

Association of British Travel Agents

VisitBritain

VisitEngland

Travel Guide Writers/Editors

Total

55

Our data analysis was exploratory, and followed a grounded theory approach (Glaser and Strauss, 1967; Dougherty, 2002; Eisenhardt and Graebner, 2007), beginning with careful reading and content analysis of the interview transcripts, observations, evaluations, and other documentary materials. Our process was inductive and iterative, focused on identifying threads associated with the nature, influence, and implications of the two hotel evaluation schemes in practice. We anticipated that in the production of knowledge about hotels, the calculative apparatus relating to each valuation scheme would make a difference in what was valued, how, and with what consequences. Our data analysis thus focused on examining this in detail.

Our analysis generated important insights into the differences in hotel assessments performed by the TripAdvisor and AA hotel evaluation schemes. We found that TripAdvisor’s user-generated ratings and rankings produce a different form of valuation of hotels than those generated by the AA’s standardized and professional inspections. These differences in valuation have significant implications for hoteliers’ strategies and practices. Accustomed and entrained to receiving routine, annual evaluations based on standardized criteria performed by trained inspectors, hoteliers now have to contend with the uncertainty and ambiguity of reviews and ratings performed by large numbers of anonymous, online users (who may or not have visited the hotel and experienced its facilities and services).

6.4 Valuation of Hotels

Before comparing the traditional AA and emergent TripAdvisor hotel valuation schemes, we first offer a brief account of each of the scheme’s strategies and practices.

(p.129) 6.4.1 Two Valuation Schemes

The UK-based Automobile Association was founded in London in 1905 by a small group of motoring enthusiasts, initially to help each other avoid speeding fines and subsequently to provide a wide range of travel services to its now 15 million members, including breakdown and road assistance, route maps, auto insurance, financial services, hotel accreditation, and travel guides. The AA is one of the UK’s most widely recognized and respected brands, and in 2004 it was acquired by a group of private equity funds for $3 billion. The AA has been evaluating hotel accommodation since 1909, and for its most recent 2012 Guides, its 30 professional inspectors visited and evaluated almost 4,000 hotels throughout the UK.

The AA hotel valuation scheme is based on a well-defined set of standards, criteria, and metrics that have been developed, honed, and configured over time through detailed, on-the-ground practices, inspections, and audits, and changed in relation to shifting norms and expectations of the hospitality industry, tourist authorities, consumers and hoteliers. Updated over time, these standards are today defined in the AA’s Hotel Quality Standards, a 65-page brochure that articulates the specific standards for facilities and services that must be met for a hotel to be awarded a particular star rating. The AA has an explicit system of star ratings—from one to five stars designating different classes and qualities of accommodation. This system uses black stars to denote the level of award and red stars to distinguish those hotels deemed “exceptional” at a certain level. The AA also distinguishes between types of accommodation—having different criteria and standards for hotels, B&Bs, inns, guesthouses, etc.

The process of valuation involves each hotel being visited annually by one of the AA’s 30 full-time inspectors. Inspectors are hired from the hospitality industry and are then thoroughly trained over nine weeks in the inspection procedures of the AA, including the meaning and definition of the standards and criteria, the use of the “AA Dashboard”—a spreadsheet that incorporates hundreds of quantitative metrics and qualitative assessments structured in terms of facilities (e.g., bedrooms, bathrooms, lobby, etc.), services (e.g., reception, room service, food quality), and operational issues (e.g., levels of cleanliness, furnishings, maintenance, etc.). These ratings are based on the quality standards for each level of star award (from one to five). Inspectors’ training, which includes a period of shadowing where more junior inspectors accompany more senior ones on their inspections, teaches the inspectors how to perform an inspection, what to look for, what to include in their notes, what to exclude.

(p.130) Hotel inspections are usually conducted as an overnight stay, with the inspector booking and staying incognito. During his or her visit to the hotel, the inspector performs a detailed evaluation of the hotel that includes capturing data about its facilities and services within the AA Dashboard operating on the inspector’s laptop computer. At the end of the hotel stay, the inspector uses the Dashboard to generate a report that computes a rating for each of the key categories of facilities and services and produces an overall graph summarizing the hotel’s performance. This report forms the basis of the star rating that is awarded to the hotel, and the face-to-face discussion the inspector has with the manager after checking out of the hotel at the end of his/her stay.

Within three days of the inspection, the inspector sends one copy of the final report to the hotel manager and one to the editor of the AA guides, who will use this report to prepare the review published about the hotel in the annual AA travel guides. Hoteliers may seek clarification about their evaluation and even challenge their star rating. A re-evaluation may then be performed to ensure validity and consistency of rating. The AA also periodically performs “benchmarks” when multiple inspectors gather at a single hotel, individually perform their inspections, and then meet to compare their scores and ratings. In addition, and as part of a staff appraisal process, senor inspectors will periodically call hoteliers to get their feedback on the inspection process.

We turn now to the TripAdvisor hotel evaluation scheme. TripAdvisor is a social media organization founded in Boston in 2000 by four software entrepreneurs, and its mission is to “Help travelers around the world plan and have the perfect trip.” After a shaky first year, TripAdvisor’s growth has been rapid, and it now hosts the world’s largest online travel website, including over 60 million user-generated reviews and opinions on approximately 1 million hotels, restaurants, and venues. TripAdvisor operates websites for 30 countries with content available in 18 languages, and these receive almost 50 million unique users per month. The company currently employs just over a thousand people located around the world, and in 2010 it generated revenues of nearly $500 million, derived from “click through” fees, advertising, and hotel listings.

Hotel reviews on TripAdvisor are produced by its 20 million members who have established a pseudonymous profile with the website. A review includes both qualitative assessments (in the form of free text comments) and quantitative ratings on six criteria (value, rooms, location, cleanliness, service, and sleep quality—none of which are defined). Members may also post their own photos to accompany their reviews. TripAdvisor uses the (p.131) quantitative ratings to compute a Traveler Rating—an overall rating across all reviewers for each hotel—and the Popularity Index—a ranking of all hotels within a specific geographic city or region. Both the Traveler Rating and Popularity ranking are highly influential in practice, and many travelers pay considerable attention to these scores.

Hotel reviews on TripAdvisor have recently been attracting considerable attention as stories in various media outlets—NBC’s Today, the New York Times, the Guardian, the Telegraph, and the Caterer and Hotelkeeper—draw attention to apparently false or malicious content. With over 60 million reviews and opinions on the website, verification and validation of content is clearly a difficult task. TripAdvisor uses both humans and tools to screen reviews, but problems persist. Both hoteliers and the press report concerns about the accuracy and authenticity of content that is unregulated and pseudonymous. They point to multiple reviews that contain defamatory comments or incorrect information, as well as to job posting websites that seek to hire people to post (apparently fabricated) hotel reviews for $5/review. These concerns about the quality and liability of TripAdvisor content have also triggered legal and regulatory scrutiny, including a number of lawsuits and an inquiry by the UK Advertising Standards Authority.

6.4.2 Comparing Valuation Schemes

In comparing the traditional practices of hotel evaluation performed by the AA with those performed by the TripAdvisor social media website, we found important differences in what was valued, how, and with what consequences. Our approach to understanding these schemes is to consider the arrangements through which different concepts of valuation are expressed not as merely embodied in apparatus but as enacted through practice. Each scheme is an apparatus that enacts what matters and what is excluded from mattering (Barad, 2007: 148). These differences are produced by the different agential cuts enacted by the AA and TripAdvisor valuation schemes in practice. Table 6.2 offers a sense of these differences in practice.

For example, when a hotel inspector makes observations in a hotel during an overnight mystery guest stay, he or she focuses attention on matters that fall within the criteria of the AA standards and pays no attention to those that don’t. So whereas an entry in the AA Hotel Guide based upon a sample of rooms inspected may describe them as generally “well appointed,” a posting on TripAdvisor may include a “room tip” that specifically categorizes room 312 as “overlooking the road with street noise at the weekend.” An AA inspector takes into account 20 percent of room stock (p.132)

Table 6.2 The AA and TripAdvisor Valuation Schemes

The AA

TripAdvisor

Practices

Dense, long-standing practices (a hundred years) honed by an institutionalized association (e.g., awarded a royal warrant), with a widely respected brand.

Light-weight, recent practices (ten years) of a start-up organization, that has developed an innovative and emergent social media brand on the internet.

Valuation practices entail routinized, regular, and regulated activities based on well-defined, standardized expertise embodied in a well-trained inspectorate.

Valuation practices entail ongoing experiences of consumers reflecting multiple, diverse, and dynamic ways in which they do (or do not) experience the hotel.

Inspections are done cyclically, are planned in advance, and subject to audit.

Reviews are posted spontaneously, at any time, and are not subject to audit.

Knowledge

Depth of knowledge of hotels—in terms of criteria, standards, facilities, and instances, articulated in the 65-page Hotel Quality Standards guide for star ratings.

Breadth of knowledge of hotels—in terms of criteria, standards, facilities, and instances, represented in 6 undefined criteria, a free-text box, and a facility to upload photos.

Each hotel receives an annual inspection conducted by a trained professional who has experienced many hundreds of hotels in his/her career.

Each hotel receives multiple, ongoing reviews by many untrained users who have each experienced a few hotels over the course of their lives.

Inspection reports are formal and standardized, and discussed immediately and face-to-face with hoteliers.

Opinions are casual and informal with reviews posted at varying times and at a remove and distance from the hoteliers.

Controls

Inspections are verified and regulated through shadowing, benchmarking, follow-up feedback, and an audit process.

Reviews are examined for inappropriate content but not verified or regulated with respect to content veracity or reviewer credibility.

Hoteliers have recourse to appeal any valuation they are concerned about or do not agree with, and request a re-inspection.

Hoteliers have limited recourse to appeal a review, and typically can only act by responding online to the contested review.

Outputs

Inspection reports are edited by professional writers for annual publication in the AA travel guidebooks

Reviews are posted to the website and aggregated via a proprietary algorithm into a real-time ranking for hotels within a region.

during his/her tour of the hotel conducted with the manager and is unlikely to stay on a weekend. Whereas we discover from an anonymous reviewer on TripAdvisor that last Saturday night there was a brawl on the street below his hotel room. The boundaries defining what comes to matter are enacted differently by the AA and TripAdvisor.

The agential cuts made by each of the two hotel evaluation schemes designed to produce knowledge about travel are significant not only (p.133) because of their capacity for making boundaries, but also because they enact change differently. This involves the reconfiguration of practices and actors but extends to materiality as well. We propose two metaphors with which to think about the outcomes involved here. On the one hand, we have institutionalized knowledge production through the AA. This process of knowledge production may be termed fluvial, a notion taken from geology to denote the formation of landscapes by the interaction of flowing water, carried through multiple regions, and across many levels and channels. In this way, the standards and criteria have flowed through the hospitality sector making differences over time. For example, to qualify for government business development grants in the 1970s and 1980s, UK hotels had to sign up with a recognized hotel accreditation scheme that would tie their practices to inspection processes and industry standards.

On the other hand, we have TripAdvisor’s rating and ranking valuation scheme that has developed over a shorter timeframe and is structured by a different set of interests, priorities, practices, and technologies. TripAdvisor’s founding strategy was tied to developing a travel portal for travel intermediaries, but over time it has evolved to servicing consumers directly by both providing them travel information they find useful and offering them the opportunity to share information about their own travel experiences. The operation of TripAdvisor’s business model relies on a constant supply of user-generated content provided as part of consumers’ ongoing practice of traveling and posting to the website. This content is aggregated by algorithms that combine users’ subjective interpretations and assessments of the quality, value, and service that they have experienced at particular times in specific hotels. The knowledge thus produced by Trip Advisor is a mix of user views, algorithms, content management, weighted priorities, and filtering practices that compute specific ratings and rankings about hotels within specific regions.

This knowledge about hotels produces what Giddens (1991:26) has termed the “collage effect”—a dynamic juxtaposition of loosely linked items that is a central aspect of contemporary experience. TripAdvisor offers us review after review after review for any particular hotel, across hundreds of thousands of hotels—a process of knowledge production that may be termed fractal. Over time, such knowledge becomes progressively recursive and self-referential, as ratings are disconnected from any benchmarks or industry standards, reviews reference other reviews, reviewers copy themselves and each other, and the genre norms shape conformity in form and content. While the production and consumption of this (p.134) knowledge is distributed and fragmented in time/space, it is always virtually present within a single time/space on the website.

In both the AA and TripAdvisor hotel valuation schemes, qualculation is being enacted as an intra-action (Barad, 2007). The specific details of this qualculation make a difference to how this intra-action is enacted in practice. For example, the AA qualculation works through industry standards, spreadsheet metrics, commensurable datasets, and a professional narrative involving identifiable relations of accountability that are threaded through its practices. The qualculation of TripAdvisor involves intersecting algorithms with non-standard and variably weighted metrics sorting through a multiplicity of personal narratives creating the perception of commensurability, all managed with a light form of (mainly) automated validation mechanisms.

Together these two hotel valuation schemes—the institutionalized star accreditations and the polysemous voices of the customer—have come to constitute the material-discursive practices characterizing value in contemporary travel. The AA configures value through its centering practices that produce fluvial knowledge honed over time, including objective criteria, professionalized standards, trained inspectors, a single dashboard, benchmarking and audit processes, and relations of accountability. A single report, a single star rating, and a single guidebook entry define an AA accredited hotel. TripAdvisor configures value through both decentering and centering practices. On the one hand, its unregulated reviews posted by many, anonymous users, using undefined criteria produce fractal knowledge that defines value in terms of the crowd—a collective, distributed, heterogeneous multiplicity. On the other hand, TripAdvisor also configures value through its calculative algorithms, mechanisms that aggregate and order, erasing difference, reducing variability, and glossing over ambiguity, inconsistency, and duplicity to produce a single ranking of hotels within a region.

6.5 Discussion

Reflecting on the reconfigurations of travel knowledge through the years, we see not only that multiple forms of knowledge and valuation have emerged at different times but also that they entail different “strategies of coordination”—mechanisms that work to connect and coordinate disparate elements so as to enact singularity (Law, 2002: 15). Some of these—such as the two hotel valuation schemes we studied—work in a “looplike and (p.135) self-sustaining” (Law, 2002: 28) manner to hold certain things together in practice at particular times and places.

In the case of the AA hotel valuation scheme, and related accreditation mechanisms, the primary strategy of coordination is through standards. The careful development and ongoing maintenance of industry standards work to coordinate relations and practices of hoteliers and inspectors across time and place. Thus, the experiences of all guests are centered and coordinated through the inspector who draws on his/her trained experience and the AA dashboard to produce a single account of a single overnight stay. These valuation practices enact a single hotel out there, one that is assessed straightforwardly through institutional standards and the objective criteria of star ratings. The editors at the AA are conscious that this “cut” excludes the personal voice of the traveler but their efforts are focused on drawing together a guide that provides a certain standard of assurance to consumers. The report sent to the hotelier is not intended to coordinate guest feedback into a single source of truth, but rather to provide feedback to hoteliers about where and how their management strategies are aligned with industry standards and quality benchmarks, thus providing them with a mandate for action going forward.

In the case of TripAdvisor’s hotel valuation scheme, users’ reviews include open-ended ratings criteria and comments that are posted without verification or audit. These postings are de-centered in terms of institutionally recognized relationships such as standards, but they claim to be locatable in terms of the “truth” entailed in in situ accounts of how the hotel is performing at any point in time. Difference in the timing and placing of reviews produce differences in ratings, and disagreements and contradictions abound. Such valuation practices enact multiple hotels, one for each user who posts a review about his or her experiences of the hotel. Yet, alongside this user-generated multiplicity, TripAdvisor also produces singularity. Its strategy of coordination entails physical juxtaposition and algorithmic aggregation. The reviews of many anonymous users, posted at various times and places, are arranged alongside each other on the website. Its Popularity Index then performs a hierarchical ordering of these disparate ratings, achieving an algorithmically coordinated hotel that hangs together in the regional rankings displayed on the site.

Following Law (2002: 188–9), we suggest that the practices producing the AA Guidebooks enact a consistent and coherent story, one that “conforms to a tradition of continuity, of narrative,” whereas the practices involving TripAdvisor are more akin to setting up a pinboard—juxtaposing traveler photos and reviews of hotels on TripAdvisor, friends’ Facebook pages, (p.136) entries from accreditation schemes, digital guidebook recommendations, Google Earth satellite images, and online discussion forums—claiming these are more or less equivalent, “without the necessity of a single order…[or] same narrative.” Law (2002: 188) notes:

A story and a pinboard do different jobs of work. They exist in different worlds. Crucially, they also help to make different kinds of worlds. And it is the making that is interesting, the performativity of storytelling on the one hand and pinboards on the other

As he argues, these differences are consequential and raise important questions (Law, 2002: 189):

How might we think about this difference, the difference between the story and the pinboard? How might we think about the difference in the work that each does? How might we think about their performative effects?

The agential cut performed by the AA hotel valuation scheme produces a particular form of centered, locatable, relational knowledge that enacts institutionalized singularity. While this cut deliberately excludes accounts that matter to the everyday traveler, its purpose is less to efface the performances of the multiple than to provide assurance through relations and processes of accountability. The shift to social media reviews involves losing the illusion of singularity produced by institutionalized, expert-based, accreditation schemes, a singularity that enables consumers to be passive recipients of knowledge.

In contrast, social media websites such as TripAdvisor invite consumers to be active participants in knowledge construction, working through the multiple, different accounts posted on the site as well as contributing their own. Travelers making travel arrangements open multiple windows on their digital desktops, keeping TripAdvisor open as they navigate through booking engine websites and pull up hotel home pages in an effort to draw their own conclusions. While travelers lose the certainty of the AA’s specialized expertise and singular valuation, they gain insights through the creative tensions afforded by TripAdvisor’s juxtaposition of multiple, heterogeneous reviews. Of course, travelers need to develop the skill to engage these tensions and parse algorithmic ratings rather than be the passive recipients of them. But doing so may allow them to reclaim their status as active, critical travelers instead of having their experiences pre-digested and framed by industry experts.

Performing travel in the twenty-first century means experiencing the discomforting juxtaposition of uncertainty through the multiplicity (p.137) offered by TripAdvisor. This applies to hoteliers as much as it does to travelers. Hoteliers in our study say that they live with the anxiety of “unjust” or “skewed” postings, regarding every guest as a potential bad review in process, taking reviews into staff meetings, and struggling to decide whether or not to make management changes based on user-generated content. Travelers report spending more time booking travel, making reservations at hotels they would not have otherwise chosen, drawing in situ comparisons with reviews, and occasionally rueing both their choices and postings.

The rise of TripAdvisor marks the tensions characterizing contemporary processes of knowing: while we continue to search for what Law (2002) describes as “neat tree-like arborescence, for singular stories,” there is recognition that such singularities are illusory, one of multiple orderings in the world. In the hospitality sector, these definitive accounts and fragments coexist as each other’s creatures. The enthusiasm with which TripAdvisor has been received may lie in its ability to manifest this in practice. By design, diverse accounts and multiple orderings confront each other. One could argue that TripAdvisor is realizing the generative possibilities of juxtaposition through which travelers have an explicit experience of what it means to know in tension, and to identify possibilities for action in the creative dissonance that comes from reconciling rather than concluding (Stark, 2009).

By redrawing the boundaries of travel planning practice, TripAdvisor confronts us with difficulties that may have not troubled us before and brings us to ask questions about the basis for travel knowledge that have remained dormant for many decades. To frame this in theoretical terms, it raises issues about the ordering logics of fractionally coherent objects. It also leaves us with the challenge of working out what is needed to support fractal knowing. This is important because TripAdvisor has raised many issues within the hospitality sector. To regard this decentered apparatus as entirely transparent and open is to misunderstand the specific centering cuts that are enacted. TripAdvisor has steadfastly declined to reveal the technical details of its Popularity Index algorithm on the grounds that it is proprietary and releasing it would only lead to “gaming” by hotels. Ironically then, in the midst of the multiple heterogeneity of reviews posted by the crowd, TripAdvisor’s ranking algorithm reintroduces homogeneity through imposing a single ordering that determines which hotel is valued as number one in a region. Through TripAdvisor we thus learn in detail about the microphysics of power invested in the qualculative apparatus of valuation.

(p.138) 6.6 Conclusion

As part of the ongoing project of travel and knowledge about travel, we are seeing the latest knowledge eclipse (Strathern, 2000). In this case, expert valuation schemes in the hospitality sector are being supplanted by user-generated reviews posted on social media websites. Such a reconfiguration of valuation in the hospitality sector redistributes resources among businesses, shifts travel practices, redefines what counts and who counts, and changes the habitats in which these reconfigurations are realized. We suggest that such reconfigurations of valuation do not just raise epistemological issues but also raise questions of ontology. Producing stories and producing pinboards entail different kinds of work. They produce different kinds of valuations of the real, and in so doing enact different (singular and multiple) realities (Law, 2002).

The particular relations through which valuation practices are enacted raises interesting questions for scholars interested in studying the entanglement of materiality in the process of qualculation, knowledge production, and responsibility. Lucy Suchman (2002b: 142) has noted:

As Haraway made clear, the fact that our knowing is relative to and limited by our locations does not in any sense relieve us of responsibility for it. On the contrary, it is precisely the fact that our vision of the world is a vision from somewhere—that it is based in an embodied, and therefore partial, perspective—that makes us personally responsible for it. The only possibility for the creation of effective technologies, from this perspective, is through collective awareness of the particular and multiple locations of their production and use.

Organizing a mass of data inevitably demands particular strategies of coordination. We would suggest that the current approach pursued by TripAdvisor stops short of its mission. The distinctive located accountability achieved by TripAdvisor is being offset by agential cuts that obscure its algorithms and allow anonymous, unverified postings. However, TripAdvisor has only been in operation for just over a decade, and its developments and consequences are relatively brief in the history of travel. It regularly releases new versions of its website and its policies, and its practices and technologies are thus fluid and emergent.

We believe that the reconfigurations of valuation exemplified by the case of TripAdvisor offer important insights into the broader issues associated with the role of social media in knowledge practices and the eclipsing of expert valuations by ones grounded in user-generated content. Our purpose in this chapter has been to present an account of these (p.139) reconfigurations in play and to emphasize that websites hosting user-generated content are as responsible as any other form of organization for the location and partiality of the knowledge that they produce. The apparent multiplicity of valuation criteria and decentering of knowledge production to the crowd is not an excuse to lose sight of located accountabilities.

A version of this chapter was delivered as a keynote address to the Third International Symposium on Process Organization Studies, Corfu, June 2011. We thank Hari Tsoukas for his invitation to develop these ideas here. We are grateful to Vasiliki Baka for her research assistance, and the study participants for their contributions to the fieldwork. This research was supported in part by the Centennial Visiting Professors Program at the London School of Economics.

References

Bibliography references:

Barad, K. (2003). “Posthumanist Performativity: Toward an Understanding of How Matter Comes to Matter,” Signs, 28/3: 801–31.

——  (2007). Meeting the University Halfway: Quantum Physics and the Entanglement of Matter and Meaning. Durham, NC: Duke University Press.

Callon, M. and Law, J. (2005). “On Qualculation, Agency and Otherness,” Environment and Planning D: Society and Space, 23/5: 717–33.

——  and Muniesa, F. (2005). “Economic Markets as Calculative Collective Devices,” Organization Studies, 26/8: 1229–50.

David, S. and Pinch, T. (2008). “Six Degrees of Reputation: The Use and Abuse of Online Review and Recommendation Systems,” in T. Pinch and R. Swedberg (eds.), Living in a Material World: Economic Sociology meets Science and Technology Studies. Cambridge, MA: MIT Press, pp. 341–74.

Dougherty, D. (2002). “Grounded Theory Building: Some Principles and Practices,” in J. A. C. Baum (ed.), Companion to Organizations. Oxford: Blackwell Publishers, pp. 849–67.

Eisenhardt, K. M. and Graebner, M. E. (2007). “Theory Building from Cases: Opportunities and Challenges,” Academy of Management Journal, 50/1: 25–32.

(p.140) Espeland, W. N. and Sauder, M. (2007). “Ranking and Reactivity: How Public Measures Recreate Social Worlds,” American Journal of Sociology, 113/1: 1–40.

——  and Stevens, M. (1998). “Commensuration as a Social Process,” Annual Review of Sociology, 24: 312–43.

Giddens, A. (1991). Modernity and Self-Identity: Self and Society in the Late Modern Age. Stanford: Stanford University Press.

Glaser, B. G. and Strauss, A. L. (1967). The Discovery of Grounded Theory: Strategies for Qualitative Research. Chicago: Aldine Publishing Company.

Gretton, J. R. (1993). “Introduction,” in W. B. C. Lister, A Guide to the Microfiche Edition of Murray’s Handbooks for Travellers. Frederick, MD: University Publications of America, pp. vii–xlviii.

Gretzel, U. and Yoo, K. H. (2008). “Use and Impact of Online Travel Reviews,” in P. O’Connor, W. Hopken, and U. Gretzel (eds.), Information and Communication Technologies in Tourism. New York: Springer, pp. 35–46.

Haraway, D. (1988). “Situated Knowledges: The Science Question in Feminism and the Privilege of Partial Perspective,” Feminist Studies, 14: 575–99.

Hsu, G., Roberts, P. W., and Swaminathan, A.(2012). “Evaluative Schemas and the Mediating Role of Critics,” Organization Science, 23/1: 83–97.

Jeacle, I. and Carter, C. (2011). “In TripAdvisor We Trust: Rankings, Calculative Regimes and Abstract Systems,” Accounting, Organizations and Society, 36/4–5: 293–309.

Karpik, L. (2010). Valuing the Unique: The Economics of Singularities. Princeton: Princeton University Press.

Law, J. (2002). Aircraft Stories: Decentering the Object in Technoscience. Durham, NC: Duke University Press.

——  and Urry, J. (2004). “Enacting the Social,” Economy and Society, 33/3: 390–410.

Mesquita, S. (2011). Travel Publishing Yearbook. Surrey, UK: Nielsen BookScan.

Moser, I. and Law, J. (2006). “Fluids or Flows? Information and Qualculation in Medical Practice,” Information Technology & People, 19/1: 55–73.

Sattin, A. (2008). “Baedeker Guidebooks are Back,” Sunday Times, January 13. http://www.timesonline.co.uk/tol/travel/holiday_type/travel_and_literature/article3159576.ece.

Sauder, M. and Espeland, W. N. (2009). “The Discipline of Rankings: Tight Coupling and Organizational Change,” American Sociological Review, 74: 63–82.

Stark, D. (2009). The Sense of Dissonance: Accounts of Worth in Economic Life. Princeton: Princeton University Press.

Starke, M. (1826). Information and Directions for Travellers On the Continent. Paris: A. & W. Galignani Publishers.

Starkov, M. and Price, J. (2007). “Web 2.0 vs. Search Engines: Are Search Engines Becoming Obsolete in the Web 2.0 Frenzy?,” Hospitality News, September 17, 2007. http://www.hospitalitynet.org/news/4034173.html.

Strathern, M. (2000). “The Tyranny of Transparency,” British Educational Research Journal, 26/3: 309–21.

(p.141) Suchman, L. A. (2002a). “Located Accountabilities in Technology Production,” Scandinavian Journal of Information Systems, 14/2: 91–105.

——  (2002b). “Practice-Based Design of Information Systems: Notes from the Hyperdeveloped World,” The Information Society, 18: 139–44.

——  (2007). Human–Machine Reconfigurations. Cambridge: Cambridge University Press.

Tisdall, N. (2007). “Baedeker is Back,” The Telegraph, http://www.telegraph.co.uk/travel/artsandculture/739083/Baedeker-is-back.html.

Vermeulen, I. and Seegers, D. (2009). “Tried and Tested: The Impact of Online Hotel Reviews on Consumer Consideration,” Tourism Management, 30: 123–7.

Willmott, H. (2010). “Creating ‘Value’ Beyond the Point of Production: Branding, Financialization and Market Capitalization,” Organization, 17/5: 517–42.

Xiang, Z. and Gretzel, U. (2010). “Role of Social Media in Online Travel Information Search,” Tourism Management, 31: 179–88.

Notes:

(2.) Published with the subtitle, The Traveller’s Guide through the Commercial Metropolis of the United States by a Gentleman residing in this City.