Jump to ContentJump to Main Navigation

Volker Wulf, Volkmar Pipek, David Randall, Markus Rohde, Kjeld Schmidt, and Gunnar Stevens

Print publication date: 2018

Print ISBN-13: 9780198733249

Published to Oxford Scholarship Online: April 2018

DOI: 10.1093/oso/9780198733249.001.0001

Show Summary Details
Page of

PRINTED FROM OXFORD SCHOLARSHIP ONLINE (www.oxfordscholarship.com). (c) Copyright Oxford University Press, 2019. All Rights Reserved. Under the terms of the licence agreement, an individual user may print out a PDF of a single chapter of a monograph in OSO for personal use (for details see www.oxfordscholarship.com/page/privacy-policy). Subscriber: null; date: 15 February 2019

Research into Design-Research Practices

Research into Design-Research Practices

Supporting an Agenda toward Self-Reflectivity and Transferability

(p.491) 16 Research into Design-Research Practices

David Randall

Tobias Dyrks

Bernhard Nett

Volkmar Pipek

Leonardo Ramirez

Gunnar Stevens

Ina Wagner

Volker Wulf

Oxford University Press

Abstract and Keywords

The aim of this chapter is to outline a research agenda. Here, this agenda has been termed “meta-research into practice-based computing”; the chapter uses the authors’ own work to exemplify elements of it. Such an agenda requires coherent framing and needs to be conducted in sufficient breadth and depth and so is still evolving. The chapter first presents some of the driving forces behind “post-normal” interdisciplinary science carried out in research consortia, as well as a methodological approach to performing ethnographies on research projects. It then describes two examples of meta-research performed by the Siegen group. Finally, the concluding part highlights the main insights from these projects, outlining a more detailed research agenda.

Keywords:   meta-research, practice-based computing, ethnography, research agenda, interdisciplinary science

16.1 Introduction

In the introduction to this book, the view was expressed that any “practice-based” approach to computing would necessarily entail reflection about whose practices we might be concerned with. It is one thing to examine the practices of a group of workers or indeed any other group of people engaged in certain practices that we feel might require empirical analysis to ground design—if computing is to be improved. It is equally true that any such examination requires us to investigate to the same extent the ways in which organizations manage IT design in practice. In particular, as design researchers we should be examining our own practices.

A practice-based research approach is based on case studies that are shaped by the particularity of the individual field of application, as well as by the specific design (research) team. While the individual cases are particular, an overall research agenda should document these cases to allow for a comparative analysis (e.g., see Chapter 1). To this end, we suggest introducing a specific genre of empirical studies to investigate the particularities of individual design case studies that typically follow project organization. In a practice-based research paradigm, such a genre of study should

  • (p.492) encourage the reflexivity of IT designers and contribute to the improvement of a particular design (research) team’s practices,

  • document the particular setting of individual design cases, thus fostering the transferability of design-relevant insights across different settings, and

  • be fundamental to the building tools of to support a practice-based design paradigm (e.g., the ePortfolio; see Chapter 1).

Moreover, as any intervention into social practices may be confronted with the question of its legitimacy, research into the design practices of individual projects may offer an empirical basis for both reflexivity and normative discourses.

Empirical investigation can deal with many different aspects of a design (research) project and can apply various methods. At the University of Siegen, for instance, research projects typically follow the grounded-design paradigm. These projects are mostly funded by public sources and follow a specific (Central) European approach to the institutionalization of research. The European Commission, the German Federal Government, and other government agencies at the German state level try to foster IT-induced social innovation by funding research projects in which academia, IT industries, and, often, application partners submit joint proposals in a competitive manner. In case of success, research is often conducted in consortia of partners representing different interests and perspectives. These funding schemes are quite distinct from the institutionalization that is required, for instance, to apply for funding from the National Science Foundation in the United States. These schemes try to incentivize IT-induced innovations that tackle societal problems. In principle, they can facilitate practice-based design research, but in practice, they can lead to a complex institutionalizations of research that requires critical reflection on the research practices that they result in (see Dachtera et al. 2014).

The aim of this chapter is to outline a research agenda. We have termed this “meta-research into practice-based computing” and use our own work to exemplify elements of it. Such an agenda requires coherent framing and needs to be conducted in sufficient breadth and depth. We are still evolving this research agenda. Our first step was to introduce a distinction between levels on which investigations into practice-based design research might be positioned. Consortia settings may vary in terms of the number of participating partners and the heterogeneity of interests and perspectives. In any case, design research needs to bridge the different practices of the participating IT professionals, whether they are from academia or from IT industries. Moreover, there is a need for techniques to be developed to align with the social practices to be supported in the fields of application. The kind of meta-research we propose can focus on different entities:

  1. 1. A design case study that is conducted by a research consortium: We have started to investigate the emerging practices of IT professionals in different design case studies using an ethnographic approach. Methodologically, we suggest ethnographical investigations into the design practices of specific research projects. Some studies of this kind already exist. Dachtera et al. (2014) for instance, examined the (p.493) cooperation between academic, application, and industry partners during participatory-design activities in the firefighting domain. Wan et al. (2016) conducted a similar study looking at a project in which a GPS tracker was designed for elderly dementia patients suffering from “wandering syndrome.”

  2. 2. A research group that runs different practice-based design projects in parallel over a longer period of time: Some of the authors of this chapter have begun an ethnographic study of the research practices of our own group (WiNeMe). To extend these insights, we follow up by looking specifically at the subgroup working on IT for the aging society. We are particularly interested in patterns of knowledge sharing and cooperation, as well as the use of IT tools. As a result of the internal presentation of the empirical findings of this study, a workshop process was set up to deal with critical issues that had not been visible before. The results of this project, it is argued, should ground the design of IT tools in supporting recurrent research practices (e.g., the ePortfolio; see Chapter 1).

  3. 3. Commercial companies that work on design projects from a practice-based angle (IT industries as well as application partners): When Nett et al. (2008) conducted action research to understand user-centered development practices at a particular software company, they found that the company had, for various reasons, difficulties in incorporating user perspectives into the design process. To analyze these difficulties, the authors used the conceptual lens of transferring, translating, and transforming knowledge across boundaries and concluded by suggesting that one should distinguish between user-centered design as an espoused theory (which has become quite popular in recent years), and user-centered design in practice. Providing practitioners with a blueprint for the “right” method was judged to be insufficient. Instead, one has to take their given constraints and work practices seriously in developing a solution that fits the context (Stevens et al. 2009).

  4. 4. A practice-based academic IT design community: Initial investigations have begun into the practices of our two international reference communities, the European Society of Socially Embedded Technologies (EUSSET) and ACM SIG-CHI. One of our researchers has observed program-committee meetings and interviewed senior members of these academic communities. Here again, opportunities for IT support were a key issue. The redesign of the EUSSET Web site and its digital library were based on elements of the empirical findings.

In the field of human-centered computing, the research program outlined here is quite novel—to the best of our knowledge. Post-disciplinary research, whether within a project, a research group, a business organization, or an academic community, depends on and deals with a complex constellation of interests, motivations, and historically grown practices. We need to understand these design activities better—and also to support them better. A corresponding research program needs to be outlined in more detail.

This is done in three steps. First, we present some of the driving forces behind “post-normal” interdisciplinary science carried out in research consortia, as well as our methodological approach to performing ethnographies on research projects. We then describe (p.494) two examples of meta-research performed by the Siegen group. Finally, the concluding part of this chapter highlights the main insights from these projects, outlining a more detailed research agenda.

16.2 The State of the Art

Our research agenda reflects a historic development in the ways research has been practiced and theorized. It is influenced by a set of concepts and approaches that emerged as a result of this debate. While the research programs that inspire our approach range from social studies of science (e.g., science and technology studies [STS]), to action research and participatory design, to newer developments in the field of human–computer interaction (HCI), the notion of practice and a commitment to ethnographic studies of practices are key to our approach. In the field of design, the notion of “reflective practice,” which is associated with the writings of John Dewey and Donald Schön, is of particular relevance (see Chapter 1).

16.2.1 New Forms of Research Collaboration

Some time ago, it was recognized that the production of scientific knowledge was undergoing a structural change—one that involved stronger participation by nonacademic social actors, with a consequent focus on knowledge production as contributing to socially relevant problems (Funtowicz and Ravetz 1993; Gibbons et al. 1994). In passing, we should acknowledge that the direction of this change was not endorsed by some, nor was it always accepted that it was, in fact, taking place. It was suggested that this “scientification” of society and its correlate, the “politicization of science” (Weingart 1997), might lead to increasing populism and commercialization and might possibly culminate in the erosion of scientific reliability and “neutrality.” The increasing demand for practical relevance in research has also stimulated a discourse about the commercialization of research, and the role of the university as a business actor in the knowledge society (e.g., Slaughter and Leslie 1997; Carrier and Nordmann 2011). Authors such as these investigate the consequences that arise from the increasing pressure for practically relevant results for different disciplines. Often, these contributors adopt a critical approach toward the demands placed on science.

Regardless, Weingart (1997) explains these changes in the role of science in society as follows:

The university has lost its monopoly on knowledge production. Research centers, government agencies, industrial laboratories, think-tanks, consulting firms are also producing new knowledge. Through their interrelations in networks, contexts are created which replace the traditional disciplines. … Knowledge production becomes socially accountable and reflexive. Research comes under new pressures of legitimation. (Weingart 1997, p. 596)

This new mode of involving social actors in scientific knowledge production, also referred to as “Mode 2” (Weingart 1997; Bartunek 2011), offers new opportunities for (p.495) transdisciplinary research; these are understood as representing an attempt to go beyond interdisciplinary collaboration among disciplines to develop approaches that transcend not only the different academic disciplines but also the different interests of commercial, academic, and other partners, thereby providing a foundation for solutions to socially relevant problems (Pohl and Hirsch Hadorn 2007; Hirsch Hadorn et al. 2008).

The different ways in which scientific research is “contextualized” (Nowotny et al. 2001) is one of the main focuses of STS. On the one hand, ethnomethodologists have produced close descriptions of scientists’ shop work, revealing the local, situated rationalities of everyday scientific practice (e.g., Lynch, 1993), while on the other the famous “Laboratory Studies” (Latour and Woolgar 1979; Knorr-Cetina 1981) also focused on how science was practiced in the laboratory. Among other things, these studies demonstrated that the world of science “is rife with political passions, contestations of power, ever-changing definitions of logic and proof” (Doing 2008, p. 286). Other STS researchers looked at the kind of “consortium research” we are interested in. For example, Michel Callon, one of the founders of actor–network theory, carried out a study on a French electric car project from 1960 to 1977. In “L’État face à l‘innovation technique: le cas du véhicule électrique” (Callon Michel 1979), he describes how the different actors (the most powerful among them being EDF, Renault, and the state) shaped the research agenda of the electric car. While these (and other) studies focused on the collaborative nature of research and its framing by networks of researchers, funding agencies, industrial partners, and so on, there was also an “engaged program” of STS (Sismondo 2008) that promoted the social responsibility of science. On a practical–political level, this engagement, which also went under the label “democratization of science,” was reflected in various initiatives. Among these were consensus conferences, the first of which was held in Denmark, and so-called science shops, which provided technical advice to citizens, associations, and nonprofit organizations and also carried out small-scale research projects. Reflecting on steps toward bringing science into the “public arena,” making it more democratic, Callon (1999) stated:

Since science produced in laboratories is at best incomplete, at worst unrealistic and, in any event, incapable of accounting for the complexity of the specific problems to which it is applied, it is advisable to open the forum for discussion and deliberation so as to create the conditions of its enrichment. (Callon 1999, p. 86)

For several reasons, such endeavors have become much more common in design-oriented research cooperatives. Some research methodologies postulate the need for cooperation with practitioners during the evaluation of innovative artifacts to address stakeholder-relevant problems (Hevner et al. 2004; Otto and Österle 2010; Wulf et al. 2011, 2015; Rohde et al. 2017). The explicit aim of early Scandinavian participatory design was to foster workplace democracy by involving workers in learning about new technologies and participating in their design (see Chapter 7). Furthermore, as a precondition for the approval of joint research projects, specifically in Europe, publicly funded research typically requires the involvement of entrepreneurs, end users, or experts of a specific social domain to ensure application-oriented results through the emergence of innovative products or services (Laudel 2006; The Federal Ministry of Education and Research 2009; EU 2013).

(p.496) Even so, and intertwined with the changes in funding policies, a problem-solving perspective on scientific knowledge production has arguably become more prevalent in the research discourses and methodologies of several scientific disciplines (Van De Ven 2007; Mathiassen and Nielsen 2008). The emergence of a “relevance gap,” or a need to identify and tackle practice-relevant problems, was promptly diagnosed in several disciplines (Rosemann and Vessey 2008). For information systems research, the “relevance gap” translates into research on how specific nonacademic stakeholders can be involved in design activities and the evaluation of design artifacts (Rohde et al. 2009; Otto and Österle 2010).

Nevertheless, we argue that analytic approaches to research activity have thus far paid too little attention to the problems of interorganizational collaboration, having typically focused on disciplinary perspectives and their internal mechanisms of stabilization. In these latter studies, socialization into a research community goes hand in hand with the acquisition of a certain world view and way of life (Stichweh 1992). Weingart (1997) looks specifically at interdisciplinary research and distinguishes between the level of the funding agencies’ rhetoric and the level of actual research work. He finds that on the rhetoric level, interdisciplinarity is discursively connected with dynamics, flexibility, and liberality, while the level of actual interdisciplinary research as yet remains only an ambition. That is, interdisciplinary collaboration may be challenging in a variety of ways, although we currently have relatively little empirical evidence to demonstrate how, in practice, outcomes are realized. There are some exceptions. In an empirical study, Mansilla et al. (2015) found that “shared socio-emotional-cognitive platforms” decisively contribute to the success of interdisciplinary synthesis. In a similar vein, McCallin and Bamford (2007), in a grounded-theoretical study, highlight the importance of emotional intelligence in interdisciplinary teamwork. There is, nonetheless, a pressing need for a “practice” orientation to the processes of interdisciplinary enquiry.

16.2.2 Why “Meta-Research” into “Practice-Based Computing”?

Two methodological problems make a reflexive practice-based approach attractive. The first is that many engineering conceptions are focusing more on the application of formalistic knowledge to known products, rather than on the analysis of applied solutions for individual cases. This partly has to do with the established form of industrial technology development. However, the historical relation between engineering and mass production has become threatened by mass customization on the one hand and the rise of the “information economy” on another. The invention of “software” has made value enhancement by the individualization of products much easier and thus has motivated business concepts in which such individualization plays a larger role. A substantial part of commercial software development is about custom-built products, which are poorly supported by (mass-production) engineering conceptions.

A second motivation for conducting reflexive project ethnographies is that assessment of the value of technology products is often conducted by stakeholders with heterogeneous expertise. This complicated problem is scarcely solved by naive consensus building, since (p.497) unexpected inputs can dramatically affect one’s plans (see Fischer and Giaccardi 2006; Stevens and Nett 2009).

A more recent proposal, emanating from the fields of computer-supported cooperative work (CSCW) and HCI, comes from Jackson et al. (2013), who have suggested that science policy should build on the manifold experiences and insights from the discourse on “cyberinfrastructure” within CSCW. They argue that as “we are likely to see interdisciplinary and translational mechanisms and mandates expand,” research will in the future “need many of the things that cyberinfrastructure promises to deliver—collaboration, interdisciplinarity, larger and multiple perspectives on problems of public import, etc.” (Jackson et al. 2013, p. 1118). Publications in this tradition, as well as in the areas of e-research and e-science, have particularly emphasized the human aspects of “infrastructuring” as an activity in collaborative research work (Lee et al. 2006; Pipek and Wulf 2009; Barkhuus and Brown 2012). These studies have deployed a practice-oriented perspective on the research activities of different scientific fields.

The need for such a “meta-analytic” view is arguably becoming more pressing as our vambitions in respect of the analysis of practice develop. For instance, it is evident in the abovementioned interest in sociotechnical infrastructures and, closely related to it, our interest in the ebb and flow of relations inherent in technological development, organizational and infrastructural change, and work practice (e.g., Lee et al. 2006; Ribes 2014). Given that the outcomes associated with design are inevitably uncertain, it makes sense that some investigation of the processes in and through which design outcomes are arrived at should take place and that empirical materials concerning those different outcomes might be a source of material for our reflections. However, it remains the case that such enquiries remain, at the time of writing, relatively unusual (but see Dachtera et al. 2014). We need, we would argue, empirical knowledge to inform how the claim of cooperation can be concretely operationalized in research (Rosemann and Vessey 2008; Österle and Otto 2010). Particularly for the practice-oriented approach presented in this book, the analysis of adequate collaborative project cases can help researchers focus on critical moments of the process, which are significant for building a profound knowledge base from design studies (Krohn 2008). This implies, of course, an appeal to a certain kind of reflexivity, and this bears some examination.

16.2.3 The Notion of “Reflective Practice”

As stated before, the notion of “reflective practice” goes back to John Dewey, who considered reflection to be essential to the development of practices. Dewey defined reflective thinking as a number of phases in thinking: “(a) a state of perplexity, hesitation, doubt; and (b) an act of search or investigation directed toward bringing to light further facts which serve to corroborate or to nullify the suggested belief” (Dewey 1910, p. 9). Schön (1983) took up this notion, using it in describing design as a reflective process, with designers engaging in two modes of reflection: while performing and evaluating a particular design move (“reflection in action”) and when looking back at a project, evaluating the approach s/he has taken (“reflection on action”). Both Dewey and Schön focused on the individual designer and, at the time, did not think of the complex constellations of research and design today.

(p.498) Reflexivity has become a much used, and much abused, term that is often associated with sociological output. However, it can imply a number of different things. It implies some interrogation of the concepts and theories one deploys, and one’s reasons for doing so; some critical reflection on one’s behavior in relation to the people one studies; a degree of self-awareness; and an awareness of the institutional forms that construct one’s thinking. As indicated at various points throughout this book, however, a concern for sociological concepts and theories is not part of our project here. Rather, we are interested in the way in which a specific project produces certain outcomes, the way in which these outcomes are managed and negotiated, how partners in projects might learn from such examinations, and how we might make a more generic sense of, and document, the organizational processes that influence these outcomes.

Reflection is one of the cornerstones of action research. Action research is a framework for inquiry that “seeks to bring together action and reflection, theory and practice, in participation with others, in the pursuit of practical solutions to issues of pressing concern to people” (Bradbury and Reason 2001, p. 1). While not design oriented as such, it postulates a strong link between inquiry and action and “the credibility and validity of [action research] knowledge is measured to a large degree by the ‘workability’ of solutions—that is, their ability to address real problems in the lives of the participants” (Hayes 2011, p. 158). One of the aims of action research is to make the research part more participatory. Reaching mutual understanding in such a process requires a high degree of reflexivity—to reflect on one’s own positions, on the research situation, and on the research process. Bergold and Thomas (2012) suggest several types of reflection: reflection on participants’ “personal and biographical attributes and dispositions,” reflection on the social relationships between partners, “structural reflection on the social field of the research project,” and reflection on the research process itself (Bergold and Thomas 2012, p. 205 f.) In this tradition, Bjørn and Boulus demonstrate how “reflection on our own assumptions, thoughts, and beliefs” (Bjørn and Boulus 2011, p. 283) in two ethnographic studies in the field of healthcare, as well as on “the work that went into constructing the project, making it function as a coherent whole” (Bjørn and Boulus 2011, p. 285), provided an important opportunity for learning. There was a reflection “ex post” when they came together to conduct a conversation about their projects, challenging each other’s views.

Participatory design also advocates reflection as an important principle. For example, Bødker and Iversen state that “professionalism is dependent on an ongoing reflection and off-loop reflection among practitioners in the [participatory design] process” (Bødker and Iversen 2002, p. 11). However, there are few examples of more in-depth studies of participatory-design practitioners systematically reflecting on their practices. This does not mean that the practice of participatory design is “unreflective”—there are fine examples of participatory-design practitioners critically examining their practices, as for example, that by Winschiers-Theophilus et al. (2010), who analyzed some of the problems they encountered when working with communities in Namibia. While these personal reflections of a research team are valuable, they are selective and partial. For example, Bratteteig and Wagner (2014, 2016) went back to several of their own participatory-design projects, attempting to reconstruct the important moments in the design process. (p.499) Revisiting project diaries and other documents helped them piece together the stories of these different projects, including moments of conflict in the project teams. Even so, their analysis is limited to what they were able to reconstruct from their own perspective (and with a view to decision-making) on the design ideas that were put forward and how some of them were selected and further developed.

One exception is the work of Pedersen (2007), who, referring to his own participatory-design project in the area of maintenance work, critically examined his (and his coresearcher’s) efforts at making this work “public” and “demonstrating promising results” to the participating stakeholders. For example, he describes how the fieldwork itself had to be formatted for different uses, and the constraints of having to avoid conflict with the people under study. In his analysis, which is influenced by actor–network theory literature, he suggests a distinction between “representatives” and “constituencies,” arguing that partners in a design project not only bring their own knowledge and commitments into a project but are also accountable to the communities to which they belong.

CSCW researchers have studied how designers, including IT professionals, work. But these studies do not necessarily look inside large and complex projects with multiple stakeholders. Again, an exception here is an ethnographic study by Martin et al. (2009) of the introduction of electronic patient records within all hospitals of the National Health Service in England and Wales. They, for example, point at the difficulty of understanding the work practices of all the different user groups and taking account of them, how healthcare policies set targets the project could not afford to fail, and how a myriad of organizational issues made it difficult to coordinate work. While, in some respects, these studies come close to what we think doing ethnography on projects is or should be, their aim is not necessarily to feed observations back into the project team in order to help them improve their practices and build a corpus of knowledge with a view not only into a particular research group but the whole community of practice-based research.

16.2.4 Doing Ethnography on Projects: The Approach

The first, and perhaps most obvious, complication to deal with is how best to conceptualize the relationship between observation of practice and subsequent design interventions. As Dourish (2006) has argued, the idea that there is, or can be, any direct and determinate relationship between the two is an oversimplification, to say the least. First, and most clearly, observations of practice cannot eliminate all the various possibilities that are inherent in design decisions. There are cost–benefit considerations that implicate, at best, estimates concerning the future. There are decisions to be made concerning the generic applicability of new technologies across settings (or domains), about the degree of modularity that might be required, or about the limits to be placed on deployment. There are constellations of interests, ethics, and commitment to be considered. As Schmidt has pointed out, when discussing the impact of ethnography for CSCW, the term “shared goals” in respect of project outcomes is somewhat misleading: “the ‘shared goal’ is not there in advance. It is constructed by the members in the course of the project, and it is in the course of agreeing to a ‘shared goal’ that the designers arrive at an agreed-to design. (p.500) When the designers “have a ‘shared goal’ they have—for all practical purposes—finished the design task” (Schmidt 2011, p. 151 f.). Schmidt goes on, citing Bucciarelli (1988), to argue that “a design decision in this instance is best seen as an overlay of interests rather than their synthesis within some flat, cognitive domain” (Bucciarelli 1988, p. 164). This is exactly the reason why documenting design decisions, who contributed to them, how priorities were set, which kinds of resources were available, and so on, is crucial for an understanding of design practices and why a particular design result has been achieved (Bratteteig and Wagner 2016).

Gaver and Bowers have put forward the idea of creating “annotated portfolios” that bring together design artifacts “as a systematic body of work,” with annotations capturing “family resemblances between designs in a mesh of similarities and differences” (Gaver and Bowers 2012, p. 46). Through this work, they seek to identify “conceptual schemes” that may lend themselves to some generalization. Our approach is different, insofar as we aim to investigate the practices of our own community in line with the practice-based perspective rehearsed throughout this book, and, perhaps most importantly, do not see this as something that is done after the serious business of design is finished but as something that should take place simultaneously with the design process. It is, put simply, constituted as an ongoing reflection about what we do and how we do it. The need for this is reflected in the fact that a community dedicated toward analyzing and supporting practices has, to date, provided very few empirical accounts of their own research practices (but see Dachtera et al. 2014). It would be naive in the extreme to imagine that the move toward any kind of model of heterogeneous collaboration comes without challenges, and empirical work ought to reveal something of what those challenges look like. It is to be hoped that, eventually, such accounts of research and development practice might serve both to design supporting IT artifacts and to inform research policy.

It makes sense to think of these aims in terms of organizational (or rather interorganizational) learning (Argyris and Schön 1996) and, more specifically, second-loop learning (Argyris et al. 1985). As such, it would have something in common with organization and technology development, as first suggested by Wulf and Rohde (1995), but with a more explicit ethnographic focus, one that addresses the meaning of a project for the participants. Institutional ethnography (e.g., Smith 2006) is one approach to organizational learning that “encompasses participant observation … and an encouragement of people to ‘tell their story’” (Bell and Morse 2007, p. 100).

We also may take inspiration for doing ethnographies of projects from recent work in other fields, such as sustainable development, or in large-scale interventions in third-world countries. Looking back at a project about implementing community interventions for HIV prevention, Evans and Lambert (2008) make a distinction between a project’s internal documentation, and the donor agencies’ insistence on ordering and systematizing practice by creating models, manuals, and guidelines. This represents two aspects of what we call “public accounts of practice” (Evans and Lambert 2008, p. 475) and “insider accounts” that are written to understand the more subtle aspects of project collaborations. The authors use ethnographic material from one particular intervention project in India to demonstrate that “observation of actual intervention practices can reveal insights that may be hard for project staff to articulate or difficult to pinpoint … and can highlight (p.501) important points of divergence and convergence from intervention theory or planning documents” (Evans and Lambert 2008, p. 477).

Pottier (1993) reflects on the possibilities of “project appraisal through ethnography” to improve the management of development projects. He makes his case by referring, among other things, to a study by Moris and Hatfield (1982), who demonstrated how a project introducing Western technology clashed with the pastoralism of the Maasai. An important argument for doing ethnography on projects is made by Mosse (2007), who examines how expertise is constructed in international development. He argues that project work in development is driven by “global policies,” which he characterizes as “‘travelling rationalities’ with general applicability that assert the technical over the political, the formal over the substantive … , the categorical over the relational” (Mosse 2007, p. 4). Mosse suggests conducting ethnographies of how development work is practiced in different contexts by different types of experts in collaboration with local participants—work that, in some respects, may be considered similar to an IT R & D project. While the idea of “meta-research into practice-based computing” is about learning across projects for future work, project ethnographies in international development seem more intended to deconstruct the dominance of “global policies” that do not sufficiently take account of local practices.

When we examine what can be learned from different approaches to doing ethnographies of projects in different contexts, three points stand out:

  • the issue of “positionality” that action research addresses: from whose perspective are these ethnographies written, as reflected in the differences and potential tensions between “insider accounts” and accounts by an external observer?

  • the “focus” on data collection: is it on the actual practices of collaborative design as they unfold in everyday project work or are these observed with a particular lens onto, for example, decision-making, situations of conflict, how these are handled, and with which result?

  • the main objectives of such studies: is it to improve collaboration in a particular project, to learn across projects, to develop guidelines, an analytical framework, and so on?

The meta-research program outlined in the introductory remarks of this chapter is quite comprehensive. However, so far we have elaborated only on a small part of the themes brought up there (Dachtera et al. 2014; Wan et al. 2016). In the following, we want to clarify the research program, with examples.

We first examine the Landmarke project in detail, focusing on the way in which a successful design outcome is reached and what kinds of activity produce that outcome. Here, the role of experience, communication across professional boundaries, and an explicit focus on the exchange of ideas is apparent. Then we investigate the collaborative practices of WiNeMe, a research group conducting projects that follow the practice-based research paradigm of grounded design. Yet another under-rehearsed aspect of meta-research is to understand research groups, specifically, the roles of collaboration, knowledge sharing, and their interactions with a wider academic community.

(p.502) 16.3 Ethnographies of Design Case Studies and Projects

We want to start by outlining a methodological framework to investigate specific design (research) case studies and projects. This framework is inspired by an approach called business ethnography (Nett and Stevens 2008; Nett 2010). Ethnographies of IT projects do not refer to any specific set of commitments but can be understood as reflexive activity within design case studies, as discussed in Section 16.2.4. The reflexive ethnographer needs (and generally has) some practical understanding of what the project in question is about and what role he/she is undertaking. While the role of the ethnographer as producer of data about, and communication of, practitioner practices in interdisciplinary work is by now (at least arguably) well understood, his/her potential role as a communicator of data about practices within a given project is not. To undertake such an activity, of course, requires explicit commitment from partners, and a clear understanding of the meta-process involved and its implications.

While, from an analytical point of view, development and documentation have to be separated, this is not done ex ante in project ethnography. Instead, the project ethnographer acts as a project partner who, at the same time, continues to communicate and discuss his/her role throughout. In doing so, he/she opens his/her own separation of development and documentation to critique from the project partners. The project ethnographer may even take a position on his/her own, but this must be done transparently, with the reasons documented and an analysis provided of how this conflicts with the task of mediation (Alexander et al. 2006) and how the reactions of project partners might be negotiated. As such, an ethnography of an IT design project can be thought of as a form of action research. The concept of action research was first developed by the psychologist and field theoretician Kurt Lewin 1946, who was critical of the ignorance in his discipline of its own sociohistorical framing. Instead of reflecting on its impact on the understanding of the task to be accomplished, psychology would simply focus on the impact of the individual psyche on society, thus reducing the interplay between the individual mind and its sociohistorical conditions to unidirectional causation. In order to contrast his action research against mainstream psychology, Lewin borrowed overtly interventionist terms (from medicine) to describe a process that starts with the examination of the status quo (“diagnosis”), continues with the identification of a change strategy (“treatment”), and is studied in the “evaluation” of the intervention’s impact upon the situation. The evaluation can become a new “diagnosis,” and the process repeated. Action research was thus essentially critical, especially of disciplinary assumptions, if they limited the space for solutions to be considered.

While Lewin has been credited as having been one of the founders of action research, a similar approach was developed at the Tavistock Institute (Emery and Trist 1969). It was in the perspective of achieving social change and therapeutic action that the method was suggested and probed, starting with a “diagnostic stage,” which included a collaborative analysis of the social situation by the researcher and the subjects of research. The “therapeutic stage” involved collaborative change experiments. Moreover, although members of the target community were considered important collaborators from the (p.503) beginning, in early action research, the researcher remained in control of the scientific investigation itself, directing the research design, the definition of key concepts and variables, data collection and analysis, and the interpretation of results. Various forms of action research have been developed in different disciplines such as social science (Moser 1980), organizational development, pedagogics (Moser 1978, 1980; Kemmis and McTaggart 1988; Altrichter and Posch 1990), and humanities (Bortz and Döring 2002). It was also—and this is central to the need for an ethnography of project processes—transdisciplinary. Thus, it sought precisely to overcome the limitations of disciplinary thinking (and, at least by implication, other sources of heterogeneity [see Davison et al. 2004]) and to refocus disciplinary approaches from a more general perspective. In effect, this necessitates a move from a conception of ethnographic work as observational in its early stages, and interventionist in some (still rather undecided) sense, to a conception that sees observation and intervention as closely tied together. The intervention, equally, will be rather more than simply providing some kind of support to design decision-making but will entail assistance in the management of its evolution. Again, it is necessary to recognize the fact that project practices are underdetermined by formal organizational roles and initial estimates of how the division of labor is constituted.

16.4 Critical Moments: Understanding Collaborative Exchanges within an IT Design Project

We exemplify our argument by investigating the practice of design research in the Landmarke project, which is described in Chapter 15. Here, however, our focus is on research processes rather than research outcomes.

The Landmarke project, involving collaboration between researchers at the University of Siegen and a consortium including practitioners, industries, and domain-specific experts, was concerned with the work of firefighting. In total, it involved collaboration between nine different partners:

  1. 1. The University of Siegen Department of Information Systems, which coordinated the project

  2. 2. The Fraunhofer Institute for Applied Information Technology

  3. 3. A research institute focusing on mobile sensor networks and ubiquitous computing

  4. 4. A research institute focusing on manufacturing science and rapid prototyping

  5. 5. A small-enterprise business specializing in technical systems for logistics support

  6. 6. A small-enterprise business area specializing in wearable electronics

  7. 7. A large-enterprise business providing protective equipment for firefighters

  8. 8. The Cologne professional fire service, which attends 55,000 incidents per year

  9. 9. The firefighting training institution in Münster, the Institut der Feuerwehr Nordrhein–Westfalen (IDF), which has 15,000 students per year

(p.504) The observations we make in this chapter about the working of the research project are founded on 60 hours of audio and video recordings, a personal research diary, and two series of interviews with actors in the consortium. The first series was conducted in project workshops between 2008 and 2011. The interviews provide access to the inner workings of workshops in which actors from a wide range of professional backgrounds and including academic researchers, practitioners, and entrepreneurs from both small and large companies participated. Here, we are particularly interested in what we will call “critical moments” in the collaboration among partners, as these seem to be critical in identifying and developing technical innovations with the intention of producing significant practical benefits for a specific social domain. An analysis of these “critical moments” shed some light on how we might bridge the “relevance gap” (Hodgkinson and Rousseau 2009; Kieser and Leiner 2009) and how we could improve grounded design from an interactional perspective.

The Landmarke project was predicated on what is assumed to be a critical need in the work of firefighters: to need to perform a reliable assessment of an incident in a timely fashion. This is, for fairly obvious reasons, particularly crucial during indoor reconnaissance missions (see, e.g., Ramirez et al. 2007; Denef et al. 2008; Ramirez et al. 2012). “Getting lost” on a mission inside an unknown building is one of the most frequent causes of accidents in firefighting (Fahy 2002). The project we discuss here leveraged the idea of an ad-hoc reference system that would be laid down during the firefighting itself. The envisaged system involved a mesh of small deployable tokens that, once dropped, worked as reference points to support the spatial decision processes of firefighters, creating a rich layer of information (Ramirez et al. 2012). Tokens might communicate with each other or with the firefighters. Analysis of work practices thus focused on the identification and definition of problems relevant to the reconnaissance work of firefighters, the invention of possible improvements through the design of artifacts, and the evaluation of their possible benefits and any difficulties to be addressed.

In this section, however, we focus on the collaboration within the research consortia and on the themes that emerged among participating actors. In so doing, we examine the dialogic processes that led to an emerging understanding of the various issues that underpinned the research, the agreements and disagreements that reflected distinct interests and understandings, and the resulting changes in project direction. As Table 16.1 shows, these issues became salient during the conduct of various collaborative activities.

Table 16.1 Collaborative activities in the Landmarke project

Activities in Collaboration


Administration and planning

Negotiation of roles

Coordination and planning of subsidiary tasks among technical developers

Budget planning for purchasing technical equipment

Negotiating a common understanding of public relations work

Bilateral meetings and multilateral workshops

Conference calls


Explaining firefighting work, including practical demonstrations

Conducting firefighting exercises to demonstrate current work practices and tactics with regard to reconnaissance in danger zones and extensive areas

Recounting anecdotes with in-depth descriptions of personal experiences gathered in real firefighting missions

“Empathic workshops”: professional firefighters guide scientists and industrial partners in simplified firefighting exercises as a means to presenting a first-person perspective on their work (Dyrks et al. 2009)

Workshops at the IDF training center

Bilateral and multilateral meetings at the Cologne professional fire service as well as at the IDF training center

Technical development of the Landmarke platform

Specification of user and system requirements, using agile approaches

Specification of system interfaces, protocols, and system behavior


Chat clients

Microsoft Word documents

Collaborative research, “dialog process”

Joint preparation and observation of experiments of Landmarke components in specially designed firefighting exercises

Joint analysis of experiments using Landmarke components with consideration of various observers with contrasting perspectives (e.g., mission leader vs. members of different reconnaissance teams vs. outside observers)

Workshops at the IDF training center

Bilateral and multilateral meetings at the Cologne professional fire service, and the IDF training center

As Table 16.1 shows, four generic types of collaboration activities were identified as being predominant in the project between 2008 and 2011. Even though this analysis is rather sketchy and incomplete, it facilitates a focus on collaborative activities, explaining how the designs of technical artifacts emerge from un(der?) specified ideas and how design decisions are linked to the analysis of firefighting work practice.

16.4.1 Dialogs and Discussions

After identifying the activities being carried out by the principal actors, analysis focused on the contents of the various dialogs emerging in the conduct of those (p.505) activities. Table 16.2 provides a list of identified topics. We discuss three of them in this section.

Table 16.2 Topics addressed in the Landmarke dialog during Project months 1 to 36

Project Months


Actors Involved



Technical feasibility of design ideas with regard to restrictions of the technology platform chosen at the beginning of the project

Industrial partners


Considered in 60% of the workshops

Became clear in the course of the technical development process, in particular concerning problems with hardware platforms selected to become the technology basis for the project


Tactical feasibility of Landmarke-based reconnaissance tactics with regard to tactical firefighting procedures and obligatory safety regulations

Firefighting working group (actors from the Cologne professional fire service and from IDF)


Considered in all workshops

Assessments on this aspect might be supported by a large number of experiments with experience prototypes


Good working practice of firefighting

Behavior of officers-in-charge when quickly assessing a situation

Cooperation of reconnaissance teams working in the danger zone and under limited visibility

In particular, firefighters who frequently act as brokers between the firefighting working group and scientists as well as industrial partners

In particular, the importance of preparation in workshops when experiments and exercises are planned with regard to research objectives and evaluations of Landmarke artifacts

Supported by storytelling from individual knowledge from operational experience


Performance of current firefighting tactics for reconnaissance in terms of accuracy, quickness, and safety

Firefighting working group


Discussed in 10%–20% of the workshops

Supported by experiments specially designed to provoke disorientation


Economic value of the Landmarke design and estimated development costs

Industrial partners


Discussed in 30%–40% of the workshops, in particular when discussing hardware designs


Value of Landmarke in everyday missions and established as obligatory safety equipment

Firefighting working group


Considered in all workshops


Further potential of the Landmarke platform beyond navigation support

Some members of the firefighting working group


The availability of the stable wireless network connection among landmarks in the last quarter of the project duration triggered a reflection on the relevance of communication breakdowns for firefighting practice

The “Koordinator” project (2011–14) explored the practical benefit of using landmarks as infrastructure for communication and support (Betz and Wulf 2014; also see Chapter 15)

The first topic identified was the question of technical and tactical feasibility. Discussions on technical feasibility explored the “opportunity space” of design ideas and whether concepts were technically feasible for implementation. Restrictions to the feasibility of designs could arise from limitations of the technology platform chosen in the beginning of the project (with the platform consisting of specific hardware and software components). A good example of one restriction was the limitation on the wireless range between two tokens; (p.506) (p.507) this arose because of the use of commercial off-the-shelf hardware components. Therefore, a minimum number of tokens required for reliable connectivity had to be taken into account in the theoretical exploration of ideas for reconnaissance tactics based on the tokens.

The second topic consisted of the explanations and reflections around the work practices of professional firefighting and its potential risks. There were two sets of reflections on the work practice: an internal dialog among members of the firefighting practitioners group, characterized by an exchange of knowledge and experience between experts in the topic of firefighting, and a dialog that took place with scientists and industrial partners and that centered on how and why an expert firefighter would likely behave in a certain situation. The internal dialog among firefighters was characterized by assessments of risks and considerations of improvements motivated by changes in the reconnaissance procedures created by the use of the tokens as a means for navigation. Over the course of the project, these dialogs moved from an exchange of opinions toward evaluations of tactical changes informed by experimental exercises conducted in the project.

The third topic area we were able to identify comprised assessments on the value of the emerging technology—the tokens as a navigation tool or as a commercial product. This reflection went beyond viewing the token as a technical artifact and considered it, in the first instance, to be a navigation tool that offered support to its users. In Month 27 of the Landmarke project, however, we found dialogs, which had been initiated by a member of the firefighting working group, about potential uses of the Landmarke platform, where these uses went beyond its use as a navigation support tool (e.g., whether it could help during communication breakdowns by providing a second communication channel during reconnaissance).

16.4.2 Leading and Facilitating Discussions

Workshops were, in the main, hosted at the IDF firefighting training center. As part of these workshops, IDF trainers and lecturers hosted discussions, in particular before and after exercises, experiments, technical evaluations, or training missions were conducted, recapitulated, and analyzed.

The substantial experience of the lecturers in hosting discussions in large groups of people became visible particularly when discussions began to drift into unproductive controversies which hindered a deeper exchange of arguments. Some of the trainers applied different techniques to overcome this kind of situation. For example, as a means of overcoming controversies, some trainers demanded an opinion-forming process from discussion partners via majority decision-making; opinions defeated by the majority were explicitly acknowledged as “important opinions showing aspects not to be forgotten in future work.” Questions were raised in order to foster an exchange of opinions, or to focus the discussion on a specific aspect. Professional firefighters as well as IDF trainers recounted anecdotes from their personal experience, illustrating how they experienced critical situations in real incidents; these anecdotes had the effect not only of underpinning opinions by experiences from real life but also of focusing and informing debates on a specific aspect. In addition, trainers supported the negotiation of meaning between the discussion partners as they hosted discussions handling both unexplored and undecided (p.508) design alternatives. For example at one point, the term “programming” was used by the participants to reflect on the decision as to whether the color shown in a token could be changed by a firefighter team in life-threatening situations (e.g., when firefighters need to act to rescue a person). The subject of the discussion was whether, in this situation, a firefighting team should be able to change the color of a token (which is referred to in the discussion as a “Landmark”) in order to leave information for other supporting teams in their reconnaissance:

Professional firefighter: “A [firefighting] team finds a person and grabs them to take them out. This is now an ‘uncritical’ situation. But—and I’m willing to bet on this—the team won’t reprogram the Landmarks.”

IDF trainer: “So the team explores the room, and then Landmark Number 4 is reprogrammed to the color green, ‘room has been inspected’, this is uncritical now. According to our new terminology, Landmarks are not being reprogrammed anymore, because the information on directions is programmed automatically by the Landmarks as they are removed from the carrying container.”

(Note: The color green on the token represents the tactical information “the room has been checked; nobody is inside.” A token showing the color blue at a specific location means “the firefighting team passed this location.”)

By using the expression “new terminology” with the term “programming,” the host changed the meaning of “programming” explicitly and thus changed a decision on the design: tokens should not need to be changed manually to blue to indicate that a firefighting team passed the marked location, but they should reprogram themselves to blue when they are removed from the carrying container.

In another case, a trainer used written scenarios to guide a 2-hour discussion that resulted in a deepening of the discussion, so that a topic was thoroughly reflected from multiple perspectives. He prepared a 2D floor plan on paper, including markings showing the locations of tokens that had been deployed. The position, color, and number of deployed tokens corresponded to a deployment tactic that had been collaboratively defined in a previous workshop. By thinking aloud, the participants were able to explore from an imaginary first-person perspective whether the deployment strategies would work as a systematic approach for any possible cases. The deep reach of the dialog was possible because controversial opinions on feasibility and possible modifications to the deployment tactics had to be justified concretely on the deployment situation described on the floorplans. The discussion produced revised versions of the deployment tactics.

16.4.3 The Role of Emerging Expectations in the Dialog Process

We expanded our analysis by considering the question of how actors produce and react to specific workshop situations. Taking the dual perspective of human actions proposed by Lofland (1995) helped us to understand how the dialog process in Landmarke was (p.509) shaped over time. The following transcript reveals how eight participants from six different partner institutions exchange ideas on how the components of the platform could be specified and what functionalities would make sense for its intended users.

The data comes from a technical workshop that took place in the sixth month of the project, shortly after the development of a tactical concept by a team of firefighters participating in the project. The concept proposed the use of colored tokens to codify information relevant to supporting collaboration among different firefighter teams performing reconnaissance.

In Month 5 of the project, the tactical concept had been explored practically in a training mission. We should note that the participants in this dialog—“L” and “S,” who are from the Fraunhofer Institute for Applied Information Technology, “I,” who is from a research institute focusing on manufacturing science and rapid prototyping, and “M,” who is from a large-enterprise business providing protective equipment for firefighters—come from industrial/scientific partners and therefore have no background in firefighting.

L: “And how do I choose a color?”

(This question triggers a rough description of the expected use of the tokens.)

M: “It is only about activating [tokens]. Just blue.”

L: “As default, Landmarks are blue?”

(The second question is for clarification.)

M: “Blue is default. […] This is the requirement: the default setting is what we currently call ‘mode blue’. Landmarks only have the function of a beacon. This would be a requirement for the specification.”

S: “Right!!”

I: “This means that we always see ‘blue’ on all [tokens]?”

S: “No, we don’t.”

M: “We see blue, at least.”

L: “So blue codifies the presence of a Landmark?”

S: “Yes.”

(The dialog continues, with further refinement of the anticipated use of the tokens, with the following characteristics: (1) the tokens always carry location-related information, by their presence alone, and (2) this attribution is described as the “blue” mode in the conversation.)

16.4.4 The Role of Anticipation and Speculation in Dialogs

Such speculative descriptions of the use of technical artifacts were very common in Landmarke workshops. Regardless of whether firefighters were involved or not, anticipation (p.510) was often used to argue for or against statements. These speculations can be characterized as follows:

  • anticipation was often speculative: actors in the project expressed views that were frequently speculative descriptions of a specific subject

  • such speculations were sometimes made spontaneously: when new possibilities were suggested by participants in the course of the conversation in workshops, they seemed to be developed spontaneously as the result of creative thinking and were not always well informed by expert knowledge

  • anticipation and speculation also came from “less expert” sources: suggestions were often made as a first-person narration, as if one were directly confronted with the situation described in the anticipation (e.g., “the display shows me the number of deployed landmarks”) and were made by actors even if those actors were not experts in a related field; hence, the remarks allowed actors to participate in dialogs, even if the actors came from a different professional background

  • suggestions included a context: anticipatory narrations included the dimensions of time and place and, more importantly, a concrete situation that was used to motivate the idea (e.g., a firefighting team having an accident during reconnaissance, creating the need to guide a rescue team to the incident location); in the case of discussing possibilities of the future use of technical artifacts, these narratives were based on previous experience from real use, even if the artifact was nothing more than a vague idea

Types of Speculation

We were able to identify four types of anticipation of development, based on the subject to which anticipation was related:

  1. 1. Speculations referring to an artifact in use: We observed suggestions about the technical artifact, including artifacts that were sometimes nothing more than a vague idea. This type of speculation is highly relevant for innovative-oriented project work, because it plays the role of a metaphorical placeholder, storing ideas of characteristics or features that can be modified over time by dialog among partners. The following comment by S, a scientist from the Fraunhofer Institute for Applied Information Technology, on whether the color of tokens should be changed remotely over a wireless connection is an example:

    The question is whether [the tokens] can be remotely controlled at all. […] Until now, we have been talking about one change of status only, from yellow to green. [The question is] whether I can make this change of status using my remote device, when the thing is lying in the corner over there.” (He points with his hand.)

  2. 2. Speculations referring to feasibility: We observed anticipation focused on assessments of technical or tactical feasibility or whether landmark tokens as a technical concept might or could be further developed to a marketable product. For example, see the dialog below, which a continuation of the discussion in which the comment (p.511) above was made. In it, S refers to problems with the feasibility of remote color change and articulates the problem of designing an HCI concept that would be usable by firefighters under harsh conditions:

    S: “And therefore we don’t need to change the status remotely.”

    B (from a research institute focusing on mobile sensor networks and ubiquitous computing): “But we have a radio connection!”

    S: “We need interaction. The problem is interaction in a wearable system, which we would need.”

  3. 3. Reflection on work practice, including speculations referring to problems: Firefighters often reflected on their own experience and speculated on possible problems emerging from confronting dangerous situations. This speculation was strongly based on experience; for example, H, who is a professional firefighter, stated the following:

    There are three [situations] that happen very often. The first is the power is out on the radio device and you’ve forgotten to bring a spare battery. […] The second is that you can’t get a radio connection to the outside of the building due to its structure, and the third is that somebody is pressing the pushbutton of the radio while they’re crawling, and you can hear how they are breathing, but no one else is able to talk on the radio.

  4. 4. Speculations on the relevance of problems: In the following dialog, a group of participants speculate on the potential for work improvement enabled by using the tokens. The firefighting domain finds this speculation on relevance very difficult because they have to take into account two types of situations: (1) improvements for day-to-day situations being predictable for well-experienced firefighters, and (2) situations that evolve from routine situations into dangerous incidents in which even well-trained firefighters are under threat:

    MS (from a research institute focusing on mobile sensor networks and ubiquitous computing): “Think differently. Suppose we are able to produce this added value that we talked about […]. Then it would be amazing if we could do the same as we do today with the thermal imaging camera. […] I have a system that could perhaps replace the thermal imaging camera someday.”

    JH (from the IDF): “No, this system will always be worse than the thermal imaging camera. With the thermal imaging camera, I can find people with certainty.”

16.4.5 How Suggestions Emerged in Dialogs

Weakly Substantiated or Purely Fictional Suggestions

We classify suggestions as weakly substantiated when it is not plausible to consider them as being based on any robust external reference. They are, thus, highly speculative. They emerge as moments of inspiration and typically spark a dialog, in particular at the beginning of the design process. We classify suggestions as fictional when they describe what could theoretically happen but are grounded neither in informed opinion nor in experiences articulated.

(p.512) By way of example, in the sixth month of the project, and with no functional artifact yet developed, S, who is not a firefighter but a scientist, anticipated how a firefighter could be faced with a number of tokens already deployed in a building:

You could have a radio connection to the last Landmark. You could have a radio connection with the last one. […] Then they go into a room [ . . ] and set the yellow one, go to the next, to the next [and turn it to] green, to the next [and turn it to] green, then they eventually come back and complete this room then.

This anticipation of an artifact in use is rather imprecise and hardly referenced to intersubjective accessible facts that could be verified by the other peers taking part in the dialog. Why a token should be deployed and what the last deployed token should be for are important points that should be determined by a firefighting team. Furthermore, the need for the tokens to be switched to different colors is not explained by S and remains unclear. Later on in this workshop, technical developers shared initial ideas on how the “last token” should be properly visualized on a mobile user interface, even in cases when up to ten tokens would be in range. At that time, the value of the design idea of having a mobile interface had not been discussed with the professional firefighters.

Informed Opinion

In Month 24 of the Landmarke project, an IDF trainer and a professional firefighter had a discussion about what scenarios could be used to provide evidence of whether the designers could modify the Landmarke system could to support more than just navigation during reconnaissance, thus creating added value by providing support for other aspects of delicate operating conditions. In the following, we see one of the exchanges between the IDF trainer JH and T, a professional firefighter with years of operational experience:

JH: “The second one is a scenario which has no practical benefit for the fire service. Because such systems are already available. For example, the system developed by [Company A] provides this …”

(The conversation pauses for a moment while the men look at the device.)

T: “[It’s a] dead-man’s handle!”

(A dead-man’s handle is a small device that can be mounted firmly to the safety equipment and produces noise when a firefighter has not made any movements for a certain span of time, indicating a critical situation.)

These observations were made on the basis of knowledge held by both participants, knowledge that came from the performance and evaluation of experiments with technology but was confirmed by T, who had extensive experience of firefighting practice. Thus, information from experiments and from real-world experience can provide the basis for an emerging consensus. Incident analyses also prompt reflection. In Month 19 of the project, Firefighter U described a situation he experienced in an incident: (p.513)

We checked a floor. The mission was quite normal until we found the burning area, where we found a woman sitting in front of a bed. At this point there was obviously a shift in the mission. To make matters worse, a colleague tried to get the woman out of the building, but he fell down with her sitting on his lap, and hurt himself. Of course the situation changed, they started out quite relaxed, checking every flat, and then all of a sudden there were problems at several locations, and all at once more task forces were required.

Firefighter U then described a possible use for the tokens:

It would have been helpful to have Landmarks at hand, because we were withdrawn to support our colleagues and no one knew which areas had already been checked. The smoke spread through open doors because we had to open all the doors that had previously been closed.

This shows a reflection on work practice, with clear implications for the design of the tokens. On the one hand, Firefighter U reflected on his individual experience of being confronted with a critical situation and the lack of information about which apartments had already been checked for people. On the other hand, the narrative implies the value of a technical tool for marking the doors of the apartments already checked by a reconnaissance team. We called such narratives anticipations modified by reflected experience.1 The use of the tokens in this specific mission was fictional (as they were not yet available when the mission took place). However, since it was informed by real-life experience, this anticipation, as a sort of thought experiment, pointed to a clear practical value for the tokens in support of cooperation in reconnaissance missions.

16.4.6 Supporting Arguments in Dialogs

The kinds of anticipation we have referred to play an important role in producing qualitative assessments of the value of technical artifacts in supporting work practices. In the next lengthy example, we see a discussion of the value of providing a remote-control functionality to change the colors of the markings on the tokens; this discussion illustrates the role of anticipated value. The discussion is from a workshop with eight participants from the scientific and industrial partners of the Landmarke project:

S (from the Fraunhofer Institute for Applied Information Technology): “What is your argument for the remote control?”

M (from a large-enterprise business providing protective equipment for firefighters): “What can we do with it that we couldn’t do before without it?”

I (from a research institute focusing on manufacturing science and rapid prototyping): “I no longer need to go to the Landmark.”

(p.514) S: “You could be connected [by radio] to the last Landmark. You could have a radio connection with the last one. But this is not the case; for example, for the scenarios that we had, the last Landmark is not always the one that got set [to another color]. They enter a room […], they set a yellow one, go to the next and the next [and turn it to] green, green, until they eventually come back to the first room and they complete this new room …”

(S wants to change the color of a specific landmark. He goes on to argue that firefighters may not be able to remember the exact numbers of landmarks as well as the places where firefighters have deployed them in the course of a reconnaissance.)

S: “Whether they still remember how many they have deployed in this room is rather questionable.”

(Here, then, an argument against remote control is being made. M supports this view.)

M: “They are not able to remember a number, they don’t care. You cannot request a firefighter to shout [speaking loudly] ‘I’ve placed Number 3 at the door, 2 is in the bathroom and over there is something like a gas cooker, that is where 4 is then.’”

S: “It could indeed happen that they do just that.”

M: “Oh no!”

S: “When I find [Landmark number] 3 again, I need a reference in my mind, of course it is linked to the surroundings, so that I link the moment now to the moment before. This is important for me!”

M: “That is quite true.”

S: “So, knowing the number would help!”

M: “Yes, of course the number helps! But do we want to force them to remember the number? […]”

L (from the Fraunhofer Institute for Applied Information Technology): “It could strongly be recommended to them, but you couldn’t force them. We cannot make further assumptions at this point.”

(That is, it is uncertain whether firefighters would remember the numbers and their locations.)

M: “That’s my point!”

S: “But then we cannot operate them remotely.”

L: “Unless you could perform actions in some kind of generic way: For example, switching off all landmarks in range.”

Although S and M argue against the idea of having a remote control, L utilizes this argument not to disagree with the assumptions but to provide some closure for the discussion by suggesting that remote control might still be a possibility in situations where reliability is less crucial. What we see here is a careful assessment of the feasibility of an idea, done by examining a variety of scenarios in which it might be useful or, conversely, cause problems. The uncertainty about whether firefighters will be able to remember (p.515) deployment numbers in combination with information about their surroundings leads to the conclusion that the tokens cannot be selected and then changed remotely. L makes the uncertainty explicit (we cannot make further assumptions on the idea of forcing firefighters to remember deployment numbers). It is worth noting that at this point, the discussion partners could have decided to involve the firefighting group on this question but do not. We can assume that their combined experience and knowledge is deemed sufficient for them to examine all realistic possibilities and reach a determination.

16.4.7 Discussion

The rehearsal of the dialog involved in reaching design decisions, and the various interests and experiences that go into the reaching of those decisions, serve to flesh out observations already made about reflexive ethnography. By focusing on deliberations in project workshops, we develop a deeper understanding of how stakeholders in a collaborative project react when confronted with technical artifacts while being involved in design activities informed by analytical reflections on firefighting practice. Our analysis identified several issues as being relevant to the value of anticipation, speculation, and suggestion in informing decisions in the design of technical artifacts. In the case of the Landmarke project, artifacts functioned not only as boundary objects (Star and Griesemer 1989; Stevens 2010) but also as a context from which anticipation, speculation, and suggestion emerged. Such technical “prompts,” which have something in common with the PRAXLABS approach discussed in Chapter 10, not only allow for creative thinking (Kensing and Madsen 1991) but also allow for actors to contribute to the construction of meaning for the central concepts that emerge. Coyne and Snodgrass (1995), when considering the role of metaphors in design thinking and in communicating while solving design problems, remarked that

The emphasis is on design as a collaborative enterprise. … The power of metaphors to define problem regimes suggests a particular approach to design practice. The practitioner does not come to a situation with fixed, pre-defined problem statements, but undertakes an investigation and engages in dialogue through which appropriate metaphors emerge. These metaphors are arrived at by both the practitioner and the client in the specific situation. Problems are presented and addressed through such exchanges and collaborations. (Coyne and Snodgrass 1995, p. 61)

Anticipations in the Landmarke project emerged from creative thinking and were informed by individual experiences from real incidents or by practical evaluations in training missions. They were intersubjectively reflected upon and served as a means to support or reject arguments on the practical value of landmarks. Through observing these anticipations and how they were embedded in the Landmarke dialog, we developed hypotheses that explain how the concept of landmark tokens evolved and, hopefully, provide a more general understanding of how technical artifacts come to be designed. The result of all these interactions represented a form of innovation across professional boundaries (Kimble et al. 2010). In this particular case, reflection on navigation (p.516) practices conducted in the dialog was extremely complex. Some of the issues that concerned the actors in the consortium were as follows:

  • anticipations of problems in real-life reconnaissance missions and in the dynamics of emergency situations, such as how firefighters might respond to specific situations, what tactical considerations are relevant to resolve critical situations, and so on

  • assessments on the relevance of problems for work practice, including estimations of the likelihood of a certain problem to occur

  • standard firefighting practices

  • specification of the Landmarke concept both as a technical component and as a set of services

  • learning about Landmarke features and services and how they could support reconnaissance work (e.g., the practical benefits of having an ad-hoc network that would allow transferring relevant tactical information between firefighters)

  • assessments on the impact of having a token as a navigation tool for ordinary missions

  • assessments on the impact of having a token as a navigation tool in the case of rare but very dangerous situations

An important consequence that we can draw for research practice is that anticipations need to be primarily substantiated by knowledge from experience in order to check whether the assumptions made can be confirmed. Real-world experiments are essential in coping with uncertainty (Klinke and Renn 2002) and, in particular, to reveal unexpected effects (Groß et al. 2005). In the case of firefighting, the latter aspect was very important. Decisions about the feasibility of new reconnaissance tactics were very difficult to make and rather uncertain. Firefighting tactics are procedures conducted in cooperation, which makes them dynamically complex. They have emerged from years of experience in coping with critical incidents in firefighting operations. Furthermore, emergency situations can change in very unexpected ways, and these changes always have an impact on the chosen firefighting tactics. Speculation and anticipation, then, are grounded in existing practice but just as much entail reflection on the degree to which existing practices can change. As we have discussed, anticipations on possible problems in reconnaissance missions, on the practical value of tokens, and so on, were not only supported by experiences (e.g., by working with Landmarke artifacts and by analyzing critical incidents in real missions) but also involved speculation. More specifically, actors oriented toward a specific set of questions, such as the following:

  • Which problematic situations are likely to occur during reconnaissance?

  • What technical option could solve the situation that an anticipation is uncovering?

  • What functions of the artifact are really useful in improving navigation practices, and in what kind of situations (e.g., rare but critical situations, day-to-day missions, or both)?

(p.517) Particularly critical were the assessments of whether a certain problem was worth being researched. As Volkema argues,

Problem formulation plays a crucial role in conducting research and potentially affects succeeding phases, including theory building, research design and conduct, and conclusions. Yet problem formulation is often rushed or taken for granted. People tend to be solution-minded, rather than problem-minded. When problem formulation is rushed or taken for granted, in all likelihood important dimensions of the problem go undetected and opportunities are missed. (Volkema 1995, p. 28)

In this case, the problems of reconnaissance teams in navigation were formulated during the setup of the project, which started from a set of pre-studies conducted with the Paris fire brigade (Denef et al. 2008; Ramirez et al. 2009).

An essential tool for exploring anticipation and confirmation was, in our case, provided by prototyping, a form of the working-with-the-artifact stream proposed by grounded design (see Chapter 1). A good example of the dynamics between anticipation and confirmation can be found in the original assumption that the last deployed token would be highly relevant for improving the navigation capabilities of a firefighting team. This anticipation emerged as a speculation but was confirmed by a number of training missions with firefighters using appropriate mobile user-interface prototypes (Ramirez et al. 2012).

The Landmarke project, as we have seen, allowed us to study research-oriented project work with actors from different professions and with varied experiences. The funding model of the civil security research program of the German Federal Government encourages project proposals aimed at having practical benefits and improving civil security. This design results in a strong involvement of stakeholders coming both from industry and from civil security authorities. This made a good fit with the practice-based research philosophy that provided the foundation of the researchers leading the project.

We have shown how design emerged not only from technical artifacts created as visions of future navigation practices but also from reflection on existing navigation practices as described and embodied by professional practitioners. This served as the argument to explore feasible changes in reconnaissance tactics. Decisions were informed by reflection on experiences coming from real firefighting missions, as well as from practical evaluations of technical artifacts in safe training exercises. What is important about the use of experience in assessing the potential value of artifacts is the fact of communication. Whereas, in other cases, experience is used on an individual basis to provide justification for decisions that went unchallenged in the organization, in this case, the constant exchange of ideas was central to the success of the project. Collaborative reflections on navigation practice justified technical designs to support navigation.

Collaborative evaluations of artifacts allowed firefighting practitioners to reflect on possible problems in relation to navigation tactics. This was critical for progress in the project and helped industry partners, scientists, instructors, and firefighters as a group to identify the potential of the Landmarke technology as a promising tool that could go way beyond simply supporting navigation.

(p.518) 16.5 Historically Grown: Understanding the Design Practices of an IT Research Group

A second ethnography investigates the research practice of our group at the University of Siegen (WiNeMe) and, to some extent, its linkage with EUSSET, its academic reference community at European level. On both levels, the ethnographies deal with community formation and knowledge sharing beyond individual (design) case studies. One of the relatively unexamined set of relationships we have alluded to is that of those that exist above and beyond the ones entailed in a specific case study. Here, then, we outline some features of an ongoing attempt to generate community participation on a more permanent basis.

As we suggested early on in this chapter, so-called post-normal or “Mode 2” research may constitute something of a new paradigm for research activity. One of the less frequently discussed aspects of this putative shift, however, is the degree to which the kinds of disciplinary and organizational heterogeneity that result may well require collaboration and engagement over time. We engage in some early reflections on our own activities with a view to a better understanding of the way in which both an individual research group, WiNeMe, and an academic communities’ research group, EUSSET, operate. EUSSET, comprised of actors from both academia and industry, aims to foster academic and practical exchange between its members, end users, and other practitioners. Ironically, little is known at present about the work practices of these participants engaged in practice-based computing.

Here then, we take early steps as part of a continuing “research about research” to identify how design case studies would be influenced by factors external to the project. Stakeholders coming together in a case study have, as we have already seen, heterogeneous expectations about project work, expectations which are shaped by the traditions, routines, and expectations of their home institutions (Dachtera et al. 2014).

With regard to our investigation of WiNeMe, the specific focus was on the everyday work of interdisciplinary, practice-oriented design activity by (mainly) PhD researchers. While becoming acquainted with their role, practice-oriented researchers developed their own set of practices, which, given the increasing demand for practice-oriented research, deserve to be described in detail.

We will start by introducing EUSSET. It was funded in response to newly emerging European funding schemes that focus on practice-oriented research. European actors (mainly) from the fields of participatory design, CSCW, and HCI found that their voices were not well enough represented in Brussels. Moreover, parts of the European CSCW community did not feel very well treated by ACM SIG-CHI when this US organization moved to rearrange its respective conference schedules. EUSSET, founded in 2011, relies on an infrastructure of personal relations, conferences, and publication venues, most of which existed before, but, it was felt, were too little interconnected to deal with the abovementioned challenges.

EUSSET was founded as a community of mainly university-related actors who were oriented toward practically relevant research and who recognized some of the difficulties (p.519) entailed in managing collaboration across different institutions and with heterogeneous actors. Most of the early participants work at universities. Those who do not still usually have a university background and, often, a PhD degree. Furthermore, their professional positions are often research related (e.g., innovation manager in a company). The community aims to integrate perspectives on empirical research (mainly qualitative, with methods from the social sciences) with technical knowledge and design-related approaches. By aiming to be practically relevant, the community concentrates on the practices of other groups of people (“practitioners” or “end users”) by studying them, publishing empirical material about them, and designing and evaluating IT artifacts together with them in support of their practices. The practices of the community are quite diverse; this is, along with other factors, a function of the differences in national science systems. While there are efforts to integrate these different steps into coherent frameworks, such as “design case studies” (Wulf et al. 2011, 2015), in fact most individual actors have a focus on the particular steps that their disciplines and commitments orient them toward. Relating to and negotiating these concerns with others is a large part of their work.

Much of what is reported here is the work of one of the authors, who, in the course of his PhD research, undertook a lengthy qualitative study. This involved, as is typically the case in such work, an ad-hoc mixture of interview and observation and was made more complex by the very heterogeneity and geographic dispersal we have already referred to. Considerable parts of the research were conducted by the researcher through involvement in project work, project acquisition, university politics, and bureaucracy, along with attendance at international conferences and program-committee meetings (as an observer). Field notes, memos, and group discussions involving other authors played a crucial role in the development of the analysis; they were used to triangulate the interview statements with the results from participant observations and with preexisting knowledge from a literature review. While generally the study had an exploratory character, it was conducted with an eye on the design of IT support and can hence be seen as representing the first step of a “design case study” (cf. Randall et al. 2007; Wulf et al. 2011, 2015). Finally, since EUSSET is an organization in the process of establishing itself, we were interested in how it was constructed by the interview partners. Hence, the method of “ethnography of scaling” proved to be a valuable sensitizing concept for analysis (cf. Ribes 2014). We begin here, however, with an account of WiNeMe. Compared to research into individual design case studies, research into larger research groups or academic communities is still less well understood.

16.5.1 WiNeMe: A Practice-Based Group of’ Design Researchers

The research group in question is based in Siegen, at a midsized German university. Universities typically rely on formal role definitions, such as student, PhD student, professor, administrator, research manager, and so on. These roles entail a certain positioning on the academic career ladder and carry assumptions about the next step to which the holder might aspire. In this manner, master’s students, for example, strive for their master’s degree, and PhD students aim to complete their theses. How such matters are managed in (p.520) practice forms part of our enquiry. WiNeMe, at the time of writing, comprises approximately thirty PhD students, eight postdocs, four faculty members (two with tenure), and thirty-three student researchers. Most of the staff are financed through soft-money projects funded by German government agencies or European institutions. Only the most senior staff hold tenured positions. Most of the external funding comes from project calls, as is typical of the European funding model. Hence, the projects usually run in consortia comprising academic, industry, and end-user partners. Project topics are dependent on the current calls of the funding agencies; at the moment, they are mainly in the fields of IT for the aging society, civil security, energy efficiency, and sustainability.

Members of the senior staff have worked together for approximately 20 years, even though changing academic institutions. One of the senior researchers, whom we shall refer to as Sen2, recalled the days when they started to build the group (still at a different university):

WiNeMe was one of seven areas at [one chair in computer science at the University of Bonn]. […] We had to acquire our positions ourselves. Basically, we acquired our asses off … […] The only thing we could rely on from [the chair] was that when there was money on the horizon, we could sometimes get interim financing from the institute for, say, 3 months.

When the senior researchers started to work in academia as PhD students, they were based at an institute where human-centered or even practice-oriented research was only tolerated at best—if it financed itself from external sources. The institute’s chair was not particularly interested in financing the then emerging senior researchers’ research agenda from his university budget. Hence, the researchers had to rely on third-party funds in order to secure their own positions. On the one hand, the orientation toward project-based funding schemes grew out of a necessity. On the other hand, relying on them also enabled the now-senior researchers to build a research group in an environment that pursued a completely different research agenda (see Wulf et al. 2011). Today, this ambiguity is still reflected in the daily work routines of the researchers, as can be seen by one PhD student, whom we shall call PhD1, when he described his tasks:

So, my practice, what I do is, my work basically consists of project work, publishing and writing project proposals again. What comes into play with the publications is supervising students and teaching, a job which is on the one side […] a bit laborious, but on the other side of course one gets input from the students, who often undertake a lot of project- and practice-related implementations or research work. You can get a lot of input for publications from there.

The daily work practice in the research group is mainly defined by the three interrelated themes of writing proposals, “doing projects,” and writing publications. Furthermore, students are involved in all three themes. This mode of working sustains the group both financially and academically. In Section 16.5.2, we will explore how senior researchers, students, and PhD students accomplish these tasks together, and which work steps are involved therein.

(p.521) 16.5.2 Project Acquisition and Project Work

As mentioned in Section 16.5.1, research projects in WiNeMe are conducted in a variety of thematic areas. The concrete tasks of such research projects vary, but in general, the goals are related to studying the practices of (IT-)users, designing technology for them, or both. These research projects usually combine stakeholders from different research areas with industry partners and end users for a certain period of time (2–4 years). Financing of staff who do not have a tenured position (e.g., PhD students and some postdocs) is hence always for fixed periods of time; the ongoing acquisition of projects is a practical necessity. When asked about how he perceives project acquisition, PhD1, who was in his second year at the time, replied:

Well, writing proposals. I more or less came into my first project because other people had got money in, more or less for me, so that I could get my position. And, because of that, I think it is only fair to take care of the generation after me by getting in money for them. I see it as a cycle […] And thus, for me, writing proposals is just a fixed part of my work. […] And, actually, I personally enjoy writing proposals, especially in a team [ … ] Of course, I particularly include things in them which I personally would like to do—for instance exploring new technologies, if I would like to take a step towards industry […] I also do it because I am personally interested in the results, or in the projects in general […], although it is clear that I will not work in all of them.

Here, the work of writing proposals is portrayed in two ways: on the one hand, for the PhD student, it is morally justified to spend a considerable part of his working time on acquiring projects. He accepts the necessity of doing this work because he also profits from the work of others. On the other hand, he also enjoys this kind of work and sees it in a positive light, as it allows him to participate in decisions about the research topics. The projects are typically acquired in response to funding agencies’ calls for proposals; these calls are structured according to topics that are deemed “societally relevant.” The fact that financing mainly comes from these third-party funds forces WiNeMe members to address the topics that the funding agencies deem relevant, and they have to adhere to the project structure required in order to obtain the funding money.

16.5.3 Acquiring a Research Project

Another PhD student, PhD3 reflected on how the process works in general and refers to a recent experience:

So, there is the set target [to acquire projects], which stands at the beginning and then research opportunities appear, possibilities to research, like the call from the BMBF [German Ministry for Research], Civil Security […] And then […] you start reflecting about it. […] You have to say, [Sen2] has a lot of ideas […] And then he says, write something down and we talk in passing […] and then you talk to other colleagues […] talk to this person and that person, because there once was an application which was rejected but maybe we can still make something out of it.

(p.522) This PhD student is an experienced researcher, having worked in the group for some 8 years and is now in the final phase of his PhD thesis. He has been involved in both successful and unsuccessful project acquisition processes. Here it becomes clear that project acquisition is always a team process, where reflecting about possible research ideas takes place among many different actors in the group. Furthermore, in this reflection process, previous proposal documents with related ideas are taken into account. While PhD3 locates the actual writing work on the level of the PhD students, the more coordinative and strategic work is attributed to the senior researchers:

Mostly it just works out somehow with regard to who is involved in the project acquisition; these are “magical forces” to me, which I have not yet fully understood, because [I am just asked]: […] can you do this and that? […] And it just happens that [Colleague 1] and [Colleague 2] and [Colleague 3] join, which is wonderful, great.

This student is not concerned with the more strategic management work and states he is not entirely aware of the criteria used for selecting colleagues in the division of labor. The PhD student PhD2 held a similar view:

Faculty, then, think much more strategically, also in the long term, which reaches beyond my time as a PhD student, for example. And … where they want to place themselves, where they want to get money from. […] I think I take their advice because they think in terms of the chair’s strategy, I don’t do this myself. […] I [would] rather concentrate on my own work.

Senior researchers thus organize the acquisition process, set the agenda in terms of selecting relevant research calls, and represent their respective teams at the official meetings. It is mainly the quality of their political network that determines the quality of the project team. When it comes to serious issues, they take over the negotiations, and they have to sign the official documents. The main work of writing the proposal document and contacting partners for the first time is, however, mainly done by PhD students, who are motivated by, inter alia, prolonging their work contracts. The exact division of labor and the distribution of competences thus varies considerably. Section 16.5.4 sheds some more light on how a proposal document comes into being.

16.5.4 Writing a Proposal Document

A completed project proposal mainly consists of a single text document. PhD3 explained how this document grows over time:

It is about the project sketch at first […] because besides the project idea and everything concerned with that, you need a consortium […] well, cold calling, well, somebody usually knows somebody […] it is not really cold [ … ] Well. Sometimes it is […] but somehow we have the contacts from somewhere [previously] […] So, what we did before is to develop a one-pager about the project goal, the call, briefly […] what the idea is and how to go about achieving these goals.

(p.523) As can be seen, the first step in getting funding is to assemble a project team according to the criteria put forward by the funding agency. This process is inherently political since it involves negotiating not only the research topic but also the distribution of financial means and tasks, the research approach, and the duties and responsibilities of each of the prospective project partners. Hence, the personal network(s) of the applicant(s) plays a crucial role. Often, partners know each other from previous cooperation, be it from a project or from, for example, jointly authored publications. Acquiring partners completely “cold” is therefore the exception rather than the rule. When assembling a project team, it seems it is crucial to align the requirements from the funding side (e.g., desired research perspectives on the problem, types of partners, distribution of results) with the possibilities allowed by the personal network. This is also the initial spark for the proposal document, which typically emerges as a “one pager” and grows subsequently to eventually become a fully fledged proposal.

The document circulates among the prospective project partners as an internal plan for how to distribute financial means among the project partners and as a convincing research outline for the duration of the project (usually 2–3 years). While writing the document is a team activity, usually the prospective project-leading institution coordinates the process. Building a practice-oriented project means that, besides stakeholders from a variety of scientific disciplines and industry partners, “practitioners” from “society” plays a key role. PhD3 reported on the tension that frequently occurs when negotiating with other partners:

The development and practice partners of course want a rather precise account of what will happen, what they have to do and also, if they come in with a product, have an interest in […] further developing their product. We, however, don’t want to tie ourselves down too much […] rather look at how it will develop […] because it contradicts our approach a bit, because I first want to look at what the practice yields. […] One has to find a way […] concrete enough, but also with enough leeway.

The proposal document is separated into several sections, including a justification as to why the proposed research is societally and scientifically important, a report on the state of the art of the respective area, and a work plan laying down the prospective division of labor. Furthermore, cooperation between the different partners is divided up into discrete tasks, such as empirical studies and knowledge transfer from the end users to the researchers; technical development work for prototypes; evaluation; and others. It is a challenge to account for the openness of the research process on the one hand and to provide enough detail so that a commonly shared idea becomes clear on the other. Even if the idea is clear to all project partners, the project holder still requires a precise account of what will be done during the funding phase. Unsurprisingly, this is done by means of a detailed work plan describing the division of labor in the prospective project on a formal level.

Project acquisition also requires communication with the funding agency or the project holder. Some aspects of this are obvious. The success of a project proposal becomes more likely the more it matches the expectations of the funding agency (and of reviewers). (p.524) While calls already contain a number of requirements for proposals, applicants can fine-tune their project in all respects and hence increase their opportunities of actually obtaining funding by talking personally to the agency’s employees. The political dimension of proposal writing is more complex, as the interests of the participants are often not openly debated, with discussion rather taking place on an ideological and rhetorical level. Following a certain rhetoric or adhering to a particular research tradition may be beneficial for some participants while being problematic for others, as we have previously seen. In addition, the funding agency may also expect a particular rhetoric, for instance, relating the research topic to a particular “societal problem” that it is deemed politically feasible, although not to say desirable, to tackle.

Besides the traditions and interests of the respective institutions, individual researchers also have preferences regarding research traditions, methods, and the topic to be followed in the project. In addition, there may be diverging conceptions of how to involve end users in the proposed cooperation. All these differences need to be mediated in the process of writing the proposal document. The end result is a document that is, in principle, accepted by all partners and that is uploaded in time on the funding agency’s Web portal.

16.5.5 Project Work

Once the acquisition process has been successfully mastered and the project is accepted for funding, the actual project work begins. While, in the proposal document, the project work practices are described on a formal level, it will come as no surprise to experienced researchers that the actual project work may substantially deviate from this description. One reason for this is the legitimation character of the project proposal. After all, the purpose of an application is to get funding money—and its content is designed accordingly. The actual project work may require a different distribution of labor for various reasons and, again, as we have seen, regardless of formal commitments, practical understanding of what these commitments may mean often varies. WiNeMe projects are particularly prone to such changes, because what is to be developed technically is a function of the empirical work that precedes it. As PhD1 said,

So, we are dominated by coordinative meetings and telephone conferences, simply to check the current state of the deliverables or work packages […] Which result documents will be sent where and when? The actual project work is really accomplished together with students; we have employed three or four student assistants who do investigations for us, and we often just finalize and put together the students’ work. And we do the administrative part; that is, the work with other project partners, the writing of the final deliverables. […] Our actual project work is very much coordinative, administrative activity.

WiNeMe frequently takes over the role of the project coordinator and is therefore located at the interface between the other partners. It also typically acts as the expert for end-user research within the consortia. A crucial part of the work hence concerns coordinating activities within the project, organizing meetings, and writing reports. (p.525) Furthermore, student assistants are actively involved in the projects and contribute with implementations and research, often in the context of their BA or MA thesis, to the projects. In addition to this work, a lot of other things have to be done, as PhD2 remarked:

Then all these things crop up that you also need to do, like teaching, small things, organizational stuff like filling out forms for travel expenses, collecting bills, searching for hotels […] or [somebody asks you] can you write a short paragraph on this and that, which doesn’t actually have anything to do with the projects.

University life also includes dealing with the university administration as well as with other bureaucratic and teaching duties. PhD2 found getting acquainted with these procedures particularly irksome. In addition, the culture within the group also includes helping out other colleagues with their work. Although employees are assigned to individual projects, there is a substantial amount of cooperation between them, which in turn requires coordination activities. As PhD2 said,

You prioritize things internally with yourself. […] I have a piece of paper and actually prioritize the tasks and clarify with the relevant persons what the greatest priority is.

Formally, individual researchers are responsible for their own project; in fact, a culture of mutual learning from others’ experiences exists among WiNeMe researchers. Prioritizing tasks is thus a team process in which colleagues synchronize and negotiate prioritization of joint activities.

Even if the work practice within a project deviates from the work plan previously outlined, there are still formal requirements that need to be met throughout the course of the project. These involve regular reports and milestone meetings with the project holder, with these serving to account for the project’s progress. Additionally, bearing in mind the limited financing period of the research project, researchers need to start preparing further applications in order to ensure their continuing employment after the project has ended. It is important to understand this context when considering how the practice-oriented knowledge particular to WiNeMe is produced. Research is always a compromise between the funding agency’s agenda, the necessities within the project, the demands of the end users, and the individual researcher’s personal interests.

16.5.6 Knowledge Creation and Publishing

Publications in WiNeMe usually cover one of the steps of a design case study (Wulf et al. 2011, 2015): a pre-study of the field, or the design and/or evaluation of an IT artifact. Publication venues tend to be decided depending on which step is salient. For WiNeMe PhD researchers, publications usually stem from work related to their respective third-party-funded projects. A complicating issue that constrains practice-oriented research is the disciplinary structure of publication venues. WiNeMe results are typically published outside these mainstream venues. The journals and conferences mentioned by the interviewees are not within the usual scope of any individual discipline. CHI, the best-known (p.526) conference outlet for human-centered computing, may well be the “the premier international conference” of HCI, but HCI is not a well-established or well-understood discipline in the German (or European) academic system as of yet.

16.5.7 Publications as a Part of the Dissertation Process

In order to understand the role of research publications in WiNeMe, it is important to know how the process of acquiring a PhD degree is organized at the University of Siegen, where WiNeMe is based. Faculty guidelines require a thesis including, inter alia, material from two top-tier publications, before the PhD process can officially start. The most important publications for WiNeMe are either ACM CHI conference publications or publications in journals with an International Scientific Indexing impact factor higher than 1. From the perspective of individual PhD researchers, the main role of their publications is to serve as part of their dissertation thesis. The following quotation illustrates how PhD2 saw his publication strategy:

Well, because I want to go back into practice one day, into industry, I don’t have endless time [to complete the PhD]. I would say … if I had a CHI publication sometime soon I would feel much better and would be more relaxed about it … Because you just don’t know … if it doesn’t work out now … you don’t know if it will work out next year either … Of course there’s always the possibility of publishing in journals … but if I had one, let’s say, top-level publication already, I would be more relaxed and I would be able to say, ok, I might publish somewhere else as well [at ACM CSCW, for instance].

Here, PhD2 referred to the annual CHI conference, which, for him, represented an opportunity to take another step toward his PhD degree by having a full paper accepted there. This view was confirmed by other PhD students in many informal interviews. It became clear that among other things, the relevance of CHI papers for the dissertation process represents a major incentive for PhD students to invest time in publishing there. For this reason, it is very important which papers count for the degree at faculty level and which do not. Another relevant issue is that publications also carry politics within the group. The reason behind this is that each paper can only be used by one PhD student for the dissertation. Usually, however, writing is a team process and, hence, authorial rights have to be negotiated. At this point, the distribution of labor within a project becomes relevant, as mentioned in Section 16.5.6. It matters whether researchers mainly spend their time with publications or with other types of work, such as administration or management.

16.5.8 Publication Policy of the Project Holder/Funding Agency

Publications are usually also part of the contract agreement with the funding agencies or the project holder. The writing of several project acquisition processes entails detailing a publication strategy as an explicit work package within a project proposal. In addition, it needs to be borne in mind that different funding authorities follow different policies with (p.527) regard to the venues in which publishing is appreciated. PhD2 mentioned several German agencies, some of which are more oriented toward scientific venues. Others put more emphasis on publication media related to the respective impact on IT industries or practitioners. Students also engage in publishing to satisfy the funding agency’s demands.

16.5.9 IT Media Use in WiNeMe

A range of different IT tools support the described practices. However, a general policy for particular tools does not exist within WiNeMe, as described by one senior researcher:

It has to do with the culture […] there is no tool except email, which is used throughout the entire group […] there are no formalities and also no traditions of use, which leads to a kind of uncontrolled growth.

However, within smaller groups and project teams, a multitude of different tools are used. When Sen2 reflects about his own usage patterns, he argues that he adapts to different audiences with regard to the tools he uses:

I have an account on ResearchGate, […], Facebook, […] Twitter, [ … , and] Google+, but I don’t use them actively. I was invited several times by my employees to build Mendeley groups, which I have accepted [to do] although I haven’t done anything actively yet. As for the future, […] I have planned to work on my infrastructure a bit […] With regards to content, if I work together with Americans, we mainly hold video conferences using Google+ Hangouts and Skype, for instance and I have Dropbox, Skydrive, and other cloud folders in which I store research-related stuff.

Interviewer: “So you do this when working internationally, but with your own employees, with people from our group, you don’t … ?”

Sen2: “Yes, I do! With [Colleague 1] and [Colleague 2], I regularly collaborate on Skydrive […] For years I used Endnote as a database, but don’t do it anymore; [reports on technical problems]. What else? Ok, for investigations I use the ACM Digital Library, Google Scholar, and stuff like this … My own Google Scholar page is not published yet, because […] it needed to be cleaned up first. And besides that, I do an awful lot via email.

The quoted passage makes clear that tool use is heavily dependent on personal preferences and on the concrete social situation with specific individuals. Sen2 uses different tools with different people, but in general he is, at the moment, still mostly reliant on email communication, since it represents the lowest common denominator inside and outside the group; as he says, “It is difficult to change customs and this here has to do with customs.”

16.5.10 Emotional Tensions in Practice-Oriented Work

The previous sections described the circumstances under which publicly funded practice-oriented research takes place. It has been made clear that the position of PhD students is (p.528) characterized by tension. This tension originates from the multitude of tasks that they are confronted with and need to become acquainted with. As PhD2 said,

You are thrown in at the deep end sometimes, you don’t know these things as a student. But it is interesting, you also grow into it. […] But there really is a lot of work which has nothing to do with the concrete research work. It is a lot of networking […] it is also work, because it costs you quite an effort, you have to develop strategies for yourself, you have to learn that for yourself. It is not like: you read it and then you know it, but it has very much to do with personality and personality development, being in this field.

PhD2 described his personal learning process as being, to an extent, painful—and as one in which he had to develop strategies to deal with situations he was not prepared for during his studies. A crucial yet underestimated aspect of work in general, and of interdisciplinary, practice-oriented research work in particular, is the role of emotions. In the process of becoming acquainted with WiNeMe, PhD2 had to learn the aforementioned practices—and not just in a formal and superficial sense. Rather, learning took place in practice, which meant that he was “thrown in at the deep end.” One strategy for dealing with this challenge is for students to concentrate on their respective strengths. This has, however, certain disadvantages as may be seen in the following statement by PhD3:

Then somebody asks you: ‘Why don’t you write publications?’ And I am like ‘Wait a second, I do the project management.’ […] And then, there are all these other things, like teaching, which you cannot skip […] then something has to be dropped and this is usually publishing […] your own research endeavor […] and then again the question: ‘Why don’t you publish?’ And I’m like: Well, if I am working on two project proposals, how should I publish? […] and, well, writing papers is also relevant for one’s cumulative doctorate. So, what counts at the end of the day are publications.

Here, PhD3 mentions the fact that if one solely concentrates on the project management and on writing project proposals, it becomes difficult to work on publications at the same time. This can become a problem, insofar as publications are the basis for the cumulative dissertation model, as explained before. PhD3 hence sees a disadvantage for those researchers mainly concerned with the project management, as they lack the relevant publications in the end.

In a similar manner, other PhD students doing more analytic work expressed the desire to have more control over the practicalities of project work. Since they were less involved in the management decisions of the project work, they felt somewhat marginalized from the course of the projects and felt a lack of control over their own work conditions and about the direction of the work with end users. There is an evident balancing act to be performed, and the balance has to entail both a collective and an individual perspective.

As we have indicated, researchers are usually trained in a particular field, with particular expectations about professional work in that field. Furthermore, the academic cultures in which students are educated during their studies not only consist of a body of knowledge but also carry a certain “habitus” (Bourdieu 1977). When comparing the discussions of more “academically” oriented researchers with those of design-oriented (p.529) researchers, it appeared that this is especially true with regard to issues of understanding and intervention. Empirically oriented subdisciplines from the humanities and the social sciences tend to focus on qualitative fieldwork, with explicit reservations about intervention. Intervention-oriented fields, such as business management, information systems, or design, in turn, usually base their intervention decisions not on extensive and systematic empirical work but rather on models of the world, and these models do not necessarily correspond to the situation at hand. Given that such things are also acquired largely through osmosis, negotiating them entails self-reflection. In addition, the process of socialization into the WiNeMe community of practice goes in hand with learning about one’s own unconscious presuppositions about the science system in general and about related career options in particular.

New members of staff face the project reality with little knowledge about what is actually going to happen. Students educated in academic fields such as informatics, sociology, interaction design, media studies, or other areas, which go in hand with images of particular professional roles, are subjected to interdisciplinary practices with the other project stakeholders from the outset. As previously explained, solely focusing on one’s strengths is not a viable option over time. In addition, as mentioned, tasks are sometimes ambiguously distributed, and participants are irritated by the fact that they have to take on tasks for which they were not prepared for in their academic education. Being confronted with the requirement to carry out such tasks while at the same time maintaining a professional attitude may lead to feelings of stress. As PhD2 said,

And sometimes when the weekend comes, I say I cannot really take a weekend, somehow. And this creates stress and I take the stress with me into the weekend and it happens that I wake up at night and think “Oh dear, I still have to do this and this.” And that is a tendency which is getting stronger at the moment and I actually want to work against it … otherwise the quality of my work declines.

Given this demanding work situation, the PhD degree assumes the function of a reward after a while. As PhD2 said,

Well, at first not really … but then I realized that it doesn’t actually make sense, to take all this […] well, this scientific method, there is an intrinsic motivation, but I think, if you spent so much time doing this, then in the end you also want the PhD degree as an appreciation […] I think intrinsic motivation alone is not enough […] It needs to be compensated and that is the doctoral degree as a reward in the end.

PhD2 is intrinsically well motivated. He is interested in the topics he investigates and is positive about his personal learning process in the group. When he started working in the group, attaining a PhD degree did not have a high priority for him. In the course of his studies, this changed, due to the compensatory nature of the degree. Other interviewees shared this view; it was also mentioned that, in contrast to industry positions, which are often characterized by a high amount of stress as well, being a PhD student had the advantage that the degree represented a challenge as well as a personal benefit.

(p.530) 16.5.11 A Comparative Perspective toward Other EUSSET Groups

Our empirical study included not only actors from WiNeMe but also other researchers related to the European network EUSSET, as previously described. When comparing the insights gained about WiNeMe practices to those gained about other actors, it became obvious that the insights were only transferable to a limited degree. This is the consequence partly of the radically different circumstances under which researchers work, as well as of different epistemological preconditions. The German academic system carries certain particularities that do not necessarily translate to other European countries. While our data concerning different institutions is limited and hence it would be wrong to overgeneralize from our presentation of WiNeMe’s research practices to the whole of EUSSET, it became clear that there are certain similarities and differences. The following findings should stimulate further investigation in the future:

  • As already mentioned, EUSSET members are at the intersection between different scientific disciplines. One feature shared by most EUSSET actors is thus that they pursue a publishing strategy in interdisciplinary venues. Within their home discipline, or the one to which they are formally assigned, EUSSET actors thus frequently experience a problematic relationship vis-à-vis other interests.

  • WiNeMe is probably one of the largest EUSSET-related groups in Europe and has grown considerably during the last decade by means of third-party funding. As previously outlined, this has certain implications concerning project acquisition, project work, and publishing. In our study, it became clear that other EUSSET groups have different ideas about, for instance, their PhD students’ education, and the relationship between senior researchers and PhD students.

  • WiNeMe’s financial base heavily relies on public funding, and this has implications with regard to project length. EUSSET actors differ with regard to a project’s typical duration, since some of them (especially those with a strong tradition in participatory design) can build on a culture of long-term involvement with practitioners, especially in the public sector.

  • While the majority of EUSSET actors are based at universities, the community also includes practitioners from industry. Since these actors’ organizations do not solely rely on public funding to be sustained financially, their daily work practices differ significantly from those of the researchers at universities. An idea of these differences is given in Section 16.5.4.

16.5.12 Discussion

This section has described in some depth the research practices of WiNeMe, one of the German research groups in the EUSSET network. The empirical findings testify to the very particular conditions under which WiNeMe emerged within the German academic system. Without strong governmental funding programs, the origins of the research group (p.531) could not have been established—in an otherwise rather hostile academic environment. The empirical data also indicates how the conditions of WiNeMe’s emergence have shaped the research practices of the group: acquisition and soft-money orientation have become major factors of the group’s identity, since the senior researchers survived in academia in this manner. Thus, WiNeMe has developed elaborated practices of acquiring and working in soft-money projects, which are handed over from generation to generation of PhD students. Interestingly, the group has only a rather basic common IT infrastructure, while subgroups can be rather sophisticated in their technology appropriation. So, IT infrastructures following the CSCW design paradigm could be of value within the group.

WiNeMe is characterized by a certain normative consensus, at least among the group’s senior researchers, that research should be directed toward societal ends, with the goal of contributing to “a better world.” Standing in the tradition of the European CSCW community, the group’s design agenda is directed toward IT support for human activities rather than toward full automatization. There is, then, a somewhat vague but still important consensus that creates a normative grounding and is an important element of group identity. The positioning toward societal ends corresponds often with governments’ funding agenda in the sense that they address societal problems as well.

To understand knowledge sharing within WiNeMe, the concept of “communities of practice” comes to mind (Lave and Wenger 1991; Wenger 1998). About two-thirds of the PhD students and postdocs have been recruited internally—from a wide range of disciplinary backgrounds such as computer science, information systems, anthropology, psychology, sociology, journalism, and design. Being hired as students, they typically are already working in research projects—often writing their master’s thesis and starting to coauthor academic papers. Those who do a good job as a student and want to stay for a PhD degree get hired—in cases where the financial conditions permit.2 There is quite a long process of enculturation into WiNeMe’s community of practice (Wenger 1998). However, the members’ different disciplinary backgrounds, identities, and job trajectories tend to create tensions and insecurities at times and prevent certain actors from full enculturation.

To make a practice-based research paradigm work, one needs an established network of partners in the IT industry as well as among the relevant fields of application. Such a network requires acting, at least in part, on a regional basis, because such a design-research practice and the related knowledge-sharing practices require some level of local proximity (Fischer et al. 2007; Rohde et al. 2007).

The empirical findings make it clear that it is hard work to establish a new interdisciplinary research paradigm in a landscape of traditional academic fields. Occasionally, the senior researchers in the group joke that they would never have started that endeavor if they had known how difficult it would have been and how unlikely final success was at the beginning in the 1990s. In particular, when in the process of conducting the research (p.532) (since the group is only somewhat institutionally settled), the PhD students suffer occasionally from challenging working conditions. This is due to the triple demand of having to acquire externally funded projects, work in those projects, and, at the same time, publish the results at top venues. It is due to the personal structure of the group that, at times, even rather inexperienced PhD students are responsible for managing complex research consortia in a practice-based design paradigm (Dachtera et al. 2014). At the same time, the financial resources enable WiNeMe to invite senior researchers who consult with PhD students in their research and specifically support them in framing high-end publications.

So, the very particular funding situation made the emergence and institutionalization of WiNeMe possible—within the German academic system. It took a long time—from the mid-nineties until now—to elaborate on the practice-based design paradigm documented in this volume. It also demanded, and still demands, quite a bit of effort to position this paradigm in the international academic community of human-centered computing, specifically inside ACM SIG-CHI. The creation of EUSSET was a major step in this direction on the European level. These challenges rest mainly with the group’s senior researchers and other researchers affiliated with WiNeMe and its research paradigm.

16.6 Conclusion

This chapter has presented two ethnographies on design-research practices. We focused on the aspects of how IT design projects are developed and how they are carried out in practice. Each of these studies has a particular focus and style of meta-research. Together, these studies seek to contribute to a better understanding of the domain of practice-based computing.

In so doing, we provide a reflexive account of our own practices. In some ways, our descriptions will appear rather familiar to experienced practitioners in HCI and similar communities. We should remember, however, that the same would be true of any group of people subject to a broadly ethnographic enquiry. Our “subjects” are supposed to be familiar with their own practices. Not all people working in the domain of human-centered computing have that degree of experience with practice-based research, however; and not all of them are university employees. We hope our work will help to familiarize those in more distant situations with the day-to-day reality of this kind of work. Our analysis contributes to the existing literature with regard to “constraints on practice orientation” and “management of practice orientation.”

Ample attention has already been paid in HCI literature to bridging the gap between users and researchers, specifically in the field of participatory IT design. However, given the circumstances in which researchers work, managing end-user involvement in a project context still remains a challenge. Our main point, then, is that practice-oriented research, as a subset of a broader agenda in human-centered computing, remains an even more challenging activity for all participants.

As we have seen, one of the means of achieving practice orientation is, in principle, combining researchers from different disciplines to investigate and intervene in social (p.533) practices. When these “lifeworlds” (Stichweh 1992), “ecological niches” (Lenoir 1997), and “communities of practice” (Lave and Wenger 1991; Wenger 1998) are combined, it is supposed to foster creativity (Fischer 2000) and lead to innovation (Weingart 1997). What is clear, however, is that the learning process implicated in this is problematic across organizations, across disciplines, and between individuals. The observation by one of the PhD students at WiNeMe that felt he had been “thrown in at the deep end” would appear to be characteristic of many experiences at different levels.

We have tried to show that the learning that takes place often has an informal and non-explicit character and, as we saw in our first example, is not always realized. In the light of our examples, it is perhaps useful to distinguish between self-management, collaborative management, and the management of others. With regard to the former and in line with Mansilla et al. (2015) and McCallin and Bamford (2007), we find that heterogeneous work involves a high degree of what Hochschild (2003) calls “emotional labour” and Strauss et al. (1985) describe as “sentimental work.” Both have pointed to the competent handling of emotions as an important dimension of work practices—of flight attendants in the case of Hochschild’s study, and of clinical staff in the case of Strauss et al. This aspect of negotiated academic outcomes is often neglected. The social situation of interdisciplinary work does not encourage the expression of “negative” feelings, given that certain “commitments” are required. This aspect may deserve to be investigated in further research, since recent studies suggest that emotions may play a far greater role for the social structure than has been assumed in the past.

Collaborative management and the management of others involve a range of skills, interests, and practices that we make visible through the reflexive approach we discuss. As we argued in the introduction to this chapter, both practices and the assumptions behind them are heterodox. Consequently, identifying the locus of decision-making in respect of design is not straightforward. Moreover, practices and assumptions originate from a complex network of personal relations, rules, regulations, instructions from external bodies, and so on. In the examination of our own work in the context of the two different studies, we begin to see what those complexities look like and perhaps provide the teachable moments we have recommended.

From a methodological perspective, investigations into research practices often have an auto-ethnographical character—in the sense that the researchers are part of the institutional framing of the research project, group, or community. This is also the case for the two studies presented in this chapter. In the case of the study on “critical moments,” the study was conducted by the coordinator of the Landmarke project as part of his PhD work. In the case of the WiNeMe study, the main researcher—also a PhD student—was part of the group, although not working on any of the design-oriented projects. In the Wan et al. (2016) study, the meta-research was also part of the research activities of the design team, whereas in other cases we created the role of a specific outsider to conduct this type of meta-study (Dachtera et al. 2014). Indeed, the complexity of practice-based research endeavors requires deep and long-term engagement. Investigations into research practices need a culture of openness if an external researcher is working with a design team, and of honesty if the study is conducted by members of the design team themselves.

(p.534) Carrying out ethnographies and communicating their results can be painful and full of conflicts—particularly whenever they try to strengthen reflection and learning within a research team. The meta-researcher may come to grapple with what he might believe to be recurring problems, structural inefficiencies, elements of incompetency, or personal conflicts. If this is the case, the researchers’ analysis is likely to criticize the given practices; some actors may not agree this criticism and might even construe it as personal criticism. In a constellation such as this, the researcher may lose access to the field or even his/her peace of mind, because of the conflicts. From the point of view of the practice-based designer as the object of meta-studies, being challenged or criticized while conducting complex design interventions can be quite stressful. Finally, meta-research can create conflicts for the project coordinators as well as for the group’s management, whose positioning and conduct may be subject to critical analysis and challenged as a result. We have seen instances of all these painful phenomena when setting up a meta-research agenda inside our own research group.

Interestingly, there is little institutional recognition of the necessity of such a meta-research program. In publicly funded research schemes, there are typically not any resources designated for the role of a “meta-researcher.” However, such research findings would be very helpful for improving the performance of practice-based research consortia. Honest and self-reflective empirical analysis could also be a permanent trigger for improving the mechanisms applied within the Central European approach to research funding. The practice-based design community should, however, try to avoid the pitfalls from which the international community of development cooperation suffers—namely, the ongoing creation of sugar-coated evaluation reports.

This paper offers a first outline of a potential meta-research program in practice-based computing. Yet, it is not fully conceptualized and has not been even half carried out. So far, the main focus has been on self-reflectivity and refinement in the field of practice-based computing. Moreover, we need to tackle one of the central challenges in the field: how can meta-research contribute to a comparative analysis of design case studies and thus help the transferability of design-relevant insights? Finally, it is also possible for meta-research to be directly relevant to the design of IT in support of the analyzed research practice. In this sense, the two studies presented so far can also be understood as context studies for the design of IT tools to support practice-based design research. Thus, we are engaged in something we might call “computer-supported cooperative research,”3 the self-application of the CSCW paradigm.


Bibliography references:

Alexander, N., Gottwald, W., and Trenczek, T. 2006. “Mediation in Germany.” In N. Alexander, ed., Global Trends in Mediation (2nd edition). Köln: Otto Schmidt, pp. 179–212.

Altrichter, H., and Posch, P. 1990. Lehrer erforschen ihren Unterricht: Einführung in die Methode der Aktionsforschung. Bad Heilbrunn: Julius Klinkhardt.

(p.535) Argyris, C., Putnam, R., and Smith, D. M. 1985. Action Science: Concepts, Methods and Skills for Research and Intervention. San Francisco: Jossey-Bass.

Argyris, C., and Schön, D. A. 1996. Organizational Learning II. Boston: Addison-Wesley.

Barkhuus, L., and Brown, B. 2012. “The sociality of fieldwork: Designing for social science research practice and collaboration.” In GROUP’12. New York: ACM, 35–44.

Bartunek, J. M. 2011. “What has happened to Mode 2?” British Journal of Management 22 (3): 555–8.

Bell, S., and Morse, S. 2007. “Story telling in sustainable development projects.” Sustainable Development 15 (2): 97–110.

Bergold, J., and Stephan, T. 2012. “Participatory research methods: A methodological approach in motion.” Historical Social Research/Historische Sozialforschung 37 (4): 191–222.

Betz, M., and Wulf, V. 2014. “EmergencyMessenger: A text based communication concept for indoor firefighting.” In CHI’14. New York: ACM, 1515–24.

Bjørn, P., and Boulus, N. 2011. “Dissenting in reflective conversations: Critical components of doing action research.” Action Research 9 (3): 282–302.

Bødker, S., and Iversen, O. S. 2002. “Staging a professional participatory design practice: Moving PD beyond the initial fascination of user involvement.” NordiCHI’02. New York: ACM, 11–18.

Bortz, J., and Döring, N. 2002. Forschungsmethoden und Evaluation für Human—und Sozialwissenschaftler. Heidelberg: Springer.

Bourdieu, P. 1977. Outline of a Theory of Practice. Cambridge: Cambridge University Press.

Bradbury, H., and Reason, P. 2001. “Introduction: Inquiry and participation in search of a world worthy of human aspiration.” In P. Reason and H. Bradbury, eds., Handbook of Action Research: Participative Inquiry and Practice. London: Sage, 1–14.

Bratteteig, T., and Wagner, I. 2014. Disentangling Participation: Power and Decision-Making in Participatory Design. Cham: Springer International.

Bratteteig, T., and Wagner, I. 2016. “Unpacking the notion of participation in Participatory Design.” Computer Supported Cooperative Work 25 (6): 425–75.

Bucciarelli, L. L. 1988. “An ethnographic perspective on engineering design.” Design Studies 9 (3): 159–68.

Callon, M. 1979. “L’État face à l’innovation technique: le cas du véhicule électrique.” Revue française de science politique 29 (3): 426–47.

Callon, M. 1999. “The role of lay people in the production and dissemination of scientific knowledge.” Science, Technology and Society 4 (1): 81–94.

Carrier, M., and Nordmann, A. 2011. “Science in the context of application: Methodological change, conceptual transformation, cultural reorientation.” In M. Carrier and A. Nordman, eds., Science in the Context of Application. Dordrecht: Springer, 1–7.

Coyne, R., and Snodgrass, A. 1995. “Problem setting within prevalent metaphors of design.” Design Issues 11 (2): 31–61.

Dachtera, J., Randall, D., and Wulf, V. 2014. “Research on research: Design research at the margins: Academia, industry and end-users.” In CHI’14. New York: ACM, 713–22.

Davison, R., Martinsons, M., and Kock, N. 2004. “Principles of canonical action research.” Information Systems Journal 14 (1): 65–86.

Denef, S., Ramirez, L., Dyrks, T., and Stevens, G. 2008. “Handy navigation in ever-changing spaces: An ethnographic study of firefighting practices.” In DIS’08. New York: ACM, 184–92.

Dewey, J. 1910. What Is Thought? How We Think. Columbia University Press.

Doing, P. 2008. “Give me a laboratory and I will raise a discipline: The past, present, and future politics of laboratory studies in STS.” In E. J. Hackett, O. Amsterdamska, M. Lynch, and J. Wajcman, eds., The Handbook of Science and Technology Studies (3rd edition). Cambridge, MA: MIT Press, pp. 279–95.

(p.536) Dourish, P. 2006. “Implications for design.” In CHI’06. New York: ACM, 541–50.

Dyrks, T., Ramirez, L., Denef, S., Meyer, D., and Penkert, B. 2009. “Designing for firefighters: Building empathy through live action role-playing.” Available at http://pdfs.semanticscholar.org/e9ca/8cf8f28bdaf66b73162434a139b191885f64.pdf (accessed October 17, 2017).

Emery, F. E., and Trist, E. 1969. Form and Content in Industrial Democracy. London: Tavistock.

EU. 2013. FP7-COOPERATION: Specific Programme “Cooperation” Implementing the Seventh Framework Programme of the European Community for Research, Technological Development and Demonstration Activities. Available at http://cordis.europa.eu/programme/rcn/846_en.html (accessed October 17, 2017).

Evans, C., and Lambert, H. 2008. “Implementing community interventions for HIV prevention: Insights from project ethnography.” Social Science and Medicine 66 (2): 467–78.

Fahy, R. 2002. US Fire Service Fatalities in Structure Fires, 1997-2009. Available at http://tkolb.net/FireReports/2011/US_FatalitiesInStructures77-09.pdf (accessed October 17, 2017).

Fischer, G. 2000. “Symmetry of ignorance, social creativity, and meta-design.” Knowledge-Based Systems 13 (7): 527–37.

Fischer, G., and Giaccardi, E. 2006. “Meta-design: A framework for the future of end-user development.” In H. Lieberman, F. Paternò, and V. Wulf, eds., End User Development, Dordrecht: Springer, 427–57.

Fischer, G., Rohde, M., and Wulf, V. 2007. “Community-based learning: The core competency of residential, research-based universities.” International Journal of Computer Supported Collaborative Learning 2 (1): 9–40.

Funtowicz, S. O., and Ravetz, J. R. 1993. “Science for the post-normal age.” Futures 25 (7): 739–55.

Gaver, W., and Bowers, J. 2012. “Annotated portfolios.” Interactions 19 (4): 40–9.

Gibbons, M., Limoges, C., Nowotny, H., Schwartzman, S., Scott, P., and Trow, M. 1994. The New Production of Knowledge: The Dynamics of Science and Research in Contemporary Societies. London: Sage.

Groß, M., Hoffmann-Riem, H., and Krohn, W. 2005. Realexperimente: Ökologische Gestaltungsprozesse in Der Wissensgesellschaft. Bielefeld: transcript Verlag.

Hayes, G. R. 2011. “The relationship of action research to human-computer interaction.” ACM Transactions on Computer-Human Interaction 18 (3): 15.

Hevner, A. R., March, S. T., Park, J., and Ram, S. 2004. “Design science in information systems research.” MIS Quarterly 28 (1): 75–105.

Hirsch Hadorn, G., Hoffmann-Riem, H., Biber-Klemm, S., Grossenbacher-Mansuy, W., Joye, D., Pohl, C., Wiesmann, U., and Zemp, E. 2008. Handbook of Transdisciplinary Research. Dordrecht: Springer.

Hochschild, A. R. 2003. The Managed Heart: Commercialization of Human Feeling. Oakland: University of California Press.

Hodgkinson, G. P., and Rousseau, D. M. 2009. “Bridging the rigour-relevance gap in management research: It’s already happening!” Journal of Management Studies 46 (3): 534–46.

Jackson, S. J., Steinhardt, S. B., and Buyuktur, A. 2013. “Why CSCW needs science policy (and vice versa).” In CSCW’13. New York: ACM, 1113–24.

Kemmis, S., and McTaggart, R. 1988. The Action Research Planner. Victoria: Deakin University Press.

Kensing, F., and Halskov Madsen, K. 1991. “Generating visions: Future workshops and metaphorical design.” In J. Greenbaum and M. Kyng, eds., Design at Work Cooperative Design of Computer Systems. Hillsdale, NJ: Lawrence Erlbaum Associates, 155–68.

Kieser, A., and Leiner, L. 2009. “Why the rigour–relevance gap in management research is unbridgeable.” Journal of Management Studies 46 (3): 516–33.

(p.537) Kimble, C., Grenier, C., and Goglio-Primard, K. 2010 “Innovation and knowledge sharing across professional boundaries: Political interplay between boundary objects and brokers.” International Journal of Information Management 30 (5): 437–44.

Klinke, A., and Renn, O. 2002. “A new approach to risk evaluation and management: risk-based, precaution-based, and discourse-based strategies.” Risk Analysis an Official Publication of the Society for Risk Analysis 22 (6): 1071–94.

Knorr-Cetina, K. D. 1981. The Manufacture of Knowledge: An Essay on the Constructivist and Contextual Nature of Science. Oxford: Pergamon Press.

Krohn, W. 2008. “Epistemische Qualitäten Transdisziplinärer Forschung.” In M. Bergmann and E. Schramm, eds., Transdisziplinäre Forschung : Integrative Forschungsprozesse Verstehen Und Bewerten. Frankfurt am Main: Campus Verlag.

Latour, B., and Woolgar, S. 1979. Laboratory Life: The Social Construction of Scientific Facts. London:Sage.

Laudel, G. 2006. “The art of getting funded: How scientists adapt to their funding conditions.” Science and Public Policy 33 (7): 489–504.

Lave, J., and Wenger, E. 1991. Situated Learning: Legitimate Peripheral Participation. Cambridge: Cambridge University Press.

Lee, C., Dourish, P., and Mark, G. 2006. “The human infrastructure of cyberinfrastructure.” In CSCW’06. New York: ACM, 483–92.

Lenoir, T. 1997. “The discipline of nature and the nature of disciplines.” In Instituting Science: The Cultural Production of Scientific Disciplines. Redwood City: Stanford University Press, 45–74.

Lewin, K. 1946. “Action research and minority problems.” Journal of Social Issues 2 (4): 34–46.

Lofland, J. 1995. “Analytic ethnography, features, failings, and futures.” Journal of Contemporary Ethnography 24 (1): 30–67.

Lynch, M. 1993. Scientific Practice and Ordinary Action: Ethnomethodology and Social Studies of Science. Cambridge: Cambridge University Press.

Mansilla, V. B., Lamont, M., and Sato, K. 2015. “Shared cognitive–emotional–interactional platforms markers and conditions for successful interdisciplinary collaborations.” Science, Technology, & Human Values 41 (4): 571–612.

Martin, D., Mariani, J., and Rouncefield, M. 2009. “Practicalities of participation: Stakeholder involvement in an electronic patient records project.” In M. Büscher, R. Slack, M. Rouncefield, R. Procter, M. Hartswood, and A. Voss, eds., Configuring User-Designer Relations. London: Springer-Verlag London, 133–55.

Mathiassen, L., and Nielsen, P. A. 2008. “Engaged scholarship in IS research.” Scandinavian Journal of Information Systems 20 (2): 3–20.

McCallin, A., and Bamford, A. 2007. “Interdisciplinary teamwork: Is the influence of emotional intelligence fully appreciated?” Journal of Nursing Management 15 (4): 386–91.

Moris, J., and Hatfield, C. 1982. “A new reality: Western technology faces pastoralism in the Maasai Project.” In International Rice Research Institute, ed., The Role of Anthropologists and Other Social Scientists in Interdisciplinary Teams Developing Improved Food Production Technology. Manila: International Rice Research Institute, 43–61.

Mosse, D. 2007. Notes on the Ethnography of Expertise and Professionals in International Development. Available at http://ceas.iscte.pt/ethnografeast/papers/david_mosse.pdf (accessed October 17, 2017).

Moser, H. 1978. Didaktisches Planen und Handeln. München: Kösel.

Moser, H. 1980. Aktionsforschung als Sozialforschung. Hagen: Studienbrief der Fernuniversität Hagen.

Nett, B., and Stevens, G. 2008. “Business ethnography: Aktionsforschung als Beitrag zu einer reflexiven Technikgestaltung.” In J. Becker, H. Krcmar, and B. Niehaves, Wissenschaftstheorie und gestaltungsorientierte Wirtschaftsinformatik. Dordrecht: Springer, 43–67.

(p.538) Nett, B. 2010. Konstruktion und Rekonstruktion von Technik. Business Ethnography als reflexives Forschungsdesign für Technikentwicklungsprojekte. Habilitation thesis. Siegen: University of Siegen.

Nett, B., Meurer, J., and Stevens, G. 2008. “Knowledge management-in-action in an EUD-oriented software enterprise.” In M. Ackerman, R. Dieng-Kuntz, C. Simone, and V. Wulf, eds., Knowledge Management in Action. Dordrecht: Springer, 139–49.

Nowotny, H., Scott, P., and Gibbons, M. 2001. Re-Thinking Science: Knowledge and the Public in an Age of Uncertainty. Cambridge: Polity Press.

Pedersen, J. 2007. Protocols of Research and Design. Reflections on a Participatory Design Project (Sort of). PhD thesis. Copenhagen: IT University of Copenhagen.

Pipek, V., and Wulf, V. 2009. “Infrastructuring: Towards an integrated perspective on the design and use of information technology.” In Journal of the Association of Information System 10 (5): 306–32.

Pohl, C., and Hirsch Hadorn, G. 2007. Principles for Designing Transdisciplinary Research. München: Oekom Verlag.

Pottier, J. 1993. “The role of ethnography in project appraisal.” In J. Pottier, ed., Practicing Development: Social Science Perspectives. London/New York: Routledge, 13–33.

Österle, H., and Otto, B. 2010. “Konsortialforschung.” Wirtschaftsinformatik 52 (5): 273–85.

Otto, B., and Österle, H. 2010. “Relevance through consortium research? Findings from an expert interview study.” In DESRIST’10. Berlin/Heidelberg: Springer, 16–30.

Ramirez, L., Denef, S., and Dyrks, T. 2009. “Towards human-centered support for indoor navigation.” CHI’09. New York: ACM Press, 1279–82.

Ramirez, L., Dyrks, T., Denef, S., and Stevens, G. 2007. Context as a Resource for Diagnostic Work. Available at http://s3.amazonaws.com/academia.edu.documents/36235316/ecscw07-diagnosing-Ramirez.pdf?AWSAccessKeyId=AKIAIWOWYYGZ2Y53UL3A&Expires=1508270374&Signature=ujY9L%2BWzMOm9nY8ReE0Cz7eB3Ng%3D&response-content-disposition=inline%3B%20filename%3DContext_as_a_resource_for_diagnostic_wor.pdf (accessed October 17, 2017).

Ramirez, L., Dyrks, T., Gerwinski, J., Betz, M., Scholz, M., and Wulf, V. 2012. “Landmarke: An ad hoc deployable ubicomp infrastructure to support indoor navigation of firefighters.” Personal and Ubiquitous Computing 16 (8): 1025–38.

Randall, D., Harper, R., and Roundefield, M. 2007. Fieldwork for Design. Berlin: Springer.

Ribes, D. 2014. “Ethnography of scaling, or, how to a fit a national research infrastructure in the room.” In CSCW’14. New York: ACM, 158–70.

Rohde, M., Klamma, R., Jarke, M., and Wulf, V. 2007. “Reality is our laboratory: Communities of practice in applied computer science.” Behaviour and Information Technology 26 (1): 81–94.

Rohde, M., Stevens, G., Brödner, P., and Wulf, V. 2009. “Towards a paradigmatic shift in IS: Designing for social practice.” In DESRIST’09. New York: ACM, 15.

Rohde, M., Brödner, P., Stevens, G., and Wulf, V. 2017. “Grounded design: A praxeological IS research perspective.” Journal of Information Technology 32 (2): 163–79.

Rosemann, M., and Vessey, I. 2008. “Toward improving the relevance of information systems research to practice: The role of applicability checks.” MIS Quarterly 32 (1): 1–22.

Schmidt, K. 2011. Cooperative Work and Coordinative Practices: Contributions to the Conceptual Foundations of Computer Supported Cooperative Work. London: Springer-Verlag London.

Schön, D. A. 1983. The Reflective Practitioner. New York: Harper Collins.

Sismondo, S. 2008. “Science and technology studies and engaged program.” In E. J. Hackett, O. Amsterdamska, M. Lynch, and J. Wajcman, eds., The Handbook of Science and Technology Studies (3rd edition). Cambridge, MA: MIT Press, 13–31.

(p.539) Slaughter, S., and Leslie, L. L. 1997. Academic Capitalism: Politics, Policies, and the Entrepreneurial University. Baltimore: Johns Hopkins University Press.

Smith, D. E. 2006. Institutional Ethnography as Practice. Lanham: Rowman & Littlefield.

Star, S. L., and Griesemer, J. R. 1989. “Institutional ecology, ‘translations’ and boundary objects: Amateurs and professionals in Berkeley’s Museum of Vertebrate Zoology, 1907–39.” Social Studies of Science 19 (3): 387–420.

Stevens, G., and Nett, B. 2009. Business Ethnography as a Research Method To Support Evolutionary Design. http://s3.amazonaws.com/academia.edu.documents/8809932/stevens-nett_ethnography_evolutionary-design_2009.pdf?AWSAccessKeyId=AKIAIWOWYYGZ2Y53UL3A&Expires=1508271397&Signature=Y0%2BqNnOoVrivSdA4BHyhQRKGB%2F8%3D&response-content-disposition=inline%3B%20filename%3DBusiness_Ethnography_as_a_research_metho.pdf (accessed October 17, 2017).

Stevens, G. 2010. Understanding and Designing Appropriation Infrastructures: Artifacts as Boundary Objects in the Continuous Software Development. Available at http://dokumentix.ub.uni-siegen.de/opus/volltexte/2010/433/ (accessed October 17, 2017).

Stevens, G., Schwartz, T., and Meurer, J. 2009. “A dialectic view on Open Innovation.” In AMCIS’09. Atlanta: AIS, 668.

Stichweh, R., 1992. “The sociology of scientific disciplines: On the genesis and stability of the disciplinary structure of modern science.” Science in Context 5 (1): 3–15.

Strauss, A., Fagerhaugh, S., Suczek, B., and Wiener, C. 1985. Social Organization of Medical Work. Chicago:University of Chicago Press.

The Federal Ministry of Education and Research. 2009. Research for Civil Security: Protection Systems for Security and Emergency Services. Available at www.straz.gov.pl/download/1133 (accessed October 17, 2017).

Van De Ven, A. H. 2007. Engaged Scholarship: A Guide for Organizational and Social Research. Oxford: Oxford University Press.

Volkema, R. J. 1995. “Managing the problem formulation process: Guidelines for team leaders and facilitators.” Human Systems Management 16 (1): 27–34.

Wan, L., Müller, C., Randall, D., and Wulf, V. 2016. “Design of a GPS monitoring system for dementia care and its challenges in academia–industry project.” ACM Transactions on Computer Human Interaction 23 (5): 31.

Weingart, P. 1997. “From ‘finalization’ to ‘Mode 2’: Old wine in new bottles?” Social Science Information 36 (4): 591–613.

Wenger, E. 1998. Communities of Practice: Learning, Meaning, and Identity. Cambridge: Cambridge University Press.

Winschiers-Theophilus, H., Chivuno-Kuria, S., Kapuire, G. K., Bidwell, N. J., and Blake, E. 2010. “Being participated: A community approach.” In PDC’10. New York: ACM, 1–10.

Wulf, V. and Rohde, M. 1995. “Towards an integrated organization and technology development.” In DIS’95. New York: ACM, 55–64.

Wulf, V., Rohde, M., Pipek, V., and Stevens, G. 2011. “Engaging with practices: Design case studies as a research framework in CSCW.” In CSCW’11. New York: ACM, 505–12.

Wulf, V., Müller, C., Pipek, V., Randall, D., Rohde, M., and Stevens, G. 2015. “Practice-based computing: Empirical grounded conceptualizations derived from design case studies.” In V. Wulf, K. Schmidt, and D. Randall, eds., Designing Socially Embedded Technologies in the Real-World. London: Springer-Verlag London, 111–50. (p.540)


(1) A sort of “alienation effect” used as a resource to reflect on project work (Stevens and Nett 2009).

(2) PhD students are rather well paid in the German academic system. In computer science, they typically get a full position at an annual gross salary of some €40,000.

(3) We owe this term our dear and invaluable colleague Erhard Schüttpelz.