[ NB: “Annotations” are occasional posts that explore selections from my research reading—articles or books—in rhetoric, technical and professional communication, and related fields. ]
Hutchins, E., & Klausen, T. (1996). Distributed cognition in an airline cockpit. In Y. Engeström and D. Middleton (Eds.), Cognition and communication at work (15–34). Cambridge: Cambridge University Press.
This piece is interesting from a number of perspectives, but I want to focus here on the methods section, which is both fascinating and disappointing to me. So, this isn’t a full annotation of the piece, though I will provide a bit of context up front.
The chapter begins with a transcript from a flight simulation involving a commercial airline Captain, First Officer, Second Officer, and two different ground crews (air traffic control). This transcript, of course, makes little sense unless you are familiar with the vagaries of commercial airline flights; indeed, this is part of the point.
The authors foreground a systemic approach for analyzing this scenario early in the chapter. Analysis of a system, rather than an actor or even (only) the interactions between actors, calls for a different unit of analysis, they contend, one that must foster the description and exploration of “the cognitive properties of the cockpit system that is composed of the pilots and their informational environment” (p. 17).
The unit of analysis, then, is a system of distributed cognition (p. 17). The chapter, Hutchins and Klausen explain, is a descriptive implementation of a theory of distributed cognition—they’ll go through the episode of flight simulation chronologically and describe the actions and operations of actors as occurring within a system of distributed cognition.
“We will attempt to show,” they argue, "that certain observed behaviors are instances of certain theoretical concepts. It is only by mapping from the data to a theory that we can generalize beyond the specifics of these observations” (p. 17). This leads them into the cognitive problems of actually doing what they say they’ll do, a meaningful digression, therefore, into their methods section.
In their study, “the theoretical interpretation of some events may depend on the meanings that the participants themselves attach to those events” (p. 17). There are no simple operational definitions of terms that will easily and seamlessly allow readers to interpret, with the authors, the meanings at the research site. So, the authors argue, “we must rely on an ethnography of the setting to provide the interpretive bridge from the structure of the recordings of activity to the terms of the theory of distributed cognition” (p. 17). This moves them into a discussion of their data collection, and the specific affordances of those forms of data. Some of this is very Inside Baseball (which is worth attending to), but it’s also somewhat cynical, as we’ll see below.
They note that “One way to protect oneself from the possibility of unexamined assumptions” in writing up ethnographic research is to try to build some kind of “objectivity” through which such assumptions are banished (p. 18). They point out that these kinds of approaches “cling to a ‘coding scheme,’ a set of ‘objective criteria’ for the existence of instances of various classes of events” (p. 18). Here’s where the cynicism creeps in, for me. Beyond the scare quotes, this seems a very cynical view of qualitative coding; they note that coding schemes are a function of the skill and situated interpretations of particular researchers. Well, yes. Who is arguing otherwise?
I don’t know any qualitative researcher who would claim objectivity because of a coding scheme. I don’t recall reading any key sources in qualitative methodology from around the time of this chapter (cf. Miles & Huberman, 1994; Strauss, 1987; Strauss & Corbin, 1990; etc.) that claims as much (I could certainly be wrong, of course). Instead, coding schemes provide, among many other things, analytic systematicity and transparency.
But the authors argue for another approach: making sure that their assumptions do not remain unexamined (p. 18). They “ground the translation from video and audio record to transcription in an explicit set of propositions that are independently verifiable in the ethnography of the setting” (p. 18). But shouldn’t a grounded coding schema be expected to do the same? This is where they’ve lost me to a certain extent. They seek to be “completely explicit about the grounds for the composition of every action” (p. 19). “The development of ethnographic grounding,” they argue, “leads us to many sources of cultural knowledge,” including documentation, layout and arrangement of cockpit instrumentation, heuristics, interviews, observations, etc. (p. 19).
I’m with them here, but I’m not sure how this differs from other forms of qualitative analysis except for the fact that they intend to be more narrative about an ethnomethodological dataset (looking at very small episodes of high granularity, using a narrative to richly describe each action and operation rather than a coding scheme to represent broader chains of actions).
Make no mistake: this is a worthwhile read, but I was really puzzled by their methods section. Have you read this piece? What do you make of the methods section?
0 comments:
Post a Comment