2
Foundations and New Directions

VALERIE CHEPP

Hamline University

CAROLINE GRAY

Research Institute of the Palo Alto Medical Foundation

2.1 INTRODUCTION

Theory has played a prominent role in the advancement of question design and evaluation. This advancement was ushered in as theories of cognitive psychology were applied to survey methodology. Prior to the advent of the cognitive aspects of survey methodology (CASM) movement, there was little theoretical discussion regarding question response. As Sudman et al. (1996) note, before this time “the work conducted in this domain suffered from a lack of theoretical perspective” (p. 7).

CASM is a critical achievement for survey methodology since theory guides the ways in which empirical research is conducted, as well as why it is conducted in the first place. It also provides insight into why some methods are more appropriate for specific types of research questions than others. Succinctly, CASM established a basis for scientific inquiry into question response and question evaluation. It also laid the foundation for establishing methodological approaches for conducting question evaluation studies.

This chapter will first describe the theoretical perspective underlying the method presented in this book. Specifically, this book is set within an interpretivist framework in which the construction of meaning is considered elemental to the question-response process. The method and methodological considerations presented in this book focus on the collection and analysis of interpretive patterns and processes that constitute the question-response process. This chapter will also describe implications for question response and question evaluation as well as recent directions in the study of interpretation and cognition as it pertains to cognitive interviewing. This discussion focuses on an emerging subfield of interpretivism: cognitive sociology. In addition, three key methodological concepts central to this tradition (narrative, Verstehen, and thick description) are examined in relationship to cognitive interviewing methodology.

2.2 SOCIOLOGY AND THE INTERPRETIVIST TRADITION

To date, insights from sociological theory have not been fully integrated into the study of questionnaire design and evaluation. Given that the survey process is fundamentally a social encounter (Sudman and Bradburn 1983; Sudman et al. 1996; Groves and Couper 1998), and given that sociologists have spent the past century theorizing the rules of social interaction (Cooley 1902; Mead 1934; Goffman 1959; Blumer 1969), sociological thought has much to offer the field of question evaluation. In fact, Sudman et al. (1996) argue that “the rules that govern conversations and social encounters in general should help us understand how survey questions are being understood and answered” (p. 1).

A sociological approach to any field of study recognizes that all human behavior takes place in a social environment. The social environment includes many different components, but all sociological work seeks to uncover patterns that arise out of humans interacting with their social worlds. In other words, there is some regularity or “structure” to the social world. Social location refers to the ways in which individuals and groups are differently located within a social structure, based upon socially constructed cultural markers such as race, ethnicity, gender, social class, sexuality, and disability status, among others. Although these cultural markers are socially constructed (i.e., they are not tangible objects but rather meaningful constructs, ideas, or perceptions that are generated through social processes), they have very real consequences in that they intersect to systematically shape individuals' and groups' social experiences as well as their worldviews or interpretations of their experiences (Collins and Chepp 2013). This does not imply that everyone of a particular gender, race, or class interprets the world in the same way, rather social experiences and opportunities that arise out of one's social location shape their interpretations. This includes interpretation of survey questions.

While interpretivist approaches to human cognition have largely been absent from discussions of the method, some researchers have drawn upon interpretivist sensibilities in their conceptualizations of cognitive interviewing (Gerber and Wellens 1997; Gerber 1999; Miller 2011). For example, Gerber (1999) argues that the utility of incorporating interpretivist modes of analysis—and the focus is specifically on ethnography—into the survey context pivots on its “ability to represent complexity” (Gerber 1999, p. 219; see also Miller 2011).

Although sociologists are generally interested in identifying patterns of human interaction with the social world, like any discipline, sociology has numerous subfields, each rooted in different intellectual traditions. Interpretivism emphasizes the meaningful quality of individuals' engagement in the social world. Furthermore, it recognizes that understanding of the social world is filtered through a complex set of interpretations that are variously informed by social experiences and cultural contexts.

Interpretivist approaches attempt to not only identify but also to understand the different realities that social actors construct. For instance, in the case of a survey question that asks respondents how many times they visited a doctor in the past year, there are numerous potential interpretations that this question might elicit that may be shaped by social factors such as a respondent's age, education level, cultural background, health insurance status, or health condition. Some respondents, for example, may interpret the term “doctor” to mean general practitioner, while others may include specialists such as surgeons, gynecologists, dermatologists, allergists, or podiatrists. Still others may understand the term more broadly to include visits to the dentist or eye doctor. In addition to conventional definitions of “doctor” rooted in a Western medical model, respondents might variously include (or not include) visits to non-traditional practitioners, such as midwives, chiropractors, or acupuncturists.

Although some of these interpretations may not be the intent of the question designer, none of these interpretations are inherently “wrong.” It is understandable that interpretations may vary across respondents given their different circumstances and experiences. Moreover, individual respondents' interpretations could shift if their circumstances or experiences change. Among interpretivists, it is widely accepted that multiple and fluid meaning patterns can exist and shift over time. It is the analyst's responsibility to identify and make sense of these varying and potentially shifting interpretations across respondents. As a result of this analytic work, it is possible to identify how a question performs across a range of respondents with differing backgrounds or experiences. Within a framework that emphasizes meaning as an elemental component of question response, a cognitive interviewing study can detail the various phenomena captured by a question and, ultimately, represented in a survey statistic.

2.3 NEW DIRECTIONS: INTERPRETATION AND COGNITION

Although the four stages of question response—comprehension, retrieval, judgment, and response—have traditionally been understood as uniquely cognitive psychological processes, cognitive psychologists are not the only scholars interested in mental processes. Cognitive sociology, a relatively recent theoretical and empirical development in sociology, is a subfield of interpretivism that provides particularly fertile ground for thinking about cognitive interviewing methodology within an interpretivist framework.

The aim of cognitive sociology is to demonstrate the numerous ways in which cognitive processes can be understood from a sociological perspective, suggesting, above all, that cognitive processes are shaped by cultural phenomena. The field is interested in understanding the categories, schemes, and codes that individuals use to organize their thoughts and make sense of the world around them. Rather than view these categories, schemes, and codes as universal as many cognitive psychologists do, cognitive sociology instead argues that these “thought structures” are the product of the social environment (Zerubavel 1997). However, although such thought structures are not universal and static, they are not idiosyncratic or individual either. The thoughts that enter minds are rooted in broader social processes and social relations. The most obvious example of this is language itself. Languages are only meaningful insofar as members of a language community share that same language and can understand what the different words it encompasses mean. But language can be understood more figuratively as shared meaning systems where individuals become socialized into shared (though not always) ideas and thought patterns. Members of society become socialized into distinctive “thought communities” (Zerubavel 1997) that both inform and reflect the inter-workings of their minds.

Cognitive sociology is interested in many of the same processes as cognitive psychology, but analyzes these cognitive processes in a different way. For example, DiMaggio (1997) outlines how cultural mental schemata inform “the way we attend to, interpret, remember, and respond emotionally to the information...