Transcription
CHAPTER NINESurvey ResearchPBNNY S. VISSBR, JON A. KROSNICK, AND PAUL J. LAVRAWSSocial psychologists have long recognized that everymethod of scientific inquiry is subject to limitationsand that choosing among research methods inherentlyinvolves trade-offs. With the control of a laboratoryexperiment, for example, comes an artificiality thatraises questions about the generalizability of results.And yet the 'naturalness" of a field study or an observational study can jeopardize the validity of causal inferences. The inevitabilityof such limitationshas led manymethodologists to advocate the use of multiple methods and to insist that substantive conclusions can bemost confidently derived by triangulating across measures and methods that have nonoverlapping strengthsand weaknesses (see, e.g., Brewer, this volume, Ch. 1;Cpmpbell 6 Piske, 1959; Campbell 6 Stanley, 1963;Cook 6 Campbell, 1969; Crano 6 Brewer, 1986; B.Smith, this volume, Ch. 2).This chapter describes a research methodology thatwe believe has much to offer social psychologists interested in a multimethod approach: survey research.Survey research is a specific type of field study that involves the collection of data from a sample of elements (e.g., adult women) drawn from a well-definedpopulation (e.g., all adult women living in the UnitedStates) through the use of a questionnaire (for morelengthy discussions, see Babbie, 1990; Fowler, 1988;This chapter was completed while the second author was aFellow at the Center for Advanced Study in the BehavioralSciences, supported by National Science Foundation GrantSBR-9022192. Correspondence should be directed to Penny S.Visxr, Department of Psychology, Princeton University, Princeton, New Jersey 08544 (e-mail:pvisseSPrinceton.edu). or JonA. Krosnick. Department of Psychology, Ohio State University,1885 Neil Avenue, Columbus, Ohio 43210 (e-mail:[email protected]).Frey, 1989; Lavrakas, 1993; Weisberg, Krosnick, 6Bowen, 1996).We begin the chapterby suggestingwhysurvey research may be valuable to sodal psychologistsand then outline the utility of various study designs.Next, we review the basics of survey sampling andquestionnaire design. F i y , we describe proceduresfor pretesting questionnairesandfor data collection.REASONS FOR SOCIAL PSYCHOLOGISTS TOCONDUCT SURVEY RBSEARCHSocial psychologists are interested in understandinghow people influence, and are influenced by, their social environment. And to the extent that social psychological phenomena are universal across differenttypes of people, it makes little difference precisely withwhom social psychological research is conducted even data collectedfrom samplesthat are decidedly unrepresentative of the general population can be used todraw inferences about that population.In recent years, however, psychologists have become increasingly sensitive to the impact of dispositional and contextual factors on human thought andsocial behavior. Instead of broad statementsabout universal processes, social psychologists today are far morelikely to offer qualified accounts of which people, under which conditions, are likely to exhibit a particular psychological phenomenon or process. And accordingly, social psychologists have increasingly turnedtheir attention to interactions between various socialpsychological processes and characteristics of the individual, such as personality traits, identification with asocial group or category, or membership in a distinctculture. In many cases, the nature of basic social psychological processes has been shown to depend to alarge degree on characteristics of the individual.
2 UPENNY S. VISSER, JON A. KROSNICK, AND PAUL J. LAVRAKASThe process by which attitude change occurs, forexample, has been shown to differ for people who arelow and high in what Petty and Cacioppo ( 1986)havetermed 'need for cognition," a general enjoyment ofand preference for effordul thinking. Attitude changeamong people high in need for cognition tends to bemediated by careful scrutiny of the arguments in a persuasive appeal, whereas attitude change among peoplelow in need for cognition tends to be based on cues inthe persuasive message or context, such as the attractiveness of the source.Similarly, attributionshave been shown to differ depending on social group membership (see, e.g., Hewstone, Bond, 6 Wan, 1983). People tend to attributepositive behaviors by members of their own socialgroup or category to stable, internal causes. Those samepositive behaviors performed by a member of a different social group, however, are more likely to be attributed to transitory or external factors.According to much recent research, culture mayalso moderate many social psychological phenomena.Markus and Kitayama (1991), for example, have argued that members of different cultures have different construals of the self and that these differencescan have a profound impact on tlye nature of cognitive, emotional, and [email protected]. Similarly,Nisbett and his colleagues'(Coheh, Nisbett, Bowdle, 6Schwan, 1996; Nisbra, 1993; Nisbett 6 Cohen, 1996)have exploredwhat they tcnned the 'culture of honorwof the American South and have demonstrated markeddifferencesin the cognitive, eimotiod, behavioral, andeven physiological reactions af southem men (relativeto their northern counterparts) when confronted withinsult.These kinds of process-by-indi--differenceinteractions suggest that precisely who participates in social psychological research can have a profound impact on what results are obtained. And of course, forthe vast majority of social psychological research, that'who" has been the infamous college sophomore. Sears(1986) has argued that the field's overwhelming reliance on this narrow base of research -stmay represent a serious problem for s o d a l psychology. Pointing to various attributes that are characteristic of young adults, Sears (1986) suggested thatthe 'college sophomore" partidpant pool is unrepresentative of the general population in a number of important ways. Among other things, youngadults are more susceptible to attitude change (Alwin,Cohen, 6 Newcomb, 1991; Glenn, 1980; Krosnick 6Alwin, 1989; Sears, 1983), exhibit less stable personality traits (Caspi, Bem, 6 Elder, 1989; Costa, McCrae,6 Arenberg, 1983; Nesselroade 6 Baltes, 1974), havemore weakly established self-images (Mortimer, Pinch,6 Kumka, 1982), and have less well-developed socialidentities (Alwin et al., 1991)than-do older adults.Because of these kinds of differences, Sears (1986)argued, the field's reliance on participant pools ofcollege-aged adults raises questions about the generahability of some findings from social psychologicallaboratory research and may have contributed to a distorted portrait of 'human nature." However, the evidence Sears (1986) cited largely reveals the prevalenceof certain characteristics (e.g., the frequency of attitude change or the firmness of social identities),ratherthan differences in the processes by which these characteristics or others emerge in different age groups. Wecurrently know so little about the operation of socialpsychological processes in other subsets of the population that it is impossible to assess the extent of bias inthis regard.Doing so will require studies of samples that arerepresentative of the general population, and inducing most members of such samples to visit a laboratory seems practically impossible. Studying a representative sample through field research, however, isrelatively easy and surprisingly practical. Using the basic tenets of probability theory, survey researchershavedeveloped a number of effident strategies for drawingrepresentative samples that are easy to contact. Andwhen samples have been selected in such a manner,social psychologists can confidently generalize findingsto the entire population. Furthermore, survey researchprovides ideal conditions for the exploration of Process x Individual Difference interactions because carefully selected samples reflect the full heterogeneity ofthe general population.There are two primary limitationsof survey researchfor social psychologists. First, surveys are more expensive and time-consumingthan most laboratory experiments using captive participant pools. However, manycost-saving approaches can be implemented. Second isthe impracticality of executing elaborate scripted scenarios for social interaction, especially ones involvingdeception. Whereas these sorts of events can be a e ated in labs with undergraduate participants, they aretougher to do in the field. But as .we discuss shortly,many experimental procedures and manipulations canbe incorporated in surveys.Put simply, social psychology can happily proceeddoing most of our research with college sophomores,assuming that our findings generalize. And we can livewith the skepticism of scholars from other disciplineswho question that generalizabiity, having documented
that can be brought about by respondents' own behavior (e.g., misreporting true attitudes, failing to payclose attention to a question), interviewer behavior(e.g., misrecording responses, providing cues that leadpartidpants to respond in one way or another), andthe questionnaire (e.g., ambiguous or confusing question wording, biased question wording or responseoptions).The total s w e y error perspective advocates explicitly taking into consideration each of these sources ofTOTAL SURVEY ERRORerror and making decisions about the allocation of finiteresources with the goal of reducing the sum of theEven researchers who recognize the value of s w e ymethodology for socialpsychological inquiry are some- four. In the sections that follow, we consider each oftimes reluctant to initiate s w e y research because of these potential sources of survey error and their immisconceptions regarding the feasibility of conducting plications for psychologists seeking to balance praga survey on a limited budget. And indeed, the cost of matic budget considerations against concerns aboutprominent large-scale national sweys conducted by data quality.major s w e y organhtions are well outside of the Rsearch budgets of most social psychologists. But survey STUDY DESIGNSmethodologists have recently begun to rekindle andexpand the early work of Hansen and his colleagues Sweys offer the opportunity to execute studies with(e.g., Hansen 6 Madow, 1953)in thinking about survey various &signs, each of which is suitable for addressingdesign issues within an explicit cost-benefit framework particular research questions of long-standing interestgeared toward helping researchers make design dea- to social psychologists. In this section, we will reviewsions that maximhe data quality within the constraints several standard designs, including aoss-sectional, reof a limited budget. This approach to survey methodol- peated cross-sectional, panel, and mixed designs, andogy, known as the 'total s w e y error" perspective (d. discuss when each is appropriatefor social psychologiDillman, 1978, Fowler, 1988; Groves, 1989; Lavrakas, cal investigation. We will also review the incorporation1993), can provide social psychologists with a broad of experiments within sweys.framework and specificguidelinesfor making decisionsto conduct good sweys on limited budgets while maxCross-Sectional Surveysimizing data quality.Cross-sectional surveys involve the collection ofThe total survey error perspective recognizes thatthe ultimate goal of s w e y research is to accurately data at a single point in time from a sample drawn frommeasure particular constructs within a sample of a speci6ed population. This design is most often usedpeople who represent the population of interest. In any to document the prevalence of particular characteristicsgiven survey, the overall deviation from this ideal is the in a population. For example, aoss-sectional sweyscumulative result of several sources of survey error. are routinely conducted to assess the frequency withSpedfically, the total s w e y error perspective disag- which people perform certain behaviors or the numgregates overall error into four components: coverage ber of people who hold particular attitudes or beliefs.error, sampling error, nonresponse error, and measure- However, documenting prevalence is typically of littlement error. C w a g e mor refers to the bias that can interest to social psychologists, who are usually moreresult when the pool of potential survey partidpants interested in documenting assodations between varifrom which a sample is selected does not include some ables and the causal processes that give rise to thoseportions of the population of interest. Smnpling mor associations.Cross-sectional surveys do offer the opportunity torefers to the random differences that invariably existbetween any sample and the population from whicb it assess relations between variables and differences bewas selected. Nonresponse m r is the bias that can re- tween subgroups in a population. But although manysult when data are not collected from all of the mem- scholars believe their value ends there, this is not thebers of a sample. And musumnmt mor refers to all case. Cross-sectional data can be used to teit causaldistortions in the assessment of the construct of inter- hypotheses in a number of ways. For example, usingest, including systematic biases and random variance statistical techniques such as two-stage least squaresthe profound impact that context and history have onsocial processes. Or we can accept the challenge andexplore the replicabiity of our findings in the generalpopulation. Either we will confirm our assumptions ofgeneralizability or we will reline our theoriesby addingto them new mediators and moderators. The explication of the survey method offered below is intended tohelp those who accept the challenge.
226PENNY S. YISSBR, JON A. KROSNICK, AND1 PAUL J. LAVRAWSregression, it is possible to estimate the causal impactof variable A on variable B, as well as the effect of variable B on variab
Smith, this volume, Ch. 2). This chapter describes a research methodology that we believe has much to offer social psychologists in- terested in a multimethod approach: survey research. Survey research is a specific type of field study that in- volves the collection of data from a sample of ele- ments (e.g., adult women) drawn from a well-defined population (e.g., all adult women living in the .
