Transcription
DOCUMENT RESUMECS 207 189ED 221 867TITLEINSTITUTIONSPONS AGENCYREPORT NOPUB DATECONTRACTNOTEEDRS PRICEDESCRIPTORSIDBNTIFIERSDocument Design Project. Final Report.American Institutes for Research, Washington, D.C.;Caenegie-Mellon Univ., Pittsburgh, Pa.; Siege1,1,Gale, Inc., New York, N.Y.National Ingt. of Education (ED), Washington,,DC.Teaching and Learning Program.AIR-75003-11/81-FRNov 81400-78-0043130p.; Several attachments will not reproduce.MF01/PC06 Pius Postage.*Government.publications; Higher Education; *Layout(Publications); Program Deacriptions; *Readability;*Technical Writing; *Writing (Composition); WritingInstruction; Writing Research*Document Design ProjectABSTRACTThe accomplishments of a team of writers, designers,,and education specialists commissioned by the National Institute otEducation (NIE) to suggest solutions to the problems that publicdocuments often pose for readers are summarized in this report.Following an overview, the report offers a section on the history andrationale of the Document Design Project. The next two sectionsexplain the research and technical assistance and training componentsof the project. The fourth section discusses the project'sundervaduate curticulum to improve the teaching of writing, whilethe fifth section presents graduate programs in rhetoric and writingat Carnegie-Mellon University. Section 6 describes efforts atdisseminating information about the project's work, and the reportconcludes with project plans beyond NIE funding. An appendix containstechnical reports, publications, articles, and presentations byDocument Design Project staff. *****************************Reproductions supplied by EDRS are the best that can be made**from the original ********.**********************
UAL mamma Os COMATIOISNATIONIA IMITITUTI OP IMUCATION.row A HtIA At 10-11,CA,mCf st gmeOgwArgON)(CfMttil 4416COAIR-75003.11/81.FR1""4CIeh*ue.s N. t, kon,1,040 0#.4 1#1., if Pee** 0, 0,0 011.:okzenA.iosIA.** Ovipor) hay. kOlsii -0,404 kr grgiti4.pit4.4 Niel 04A4yrJpPty.urni*(((, .60 .rnatoy Nphwrit ortErt Ni(the( *AutfroNs-totkuDocument Design ProjectFinal ReportPrepared for the National Institute of Education, Washington; D.C.under contract No. NJE-400-78-0043November 1981AMERICAN INSTITUTES FOR RESEARCH/1055 Thomas Jefferson Street, NW, Washington, DC 20007 202/342-5000with CARNEGIE MELLON UNIVERSITYand SIEGEL & GALE2
L ,This report is part of the work done under contract #40078-0043 of the. National /nstitute of Education.The projectofficer is Dr. Candace Miyamura of the Teaching and Learning/Reading and Language Group. This report does not necessarilyreflect the views of khe funding agency.3
ABOUT THE DOCUMENT DESIGN PROJECTInk September 1978, the American Institutes for Research(AIR) began the Document Design Project to foster clear andsimple writing and design of public documents. The purposeof the Document Design Project (DDP) was to help make forms,regulation's, brochures, and other written materials easierfor people to read, to understand, and to use. CarnegieMellon University and Siegel & Gale, Inc. worked with AIRon this project. The project was funded by the Teaching andLearning/Reading and Language group.at the National Instituteof Education,,The Project's goal was to increase the knowledge andskills of people who produce public documents. To accomplishthis goal, the Document Design Project had three tasks:Task 1: To conduct theoretical and appliedresearch studies on language comprehension,on the ways in which skilled and unskilledwriters work, on problems associated withdifferent docuMent features;Task 2: To bring research into practice byTTUTETWg with government agencies as theyproduce materials for public use;Task 3: To bring research-and practice intoeducation by developing courses on writingand design for graduate ,iitudents and undergraduates.If you have questions or comments on this report or onother work of the Document Design Project, contact Dr. JaniceC. Redish, Director, The Document Design Center, AIR, 202/342-5071.
TABLE OF CONTENTSOverview11.The History and Rationale of the Pro eCt52.The Research Component93The Technical Assistance and Training Component4.The New Undergraduate Curriculum to Improve theTeaching of Writing57New Graduate Programs in Rhetoric and Writing atCarnegie-Mellon University67Disseminating Information about the DocumentDesign Project's Work93Continuing the Work of the Document Design Project95APPENDIX A: Technical Reports, Publications, Articlesabout the Document Design Project, Presentations byDocument Design Project StaffA-15.6.7.List of Figures ,Figure 1:' ProgramFigure 2: ProgramFigure 3: ProgramFigure 4: ProgramProgramFigure 5:tototototoProduce "Guidelines" ,Identify ProblemsSolve ProblemsEvaluate DocumentsTest Methods.N./451101314'1517
The Document Design ProjectOverviewDuring the past 39 months, a team of linguists,psychologists, English professors, instructional researchers,bilingual education specialists, writers, and graphic desi4ners,worked together to increase our understanding of the problemsthat public documents create for readers and to suggest and tryout solutions to the problems that were identified.NIE gavethis teaffrthree tasks:1.to conduct research on why so thany documentscause proble711776-ireaders and on why thepeople who write and design documents so oftendevelop material that other people can'tunderstand;to test the usefulness of what we learn fromresearch by conducting technical assistanceand training projects With writers in publicagencies;and3.to put what we learnfrom research andpractice ,into improving the teaching ofwriting by creating new uri3iF4Taduate andgraduate curricula.This report summarizes the accomplishments of the DocumentDesign Project.We review briefly:
1.the history and rationale of the pro ect2.the research component3.the technical assistance and trainingcomponent4.the new undergraduate curriculum to improvethe teaching of writing5.the new graduate curriculum to train futureresearchers, practitioners, and educators6.the dissemination of information about theDocument Design Project7.next steps:andcontinuing what 14IE began.We hope that this report gives the reader an.understandingof the scope and the major results of the Document DesignProject, but the accomplishments of the project are reallyembodied, not in this report, but in our products.In threeyears, the Document Design Project staff produced: 17 technical reportsa 171-page monograph on relevant research fromsix disciplines36 papers published or accepted for publicationin books and professional journalsmore than 150 papers, workshOps, and symposiapresented at conferences and professionalmeetings35 technical assistance projects covering awide range of documents and agencies2
\two sets of training Materie& (one forwriters; one for people who supervise writers)andthree booksWriting in the Professions (a course guideand instructional materials for an advancedcomposition course)Guidelines for Document Designers (ateaching and reference book for practicingwriters)andComposing and Reading: A Multi.-DisciplinaryView of Theory and Research (a book ofreadings for a first-year graduate schoolcourse).eN.3'
1.The History and Rationale of the ProjectThe Document Design Project was funded by the teaching andLearninggading and Language Group at the National Institute ofEducation.'The project was created by NIE through arequest-for-proposal and was carried out by a conlortium of threegroups.The American Institutes for Research (AIR) was the primecontractor,,taking the .lead on the tasks of research, technicalassistance and training, and the development of a newundergraduate curriculum.Carnegie-Mellon University, throughttie Det5artments of Psychology and English, conducted research andcreated a new interdisciplinary graduate progtam. Siegel-E, Gale,a private design firm in New York, contributed their practicalexperience and knowledge of clear writing and design to all thecomponents of the project.NIE developed the Document Design Project because theyrecognized that the problem of functiona10.iteracy in Americatoday is as much a problem of incomprehensible documents as it isof people who have poor reading skills.The skills of readersand the demands made on them by the documents they have to readdon't match.The mismatch can be 'attacked from both sides.In additionto-teaching functional reading skills to the people who must usethe documents, we should also make the documents'easier to read5
and understand.In order to do the latter, we must teachfunctional Writing skills to'the people who create the documents.We still need to develop better teaching Methcids in reading,particularly in teaching students how to read.and deal with thedocuments that are part of adult life in America.When weexamine typical documents, however, we realize that even expertreaders have trouble understanding many of them.Consider, forexample, job application forms like the Federal Government'sSF-171, forms for educational loans or scholarships, informationand applications for benefits progr4ms, consumer creditdocuments, rental agreements, or medical consent forms, to namejust a few.Three-and-a-half years ago,"When NIE developed the DocumentDesign Project, most of thesedocuments were poorly-written,poorly-organized, and poorly-designed.They presented materialand asked questions in ways/ that Contradict rather than takeadyantage of the reader's expectations and cognitive processes.NIE wanted tO see what could be done to improve public documentsby improving the knowledge and skills of the document designers.The Document Design Project was, thus, part of NIE's'growinginterest in research on effective writing skills, which they hadcome to realize was a necessary counterpart to the development ofeffective reading skills that they had been sUpporting for many.years.Io6
NIE was also concerned with the inequities in the problemscreated by incomprehensible public documents.Although we allsuffer because of the time and effort that we must devote todifficult documents, and the paychologiCal distress they cause,the poor and least educated suffer most.The hurt is greatestfor people who are dependent on the programs that generate some9,f, the most difficult public documents.Documents are, inffect, gatekeepers, controlling access to many jobs, to moneyfor education, and to many kinds of benefits.Those who canunderstand the rules and complete the forms get in; those whocannot are kept out.The Document Design Project focused primarily on documentsthat are of critical importance to large numbers of people whocould be badly hurt if they did not understand the documents.Weconducted studies of how people in an Hispanic community copewith difficult documents, of how people with different levels ofeduCation, experiende, and fluency in English approach the taskof filling out a complex job application form, of how the owner.of a section 8A small business manages with the regulations thatgovern the program on ininority-owned businesses.We also helped .writers create moddls of well-organized, well-written, andwell-designed documents, such as college eid forms, immigrationforms, income tax instructions (in English and Spanish), medicalconsent forms for adults'and children,7ls:aflets about11
prescription drugs,sand material on fireisafety In group homesfor the mentally and physically disabled.In setting up the Document Design Project, NIg had, theforesight to include, within one team, all of the people andtasks needed to move from research to practice to improvedinstruction.This went far beyond the usual arrangements.In arecent issue of Visible Language, British researcher Patrict*Wright describes the communication gap that usually.existsbetween academic researchersand the writers and designers whocreate documents.*She discusses the value of a "bilingual".intermediary who can sPeak the languages of both researchers andpractitioners to bring together and interpret research findingsfor document designers.Because of the broad scope of theproject, this was a role that the Difumint Design,Projedt wasable to play.She also discusses the importance of greater'interaction between reseaechers and practitioners to make theresearch respond to practical issues and to make sure that 'thepractical appli,cations of the research are, in fact, evaluatedand added to the research literature.This type of interlictionhas also been part of the Document Design Project.NIE, in the'Document Design Project, also went a step.beyond Wright an0brought ,the research and practide into new curricula lor future('researchers and future document designers.*p Wright, Strategy and tactics in the design of forms.Visible Language, 1989, 14 (2), 151-193.812
2.The:Research COMponentOverVieW.A major goal of 'the,Dotument Design Projebt has been todeSignConduct,,evaluate, and integrate basic and appliedresearch in order to'iMprove public documents.of'the prOectOver the,Coursethe rese-arch group has produced 17 technicalreports and 36 Published articles in meeting this goal.rePorts ancrpublished articleSTheieWell as papers prespnted, atmeetings and other unpublished.reports) represent the principal-results of the.research component."go,to-the source" to get a comtleWe encourage thd reader topicture of all of our.0research activities and findin§s.'l'ist.technical reports,published, papers, and presentations inidix A.)In summarizing these activities for the purpose of thepresent reportwe will begin with the conceptual framework thatdrove the research effort.Then we will trace the evolution ofeach of the aspects of the research program'from this overallframework and present brief summaries of the studies weconducted.130
A program 'to produce reliable andThe conceptual framework:Valid guidelinesOne of the principal objectives of the research component ofthe Document Design Project was to add to tfie knowledge that:isneceseary to produce a set of reliab e and,valid guidelines thatdocument designers can use.In Figu e 1 below, we picture aplausible sequence of activities foachieving this sFind SolutionsEvaluate Evaluat effectsof writer'simplementationof guidelinesSolutionto problemDiagnoseproblemsunaList ofproblamsguidelinesFIGUAE 1. Program to Produce "Guidelines"10,
The input to this "program" is a specific problem--from thepoint of view of the user, the producer, or a third-party'analyst.The first two processes,that occur are to diagnose theproblem--that is, to find the particular sources ofdifficulty--and to find a solution.Solutions can come fromexisting theories, current practices, experimental results,And/or existing guidelines.If we find a. solution, we can proceed to the next set of1processes, which convert the solution into a practical guidelinethat typical writers Can use.In these processes, we firstcreate a tentative guideline and then evaluate whether writerscan use it.If they cannot, we must revise the guideline andre-evaluate it until it can be used.-When we have a usable guideline, we can go on to the nextprocess--evaluating whether the guideline, as the writer uses.it,resolves (or at least reduces) the original problem.If it doesnot, we must return to the step of creating the, guideline, reviseit and repeat each of the testing processes.The overall program has three tangible products.The firstis a specific solution to each problem that set the program inmotion; the second is a compilation of the problems that we haveidentified and diagnosed; the third is a compilation ofguidelines that have been empirically validated by actual15
writers.If repeated several times, this program will alsoproduce several secondary producti, such as new experimentalresults and new-methods for diagnosing problems, for evaluatingwriters, and for evaluating documents.For this overall program to work, we must make severalassumptions.For example, we must assume that we know how toidentify and diagnose problems, that We have (or know how togenerate) solutions, that we know, how to create guidelines, andthat we knoW hoW to evaluate both writers and documents.-fact, in the course of the Document Design Projectassumptions led to its oWn research program.Ineactl of theseLet us briefly-describe the five research programs that spun out of our generalprogram to produce,reliable and valid guidelines that writers,canUse.,The program to identify problemsThe first major activity in the'development of guidelines- sto identify and isolate problems.,several processes must occur:As we show, in Figure 2,'In order to understand theproblem, we must analyze the users, the producers, and thedocuments; we must review the experimental, theoretical, andapplied literature; and we must develop new analytical methodswhere necessary.1216
AnalyticalmethOdsReviewuperlmental.theoretkal.Ana lyZeAnalyzedocumentsWW1Analyzeproducwsapplied literatureProblemsFIGURE 2. Program to Identify ProblemsThe principal outputs of this program,are, first,.a "database" of problems, and second, A set of analytical methOds foridentifying and isolating problems.The "data base" could beorganized by types of users, tYpes of documents, types ofproducers, or the interactions among these types.The analyticalmethods could alsoibe organized around specific oblectives--user:analysis, producer analysis, and so forth.The program to solve problemsFollowing the process of identifying problems, the nextactivity in the guidelines program is to find solutions tospecific problems.Figure 3 represents a "blow-up" of thiscomponent.1317
/EPecificproblemexperimental,end utionSolutionto problemFIGURE 3. Program to Solve ProblemsThe primary input is a particular problem, identified andisolated by the previous progrIa. We then analyze this specificproblem with respect to existing theoretical, experimental, andapplied literature.Based on this analysis, we propose atentative solution.We then carry out a test of this solution.If it does not "work," we must propose new or different solutionsand repeat thd process until a solution is found.This program is designed to solve a particular problem;thus, the principal product is problem-specific.Again, howeverrepeated applications of this program will result in importantsecondkry products, such as new methods for analyzing and testingsolutions and additions to the"tJeoretical, experimental, andapplied literature.1418
The program to evaluate guidelinesThe remainder of the guidelines program consists of twobasic activities: -creating and evaluating guidelines.Figure 4represents, in general form, the program by which a guideline ordocument could be stDevelop testFIGURE 4. Program to Evaluate DocumentAs before, we first revise the docuMent (using- the.guideline);then we develop and conduct a-test.If the test provesunsuccessful, iteratiVe revisions will be necessary Until a"good" document or guideline is produced.In the specific instance of evaluating a guideline, thisprogram implies or assumes that we know .enough about writers togenerate a non-arbitrary guideline.In fact, a distinct programof research has developed during the course of the DocumentDesign Project which looks at what writers and readers do.1915!:-
The prograM to study what writers and readers doA growing number of researchers in cognitive vsyChology ancipsycholinguistics are focusing their attention on Writer andreader processes.Carnegie-Mellon University is at the forefrontof these theoretical and applied advances.As part of theDocument Design Project, these reteardhers have continued toanalyze, develop, and test models of writer and reader processes.Particularly relevant to the guidelines.program is work that theyhave done concerning writers' implementation of guidelines.CMUhas produced several reports addressing this general issue.The program.to test methodsIn all of these programs, we must* develop new methods oradopt existing methods in order to generate specific information.For eXample, the program to identify problems requirei a methodor set of methods for .analyzing documents, users, and producers.In.mbst sitUations, finding an appropriate method is an importantproblem in its own right.Figure 5 represents the generalsequence of activities necessary to address the problem ofselecting and testing methods.2o16
FIGURE IL Program to Tort*hothInput to this program is the need for information.We firstevaluate existing methods and techniques to see if they cangathethe needed information.The program allows for.developingnew methods if existing ones do not "work" and for revising oradapting old ones.The two principal products of,this program.are, first, the specific requIred information; and second, acompilation and expansion of valuable methods and techniques.In the oourse of the Documeht Design Project, we have workedon each of these "programs" as interesting research questionshave arisen.As might be expected, many of the studies do not"fit" neatly or uniquely in a particular program; some addressissues in several at once.Nevertheless, the conceptual1721
framework and the programs within it serve well as a heuristicdevice forrganizing groups of studies.We will u4e them as astructure for presenting summaries of individual studiesconducted by the Document Design Project.Summaries of individual studies)The programo develop guidelines/As mentioned above,'thiS program repreients the generalframework from which we derive the other programs;'The twoprimary pvoducts associated exclusively with this program arealso general.The first is a major review of the literaturelandthe second is a compilation of document design gOdelines.A major endeavor of the Document Design Project was tosynthesize the literature in research disciplines that contributeto our knowled9e of validated principles of document-design.completed a narrative literature review in April, 1980.WeDocument,,Design: A review oE the relevant research (D. Felker, ed.) isessentially a data base developed from existing literature.includes chapters on:psycholinguistics,cognitive psychology,instructional research,readability,human factors, andtypography.1822It
In addition, there is a chapter in the form of a case studythat illustrates the development of an actual document designexperiment.The case study shows how research from differentdisciplines can be pulled together in examining a typicaldocument design issue.Within each discipline', we concentrated on studies that havedirect relevance to designing public documents.We primarilyconsidered research done with adult readers or learners (highschool and above).We focused on research that involvedmeaningful prose of sentence length or greater.We stressedresearch that required memory, recall, and comprehension tasksbecaUse public documents require similar'information procesingadtivities.-Guidelines for Document Designers (D. Felker-, et al.), thesecond major product of this program, is a job aid for people whowrite as part of their jobs, but who are not professionalwriters.It presenti twenty-five document design-principles thatcan make public documents easier to read and use.is piesented in the form of a guideline.Each principleThe guideline explainsthe principle, gives examples and common-sense adliice aboutapplying it, relates it to other principles, and summarizes someof the research on it.1923
The guidelines are divided into four sections:organization, writing, typography, and graphics.The,organizational guidelines discuss how to order paragraphs andsentences, how to show the document's organization to readers,,and how to belp,readers find information in the document.Thewriting gu'idelines are about sentences: how to make theconnections among information and ideas in them clear to readers,and how to avoid words and phrases that most readers have teoubleunderstanding.The typographical guidelines explain some basicdesign principles that can make documents visually appealing,physically easier to read, and that help readers' understandingby physically illuminating the contents.The graphic guidelinespresent some alternatives to prose (illustrations, tables,charts, graphs) that help readers to understand quantitative andtechnical information.Problem identificatioh studiesOne of the primary goals of our research effort has been toidentify and isolate problems.We have approached this task fromSeveral directions, including theoretichl analyse8 of languageuse, and laboratory and real-world expeiiments of documentr-.\writers and users.2024
Theoretical analyses.In Linguistic theory ana the study oflegal and bureaucratic language (Technical Report 415), VedaCharrow examines aspects of linguistic theory in,light of thereal-world phenomena of legal and bureaucratic'language.On thebasis of experimental and practical studies by researchers in theDocument Design Project and elsewhere, ChariOw presentsdescriptions of legal and bureaucratic language.She discusseshow 6,4E,aspects of linguistic theory--historicaltheories of grammar', sociolinguisqc theories, and theories oflanguage competence and performance (meta-theory)--havecontributed to understanding language.use-in the real world oflaw and government.Charrow's review of these major areas of linguistic thqoryreveals, however, that there are flaws and gaps in linguistictheory, which therefore cannot account for certain aspects ofShe also points out areas wherelegal and bureaucratic language.the study of real-world language can be,used to find these gapsand flaws, and a few areas where an understanding Of legal andbureaucratic language is potentially useful as a metric for,choosing 'among competing linguistic theories.2125PA).
In Strategies for understanding forms.and Other publicdocuments (Technical.Reporf 13), Melissa fiolland and JaniceRedish examine forms as discourse--in particular, how fo.rms aresimilar to or different.fromother types 'of text.The authorspresent a model for "functional readine-reading to learn or toactthat specifies the factors that are critical tounderstanding functional reading.The model is diVided itothree main factors--user characteristics, user behavior, and-4.document characteristics.The area 6hat'they focus, on is the'mildlebox of the model: the processes and strategies that.co stitute the usee's behavior With the dobumeni.Ho11and and Redish then report on the preliminary results ofDocument Design Project protocol study,which looked at thestrategies of forms users-,-a study of expert and nov'ice formsusers filling out the SF-171.From this study, they have beenable to postulate three levels of strategies that people use tosuccessfully fill out a 'form:decoding strategies, to figure out word meanings and todisambiguate sentences;form-using strategies, to relate items across the formor to draw on personal knowledge to clarify the meaningof an item; andglobal strategies, to put the document in a societal andinstitutional context.22
The authors' interest is ih strategies that experts use thatnovices do not.They suggeit that the experts' successfulstrategies can be taught to less successful forms users.Theyalso suggest that 'the results of this study can be used togenerate principles Tor designing better forms.A third primarily theoretical paper, is The lanpage of the ,bureaucracy (Technic4l Report#16),'in which Janice Redish,presents a ltnguistic .analysiof the salient features thatmakeTh.is paper is alsobureaucratic prose-so difficult for readers.a synthesis of the findOgs from the Document Design Project sextensive experience in working,with%government documents andgovernment writers. v As such, it presenti a sociolinguisticanalysis of the envtronment in which government writers work.8'YThese tWo theoretical analyseslinguisticandsociolinguisticlead to a better understanding' of the problemr rbureaucratese:sufficient.Deve1oping ,alid guidelines is necessary but not,We must alsO create a climate in which theAbureaucracy wants to change.Redish suggests some reasons thatmight motivate governmemt writers to apply the linguisticsolutions that the DocuMent Design Project has developed.rr,237
In Psycholinguistic alternatives to readability formulas(Technical Report *12), Holland discusses the assumptions ofreadability formulai,vand cites findings from psycholinguisticstudies of how people understand and use language that Cast doubton readability assumptions.Holland then takes results from arange of studies in psYcholinguistics-and related areas--prosecomprehension and' rneutdty, discourse analysis, sociolinguistics,and graphics in text--to, suggest features beyond sentence lengthkand Word frequency that writers and designers mdy consider toimprove their texts.The paper concludes with examples ofstudies of documents Conducted by the Document DeSign Project'that-'have considered several of these aliernatil;e features, such-using familiar text schemas, rhetorical,cues, scenarios, andThe studies,show that whencul.ture-specifid word meanings.designers use these other features, they can do more to improve adocument than they 'can !linen thpy ohly-use a readability formula.Labóratory and field studies.Technical Report No. 8,Translating the law into common language:A protocol study, byBondf-mayes, and Plower.,of CarnegieTMellon, explores the,problemof clatifying government language.The authors interviewedgovernmentwriters- and observed them as they rewrote a portiona Small Business Administration regulation so that it could be2428
understood by the general public.This document was chosen forrevision because of its extreme difficulty.The writers werealso quedtioned about the con
SF-171, forms for educational loans or scholarships, information and applications for benefits progr4ms, consumer credit documents, rental agreements, or medical consent forms, to name just a few. Three-and-a-half years ago,"When NIE developed the Document
