JRRDVolume 49, Number 1, 2012Pages 139–154Measurement of social participation outcomes in rehabilitation ofveterans with traumatic brain injuryWilliam Stiers, PhD;1* Noelle Carlozzi, PhD;2 Alison Cernich, PhD;3 Craig Velozo, PhD;4 Theresa Pape, DrPH;5Tessa Hart, PhD;6 Suzy Gulliver, PhD;7 Margaret Rogers, PhD;8 Edgar Villarreal, MS;9 Shalanda Gordon,PhD;10 Wayne Gordon, PhD;11 Gale Whiteneck, PhD121Department of Physical Medicine and Rehabilitation, Johns Hopkins University School of Medicine, Baltimore, MD;2Kessler Foundation Research Center, West Orange, NJ; and Department of Physical Medicine and Rehabilitation, NewJersey Medical School, University of Medicine & Dentistry of New Jersey, Newark, NJ; 3Neuropsychology, Department ofVeterans Affairs (VA) Maryland Health Care System, Baltimore, MD; and Departments of Neurology and Psychiatry, University of Maryland School of Medicine, Baltimore, MD; 4North Florida/South Georgia Veterans Health System, Gainesville, FL; and Department of Occupational Therapy, University of Florida, Gainesville, FL; 5Edward Hines, Jr. VAHospital, Hines, IL; and Feinberg School of Medicine, Northwestern University, Chicago, IL; 6Moss RehabilitationResearch Institute, Elkins Park, PA; and Department of Psychiatry, Einstein Medical Center, Einstein Healthcare Network, Philadelphia, PA; 7Center of Excellence for Research on Returning War Veterans, Central Texas Veterans HealthCare System, Waco VA Medical Center, Waco, TX; and College of Medicine, Texas A&M Health Science Center, Bryan,TX; 8American Speech-Language-Hearing Association, Rockville, MD; 9VA Eastern Colorado Health Care System, Denver, CO; 10Austin Outpatient Clinic, Central Texas Veterans Health Care System, Austin, TX; and St. Phillip’s College,San Antonio, TX; 11Department of Rehabilitation Medicine, Mt. Sanai School of Medicine, New York, NY; 12Craig Hospital, Englewood, COAbstract—Traumatic brain injury (TBI) is a significant concern in the veteran population, and the Department of VeteranAffairs (VA) has devoted substantial healthcare resources tothe rehabilitation of veterans with TBI. Evaluating the outcomes of these rehabilitation activities requires measuringwhether they meaningfully improve veterans’ lives, especiallywith regard to community and vocational participation, whichare strongly linked to perceived quality of life. In January2010, the VA Rehabilitation Research and Development Service convened an invitational conference focused on outcomemeasurement in rehabilitation with a specific focus on veterans’ community and vocational participation. This articlereports on the working group, addressing the issues of conceptualizing and operationalizing such participation outcomemeasures for veterans with TBI; we discuss conceptual modelsof participation, review participation subdomains and theirinstruments of measurement, and identify current researchissues and needs. Two avenues are identified for advancingparticipation measurement in veterans with TBI. First, wedescribe suggestions to facilitate the immediate implementation of participation measurement into TBI clinical practiceand rehabilitation (cont)Abbreviations: CAT computer-adaptive testing; CHART Craig Handicap Assessment and Reporting Technique; CIQ Community Integration Questionnaire; FDA Food and DrugAdministration; ICF International Classification of Functioning, Disability, and Health; M2PI Mayo-Portland Adaptability Inventory 8-Item Participation Index; MPAI MayoPortland Adaptability Inventory; NIDRR National Instituteon Disability and Rehabilitation Research; NIH NationalInstitutes of Health; PART Participation Assessed withRecombined Tools; PDA personal digital assistant; Polytrauma-QOL Polytrauma-Quality of Life; POPS Participation Objective–Participation Subjective Scale; PTSD posttraumatic stress disorder; RR&D Rehabilitation Researchand Development; TBI traumatic brain injury; TBI-QOL TBI-Quality of Life; VA Department of Veterans Affairs.*Address all correspondence to William Stiers, PhD; JohnsHopkins University School of Medicine, Department ofPhysical Medicine and Rehabilitation, 5601 Loch RavenBlvd, Suite 406, Baltimore, MD 21239; 443-444-4700; fax:443-444-4770. Email: 10.07.0131139
140JRRD, Volume 49, Number 1, 2012(cont) Abstract— research within the VA healthcare system.Second, we describe recommendations for future VA researchfunding initiatives specific to improving the measurement ofparticipation in veterans with TBI.Key words: clinical practice, measurement, outcomes, participation, rehabilitation, research, social participation, traumaticbrain injury, VA, veterans.INTRODUCTIONTraumatic brain injury (TBI) is a problem of greatconcern for the veteran population, and the Departmentof Veteran Affairs (VA) has devoted significant healthcare resources to the rehabilitation of veterans with TBI.Evaluating the outcomes of these rehabilitation activitiesrequires measuring whether they improve veterans’ livesin a meaningful way, with specific focus on improvingthe ability of veterans with TBI to join in normal workand community activities. This type of social activity isreferred to as “participation”; participation in work andcommunity activities is strongly linked to perceived quality of life. This article considers issues related to improving the measurement of social participation in veteranswith TBI to help improve the meaningful evaluation ofrehabilitation efforts.TBI is an injury that disrupts the normal function ofthe brain. TBI severity may range from mild (a briefchange in mental status or consciousness) to severe (anextended period of unconsciousness or amnesia after theinjury) . TBI is often classified as either closed or penetrating and may result from a number of different types ofevents, including sudden acceleration or deceleration ofthe head (e.g., motor vehicle accident or fall), penetratinginjury (e.g., gunshot wound), or blast injury (e.g., explosion). TBI of all types can result in the immediate ruptureof cellular and vascular membranes with impaired cerebral blood flow followed by necrotic and apoptotic celldeath and with subsequent hypoxia, hypotension, andincreased intracranial pressure . Newer imaging techniques have indicated that cytoskeletal damage may contribute to diffuse axonal injuries that disrupt nerve cell andnetwork functioning . However, understanding that alltypes of TBI can and often do co-occur with multipleother injuries, including significant cardiopulmonary,burn, or musculoskeletal injuries and posttraumatic stressdisorder (PTSD), is important [2–3]. This is especially truewith TBI in a combat environment. Mild TBI and PTSDpresent diagnostic challenges because symptoms mayoverlap, and the entire constellation of injuries complicateand compound each other.Population-based epidemiological studies in the U.S.civilian population provide insights into the magnitude ofTBI in the United States. The annual incidence of U.S.civilians with TBI is estimated to range from 0.4 to0.8 percent [1–2], or 1.2 million to 2.4 million personsexperiencing TBI each year. A 2010 Centers for DiseaseControl and Prevention report identified 1.7 million U.S.civilians incurring a TBI each year, with 80 percent of thesebeing treated and released from hospital emergency departments, 16 percent hospitalized, and 3 percent fatalities .Although no population-based epidemiological studies stratify incidence and prevalence data by TBI severityin veteran and military populations, some data provideinsight into the occurrence of TBI for these groups. Estimates of the number of veterans and military servicemembers with TBI vary widely according to the datasources and research methods employed in a given analysis. According to the Department of Defense, about35,000 servicemembers have been wounded in action asof 2009 , or about 2 percent of the 1.6 million servicemembers who have served or are serving in OperationIraqi Freedom and Operation Enduring Freedom over thepast decade. However, some researchers have suggestedthat perhaps around 70,000 of these 1.6 million servicemembers (4%) have sustained a TBI , while othershave suggested much higher numbers, perhaps as manyas 320,000 (20%) [7–9]. These estimates of the magnitude of possible TBI in military personnel range fromapproximately 2 to 10 times greater than the total numberof servicemembers identified as wounded in action, andtherefore include individuals who were not officiallyidentified as wounded during their tour of duty. Injuriesnot initially recognized during tour of duty would almostexclusively be expected to be mild injuries.While the data regarding the magnitude of mild TBIin military servicemembers and veterans are very concerning, it is also important to recognize that mild TBIand what is sometimes called “postconcussive disorder”have poor diagnostic reliability and validity. Many postconcussive symptoms such as headache, sleep disturbance, and difficulty concentrating have a base rate in thegeneral nondisabled population ranging from 6 to 80 percent depending on the specific symptom [10–13] and arealso found in individuals with anxiety , depression
141STIERS et al. Social participation outcomes in veterans with TBI, PTSD , and pain [17–18]. In addition, TBI iscommonly diagnosed using neuropsychological assessments, but interpretation of such assessment results can beconfounded by the 5 to 85 percent incidence of false positive neuropsychological findings in nondisabled populations (depending on the number of tests and the cutofflevel for identifying positive findings ).Using the most common definitions of severity, a2009 Institute of Medicine study considered over 30,000published studies and found an association between moderate to severe TBI and neurocognitive deficits, as well asproblems with long-term employment and social relationships. However, no clear association was foundbetween mild TBI and objectively measured neurocognitive deficits or long-term employment, social relationships, and ability to live independently .Although it is important to consider the multiple factors and complex issues related to accurately identifyingand diagnosing TBI, it is also important to provide assistance to military servicemembers and veterans who experience functional difficulties. It is important that the VAprovide resources to improve veterans’ ability to function,regardless of the relative contribution of TBI, anxiety,depression, PTSD, sleep disorder, pain, substance abuse,or other factors confounding diagnosis. Targeted interventions to improve specific areas of difficulty, especially difficulties in community and vocational participation, maybe more important than diagnostic categorization.It is important that interventions designed to improveveterans’ social and vocational participation be studiedusing valid, reliable, and sensitive measures. One set ofparticularly relevant measures that can help inform therehabilitation process involve measuring how successfulindividuals are in participating in usual social role activities, i.e., working; engaging in leisure and recreationalactivities with others; and being part of family, neighborhood, and community interactions. Precise and comprehensive measurement of rehabilitation outcomes relatedto participation is needed to meaningfully evaluate andcompare the results of different rehabilitation interventions designed to improve the lives of veterans withknown or suspected TBI.Therefore, this article considers the issues involvedin measuring rehabilitation outcomes regarding socialparticipation in veterans with possible, probable, andconfirmed TBI. Specifically, we will— Discuss conceptual models of participation. Review participation subdomains and their instruments of measurement. Identify current research issues and needs related tomeasurement of participation. Make suggestions for the future development of participation measures.CONCEPTUALIZING PARTICIPATIONMedical advances have led to more veterans andmilitary personnel surviving acute injury and illness andthen living with chronic health conditions. This creates aneed for conceptual models to understand disablement. Inthe conceptual model of the World Health OrganizationInternational Classification of Functioning, Disability,and Health (ICF) , rehabilitation outcomes can beconsidered in a three categories. “Body Functions andStructures” refers to the physical level of body structuresand their associated functions. “Activities” occurs at thetask level and involves such things as feeding, dressing,shopping, and operating a motor vehicle. “Participation”occurs at the societal level and is an interaction betweenthe person and the environment in social roles, involvingsuch things as being a worker, student, friend, spouse,parent, or citizen. While rehabilitation interventions toimprove body functions and structures and activities arecertainly important, it is participation that is moststrongly linked to perceived quality of life. However, participation in social role activities is challenging to conceptualize and measure.DEFINING PARTICIPATIONThe ICF was the first model of disability to use theterm participation to refer to “involvement in life situations,” but the concept of participation as involvement atthe societal level has appeared in all prior models of disability. For example, earlier models used different termsthan participation, including “handicap” to indicate a disadvantage in society  and “disability” to indicateproblems with social role fulfillment [22–24]. Althoughparticipation is a useful term to describe involvement infamily, community, and work roles, the ICF does notoperationally define participation in a way that distinguishes it from other concepts such as activity and qualityof life.
142JRRD, Volume 49, Number 1, 2012It is important to conceptually distinguish participation from activity. As Whiteneck and Dijkers state,“activities are the physical and cognitive tasks performedby individuals, while participation is social role performance as a member of society” [25; p. 24]. The ICF hasbeen criticized for not adequately differentiating participation from activity (at the person level) either definitionallyor in its single taxonomy of activity and participation category codes [26–27].Measures of participation are also difficult to distinguish from measures of global life satisfaction such asquality of life . In contrast to the multidimensionalhealth-related quality of life construct , unidimensional quality of life measures evaluate general wellbeing [28–29]. It is unclear whether subjective measuresof one’s satisfaction with participation differ from subjective judgments of well-being, which have been conceptualized as encompassing eudemonic concepts suchas purpose in life, personal growth, positive relationswith others, environmental mastery, self-acceptance, andautonomy, as well as hedonic concepts such as happinessand life satisfaction .The ICF also does not clearly conceptualize how theinteraction between the person, the task, and the environment affects participation. It is important to recognizethat disability is a person-task-environment interactionarising from the individual’s condition, the task in whichthey are engaged, and the environment in which theyengage with the task. For example, wheelchair users areless disabled in communication tasks than in mobilitytasks, less disabled in physical environments with pavedsidewalks and ramps than in physical environments without such structures, and less disabled in social environments with acceptance of variations in body functionsand structures than in social environments without suchacceptance. Therefore, participation is affected byaspects of an individual’s physical condition; motor andcognitive functioning; availability and use of assistivedevices; and the physical, social, and public-resourceenvironments in which they live.In addition, we may need to expand definitions ofparticipation to include some unexplored areas that couldbe important from a conceptual standpoint. Attention hasnot been given to the concept of negative participation,wherein individuals engage in antisocial or illegal interpersonal activities. Cyberparticipation, wherein individuals interact without direct or even real-time contact withothers, is beginning to be incorporated into concepts ofparticipation. Hyperparticipation, wherein individualsengage in greater amounts of interpersonal activities thanthey desire (when family, school, community, and workdemands exceed their preferences), has also not been discussed in models of participation.In summary, a clear and commonly accepted definition of participation, distinct from activity and quality oflife, is needed to develop operational definitions andmeasurement instruments . The field of rehabilitationoutcome assessment could also be enhanced by moreexplicit incorporation of person-task-environment interactions into these assessments. Inclusion of conceptualvariations on participation, such as negative participationor overparticipation, may also be helpful.MEASURING PARTICIPATIONIn addition to conceptual clarification, improving thepractical enumeration of types of participation (distinctfrom activity and quality of life) is also needed toadvance the field of rehabilitation outcome assessment. As with the overall definition of participation,agreement on the key subdomains of participation andtheir measurement would be helpful. Whiteneck andDijkers discuss that participation can be measured usingobjective indices such as marital and employment status,counts of social activities engaged in, and inventories ofcommon social activities . These objectiveapproaches have been criticized as neglecting importantindividual variation, especially with respect to how people with disabilities may prefer to participate and how asa group, they may differ in important ways from the nondisabled population upon which such measures are typically based. It is not clear that participation is bestmeasured by counting the number of social engagements.Participation may include issues related to amount, butpeople pick and choose their participation according topreferences and perceived importance and difficulty, andthus the subdomains are not hierarchical in nature (onetype of activity does not necessarily precede or followanother) . Optimal participation, rather than maximalparticipation, may be most important, involving the perceived characteristics of those interactions (preference,importance, and difficulty) .Measuring participation that is specific to individualswith TBI is also complex. First, individuals may have mild,moderate, or severe injuries, and their specific injuries may
143STIERS et al. Social participation outcomes in veterans with TBIresult in different types of disabilities. Second, we mustconsider the stage and trajectory of recovery during whichassessments are made. Third, although this population isheterogeneous, some types of impairment frequently occurthat may directly affect a person’s ability to reliably andaccurately communicate their thoughts and behaviors.These include deficits in motor function, thinking andmemory, communication and social pragmatics, affectiveself-regulation, and accurate awareness of self and others,which may compromise the validity of self-report measures with this population [31–32]. Fourth, some participation measures are completed by knowledgeable caregivers,and the degree to which these reports may or may not agreewith the person’s own report varies according to the typeand degree of injury and the type of participation measured[33–34]. Despite these difficulties, valid, reliable, and sensitive outcome measures are needed that can help informthe rehabilitation process by examining how successfulindividuals with TBI are in returning to their families, communities, and work and social involvements. This articlediscusses existing and developing instruments for measuring participation in persons with TBI in relation to theseissues.ESTABLISHED ASSESSMENT TOOLSAlthough a number of different assessment toolshave been developed to assess participation in individuals with a variety of disabilities, the majority of these aregeneric and not specific to persons with TBI. Fortunately,recent measurement developments specifically target theassessment of TBI-related participation issues. This article does not attempt to provide a systematic or exhaustivereview of all assessment tools for measuring participation; rather, it discusses assessment tools that have beenspecifically developed to evaluate participation for persons with TBI.One of the first assessment tools developed to examine participation was the Community Integration Questionnaire (CIQ), which includes an objective measure ofparticipation in TBI samples (type and frequency ofactivities). The complementary development of the Participation Objective–Participation Subjective Scale(POPS) and the Mayo-Portland Adaptability Inventory(MPAI) provides subjective assessment of participation inTBI (amount of assistance needed). This developmentwork was further extended by the Participation Assessedwith Recombined Tools (PART), which takes items fromeach of these measures (CIQ, POPS, MPAI) as well as theCraig Handicap Assessment and Reporting Technique(CHART), to develop a more sensitive tool appropriatefor use in TBI.Community Integration QuestionnaireThe CIQ is a 15-item questionnaire developed toassess three domains of community participation: homeintegration (4 items involving household chores and childcare), social integration (8 items involving leisure andsocial networks), and productivity (3 items involvingwork and school or volunteer participation) . Theitems in the home integration and social integration subscales are rated for levels of independence in participation, the frequency of participation, and if socialinteraction is involved, with whom the individual participates in the activity. For the productivity subscale, thelevel of participation is rated on the hourly amount ofparticipation (full- vs part-time participation in school orwork) and the frequency of participation. The questionnaire yields a total score and three subscale scores, withhigher scores representing higher levels of communityintegration. The CIQ can be conducted in person andover the telephone and is designed for use with the individual and/or a significant other.The CIQ is the most widely used community outcomemeasure in TBI rehabilitation . Its strengths includebrevity [36–37], orthogonal subscales , and high testretest reliability . The CIQ was able to discriminateindividuals with TBI from nondisabled individuals andalso showed differentiation of scores in a TBI samplegrouped according to living situation (e.g., independentliving, living with supports, institutional setting) .However, a number of critiques of the CIQ exist. First,the development of the instrument was based on principalcomponents analysis of data from a sample of 49 individuals with TBI, which may have been too small to provideadequate statistical power. Second, although subsequentreplication of the analysis verified the factor structure, thisreplication did not include each of the original 47 itemsand indicated lack of fit for two items (shopping and childcare) [36,38]. Third, interrater reliability was initiallyassessed to be acceptable , but later studies using themore accurate intraclass correlation coefficient producedless robust results, especially for the social integration andproductivity scales [38–39]. Fourth, the CIQ does not correlate highly with other measures of participation and/or
144JRRD, Volume 49, Number 1, 2012disability and thus may represent measurement of aunique domain of participation or may not assess somerelevant constructs [40–41]. Fifth, an individual’s scorecan be adversely affected by their level of premorbid participation (e.g., if they never did household chores), needfor less supervision, and/or availability of the activity orneed for participation in the activity (e.g., child care)[36,38,42]. Sixth, demographic factors and cultural valuescan influence the findings, but no controls are included forthese effects in scoring or normative development [43–47]. Seventh, although the sensitivity of this assessmenttool’s ability to distinguish individuals with and withoutdisability is well established, it is less able to detectchange as the result of an intervention . Finally, thepriorities of the individual are not included in the CIQ, andcritiques have been directed at the lack of consideration ofthe relevance of specific areas of participation to certainTBI stakeholders (e.g., patients, families, healthcare professionals, employers) .Participation Objective–Participation Subjective ScaleThe POPS was developed to prioritize the preferences and goals of the individual with TBI and to mapmore closely to the ICF . The POPS is a 26-item selfreport assessment tool evolved from a research measurewith data gathered from 454 individuals with TBI in thecommunity and 126 nondisabled individuals. It includesboth an objective (participation objective: frequency orduration of engagement) and a subjective measure (participation subjective: importance of activity and satisfaction with level of engagement) of participation. Areas ofparticipation are organized into five subscales that wereconceptually derived and conform to ICF categorization:domestic life; interpersonal interactions and relationships; major life areas; transportation; and community,recreational, and civic life. The participation objectiveportion of the questionnaire quantifies participation asproportion of responsibility, number of hours, or frequency of participation, depending on the area of lifemeasured. The participation subjective portion quantifiesthe importance of the activity to the individual and thedesire for change in the level of participation. The twosubscales are summed into weighted averages, eitherusing the mean ratings of the standardization sample(participation objective) or using the importance rating ofthe individual (participation subjective).The test-retest reliability coefficients for the totalscores were adequate (intraclass correlation coefficient of0.75 and 0.80, respectively, for the participation objectiveand participation subjective total scores); however, therange for the subscale scores was more variable, withweaker reliability for the transportation and community,recreational, and civic life subscales (0.28 and 0.37,respectively) . The subscales appear to be orthogonalon the participation objective dimensions, indicating thatthe types of activity are different, but share correlationson the participation subjective dimension, indicating acoherent measurement of value.The authors compared the scores of individuals withTBI with those without disability and found that the ratings of the importance of various activities did not significantly differ between the groups . The authorscompared the scores of individuals with mild TBI withthose with moderate to severe TBI and found that level ofparticipation did not significantly differ, but that satisfaction with participation was lower for the individuals withmild TBI .To date, the literature has reported no validation ofthe POPS using other instruments of participation, andthis remains a major limitation. In addition, although theauthors note that environmental factors could affect theratings of participation and/or satisfaction, this potentialconfound is not included in the measure as a factor thataffects level of participation. In sum, while the POPSholds some promise, further evaluation is needed todetermine whether it is valid and reliable in outcomeassessment of participation in rehabilitation trials.Participation Assessed with Recombined ToolsThe PART represents an attempt to select the bestitems from existing instruments of participation [49–50]:the second edition of the CIQ , the POPS , and theCHART . Items from these instruments were mergedinto a single instrument and administered to a sample of400 individuals with TBI at 8 of the 16 Federal TBIModel Systems programs as they crossed their 1st, 2nd,5th, 10th, and 15th year anniversaries of injury. Usinga combination of factor analysis, Rasch analysis, andcontent analysis, the authors identified 24 items thatformed a new scale (PART) of participation (being productive, socially integrated, and engaged in communitylife) demonstrating good psychometric properties .The authors selected all 24 PART items to measure participation objectively (hours spent in productive activities,
145STIERS et al. Social participation outcomes in veterans with TBIfrequency of involvement in social and communityaspects of participation, etc.), but they also identified theneed for subjective measures of participation, and a subjective tool is currently under development. The PART islimited by the limitations of the measures it incorporatesbecause it did not include the validation of new items.The PART correlates strongly with other measures ofparticipation and with measures of impairment, physicaland cognitive functional performance, and satisfaction.Specifically, the PART demonstrates significant correlations with the MPAI 8-Item Participation Index (M2PI),the Cognitive Functional Independence Measure, theMotor Functional Independence Measure, the Supervision Rating Scale, the Glasgow Outcome ScaleExtended, and the Dementia Rating Scale, indicating thatgreater functional independence is associated withgreater participation . In addition, the PART is significantly associated with the Satisfaction with Life Scale. Because of these strengths
139 JRRD Volume 49, Number 1, 2012 Pages 139-154 Measurement of social participation outcomes in rehabilitation of veterans with traumatic brain injury William Stiers, PhD;1* Noelle Carlozzi, PhD;2 Alison Cernich, PhD;3 Craig Velozo, PhD;4 Theresa Pape, DrPH;5 Tessa Hart, PhD;6 Suzy Gulliver, PhD;7 Margaret Rogers, PhD;8 Edgar Villarreal, MS;9 Shalanda Gordon,