The DEcIDE (Developing Evidence to Inform Decisions about Effectiveness) network is part ofAHRQ's Effective Health Care Program. It is a collaborative network of research centers thatsupport the rapid development of new scientific information and analytic tools. The DEcIDEnetwork assists health care providers, patients, and policymakers seeking unbiased informationabout the outcomes, clinical effectiveness, safety, and appropriateness of health care items andservices, particularly prescription medications and medical devices.This report is based on research conducted by the Brigham and Women’s Hospital DEcIDE(Developing Evidence to Inform Decisions about Effectiveness) Center under contract to theAgency for Healthcare Research and Quality (AHRQ), Rockville, MD (Contract No. 290-20050016) with funding from the National Cancer Institute. The AHRQ Task Order Officer for thisproject was William Lawrence, M.D., M.S.The findings and conclusions in this document are those of the authors, who are responsible for itscontents; the findings and conclusions do not necessarily represent the views of AHRQ or theNational Cancer Institute. Therefore, no statement in this report should be construed as an officialposition of AHRQ, the National Cancer Institute, or the U.S. Department of Health and HumanServices.None of the authors has a financial interest in any of the products discussed in this report.An article based on this report has been published: Setoguchi S, Earle CC, Glynn R, et al.Comparison of prospective and retrospective indicators of the quality of end-of-life cancer care. JClin Oncol 2008 Dec 10;26(35):5671-8.Suggested citation:Setoguchi S, Earle CC, Glynn R, Stedman M, Polinski J, Corcoran CP, Haas JS. Testing cancerquality measures for end-of-life care. Effective Health Care Research Report No. 21. (Prepared byBrigham and Women’s Hospital DEcIDE Center Under Contract No. 290-2005-0016). Rockville,MD: Agency for Healthcare Research and Quality. April 2010. Available al.cfm.

Effective Health Care Research Report Number 21ContentsIntroduction .1Methods.2Data Sources .2Definition of Cohorts .2Independent Variables .3Data Analysis .5Results .6Description of the Cohorts .6Physician Characteristics .6Hospital Characteristics .6Prospective Benchmark Measures .6Retrospective Benchmark Measures .7Multivariate Models for Retrospective and Prospective Cohorts .7Discussion .8Translation of Findings .10Acknowledgment .11References .11Tables and Figure .15Appendixes .35Author affiliations:Soko Setoguchi, M.D., Dr.P.H.aCraig C. Earle, M.D.bRobert Glynn, Ph.D., Sc.D aMargaret Stedman, M.P.H.aJennifer Polinski, M.P.H.aColleen P. Corcoran, M.S.N., M.P.H.aJennifer S. Haas, M.D., M.S.P.H.caDivision of Pharmacoepidemiology, Department of Medicine, Brigham and Women’s Hospitaland Harvard Medical School, Boston, MAbDepartment of Medical Oncology, Dana-Farber Cancer Institute, Boston, MAcDivision of General Medicine and Primary Care, Department of Medicine, Brigham andWomen’s Hospital and Harvard Medical School, Boston, MAi

Effective Health Care Research Report Number 21AbstractBackground. The quality of end-of-life care for cancer has important deficiencies. There are twoapproaches to measuring this care: retrospectively prior to death, or prospectively for patients witha poor prognosis.Objectives. To examine (1) the performance of existing “retrospective” quality indicators; (2)novel indicators for opiate analgesia and chemotherapy toxicity; and (3) whether patterns of usevary for retrospective and prospective approaches.Data. Linked Medicare claims, pharmaceutical claims, and cancer registry data from 1994 - 2003for New Jersey (NJ) and Pennsylvania (PA).Subjects. Seniors with breast, colorectal, lung, or prostate cancer who participated in statepharmaceutical benefit programs for near-poor seniors.Measures. Previously validated retrospective indicators, and new measures to reflect the use ofopiate analgesia and chemotherapy toxicity.Results. Use of chemotherapy and opiates were more common, but use of hospice was lesscommon, in the prospective vs. retrospective cohort. In multivariate models, visit with a surgeonwas positively associated with use of chemotherapy and opiates, toxicity, and negativelyassociated with hospice (both cohorts). Visit with an oncologist was positively associated withchemotherapy, opiates, and hospice. Patients cared for by oncologists in a small group practicewere more likely to receive chemotherapy (retrospective only) and less likely to receive hospice(both) than those in a large group. Compared to patients cared for in teaching hospitals, those inother hospitals were more likely to receive chemotherapy (both) and have toxicity (prospective),but less likely to receive opiates (both) and hospice (retrospective).Discussion. Several of the existing retrospective measures could be replicated in these data. Newindicators for opiate use and toxicity appear feasible and potentially important. Retrospective andprospective measures identify some similar physician and hospital patterns of end-of-life care.ii

Effective Health Care Research Report Number 21IntroductionWhile cancer causes more than a half million deaths each year in the United States (US),1little is known about the quality of end-of-life care. Initiatives to study the quality of cancer carehave focused largely on initial treatment decisions.2-5 Yet prior work on the quality of end-of-lifecare suggests several important deficiencies; many patients with advanced cancer continue to getaggressive chemotherapy, and may not receive hospice or other palliative care services.6-8Adequate pain control for patients dying from cancer has been highlighted as an area in particularneed of improvement.9 Prior work suggests that 25–70% of patients suffer from considerable painat the end-of-life.10-13 Hospice is also a fundamental part of end-of-life care for patients withcancer and has been associated with less suffering and better satisfaction than conventionalhospital care.11,14 Because hospice may improve the quality of life of patients at the end-of-life,Medicare has provided coverage for hospice services since 1983.14,15 While the use of hospice carehas been increasing over time, it remains broadly underutilized.15-20 These findings suggest thatthere is much room for improvement in the quality of end-of-life care.Quality measurement is the foundation for interventions and policies to improve thequality of care. Reliable and valid measures of the quality of end-of-life care are necessary todefine targets for improvement.21 Two approaches have been used to examine the quality of endof-life care. Several studies have identified patients who have died and then have lookedretrospectively at the care received during some time period prior to death.6,22,23 The alternativeapproach is to identify patients who have a poor prognosis and then prospectively examine thecare that they receive.16,24 Because of the complexities of accurately predicting when a patient isapproaching death,25 the later approach may result in a less representative sample.26 Conversely,retrospective studies do not include information about patients who are expected to die, but thenrecover, and may lead to biased estimates of utilization particularly when the time intervalbetween diagnosis and death is longer.27 Retrospective designs are efficient and provideinformation about care received in the period immediately prior to death, yet it is hard to useretrospective measures for quality improvement.Prior claims based analyses have largely focused on the utilization of hospital-basedservices and hospice care.6,22,23 Existing claims-based quality indicators have not includedmeasures that include outpatient prescription drugs. As these claims become more available, it isappealing to develop measures for the use of opiate analgesia at the end-of-life. The adequatetreatment of pain at the end-of-life is a cornerstone of end-of-life care,12 and opiates are a centralto these pain treatment regimens. As chemotherapy-related adverse effects (i.e., toxicity) may alsobe a more specific indicator of overly aggressive treatment near the end-of-life than the use ofchemotherapy more broadly, this is also a potentially important indicator of quality of care at theend-of-life.The purpose of this analysis is to further evaluate claims-based indicators of the quality ofcare at the end-of-life for seniors with cancer. Specifically, our goals were to: (1) Provide furtherdata about the performance of existing retrospective quality indicators in new populations;6,8(2) Develop novel indicators of the quality of care at the end-of-life using outpatient pharmacydata to create benchmarks for the use of opiate analgesia and chemotherapy toxicity; and(3) Examine whether patterns of variation in benchmark utilization by physician and hospitalcharacteristics is similar or different for retrospective and prospective measures.1

Effective Health Care Research Report Number 21MethodsData SourcesLinked Medicare claims, pharmaceutical claims and cancer registry data for the periodJanuary 1, 1994 through May 31, 2003 were used for two states, New Jersey (NJ) andPennsylvania (PA). Medicare Part A (hospitalization and nursing home stays), Part B (outpatientservices and procedures), and patient enrollment data were linked with pharmaceutical claimsfrom the NJ Pharmaceutical Assistance for the Aged and Disabled (PAAD) Program, and the PAPharmaceutical Assistance Contract for the Elderly (PACE) Program, respectively.28,29 Both thePAAD and PACE programs provide pharmaceutical benefits to near-indigent residents age 65 andolder. PACE is the largest US state prescription benefits program for the elderly. PACE has nodeductibles or maximum annual benefit and charges a modest co-payment of 6 for each genericprescription and 9 for brand name prescription. The current income ceiling for eligibility (2007)is 14,500 if single and 17,700 for a couple, resulting in a recipient population of both indigentand near-poor elderly p?a 554&Q 254019&agingNav 6658 foradditional information). The PAAD program is quite similar to PACE. There is no deductible, anda small co-payment of 5. The current (2007) income eligibility criterion for PAAD is an annualincome between 22,752 and 27,676 for a married couple aad.shtml). These generous benefits andrequirements for financial need result in essentially no out-of-pocket (i.e., out-of-system)medication use. Eligibility is determined annually for both of these programs. Electronic pharmacydispensing records from these programs are considered accurate because pharmacists fillprescriptions with little room for interpretation, and are reimbursed by insurers on the basis ofdetailed, complete, and accurate claims submitted in electronically.30,31Health care and pharmaceutical claims for this population were further linked to the NewJersey State Cancer Registry (NJSCR) and the Pennsylvania Cancer Registry (PCR).29 The NJCRis one of the population-based cancer registry programs that participate in the Surveillance,Epidemiology, and End Results (SEER) Program. The PCR is certified as “gold” (the highestquality) by the North American Association of Central Cancer Registries. Both cancer registriesare population-based and collect data on patient demographics, primary tumor site, morphology,cancer stage at diagnosis, first course of treatment, and follow-up for vital status. The Brigham andWomen’s Hospital Institutional Review Board reviewed and approved this study.Definition of CohortsRetrospective CohortThe retrospective cohort includes patients with a primary diagnosis of primary breast,colorectal, lung, or prostate cancer who died during the study period. Patients included in thisanalysis were required to have at least one Medicare claim within 14 months prior to death and tohave filled at least one prescription in each of two consecutive 7-month periods prior to death, toensure continuous eligibility (and therefore complete data ascertainment) during the study period.The date of death was considered to be the index date in all retrospective cohort analyses. Data oncause of death was only available for patients from NJ (1994-2002). These individuals wereexamined as a subgroup of the retrospective cohort as prior benchmarks for end-of-life care havefocused specifically on cancer-related deaths.6,82

Effective Health Care Research Report Number 21Prospective CohortThe goal of the prospective cohort was to identify individuals with these common cancersat higher risk of death. First, we identified all patients with a cancer registry-based primarydiagnosis of breast, colorectal, lung, or prostate cancer and had at least one Medicare claim withinthe 14 months prior to and following cancer diagnosis and at least one PAAD/PACEpharmaceutical claim in each of two consecutive 7-month periods prior to and following cancerdiagnosis. Eligibility for the prospective cohort was further restricted to those patients who had atleast 14 months of claims data following their cancer diagnosis date (or until death if deathoccurred within the 14 month period). The date of cancer diagnosis was considered the index datefor the prospective cohort.We applied a prediction algorithm to the prospective cohort to identify three subgroups ofpatients with a higher probability of mortality (those with a 40-, 60-, and 80% probability of deathwithin 14 months following cancer diagnosis). Patients were identified for these cohorts based ontheir predicted probability of death with logistic regression models using baseline characteristicsincluding age (5-year age groups), gender, race (white, non-white), income category ( 25,000;25,0000-49,999; 50,000-74,999, 75,000), primary cancer site, Charlson comorbidity score(continuous), histology type (e.g., ductal carcinoma, adenocarcinoma), cancer stage (in situ,localized, regional, distant, unknown), and presence of a metastasis to a solid organ (defined bythe presence of an ICD9 diagnosis code 197.xx – 198.xx during a hospitalization or ED visitwithin 2 weeks of diagnosis). The c-statistics for these models ranged from 0. 78 - 0.83Independent VariablesPatient CharacteristicsSociodemographic data (age, gender, race, zip code) were drawn from Medicareenrollment files. We matched each individual’s zip code to the median household income usingCensus 2000 data to act as a proxy for socioeconomic status.32 A comorbidity score was calculatedfor each patient using the algorithm defined by Charlson.33 For each of the four cancers, wedefined stage at diagnosis from cancer registry data as categorized above. In addition, Medicareclaims were used to define distant spread of the cancer if a claim with these codes was apparentwithin 2 weeks of the cancer diagnosis. Cancer type was defined by SEER histology codes. Bothcancer diagnosis date and death date were obtained from cancer registry data. Cause of death wasavailable for NJ patients from a cause of death file, extracted from NJ death certificates.Physician and Hospital CharacteristicsFor patients from Pennsylvania, Unique Provider Identification Numbers (UPINs) for eachtreating physician were linked with the 2003 American Medical Association (AMA) Masterfile.UPINS were only available in the Pennsylvania data. Using physician specialty codes, treatingphysicians were identified for each patient. For the prospective cohort, these were defined as thefirst surgeon, primary care physician, medical oncologist, and radiation oncologist following thecancer diagnosis date. For the retrospective cohort, these were defined as the last physician seenbefore death. We defined surgeons as those with one of the following primary or secondaryspecialties: colon and rectal surgery, surgical oncology, thoracic surgery, urology, generalsurgery, cardiothoracic surgery. Physicians who identified a specialty of hematology/oncology,oncology, or medical oncology were included as medical oncologists. Physicians who identifiedtheir specialty as radiation oncology were considered to be radiation oncologists. Primary care3

Effective Health Care Research Report Number 21physicians were identified by a specialty of family practices, general practice, general practicemedicine, internal medicine, internal medicine geriatrics, and family practice geriatrics. Becauseoncologists are more directly responsible for prescribing and managing chemotherapy, we furthercategorized oncologists by gender, type of practice (defined as solo or 2-physician practice, grouppractice, hospital-based, medical school-based, and other which included government, HMO, andno classification), number of years since medical school graduation, and whether they were aninternational medical graduate (IMG). Of the UPINS represented in the outpatient claims fromour patient population, 86% of the prospective cohort and 85% of the retrospective cohort matchedthe 2003 AMA Masterfile. Because our data included individuals with claims from 1994–2003,some of the physicians who were practicing in the earlier years may have retired by 2003.For each patient in both Pennsylvania and New Jersey, we identified a treating hospitalfrom Medicare Part A claims data. Treating hospital was defined as the first acute care hospitalvisited following the date of cancer diagnosis for the prospective cohort or the last hospital visitedbefore death for the retrospective cohort. Characteristics for each hospital were defined from the2003 American Hospital Association (AHA) file, including teaching status (defined asmembership in the Council of Teaching Hospitals)34 number of beds, ownership (for profit, nonprofit), and whether the hospital had a surgical cancer program, provided chemotherapy services,or provided hospice or palliative care services. Additionally, we determined if the hospital was aNational Cancer Institute (NCI)-designated cancer facility.35Cancer End-of-Life Quality Benchmark MeasuresWe defined benchmark measures of quality of cancer care at the end-of-life usingmeasures previously by Earle et al.6,8,26 These included the proportion of patients who: a) receiveda chemotherapy regimen, b) received a new chemotherapy regimen, c) had 1 emergencydepartment (ED) visit, d) had 1 hospitalization, e) had 1 admission to the intensive care unit(ICU), f) were not admitted to hospice, g) were admitted to hospice within 3 days of death, h) diedin an acute care setting. These measures were constructed to identify health care systems thatprovide overly aggressive disease modifying therapies at the end-of-life.As patients at the end-of-life may suffer from considerable pain,10-13 new benchmarkmeasures were also defined to reflect the appropriate use of opiate analgesia, including theproportion of patients who received an outpatient prescription for: i) a long-acting opiate, j) ashort-acting or a long-acting opiate, or k) both a short-acting and a long-acting opiate. Thecreation of benchmarks for opiate analgesia is particularly timely as pharmacy claims forMedicare Part D become available. Finally, we examined the proportion of patients who had anED visit or hospitalization related to chemotherapy toxicity. Chemotherapy toxicity may be anindicator that the risks of chemotherapy surpass the benefits.Use of chemotherapy was defined using the codes listed in Table 1. Of note, our data didnot include some of the codes used by Earle et al (HCPCS Q0083, Q0084, Q0085) and RevenueCenter Codes (RCCs: 0331, 0332, 0335).8 New chemotherapy regimen was defined only for theretrospective cohort using Medicare J-codes for drugs used by Earle.8 Additional J-codes forchemotherapy were identified for chemotherapy drugs that had received FDA approval since thebenchmarks were created (Appendix A). A new chemotherapy regimen was characterized as one,administered during the 30 days prior to death, which included a new drug or combination ofdrugs that had not been used previously within the study period (14 months prior to death). Wedid not examine the new chemotherapy benchmark for the prospective cohort as newchemotherapy is commonly given around the time of cancer diagnosis, and it therefore would notbe a valid benchmark for end-of-life care. Visits to the ED, admission to the hospital, ICU, and4

Effective Health Care Research Report Number 21hospice, and death in an acute care setting were all identified using Medicare claims (Part A orhospice file). Prescriptions for the opiate analgesics were based on National Drug Codes (NDC)for both generic and brand opiates (Appendix B). We defined chemotherapy toxicity basedInternational Classification of Diseases (ICD-9) codes and Diagnosis-Related Group (DRG) codesvalidated by Hassett et al.36 A toxicity code needed to occur within 90 days of a chemotherapyclaim to be considered a chemotherapy-related toxicity.For the prospective cohort, benchmark measures were calculated for the 14 monthsfollowing the date of cancer diagnosis (Figure 1a, period b) except for the benchmarks forchemotherapy toxicity and admission to hospice within 3 days of death. Chemotherapy toxicitywas calculated for events that occurred between the receipt of the first chemotherapy regimen andthe last chemotherapy regimen plus 90 days. Therefore, the denominator for this benchmarkmeasure included only those patients who had chemotherapy within 14 months after diagnosis.For the retrospective cohort, benchmark measures were calculated during the 30 days priorto death (Figure 1b, period c), except for the benchmarks for chemotherapy toxicity and admissionto hospice within 3 days of death.6-8 The chemotherapy toxicity benchmark was calculated fromreceipt of last chemotherapy regimen plus 90 days and truncated at date of death. Thus,chemotherapy toxicity could occur at any time during the 14 months period from diagnosis todeath.Data AnalysisWe calculated the distribution (frequencies or means) of patient, physician, and hospitalcharacteristics as well as each of the benchmark measures for each cohort. We also compared theprevalence of the benchmark measures in our retrospective cohort to those previously observed byEarle.8 Because the population in Earle’s study included gastric cancers, excluded prostate cancer,and had more males and younger patients, we excluded gastric cancer from Earle’s original datasetand standardized the rates of their retrospective benchmark measures for the age, cancer type, andgender of our population. To compare the performance of the benchmarks between prospectivecohorts and retrospective cohorts, we estimated the effect of physician and hospital characteristicson a subset of clinically important benchmarks: any opiate use (long or short-acting), not admittedto hospice, use of chemotherapy, and chemotherapy toxicity. For each benchmark, we developeda series of three multivariate logistic regression models: (1) a model that included only patientlevel characteristics (patient model), (2) a model that included patient characteristics andphysicians’ characteristics (physician model), and (3) a model that included patient and hospitalcharacteristics (hospital model). Each of these models adjusted for age, gender, race, income,primary site of cancer, Charlson score, cancer stage, and year of diagnosis. In addition, the patientmodels adjusted for whether a patient had a visit with surgeon, a medical oncologist, a radiationoncologist, and a visit with PCP. The physician models also adjusted for the gender of oncologist,the number of years since medical school graduation, whether the oncologist was an internationalmedical graduate, and practice size and type. The hospital models also adjusted for whether thehospital was a NCI-designated cancer center, teaching status, number of hospital beds, profitstatus, and the presence of chemotherapy, and hospice or palliative care services. Because the dataon physician characteristics are available only in the PA Medicare data and the data on cause ofdeath is available only in NJ data, we could not build a single model with all three components.Generalized estimating equations were used to adjust for the clustering of patients withinphysicians and hospitals. Odds ratios with 95% confidence interval (95% CI) for each physicianand hospital characteristic were summarized and compared among different cohorts. SAS forWindows software (release 9.2) was used for all statistical analyses (The SAS Institute, Cary, NC)5

Effective Health Care Research Report Number 21ResultsDescription of the CohortsWe identified total of 33,675 patients for the prospective cohort and 32,810 patients in theretrospective cohort (Table 2). Compared with the full retrospective cohort, individuals in the fullprospective cohort were younger, less likely to be male, and had a lower Charlson comorbidityscore. The majority of individuals lived in a ZIP code with a median income less than 50,000.While there were income eligibility criteria for these programs, we did not have access to dataabout individual income and therefore used ZIP code medians as a proxy. In both cohorts,colorectal was the most common cancer, and prostate the least common. The median number ofdays between diagnosis and death was 539 days for the full-prospective cohort, and 1,053 days forthe full retrospective cohort.Within the prospective cohort, individuals with a higher predicted probability of deathwere older, more likely to be male, have lung cancer, and have later stage disease than individualswith a lower probability of death. The median number of days from diagnosis to death decreasedfrom a median 539 days for the full cohort to 73 days for the sub-group with a predictedprobability of death of 80%.Physician CharacteristicsAmong patients in both study cohorts, we identified those who had at least one physicianvisit and summarized the characteristics of their physicians, and in particular their oncologist(Table 3). Because physicians could not be identified in the NJ claims data, this analysis wasrestricted to PA patients. A PCP saw almost all patients, in both the retrospective and prospectivecohorts, whereas only 46–62% were seen by medical oncologist. In the prospective cohort, theproportion of patients seen by a surgeon declined as the predicted probability of death increased,whereas the proportion seeing a medical or radiation oncologist increased. The characteristics ofoncologist were similar in the different cohorts.Hospital CharacteristicsAmong patients who had at least one hospitalization, approximately 40% of patients inboth cohorts received care at an NCI-designated cancer facility and 20% were hospitalized at ateaching hospital (Table 4). The vast majority of care was provided at a non-profit facility. Themajority of patients in all cohorts received care at a hospital with a surgical cancer center,chemotherapy services and hospice or palliative care services. These characteristics did not varyamong the different cohorts, with the exception that individuals in the retrospective cohort whodied from cancer were more likely to have had their last hospitalization at a hospital with asurgical cancer program or hospice/ palliative care services than individuals in the fullretrospective cohort.Prospective Benchmark MeasuresTable 5 shows proportion of patients receiving our benchmark measures for end-of-lifecare in the prospective cohorts. Approximately one-quarter of these patients receivedchemotherapy, 1/4th had 1 ED visit, and 1/3rd had 1 hospitalization during the study period.The majority of patients were never admitted to hospice. Approximately one-quarter of patientswere admitted to hospice wi

hospital care.11,14 Because hospice may improve the quality of life of patients at the end-of-life, Medicare has provided coverage for hospice services since 1983.14,15 While the use of hospice care has been increasing over time, it remains broadly underutilized.15-20 These findings suggest that