Association of Magnet Status With Hospitalization Outcomes for Ischemic Stroke Patients
Background It is not clear whether Magnet recognition by the American Nurses Credentialing Center (nursing excellence program) is associated with improved patient outcomes. We investigated whether hospitalization in a Magnet hospital is associated with improved outcomes for patients with ischemic stroke.
Methods and Results We performed a cohort study of patients with ischemic stroke from 2009 to 2013, who were registered in the New York Statewide Planning and Research Cooperative System database. Propensity‐score‐adjusted multivariable regression models were used to adjust for known confounders, with mixed effects methods to control for clustering at the facility level. An instrumental variable analysis was used to control for unmeasured confounding and simulate the effect of a randomized trial. During the study period, 176 557 patients were admitted for ischemic stroke, and met the inclusion criteria. Of these, 32 092 (18.2%) were hospitalized in Magnet hospitals, and 144 465 (81.8%) in non‐Magnet institutions. Instrumental variable analysis demonstrated that hospitalization in Magnet hospitals was associated with lower case‐fatality (adjusted difference, −23.9%; 95% CI, −29.0% to −18.7%), length of stay (adjusted difference, −0.4; 95% CI, −0.8 to −0.1), and rate of discharge to a facility (adjusted difference, −16.5%; 95% CI, −20.0% to −13.0%) in comparison to non‐Magnet hospitals. The same associations were present in propensity‐score‐adjusted mixed effects models.
Conclusions Using a comprehensive all‐payer cohort of patients with ischemic stroke in New York State, we identified an association of treatment in Magnet hospitals with lower case‐fatality, discharge to a facility, and length of stay. Further research into the factors contributing to the superiority of Magnet hospitals in stroke care is warranted.
Public reporting is at the core of recently enacted legislation aimed at improving quality, and empowering shared decision‐making.1, 2, 3, 4, 5, 6 The Magnet Recognition Program of the American Nurses Credentialing Center7 is one such initiative designed to identify healthcare facilities with a commitment to quality improvement, and excellent nursing care delivery. This program involves rigorous documentation and site visits to evaluate institutions across 5 core principles: transformational leadership, a structure that empowers staff, an established professional nursing practice model, support for knowledge generation and application, and robust quality improvement mechanisms (benchmarking, morbidity review, etc).7 The stated goal of the Magnet program is to “improve patient care.”7 Through recent inclusion in US News and World Report rating,8 endorsement by the Leapfrog Group,9 and media attention, these initiatives are increasingly recognized by the public.
Prior studies have investigated the association of Magnet recognition with outcomes for different patient groups. Some researchers have shown improved outcomes in Magnet hospitals for elderly Medicare medical and surgical patients.10, 11 However, others failed to demonstrate a similar benefit of Magnet status.12, 13, 14 These and other retrospective analyses failed to control for unmeasured confounding, stemming from the nonrandom patient allocation to particular hospitals. There has been no previous investigation attempting to answer this question in a comprehensive all‐payer cohort, while controlling for unmeasured confounding.
We used the New York Statewide Planning and Research Cooperative System (SPARCS)15 to study the association of being hospitalized in a Magnet hospital with case‐fatality, discharge to a facility, and length of stay (LOS) for patients with ischemic stroke.
New York Statewide Planning and Research Cooperative System
This study was approved by the Dartmouth Committee for Protection of Human Subjects. The study was based on de‐identified data and the consent process was waived. All patients who were hospitalized for acute ischemic stroke, and were registered in the SPARCS (New York State Department of Health, Albany, NY)15 database between 2009 and 2013 were included in the analysis. For these years, SPARCS contains patient‐level details for every hospital discharge, ambulatory surgery, and emergency department admission in New York State as coded from admission and billing records. More information about SPARCS is available at https://www.health.ny.gov/statistics/sparcs/.
Magnet Recognition Program
The Magnet Recognition program of the American Nurses Credentialing Center was established in 1994 by a subsidiary of the American Nurses Association.7 Magnet recognition is a voluntary program that lasts for 4 years. As of 2015, 402 facilities in the United States were recognized by the program. More information on this process can be found at http://www.nursecredentialing.org/Magnet. The program's website was used to identify hospitals in New York State that obtained Magnet recognition and the year this was achieved. Hospitals were classified as having Magnet recognition in the corresponding year of the analysis. Classifications were updated each year of the study period in case of mergers or closures.
In order to establish the cohort of patients, we used International Classification of Disease‐9‐Clinical Modification codes to identify patients in the database who were hospitalized for acute ischemic stroke (International Classification of Disease‐9‐Clinical Modification code 433.x1, 434.x1) between 2009 and 2013.
The primary outcome variable was case‐fatality during the initial hospitalization for ischemic stroke. Secondary outcomes were LOS during the initial hospitalization, and the rate of discharge to a facility. Discharge to a facility was defined as discharge to any location with services other than primarily to the patient's home.
The primary exposure variable was whether the stroke patient was admitted to a Magnet hospital for their care.
Covariates (Table S1) used for risk‐adjustment were age, sex, race (black, Hispanic, Asian, white, other), insurance (private, Medicare, Medicaid, uninsured, other), patient location during the stroke (inpatient versus outpatient setting), and stroke intervention either via administration of intravenous tissue plasminogen activator (International Classification of Disease‐9‐Clinical Modification 99.10, V45.88) or mechanical thrombectomy (International Classification of Disease‐9‐Clinical Modification 39.74). The comorbidities used for risk adjustment were diabetes mellitus, smoking, chronic lung disease, hypertension, hypercholesterolemia, peripheral vascular disease, congestive heart failure, coronary artery disease, history of stroke or transient ischemic attack, alcohol abuse, obesity, chronic renal failure, and coagulopathy. Only variables that were defined as “present on admission” were considered part of the patient's preadmission comorbidity profile.
We additionally controlled for hospital characteristics including primary stroke center or comprehensive stroke center status, hospital size, and Get with the Guidelines program participation (70.8% of Magnet hospitals, and 30.4% of non‐Magnet institutions).16
The association of Magnet recognition with our outcome measures was examined in a multivariable setting. Patients admitted to a Magnet hospital or a non‐Magnet institution were nonrandomly directed to either facility (depending on presentation, ambulance protocols, etc). In order to account for this unmeasured confounding, and to simulate the effect of randomization, we used an instrumental variable analysis, an econometric technique.17 The differential distance of the patient to the closest Magnet hospital (distance to a Magnet hospital minus distance to a non‐Magnet institution) was used as an instrument for the treatment facility. This advanced observational technique has been used before by clinical researchers, to answer comparative effectiveness questions for different interventions, including the value of primary stroke centers.18 The goal is to simulate randomization, especially when the baseline functional characteristics of the patients (including the functional status of patients with stroke) are unknown (similar to our application).19, 20, 21 The instrumental variable analysis is utilized to minimize the impact of such unmeasured confounders. This analysis uses the differences across regions to simulate the structure of a randomized trial, in an observational setting. It attempts to create balance of unmeasured covariates among treatment groups.
A good instrument is not associated with the outcome other than through the exposure variable of interest (a requirement known as the exclusion restriction criterion).22 In our case it is unlikely that the differential distance to the closest Magnet hospital would be associated with case‐fatality in any way other than the choice of treatment facility. A 2‐stage least‐squares method was used for the calculation of the coefficients. The value of the F statistic in the first stage of the 2‐stage least‐squares approach was 112, which is consistent with a strong instrument (F statistic >10), based on a practical rule.17
A probit regression was used for the categorical outcomes (case‐fatality, and discharge to a facility),23 and a linear regression for the linear outcomes (LOS). The covariates used for risk adjustment in these models were as follows: age, sex, race, insurance status, and all the comorbidities and hospital characteristics mentioned previously. Since the coefficients produced by the probit function are not interpretable, we used the marginal effects of our independent variables instead. The marginal effects are the partial derivatives of the coefficients, and reflect the change in the probability of the dependent variable, for 1 unit change in the independent variable, at the average value of all other covariates.
In order to demonstrate the robustness of our data in a sensitivity analysis, we used standard techniques to account for measured confounding, while accounting for clustering at the hospital level. For categorical outcomes, we used a probit regression model with hospital ID as a random effects variable, while controlling for all the covariates mentioned previously. In an alternative way to control for confounding, we used a propensity‐adjusted (with deciles of propensity score) probit regression model. We calculated the propensity score of admission to a Magnet hospital with a separate probit regression model, using all the covariates mentioned previously. For continuous outcomes, we performed similar analyses using linear models. Logarithmic transformation of the values of LOS yielded identical results and is therefore not reported further.
Regression diagnostics were used for all models. Number needed to treat was calculated when appropriate. All results are based on 2‐sided tests, and the level of statistical significance was set at 0.05. This study, based on 176 557 patients, has sufficient power (80%) at a 5% type I error rate to detect differences in case‐fatality, as small as 0.7%. Statistical analyses were performed using Stata version 13 (StataCorp, College Station, TX).
In the selected study period, there were 176 557 patients hospitalized for acute ischemic stroke (mean age was 71.3 years, with 53.0% females) who were registered in SPARCS. There were 32 092 (18.2%) patients hospitalized in Magnet hospitals, and 144 465 (81.8%) in non‐Magnet institutions. The characteristics of the 2 cohorts at baseline can be seen in Table 1.
Overall, 2525 (7.9%) inpatient deaths were recorded in Magnet hospitals and 12 855 (8.9%) in non‐Magnet institutions (Table 2). Hospitalization in a Magnet hospital for acute ischemic stroke was associated with lower case‐fatality in comparison to non‐Magnet institutions (difference, −6.7%; 95% CI, −8.9% to −4.5%) in unadjusted analysis. Likewise, using a probit regression with instrumental variable analysis, we identified that Magnet hospitals were associated with a 23.9% decreased case‐fatality (95% CI, −29.0% to −18.7%), in comparison to non‐Magnet institutions (Table 3). This persisted in a mixed effects probit regression model (adjusted difference, −10.5%; 95% CI, −12.8% to −8.1%) and a propensity score adjusted probit model (adjusted difference, −9.7%; 95% CI, −11.9% to −7.4%). This corresponded to 5 patients with stroke needing to be treated in a Magnet hospital to prevent 1 death.
Length of Stay
The average LOS was 8.8 days (SD 14.1) in Magnet hospitals, and 9.1 days (SD 15.9) in non‐Magnet institutions (Table 2). Magnet hospitals were associated with lower LOS than non‐Magnet institutions (difference, −0.3; 95% CI, −0.5 to −0.1) in the unadjusted analysis. Using a linear regression with instrumental variable analysis, we demonstrated (Table 3) that hospitalization in a Magnet hospital was associated with 0.4 days shorter LOS in comparison with non‐Magnet institutions (95% CI, −0.8 to −0.1). We found similar results in a mixed effects linear regression model (adjusted difference, −0.2; 95% CI, −0.3 to −0.1).
Discharge to a Facility
Overall, 14 390 (48.7%) patients were discharged to a facility from Magnet hospitals and 658 589 (52.1%) from non‐Magnet institutions (Table 2). Hospitalization in a Magnet hospital for acute ischemic stroke was associated with lower rate of discharge to a facility in comparison to non‐Magnet institutions (difference, −8.6%; 95% CI, −10.2% to −7.0%) in unadjusted analysis. Likewise, using a probit regression with instrumental variable analysis, we identified that Magnet hospitals were associated with a 16.5% lower rate of discharge to a facility (95% CI, −20.0% to −13.0%), in comparison to non‐Magnet institutions (Table 3). This persisted in a mixed effects probit regression model (adjusted difference, −10.1%; 95% CI, −11.7% to −8.4%) and a propensity score adjusted probit model (adjusted difference, −9.9%; 95% CI, −11.6% to −8.3%). This corresponded to 6 patients needing to be treated in a Magnet hospital to prevent 1 discharge to a facility.
Using a comprehensive all‐payer cohort of patients in New York State with acute ischemic stroke, we identified an association of hospitalization in a Magnet hospital with lower case‐fatality, LOS, and discharge to rehabilitation.1, 2, 3, 4, 5, 6 These results were consistent across statistical techniques used to control for measured and unmeasured confounders. Although we can only speculate about the reasons behind the more profound effect observed with the instrumental variable analysis, we hypothesize that controlling for unmeasured confounders (ie, possibly sicker patients being preferentially hospitalized in magnet hospitals) results in a more clear association of Magnet hospitals with improved outcomes. The clinical significance of the observed differences should be assessed based on the individual patient and practice characteristics of every provider. In recent years, there is increasing emphasis on public reporting, center of excellence recognition, and patient engagement in treatment decisions. Magnet recognition is heavily advertised by hospitals.7 These facilities have been found to have lower rates of nursing burnout and improved overall financial performance.7, 24, 25, 26, 27, 28 However, the association of this recognition with improved outcomes in ischemic stroke has not been studied before.
Prior investigations have demonstrated conflicting results regarding the association of Magnet recognition and patient outcomes. The first study linking Magnet status to improved outcomes was published in 1994.29 However, it was based on the original 1983 cohort of Magnet hospitals recognized by reputation, and not the formal review process currently used.29 Most recently, Friese et al10 in a longitudinal analysis of elderly surgical Medicare patients demonstrated that hospitalization in a Magnet hospital was associated with lower 30‐day mortality and rates of failure‐to‐rescue. Similar results were demonstrated in a regional analysis in Pennsylvania.11 Other groups have identified superior outcomes for Magnet facilities in terms of falls,30 mortality after trauma,31 and outcomes of very low‐birth‐weight infants.32 On the contrary, several researchers were not able to demonstrate a benefit of Magnet status.12, 13, 14 The lack of adjustment for clustering, and rigorous control for measured and unmeasured confounders (especially the fact that patients were nonrandomly selected for either treatment center), significantly limit the interpretation of the results of these investigations. A meta‐analysis by Petit Dit Dariel and Regnaux reached the same conclusions.33
Our current study purposefully addresses many of these methodologic limitations. First, we created a cohort of all patients in a major state, giving a true picture of practice in the community. Second, we used advanced observational techniques to control for confounding. Propensity score stratification was used to adjust our analyses for known confounders. The possibility of clustering, which can bias the results of multicenter national studies, was accounted for by using mixed effects methods. Most importantly, an instrumental variable analysis was used to control for unmeasured confounders (mainly the a priori selection of treatment facility), and simulate the effects of randomization. The instrumental variable analysis is expected to control for such factors and report results for patients of similar functional status. Results were consistent across techniques, supporting the validity of the observed associations.
Further research into the factors contributing to the potential superiority of Magnet hospitals in stroke care is warranted. Previous work has demonstrated that Magnet hospitals have less organizational hierarchy and increased nursing autonomy, invest in quality benchmarking and reporting, and have higher nursing satisfaction.7, 24, 25, 26, 27, 28, 34, 35, 36 Such institutional commitment to quality improvement empowers nurses and physicians to deliver evidence‐based care, establish effective communication, and identify patient problems more quickly,34, 35, 36 which is particularly critical in the largely protocol‐driven stroke care. A culture that encourages nursing participation in the decision‐making additionally supports effective interdisciplinary care.34, 35, 36 From a policy perspective, it is important to recognize effective quality‐reporting initiatives, given the growing body of such reports, and the resulting hesitancy of the public to adopt them. Some of the most prominent such efforts such as Hospital Compare, and the ProPublica Surgeon Scorecard have been criticized for their accuracy.37, 38, 39
Our study has several limitations. Residual confounding could account for some of the observed associations. However, this is minimized to the extent that we are using a good instrument for treatment facility. The F statistic in our analysis suggests a strong instrument. In addition, coding inaccuracies will undoubtedly occur and can affect our estimates. However, in several reports the coding for stroke has been shown to have a near perfect association with medical record review.40, 41 Although SPARCS includes all hospitals from the entire New York State, the generalization of this analysis to the entire US population is uncertain. SPARCS does not provide any clinical information on the treatment times, or the functional status of the patients (National Institutes of Health Stroke Scale), which can affect the choice of institution. However, the use of the instrumental variable analysis is attempting to control for unknown confounders such as these, and has been used before in patients with stroke from this database.19, 20, 21
Additionally, we were lacking posthospitalization and long‐term data, or timing of inpatient mortality on our patients. Quality metrics (ie, modified Rankin score) are also not available through SPARCS, and therefore we cannot compare the 2 hospital settings on these outcomes. The definitive comparison of the 2 hospital settings on functional outcomes can only be done in prospective registries. In this direction, the NeuroPoint Alliance has created the first module for a cerebrovascular registry, with results expected in the near future.42 Finally, causality cannot be definitively established based on observational data, despite the use of advanced techniques, such as the instrumental variable analysis.
It is not clear whether Magnet recognition by the American Nurses Credentialing Center is associated with improved patient outcomes. We investigated the association of Magnet recognition with case‐fatality, LOS, and discharge to a facility for ischemic patients with stroke. Using a comprehensive all‐payer cohort of patients with ischemic stroke in New York State, we identified that an association of treatment in Magnet hospitals was associated with lower case‐fatality, discharge to a facility, and LOS. Further research into the factors contributing to the superiority of Magnet hospitals in stroke care is warranted.
Sources of Funding
This work was supported by grants from the National Institute on Aging (PO1‐ AG19783), the National Institutes of Health Common Fund (U01‐AG046830), and the National Center for Advancing Translational Sciences (NCATS) of the NIH (Dartmouth Clinical and Translational Science Institute—UL1TR001086). The funders had no role in the design or execution of the study.
Table S1. Coding Definitions
- ↵Centers for Medicare and Medicaid Services . Physician quality reporting system. 2015.
- ↵Centers for Medicare and Medicaid Services . Qualified clinical data registry reporting. 2015.
- ↵Centers for Medicare and Medicaid Services . Quality measures. 2015.
- ↵Centers for Medicare and Medicaid Services . Hospital compare. 2015.
- ↵Centers for Medicare and Medicaid Services . Physician compare. 2015.
- ↵H.R. Medicare access and chip reauthorization act of 2015. 2015.
- ↵American Nurses Credentialing Center . ANCC magnet recognition program. 2016.
- ↵US News and World Report . Best hospitals. 2016.
- ↵The Leapfrog Group . 2016. Available at: http://www.leapfroggroup.org/. Accessed April 4, 2017.
- ↵Friese CR, Xia R, Ghaferi A, Birkmeyer JD, Banerjee M. Hospitals in ‘magnet’ program show better patient outcomes on mortality measures compared to non‐‘magnet’ hospitals. Health Aff (Millwood). 2015;34:986–992.
- ↵Health NYSDo . Statewide planning and research cooperative system (SPARCS). 2015.
- ↵American Heart/Stroke Association . Get with the guidelines‐stroke. 2017.
- ↵Bekelis K, Marth NJ, Wong K, Zhou W, Birkmeyer JD, Skinner J. Primary stroke center hospitalization for elderly patients with stroke: implications for case fatality and travel times. JAMA Intern Med. 2016;176:1361–1368.
- ↵Petit Dit Dariel O, Regnaux JP. Do Magnet®‐accredited hospitals show improvements in nurse and patient outcomes compared to non‐magnet hospitals: a systematic review. JBI Database System Rev Implement Rep. 2015;13:168–219.
- ↵Missios S, Bekelis K. How well do subjective hospital compare metrics reflect objective outcomes in spine surgery? J Neurosurg Spine. 2016;25:264–270.
- ↵Bekelis K, Missios S, Coy S, Rahmani R, MacKenzie TA, Asher T. Correlation of subjective hospital compare metrics with objective outcomes of cranial neurosurgical procedures in New York state. Neurosurgery. 2017;80:401–408.
- ↵Bekelis K, Missios S, MacKenzie TA, Johnson J. Does ranking of surgeons in a publicly available online platform correlate with objective outcomes? J Neurosurg. 2017. Available at: http://thejns.org/doi/full/10.3171/2016.8.JNS16583. Accessed April 4, 2017.
- ↵Kokotailo RA, Hill MD. Coding of stroke and stroke risk factors using international classification of diseases, revisions 9 and 10. Stroke. 2005;36:1776–1781.
- ↵Tirschwell DL, Longstreth WTJ. Validating administrative data in stroke research. Stroke. 2002;33:2465–2470.
- ↵NeuroPoint Alliance . The national neurosurgery quality and outcomes database (N²QOD). 2015.