How We Reported Our Nursing Home Investigation

Nearly a year ago, we began the process of requesting and analyzing complicated federal data to examine our starting question: How many people with serious mental illness are residents in California nursing homes?

Here are the steps we took:

Creating a yearly census of nursing home residents

We used the Minimum Data Set (MDS) from the U.S. Centers for Medicare and Medicaid Services (CMS) as our source, obtained via a Freedom of Information Act request. The data we requested and received covered the years 2013 through 2022. The MDS is based on information CMS receives from nursing homes that are eligible to receive funding from Medicare or Medicaid. We used the MDS data to analyze the approximately 1,140 California skilled nursing facilities (“nursing homes”) appearing in records and that are not certified by the state as "special treatment programs" (STPs) for residents with psychiatric care needs. (There are 31 such STP facilities in California in 2023.) Comparing with the list of all nursing homes in California available from the California Department of Public Health, we have likely accounted for all but about 20 nursing homes in the state.

We identified all people from each year of data whose final assessment was not a discharge, thereby creating a census of people in a skilled nursing facility on Dec. 31 of each year. We then found the latest-dated comprehensive assessment, if available, for each of those people. A comprehensive assessment covers hundreds of data points about a resident’s health status, including diagnoses, medications and therapies. For a small percentage of patients, there was no comprehensive assessment available. Some of those patients had an assessment containing diagnosis information, and so those were retained in our census. If a patient did not have diagnosis information available, they were excluded from the analysis. The share of total nursing home residents each year who were excluded was between 3% and 5%.

Identifying patients with serious mental illness (SMI)

MDS has specific variables indicating whether or not a resident has a certain condition, including a range of mental illness diagnoses. We used the variables identifying whether a patient has a diagnosis of bipolar disorder and/or schizophrenia and/or psychotic disorder. While depression, anxiety and post-traumatic stress disorder can fall in the category of serious mental illness, we used the "narrow definition" as described in the 2019 PASRR National Report.

(PASRR, the Preadmission Screening and Resident Review, is conducted when people are being considered for admission to nursing homes.)

In some legal contexts, whether or not a resident with a serious mental illness also has dementia is relevant. We also calculated rates by facility of residents who have a serious mental illness diagnosis and do not have a dementia or Alzheimer's diagnosis. Those diagnoses are also indicated via a specific variable in MDS.

Analyzing the data

We grouped patients by facility to get a percentage at each facility who had either bipolar disorder and/or schizophrenia and/or psychotic disorder. We then merged our data with facility-level data from the CMS provider catalog to get facility names and locations.

For this analysis, we defined “long-term residents” as those whose latest-dated comprehensive assessment was an annual assessment, indicating they had been at the same facility (not including brief hospital stints) for at least a year.

To assess prevalence of therapy for residents, we used the variable in MDS that indicates the number of minutes therapy was administered to the resident by a licensed mental health professional in the last seven days.

Data regarding legal citations was downloaded from the California Department of Public Health (CDPH). Only higher-level citations — class "A", "AA", and "A Trebled" — were included in our analyses.

• Class AA citations are given when CDPH determines that a violation has been “a substantial factor in the death of a resident of a long-term health care facility.” 
• Class A citations are given when CDPH determines there was “imminent danger of death or serious harm to patients … or a substantial probability of death or serious physical harm to patients.” 
• A Trebled indicates an A-class citation that is the third such citation in a year. 

We combined this data with the SMI percent by facility to calculate and compare citation numbers in facilities with lower and higher percentages of SMI. For Hyde Park Healthcare Center, whose citations we mention in the story, we found 16 "A" level citations — two of which were later downgraded.

In our map and lookup tool, facilities with greater than 75% of residents with an SMI diagnosis were listed as such instead of reporting their exact percentages, out of concern for resident privacy. Ten facilities without a clear name match in the CMS directory were removed from the map and lookup tool, but their data was used in the summary statistics.

Comparing our data to data provided in state’s response

When we submitted our analysis results to the California Department of Health Care Services (DHCS), it responded with the following:

• DHCS: An initial analysis of data available to DHCS from Preadmission Screening and Resident Review (PASRR) records indicates that the proportion of individuals with SMI among the overall SNF [skilled nursing facility] population is likely lower than asserted by LAist. In 2022, there were 311,473 Level 1 PASRR screenings conducted in conjunction with SNF placement. Of those, 130,831 (42%) were positive and required Level 2 evaluations. True presence of SMI was then identified in 12,968 individuals, or about 10% of that group, with determinations made that mental health services (among others) were required for them.

• LAist follow-up: Were the 12,968 individuals you identified all admitted to skilled nursing facilities?

• DHCS: Yes. The data set reflects calendar year 2022, when California had not yet achieved preadmission compliance. PASRRs that year were all submitted by SNFs for individuals they admitted. 

We did not have the full PASRR records and could not directly replicate their analysis with our data. We did, however, have some PASRR data in our MDS records that allowed us to replicate their final finding, that “True presence of SMI was then identified in 12,968 individuals.”

Using the MDS data to identify how many residents admitted to nursing homes in 2022 were identified via PASRR as having an SMI, we found 12,850 residents — very close to what the state found. By our calculation, this represents 6% of admissions.

While it may be true that the percentage of SMI detected through PASRR in overall admissions assessments is lower than our finding, which shows that a quarter of all residents have an SMI, it is a significantly different analysis. First, since their analysis only looked at 2022 admissions, it did not include residents who’d been living in nursing homes prior to that year. Because our analysis shows that residents with an SMI tend to stay longer than residents without an SMI, this skews the state’s data toward lower rates. Second, the state was not in compliance with PASRR in 2022, as they note, so the quality of this data point compared to the direct diagnoses fields used in our analysis is questionable.

Identifying residents with SMI who were not identified as such through PASRR

We identified approximately 16,000 residents in our census who had a serious mental illness diagnosis in 2022, but were not identified as such through the PASRR process.

However, not all people with an SMI will necessarily be identified via the PASRR process, even if the process is working as intended. One reason: If someone has a primary diagnosis of dementia, they are exempt from the PASRR mental illness screening process. Looking solely at residents in our 2022 census with a serious mental illness diagnosis and no dementia or Alzheimer’s diagnosis, we found 8,755 residents who were not identified as having a serious mental illness via PASRR. (There were 2,890 residents in this category who were identified as having an SMI via PASRR.)

Another reason someone with a serious mental illness may not be identified via PASRR, according to MDS: If “the resident was admitted from a hospital after requiring acute inpatient care, is receiving services for the condition for which they received care in the hospital, and the attending physician has certified before admission that the resident is likely to require less than 30 days of nursing home care.” (See pg. 113 of the MDS manual here.)

We therefore further narrowed our estimate to long-term residents with serious mental illness and no dementia or Alzheimer’s diagnosis. There were 4,764 such residents on Dec. 31, 2022 who were not identified in our analysis as having an SMI via PASRR. (There were 1,359 residents in this category who we identified as having an SMI via PASRR.)

More on calculating joint dementia and SMI patients; DOJ Louisiana investigation

As noted above regarding PASRR, we calculated rates of serious mental illness including and excluding those who also had a dementia-related diagnosis. The intersection of dementia and SMI also appeared in two more noteworthy instances.

1. The DHCS response to our findings: The agency listed dementia as one reason why someone may reside in a nursing home while having a serious mental illness. While this may be true, experts we spoke with noted that residents who have a dual diagnosis of dementia and a serious mental illness need both conditions addressed. David Grabowski, professor of health care policy at Harvard University, noted the gains that have been made in dementia care at many facilities, but said there needs to be additional care and resources for a serious mental diagnosis on top of that. Our analysis found 9,390 residents with a dementia or Alzheimer diagnosis along with a serious mental illness diagnosis.

2. Comparing California to Louisiana: In the story, we mention a comparison of Louisiana data at the time of the U.S. Department of Justice’s (DOJ) 2016 investigation and California in 2022. The rate of serious mental illness among residents of Louisiana nursing homes in 2014, cited by the DOJ in its findings, was 14.5%.

The data in the DOJ’s findings — which resulted in a lawsuit — excluded individuals with dementia-related diagnoses from their estimate of people with a serious mental illness. “[W]e adopted the methodology used by the State to estimate the number of people with serious mental illness in Louisiana’s nursing facilities. As the State did,” the DOJ wrote at the time. However, the DOJ did not note exactly which diagnoses were included as a “serious mental illness,” and they also noted that the data included only Medicaid beneficiaries.

So, we replicated our methodology using MDS data for both states to provide a direct comparison, thus: Serious mental illness was defined as bipolar disorder and/or schizophrenia and/or psychotic disorder, and we removed from the numerator anyone who also had a diagnosis of dementia or Alzheimer’s. When doing this, we got a result of 14.8% of Louisiana nursing home residents in 2014 having SMI but no dementia, which is very close to the DOJ’s calculation. For California in 2022, the equivalent percentage was 14.0%.

Who we talked to for guidance

Brian McGarry, a physical therapist and health services researcher at the University of Rochester, offered guidance on our methods and findings throughout the project. David Grabowski, a professor of health care policy at Harvard University, and Orna Intrator, professor of health services research and policy at the University of Rochester, also provided guidance and expert opinion.

Explore the code

Further details of the methods of these and other statistics in the story are available in our code on GitHub.

Read the investigation

• Investigation: California Nursing Homes Are Becoming ‘De Facto Mental Health Centers’
• Takeaways: Key Analysis And Findings