This study aimed to assess the reliability and validity of specific sections within the Centers for Medicare and Medicaid Services Inpatient Rehabilitation Facility Patient Assessment Instrument (CMS-IRF PAI) Version 1.4. Specifically, the research focused on Sections GG (Self-Care and Mobility Assessment), B (Hearing, Speech, and Vision), and C (Cognitive Patterns). The goal was to compare these sections against the established Functional Independence Measure (FIM), a widely used tool for evaluating patient function in rehabilitation settings. The comparison utilized the FIM motor subscale (FIMm) and cognitive subscale (FIMc) to determine how well the CMS-IRF PAI sections aligned with existing standards.
This retrospective cohort study was conducted at a single inpatient rehabilitation facility. The participant group consisted of 1296 patients admitted for rehabilitation following a stroke, admitted between October 1, 2016, and October 1, 2019. No specific interventions were applied for the purposes of this study, as it was a retrospective analysis of existing patient data.
The primary outcome measures involved comparing scores from Sections GG, B, and C of the CMS-IRF PAI with scores from the FIMm and FIMc. Spearman’s rank correlation coefficient and Bland-Altman analyses were used to statistically analyze these comparisons and determine the level of agreement and any potential biases between the assessment tools.
The results indicated a strong correlation between Section GG and FIMm scores, both at the time of admission (ρ=0.919, P<.001) and at discharge (ρ=0.929, P<.001). However, the study also revealed that Section GG exhibited more pronounced ceiling effects at discharge (8.6%) compared to FIMm. Bland-Altman analysis further showed a systematic bias towards higher scores on Section GG compared to FIMm, observed at both admission (Bias=2.3%, P<.001) and discharge (Bias=6.2%, P<.001). This bias extended to measuring functional gains, with Section GG indicating greater gains in function (Bias=3.9%, P<.001), particularly in areas like walking and stair climbing (bias=3.71%, P<.001). Notably, self-care items contributed less to the perceived gains in Section GG when compared to FIMm (bias=-7.5%, P<.001).
For cognitive assessment, a combined scale (B+C scale) using Section B and C demonstrated good internal validity (Cronbach’s alpha=0.868). This B+C scale was also highly correlated with FIMc (ρ=0.745). Despite the correlation, the B+C scale tended to rate patients at a higher cognitive level than FIMc (bias=20.0%, P<.001) and presented a greater ceiling effect at admission (20.4%) compared to the minimal ceiling effect observed with FIMc (0.6%).
In conclusion, while both the Section GG and the B+C scale showed significant correlations with the FIM motor and cognitive subscales, respectively, they also exhibited a bias towards higher ability ratings. The presence of ceiling effects, especially at higher levels of cognitive and functional independence, may limit the ability of these CMS-IRF PAI sections to discriminate effectively among higher-functioning patients. Further research is needed to fully understand the clinical acceptability of using Section GG total scores as a reliable outcome measure and predictor of long-term patient outcomes in rehabilitation settings.
