Reproducibility and Stationarity for the Schizophrenia Quality of Life Scale Revision 4 (SQLS-R4) on Taiwan Population

*Corresponding Author:

Chung-Ying Lin
Department of Rehabilitation Sciences
The Hong Kong Polytechnic University
11 Yuk Choi Road, Hung Hom, Hong Kong
Tel: +852-2766-6755
Fax: 852-2330-8656

Abstract

Objective

Although the Schizophrenia Quality of Life Scale Revision 4 (SQLS-R4) is a feasible quality of life (QoL) instrument for people with schizophrenia, the reproducibility and stationarity of SQLS-R4 are underdeveloped. The purposes of this study were (1) to examine the reproducibility, including the relative and the absolute reliabilities of the SQLS-R4; (2) to establish the stationarity of the SQLS-R4 through measurement invariance.

Methods

One hundred people with schizophrenia filled out the SQLS-R4 twice with an interval of 2 weeks. The relative and absolute reliabilities were examined using intraclass correlation coefficients (ICC), the percentages of standard error of measurement (SEM%), and the smallest real difference (SRD%). The measurement invariance was done through nested models of confirmatory factor analyses.

Results

Both the relative (ICC=0.728 to 0.886) and absolute reliabilities (SEM%=3.13% to 8.47%; SRD%=8.64% to 16.58%) were satisfactory for SQLS-R4, which suggested its adequate stability. The measurement invariance of the three-factor construct of SQLS-R4 remained the same across time for people with schizophrenia. Also, factor loadings (P of Δχ2=0.122), item intercepts (P of Δχ2=0.516), and residual variances of measured items (P of Δχ2=0.370) of SQLS-R4 were partial invariance.

Conclusion

The reproducibility and stationarity were established for SQLS-R4, and the results suggested that the SQLS-R4 is reliable and valid across time. Healthcare professionals can use the SQLS-R4 to longitudinally monitor the QoL of people with schizophrenia. In addition, the SQLS-R4 may be used to measure intervention effects on people with schizophrenia.

Keywords

Reliability, Confirmatory Factor Analysis, Measurement Invariance, Psychometrics, Quality Of Life, Schizophrenia, Taiwan

Introduction

People with schizophrenia confront their difficulties of delusions, hallucinations, and disorganized thinking and speech [1], and are found to have worse quality of life (QoL) than healthy individuals do [2]. Although the QoL is useful for healthcare professionals understanding the overall health status [3] and assessing effects of medical interventions [4], the QoL measures specifically designed for people with schizophrenia are still under development. Only the Quality of Life Index for Mental Health, Schizophrenia Quality of Life Scale (SQLS), and S-QoL were specifically developed for schizophrenia [5]. Among these three, the SQLS has been revised several times to the fourth revised version (viz., SQLS-R4) [6], and has acceptable psychometric properties in the UK [7,8], Singapore [9], Malaysia [10], Korean [11], and Taiwan versions [12-14]. Therefore, the SQLS has been widely used for healthcare professionals to assess the QoL of people with schizophrenia cross culturally.

Although the SQLS-R4 is evidenced to have acceptable psychometric properties, the information of its reproducibility and stationarity remains unclear. The people with schizophrenia have cognitive difficulties [15], and may not have sufficient cognition to answer some difficult items [16]. Therefore, adequate reproducibility and stationarity of the SQLS-R4 are important for healthcare professionals to rigorously measure the QoL of people with schizophrenia. As for an instrument used in clinic, reproducibility can be defined as the ability to measure attributes in a consistent manner when administered on several occasions to the target population [17], and is often examined using test-retest reliability. Better reproducibility of the instrument suggests more precise measurement and better ability of detecting changes in research or clinical practice [17,18]. To the best of our knowledge, only two articles reported the test-retest reliability for SQLS/SQLS-R4 [9,12]. Moreover, its absolute reliability has not been examined. The absolute reliability indices, e.g., the standard error of measurement (SEM) and the smallest real difference (SRD), can provide us with the information to compensate the limitations that relative reliability faces, such as limited information about agreement between repeated measures [19].

Stationarity of an instrument demonstrates that the factor structure is being measured as the same across time [20], and an acceptable stationarity indicates that the target population will not differently interpret the instrument across time. To the best of our knowledge, however, the stationarity of any QoL measure for people with schizophrenia, including SQLS/SQLS-R4, has never been examined. Therefore, we do not know whether the people with schizophrenia interpret the construct of SQLS-R4 the same across time. An advanced statistical method of using confirmatory factor analysis (CFA) to test the measurement invariance can help us understand the stationarity of SQLS-R4. Measurement invariance across time directly assesses the equivalence of factor structure, factor loadings, item intercepts, construct intercepts, and residual variances across time [20], and indicates whether the construct remains the same across time.

The study had two aims: (1) to evaluate the reproducibility, including relative and absolute reliabilities, and (2) to examine the stationarity of the SQLS-R4 for people with schizophrenia.

Methods

▪ Participants and instrument

This was a prospective study with a convenience sample recruited in a psychiatric institution in southern Taiwan. The participants, aged > 18 years, were diagnosed with schizophrenia for more than 2 years according to DSM-5 [21]. All of them remained a stable mental and psychiatric function, i.e. no medication adjustment within two months and a score ≤ 48 on the Brief Psychiatric Rating Scale (BPRS). Also, they had to meet the basic criteria for cognitive ability of scoring ≥ 24 on the Mini Mental Status Examination (MMSE). A total of 100 participants were included in this study. The Institutional Review Board (IRB) of National Cheng Kung University Hospital approved this study.

The Schizophrenia Quality of Life Scale Revision 4 (SQLS-R4) is a self-reported 33- item questionnaire with each item coded from 0 (=never) to 4 (=always), except for 4 items coded from 0 (=always) to 4 (=never). The 4 oppositional coded items are scored reversely to have the same tendency as the other items, and make the higher SQLS-R4 score represents a worse QoL. Several published articles [8,10,12-14] have shown the satisfactory psychometric properties of the SQLS-R4, including the Taiwan version [12-14]. The former SQLS-R4 (i.e., SQLS) was firstly developed with three factors of psychosocial, motivation and vitality, and symptoms and side-effects [7], and was revised to two factors of psychosocial and vitality afterward [6]. However, the item numbers and the factor structure have been respectively suggested as 29 and 3 (psychosocial, physical, and vitality) recently, especially for the East population [13,14]. Thus, we analyzed our data based on the 3-correlated-factor model with 29 items.

▪ Procedure and data analysis

All the participants firstly signed the informed consents, and then they filled out a background information sheet and the SQLS-R4 under the supervision of one experienced occupational therapist. After two weeks, the participants completed the SQLS-R4 under the supervision of the same occupational therapist again.

Reproducibility was examined using both absolute and relative reliabilities. Absolute reliability was indicated by the SEM/SRD and the percentage of SEM/SRD (SEM%/SRD%) with a SEM%/SRD%<10%/30% indicating acceptable [22]. In addition, the Bland-Altman figures were used to visualize the absolute reliability [23]. Relative reliability was assessed by using intraclass correlation coefficients (ICC) with ICC value >0.4 indicating fair to excellent [24]. Also, a paired-t test was used to determine whether the difference was significant between the first- and the second-time measures of SQLS-R4.

Measurement invariance across time was evaluated using five nested CFA models. The measurement invariance was assessed in a repeated-measures design based on the suggestion of one review article [25]. Because the sample size was not large enough to examine all items of the SQLS-R4 in a CFA model, we used the factor scores (viz., the scores of psychosocial, physical, and vitality) to represent the items instead. Therefore, as (Figure 1) presents, our proposed models contained two correlated latent constructs (first- and second-time SQLS-R4) and six item scores with first-time factors correlated to second-time factors (i.e., correlated first- and second-time psychosocial; correlated first- and second-time physical; correlated firstand second-time vitality).

The nested models were configural model (Model 1), model with factor loadings constrained (Model 2), model with factor loadings and item intercepts constrained (Model 3), model with factor loadings, item intercepts, and construct intercepts constrained (Model 4), and model with factor loadings, item intercepts, construct intercepts, and residual variances of measured items constrained (Model 5). The five models were determined as acceptable based on P-value of χ²>0.05 [26]. In addition, the data-model fit indices of comparative fit index (CFI) >0.9, root mean square error of approximation (RMSEA) <0.08, and standardized root mean square residual (SRMR) <0.08 also indicate a satisfactory model [26].

The model comparisons were examined using P-value of Δχ², ΔCFI, ΔRMSEA, and ΔSRMR. The measurement invariance was supported for both invariant loadings and intercepts if P-value of Δχ²>0.05 and ΔCFI>-0.01 with ΔRMSEA<0.015; for invariant loadings if ΔSRMR < 0.03; for invariant intercepts if ΔSRMR<0.01. When the criteria of invariance were not fulfilled, a partial invariance with some constraints being relaxed was then examined for each model [27,28]. Finally, the study used the maximum likelihood estimation for all CFAs.

The paired-t test and ICC were tested using SPSS 16.0 (SPSS Inc., Chicago, IL, USA), the SEM/SRD and SEM%/SRD% were calculated using Microsoft Excel, and the CFAs were conducted using Lisrel 8.8 (Scientific Software International, Lincolnwood, IL, USA).

Results

Sixty-six of the 100 participants were male, and 34 were female. About half participants had an educational level of above senior high (n=56), and most were single (n=76) and had no employed experience (n=61). Mean ± SD age, onset age, duration of diseases, and duration in institution of the participants were 49.16 ± 7.85, 22.57 ± 5.79, 26.59 ± 8.08, and 17.40 ± 8.70, respectively.

Table 1 reported the first- and second-time observed SQLS-R4 scores, and no significant differences were found in all three constructs and the total score (P = 0.29 to 0.87). In addition, all ICC values were >0.4 (range: 0.728 to 0.886), all SEM% values were <10% (range: 3.13% to 8.47%), and all SRD% values were <30% (range: 8.64% to 16.58%). Moreover, the satisfactory absolute reliability can be seen on Figure 2.

Table 1: Observed first- and second-time SQLS-R4 scores (N = 100).

Five nested models were conducted, and 3 of them (viz., Models 2, 3 and 5) had some constraints been relaxed. Before the constraints were relaxed, the fit indices of Models 1, 4 were all satisfactory, while some fit indices of Models 2, 3, and 5 were unsatisfactory (Table 2). However, all fit indices were acceptable after the factor loading of psychosocial, item intercept of physical, and variance residual of psychosocial were relaxed for Models 2, 3, and 5, respectively, as seen on Table 2.

Table 2: Data-model fit indices and model comparisons (N = 100)

Discussion

This study appears to be the first one to thoroughly examine the reproducibility (e.g., absolute reliability) and stationarity (e.g., measurement invariance across time) for SQLS-R4. We found that the SQLS-R4, a newly developed schizophrenia-specific self-repot QoL measure, has satisfactory reproducibility for measuring people with schizophrenia. In addition, the measurement invariance suggests that the threefactor construct of SQLS-R4 remains the same across time for people with schizophrenia.

Although the previous studies on SQLS/SQLS-R4 [9,12] do not use the same constructs and items as our study, the test-retest reliability of ICC in our study (ICC values = 0.728-0.886) supported the good reproducibility of our suggested constructs as well as the total score of SQLS-R4. In addition, we extended the information of test-retest reliability of SQLS-R4 to the absolute reliability, and found that each construct as well as the total score of SQLS-R4 was acceptable (all SEM%<10% and all SRD%<30%). Moreover, the SEM values in this study help healthcare professionals understand whether the change of score is beyond measurement errors in our schizophrenia population [29]. Based on our results of the SEM values, psychosocial, physical, vitality, and the total score of SQLS-R4 need to be changed beyond 0.156, 0.206, 0.423, and 0.159, respectively, over time to represent a real change rather than chance variation or measurement error in a schizophrenia population. The SRD values suggest the real change at the 95% confidence interval for an individual [30], and our results demonstrated that real changes of psychosocial, physical, vitality, and total score of SQLS-R4 for a person with schizophrenia are 0.432, 0.571, 0.829, and 0.441, respectively. Moreover, our Bland-Altman figures showed that most participants had their difference of SQLS-R4 scores between first- and second-time fell in 95% confidence interval. Due to the satisfactory ICC, SEM%, and SRD% values, the reproducibility of the three constructs and the total score of SQLS-R4 were established.

The measurement invariance results provided the evidence that the three-correlated-factor structure of the SQLS-R4 with 29 items was the same across time. Factor loading invariance suggested that the unit of the scale was the same across time [31]. Therefore, one point of either physical or vitality SQLS-R4 score reported by people with schizophrenia had the same weight as one point reported two weeks later. However, our results showed that the score of psychosocial did not have the same weight across time. One reason may be that the emotional problems of people with schizophrenia [1,32] made them more difficult to perceive some abstract psychological feelings (e.g., lonely, hopeless) the same across time. Another reason may be due to their negative symptoms, such as social interaction avoiding [1], which influenced their ratings for social items (e.g., difficult to mix with people). The invariance of item intercepts indicated that people with schizophrenia rated their QoL score at the same origins across time in psychosocial and vitality but not in physical domain. The result may be contributed by the gradually decreased physical functioning of people with schizophrenia [33], which may decrease the origin of their physical conditions.

We acknowledged several limitations in this study. First, the sample in this study was from one institution only, and may have limited generalizability toward other schizophrenia population (e.g., the population in communities). Second, the interval between test and retest is two weeks, and may not be long enough for testing the stationarity of SQLS-R4. Although we considered the interval appropriate for examining reproducibility, a longer period to test the stationarity is suggested for future studies. Specifically, a recent review found that the test-retest intervals for patientreported outcome measures such as SQLS-R4 varied among studies: 18% had an interval between 1 day to 1 week; 25% between 1 and 2 weeks; 21% between 2 and 4 weeks; 9% between 1 and 2 months; 13% more than 2 months [34]. Given that more than 40% of the reviewed literature using an interval more than 2 weeks [34], it would be better if future studies can use an interval longer than 2 weeks to corroborate our findings in the stationarity. Third, the sample size in this study was not large enough for us to analyze every item on SQLS-R4, and we only can analyze the staionarity for the domain scores only. The rule of thumb for calculating sample size in a factor analysis is 5 to 10 respondents per item [35]. Therefore, the sample size will be between 290 (5 respondents x 29 items x 2 time points) and 580 (10 respondents x 29 items x 2 time points) if we want to analyze the stationarity for the item scores using the CFA. We thus suggest recruitment of a larger sample size in the further studies to analyze the staionarity of each item on SQLS-R4.

Conclusion

In conclusion, we demonstrated that the SQLS-R4 had the three-factor construct as suggested recently [13,14], and the construct was the same across time. In addition, we confirmed that the SQLS-R4 had good reproducibility and stationarity, and was useful for healthcare professionals assessing the QoL of people with schizophrenia.

Funding

This research was supported in part by (received funding from) the startup fund in the Department of Rehabilitation Sciences, The Hong Kong Polytechnic University, Hong Kong.

Conflict of Interest

The authors declare that they have no conflict of interest.

Ethical Approval

“All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.”

Informed Consent

“Informed consent was obtained from all individual participants included in the study.”

References

1. American Psychological Association. Diagnostic and Statistical Manual of Mental Disorders, Text Revision. 4 ed. Washington, DC: Author; (2000).
2. Su C-T, Ng H-S, Yang A-L, et al. Psychometric evaluation of the Short Form 36 Health Survey (SF-36) and the World Health Organization Quality of Life Scale Brief Version (WHOQOL-BREF) for patients with schizophrenia. Psychol. Assess 26(3), 980-989 (2014).
3. Salek S. Compendium of quality of life instruments. Chichester, NY: John Wiley & Sons; (1998).
4. Adelufosi AO, Ogunwale A, Abayomi O, et al. Socio-demographic and clinical correlates of subjective quality of life among Nigerian outpatients with schizophrenia. Psychiatry. Res 209(3), 320-325 (2013).
5. Chou C-Y, Ma M-C, Yang T-T, et al. Psychometric validation of the S-QoL Chinese (Taiwan) version for patients with schizophrenia. Qual. Life. Res 20(5), 763-767 (2011).
6. Oxford Outcomes Ltd. The Revised Schizophrenia Quality of Life Questionnaire (SQLS-R4): User Manual for the SQLS-R4. Cassington: Oxford Outcomes; (2004).
7. Wilkinson G, Hesdon B, Wild D, et al. Self-report quality of life measure for people with schizophrenia: the SQLS. Br. J. Psychiatry 177(1), 42-46 (2000).
8. Martin CR, Allan R. Factor structure of the Schizophrenia Quality of Life Scale Revision 4 (SQLS-R4). Psychol. Health. Med 12(2), 126-34 (2007).
9. Luo N, Seng BK, Xie F, et al. Thumboo J. Psychometric evaluation of the Schizophrenia Quality of Life Scale (SQLS) in English- and Chinese- speaking Asians in Singapore. Qual. Life. Res 17(1), 115-122 (2008).
10. Taha NA, Ibrahim MI, Rahman AF, et al. Validation of the Schizophrenia Quality of Life Scale Revision 4 among chronic schizophrenia patients in Malaysia. Value. Health. Reg 1, 82-86 (2012).
11. Kim JH, Yim SJ, Min SK. The Korean version of the 4th Revision of Schizophrenia Quality of Life Scale: validation study and relationship with PANSS. J Kor. Neuropsychiatry 45, 401-410 (2006).
12. Kuo P-J, Chen-Sea M-J, Lu RB, et al. Validation of the Chinese version of the Schizophrenia Quality of Life Scale Revision 4 (SQLS-R4) in Taiwanese patients with schizophrenia. Qual. Life. Res 16(9), 1533-1538 (2007).
13. Kuo P-J, Ma H-I, Kuo C-C, et al. Factor analysis of the Schizophrenia Quality of Life Scale Revision 4 (SQLS-R4) Chinese version and related factors. Int. J. Psychiatry. Clin. Pract 13(4), 278-284 (2009).
14. Su C-T, Yang A-L, Lin C-Y. The construct of the Schizophrenia Quality of Life Scale Revision 4 (SQLS-R4) for the population of Taiwan. Occupational. Ther. Int 2017(2017), 5328101 (2017).
15. Mas-Expósito L, Amador-Campos JA, Gómez-Benito J, et al. Research Group on Severe Mental Disorder. The World Health Organization Quality of Life Scale Brief Version: a validation study in patients with schizophrenia. Qual. Life. Res 20(7), 1079-1089 (2011).
16. Bobes J, García-Portilla P, Sáiz PA, et al. Quality of life measures in schizophrenia. Eur. Psychiatry 20(1), S313-S317 (2005).
17. Beckerman H, Roebroeck ME, Lankhorst GJ, et al. Smallest real difference, a link between reproducibility and responsiveness. Qual. Life. Res 10(7), 571-578 (2001).
18. Hopkins WG. Measures of reliability in sports medicine and science. Sports. Med 30(1), 1-15 (2000).
19. Chen H-M, Hsieh C-L, Lo SK, et al. The test-retest reliability of 2 mobility performance tests in patients with chronic stroke. Neurorehabil. Neural. Repair 21(4), 347-352 (2007).
20. Motl RW, DiStefano C. Longitudinal invariance of self-esteem and method effects associated with negatively worded items. Structural. Equation. Modeling 9(4), 562-578 (2002).
21. American Psychological Association. Diagnostic and statistical manual of mental disorders (5th ed.). Washington, DC: Author; (2013).
22. Flansbjer UB, Holmbäck AM, Downham D, et al. Reliability of gait performance tests in men and women with hemiparesis after stroke. J. Rehabil. Med 37(2), 75-82 (2005).
23. Bland JM, Altman DG. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet 1, 307-310 (1986).
24. Lin C-Y, Luh W-M, Yang A-L, et al. Psychometric properties and gender invariance of the Chinese version of the self-report Pediatric Quality of Life Inventory Version 4.0: short form is acceptable. Qual. Life. Res 21(1), 177-182 (2012).
25. Vandenberg RJ, Lance CE. A review and synthesis of the measurement invariance literature: suggestions, practices, and recommendations for organizational research. Organizational. Research. Methods 3(1), 4-70 (2000).
26. Hoyle RH, Panter AT. Writing about structural equation modeling. In: Hoyle ed, Structural equation modeling: Concepts, issues, and application. Thousand Oakes, CA: Sage; (1995).
27. Byrne BM, Schavelson RJ, Muthen B. Testing for the equivalence of factor covariance and mean structures: the issue of partial measurement invariance. Psychol. Bull 105:456-466 (1989).
28. Steenkamp J-BEM, Baumgartner H. Assessing measurement invariance in cross-national consumer research. J. Consumer. Res 25, 78-90 (1998).
29. Lexell JE, Downham DY. How to assess the reliability of measurement in rehabilitation. Am. J Phys. Med. Rehabil 84(9), 719-723 (2005).
30. Liaw L-J, Hsieh C-L, Lo SK, et al. The relative and absolute reliability of two balance performance measures in chronic stroke patients. Disabil. Rehabil 30(9), 656-661 (2007).
31. Lin C-Y, Luh W-M, Cheng C-P, et al. Measurement Equivalence across child self-reports and parent-proxy reports in the Chinese version of the Pediatric Quality of Life Inventory Version 4.0. Child. Psychiatry. Hum. Dev 44(5), 583-590 (2013).
32. van Os J, Kapur S. Schizophrenia. Lancet 374(9690), 635-645 (2009).
33. Connolly M, Kelly C. Lifestyle and physical health in schizophrenia. Advances. Psychiatric. Treatment 11, 125-132 (2005).
34. Quadri N, Wild D, Skerritt B, et al. A literature review of the variance in interval length between administrations for assessment of test retest reliability and equivalence of PRO measures. Value. Health 16(3), A40-A41 (2013).
35. Lin C-Y, Lee T-Y, Sun Z-J, et al. Development of diabetes-specific quality of life module to be in conjunction with the World Health Organization quality of life scale brief version (WHOQOL-BREF). Health. Qual. Life. Outcomes 15(1), 167 (2017).