1. Introduction
ince the study on the elderly people and recording related data requires more accuracy, the calculation of measurement error is very important. Each questionnaire must have good validity and reliability in order to be able to collect acceptable data. Although many studies use Intraclass Correlation Coefficient (ICC) (or Cronbach’s alpha) as the reliability coefficient of the questionnaire, this indicator cannot determine how much of the difference in the subscales is related to measurement error. This study aims to introduce a simple formula to calculate the Total Deviation Index (TDI), interpret it and encourage researchers to use this index as a suitable tool to determine the reliability of psychometrics studies and its application in the Quality of Life questionnaire.
2. Methods & Materials
In this cross-sectional study, 105 elderly people living in Tehran, Iran were selected using a convenience sampling technique in 2019. In order to determine their quality of life, 36-Item Short Form Health Survey (SF-36) questionnaire was completed in a period of two weeks and the score of each person (ranged 0-100) was calculated for the whole questionnaire as well as for its 8 sub-scales. Then, the ICC and TDI values for the subscales of the questionnaire were calculated and compared.
3. Results
The Results showed that the lowest value of the TDI was related to the subscale “physical functioning” and the highest value was related to the subscale “emotional problems”. At a 95% confidence interval, the maximum measurement error under the physical functioning and emotional problems subscales was 22 and 34 (out of 100), respectively. For the sub-scales of physical health problems, bodily pain, general health, vitality, social functioning, and mental health, the maximum measurement error values were 31.59, 24.54, 33.69, 29.32, 78.33 and 24.43, respectively (out of 100). According to Pearson correlation test Results, the TDI values were not significantly correlated with the ICC values. That is, if the ICC value is high, the TDI value is not necessary to be low, and vice versa. Although a scale from 0 to 100 was used for scoring the SF-36 subscales, the TDI can be obtained and interpreted similarly in any case where any other scoring methods are used. Moreover, although the TDI values in our study were interpreted at a 95% confidence interval (due to its popularity), it can also be interpreted at any other probability rates.
4. Conclusion
Since the ICC cannot determine how much of the difference in the subscale scores is related to the measurement error, and given the simple interpretation of the TDI, it is recommended that researchers report the TDI value using the proposed formula for the reliability in psychometrics studies.
Ethical Considerations
Compliance with ethical guidelines
This study was approved by the Ethics Committee of University of Social Welfare and Rehabilitation Sciences (Code: IR.USWR.REC.13960379).
Funding
This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.
Authors' contributions
Both authors contributed equally in preparing all parts of the research.
Conflicts of interest
The authors declared no conflict of interest.
References
- Lin LI. Total deviation index for measuring individual agreement with applications in laboratory performance and bioequivalence. Statistics in Medicine. 2000; 19(2):255-70. [DOI:10.1002/(sici)1097-0258(20000130)19:2<255::aid-sim293>3.0.co;2-8] [PMID]
- McHorney CA, Ware Jr JE, Rachel Lu JF, Sherbourne CD. The MOS 36-item Short-Form Health Survey (SF-36): III. Tests of data quality, scaling assumptions, and reliability across diverse patient groups. Medical Care. 1994; 32(1):40-66. [DOI:10.1097/00005650-199401000-00004] [PMID]
- Pan Y, Barnhart HX. Methods for assessing the reliability of quality of life based on SF-36. Statistics in Medicine. 2016; 35(30):5656-65. [DOI:10.1002/sim.7085] [PMID]
- Montazeri A, Goshtasebi A, Vahdaninia M, Gandek B. The Short Form Health Survey (SF-36): Translation and validation study of the Iranian version. Quality of Life Research. 2005; 14(3):875-82. [DOI:10.1007/s11136-004-1014-5] [PMID]
- Lin L, Pan Y, Hedayat AS, Barnhart HX, Haber M. A simulation study of nonparametric total deviation index as a measure of agreement based on quantile regression. Journal of Biopharmaceutical Statistics. 2016; 26(5):937-50. [DOI:10.1080/10543406.2015.1094812] [PMID]