STUDY DESIGN: Observational inter-rater reliability study. OBJECTIVES: To examine: (1) the inter-rater reliability of a modified version of Delitto et al.'s classification-based algorithm for patients with low back pain; (2) the influence of different levels of familiarity with the system; and (3) the inter-rater reliability of algorithm decisions in patients who clearly fit into a subgroup (clear classifications) and those who do not (unclear classifications). METHODS: Patients were examined twice on the same day by two of three participating physical therapists with different levels of familiarity with the system. Patients were classified into one of four classification groups. Raters were blind to the others' classification decision. In order to quantify the inter-rater reliability, percentages of agreement and Cohen's Kappa were calculated. RESULTS: A total of 36 patients were included (clear classification n = 23; unclear classification n = 13). The overall rate of agreement was 53% and the Kappa value was 0.34 [95% confidence interval (CI): 0.11-0.57], which indicated only fair inter-rater reliability. Inter-rater reliability for patients with a clear classification (agreement 52%, Kappa value 0.29) was not higher than for patients with an unclear classification (agreement 54%, Kappa value 0.33). Familiarity with the system (i.e. trained with written instructions and previous research experience with the algorithm) did not improve the inter-rater reliability. CONCLUSION: Our pilot study challenges the inter-rater reliability of the classification procedure in clinical practice. Therefore, more knowledge is needed about factors that affect the inter-rater reliability, in order to improve the clinical applicability of the classification scheme.
AD - Department of Epidemiology and Biostatistics and the EMGO+ Institute for Health and Care Research, VU University Medical Centre, Amsterdam, The Netherlands & Rehabilitation Department, Medical Centre Alkmaar, The Netherlands.STUDY DESIGN: Observational inter-rater reliability study. OBJECTIVES: To examine: (1) the inter-rater reliability of a modified version of Delitto et al.'s classification-based algorithm for patients with low back pain; (2) the influence of different levels of familiarity with the system; and (3) the inter-rater reliability of algorithm decisions in patients who clearly fit into a subgroup (clear classifications) and those who do not (unclear classifications). METHODS: Patients were examined twice on the same day by two of three participating physical therapists with different levels of familiarity with the system. Patients were classified into one of four classification groups. Raters were blind to the others' classification decision. In order to quantify the inter-rater reliability, percentages of agreement and Cohen's Kappa were calculated. RESULTS: A total of 36 patients were included (clear classification n = 23; unclear classification n = 13). The overall rate of agreement was 53% and the Kappa value was 0.34 [95% confidence interval (CI): 0.11-0.57], which indicated only fair inter-rater reliability. Inter-rater reliability for patients with a clear classification (agreement 52%, Kappa value 0.29) was not higher than for patients with an unclear classification (agreement 54%, Kappa value 0.33). Familiarity with the system (i.e. trained with written instructions and previous research experience with the algorithm) did not improve the inter-rater reliability. CONCLUSION: Our pilot study challenges the inter-rater reliability of the classification procedure in clinical practice. Therefore, more knowledge is needed about factors that affect the inter-rater reliability, in order to improve the clinical applicability of the classification scheme.
PY - 2016 SN - 1066-9817 (Print)