An Analysis of the Extent of Intra and Inter-rater Variability in OSCE

Abstract:

Background: OSCE is an objective structured clinical examination, though the tool is designed to increase the objectivity in the examination of the clinical cases, the inter and intra rater variability of the tool needs to analysed. Aim: To determine the extent of intra and inter-rater variability in OSCE. Settings and Design: The study was conducted in a medical college and is a cross sectional study. Materials and Methods: 5 OSCE stations were designed and videos recorded of the candidates performing the clinical exercise, 9 teachers were selected, three of assistant professor grade (junior level), three of associate grade (middle level) and three of professor grade (senior level), all the teachers were shown the videos on day 1 and the scores recorded, the same videos were shown to the teachers on the second day and the scores recorded, each teacher graded all the 5 OSCE recordings. Statistical Analysis Used: Student paired t test, One-Way ANOVA and multiple comparison to Tukey test and Cronbach alpha. Software used for analyses was SPSS 27.0 version. Results: when the scores were segregated according to level of Assessors it was observed that there was significant intra-rater variability at senior level, while there is significant inter-rater variability between junior and mid-level assessors. Conclusion: There is intra and inter-rater variability observed in OSCE assessment when assessors were segregated in groups as per their seniority levels.

Keywords: OSCE, Variability, Inter rater, Intra rater.