Exploring the Assessment Tool ICO OSCAR. A Comparison of Reliability of Whole of the Operation with Its Parts

Doi: 10.36351/pjo.v39i2.1545

Abstract

Purpose:  To compare the reliability of Ophthalmology Surgical Competency Assessment Rubric (ICO-OSCAR: phaco) between assessment of whole of the operation and its parts.

Study Design:  Mixed method study design.

Place and Duration of Study:  Mughal Eye Hospital, Lahore from January 2020 to January 2021.

Methods:  Three raters assessed six anonymized phaco video operations with the help of ICO OSCAR phaco first as whole operations and then as parts of the operation. Inter rater reliability of the assessments of parts of the operation were compared with inter rater reliability of whole of the operation. Intra class coefficient (ICC) was used in SPSS – version 20. Raters also filled a survey form to detect how satisfied they were with the rubric. Raters’ feedback was taken to describe shortcomings/faults found during the use of ICO OSCAR phaco form.

Results:  In the quantitative part, the ICC for the combined parts of the operation (0.910) was better than the value of ICC for the whole operation (0.904). In the questionnaire part, raters were satisfied with ICO OSCAR phaco rubric as a useful tool for learning and assessment of phacoemulsification surgical skills. In the qualitative part, many deficiencies were observed by the assessors in the rubric during videos analysis with the rubric ICO OSCAR phaco. This rubric is more accurate for assessment of operations in the operation theater.

Conclusion:  Parts of the operations can be used instead of the whole operations for the assessment of surgical skills of phacoemulsification with the help of ICO OSCAR phaco rubric. For assessment of videos, it requires some modifications as it is more valid for the assessment of surgical skills during live surgery than in operation theater.