Virtual reality surgical simulation as a tuition aid for understanding surgical temporal bone anatomy: trial on 15 ear, nose, and throat registrars

Objective: Virtual reality (VR) surgical simulation is only a supplementary teaching tool for surgical students, and current evidence base that supports its benefits in surgical training is lacking. This study aimed to evaluate VR simulation as a tool to improve the understanding of temporal bone anatomy during Ear, Nose, and Throat residency. The primary goal was to determine whether VR simulation improved written examination performance on temporal bone radiology. Notably, 15 residents were recruited and trained by the VOXEL-MAN Tempo surgical simulator system (UKE-Voxel-man Group, Hamburg, Germany, 2009). The secondary outcome was to determine whether VR simulation improved the surgical skills of the trained residents during cadaveric dissection of temporal bone. Methods: Examination performance on temporal bone radiology anatomy was evaluated in 15 residents from the university-affiliated teaching hospital before and after 5 training sessions on the VOXEL-MAN Tempo surgical simulator. Technical skills after simulation training were assessed on cadaveric temporal bones and evaluated by blinded senior otology surgeons. Results: Residents significantly improved their examination performance on their temporal bone radiology anatomy test after completing virtual training on the simulator. Second, a significant correlation was not found between virtual simulator performances and surgical performances on cadaveric model; however, a significant correlation was shown between the anatomic examination result and the performances on the cadaveric model. Conclusion: This study suggests that a high-fidelity VR simulator improves the understanding of temporal bone anatomy and specifically increased trainees’ practical knowledge regarding radiological anatomy of the temporal bone.


Introduction
Flight simulation has become a "standard of practice" in aeronautical training over the last 20 years and has enabled the prevention of risks and errors previously encountered (1). Equally, "virtual reality (VR) training" has significant potential in the field of medical training especially for residents in surgery. Training with real patients is limited by time and opportunity, whereas VR simulation allows repeated practices in a computerized environment that can mimic healthy or pathologic situations (2)(3)(4)(5)(6). The degree of realism depends upon the device's characteristics and sophistication. Repetition of technique in a controlled environment will help residents to learn the required skills for their specialty including anatomic knowledge and the confidence needed for real-life situations (1,7,8).
The evidence base to support the effectiveness of virtual learning in medical training, particularly with regard to potential patient feedback, is challenging and is currently lacking (9,10). Consequently, other performance indicators are employed to rate technical skills such as time to complete the procedure, psychomotor scores, and anatomic knowledge, which are well described in the literature (11,12). The costs and benefits of current simulation applications are also available but often difficult to interpret and appraise.
The issue over measuring patient safety is extremely challenging, and typically, a reduction in clinical errors with improved clinical care only becomes evident after many years of clinical experience (2). The inability to measure cost effectiveness of simulation training and, in particular, high-fidelity VR simulation training remains a major drawback but needs to be addressed so as to improve future investment planning in training healthcare professionals.
Within the Ear, Nose, and Throat surgery domain, simulation has been reported to provide a structured, safe, and supportive environment that is complementary to conventional training (1,2,9,10,13,14). However, research on surgical learning based on simulation and VR is still in its infancy.
In this study, we hypothesize that the integration of VR temporal bone simulation may provide an advantage over current learning methods with regard to understanding temporal bone radiology anatomy. More precisely, we aimed to determine whether it would be advantageous to integrate the use of a surgical simulator (VOXEL-MAN Tempo) in a trainee curriculum in this aspect. The first aim was to determine whether this tool with its 3-dimensional computed tomography (CT) scan guidance technology could improve knowledge base of the radiology appearance of the temporal bone and surgical anatomy. Second, we assessed the trainees' technical skills on cadaveric models after their virtual training sessions.

Methods
A total of 15 Belgian otorhinolaryngology residents affiliated to the University Hospital of Liège were recruited, and all participants completed 5 sessions of antromastoidectomy simulation training. This involved identification and preservation of important anatomic structures using the VOXEL-MAN Tempo device (Hamburg, Germany, 2009) (14). All residents were given an induction session on how to use the device before the training sessions commenced. The 5 training sessions took place in the same room and with the same simulator over a 3-month period. Residents did not have access to any other teaching tools during this period and were alone during these sessions ( Figure 1).

Primary Assessment (Knowledge Assessment of Radiological Anatomy)
Residents were asked to complete an online temporal bone radiology anatomy examination (http://www.radioanatomie. com) directly before and after simulation training, which ensured the residents had very little time to revise. The radiology examination was devised by the neuroradiology department of the Grenoble University Hospital Center and consisted of identifying anatomic structures as seen on CT scans with a range of questions of increasing difficulty. The pretest (GO) was performed just before the training sessions started, and the posttest (RETURN) was taken after the completion of the 5 training sessions. These tests were different (randomized) to limit learning by test-retest effect.
Secondary Assessment (Skills Assessment and Learning Curve) Surgical skills were assessed during training sessions on the VOXEL-MAN Tempo simulator, and the technical mistakes and surgical parameters were recorded. The simulator metrics have been validated (15). Automatically-recorded parameters within the VOXEL-MAN software, were as follows: task fulfillment (percentage of fulfillment of the complete antromastoidectomy), instrument handling (measured as seconds of "wrong handling"), efficiency (measured as minutes used to perform the whole procedure during virtual drilling), number of injured structures (auditory ossicles, dura mater, sigmoid sinus, and facial nerve), and global result.
Surgical skills were also assessed by measuring residents' surgical technique on cadaveric human temporal bones directly after the simulator training period (within a week). Blinded assessment was performed by a senior otology surgeon with a temporal bone dissection rating scale (Table 1).

Statistical Analysis
Statistical analysis was performed using the free software R (https://www.r-project.org, The R Foundation, Vienna, Austria) with R Commander (RCmdr) (R commander, Fox, 2005). Nonparametric tests were used because the data were not normally distributed. Pre-and postsimulation training scores were compared with a nonparametric Wilcoxon test for paired samples. Spearman regression was used to evaluate the correlation between survey scores and dissection scale scores. Results

104
Rogister et al. Virtual reality for understanding surgical temporal bone anatomy B-ENT 2020; 16(2): 103-8 Main Points: • Simulator was effective in improving anatomic knowledge and, in particular, radiology anatomy. • The most successful cadaver dissection subjects were those who were most familiar with radiology anatomy before and after the training period. • Most students were in favor to integrate VR in their training program. Figure 1. VOXEL-MAN Tempo's practical setting for training sessions were considered as statistically significant when a p≤0.05 was reached.

Ethical Consideration
All procedures were in accordance with the ethical standards of the Comité d'éthique Hospitalo-Facultaire Universitaire de Liège and with the 1964 Helsinki declaration and its later amendments. A waiver of informed consent was obtained for anonymized data abstracted from residents testing. Participants were all volunteers and understood their right not to participate, without any change in their ongoing training. They received all explanations concerning the study, the publication type, and the intended audience. Data were collected in an anonymized database.

First Endpoint
Improvement of temporal bone radioanatomy knowledge Each resident completed the online temporal bone radiological anatomy test before (GO) and after (RETURN) the simulation training sessions. Results showed a statistically significant increase of performance between the radiological GO survey and the radiological RETURN survey, with a score improvement of 28±12.12 (p=0.0011) (Table 2, Figure 2). For the record and for comparison, we also gave the radioanatomy test to several novice trainees with no specific knowledge of radioanatomy of the temporal bone, and the average score was 23/76, whereas the average score of the same test performed by 2 seniors' otologists from our department was 73/76, which confirms its discriminatory nature.

Secondary Endpoints
Skills development on the voxel-man tempo surgical simulator Comparison of surgical parameters between session 1 and session 5 showed a statistically significant increase in the percentage of "task fulfillment" (p=0.0033) (Table 2, Figure 3) and the "global results" (+53 ±26.7; p=0.0033) (Figure 4), whereas the "instrument handling" ( Figure 5) and "efficiency" ( Figure  6) variables significantly decreased. All these parameters have been described in the Methods section. Evolution of the du-ration of incorrect instrument handling and number of injured anatomic structures are also reported in Figures 7 and 8.
Transitions from session 1 to session 2 and session 4 to session 5 were noteworthy. Transition to session 2 showed the highest increase of global results (p=0.0053) and the highest decrease in "instrument handling" and "efficiency". Transition to session 5 showed the highest increase in "task fulfillment" (+24.5%±20.03%), a second improvement in "global results" (p=0.0248), and the decrease in "instrument handling" (Figures 3-6).

Skills on cadaveric model
With regard to the surgical results on cadaveric specimens, higher results on the GO survey were associated with a greater score on the dissection scale. Similarly, higher results on the RETURN survey were related to greater score on the dissection scale. However, the magnitude of the improvement in the survey score was not significantly correlated with the score on the dissection scale. These results also showed that the higher the scores of the GO and RETURN questionnaires, the higher the cadaver dissection score (Figure 9 and Figure 10).

Discussion
Overall, our primary outcome indicated that the simulator was effective in improving anatomic knowledge and in particular radiology anatomy. Our study was not designed to prove that simulator training directly affected surgical skills; however, we found that the most successful dissection subjects were those who were most familiar with radiology anatomy before and after the training period. The main limitation of our study was the absence of a control group to compare our results because the study design was a pre-and postevaluation of the same group of 15 residents. This was mainly caused by the small number of residents available to conduct this study, making it statistically challenging to divide them into 2 groups. The same problem arose with the cadaveric dissection assessment before the training period on a simulator to evaluate initial level. Regarding the improvement of simulation performance (measured on simulator-integrated metrics), it was also not surprising to find an improvement from having simulation training on the same device. Because a specific scale was used for cadaveric dissection scoring, simulation performances and subsequent cadaveric performance could not be properly compared. Indeed, there are many temporal bone dissection scales in the literature, but we have chosen the one usually used and adapted to our department (15). It would obviously also have been interesting to compare scores with medical students and staff members after training sessions, and it will be the next step in our research.
The importance of having a knowledge of anatomy location has not been extensively explored in the literature. A few studies previously highlighted that knowledge of anatomy can avoid unnecessary injury to structures in temporal bone surgery and cochlear implantation (14,16,17). Our results are in line with their conclusions; correlation between the anatomy examination performance and higher scores on cadaver dissection was demonstrated. We conclude that the subjects who scored the best in dissection are those who had the best understanding of radiology anatomy before and after the training period. Comparing practices with real patients is obviously limited, whereas VR simulator training allows for repeated practices. This has led to a better knowledge base of the anatomy and suitable surgical gestures, which enhances self-confidence in real situation (1,7,8,10). Companionship training left many trainees with a lack of confidence and difficulty performing suitable surgical gestures, even at the end of their residency period (6). All of the residents in our university accepted to participate in this study and stated their subjective opinion about the device. Even if several weaknesses were reported, most students were in favor to integrate VR in their training program and did not have an aversion in using this technology (3,7,8,16,17). Current literature demonstrated the utility of several high-fidelity simulators (9,14,(18)(19)(20). Previous reports have demonstrated that simulator-generated objective metrics in a standardized temporal bone task could also differentiate between individuals of varying levels of experience (14,15). Therefore, VR simulation seems to be not only a useful training tool but may also have roles in both formative and summative assessment (13-17, 20, 21). Thus, this project delves into a major current issue and offers innovative solutions. Moreover, training in anatomic knowledge is an important element to consider because all trainees will require knowledge of anatomy to correctly diagnose correlated pathologies. The next step would be to determine the optimal integration of teaching tools in the training program effectively. We also think that there is now a bigger need for simulation-based teaching after the coronavirus disease 2019 crisis experience.
This study demonstrated that the use of the VR simulator, the VOXEL-MAN Tempo® device, has led to improved performance on temporal bone anatomy testing. This is an important aspect for the global assessment of this device, which helps to define its place within the training to develop a mixed teaching protocol involving optimal use of different teaching tools.
Ethics Committee Approval: Ethics Committee Approval is not necessary due to the nature of this study.