Virtual reality as a teaching method for first-year undergraduate medical student resuscitation training during the COVID-19 pandemic: a randomized controlled trial | BMC medical training

study design

This randomized, double-blind, controlled study was performed at the Department of Anesthesiology at the University Medical Center Hamburg-Eppendorf, during the winter semester 2020/21. This study was reported in accordance with CONSORT guidelines [24].

During their first semester in medical school, all 1^st undergraduate students are assigned to mandatory BLS training, provided by the Department of Anesthesiology. Before the pandemic, this training included a theoretical part (seminar) and practical training on a mannequin. During the pandemic, face-to-face teaching was interrupted or strictly modified (strict hygiene rules – no more than three people in a room, limited number of students allowed to enter the university building per day). Consequently, the BLS training was replaced by online training, which was distributed via Cisco Webex™ Online Meetings, Milpitas, CA, USA.

A maximum of twenty-one undergraduate students were assigned to each course and a total of 19 courses were delivered. Each online training also consisted of two parts: First, a 60-minute BLS seminar was led by an instructor, covering all learning objectives as outlined by the 2021 European Resuscitation Council (ERC) guidelines [25, 26]. The seminar was followed by an online demonstration (120 min) of BLS which was carried out by two instructors, using the Resusci Anne QCPR, Laedal, Stavanger, Norway. One of the instructors demonstrated the BLS sequence, pitfalls, and typical chest compression mistakes, such as the wrong compression depth or rate. During the demonstration, the undergraduates were discussed by the second instructor. Undergraduate students were encouraged to perform cardiopulmonary resuscitation on pillows at home.

The intervention group completed additional VR BLS training within three days of the online training. The VR training consisted of an introduction to the VR module (20 min) and a training unit (35 min). At the end of the VR training, the undergraduate students completed a three-minute Structured Clinical Examination (SCE) on BLS, using the Resusci Anne QCPR (Laedal, Stavanger, Norway). The control group also completed the SCE in the same time frame after the online training. All SCEs were supervised by the same instructor, who familiarized both groups with the manikin and its functions before each SCE, in order to reduce cognitive biases.

In summary, the main differences between the training approaches were as follows: Before the pandemic, students participated in face-to-face training, in which practical skills were rehearsed directly on manikins. During the pandemic, this training was replaced by online instruction on a manikin, with no possibility of rehearsal on a manikin. The VR training allowed the students to train the skills on the manikin with direct feedback through the VR module.

Speakers

All first-year students (NOT= 360) were eligible for the study. Prerequisite for participation was participation in BLS online training prior to the intervention. Exclusion criteria were disease symptoms (undergraduate students were not routinely tested for COVID-19 at this time). An e-mail with a detailed description of the study, the VR training and the possibility of applying, has been sent to 1^st undergraduate students two weeks before the semester.

A total of 120 VR slots were incorporated into the teaching schedule for undergraduates by the deanship and undergraduates were required to apply for VR training by emailing the educational coordinator of the Department of Anesthesiology . The first 120 undergraduate students who applied were enrolled in the study and given a VR appointment. Those who confirmed their appointment were randomized to the intervention or control group (computer generated random numbers). Undergraduates were blinded, and assignment to study groups was only documented by one instructor and not disclosed to undergraduates or BLS checklist assessors. Undergraduates have been asked to keep a low profile about their education.

Intervention

BLS training in virtual reality

The individual VR training had a duration of 35 min and was supervised by the same instructor. The VR system and a pilot version of the software were developed by VIREED MED, Hamburg- Germany, a start-up founded in 2017. Thanks to a large research project by the “Jung Foundation for Science and Research”, we have was able to acquire the VR system as well as additional services from VIREED MED, which included adapting the software to our requests and needs. The VR system is connected to a small CPR manikin – and therefore training of chest compressions is possible and direct feedback on the quality of chest compressions is provided visually (Fig. 1b). Bag-mask ventilation and use of an AED are virtually implemented in the system, but no actual haptic manipulation takes place.

The VR BLS training consisted of two sections: In the first section, a correct BLS scenario was demonstrated and explained by a virtual teacher. After that, the participants had to manage and guide a BLS scenario with a virtual colleague, who performed the chest compressions. In the second section, students could practice chest compressions on the manikin and the virtual college provided the bag-mask-ventilation. Subsequently, they were faced with a real emergency scenario in which they had to provide BLS without assistance. Figure 1 summarizes the VR BLS training timeline and content.

Results

The primary endpoint was the time of absence of flow, assessed during the three-minute structured clinical examination (SCE) by the Laerdal Skill Reporter software (Laerdal, Stavenger, Norway).

The secondary outcome was the overall performance of the BLS, assessed by an adapted observation checklist which is used by the ERC and which was validated by Graham and Lewis in 2000 [27]. Each SCE was recorded and then independently reviewed by two blinded raters experienced in BLS training and medical education.

The BLS checklist is made up of ten items (Table 1) and for each item penalty points can be awarded, based on predefined performance. Penalty points are awarded for incorrect performance of each BLS component, based on the potential to compromise patient safety. The best possible BLS performance is combined with zero penalty points, the worst performance with 125 penalty points.

The tertiary outcome was learning gain which was assessed by comparative self-assessment (CSA) [28], a validated self-assessment tool, consisting of eleven questions (shown in Table 3.) that assess BLS learning gain. For each question, a six-point Likert scale is provided (1 = mainly applies; 6= Not Applicable).

Undergraduate students completed the CSA before and after the intervention/control SCE.

The learning gain was calculated with two methods. First according to the following formula which was described by Raupach and the colleges, in order to calculate the percentage learning gain [28].

CSA gain (%) = (CSA_pre—ASC_Publish) / (AUC_pre—1) × 100.

In this method, participants who gave themselves the highest possible score (1 = mainly applied) in the pre-test were, so to speak, “automatically” excluded from the analysis, because the term (CSA_pre—1) leads to a division by zero, resulting in missing % Learning Gain values ​​for these participants.

To calculate the score point differences, a subtraction of the pre-intervention and post-intervention scores of all undergraduate students was performed:

CSA gain (points) = CSA_pre—ASC_Publish.

statistical analyzes

Descriptive statistics were applied for the calculation of the average values ​​of the penalty points, attributed by the two evaluators. The penalty points of each study group were compared by applying an unpaired t-test. For the calculation of the rater’s agreement (penalty points), the intraclass correlations (ICC) were calculated, with atwo-way random effects model(definition of agreement). The ICC was interpreted according to Ciccetti: ICC values ​​below 0.40 are interpreted as low values ​​between 0.40 and 0.59 as acceptable, between 0.59 and 0.75 as good and values ​​between 0.75 and 1 as an excellent correlation [29].

Sample size calculations with PASS 2008 version 08.0.6 [30] reported that a sample size of 42 for each group achieves 81% power to detect superiority using a one-tailed two-sample t-test (assumptions: equivalence margin = 0, true ratio of means = 0.9, α = 0.025, coefficients of variation of the two groups = 0.17).

Histograms of the data distributions of the dependent variables (time out of flow, CSA difference, % CSA gain) were visually examined by intervention group (and CSA item, if applicable). Their variances were calculated by intervention group and assessed for homogeneity. Data values ​​of No Flow Time (in seconds) were transformed to ln before further analyzes because they were right-skewed. This means that the data has been transformed into its natural logarithm (= ln). This transformation is used to eliminate or reduce rightward skew in the distribution of the data, so that the data better fits general linear modeling.

A general linear model was fitted to the dependent variable (time without flow) – with intervention group as a fixed effect. For the dependent variables CSA-difference and % CSA gain, a general linear mixed-effects model was applied, considering the participant as a random effect and the CSA items with the participants as repeated measures.

The CSA difference was intervention group, CSA item, and intervention group x CSA item, adjusting the baseline analysis by the CSA pre-scoring used as a covariate. For the dependent variable % CSA gain, the same fixed effects were included in the model, except for CSA pre-scoring. Marginal means estimated by the model with 95% confidence intervals were calculated and pairwise group comparisons were made. IBM SPSS version 27 was used for all statistical analyzes using its GENLINMIXED routine for general linear (mixed) modeling work. One to two tailsp