A Framework for Measuring and Improving VR Competency

As Virtual Reality (VR) becomes more accessible and a utilized medium for research, training, and education, subjects who are not familiar with VR experiences must learn how to operate within VR to engage with the educational material, exhibit training experiences, and provide usable data for researchers. A subjects' performance can be affected by their level of familiarity with VR and the impact could appear in analysis based on those performances. This work proposes a framework to identify, measure, and attempt to improve "VR Competency", the ability for a person to understand and utilize a VR experience successfully. The framework focuses on three aspects to master: utilizing VR hardware, performing VR interactions, and understanding instructions to achieve the experience's goal. This dissertation covers three studies by exploring each for its successes and failures, analyze the interactions used, and draw conclusions within the context of the VR framework. The first study explored using VR for fire safety training where subjects would learn from the Community Emergency Response Team (CERT) fire safety module and had to apply the techniques in a simulated house fire. The experience successfully recreated the fire extinguisher operation section of the fire module, but subjects reported interaction frustrations caused from an improperly built tutorial that caused some subjects to not complete the trial. The second study explored using augmented reality (AR) to compare subject matter expertise assessment from an AR assessment tool to a traditional pen-and-paper assessment. This study shows that the AR assessment tool was comparable as an subject matter expertise assessment to the pen-and-paper assessment, but user interface and interaction problems posed difficulties for subjects. The final study explored the instruction aspect of the framework by providing different instructional techniques to subjects performing a robot assembly task. The study showed no statistically significant impact on performance, but subjects preferred hologram-based instructional techniques and performed the fastest on average. As the framework continues to be defined through existing and future works, the results would be used to establish best practices and techniques for future designers and experimenters to utilize to build the best VR experiences possible.