Skip to main content Skip to main navigation menu Skip to site footer

Application of mixed reality and artificial intelligence to assist medical students in learning injection technique


Introduction: The usage of immersive technology has advanced in a number of areas of life because of the development of technology that keeps pace with the times. Another immersive technology that combines VR and AR is mixed reality (MR), which enables us to interact with 3-dimensional objects in the real world. Since MR technology gives a more nuanced experience, the market is highly promising. This study aims to evaluate the application of mixed reality and artificial intelligence to assist medical students in learning injection technique.

Methods: This type of research is analytic with a quantitative and qualitative approach to prove the purpose of the research. This research involved 40 students. Due to the creative nature of immersive technology, it must be combined with other technologies to produce an even more complicated and engaging experience. In order to enhance the quality of the user experience, we will merge MR immersive technology with AI in this research for medical educational field. The integration of these two technologies via an application that can be launched on a Hololens 2 and Magic Leap 1 device and can identify person in a laboratory to support in student learning.

Results: For instance, students can utilize artificial intelligence (AI) to learn the names of objects in the lab and do simulation about injection technique. The study 's outcomes are presented in software testing (FPS, CPU, GPU, and load scene) an in the form of user testing utilizing the PIECES Framework (Performance, Information and Data, Economy, Control and Security, Efficiency, and Service), which evaluates the application's utility or significance as well as the satisfaction of its users.  

Conclusion: The system was able to develop a combining application of artificial intelligence and mixed reality for detecting objects in laboratories to assist learning students, according to the study's conclusions.


  1. Cen L, Ruta D, Qassem LMMS Al, Ng J. Augmented Immersive Reality (AIR) for Improved Learning Performance: A Quantitative Evaluation. IEEE Trans Learn Technol. 2020;13(2):283–96.
  2. Chan J, Leung H, Tang J, Komura T. A Virtual Reality Dance Training System Using Motion Capture Technology. Learn Technol IEEE Trans. 2011;4:187–95.
  3. Santos MEC, Chen A, Taketomi T, Yamamoto G, Miyazaki J, Kato H. Augmented Reality Learning Experiences: Survey of Prototype Design and Evaluation. IEEE Trans Learn Technol. 2014;7(1):38–56.
  4. Bacca J, Baldiris S, Fabregat R, Graf S, Kinshuk. Augmented Reality Trends in Education: A Systematic Review of Research and Applications. J Educ Technol Soc. 2014;17(4):133–49. Available from:
  5. Pavithra A. An Emerging Immersive Technology-A Survey. Int J Innov Res Growth. 2020;6:119–30.
  6. Wang P, Zhang S, Bai X, Billinghurst M, He W, Sun M, et al. 2.5DHANDS: a gesture-based MR remote collaborative platform. Int J Adv Manuf Technol. 2019;102(5):1339–53. Available from:
  7. Piumsomboon T, Dey A, Ens B, Lee G, Billinghurst M. The Effects of Sharing Awareness Cues in Collaborative Mixed Reality. Front Robot AI. 2019;6:5-11.
  8. Wang P, Zhang S, Billinghurst M, Bai X, He W, Wang S, et al. A comprehensive survey of AR/MR-based co-design in manufacturing. Eng Comput. 2020;1(1):36-43.
  9. Iwasaki Y, Nishimura S, Hamada Y, Kozono K. Development of the MR laboratory for electrical experiment using ARToolKit. 2010 9th Int Conf Inf Technol Based High Educ Train. 2010;1(1):125–38.
  10. Delmerico J, Poranne R, Bogo F, Oleynikova H, Vollenweider E, Coros S, et al. Spatial Computing and Intuitive Interaction: Bringing Mixed Reality and Robotics Together. IEEE Robot Autom Mag. 2022;29(1):45–57.
  11. Bucsai S, Kučera E, Haffner O, Drahoš P. Control and Monitoring of IoT Devices Using Mixed Reality Developed by Unity Engine. In: 2020 Cybernetics & Informatics (K&I). 2020. p. 1–8.
  12. Musharyanti L, Yusup RM, Priyatnanto H. Teaching method to increase critical thinking in health profession student: a literature review. Bali Med J. 2021;10(3):1083–7.
  13. Kiruthika J, Khaddaj S. Impact and Challenges of Using of Virtual Reality & Artificial Intelligence in Businesses. In: 2017 16th International Symposium on Distributed Computing and Applications to Business, Engineering and Science (DCABES). 2017. p. 165–8.
  14. Krishna MVP, Mehta S, Verma S, Rane S. Mixed Reality in Smart Computing Education System. In: 2018 International Conference on Smart Systems and Inventive Technology (ICSSIT). 2018. p. 72–5.
  15. Chiou Y-M, Barrnaki R. Learning Tornado Formation via Collaborative Mixed Reality. In: 2019 IEEE Conference on Virtual Reality and 3D User Interfaces (VR). 2019. p. 1369–70.
  16. Schmidhuber J. Deep learning in neural networks: An overview. Neural Networks. 2015;6(1):85–117. Available from:
  17. Szegedy C, Toshev A, Erhan D. Deep Neural Networks for object detection. Adv Neural Inf Process Syst. 2013;1(1):26-34.
  18. Girshick R, Donahue J, Darrell T, Malik J. Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation. In: 2014 IEEE Conference on Computer Vision and Pattern Recognition. 2014. p. 580–7.
  19. Sankaran NK, Nisar HJ, Zhang J, Formella K, Amos J, Barker LT, et al. Efficacy Study on Interactive Mixed Reality (IMR) Software with Sepsis Prevention Medical Education. In: 2019 IEEE Conference on Virtual Reality and 3D User Interfaces (VR). 2019. p. 664–70.
  20. Teng Z, Hanwu H, Yueming W, He’en C, Yongbin C. Mixed Reality Application: A Framework of Markerless Assembly Guidance System with Hololens Glass. In: 2017 International Conference on Virtual Reality and Visualization (ICVRV). 2017. p. 433–4.
  21. Leader JF. Mixed Reality Therapy Clinic Design. In: 2018 IEEE Games, Entertainment, Media Conference (GEM). 2018. p. 1–9.
  22. Morina N, Ijntema H, Meyerbröker K, Emmelkamp PMG. Can virtual reality exposure therapy gains be generalized to real-life? A meta-analysis of studies applying behavioral assessments. Behav Res Ther. 2015;7(4):18–24. Available from:
  23. Kolb D. Experiential Learning: Experience as the source of Learning and Development Second Edition. Pearson Education; 2015.
  24. Varela F., Thompson E, Rosch E. The embodied mind: Cognitive science and human experience. MIT Press; 1992.
  25. Heft H. Ecological psychology in context: James Gibson, Roger Barker, and the legacy of William James’s radical empiricism. J Hist Behav Sci. 2003;39(3):320–9. Available from:

How to Cite

Hanfati, K. ., Sukaridhoto, S. ., Rante, H. ., Budiarti, R. P. N., & Nadatien, I. . (2023). Application of mixed reality and artificial intelligence to assist medical students in learning injection technique. Bali Medical Journal, 12(3), 3363–3369.




Search Panel

Kirana Hanfati
Google Scholar
BMJ Journal

Sritrusta Sukaridhoto
Google Scholar
BMJ Journal

Hestiasari Rante
Google Scholar
BMJ Journal

Rizqi Putri Nourma Budiarti
Google Scholar
BMJ Journal

Ima Nadatien
Google Scholar
BMJ Journal