Home   >   CSC-OpenAccess Library   >    Manuscript Information
Assessment of Surgical Expertise in Virtual Reality Simulation by Hybrid Deep Neural Network Algorithms
Bekir Karlik, Recai Yilmaz, Alexander Winkler-Schwartz, Nykan Mirchi, Vincent Bissonnette, Nicole Ledwos, Rolando Del Maestro
Pages - 47 - 59     |    Revised - 30-11-2021     |    Published - 31-12-2021
Volume - 10   Issue - 3    |    Publication Date - December 2021  Table of Contents
MORE INFORMATION
KEYWORDS
Virtual Reality, Hybrid Deep Neural Network, FCNN
ABSTRACT
The utilization of simulation in surgical resident assessment and training allows quantitation of technical skills in risk-free environments. This transformation of current training paradigms which involve the development and application of high-fidelity virtual reality simulators may play an important role in future surgical educational curricula. With the implementation of artificial intelligence (AI) virtual reality simulators have the potential to advance the understanding, training and assessment of psychomotor performance of surgical expertise. In this study, we apply four hybrid deep neural network algorithms called Fuzzy Clustering Neural Networks (FCNN-1, FCNN-2, FCNN-3, and FCNN-4). The proposed study consists of four hybrid deep neural networks methods called Fuzzy Clustering Neural Network (FCNN) were employed to differentiate surgical expertise. Raw data was obtained from virtual reality tumor resection studies utilizing the NeuroVR simulator platform. The performance of neurosurgeons, senior residents, junior residents and medical students was assessed using a series of metrics. For two proposed algorithms (FCNN-3 and FCNN-4), we achieved a further improvement in classification accuracy of 100% correctly classifying the level of expertise of all participants. The other 2 models were also better accuracies than previous studies. The results demonstrate that the four hybrid FCNN algorithms utilized have increased the accuracy of classification compared to previous studies utilizing the same dataset.
Akyol, A., Hacibeyoglu, M., Karlik, B. (2016). Design of Multilevel Hybrid Classifier with Variant Feature Sets for Intrusion Detection System. IEICE Transactions on Information and Systems, E99-D, 7, 1810-1821. https://doi: 10.1587/transinf.2015EDP7357.
Alotaibi, F. E., AlZhrani, G. A., Mullah, M. A., Sabbagh, A. J., Azarnoush, H., Winkler-Schwartz, A., et al. (2015). Assessing bimanual performance in brain tumor resection with NeuroTouch, a virtual reality simulator. Neurosurgery, 11(2), 89-98; discussion 98, https://doi:10.1227/NEU.0000000000000631.
Bingham, G., Macke, W., Miikkulainen, R.A. (2020). Evolutionary optimization of deep learning activation functions. GECCO'20: Proceedings of the 2020 Genetic and Evolutionary Computation Conference, July 8 – 12, 2020, Cancún Mexico, 289–296. https://doi.org/10.1145/3377930.3389841.
Bissonnette, V., Mirchi, N., Ledwos, N., Alsidieri, G., Winkler-Schwartz, A., Del Maestro, R.F. (2019). Artificial ıntelligence distinguishes surgical training levels in a virtual reality spinal task. Journal of Bone and Joint Surgery, vol. 101(23) e127. https://doi:10.2106/jbjs.18.01197PMID: 31800431.
Ceylan, R., Ozbay, Y., Karlik, B. (2009). Classification of ECG arrhythmias using type-2 fuzzy clustering neural network. 14th National Biomedical Engineering Meeting, Balcova, Izmir, 2009. 1-4, https://doi: 10.1109/BIYOMUT.2009.5130250.
Ceylan, R., Ozbay, Y., Karlik, B. (2011). Integration of type-2 fuzzy clustering and wavelet transform in a neural network based ECG classifier. Expert Systems with Applications, 38, 1004-1010. https://doi: 10.1016/j.eswa.2010.07.118.
Ceylan, R., Ozbay, Y., Karlik, B. (2014). Comparison of type-2 fuzzy clustering-based cascade classifier models for ECG arrhythmias. Biomedical Engineering Applications Basis and Communications, 26(06):1450075. https://doi:10.4015/S1016237214500756.
Choudhury, N., Gélinas-Phaneuf, N., Delorme, S., Del Maestro, R. (2013). Fundamentals of neurosurgery: virtual reality tasks for training and evaluation of technical skills. World Neurosurg., 80(5), 9-19. https://doi: 10.1016/j.wneu.2012.08.022.
De Campos Souza, P.V. (2020). Fuzzy neural networks and neuro-fuzzy networks: A review the main techniques and applications used in the literature, Applied Soft Computing, 92, 106275. https://doi.org/10.1016/j.asoc.2020.106275.
Delorme, S., et al. (2012). NeuroTouch: A physics-based virtual simulator for cranial microneurosurgery training. Neurosurgery, 71(1), 32-42. https://doi:10.1227/NEU.0b013e318249c744.
Ershad, M., Rege, R., Fey A. M. (2018). Meaningful Assessment of Robotic Surgical Style using the Wisdom of Crowds. Int J Comput Assist Radiol Surg., 13(7), 1037-1048. https://doi:10.1007/s11548-018-1738-2.
Esme, E. and Karlik, B. (2016). Fuzzy c-means based support vector machines classifier for perfume recognition. Applied soft computing, 46, 452-458. https://doi.org/10.1016/j.asoc.2016.05.030.
Fried, G. M., Feldman, L. S., Vassiliou, M. C., Fraser, S. A., Stanbridge, D., Ghitulescu, G., et al. (2004). Proving the value of simulation in laparoscopic surgery. Annals of Surgery, 240(3), 518-528. https://doi:10.1097/01.sla.0000136941.46529.56.
Gelinas-Phaneuf, N., and Del Maestro, R. F. (2013). Surgical expertise in neurosurgery: integrating theory into practice. Neurosurgery, 73, 30-38. https://doi:10.1227/NEU.0000000000000115.
Hajshirmohammadi, I. and Payandeh, S. (2007). Fuzzy set theory for performance evaluation in a surgical simulator. Presence: Teleoperators Virtual Environ., 16(6), pp.603-622.
Huang J, Payandeh S, Doris P, Hajshirmohammadi I. (2005). Fuzzy classification: towards evaluating performance on a surgical simulator. Stud Health Technol Inform., 111, 194-200. http://ebooks.iospress.nl/publication/10064. Accessed August 23, 2018.
Jog, A., Itkowitz, B., Liu, M., et al. (2011). Towards integrating task information in skills assessment for dexterous tasks in surgery and simulation. IEEE Inter. Conf. on Robotics and Automation. 5273-5278. https://doi:10.1109/ICRA.2011.5979967.
Karlik, B. (2016). The positive effects of fuzzy c-mean clustering on supervised learning classifiers. Inter Journal of Artificial Intelligence and Expert Systems, 7(1), 1-8. https://doi:10.1.1.740.500&rep=rep1&type=pdf.
Karlik, B. and Olgac, A. V. (2011). Performance analysis of various activation functions in generalized MLP architectures of neural networks. Inter Journal of Artificial Intelligence and Expert Systems, 4, 111-122. https://doi:10.1.1.740.9413&rep=rep1&type=pdf.
Karlik, B., Korurek, M., Koçyigit, Y. (2009). Differentiating types of muscle movements using wavelet based fuzzy clustering neural network. Expert Systems, 26(1), 49–59. doi: https://doi.org/10.1111/j.1468-0394.2008.00496.x.
Karlik, B., Tokhi, O., Alci, M. (2002). A novel technique for classification of myoelectric signals for prosthesis, 15th Triennial World Congress, (IFAC’02), Barcelona, Spain 21 - 26 July 2002. https://doi: 10.3182/20020721-6-ES-1901.00980.
Karlik, B., Tokhi, O., Alci, M. (2003). A fuzzy clustering neural network architecture for multi-function upper-limb prosthesis. IEEE Trans. Biomedical Eng., 50(11), 1255–1261. https://doi: 10.1109/TBME.2003.818469.
Karlik, M. and Karlik, B. (2020). Prediction of student's performance with deep neural networks. Inter Journal of Artificial Intelligence and Expert Systems, 9(2), 39-47.
Karlik, M. and Karlik, B., (2021). Second language learning with affective factors and deep neural networks methods. Canadian Journal of Language and Literature Studies, 1(3), 26-43. https://doi.org/10.53103/cjlls.v1i3.20.
Kerwin T, Wiet G, Stredney D, Shen H.W. (2012). Automatic scoring of virtual mastoidectomies using expert examples. Int Journal of Comput Assist Radiol Surg., 7(1), 1-11. https://doi:10.1007/s11548-011-0566-4.
Ledwos, N., Mirchi, N., Bissonnette, V., Winkler-Schwartz, A., Yilmaz, R., Del Maestro, R.F. (2021). Virtual Reality Anterior Cervical Discectomy and Fusion Simulation on the Novel Sim-Ortho Platform: Validation Studies. Operative Neurosurgery, 20(1), 74–82. https://doi.org/10.1093/ons/opaa269.
Liang, H., Shi, M.Y. (2011). Surgical skill evaluation model for virtual surgical training. ApplMech Mater., 40-41, 812-819. https://doi:10.4028/www.scientific.net/AMM.40-41.812.
Loukas C, Georgiou E. (2011). Multivariate autoregressive modeling of hand kinematics for laparoscopic skills assessment of surgical trainees. IEEE Trans Biomed Eng., 58(11), 3289-3297. https://doi:10.1109/TBME.2011.2167324.
Megali, G., Sinigaglia, S., Tonet, O., and Dario, P. (2006). Modelling and evaluation of surgical performance using hidden Markov models. IEEE Trans Biomed Eng, 53(10), 1911-1919. https://doi:10.1109/TBME.2006.881784.
Mehta, R., Bath, S.K., Sharma, A.K. (2018). An analysis of hybrid layered classification algorithms for object recognition. IOSR Journal of Computer Engineering, 20(1), 57-64. https://doi: 10.9790/0661-2001035764.
Mirchi, N., Bissonnette, V., Ledwos, N., Winkler-Schwartz, A., Yilmaz, R., Karlik, B., Del Maestro, R.F. (2020). Artificial Neural Networks to Assess Virtual Reality Anterior Cervical Discectomy Performance. Operative Neurosurgery, 19(1), 65–75. https://doi.org/10.1093/ons/opz359.
Mirchi, N., Bissonnette, V., Yilmaz, R., Winkler-Schwartz, A., Del Maestro, R.F. (2020). The Virtual Operative Assistant: An explainable artificial intelligence tool for simulation based training in surgery and medicine. PLoS ONE, 15(2), e0229596. https://doi.org/10.1371/journal.pone.0229596.
Moorthy, K., Munz, Y., Sarker, S. K., Darzi, A. (2003). Objective assessment of technical skills in surgery. British Medical Journal, 327(7422), 1032-1037.
Rhienmora P, Haddawy P, Suebnukarn S, Dailey M.N. (2011). Intelligent dental training simulator with objective skill assessment and feedback. Artificial Intell Medicine, 52(2), 115-121. https://doi:10.1016/j.artmed.2011.04.003.
Richstone. L., Schwartz, M. J., Seideman, C, Cadeddu, J., Marshall, S., Kavoussi, L. R. (2010). Eye metrics as an objective assessment of surgical skill. Ann Surg., 252(1), 177-182. https://doi:10.1097/SLA.0b013e3181e464fb.
Sabbag, A.J., Bajunaid, H.M., Alarifi, N., Winkler-Schwartz, A., Alsideiri, G., Al-Zhrani, G., Alotaibi, F.E., Bugdadi, A., Laroche, D., Del Maestro, R.F. (2020). Roadmap for developing complex virtual reality simulation scenarios: subpial neurosurgical tumor resection model. World Neurosurgery, 139, e220-e229. https://doi.org/10.1016/j.wneu.2020.03.187.
Sewell, C., Morris, D., Blevins, N.H., et al., Providing metrics and performance feed-back in a surgical simulator. (2008). Comput Aided Surg., 13(2), 63-81. https://doi:10.1080/10929080801957712.
Siyar, S. Azarnoush, H., Rashidi, S., Winkler-Schwartz, A., Bissonnette, V., Ponnudurai, N., Del Maestro, R.F. (2018). Machine learning distinguishes neurosurgical skill levels in a virtual reality tumor resection task.https://arxiv.org/ftp/arxiv/papers/1811/1811.08159.pdf.
Wang, Z. and Fey, A.M. (2019). Convolutional Neural Network (CNN) for objective skill evaluation in robot-assisted surgery. https://arxiv.org/abs/1806.05796.
Winkler-Schwartz, A., Bissonnette, V., Mirchi, N., Ponnudurai, N., Yilmaz, R., Ledwos, N., Siyar, S., Azarnoush, H., Karlik, B., Del Maestro, R.F. (2019). Artificial intelligence in medical education: best practices using machine learning to assess surgical expertise in virtual reality simulation. Journal of surgical education, 76(6), 1681-90.
Winkler-Schwartz, A., Yilmaz, R., Mirchi, N., Bissonnette, V., Ledwos, N., Siyar, S., et al. (2019). Assessment of machine learning identification of surgical operative factors associated with surgical expertise in virtual reality simulation. JAMA Network Open., 2(8), e198363–e198363, https://doi.org/10.1001/jamanetworkopen. PMID: 31373651.
Professor Bekir Karlik
Neurosurgical Simulation and Artificial Intelligence Learning Centre, Department of Neurology and Neurosurgery, McGill University, Montreal, QC - Canada
bkarlik@hotmail.com
Dr. Recai Yilmaz
Neurosurgical Simulation and Artificial Intelligence Learning Centre, Department of Neurology and Neurosurgery, McGill University, Montreal, QC - Canada
Dr. Alexander Winkler-Schwartz
Neurosurgical Simulation and Artificial Intelligence Learning Centre, Department of Neurology and Neurosurgery, McGill University, Montreal, QC - Canada
Dr. Nykan Mirchi
Neurosurgical Simulation and Artificial Intelligence Learning Centre, Department of Neurology and Neurosurgery, McGill University, Montreal, QC - Canada
Dr. Vincent Bissonnette
Division of Orthopaedic Surgery, Montreal General Hospital, McGill University, 1650 Cedar Avenue, Montreal, Quebec, Canada. H3G 1A4 - Canada
Dr. Nicole Ledwos
Neurosurgical Simulation and Artificial Intelligence Learning Centre, Department of Neurology and Neurosurgery, McGill University, Montreal, QC - Canada
Professor Rolando Del Maestro
McGill University - Canada


CREATE AUTHOR ACCOUNT
 
LAUNCH YOUR SPECIAL ISSUE
View all special issues >>
 
PUBLICATION VIDEOS