Automatic Detection of Laparoscopic Videos Distortion Using Machine Learning Classification
,
Abstract
Purpose: In this research, using the Geant4 software toolbox and metamaterials as a neutron shield, it was tried to introduce the proper metamaterial for this matter.
Materials and Methods: In this paper, we leverage the Laparoscopic Video Quality (LVQ) database developed by Khan et al. to train and validate our model. To classify defocus blur, motion blur, and smoke in the laparoscopic video, we adopt a novel approach utilizing a cascade support vector machine (SVM) classifier, which combines decisions from three binary classifiers. The first classifier categorizes videos into two classes: good and distorted. The second classifier focuses on detecting smoke and blur, while the third is dedicated to distinguishing between defocus blur and motion blur.
Results: In this study, we calculate performance metrics, including accuracy rate, precision, recall, F1 score, and execution time, which are crucial indicators for evaluating quality detection results. The machine-learning classification demonstrates notable performance, with an accuracy rate of 96.55% for the first classifier, 100% for the second, and 99.67% for the third classifier. Additionally, the classification achieves a high inference speed of 37 frames per second (fps).
Conclusion: The experimental results showcased in this paper underscore the efficacy of the proposed approach in automatically detecting distortions in a laparoscopic video. The method exhibits high performance, excelling in both accuracy and processing speed. Notably, the method's advantage lies in its simplicity and the fact that it does not necessitate high-performance computer hardware.
1- A. Beghdadi, "Real-Time Distortion Classification inLaparoscopic Videos ICIP." ed, (2020).
2- Z. A. Khan et al., "Towards a Video Quality Assessment based Framework for Enhancement of Laparoscopic Videos." 2020/3// (2020).
3- N. Aldahoul, H.A. Karim, M.J.T Tan, and J.L. Fermin, "Transfer Learning and Decision Fusion for Real Time Distortion Classification in Laparoscopic Videos." IEEE Access, Vol. 9pp. 115006-18, (2021).
4- P. Sánchez-González et al., "Laparoscopic video analysis for training and image-guided surgery." in Minimally Invasive Therapy and Allied Technologies Vol. 20, ed, (2011), pp. 311-20.
5- J. Cartucho, S. Tukra, Y. Li, D. S. Elson, and S. Giannarou, "VisionBlender: a tool to efficiently generate computer vision datasets for robotic surgery." Computer Methods in Biomechanics and Biomedical Engineering: Imaging and Visualization, Vol. 9 (No. 4), pp. 331-38, (2021).
6- S. Voros, J.A. Long, and P. Cinquin, "Automatic detection of instruments in laparoscopic images: A first step towards high-level command of robotic endoscopic holders." in International Journal of Robotics Research, (2007), Vol. 26, pp. 1173-90.
7- D. Owen, M. Grammatikopoulou, I. Luengo, and D. Stoyanov, "Automated identification of critical structures in laparoscopic cholecystectomy." International Journal of Computer Assisted Radiology and Surgery, Vol. 17 (No. 12), pp. 2173-81, 2022/12// (2022).
8- E. Padovan et al., "A deep learning framework for real-time 3D model registration in robot-assisted laparoscopic surgery." International Journal of Medical Robotics and Computer Assisted Surgery, Vol. 18 (No. 3), 2022/6// (2022).
9- M. Carstens et al., "The Dresden Surgical Anatomy Dataset for Abdominal Organ Segmentation in Surgical Data Science." Scientific Data, Vol. 10 (No. 1), pp. 3-3, 2023/1// (2023).
10- B.F. Allen, F. Kasper, G. Nataneli, E. Dutson, and P. Faloutsos, "Visual tracking of laparoscopic instruments in standard training environments." in Studies in Health Technology and Informatics, (2011), Vol. 163: IOS Press, pp. 11-17.
11- D. Bouget, M. Allan, D. Stoyanov, and P. Jannin, "Vision-based and marker-less surgical tool detection and tracking: a review of the literature." in Medical Image Analysis Vol. 35, ed: Elsevier B.V., (2017), pp. 633-54.
12- M. Ali, G. Ochoa-Ruiz, and S. Ali, "A semi-supervised Teacher-Student framework for surgical tool detection and localization." 2022/8// (2022).
13- J. Cartucho, C. Wang, B. Huang, D. S. Elson, A. Darzi, and S. Giannarou, "An enhanced marker pattern that achieves improved accuracy in surgical tool tracking." Computer Methods in Biomechanics and Biomedical Engineering: Imaging and Visualization, Vol. 10 (No. 4), pp. 400-08, (2022).
14- S. Bernhardt, S.A. Nicolau, L. Soler, and C. Doignon, "The status of augmented reality in laparoscopic surgery as of 2016." in Medical Image Analysis Vol. 37, ed: Elsevier B.V., (2017), pp. 66-90.
15- F. Joeres, F. Heinrich, D. Schott, and C. Hansen, "Towards natural 3D interaction for laparoscopic augmented reality registration." Computer Methods in Biomechanics and Biomedical Engineering: Imaging and Visualization, Vol. 9 (No. 4), pp. 384-91, (2021).
16- J.R. Abbing, F.J. Voskens, B.G. A. Gerats, R.M. Egging, F. Milletari, and I.A. M. J. Broeders, "Towards an AI-based assessment model of surgical difficulty during early phase laparoscopic cholecystectomy." Computer Methods in Biomechanics and Biomedical Engineering: Imaging and Visualization, (2023).
17- T.A. Alshirbaji, N.A. Jalal, L. Mündermann, and K. Möller, "Classifying smoke in laparoscopic videos using SVM." Current Directions in Biomedical Engineering, Vol. 3 (No. 2), pp. 191-94, 2017/9// (2017).
18- A. Leibetseder, M.J. Primus, S. Petscharnig, and K. Schoeffmann, "Real-time image-based smoke detection in endoscopic videos." in Thematic Workshops 2017 - Proceedings of the Thematic Workshops of ACM Multimedia 2017, co-located with MM 2017, (2017): Association for Computing Machinery, Inc, pp. 296-304.
19- A. Leibetseder, M.J. Primus, and K. Schoeffmann, "Automatic smoke classification in endoscopic video." in Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), (2018), Vol. 10705 LNCS: Springer Verlag, pp. 362-66.
20- W. Reiter, "Improving endoscopic smoke detection with semi-supervised noisy student models." Current Directions in Biomedical Engineering, Vol. 6 (No. 1), 2020/5// (2020).
21- S. Salazar-Colores, H. M. Jiménez, C.J. Ortiz-Echeverri, and G. Flores, "Desmoking Laparoscopy Surgery Images Using an Image-to-Image Translation Guided by an Embedded Dark Channel." IEEE Access, Vol. 8pp. 208898-909, (2020).
22- N.A. Jalal, T.A. Alshirbaji, L. Mündermann, and K. Möller, "Features for detecting smoke in laparoscopic videos." Current Directions in Biomedical Engineering, Vol. 3 (No. 2), pp. 521-24, 2017/9// (2017).
23- J. Immerkaer, "NOTE: Fast Noise Variance Estimation." in "COMPUTER VISION AND IMAGE UNDERSTANDING," (1996), Vol. 64.
24- A. Chetouani, A. Beghdadi, and M. Deriche, "A new reference-free image quality index for blur estimation in the frequency domain." in IEEE International Symposium on Signal Processing and Information Technology, ISSPIT 2009, (2009), pp. 155-59.
25- R. Bansal, G. Raj, and T. Choudhury, "Blur image detection using Laplacian operator and Open-CV." in Proceedings of the 5th International Conference on System Modeling and Advancement in Research Trends, SMART 2016, (2017): Institute of Electrical and Electronics Engineers Inc., pp. 63-67.
26- U. Ali and M.T. Mahmood, "Analysis of blur measure operators for single image blur segmentation." Applied Sciences (Switzerland), Vol. 8 (No. 5), 2018/5// (2018).
27- A. Onan, "Bidirectional convolutional recurrent neural network architecture with group-wise enhancement mechanism for text sentiment classification." Journal of King Saud University - Computer and Information Sciences, Vol. 34 (No. 5), pp. 2098-117, 2022/5// (2022).
28- Z.A. Khan, A. Beghdadi, M. Kaaniche, and F.A. Cheikh, "RESIDUAL NETWORKS BASED DISTORTION CLASSIFICATION AND RANKING FOR LAPAROSCOPIC IMAGE QUALITY ASSESSMENT." in "2020 IEEE International Conference on Image Processing (ICIP)," (2020).
29- Z.A. Khan, A. Beghdadi, M. Kaaniche, F.A. Cheikh, and O. Gharbi, "A Neural Network based Framework for Effective Laparoscopic Video Quality Assessment." 2022/2// (2022).
30- M. Siddaiah-Subramanya, M. Nyandowe, and K.W. Tiang, "Technical problems during laparoscopy: A systematic method of troubleshooting for surgeons." Innovative Surgical Sciences, Vol. 2 (No. 4), pp. 233-37, 2017/12// (2017).
31- A.P. Twinanda, S. Shehata, D. Mutter, J. Marescaux, M. De Mathelin, and N. Padoy, "EndoNet: A Deep Architecture for Recognition Tasks on Laparoscopic Videos." IEEE Transactions on Medical Imaging, Vol. 36 (No. 1), pp. 86-97, 2017/1// (2017).
32- A. Onan, "Consensus Clustering-Based Undersampling Approach to Imbalanced Learning." Scientific Programming, Vol. 2019(2019).
33- S. Lakshmi Bhavani, "Detection and Classification of Blur Images using Multi-Class Support Vector Machine; Detection and Classification of Blur Images using Multi-Class Support Vector Machine." [Online]. Available: www.ijert.org.
34- B. Schölkopf, "SVMs - A practical consequence of learning theory." IEEE Intelligent Systems and Their Applications, Vol. 13 (No. 4), pp. 18-21, 1998/7// (1998).
35- A. Onan, "An ensemble scheme based on language function analysis and feature engineering for text genre classification." Journal of Information Science, Vol. 44 (No. 1), pp. 28-47, 2018/2// (2018).
36- Z.A. Khan, "Learning based quality assessment for medical imaging in the context of liver cancer treatment." PhD thesis, Sorbonne Paris Nord University Paris XIII, 2021PA131004, (2021).
2- Z. A. Khan et al., "Towards a Video Quality Assessment based Framework for Enhancement of Laparoscopic Videos." 2020/3// (2020).
3- N. Aldahoul, H.A. Karim, M.J.T Tan, and J.L. Fermin, "Transfer Learning and Decision Fusion for Real Time Distortion Classification in Laparoscopic Videos." IEEE Access, Vol. 9pp. 115006-18, (2021).
4- P. Sánchez-González et al., "Laparoscopic video analysis for training and image-guided surgery." in Minimally Invasive Therapy and Allied Technologies Vol. 20, ed, (2011), pp. 311-20.
5- J. Cartucho, S. Tukra, Y. Li, D. S. Elson, and S. Giannarou, "VisionBlender: a tool to efficiently generate computer vision datasets for robotic surgery." Computer Methods in Biomechanics and Biomedical Engineering: Imaging and Visualization, Vol. 9 (No. 4), pp. 331-38, (2021).
6- S. Voros, J.A. Long, and P. Cinquin, "Automatic detection of instruments in laparoscopic images: A first step towards high-level command of robotic endoscopic holders." in International Journal of Robotics Research, (2007), Vol. 26, pp. 1173-90.
7- D. Owen, M. Grammatikopoulou, I. Luengo, and D. Stoyanov, "Automated identification of critical structures in laparoscopic cholecystectomy." International Journal of Computer Assisted Radiology and Surgery, Vol. 17 (No. 12), pp. 2173-81, 2022/12// (2022).
8- E. Padovan et al., "A deep learning framework for real-time 3D model registration in robot-assisted laparoscopic surgery." International Journal of Medical Robotics and Computer Assisted Surgery, Vol. 18 (No. 3), 2022/6// (2022).
9- M. Carstens et al., "The Dresden Surgical Anatomy Dataset for Abdominal Organ Segmentation in Surgical Data Science." Scientific Data, Vol. 10 (No. 1), pp. 3-3, 2023/1// (2023).
10- B.F. Allen, F. Kasper, G. Nataneli, E. Dutson, and P. Faloutsos, "Visual tracking of laparoscopic instruments in standard training environments." in Studies in Health Technology and Informatics, (2011), Vol. 163: IOS Press, pp. 11-17.
11- D. Bouget, M. Allan, D. Stoyanov, and P. Jannin, "Vision-based and marker-less surgical tool detection and tracking: a review of the literature." in Medical Image Analysis Vol. 35, ed: Elsevier B.V., (2017), pp. 633-54.
12- M. Ali, G. Ochoa-Ruiz, and S. Ali, "A semi-supervised Teacher-Student framework for surgical tool detection and localization." 2022/8// (2022).
13- J. Cartucho, C. Wang, B. Huang, D. S. Elson, A. Darzi, and S. Giannarou, "An enhanced marker pattern that achieves improved accuracy in surgical tool tracking." Computer Methods in Biomechanics and Biomedical Engineering: Imaging and Visualization, Vol. 10 (No. 4), pp. 400-08, (2022).
14- S. Bernhardt, S.A. Nicolau, L. Soler, and C. Doignon, "The status of augmented reality in laparoscopic surgery as of 2016." in Medical Image Analysis Vol. 37, ed: Elsevier B.V., (2017), pp. 66-90.
15- F. Joeres, F. Heinrich, D. Schott, and C. Hansen, "Towards natural 3D interaction for laparoscopic augmented reality registration." Computer Methods in Biomechanics and Biomedical Engineering: Imaging and Visualization, Vol. 9 (No. 4), pp. 384-91, (2021).
16- J.R. Abbing, F.J. Voskens, B.G. A. Gerats, R.M. Egging, F. Milletari, and I.A. M. J. Broeders, "Towards an AI-based assessment model of surgical difficulty during early phase laparoscopic cholecystectomy." Computer Methods in Biomechanics and Biomedical Engineering: Imaging and Visualization, (2023).
17- T.A. Alshirbaji, N.A. Jalal, L. Mündermann, and K. Möller, "Classifying smoke in laparoscopic videos using SVM." Current Directions in Biomedical Engineering, Vol. 3 (No. 2), pp. 191-94, 2017/9// (2017).
18- A. Leibetseder, M.J. Primus, S. Petscharnig, and K. Schoeffmann, "Real-time image-based smoke detection in endoscopic videos." in Thematic Workshops 2017 - Proceedings of the Thematic Workshops of ACM Multimedia 2017, co-located with MM 2017, (2017): Association for Computing Machinery, Inc, pp. 296-304.
19- A. Leibetseder, M.J. Primus, and K. Schoeffmann, "Automatic smoke classification in endoscopic video." in Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), (2018), Vol. 10705 LNCS: Springer Verlag, pp. 362-66.
20- W. Reiter, "Improving endoscopic smoke detection with semi-supervised noisy student models." Current Directions in Biomedical Engineering, Vol. 6 (No. 1), 2020/5// (2020).
21- S. Salazar-Colores, H. M. Jiménez, C.J. Ortiz-Echeverri, and G. Flores, "Desmoking Laparoscopy Surgery Images Using an Image-to-Image Translation Guided by an Embedded Dark Channel." IEEE Access, Vol. 8pp. 208898-909, (2020).
22- N.A. Jalal, T.A. Alshirbaji, L. Mündermann, and K. Möller, "Features for detecting smoke in laparoscopic videos." Current Directions in Biomedical Engineering, Vol. 3 (No. 2), pp. 521-24, 2017/9// (2017).
23- J. Immerkaer, "NOTE: Fast Noise Variance Estimation." in "COMPUTER VISION AND IMAGE UNDERSTANDING," (1996), Vol. 64.
24- A. Chetouani, A. Beghdadi, and M. Deriche, "A new reference-free image quality index for blur estimation in the frequency domain." in IEEE International Symposium on Signal Processing and Information Technology, ISSPIT 2009, (2009), pp. 155-59.
25- R. Bansal, G. Raj, and T. Choudhury, "Blur image detection using Laplacian operator and Open-CV." in Proceedings of the 5th International Conference on System Modeling and Advancement in Research Trends, SMART 2016, (2017): Institute of Electrical and Electronics Engineers Inc., pp. 63-67.
26- U. Ali and M.T. Mahmood, "Analysis of blur measure operators for single image blur segmentation." Applied Sciences (Switzerland), Vol. 8 (No. 5), 2018/5// (2018).
27- A. Onan, "Bidirectional convolutional recurrent neural network architecture with group-wise enhancement mechanism for text sentiment classification." Journal of King Saud University - Computer and Information Sciences, Vol. 34 (No. 5), pp. 2098-117, 2022/5// (2022).
28- Z.A. Khan, A. Beghdadi, M. Kaaniche, and F.A. Cheikh, "RESIDUAL NETWORKS BASED DISTORTION CLASSIFICATION AND RANKING FOR LAPAROSCOPIC IMAGE QUALITY ASSESSMENT." in "2020 IEEE International Conference on Image Processing (ICIP)," (2020).
29- Z.A. Khan, A. Beghdadi, M. Kaaniche, F.A. Cheikh, and O. Gharbi, "A Neural Network based Framework for Effective Laparoscopic Video Quality Assessment." 2022/2// (2022).
30- M. Siddaiah-Subramanya, M. Nyandowe, and K.W. Tiang, "Technical problems during laparoscopy: A systematic method of troubleshooting for surgeons." Innovative Surgical Sciences, Vol. 2 (No. 4), pp. 233-37, 2017/12// (2017).
31- A.P. Twinanda, S. Shehata, D. Mutter, J. Marescaux, M. De Mathelin, and N. Padoy, "EndoNet: A Deep Architecture for Recognition Tasks on Laparoscopic Videos." IEEE Transactions on Medical Imaging, Vol. 36 (No. 1), pp. 86-97, 2017/1// (2017).
32- A. Onan, "Consensus Clustering-Based Undersampling Approach to Imbalanced Learning." Scientific Programming, Vol. 2019(2019).
33- S. Lakshmi Bhavani, "Detection and Classification of Blur Images using Multi-Class Support Vector Machine; Detection and Classification of Blur Images using Multi-Class Support Vector Machine." [Online]. Available: www.ijert.org.
34- B. Schölkopf, "SVMs - A practical consequence of learning theory." IEEE Intelligent Systems and Their Applications, Vol. 13 (No. 4), pp. 18-21, 1998/7// (1998).
35- A. Onan, "An ensemble scheme based on language function analysis and feature engineering for text genre classification." Journal of Information Science, Vol. 44 (No. 1), pp. 28-47, 2018/2// (2018).
36- Z.A. Khan, "Learning based quality assessment for medical imaging in the context of liver cancer treatment." PhD thesis, Sorbonne Paris Nord University Paris XIII, 2021PA131004, (2021).
| Files | ||
| Issue | Articles in Press | |
| Section | Original Article(s) | |
| Keywords | ||
| Automatic Detection of Laparoscopic Video Distortion Smoke and Blur Detection Machine-Learning Classification | ||
| Rights and permissions | |
|
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License. |
How to Cite
1.
Belmokeddem M, Khemis K, Loudjedi S. Automatic Detection of Laparoscopic Videos Distortion Using Machine Learning Classification. Frontiers Biomed Technol. 2024;.
