Temporal Registration of Cardiac Multimodal Images Using Locally Linear Embedding Algorithm
,
Abstract
Purpose: Multimodal Cardiac Image (MCI) registration is one of the evolving fields in the diagnostic methods of Cardiovascular Diseases (CVDs). Since the heart has nonlinear and dynamic behavior, Temporal Registration (TR) is the fundamental step for the spatial registration and fusion of MCIs to integrate the heart's anatomical and functional information into a single and more informative display. Therefore, in this study, a TR framework is proposed to align MCIs in the same cardiac phase.
Materials and Methods: A manifold learning-based method is proposed for the TR of MCIs. The Euclidean distance among consecutive samples lying on the Locally Linear Embedding (LLE) of MCIs is computed. By considering cardiac volume pattern concepts from distance plots of LLEs, six cardiac phases (end-diastole, rapid-ejection, end-systole, rapid-filling, reduced-filling, and atrial-contraction) are temporally registered.
Results: The validation of the proposed method proceeds by collecting the data of Computed Tomography Coronary Angiography (CTCA) and Transthoracic Echocardiography (TTE) from ten patients in four acquisition views. The Correlation Coefficient (CC) between the frame number resulted from the proposed method and manually selected by an expert is analyzed. Results show that the average CC between two resulted frame numbers is about 0.82±0.08 for six cardiac phases. Moreover, the maximum Mean Absolute Error (MAE) value of two slice extraction methods is about 0.17 for four acquisition views.
Conclusion: By extracting the intrinsic parameters of MCIs, and finding the relationship among them in a lower-dimensional space, a fast, fully automatic, and user-independent framework for TR of MCIs is presented. The proposed method is more accurate compared to Electrocardiogram (ECG) signal labeling or time-series processing methods which can be helpful in different MCI fusion methods.
1- O. World Health, World health statistics 2020: monitoring health for the SDGs, sustainable development goals. Geneva: World Health Organization, 2020.
2- Q. Zhang, A. Samani, and T. M. Peters, "MR and ultrasound cardiac image dynamic visualization and synchronization over Internet for distributed heart function diagnosis," Comput Med Imaging Graph, vol. 88, p. 101850, Mar 2021.
3- J. Jose, N. Gautam, M. Tiwari, T. Tiwari, A. Suresh, V. Sundararaj, et al., "An image quality enhancement scheme employing adolescent identity search algorithm in the NSST domain for multimodal medical image fusion," Biomedical Signal Processing and Control, vol. 66, p. 102480, 2021/04/01/ 2021.
4- M.-A. Azam, K.-B. Khan, M. Ahmad, and M. Mazzara, "Multimodal Medical Image Registration and Fusion for Quality Enhancement," Computers, Materials \& Continua, vol. 68, pp. 821-840, 2021.
5- J. Grondin, D. Wang, C. S. Grubb, N. Trayanova, and E. E. Konofagou, "4D cardiac electromechanical activation imaging," Computers in Biology and Medicine, vol. 113, p. 103382, 2019/10/01/ 2019.
6- T. Tada, K. Osuda, T. Nakata, I. Muranaka, M. Himeno, S. Muratsubaki, et al., "A novel approach to the selection of an appropriate pacing position for optimal cardiac resynchronization therapy using CT coronary venography and myocardial perfusion imaging: FIVE STaR method (fusion image using CT coronary venography and perfusion SPECT applied for cardiac resynchronization therapy)," Journal of Nuclear Cardiology, 2019/08/21 2019.
7- S. Zandieh, R. Bernt, S. Mirzaei, J. Haller, and K. Hergan, "Image fusion between 18F-FDG PET and MRI in cardiac sarcoidosis: A case series," Journal of Nuclear Cardiology, vol. 25, pp. 1128-1134, 2018/08/01 2018.
8- C. Nobre, M. Oliveira-Santos, L. Paiva, M. Costa, and L. Gonçalves, "Fusion imaging in interventional cardiology," Revista Portuguesa de Cardiologia, vol. 39, pp. 463-473, 2020/08/01/ 2020.
9- A. Peters, A. Motiwala, B. O'Neill, and P. Patil, "Novel use of fused cardiac computed tomography and transesophageal echocardiography for left atrial appendage closure," Catheter Cardiovasc Interv, Mar 9 2020.
10- Y. Takaya and H. Ito, "New horizon of fusion imaging using echocardiography: its progress in the diagnosis and treatment of cardiovascular disease," Journal of Echocardiography, vol. 18, pp. 9-15, 2020/03/01 2020.
11- A. Khalil, A. Faisal, K. W. Lai, S. C. Ng, and Y. M. Liew, "2D to 3D fusion of echocardiography and cardiac CT for TAVR and TAVI image guidance," Med Biol Eng Comput, vol. 55, pp. 1317-1326, Aug 2017.
12- Z. Luo, J. Cai, T. M. Peters, and L. Gu, "Intra-Operative 2-D Ultrasound and Dynamic 3-D Aortic Model Registration for Magnetic Navigation of Transcatheter Aortic Valve Implantation," IEEE Transactions on Medical Imaging, vol. 32, pp. 2152-2165, 2013.
13- N. Courtial, A. Simon, E. Donal, M. Lederlin, and M. Garreau, "Cardiac Cine-MRI/CT Registration for Interventions Planning," in 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019), 2019, pp. 776-779.
14- J. von Spiczak, R. Manka, A. Gotschy, S. Oebel, S. Kozerke, S. Hamada, et al., "Fusion of CT coronary angiography and whole-heart dynamic 3D cardiac MR perfusion: building a framework for comprehensive cardiac imaging," Int J Cardiovasc Imaging, vol. 34, pp. 649-660, Apr 2018.
15- B. Sturm, K. A. Powell, A. E. Stillman, and R. D. White, "Registration of 3D CT angiography and cardiac MR images in coronary artery disease patients," The International Journal of Cardiovascular Imaging, vol. 19, pp. 281-293, 2003/08/01 2003.
16- J. Betancur, A. Simon, B. Langella, C. Leclercq, A. Hernandez, and M. Garreau, "Synchronization and Registration of Cine Magnetic Resonance and Dynamic Computed Tomography Images of the Heart," IEEE J Biomed Health Inform, vol. 20, pp. 1369-76, Sep 2016.
17- G. Kiss, A. Thorstensen, B. Amundsen, P. Claus, J. D. hooge, and H. Torp, "Fusion of 3D echocardiographic and cardiac magnetic resonance volumes," in 2012 IEEE International Ultrasonics Symposium, 2012, pp. 126-129.
18- F. Tavard, A. Simon, C. Leclercq, E. Donal, A. I. Hernández, and M. Garreau, "Multimodal registration and data fusion for cardiac resynchronization therapy optimization," IEEE Trans Med Imaging, vol. 33, pp. 1363-72, Jun 2014.
19- F. Anowar, S. Sadaoui, and B. Selim, "Conceptual and empirical comparison of dimensionality reduction algorithms (PCA, KPCA, LDA, MDS, SVD, LLE, ISOMAP, LE, ICA, t-SNE)," Computer Science Review, vol. 40, p. 100378, 2021/05/01/ 2021.
20- R. Pless and R. Souvenir, "A Survey of Manifold Learning for Images," IPSJ Transactions on Computer Vision and Applications, vol. 1, pp. 83-94, 2009.
21- A. Pournemat, P. Adibi, and J. Chanussot, "Semisupervised charting for spectral multimodal manifold learning and alignment," Pattern Recognition, vol. 111, p. 107645, 2021/03/01/ 2021.
22- P. Gifani, H. Behnam, A. Shalbaf, and Z. A. Sani, "Automatic detection of end-diastole and end-systole from echocardiography images using manifold learning," Physiol Meas, vol. 31, pp. 1091-103, Sep 2010.
23- L. P. Nijhawan, M. D. Janodia, B. S. Muddukrishna, K. M. Bhat, K. L. Bairy, N. Udupa, et al., "Informed consent: Issues and challenges," Journal of advanced pharmaceutical technology & research, vol. 4, pp. 134-140, 2013.
24- P. Mallia, "WASP (Write a Scientific Paper): Informed consent in research," Early Hum Dev, vol. 124, pp. 54-57, Sep 2018.
25- P. Orzechowski, F. Magiera, and J. H. Moore, "Benchmarking Manifold Learning Methods on a Large Collection of Datasets," in Genetic Programming, Cham, 2020, pp. 135-150.
26- S. T. Roweis and L. K. Saul, "Nonlinear Dimensionality Reduction by Locally Linear Embedding," Science, vol. 290, p. 2323, 2000.
27- L. K. Saul and S. T. Roweis, "Think globally, fit locally: unsupervised learning of low dimensional manifolds," J. Mach. Learn. Res., vol. 4, pp. 119-155, 2003.
28- N. Thorstensen, "Manifold learning and applications to shape and image processing," Ph.D., Mathematics,Computer Science, 2009.
29- Z. Alizadeh Sani, A. Shalbaf, H. Behnam, and R. Shalbaf, "Automatic computation of left ventricular volume changes over a cardiac cycle from echocardiography images by nonlinear dimensionality reduction," J Digit Imaging, vol. 28, pp. 91-8, Feb 2015.
30- F. Maffessanti, K. Addetia, G. Murtagh, L. Weinert, A. R. Patel, R. M. Lang, et al., "Fusion imaging of computed tomography and 3D echocardiography: Combined assessment of coronary anatomy and myocardial function," in Computing in Cardiology 2014, 2014, pp. 701-704.
31- A. Khalil, Y. M. Liew, S. C. Ng, K. W. Lai, and Y. C. Hum, "Echocardiography to cardiac CT image registration: Spatial and temporal registration of the 2D planar echocardiography images with cardiac CT volume," in 2016 IEEE 18th International Conference on e-Health Networking, Applications and Services (Healthcom), pp. 1-5, 2016.
32- J. J. Peoples, G. Bisleri, and R. E. Ellis, "Deformable multimodal registration for navigation in beating-heart cardiac surgery," International Journal of Computer Assisted Radiology and Surgery, vol. 14, pp. 955-966, 2019/06/01 2019.
33- A. Atehortúa, M. Garreau, A. Simon, E. Donal, M. Lederlin, and E. Romero, "Fusion of 3D real-time echocardiography and cine MRI using a saliency analysis," International Journal of Computer Assisted Radiology and Surgery, vol. 15, pp. 277-285, 2020/02/01 2020.
2- Q. Zhang, A. Samani, and T. M. Peters, "MR and ultrasound cardiac image dynamic visualization and synchronization over Internet for distributed heart function diagnosis," Comput Med Imaging Graph, vol. 88, p. 101850, Mar 2021.
3- J. Jose, N. Gautam, M. Tiwari, T. Tiwari, A. Suresh, V. Sundararaj, et al., "An image quality enhancement scheme employing adolescent identity search algorithm in the NSST domain for multimodal medical image fusion," Biomedical Signal Processing and Control, vol. 66, p. 102480, 2021/04/01/ 2021.
4- M.-A. Azam, K.-B. Khan, M. Ahmad, and M. Mazzara, "Multimodal Medical Image Registration and Fusion for Quality Enhancement," Computers, Materials \& Continua, vol. 68, pp. 821-840, 2021.
5- J. Grondin, D. Wang, C. S. Grubb, N. Trayanova, and E. E. Konofagou, "4D cardiac electromechanical activation imaging," Computers in Biology and Medicine, vol. 113, p. 103382, 2019/10/01/ 2019.
6- T. Tada, K. Osuda, T. Nakata, I. Muranaka, M. Himeno, S. Muratsubaki, et al., "A novel approach to the selection of an appropriate pacing position for optimal cardiac resynchronization therapy using CT coronary venography and myocardial perfusion imaging: FIVE STaR method (fusion image using CT coronary venography and perfusion SPECT applied for cardiac resynchronization therapy)," Journal of Nuclear Cardiology, 2019/08/21 2019.
7- S. Zandieh, R. Bernt, S. Mirzaei, J. Haller, and K. Hergan, "Image fusion between 18F-FDG PET and MRI in cardiac sarcoidosis: A case series," Journal of Nuclear Cardiology, vol. 25, pp. 1128-1134, 2018/08/01 2018.
8- C. Nobre, M. Oliveira-Santos, L. Paiva, M. Costa, and L. Gonçalves, "Fusion imaging in interventional cardiology," Revista Portuguesa de Cardiologia, vol. 39, pp. 463-473, 2020/08/01/ 2020.
9- A. Peters, A. Motiwala, B. O'Neill, and P. Patil, "Novel use of fused cardiac computed tomography and transesophageal echocardiography for left atrial appendage closure," Catheter Cardiovasc Interv, Mar 9 2020.
10- Y. Takaya and H. Ito, "New horizon of fusion imaging using echocardiography: its progress in the diagnosis and treatment of cardiovascular disease," Journal of Echocardiography, vol. 18, pp. 9-15, 2020/03/01 2020.
11- A. Khalil, A. Faisal, K. W. Lai, S. C. Ng, and Y. M. Liew, "2D to 3D fusion of echocardiography and cardiac CT for TAVR and TAVI image guidance," Med Biol Eng Comput, vol. 55, pp. 1317-1326, Aug 2017.
12- Z. Luo, J. Cai, T. M. Peters, and L. Gu, "Intra-Operative 2-D Ultrasound and Dynamic 3-D Aortic Model Registration for Magnetic Navigation of Transcatheter Aortic Valve Implantation," IEEE Transactions on Medical Imaging, vol. 32, pp. 2152-2165, 2013.
13- N. Courtial, A. Simon, E. Donal, M. Lederlin, and M. Garreau, "Cardiac Cine-MRI/CT Registration for Interventions Planning," in 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019), 2019, pp. 776-779.
14- J. von Spiczak, R. Manka, A. Gotschy, S. Oebel, S. Kozerke, S. Hamada, et al., "Fusion of CT coronary angiography and whole-heart dynamic 3D cardiac MR perfusion: building a framework for comprehensive cardiac imaging," Int J Cardiovasc Imaging, vol. 34, pp. 649-660, Apr 2018.
15- B. Sturm, K. A. Powell, A. E. Stillman, and R. D. White, "Registration of 3D CT angiography and cardiac MR images in coronary artery disease patients," The International Journal of Cardiovascular Imaging, vol. 19, pp. 281-293, 2003/08/01 2003.
16- J. Betancur, A. Simon, B. Langella, C. Leclercq, A. Hernandez, and M. Garreau, "Synchronization and Registration of Cine Magnetic Resonance and Dynamic Computed Tomography Images of the Heart," IEEE J Biomed Health Inform, vol. 20, pp. 1369-76, Sep 2016.
17- G. Kiss, A. Thorstensen, B. Amundsen, P. Claus, J. D. hooge, and H. Torp, "Fusion of 3D echocardiographic and cardiac magnetic resonance volumes," in 2012 IEEE International Ultrasonics Symposium, 2012, pp. 126-129.
18- F. Tavard, A. Simon, C. Leclercq, E. Donal, A. I. Hernández, and M. Garreau, "Multimodal registration and data fusion for cardiac resynchronization therapy optimization," IEEE Trans Med Imaging, vol. 33, pp. 1363-72, Jun 2014.
19- F. Anowar, S. Sadaoui, and B. Selim, "Conceptual and empirical comparison of dimensionality reduction algorithms (PCA, KPCA, LDA, MDS, SVD, LLE, ISOMAP, LE, ICA, t-SNE)," Computer Science Review, vol. 40, p. 100378, 2021/05/01/ 2021.
20- R. Pless and R. Souvenir, "A Survey of Manifold Learning for Images," IPSJ Transactions on Computer Vision and Applications, vol. 1, pp. 83-94, 2009.
21- A. Pournemat, P. Adibi, and J. Chanussot, "Semisupervised charting for spectral multimodal manifold learning and alignment," Pattern Recognition, vol. 111, p. 107645, 2021/03/01/ 2021.
22- P. Gifani, H. Behnam, A. Shalbaf, and Z. A. Sani, "Automatic detection of end-diastole and end-systole from echocardiography images using manifold learning," Physiol Meas, vol. 31, pp. 1091-103, Sep 2010.
23- L. P. Nijhawan, M. D. Janodia, B. S. Muddukrishna, K. M. Bhat, K. L. Bairy, N. Udupa, et al., "Informed consent: Issues and challenges," Journal of advanced pharmaceutical technology & research, vol. 4, pp. 134-140, 2013.
24- P. Mallia, "WASP (Write a Scientific Paper): Informed consent in research," Early Hum Dev, vol. 124, pp. 54-57, Sep 2018.
25- P. Orzechowski, F. Magiera, and J. H. Moore, "Benchmarking Manifold Learning Methods on a Large Collection of Datasets," in Genetic Programming, Cham, 2020, pp. 135-150.
26- S. T. Roweis and L. K. Saul, "Nonlinear Dimensionality Reduction by Locally Linear Embedding," Science, vol. 290, p. 2323, 2000.
27- L. K. Saul and S. T. Roweis, "Think globally, fit locally: unsupervised learning of low dimensional manifolds," J. Mach. Learn. Res., vol. 4, pp. 119-155, 2003.
28- N. Thorstensen, "Manifold learning and applications to shape and image processing," Ph.D., Mathematics,Computer Science, 2009.
29- Z. Alizadeh Sani, A. Shalbaf, H. Behnam, and R. Shalbaf, "Automatic computation of left ventricular volume changes over a cardiac cycle from echocardiography images by nonlinear dimensionality reduction," J Digit Imaging, vol. 28, pp. 91-8, Feb 2015.
30- F. Maffessanti, K. Addetia, G. Murtagh, L. Weinert, A. R. Patel, R. M. Lang, et al., "Fusion imaging of computed tomography and 3D echocardiography: Combined assessment of coronary anatomy and myocardial function," in Computing in Cardiology 2014, 2014, pp. 701-704.
31- A. Khalil, Y. M. Liew, S. C. Ng, K. W. Lai, and Y. C. Hum, "Echocardiography to cardiac CT image registration: Spatial and temporal registration of the 2D planar echocardiography images with cardiac CT volume," in 2016 IEEE 18th International Conference on e-Health Networking, Applications and Services (Healthcom), pp. 1-5, 2016.
32- J. J. Peoples, G. Bisleri, and R. E. Ellis, "Deformable multimodal registration for navigation in beating-heart cardiac surgery," International Journal of Computer Assisted Radiology and Surgery, vol. 14, pp. 955-966, 2019/06/01 2019.
33- A. Atehortúa, M. Garreau, A. Simon, E. Donal, M. Lederlin, and E. Romero, "Fusion of 3D real-time echocardiography and cine MRI using a saliency analysis," International Journal of Computer Assisted Radiology and Surgery, vol. 15, pp. 277-285, 2020/02/01 2020.
| Files | ||
| Issue | Vol 8 No 4 (2021) | |
| Section | Original Article(s) | |
| DOI | https://doi.org/10.18502/fbt.v8i4.7757 | |
| Keywords | ||
| Multimodal Temporal Registration Manifold Learning Algorithm Locally Linear Embedding Nonlinear Dimension Reduction | ||
| Rights and permissions | |
|
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License. |
How to Cite
1.
Ghodsizad T, Behnam H, Fatemizadeh E, Faghihi Langroudi T, Bayat F. Temporal Registration of Cardiac Multimodal Images Using Locally Linear Embedding Algorithm. Frontiers Biomed Technol. 2021;8(4):292-303.
