UCGNet: GAN for Ultrasound Beamforming through Capsule Layers from Single-Plane Wave RF Data
Abstract
Purpose: This study aims to implement Capsule Networks for ultrasound beamforming and image reconstruction, addressing the limitations of conventional Convolutional Neural Networks (CNNs) in embedded systems. The goal is to enhance image quality from single-plane wave transmission using fewer parameters while maintaining diagnostic accuracy.
Materials and Methods: We propose a novel image reconstruction architecture, UCGNet (U-Caps-GAN Network), which integrates Capsule Networks (U-Caps) within a Generative Adversarial Network (GAN) framework. The method is applied to reconstruct high-quality ultrasound images from single-plane wave data and is evaluated using the Plane-wave Imaging Challenge in Medical Ultrasound (PICMUS) dataset.
Results: The reconstructed images achieved a mean signal-to-noise ratio (SNR) of 18.4383 and a peak signal-to-noise ratio (PSNR) of 41.0226, outperforming the baseline UNet model in terms of accuracy. Moreover, UCGNet used less than 25% of the training parameters compared to UNet.
Conclusion: UCGNet provides an effective and lightweight solution for ultrasound image reconstruction. Its improved accuracy and reduced parameter count make it well-suited for practical medical imaging applications, particularly in resource-constrained environments.
1- V. Kumar et al., "Automated and real-time segmentation of suspicious breast masses using convolutional neural network." PLOS One, Vol. 13 (No. 5), p. e0195816, (2018).
2- G. Pons, J. Martí, R. Martí, S. Ganau, and J. A. Noble, "Breast lesion segmentation combining b-mode and elastography ultrasound." Ultrasonic Imaging, Vol. 38 (No. 3), pp. 209-24, (2016).
3- R. Mebarki, A. Krupa, and F. Chaumette, "2-D ultrasound probe complete guidance by visual servoing using image moments." IEEE Transactions on Robotics, Vol. 26 (No. 2), pp. 296-306, (2010).
4- R. Wang, "High-resolution image reconstruction for portable ultrasound imaging devices." EURASIP Journal on Advances in Signal Processing, (2019).
5- Y. Lyu, Y. Shen, M. Zhang, and J. Wang, "Real-time 3D ultrasound imaging system based on a hybrid reconstruction algorithm." Chinese Journal of Electronics, Vol. 33 (No. 1), pp. 245-55, (2024).
6- H. Yoon, J. Kim, K. Lee, and T. K. Song, "Design and Implementation of Analog-Digital Hybrid Beamformers for Low-Complexity Ultrasound Systems: A Feasibility Study." Bioengineering, Vol. 11 (No. 1), p. 8, (2023).
7- Z. Li, A. Wiacek, and M. L. Bell, "Beamforming with deep learning from single plane wave RF data." Presented at the Proc. IEEE International Ultrasonics Symposium (IUS), (2020).
8- M. A. Lediju, G. E. Trahey, B. C. Byram, and J. J. Dahl, "Shortlag spatial coherence of backscattered echoes: Imaging characteristics." IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control, Vol. 58 (No. 7), pp. 1377-88, (2011).
9- S. Goudarzi and H. Rivaz, "Plane-Wave Ultrasound Beamforming: A Deep Learning Approach." arXiv, (2021).
10- A. K. Tehrani and H. Rivaz, "Displacement estimation in ultrasound elastography using pyramidal convolutional neural network." IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control, (2020).
11- M. Amiri, "Fine-tuning UNet for ultrasound image segmentation: different layers, different outcomes." IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control, (2020).
12- M. Azimi, A. Gharekhani, and S. Ebadollahi, "An Integration of a Neural-Network-Based Computer Vision Model and a 2-DOF Object Tracker Robot." Presented at the 2023 11th RSI International Conference on Robotics and Mechatronics (ICRoM), (2023).
13- R. Delaunay, "An unsupervised learning approach to ultrasound strain elastography with spatio-temporal consistency." Physics in Medicine & Biology, (2021).
14- S. Jeon and C. Kim, "Deep learning-based speed of sound aberration correction in photoacoustic images." Presented at the Photons Plus Ultrasound: Imaging and Sensing 2020, (2020).
15- D. Karimi, "Accurate and robust deep learning-based segmentation of the prostate clinical target volume in ultrasound images." Medical Image Analysis, Vol. 57pp. 186-96, (2019).
16- M. A. Samani, K. Hooshanfar, H. S. Jey, and S. M. Esmailzadeh, "Eye-Tracking Based Control of a Robotic Arm and Wheelchair for People with Severe Speech and Motor Impairment (SSMI)." Presented at the 2023 11th RSI International Conference on Robotics and Mechatronics (ICRoM), (2023).
17- M. Sharifzadeh, "Phase aberration correction: A convolutional neural network approach." IEEE Access, Vol. 8p. 112400, (2020).
18- K. Song, J. Feng, and D. Chen, "A survey on deep learning in medical ultrasound imaging." Frontiers in Physics, Vol. 12p. 1398393, (2024).
19- A. K. Tehrani, "Semisupervised training of optical flow convolutional neural networks in ultrasound elastography." Presented at the International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI), (2020).
20- A. K. Tehrani, "Mpwc-net++: evolution of optical flow pyramidal convolutional neural network for ultrasound elastography." Presented at the Medical Imaging 2021: Ultrasonic Imaging and Tomography, (2021).
21- G. Madhu et al., "UCapsNet: A two-stage deep learning model using U-Net and Capsule Network for breast cancer segmentation and classification in ultrasound imaging." Cancers, Vol. 16 (No. 22), p. 3777, (2024).
22- S. Sabour, N. Frosst, and G. E. Hinton, "Dynamic routing between capsules." Presented at the Advances in Neural Information Processing Systems (NeurIPS), (2017).
23- S. Survarachakan, "Capsule Nets for Complex Medical Image Segmentation Tasks." figshare, (2020).
24- M. Tran, "CapsNet for Medical Image Segmentation." arXiv, (2022).
25- A. A. Nair, "Deep learning to obtain simultaneous image and segmentation outputs from a single input of raw ultrasound channel data." IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control, (2020).
26- D. Hyun, "Beamforming and speckle reduction using neural networks." IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control, Vol. 66 (No. 5), pp. 898-910, (2019).
27- E. Dahan and I. Cohen, "Deep-learning-based multitask ultrasound beamforming." Information, Vol. 14 (No. 10), p. 582, (2023).
28- A. A. Nair, T. D. Tran, A. Reiter, and M. A. Bell, "A generative adversarial neural network for beamforming ultrasound images: Invited presentation." Presented at the 2019 53rd Annual Conference on Information Sciences and Systems (CISS), (2019).
29- S. W. Penninga, H. van Gorp, and R. J. van Sloun, "Deep Sylvester posterior inference for adaptive compressed sensing in ultrasound imaging." Presented at the ICASSP 2025 - IEEE International Conference on Acoustics, Speech and Signal Processing, (2025).
30- H. Liu, "Attention residual network for medical ultrasound image segmentation." Scientific Reports, Vol. 15 (No. 1), p. 22155, (2025).
31- M. Vafaeezadeh, H. Behnam, and P. Gifani, "Ultrasound image analysis with vision transformers." Diagnostics, Vol. 14 (No. 5), p. 542, (2024).
32- K. Xia and J. Wang, "Recent advances of transformers in medical image analysis: a comprehensive review." MedComm–Future Medicine, Vol. 2 (No. 1), p. e38, (2023).
33- B. Behboodi and H. Rivaz, "Ultrasound segmentation using U-Net: learning from simulated data and testing on real data." Presented at the 2019 41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), (2019).
34- H. Rasaee, M. Samuel, B. Behboodi, J. Afilalo, and H. Rivaz, "Ultrasound segmentation using semi-supervised learning: Application in point-of-care sarcopenia assessment." IEEE Open Journal of Engineering in Medicine and Biology, (2025).
35- V. Akoto-Adjepong, O. Appiah, P. K. Mensah, and P. Appiahene, "TTDCapsNet: Tri Texton-Dense Capsule Network for complex and medical image recognition." PLOS One, Vol. 19 (No. 3), p. e0300133, (2024).
36- S. Bonheur, D. Štern, C. Payer, M. Pienn, H. Olschewski, and M. Urschler, "Matwo-capsnet: a multi-label semantic segmentation capsules network." Presented at the International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI), (2019).
37- J. Deepika, C. Rajan, and T. Senthil, "Improved CAPSNET model with modified loss function for medical image classification." Signal, Image and Video Processing, Vol. 16 (No. 8), pp. 2269-77, (2022).
38- R. LaLonde and U. Bagci, "Capsules for object segmentation." arXiv, (2018).
39- R. LaLonde, Z. Xu, I. Irmakci, S. Jain, and U. Bagci, "Capsules for biomedical image segmentation." Medical Image Analysis, Vol. 68p. 101889, (2021).
40- F. Long, J. J. Peng, W. Song, X. Xia, and J. Sang, "BloodCaps: A capsule network based model for the multiclassification of human peripheral blood cells." Computer Methods and Programs in Biomedicine, Vol. 202p. 105972, (2021).
41- A. F. M. Saif, T. Imtiaz, S. Rifat, C. Shahnaz, W. P. Zhu, and M. O. Ahmad, "CapsCovNet: A modified capsule network to diagnose Covid-19 from multimodal medical imaging." IEEE Transactions on Artificial Intelligence, Vol. 2 (No. 6), pp. 608-17, (2021).
42- O. Ronneberger, P. Fischer, and T. Brox, "U-Net: Convolutional Networks for Biomedical Image Segmentation." Presented at the Medical Image Computing and Computer-Assisted Intervention (MICCAI), (2015).
43- I. Goodfellow et al., "Generative adversarial nets." Presented at the Advances in Neural Information Processing Systems (NeurIPS), (2014).
44- T. Karras, S. Laine, and T. Aila, "A style-based generator architecture for generative adversarial networks." Presented at the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), (2019).
45- J. Y. Zhu, T. Park, P. Isola, and A. A. Efros, "Unpaired image-to-image translation using cycle-consistent adversarial networks." Presented at the IEEE International Conference on Computer Vision (ICCV), (2017).
46- S. Ioffe and C. Szegedy, "Batch normalization: Accelerating deep network training by reducing internal covariate shift." Presented at the International Conference on Machine Learning (ICML), (2015).
47- C. Fan, H. Lin, and Y. Qiu, "U-Patch GAN: A medical image fusion method based on GAN." Journal of Digital Imaging, Vol. 36 (No. 1), pp. 339-55, (2023).
48- H. Liebgott, "Plane-Wave Imaging Challenge in Medical Ultrasound." Presented at the IEEE International Ultrasonics Symposium (IUS), (2016).
2- G. Pons, J. Martí, R. Martí, S. Ganau, and J. A. Noble, "Breast lesion segmentation combining b-mode and elastography ultrasound." Ultrasonic Imaging, Vol. 38 (No. 3), pp. 209-24, (2016).
3- R. Mebarki, A. Krupa, and F. Chaumette, "2-D ultrasound probe complete guidance by visual servoing using image moments." IEEE Transactions on Robotics, Vol. 26 (No. 2), pp. 296-306, (2010).
4- R. Wang, "High-resolution image reconstruction for portable ultrasound imaging devices." EURASIP Journal on Advances in Signal Processing, (2019).
5- Y. Lyu, Y. Shen, M. Zhang, and J. Wang, "Real-time 3D ultrasound imaging system based on a hybrid reconstruction algorithm." Chinese Journal of Electronics, Vol. 33 (No. 1), pp. 245-55, (2024).
6- H. Yoon, J. Kim, K. Lee, and T. K. Song, "Design and Implementation of Analog-Digital Hybrid Beamformers for Low-Complexity Ultrasound Systems: A Feasibility Study." Bioengineering, Vol. 11 (No. 1), p. 8, (2023).
7- Z. Li, A. Wiacek, and M. L. Bell, "Beamforming with deep learning from single plane wave RF data." Presented at the Proc. IEEE International Ultrasonics Symposium (IUS), (2020).
8- M. A. Lediju, G. E. Trahey, B. C. Byram, and J. J. Dahl, "Shortlag spatial coherence of backscattered echoes: Imaging characteristics." IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control, Vol. 58 (No. 7), pp. 1377-88, (2011).
9- S. Goudarzi and H. Rivaz, "Plane-Wave Ultrasound Beamforming: A Deep Learning Approach." arXiv, (2021).
10- A. K. Tehrani and H. Rivaz, "Displacement estimation in ultrasound elastography using pyramidal convolutional neural network." IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control, (2020).
11- M. Amiri, "Fine-tuning UNet for ultrasound image segmentation: different layers, different outcomes." IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control, (2020).
12- M. Azimi, A. Gharekhani, and S. Ebadollahi, "An Integration of a Neural-Network-Based Computer Vision Model and a 2-DOF Object Tracker Robot." Presented at the 2023 11th RSI International Conference on Robotics and Mechatronics (ICRoM), (2023).
13- R. Delaunay, "An unsupervised learning approach to ultrasound strain elastography with spatio-temporal consistency." Physics in Medicine & Biology, (2021).
14- S. Jeon and C. Kim, "Deep learning-based speed of sound aberration correction in photoacoustic images." Presented at the Photons Plus Ultrasound: Imaging and Sensing 2020, (2020).
15- D. Karimi, "Accurate and robust deep learning-based segmentation of the prostate clinical target volume in ultrasound images." Medical Image Analysis, Vol. 57pp. 186-96, (2019).
16- M. A. Samani, K. Hooshanfar, H. S. Jey, and S. M. Esmailzadeh, "Eye-Tracking Based Control of a Robotic Arm and Wheelchair for People with Severe Speech and Motor Impairment (SSMI)." Presented at the 2023 11th RSI International Conference on Robotics and Mechatronics (ICRoM), (2023).
17- M. Sharifzadeh, "Phase aberration correction: A convolutional neural network approach." IEEE Access, Vol. 8p. 112400, (2020).
18- K. Song, J. Feng, and D. Chen, "A survey on deep learning in medical ultrasound imaging." Frontiers in Physics, Vol. 12p. 1398393, (2024).
19- A. K. Tehrani, "Semisupervised training of optical flow convolutional neural networks in ultrasound elastography." Presented at the International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI), (2020).
20- A. K. Tehrani, "Mpwc-net++: evolution of optical flow pyramidal convolutional neural network for ultrasound elastography." Presented at the Medical Imaging 2021: Ultrasonic Imaging and Tomography, (2021).
21- G. Madhu et al., "UCapsNet: A two-stage deep learning model using U-Net and Capsule Network for breast cancer segmentation and classification in ultrasound imaging." Cancers, Vol. 16 (No. 22), p. 3777, (2024).
22- S. Sabour, N. Frosst, and G. E. Hinton, "Dynamic routing between capsules." Presented at the Advances in Neural Information Processing Systems (NeurIPS), (2017).
23- S. Survarachakan, "Capsule Nets for Complex Medical Image Segmentation Tasks." figshare, (2020).
24- M. Tran, "CapsNet for Medical Image Segmentation." arXiv, (2022).
25- A. A. Nair, "Deep learning to obtain simultaneous image and segmentation outputs from a single input of raw ultrasound channel data." IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control, (2020).
26- D. Hyun, "Beamforming and speckle reduction using neural networks." IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control, Vol. 66 (No. 5), pp. 898-910, (2019).
27- E. Dahan and I. Cohen, "Deep-learning-based multitask ultrasound beamforming." Information, Vol. 14 (No. 10), p. 582, (2023).
28- A. A. Nair, T. D. Tran, A. Reiter, and M. A. Bell, "A generative adversarial neural network for beamforming ultrasound images: Invited presentation." Presented at the 2019 53rd Annual Conference on Information Sciences and Systems (CISS), (2019).
29- S. W. Penninga, H. van Gorp, and R. J. van Sloun, "Deep Sylvester posterior inference for adaptive compressed sensing in ultrasound imaging." Presented at the ICASSP 2025 - IEEE International Conference on Acoustics, Speech and Signal Processing, (2025).
30- H. Liu, "Attention residual network for medical ultrasound image segmentation." Scientific Reports, Vol. 15 (No. 1), p. 22155, (2025).
31- M. Vafaeezadeh, H. Behnam, and P. Gifani, "Ultrasound image analysis with vision transformers." Diagnostics, Vol. 14 (No. 5), p. 542, (2024).
32- K. Xia and J. Wang, "Recent advances of transformers in medical image analysis: a comprehensive review." MedComm–Future Medicine, Vol. 2 (No. 1), p. e38, (2023).
33- B. Behboodi and H. Rivaz, "Ultrasound segmentation using U-Net: learning from simulated data and testing on real data." Presented at the 2019 41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), (2019).
34- H. Rasaee, M. Samuel, B. Behboodi, J. Afilalo, and H. Rivaz, "Ultrasound segmentation using semi-supervised learning: Application in point-of-care sarcopenia assessment." IEEE Open Journal of Engineering in Medicine and Biology, (2025).
35- V. Akoto-Adjepong, O. Appiah, P. K. Mensah, and P. Appiahene, "TTDCapsNet: Tri Texton-Dense Capsule Network for complex and medical image recognition." PLOS One, Vol. 19 (No. 3), p. e0300133, (2024).
36- S. Bonheur, D. Štern, C. Payer, M. Pienn, H. Olschewski, and M. Urschler, "Matwo-capsnet: a multi-label semantic segmentation capsules network." Presented at the International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI), (2019).
37- J. Deepika, C. Rajan, and T. Senthil, "Improved CAPSNET model with modified loss function for medical image classification." Signal, Image and Video Processing, Vol. 16 (No. 8), pp. 2269-77, (2022).
38- R. LaLonde and U. Bagci, "Capsules for object segmentation." arXiv, (2018).
39- R. LaLonde, Z. Xu, I. Irmakci, S. Jain, and U. Bagci, "Capsules for biomedical image segmentation." Medical Image Analysis, Vol. 68p. 101889, (2021).
40- F. Long, J. J. Peng, W. Song, X. Xia, and J. Sang, "BloodCaps: A capsule network based model for the multiclassification of human peripheral blood cells." Computer Methods and Programs in Biomedicine, Vol. 202p. 105972, (2021).
41- A. F. M. Saif, T. Imtiaz, S. Rifat, C. Shahnaz, W. P. Zhu, and M. O. Ahmad, "CapsCovNet: A modified capsule network to diagnose Covid-19 from multimodal medical imaging." IEEE Transactions on Artificial Intelligence, Vol. 2 (No. 6), pp. 608-17, (2021).
42- O. Ronneberger, P. Fischer, and T. Brox, "U-Net: Convolutional Networks for Biomedical Image Segmentation." Presented at the Medical Image Computing and Computer-Assisted Intervention (MICCAI), (2015).
43- I. Goodfellow et al., "Generative adversarial nets." Presented at the Advances in Neural Information Processing Systems (NeurIPS), (2014).
44- T. Karras, S. Laine, and T. Aila, "A style-based generator architecture for generative adversarial networks." Presented at the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), (2019).
45- J. Y. Zhu, T. Park, P. Isola, and A. A. Efros, "Unpaired image-to-image translation using cycle-consistent adversarial networks." Presented at the IEEE International Conference on Computer Vision (ICCV), (2017).
46- S. Ioffe and C. Szegedy, "Batch normalization: Accelerating deep network training by reducing internal covariate shift." Presented at the International Conference on Machine Learning (ICML), (2015).
47- C. Fan, H. Lin, and Y. Qiu, "U-Patch GAN: A medical image fusion method based on GAN." Journal of Digital Imaging, Vol. 36 (No. 1), pp. 339-55, (2023).
48- H. Liebgott, "Plane-Wave Imaging Challenge in Medical Ultrasound." Presented at the IEEE International Ultrasonics Symposium (IUS), (2016).
| Files | ||
| Issue | Vol 13 No 2 (2026) | |
| Section | Original Article(s) | |
| DOI | https://doi.org/10.18502/fbt.v13i2.21930 | |
| Keywords | ||
| Capsule Network Generative Adversarial Network Beamforming Medical Image Reconstruction Single Plane Wave | ||
| Rights and permissions | |
|
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License. |
How to Cite
1.
Samani M, Gharekhani A, Farnia P, Makki Abadi B. UCGNet: GAN for Ultrasound Beamforming through Capsule Layers from Single-Plane Wave RF Data. Frontiers Biomed Technol. 2026;13(2):317-326.
