O04_03
GAN-Based Multi-Axis Resolution-Enhanced 3D Visualization of Giant Vesicles
Soichiro HIROI *1, 2, Taro TOYOTA1, 3, Akihiko KONAGAYA2
1Department of Basic Science, Graduate School of Arts and Science, The University of Tokyo
2Molecular Robotics Research Institute, Co., Ltd.
3Universal Biology Institute, The University of Tokyo
( * E-mail: hiroi@molecular-robot.com )
In the rapidly evolving field of molecular visualization, accurately representing the 3D morphology of complex structures remains a significant challenge [1]. Giant vesicles (GVs), with their dynamic nature and sensitivity to environmental factors [2], present a particularly intriguing subject for high-resolution imaging and virtual reality (VR) representation.
Previously, we developed a GAN-based model to enhance the clarity of GV microscopy images for VR visualization [3]. Building upon this foundation, our current study introduces substantial improvements in both dataset construction and model architecture to address the persistent issues of image blurring and loss of fine structural details across multiple axes.
In this study, we have reconstructed our synthetic dataset to be more robust and capable of capturing intricate membrane details and vesicle contours in both lateral (XY) and axial (Z) dimensions. This refined dataset serves as a stronger foundation for our deep learning model. Additionally, we have developed an improved generative model with two key enhancements:
1. Implementation of specialized loss functions tailored to visualize vesicle shapes and membrane states more effectively in 3D space.
2. Integration of multi-axis resolution enhancement techniques, including Z-axis interpolation, to more accurately capture and represent the 3D structure of GVs.
[1] Konagaya, A.; Gutmann, G.; Zhang, Y. Co-creation environment with cloud virtual reality and real-time artificial intelligence toward the design of molecular robots. J. Integr. Bioinform. 2022, 20220017.
[2] Lipowsky, R. Remodeling of Membrane Compartments: Some Consequences of Membrane Fluidity. J. Biol. Chem. 2014, 395, 253–274.
[3] Hiroi, S.; Toyota, Y.; Konagaya, A. Deep Learning-Based Deconvolution of Confocal Laser scanning Fluorescence Microscopy Images for Enhanced Visualization of Giant Vesicles, Proceedings of the CBI Annual Meeting 2023, O02-02.