Novel Neural Style Transfer based data synthesis method for phase-contrast wound healing assay images

Abstract

Recent advancements in the field of image synthesis have led to the development of Neural Style Transfer (NST) and Generative Adversarial Networks (GANs) which have proven to be powerful tools for data augmentation and realistic data generation. While GANs have been widely used for both data augmentation and generation, NST has not been employed for data generation tasks. Nonetheless, the simpler structure of NST compared to GANs makes it a promising alternative. In this research, we introduce an NST-based method for data generation, which to the best of our knowledge, is the first of its kind. By taking advantage of simplified architecture of NST models attributed to the utilization of a real image as the style input, our method enhances performance in data generation tasks under limited resource conditions. Additionally by utilizing patch-based training and high-resolution inference process high quality images are synthesized with limited resources. Furthermore multi-model and noised input is utilized for increased diversity with the novel NST-based data generation approach. Our proposed method utilizes binary segmentation maps as the condition input, representing the cell and wound regions. We evaluate the performance of our proposed NST-based method and compare it with a modified and fine-tuned conditional GAN (C-GAN) methods for the purpose of conditional generation of phase-contrast wound healing assay images. Through a series of quantitative and qualitative analyses, we demonstrate that our NST-based method outperforms the modified C-GAN while utilizing fewer resources. Additionally, we show that our NST-based method enhances segmentation performance when used as a data augmentation method. Our findings provide compelling evidence regarding the potential of NST for data generation tasks and its superiority over traditional GAN-based methods. The NST for data generation method was implemented in Python language and will be accessible at https://github.com/IDU-CVLab/NST_for_Gen under the MIT licence. © 2024 Elsevier Ltd