|
We present a deep learning-based framework for super-resolution image transformations across multiple fluorescence microscopy modalities. By training a neural network using a generative adversarial network (GAN), a single low-resolution image is transformed into a high-resolution image that surpasses the diffraction limit. The deep network’s output also demonstrates improved signal-to-noise ratio and extended depth-of-field. This framework is solely data-driven which means that it does not rely on any physical models of the imaging formation process, and instead learns a statistical transformation from the training image datasets. The inference process is non-iterative and does not require sweeping over parameters to achieve optimal results, in contrast to state-of-the-art deconvolution methods. The success of this framework is demonstrated by super-resolving wide-field images captured with low-numerical aperture objective-lenses to match the resolution of images captured with high-numerical aperture objectives. In another example, we demonstrate the transformation of confocal microscopy images into images that match the performance of stimulated emission depletion (STED) microscopy, by super-resolving the distributions of Histone 3 sites within cell nuclei. We also applied this framework to total-internal-reflection fluorescence (TIRF) microscopy and super-resolved TIRF images to match the resolution of TIRF-based structured illumination microscopy (TIRF-SIM). Our super-resolved TIRF images/movies reveal endocytic protein dynamics in SUM159 cells and amnioserosa tissues of a Drosophila embryo, providing a very good match to TIRF-SIM images/movies of the same samples. Our experimental results demonstrate that the presented data-driven super resolution approach generalizes to new types of images and super-resolves objects that were not present in the training stage.
|
