Review ArticleOpen Access

Light field microscopy in biological imaging

School of Optical and Electronic Information-Wuhan National Laboratory for Optoelectronics, Huazhong University of Science and Technology, Wuhan 430074, P. R. China

Search for more papers by this author

Lanxin Zhu

School of Optical and Electronic Information-Wuhan National Laboratory for Optoelectronics, Huazhong University of Science and Technology, Wuhan 430074, P. R. China

Search for more papers by this author

Dongyu Li

School of Optical and Electronic Information-Wuhan National Laboratory for Optoelectronics, Huazhong University of Science and Technology, Wuhan 430074, P. R. China

E-mail Address: li_dongyu@hust.edu.cn

Search for more papers by this author

, and

Peng Fei

https://orcid.org/0000-0003-3764-817X

School of Optical and Electronic Information-Wuhan National Laboratory for Optoelectronics, Huazhong University of Science and Technology, Wuhan 430074, P. R. China

E-mail Address: feipeng@hust.edu.cn

Corresponding author.

Search for more papers by this author

https://doi.org/10.1142/S1793545822300178Cited by:10 (Source: Crossref)

This article is part of the issue:

Special Issue on Celebrating the 15th Anniversary of JIOHS and the 70th Anniversary of HUST
Guest Editors: Qingming Luo, Valery V. Tuchin and Lihong Wang

Abstract

Light field microscopy (LFM), featured for high three-dimensional imaging speed and low phototoxicity, has emerged as a technique of choice for instantaneous volumetric imaging. In contrast with other scanning-based three-dimensional (3D) imaging approaches, LFM enables to encode 3D spatial information in a snapshot manner, permitting high-speed 3D imaging that is only limited by the frame rate of the camera. In this review, we first introduce the fundamental theory of LFM and current corresponding advanced approaches. Then, we summarize various applications of LFM in biological imaging.

Keywords:

1. Introduction

Three-dimensional (3D) in vivo imaging is of great significance to understand dynamic biological process, such as neural activity,^1,2 immune response,³ heart beating,^4,5 blood flow,⁶ etc. Traditional techniques developed for 3D in vivo imaging, such as confocal microscopy and multi-photon microscopy,^7,8 are based on sequential point-scanning, therefore require plenty of time for obtaining a 3D volume of information. In the last decades, to improve the imaging speed, light sheet fluorescence microscopy (LSFM)^9,10,11 has been developed. According to the selective excitation for one plane of sample, LSFM only requires scanning along one direction, providing much higher throughput compared with point-scanning methods and yields image stacks with high axial resolution. However, due to this one-dimensional scanning-acquisition manner, it’s still challengeable for long-term, high-speed volumetric imaging with low photon doses.

To achieve even better temporal resolution and photon-efficiency when capturing complicated biological dynamics in three dimensions, approaches have been proposed to first map the 3D information onto one lateral plane^{12,13,14,15,16} and second to reconstruct the 3D volume by post-processing. Light field microscopy (LFM) is one such high-speed volumetric imaging method for observing fluorescent or nonfluorescent specimens. LFM employs a microlens array (MLA) to modulate the wave front of incident light which enables simultaneous recording of 2D spatial and 2D angular information in a manner of one snapshot, after which 3D structure can be reconstructed using the light-field snapshot. Therefore, this approach provides the most photon-efficient solution by imaging the entire excited volume with one exposure so that it drastically reduces the extra excitation burden that is common in conventional scanning-based approaches (e.g., confocal microscopy). Therefore, it’s suited for long-term observation of living sample with very low phototoxicity. However, due the intrinsic trade-off between the spatial resolution and angular resolution,¹⁷ the performance metrics, including spatial resolution, field of view (FOV) and depth of field of LFM, are trade-offs.

In recent years, on the one hand, advanced optics design and powerful algorithm have been developed^{6,18,19,20,21} to improve LFM’s spatial resolution, FOV and depth of field. On the other hand, with its unprecedent volumetric imaging speed and low-phototoxicity, LFM is well-applied in the field of capturing fast biological dynamics. A few reviews have introduced LFM techniques from the prospective of specific biological applications or optical systems.^22,23 In this review, we aim to introduce the theory of various state-of-the-art LFM implementations comprehensively from their optical designs, to the 3D reconstruction algorithms, especially deep-learning-based reconstruction algorithm, and finally to their biological applications (Table 1). Therefore, it’s expected to substantially help the readers to understand the technical advances and application preferences between different LFM techniques.

**Table 1. A summary of various light filed implementations.**
Hardware configuration	Reconstruction methods	Spatial resolution	Imaging volume and speed	Applications	Reference
Conventional LFM	Richard–Lucy (RL) deconvolution	Lateral: 1.4 $μ$ $μ$ m	$700 \times 700 \times 200$ $700 \times 700 \times 200$ $μ$ $μ$ m³ @50Hz	Neuron imaging	22
		Axial: 2.6 $μ$ $μ$ m
	VCD network	Lateral: 1.0 $μ$ $μ$ m	$250 \times 250 \times 150$ $250 \times 250 \times 150$ $μ$ $μ$ m³ @200Hz	Beating heart imaging	6
		Axial: 2.9 $μ$ $μ$ m
			$300 \times 300 \times 50$ $300 \times 300 \times 50$ $μ$ $μ$ m³ @100Hz	Neuron imaging
Dual view LFM	Muti-view deconvolution	Lateral: 2.0 $μ$ $μ$ m	$200 \times 200 \times n 200$ $200 \times 200 \times n 200$ $μ$ $μ$ m³ @ 200Hz	Blood flow imaging	23
		Axial: 2.9 $μ$ $μ$ m
			$300 \times 300 \times 300$ $300 \times 300 \times 300$ $μ$ $μ$ m³ @143Hz	Beating heart imaging
Scanning LFM	Phase-space deconvolution with digital adaptive optics	Lateral: 0.22 $μ$ $μ$ m	$225 \times 225 \times 300$ $225 \times 225 \times 300$ $μ$ $μ$ m³ @ 12Hz	Living cell imaging	18
		Axial: 0.4 $μ$ $μ$ m
Conventional FLFM	RL-deconvolution with aberration correction	Lateral: 0.3–0.7 $μ$ $μ$ m	$70 \times 70 \times 4$ $70 \times 70 \times 4$ $μ$ $μ$ m³ @ 20Hz	Living cell imaging	19
		Axial: 0.5–1.5 $μ$ $μ$ m
Confocal FLFM	RL-deconvolution	Lateral: 2.1 $μ$ $μ$ m	ø800 $\times$ $\times$ 15 $μ$ $μ$ m³ @70Hz	Neuron imaging	21
		Axial: 2.5 $μ$ $μ$ m
DoF-extended FLFM	RL-deconvolution	Lateral: 3.4 $μ$ $μ$ m	ø800 $\times$ $\times$ 400 $μ$ $μ$ m³ @77Hz	Neuron imaging	24
		Axial: 5 $μ$ $μ$ m
Hybrid light-sheet LFM	HyLFM-Net	Lateral: 1.8 $μ$ $μ$ m	$350 \times 300 \times 150$ $350 \times 300 \times 150$ $μ$ $μ$ m³ @40–100Hz	Beating heart imaging	25
		Axial: 7.1 $μ$ $μ$ m

2. Implementations of LFM

2.1. Light field microscopy

LFM is a volumetric imaging method which employs a microlens array at native image plane to modulate 4D light field of signals¹⁷ [Figs. 1(a) and 2(a)]. Unlike conventional wide-field microscopy, both the 2D location and 2D angular information of the incident light in a 3D space are captured by single 2D LF snapshot, which provides different views of the sample simultaneously. After image post-processing (shearing and sum) applied, a refocused 3D image stack can be obtained from these views¹⁷ [Fig. 1(b)].

Fig. 1. The principle of LFM: (a) Microlens array is placed at intermediate image plane to record directional ( $u, v$ $u, v$ ) information of incident rays from each lenlet ( $s, t$ $s, t$ ), which enables to store the 4D light field information into one snapshot ( $x, y$ $x, y$ ). (b) According to the digital refocusing, a synthetic image stack can be derived. (c) The principle diagram of LFD: This wave-optics-based algorithm adopts Richard–Lucy deconvolution which consists of alternative forward and backward projection.

Fig. 2. Different implementations of LFM and FLFM. (a) Conventional LFM. Inserting an MLA at intermediate image plane enables the detector to capture both 2D spatial information and 2D angular information of 3D sample. (b) Scanning-based LFM mitigates the artifacts and spatial resolution deterioration brought by sub-sampling. (c) Another detection path provides more angular information which contributes to achieving isotropic spatial resolution. (d) Conventional FLFM. Different from LFM, MLA is located at the Fourier plane which enables it to directly capture various views of the sample. (e) DoF-extended FLM. Two different groups of MLAs with different focal lengths are introduced to increase the DoF. (f) Confocal FLFM. This confocal detection scheme ensures selective and efficient signal collection from the in-focus volume which enables to yield 3D reconstruction from low-SNR/SBR LF measurement when imaging thick tissues.

This enables LFM to acquire 3D imaging through a single-snapshot which realizes a faster frame rate compared to the other 3D microscopy. However, in this case, the spatial resolution is limited due to the low sampling rate of microlens array.^17,26 In addition, when observing the microscale object, the diffraction effect must be considered, where refocusing algorithm is not suitable. Deconvolution algorithm [Fig. 1(c)] based on wave optics theory can produce 3D reconstruction with better spatial resolution and fidelity compared with computationally refocusing method.²⁷ This fundamental theory improvement allows the application of LFM in the field of high-speed high-resolution volumetric in vivo imaging, like simultaneous observation of neuronal activities in a certain volume at single-cell resolution.¹⁶ However, because of the limited angular and spatial information in one snapshot, it’s hard for LFM to derive a 3D image that the imaging quality is as high as scanning-based microscopy. Moreover, the nonuniform sampling pattern along the depth of LFM further degrades the spatial resolution and produces grid-like artifacts. The above problems hamper the further application of LFM in biological imaging.

In order to solve the above significant defects of LFM, the current approaches utilize two strategies, those are optical and algorithmic improvements.

The first approach is to mitigate spatial resolution degradation and artefacts in native focus plane via optimizing the optics. For instance, placing a well-designed mask at Fourier plane or image plane can make the point spread function (PSF) of LFM have more diversity of diffraction pattern, which is able to capture minimum translation of sample at focus plane and achieve relatively uniform spatial resolution.²⁸ In addition, the method of subpixel shift could break the limitation of sampling rate, but at the expense of sacrificed temporal resolution.²⁹ Fortunately, this extra scanning time can be drastically reduced by the introduction of a fast galvanometer¹⁸ [Fig. 2(b)]. Despite the lateral resolution enhancement, angular resolution improvement can further produce uniform spatial resolution. Because of the limited views provided by the finite numerical aperture of the objective, the missing cone problem that existed in LFM leads to a poor axial resolvability.³⁰ To address this issue, an additional light-field detection pathway is placed orthogonal to the primary one to obtain dual-view light-field images and achieve isotropic spatial resolution using a multi-view light-field deconvolution algorithm²³ [Fig. 2(c)]. Another approach is to employ a mirror to obtain two orthogonal views in one snapshot in a manner of information aliasing.¹⁷ This approach prevents the involvement of extra detection path through applying symmetry constraints in reconstruction algorithm but at the expense of reduction of FOV.

Furthermore, to overcome the influences caused by uneven sampling rate along the depth, directly introducing customized microlens array with various focal length,¹⁴ multi-focus elements³¹ and wave front modification²⁸ can be used to achieve more uniform spatial resolution.

For densely labeled or opaque sample, scattering-induced fluorescence background and cross-talk of signals from different depth reduce the signal-to-background ratio (SBR) of captured light-field images. This background coupling with extended DOF of LFM greatly deteriorates the fidelity of the reconstructed 3D images. Inspired by light-sheet microscopy, the combination between LFM and selective-volume illumination microscopy (SVIM) greatly reduces the background induced by unnecessary excitation far away from the native focus plane.³² In addition, combined with two-photon excitation illumination, the background brought by scattering can be further reduced due to the less scattering effects of near-infrared beams (NIR). Also, the invisible excitation light reduces the response of the animal’s visual system to the illumination, which is more suitable for living animals imaging.³³

The second strategy begins with the improvement of traditional deconvolution algorithm (light field deconvolution, LFD) and the model of wave propagation. Due to the depth-dependent sampling rate of LFM (especially low at focus plane), employing interpolation and deconvolution for different views instead of raw light field images can reduce the degradation of spatial resolution and the computational burden brought by traditional 5D PSF (three spatial dimensions and two angular dimensions).³⁴ Besides, by analyzing the frequency band of sub-aperture of MLA, depth-dependent anti-aliasing filters can be obtained which are able to remove all the artifacts at focus plane.³⁵ However, owing to the angular-aware feature of LFM, the quality of 3D reconstruction depends on the accuracy of rays’ propagation. Consequently, when applied in thick tissue, LFM would degrade rapidly due to scattering-induced crosstalk. There are several works in the literature specifically aiming to recover the fluctuation signal of spiking neurons in deep tissue, like introducing seeded iterative demixing (SID) algorithm,³⁶ which utilizes the remnant ballistic light and fixed-position of neuron to extract temporal information and spatial information; or introducing multiscale scattering model, which can remove background fluorescence.³⁷ In addition, apart from scattering influence, various noise existed in low-light imaging which will severely degrade the quality of 3D reconstruction via RL deconvolution. By introducing the mixed Poisson and Gaussian noise distribution and dictionary priors from general biological samples, the intrinsic artefacts from low-sampling rate in LFM and noise-induced ringing effects can be mitigated.³⁸ Besides, spatial-temporal low-rank prior of sequential LF data could also enhance the reconstruction performance of LFM in low-light conditions.³⁹

2.2. Fourier light field microscopy

Unlike conventional LFM, Fourier light field microscopy (FLFM) adopts an extra single lens, termed Fourier lens, to process the wavefront at intermediate image plane with Fourier transformation. After propagation, the wavefront is subsequently modulated by microlens array at frequency plane [Fig. 2(d)]. Thus, the direction information of sample can be divided into different regions in an arrangement of microlens array. The spatial information is preserved inside one microlens.⁴⁰ This configuration effectively avoids artifacts due to the redundancy sample²⁷ or angular cross-talk brought by scattering.³⁶ Like conventional LFM, FLFM also plays an important role in observing crucial biological processes, like whole-brain neuron activities^21,24 and intracellular organelles interaction.¹² However, although the configuration of inserting MLA at pupil plane prevents grid-like artifacts and provides a less computational forward model, it splits the numerical aperture into several parts at one sensor plane with limited size. Thus, different performance metrics (like spatial resolution, depth of field (DOF), FOV, etc.) are coupling together, making it hard to find optimal required optical parameters.

To solve the above problem, the variants of FLFM have been proposed to improve its performance. In 2017, Li et al. adopted two groups of microlens array to extend the DOF of the traditional FLFM.²⁴ As illustrated in Fig. 2(e), these two groups have different focal lengths and are located at different axial positions. Through deconvolution with two measured PSF, a DOF-extended volume can be reconstructed. However, considering the fluorescence fluctuations and low SNR away from the focal plane, the measured PSF may cause artifacts. Though there exists an approach⁴⁰ proposing an accurate wave-optics model to provide better reconstruction, it ignores the aberration of imaging system brought by nonideal imaging objective and MLA. Later, a hybrid PSF is proposed to improve the performance of the deconvolution algorithm.¹⁹ By minimizing the spatial distance between the simulated PSF and measured PSF, the misalignment between the numerical model and the actual optical system can be compensated. Besides the DOF extension and PSF optimization, when capturing large-scale dense signals like neuron activities and blood circulation, decomposition of time-series signals and suppression of background fluorescence should be considered. In 2020, Yoon et al. exploited the natural activity-dependent fluorescence intensity and introduced the sparse decomposition algorithm into FLFM, which enabled the same quality of reconstruction for dense signals and sparse ones.⁴¹ What’s more, reducing the unnecessary excitation/emission is also beneficial to resolve complex signals or image opaque sample. Wang et al. suppressed the background fluorescence with the combination of confocal approach when imaging scattering mammalian brains at large depth²¹ (Fig. 2(f)). This modality greatly improves the efficient imaging depth and SBR of FLFM applied in thick tissues without compromising on its high volumetric speed.

2.3. Deep-learning-based LFM

Although LFM is featured for instantaneous volumetric imaging speed and extremely low phototoxicity, which opens a new avenue in the field of neurobiology,^21,24,36 cardiodynamics²³ etc., the low-spatial resolution, obvious artifacts and limited computational speed brought by the traditional iterative-based reconstruction algorithms hamper the widespread use of LFM in the biological application.^16,27

With the fast development of deep learning (DL) and convolution neuron network in the applications of image processing,^42,43,44 numerous DL-based microscopy image enhancing algorithms have been proposed to remarkably increase the imaging resolution and SNR.^45,46,47,48 These algorithms also show excellent fitting ability with a rather promising computation speed because of parallel computing (one forward inference costs about tens of milliseconds). According to such key advantages, DL could be also beneficial to the 3D reconstruction of LFM. However, unlike traditional DL algorithms, which are used to output high-quality images from the same dimensional low-quality images, the main task of 3D reconstruction algorithm of LFM is to derive a high-quality 3D stack from a low-quality modulated 2D image. Although some researchers have combined wide-field images and corresponding explicit defocus distance by deep-learning-based refocusing framework to predict a 3D volume without mechanical scanning,^49,50 there exists refocusing depth limitation and signal loss because wide-field images can’t preserve the whole volumetric information. On the contrary, LFM, featured for its extended DOF and high-efficient encoding ability of volumetric information, provides a better choice for 3D reconstruction by deep learning. However, because LFM was proposed to capture the rapid volumetric dynamics, it’s difficult to obtain a high-resolution image stack of dynamic samples. Therefore, building a paired “2D LF-3D image” dataset to train a DL model is a challenge for this approach. More importantly, how to extract volumetric information from 2D light filed images decides the design of the neural network. Simply changing the paired data type in traditional image enhancement neural network to achieve 3D reconstruction makes no sense because of the complex 4D information that is present in the 2D light field images.

Based on the accurate wave-optics model of LFM, it’s possible to generate any number of synthetic light-field images from high resolution 3D images. In 2021, Wang et al. described a novel LFM strategy based on a VCD neural network processing of light-field data⁶ [Fig. 3(a)]. Convolved with the light-field PSF, synthetic 2D light-field images could be produced from confocal image stacks. Then, VCD neural network with the input of different views extracted from light-field images is established to learn the transformation between 2D LF images and paired confocal image stack. This view split operation disentangles the spatial information from 4D light fields and rearrange the 2D light field image to 3D ‘view stack’. According to different learned 2D convolution operations that are used, parallax information can be extracted from the input stack that is similar to the traditional refocusing algorithm but in an intelligent way (learned up-sampling operation and filtered weights). It means neural network could learn how to inference high-resolution 3D volume from 2D LF. Once the network is trained well, light-field measurement can be converted into an image stack on a millisecond timescale by network forward inference. Due to the strong priors or guidance provided by high-resolution 3D images, this method can remove the artifacts brought by traditional deconvolution algorithm and achieve uniform spatial resolution along the depth.

Fig. 3. Schematic principle and network architecture of DL-based 3D reconstruction algorithms. (a) The VCD-Net reconstruction pipeline, containing: (i) Forward light-field projection (LFP) from the HR image stacks; (ii) VCD transformation (based on Unet) of synthetic light-field inputs into intermediate 3D image stacks; (iii) network training via iteratively minimizing the difference between VCD inferences and confocal ground truths. (b) HyLFM-Net image reconstruction pipeline. Different from the semi-synthetic strategy of VCD-Net, registered experimental data are directly fed into HyLFM network (based on ResNet) to establish a mapping relationship between raw 2D light field images and SPIM stack/slice. With continuous acquisition, such a network can be trained/validated on static or dynamic sample.

Besides obtaining structural prior knowledge from the static sample, directly imaging one plane and its corresponding light-field image of dynamic sample is also a method to establish training datasets. Wagner et al.²⁵ proposed a hybrid light field microscopy (HyLFM) that can image the same sample simultaneously or successively with very brief delay using LFM and LSFM [Fig. 3(b)]. LSFM can capture high-resolution 3D images from static sample as ground truth for corresponding light-field image. Through post-registration, DL model can be trained either in the traditional way, in which DL model’s inputs are pairs of LFM images and corresponding registered high-resolution 3D images, or in “dynamic mode”, in which LSFM slices are fed into DL model as ground truth to constrain reconstruction of captured LFMs. This method is able to continuously obtain different training data from dynamic sample which contributes to the generalization of model.

Although DL outperforms in the quality of image reconstruction and computing time when compared with deconvolution-based methods, there still exists some puzzles on DL-based LFM. For semi-synthetic approach, how to minimize the gap between the simulated light-field images and experimental ones is key to producing 3D reconstruction with high fidelity and resolution. For registration-based approach, the quality of reconstructed results is limited to the pixel-wise difference between primary deconvolution results and target LSM stacks. What’s more, the label data can be only obtained from the hybrid imaging system. It’s hard to obtain label data with higher quality due to the potential existence of incompatibility between LFM and other modalities.

The above problems are common in most DL-based algorithms when applied in microscopy image enhancement. In essence, it’s about how to acquire more sophisticated prior knowledge and build a ‘more compatible’ neural network.

For most DL algorithms, numerous paired data provide powerful data-prior about how to map a degraded image to an ‘accurate’ image. DL model aims to catch this prior by iteratively learning and uses this ‘learned prior’ to finish image enhancement. What can’t be ignored is that keeping an optimal match between corrupted training data and real experimental data is key to produce reasonable reconstruction. For microscopy images, this corruption comes from optical system and acquisition equipment. Some researches proved that introducing optical aberration^18,19 in traditional light field deconvolution algorithms would suppress reconstruction artifacts. The camera-induced matched noise⁵¹ would also improve the robustness and generalization of the algorithms. Beyond this ‘learned prior’, explicit prior can directly constrain the results. For example, using total variation (TV) of the image encourages solutions to contain uniform regions. The same for microscopy image processing, sparsity and continuity of biological structures have been applied into traditional deconvolution algorithms to suppress noise and recover high-frequency information collaboratively.⁴¹ This would be also helpful for light field images processing, but what should be considered deliberately is the intrinsic property of view-wise SNR and resolution degradation during light field imaging.

Besides the aid of prior knowledge, advanced DL model design will be helpful for algorithms’ performance and generalization. As mentioned above, for ‘paired’ experimental data, local misalignment inevitably exists, which potentially influences reconstruction’s quality. This would be mitigated by implicit estimation of local displacement.⁵² Moreover, with the development of self-supervised learning in microscopy image processing,^53,54 it may provide a new strategy for 3D reconstruction of LFM.

3. Biological Application of LFM

3.1. LFM for subcellular visualization

The interaction and organization of various intracellular organelles and molecules in cellular microenvironment reflects the spatiotemporal regulation of biological processes. Long-term visualization of such subcellular dynamics in three dimensions is essential for fundamental research in biological and biomedical science.^55,56,57 Conventionally, most of the fluorescence microscopy techniques acquire 3D information in a sequential scanning or multi-frame fusion fashion,^11,41,58 which inevitably compromises the temporal resolution and light dose, leading to abnormal morphological deformation, stress response or even death of the cell.

These problems can be mitigated by LFM which provides most photon-efficient solution by recording volumetric information simultaneously.⁵⁹ In 2019, Li et al.⁶⁰ developed a HR-LFM with a defocus design for live-cell imaging with a spatial resolution of 300–700nm in all three dimensions. However, artifacts of traditional LFM and shadow DOF limit the further application in interrogation of intracellular process. In 2020, Hua et al.¹⁹ reported a high-resolution FLFM (HR-FLFM) for fast, volumetric and multi-color live cell imaging. Combined with aberration correction algorithms, HR-FLFM allowed researchers to visualize rapid interaction between peroxisomes and mitochondria [Fig. 4(a)], and track the fast-moving peroxisomes in three dimensions.

Fig. 4. Multi-scale imaging of LFM in biological application. (a) and (b) Subcellular imaging. (a) Projection of 3D reconstructed HR-FLFM images in living COS-7 cells. (b) MIPs with magnified views illustrating the formation of migrasomes during neutrophil migration along vessels. (c)–(h) Model organism imaging of LFM. (c) Motor neuron activity tracking in L4-stage *C. elegans*. (d) MIPs in x–y (left) and y–z (right) planes of one instantaneous volume of red blood cells reconstructed by VCD-LFM. (e) Velocity map computed from two consecutive volumes of RBCs during systole. (f) Neuron activity recording and corresponding analysis in mice brain. (g) Imaging and tracking of circulating blood cells in an awake mouse brain. The above figures are reproduced with permission respectively from Refs. 6, 18, 19, 21, 32 and 60.

For in vivo fluorescence imaging, aberration and phototoxicity significantly influence imaging quality. In 2021, Wu et al.¹⁸ proposed a scanning-based LFM, called digital AO scanning LF mutual iterative tomography (DAOSLIMIT), which permitted volumetric imaging across a $225 \times 225 \times 16$ $225 \times 225 \times 16$ $μ$ $μ$ m³-volume, with a resolution of up to 220 nm laterally and 400nm axially, and at the millisecond scale. The authors harnessed the 4D light field information captured by LFM to correct spatially nonuniform aberrations, which enabled 3D dynamical mitochondrial imaging without reconstruction artifacts, providing a robust and accurate tracking analysis for organelle dynamics. What’s more, due to the low phototoxicity of LFM, DAOSLIMIT clearly captured the whole process of migrasome biogenesis in mice. It was observed that the immune cells in the intravascular movement of living mouse livers will detach some migrasomes, which indicated immune cells achieved communication and long-distance interaction with each other. What’s more, the authors injected breast cancer cells into the vasculature of transgenic zebrafish. With DAOSLIMIT, they observed that the cancer cells trapped by small-bore microvessels could split large vesicles under flow stress, suggesting the cancer cells had the ability to react to different circumstances [Fig. 4(b)].

3.2. LFM for dynamics capture in living animals

There occurs many tissue-scale dynamic biological process in three dimensions with millisecond-time scales, like neuronal activities distributed over whole brains, dynamics of beating hearts and blood flow, etc. To catch such rapid biological phenomena in model organisms such as C. elegans, zebrafish, mouse is of great importance for the study of many diseases, including cardiac diseases,⁶¹ neuron diseases,^62,63,64 etc. However, resolving these transient processes needs fast volumetric imaging speed, which requires promising spatiotemporal resolution for the imaging modalities.

LFM, featured for its scanning-free and parallel acquisition methods, precisely fills this niche and has made well application in investigating biological dynamics of model organism.

3.2.1. C. elegan imaging

In 2021, Wang et al. proposed DL-based LFM (termed VCD-LFM) which is a feature with high-spatial resolution and extremely fast speed of 3D reconstruction. They demonstrated to capture the neuronal activity and locomotion of moving C. elegans at a 100Hz acquisition rate, yielding 6000 light fields in both green and red channels in a 1-min observation [Fig. 4(c)]. To further push the achievable throughput of LFM, Zhu et al. present a hybrid approach in 2021 that integrates LFM with microfluidic chip, which allowed high-speed and high-throughput activity mapping of freely-moving C. elegans at a large scale.⁶⁵ Via high-efficient reconstruction and analysis algorithm, this microfluidics-based LFM could achieve high-speed 4D screening on C. elegans managed with different genotypes and phenotype, which contributed to obtain the correlation between worms’ locomotion and neural activities. This new paradigm holds great potential for large-scale drug screening, in which the complete phenotype mapping of drug-treated organoids or small model animals are required for evaluation of drug efficacy.

3.2.2. Zebrafish imaging

In 2017, Cong et al. developed an FLFM with extended DOF to observe neural activities across the whole brain of a freely-moving zebrafish larva in a small water-filed chamber.²⁴ Owning to the instant volumetric imaging ability of LFM, it enabled to catch all neurons’ activities of zebrafish’s brain when it pursued and caught its prey. Apart from monitoring temporal signal fluctuation, LFM is able to visualize dynamics of fast blood flow and rapid morphological change of zebrafish heart while its beating.

Wagner et al.²³ presented an iso-LFM which can simultaneously record perpendicular light fields and reduce fluorescence background by selective volume illumination. It allowed to image the beating heart and blood flow dynamics of 8 d post fertilization (dpf) medaka at single-cell resolution and with a 200 Hz volume rate.

In 2021, Wang et al.⁶ proposed a VCD-LFM to mitigate spatial defects and computational burden of traditional LFM, and combined selective excitation with light field detection to record zebrafish heart beating and blood flow at a 200Hz volume rate. By tracking red blood cells and segmenting heart boundary, the ejection fraction of the heartbeat and blood flow velocity can be calculated [Figs. 4(d) and 4(e)]. Later, based on a hybrid light field imaging system integrated with light sheet microscopy, Wang established a 4D (space and time) analysis pipeline for heartbeat and blood flow.⁶⁶ This brings new insights of the intrinsic relationship between myocardial contraction and intracardiac hemodynamics and provides a powerful tool for the further investigation of cardiac function and development.

3.2.3. Mouse imaging

Except above weakly scattered sample, LFM can also capture biological dynamics in large and scattering tissue. In 2018, Skocek et al. combined head-mounted Miniscope technology with LFM and a neuron signal extraction algorithm to achieve capturing neuronal network activity at the depth of $\sim$ $\sim$ 360 $μ$ $μ$ m in the hippocampus in freely moving mice.⁶⁷ By coupling MiniLFM with a tabletop of two-photon scanning microscope, the accuracy of light field reconstruction can be validated with ground truth [Fig. 4(f)]. With the aid of SID,³⁶ MiniLFM achieved high $F$ $F$ -score and fidelity. In 2020, Zhang et al. combined confocal detection scheme with LFM to physically reject background noise in scattered tissue, which yield high-SBR raw LF measurements.²¹ This confocal LFM enables to image fast circulating blood cells in thousands of blood vessel branches simultaneously and 600 $μ$ $μ$ m deep in the awake mouse brain [Fig. 4(g)].

4. Conclusions and Prospects

Since light field technology was first proposed in 1908, featured for its ability of capturing 4D light field, it has been well developed in various fields, including photography,⁶⁸ scene rendering,⁶⁹ depth estimation,⁷⁰ digital refocusing,⁷¹ etc. The feature of preservation of spatial and angular information of incident light brings inspiration for volumetric fluorescent microscopy in temporal resolution improvement. Till now, numerous variants of LFM are developed, such as scanning-based LFM,¹⁸ FLFM,^19,40 LFM with hybrid modality,^21,32,72,73 algorithm-enhanced LFM^6,25,34 and so on. These optimizations aim to overcome the degradation of 3D reconstruction brought by the intrinsic tradeoff between angular resolution and spatial resolution. With the development of LFM, it has seen a various biological applications over the past years in terms of recording of neuron activities,^6,21,24,36 visualization of cardiac dynamics,^6,23,25 monitoring vascular circulating^6,21,72 and observing intracellular interaction.^18,60 These applications indicate LFM provides a new window for investigating complex rapid biological process in three dimensions.

However, there still exist some issues that deserve to be discussed. First, different optical performance parameters have a mutual restraint in LFM, for example, spatial resolution and DOF in FLFM.⁴⁰ Decoupling these constrains according to advanced optics and corresponding image post-processing algorithms^18,28,31 is a direct approach to promote system performance enhancement. Second, the intrinsic missing cone problem existed in LFM deteriorates the axial resolution especially with the uneven sampling rate. It could also be mitigated by advanced algorithms⁷⁴ or optics design.^20,23 Third, strong fluorescence background far from native objective plane introduces great reduction of SBR especially in strong scattering sample. Combined with confocal microscopy³² or light-sheet microscopy,⁷³ LFM can produce clean measurements with high contrast, but the reconstructed volume is limited by penetration depth or Rayleigh distance. So, generating an energy-concentrated light sheet with long effective focusing range⁷⁵ would be helpful to enlarge the imaging volume. Furthermore, nonlinear excitation is more suitable to observe high scattering sample. Therefore, the combination of two-photon fluorescence and LFM could have promising performance in thick tissue.³² Last but not the least, optical model of variants of LFM needs to be more accurate in order to match different distortion brought by imaging system. Minimizing the gap between experiments and numerical simulations would directly boost the development of model-based reconstruction algorithms.

In conclusion, with various advanced optical hardware design and imaging processing algorithm, the spatial resolution degradation in LFM could be greatly mitigated. Add to its superior volumetric imaging speed, interrogation of intracellular microenvironments and complex interaction between organelles would be possible. What’s more, this instantons 3D imaging ability has shown wide-spread application in calcium activities. Combined with well-designed illumination strategy, it’s expected to enable high-resolution and fast whole-brain imaging in diverse biological samples. In addition, studies for the miniaturization of LFM are rising gradually. These technical improvements would provide a new sight in clinical and medical application of LFM.

Conflict of Interest

The authors declare that there are no conflicts of interest relevant to this paper.

Acknowledgments

We thank the funding supports offered by the National Natural Science Foundation of China (T2225014, 21874052, 61860206009) and the National Key Research and Development Program of China (2017YFA0700501). We also thank Jiahao Sun and Mian He (School of Optical and Electronic Information, the Huazhong University of Science and Technology) for their helpful discussion.

References

1. T. Schrödel, R. Prevedel, K. Aumayr, M. Zimmer, A. Vaziri, “Brain-wide 3D imaging of neuronal activity in Caenorhabditis elegans with sculpted light,” Nat. Meth. 10, 1013–1020 (2013). Crossref, Web of Science, Google Scholar
2. J. P. Nguyen, F. B. Shipley, A. N. Linder, G. S. Plummer, M. Liu, S. U. Setru, J. W. Shaevitz, A. M. Leifer, “Whole-brain calcium imaging with cellular resolution in freely behaving Caenorhabditis elegans,” Proc. Natl. Acad. Sci. 113, E1074–E1081 (2016). Crossref, Web of Science, Google Scholar
3. A. T. Ritter, Y. Asano, J. C. Stinchcombe, N. M. G. Dieckmann, B.-C. Chen, C. Gawden-Bone, S. van Engelenburg, W. Legant, L. Gao, M. W. Davidson, E. Betzig, J. Lippincott-Schwartz, M. G. Gillian, “Actin depletion initiates events leading to granule secretion at the immunological synapse,” Immunity 42, 864–876 (2015). Crossref, Web of Science, Google Scholar
4. M. Mickoleit, B. Schmid, M. Weber, F. O. Fahrbach, S. Hombach, S. Reischauer, J. Huisken, “High-resolution reconstruction of the beating zebrafish heart,” Nat. Meth. 11, 919–922 (2014). Crossref, Web of Science, Google Scholar
5. V. Voleti, K. B. Patel, W. Li, C. Perez Campos, S. Bharadwaj, H. Yu, C. Ford, M. J. Casper, R. W. Yan, W. Liang, C. Wen, K. D. Kimura, K. L. Targoff, E. M. C. Hillman, “Real-time volumetric microscopy of in vivo dynamics and large-scale samples with SCAPE 2.0,” Nat. Meth. 16, 1054–1062 (2019). Crossref, Web of Science, Google Scholar
6. Z. Wang, L. Zhu, H. Zhang, G. Li, C. Yi, Y. Li, Y. Yang, Y. Ding, M. Zhen, S. Gao, T. K. Hsiai, P. Fei, “Real-time volumetric reconstruction of biological dynamics with light-field microscopy and deep learning,” Nat. Meth. 18, 551–556 (2021). Crossref, Web of Science, Google Scholar
7. W. Denk, J. H. Strickler, W. W. Webb, “Two-photon laser scanning fluorescence microscopy,” Science 248, 73–76 (1990). Crossref, Web of Science, Google Scholar
8. N. G. Horton, K. Wang, D. Kobat, C. G. Clark, F. W. Wise, C. B. Schaffer, C. Xu, “In vivo three-photon microscopy of subcortical structures within an intact mouse brain,” Nat Photon. 7, 205–209 (2013). Crossref, Web of Science, Google Scholar
9. J. Huisken, J. Swoger, F. Del Bene, J. Wittbrodt, E. H. Stelzer, “Optical sectioning deep inside live embryos by selective plane illumination microscopy,” Science 305, 1007–1009 (2004). Crossref, Web of Science, Google Scholar
10. T. A. Planchon, L. Gao, D. E. Milkie, M. W. Davidson, J. A. Galbraith, C. G. Galbraith, E. Betzig, “Rapid three-dimensional isotropic imaging of living cells using Bessel beam plane illumination,” Nat. Meth. 8, 417–423 (2011). Crossref, Web of Science, Google Scholar
11. B.-C. Chen, W. R. Legant, K. Wang, L. Shao, D. E. Milkie, M. W. Davidson, C. Janetopoulos, X. S. Wu, J. A. Hammer, Z. Liu, B. P. English, Y. Mimori-Kiyosue, D. P. Romero, A. T. Ritter, J. Lippincott-Schwartz, L. Fritz-Laylin, R. D. Mullins, D. M. Mitchell, J. N. Bembenek, A.-C. Reymann, R. Böhme, S. W. Grill, J. T. Wang, G. Seydoux, U. S. Tulu, D. P. Kiehart, E. Betzig, “Lattice light-sheet microscopy: Imaging molecules to embryos at high spatiotemporal resolution,” Science 346, 1257998 (2014). Crossref, Web of Science, Google Scholar
12. S. Abrahamsson, J. Chen, B. Hajj, S. Stallinga, A. Y. Katsov, J. Wisniewski, G. Mizuguchi, P. Soule, F. Mueller, C. D. Darzacq, X. Darzacq, C. Wu, C. I. Bargmann, D. A. Agard, M. Dahan, M. G. L. Gustafsson, “Fast multicolor 3D imaging using aberration-corrected multifocus microscopy,” Nat. Meth. 10, 60–63 (2013). Crossref, Web of Science, Google Scholar
13. J. Rosen, G. Brooker, “Non-scanning motionless fluorescence three-dimensional holographic microscopy,” Nat. Photon. 2, 190–195 (2008). Crossref, Web of Science, Google Scholar
14. K. Yanny, N. Antipa, W. Liberti, S. Dehaeck, K. Monakhova, F. L. Liu, K. Shen, R. Ng, L. Waller, “Miniscope3D: Optimized single-shot miniature 3D fluorescence microscopy,” Light: Sci. Appl. 9, 171 (2020). Crossref, Web of Science, Google Scholar
15. N. Antipa, G. Kuo, R. Heckel, B. Mildenhall, E. Bostan, R. Ng, L. Waller, “DiffuserCam: Lensless single-exposure 3D imaging,” Optica 5, 1–9 (2018). Crossref, Web of Science, Google Scholar
16. R. Prevedel, Y.-G. Yoon, M. Hoffmann, N. Pak, G. Wetzstein, S. Kato, T. Schrödel, R. Raskar, M. Zimmer, E. S. Boyden, “Simultaneous whole-animal 3D imaging of neuronal activity using light-field microscopy,” Nat. Meth. 11, 727–730 (2014). Crossref, Web of Science, Google Scholar
17. M. Levoy, R. Ng, A. Adams, M. Footer, M. Horowitz, “Light field microscopy,” ACM Trans. Graph. 25, 924–934 (2006). Crossref, Web of Science, Google Scholar
18. J. Wu, Z. Lu, D. Jiang, Y. Guo, H. Qiao, Y. Zhang, T. Zhu, Y. Cai, X. Zhang, K. Zhanghao, H. Xie, T. Yan, G. Zhang, X. Li, Z. Jiang, X. Lin, L. Fang, B. Zhou, P. Xi, J. Fan, L. Yu, Q. Dai, “Iterative tomography with digital adaptive optics permits hour-long intravital observation of 3D subcellular dynamics at millisecond scale,” Cell 184, 3318–3332.e17 (2021). Crossref, Web of Science, Google Scholar
19. X. Hua, W. Liu, S. Jia, “High-resolution Fourier light-field microscopy for volumetric multi-color live-cell imaging,” Optica 8, 614–620 (2021). Crossref, Web of Science, Google Scholar
20. L. Zhu, C. Yi, G. Li, Y. Zhao, P. Fei, Deep-learning based dual-view light-field microscopy enabling high-resolution 3D imaging of dense signals, Proc. Biophotonics Congr. C. M. K. H. C. W. M. Q. K. S.-K. M. D. N. E. D. C. F. O. L. E. V. M. O. Boudoux, E. Buckley, Eds., p. DTh2A.3, Optica Publishing Group, Washington, DC (2021). Crossref, Google Scholar
21. Z. Zhang, L. Bai, L. Cong, P. Yu, T. Zhang, W. Shi, F. Li, J. Du, K. Wang, “Imaging volumetric dynamics at high speed in mouse and zebrafish brain with confocal light field microscopy,” Nat. Biotechnol. 39, 74–83 (2021). Crossref, Web of Science, Google Scholar
22. R. Prevedel, Y.-G. Yoon, M. Hoffmann, N. Pak, G. Wetzstein, S. Kato, T. Schrödel, R. Raskar, M. Zimmer, E. S. Boyden, A. Vaziri, “Simultaneous whole-animal 3D imaging of neuronal activity using light-field microscopy,” Nat. Meth. 11, 727–730 (2014). Crossref, Web of Science, Google Scholar
23. N. Wagner, N. Norlin, J. Gierten, G. de Medeiros, B. Balázs, J. Wittbrodt, L. Hufnagel, R. Prevedel, “Instantaneous isotropic volumetric imaging of fast biological processes,” Nat. Meth. 16, 497–500 (2019). Crossref, Web of Science, Google Scholar
24. L. Cong, Z. Wang, Y. Chai, W. Hang, C. Shang, W. Yang, L. Bai, J. Du, K. Wang, Q. Wen, “Rapid whole brain imaging of neural activity in freely behaving larval zebrafish (Danio rerio),” eLife 6, e28158 (2017). Crossref, Web of Science, Google Scholar
25. N. Wagner, F. Beuttenmueller, N. Norlin, J. Gierten, J. C. Boffi, J. Wittbrodt, M. Weigert, L. Hufnagel, R. Prevedel, A. Kreshuk, “Deep learning-enhanced light-field imaging with continuous validation,” Nat. Meth. 18, 557–563 (2021). Crossref, Web of Science, Google Scholar
26. M. Levoy, Z. Zhang, I. McDowall, “Recording and controlling the 4D light field in a microscope using microlens arrays,” J. Microsc. 235, 144–162 (2009). Crossref, Web of Science, Google Scholar
27. M. Broxton, L. Grosenick, S. Yang, N. Cohen, A. Andalman, K. Deisseroth, M. Levoy, “Wave optics theory and 3-D deconvolution for the light field microscope,” Opt. Exp. 21, 25418–25439 (2013). Crossref, Web of Science, Google Scholar
28. N. Cohen, S. Yang, A. Andalman, M. Broxton, L. Grosenick, K. Deisseroth, M. Horowitz, M. Levoy, “Enhancing the performance of the light field microscope using wavefront coding,” Opt. Exp. 22, 24817–24839 (2014). Crossref, Web of Science, Google Scholar
29. Y.-T. Lim, J.-H. Park, K.-C. Kwon, N. Kim, “Resolution-enhanced integral imaging microscopy that uses lens array shifting,” Opt. Exp. 17, 19253–19263 (2009). Crossref, Web of Science, Google Scholar
30. Y. Sung, “Snapshot projection optical tomography,” Phys. Rev. Appl. 13, 054048 (2020). Crossref, Web of Science, Google Scholar
31. K. He, X. Wang, Z. W. Wang, H. Yi, N. F. Scherer, A. K. Katsaggelos, O. Cossairt, “Snapshot multifocal light field microscopy,” Opt. Exp. 28, 12108–12120 (2020). Crossref, Web of Science, Google Scholar
32. S. Madaan, K. Keomanee-Dizon, M. Jones, C. Zhong, A. Nadtochiy, P. Luu, S. E. Fraser, T. V. Truong, “Single-objective selective-volume illumination microscopy enables high-contrast light-field imaging,” Opt. Lett. 46, 2860–2863 (2021). Crossref, Web of Science, Google Scholar
33. T. V. Truong, D. B. Holland, S. Madaan, A. Andreev, K. Keomanee-Dizon, J. V. Troll, D. E. S. Koo, M. J. McFall-Ngai, S. E. Fraser, “High-contrast, synchronous volumetric imaging with selective volume illumination microscopy,” Commun. Biol. 3, 74 (2020). Crossref, Web of Science, Google Scholar
34. Z. Lu, J. Wu, H. Qiao, Y. Zhou, T. Yan, Z. Zhou, X. Zhang, J. Fan, Q. Dai, “Phase-space deconvolution for light field microscopy,” Opt. Exp. 27, 18131–18145 (2019). Crossref, Web of Science, Google Scholar
35. A. Stefanoiu, J. Page, P. Symvoulidis, G. G. Westmeyer, T. Lasser, “Artifact-free deconvolution in light field microscopy,” Opt. Exp. 27, 31644–31666 (2019). Crossref, Web of Science, Google Scholar
36. T. Nöbauer, O. Skocek, A. J. Pernía-Andrade, L. Weilguny, F. M. Traub, M. I. Molodtsov, A. Vaziri, “Video rate volumetric Ca2+ imaging across cortex using seeded iterative demixing (SID) microscopy,” Nat. Meth. 14, 811–818 (2017). Crossref, Web of Science, Google Scholar
37. Y. Zhang, Z. Lu, J. Wu, X. Lin, D. Jiang, Y. Cai, J. Xie, Y. Wang, T. Zhu, X. Ji, Q. Dai, “Computational optical sectioning with an incoherent multiscale scattering model for light-field microscopy,” Nat. Commun. 12, 6391 (2021). Crossref, Web of Science, Google Scholar
38. Y. Zhang, B. Xiong, Y. Zhang, Z. Lu, J. Wu, Q. Dai, “DiLFM: An artifact-suppressed and noise-robust light-field microscopy through dictionary learning,” Light Sci. Appl. 10, 152 (2021). Crossref, Web of Science, Google Scholar
39. J. He, Y. Cai, J. Wu, Q. Dai, “Spatial-temporal low-rank prior for low-light volumetric fluorescence imaging,” Opt. Exp. 29, 40721–40733 (2021). Crossref, Web of Science, Google Scholar
40. C. Guo, W. Liu, X. Hua, H. Li, S. Jia, “Fourier light-field microscopy,” Opt. Exp. 27, 25573–25594 (2019). Crossref, Web of Science, Google Scholar
41. W. Zhao, S. Zhao, L. Li, X. Huang, S. Xing, Y. Zhang, G. Qiu, Z. Han, Y. Shang, D.-E. Sun, C. Shan, R. Wu, L. Gu, S. Zhang, R. Chen, J. Xiao, Y. Mo, J. Wang, W. Ji, X. Chen, B. Ding, Y. Liu, H. Mao, B.-L. Song, J. Tan, J. Liu, H. Li, L. Chen, “Sparse deconvolution improves the resolution of live-cell super-resolution fluorescence microscopy,” Nat. Biotechnol. 40, 606–617 (2022). Crossref, Web of Science, Google Scholar
42. K. Zhang, W. Zuo, Y. Chen, D. Meng, L. Zhang, “Beyond a gaussian denoiser: Residual learning of deep cnn for image denoising,” IEEE Trans. Image Process. 26, 3142–3155 (2017). Crossref, Web of Science, Google Scholar
43. C. Ledig, L. Theis, F. Huszár, J. Caballero, A. Cunningham, A. Acosta, A. Aitken, A. Tejani, J. Totz, Z. Wang, Photo-realistic single image super-resolution using a generative adversarial network, Proc. IEEE Conf. Computer Vision and Pattern Recognition, IEEE, Piscataway, New Jersey, USA, pp. 4681–4690 (2017). Crossref, Google Scholar
44. J. Redmon, A. Farhadi, “Yolov3: An incremental improvement,” arXiv:1804.02767 (2018). Google Scholar
45. H. Zhang, C. Fang, X. Xie, Y. Yang, W. Mei, D. Jin, P. Fei, “High-throughput, high-resolution deep learning microscopy based on registration-free generative adversarial network,” Biomed. Opt. Exp. 10, 1044–1063 (2019). Crossref, Web of Science, Google Scholar
46. M. Weigert, U. Schmidt, T. Boothe, A. Müller, A. Dibrov, A. Jain, B. Wilhelm, D. Schmidt, C. Broaddus, S. Culley, M. Rocha-Martins, F. Segovia-Miranda, C. Norden, R. Henriques, M. Zerial, M. Solimena, J. Rink, P. Tomancak, L. Royer, F. Jug, E. W. Myers, “Content-aware image restoration: pushing the limits of fluorescence microscopy,” Nat. Meth. 15, 1090–1097 (2018). Crossref, Web of Science, Google Scholar
47. Y. Zhao, M. Zhang, W. Zhang, Y. Zhou, L. Chen, Q. Liu, P. Wang, R. Chen, X. Duan, F. Chen, H. Deng, Y. Wei, P. Fei, Y.-H. Zhang, “Isotropic super-resolution light-sheet microscopy of dynamic intracellular structures at subsecond timescales,” Nat. Meth. 19, 359–369 (2022). Crossref, Web of Science, Google Scholar
48. C. Qiao, D. Li, Y. Guo, C. Liu, T. Jiang, Q. Dai, D. Li, “Evaluation and development of deep neural networks for image super-resolution in optical microscopy,” Nat. Meth. 18, 194–202 (2021). Crossref, Web of Science, Google Scholar
49. Y. Wu, Y. Rivenson, H. Wang, Y. Luo, E. Ben-David, L. A. Bentolila, C. Pritz, A. Ozcan, “Three-dimensional virtual refocusing of fluorescence microscopy images using deep learning,” Nat. Meth. 16, 1323–1331 (2019). Crossref, Web of Science, Google Scholar
50. L. Huang, H. Chen, Y. Luo, Y. Rivenson, A. Ozcan, “Recurrent neural network-based volumetric fluorescence microscopy,” Light Sci. Appl. 10, 62 (2021). Crossref, Web of Science, Google Scholar
51. H. Zhang, Y. Zhao, C. Fang, G. Li, M. Zhang, Y.-H. Zhang, P. Fei, “Exceeding the limits of 3D fluorescence microscopy using a dual-stage-processing network,” Optica 7, 1627–1640 (2020). Crossref, Web of Science, Google Scholar
52. L. Kong, C. Lian, D. Huang, Y. Hu, Q. Zhou, “Breaking the dilemma of medical image-to-image translation,” Adv. Neural Inf. Process. Syst. 34, 1–15 (2021). Google Scholar
53. X. Li, G. Zhang, J. Wu, Y. Zhang, Z. Zhao, X. Lin, H. Qiao, H. Xie, H. Wang, L. Fang, Q. Dai, “Reinforcing neuron extraction and spike inference in calcium imaging using deep self-supervised denoising,” Nat. Meth. 18, 1395–1400 (2021). Crossref, Web of Science, Google Scholar
54. F. Wang, C. Wang, M. Chen, W. Gong, Y. Zhang, S. Han, G. Situ, “Far-field super-resolution ghost imaging with a deep neural network constraint,” Light Sci. Appl. 11, 1 (2022). Crossref, Web of Science, Google Scholar
55. V. Eisner, M. Picard, G. Hajnóczky, “Mitochondrial dynamics in adaptive and maladaptive cellular stress responses,” Nat. Cell Biol. 20, 755–765 (2018). Crossref, Web of Science, Google Scholar
56. D. Jiang, Z. Jiang, D. Lu, X. Wang, H. Liang, J. Zhang, Y. Meng, Y. Li, D. Wu, Y. Huang, Y. Chen, H. Deng, Q. Wu, J. Xiong, A. Meng, L. Yu, “Migrasomes provide regional cues for organ morphogenesis during zebrafish gastrulation,” Nat. Cell Biol. 21, 966–977 (2019). Crossref, Web of Science, Google Scholar
57. Y. C. Wong, S. Kim, W. Peng, D. Krainc, “Regulation and function of mitochondria–lysosome membrane contact sites in cellular homeostasis,” Trends Cell Biol. 29, 500–513 (2019). Crossref, Web of Science, Google Scholar
58. K. M. Dean, P. Roudot, E. S. Welf, G. Danuser, R. Fiolka, “Deconvolution-free subcellular imaging with axially swept light sheet microscopy,” Biophys. J. 108, 2807–2815 (2015). Crossref, Web of Science, Google Scholar
59. B. Gao, L. Gao, F. Wang, “Single-cell volumetric imaging with light field microscopy: Advances in systems and algorithms,” J. Innov. Opt. Health Sci. 2230008. https://doi.org/10.1142/S17935458223000872230008-1 Web of Science, Google Scholar
60. H. Li, C. Guo, D. Kim-Holzapfel, W. Li, Y. Altshuller, B. Schroeder, W. Liu, Y. Meng, J. B. French, K.-I. Takamaru, M. A. Frohman, S. Jia, “Fast, volumetric live-cell imaging using high-resolution light-field microscopy,” Biomed. Opt. Exp. 10, 29–49 (2019). Crossref, Web of Science, Google Scholar
61. J. Bakkers, “Zebrafish as a model to study cardiac development and human cardiac disease,” Cardiovasc. Res. 91, 279–288 (2011). Crossref, Web of Science, Google Scholar
62. P. J. Babin, C. Goizet, D. Raldúa, “Zebrafish models of human motor neuron diseases: Advantages and limitations,” Prog. Neurobiol. 118, 36–58 (2014). Crossref, Web of Science, Google Scholar
63. M. C. K. Leung, P. L. Williams, A. Benedetto, C. Au, K. J. Helmcke, M. Aschner, J. N. Meyer, “Caenorhabditis elegans: An emerging model in biomedical and environmental toxicology,” Toxicol. Sci. 106, 5–28 (2008). Crossref, Web of Science, Google Scholar
64. J. V. Cramer, B. Gesierich, S. Roth, M. Dichgans, M. Düring, A. Liesz, “In vivo widefield calcium imaging of the mouse cortex for analysis of network connectivity in health and brain disease,” NeuroImage 199, 570–584 (2019). Crossref, Web of Science, Google Scholar
65. T. Zhu, L. Zhu, Y. Li, X. Chen, M. He, G. Li, H. Zhang, S. Gao, P. Fei, “High-speed large-scale 4D activities mapping of moving C. elegans by deep-learning-enabled light-field microscopy on a chip,” Sens. Actuators B, Chem. 348, 130638 (2021). Crossref, Web of Science, Google Scholar
66. Z. Wang, Y. Ding, S. Satta, M. Roustaei, P. Fei, T. K. Hsiai, “A hybrid of light-field and light-sheet imaging to study myocardial function and intracardiac blood flow during zebrafish development,” PLoS Comput. Biol. 17, e1009175 (2021). Crossref, Web of Science, Google Scholar
67. O. Skocek, T. Nöbauer, L. Weilguny, F. Martínez Traub, C. N. Xia, M. I. Molodtsov, A. Grama, M. Yamagata, D. Aharoni, D. D. Cox, P. Golshani, A. Vaziri, “High-speed volumetric imaging of neuronal activity in freely moving rodents,” Nat. Meth. 15, 429–432 (2018). Crossref, Web of Science, Google Scholar
68. R. Ng, M. Levoy, M. Bredif́, G. Duval, M. Horowitz, P. Hanrahan, Light field photography with a hand-held plenoptic camera, Technical Report, Stanford University (2005). Google Scholar
69. M. Suhail, C. Esteves, L. Sigal, A. Makadia, “Light Field Neural Rendering,” preprint, arXiv:2112.09687 [cs.CV] (2021). Google Scholar
70. W. Williem, I. K. Park, Robust light field depth estimation for noisy scene with occlusion, Proc. IEEE Conf. Computer Vision and Pattern Recognition, IEEE, Piscataway, New Jersey, USA, pp. 4396–4404 (2016). Crossref, Google Scholar
71. Y. Wang, J. Yang, Y. Guo, C. Xiao, W. An, “Selective light field refocusing for camera arrays using bokeh rendering and superresolution,” IEEE Signal Process. Lett. 26, 204–208 (2018). Crossref, Web of Science, Google Scholar
72. D. Li, H. Zhang, L. L. Streich, Y. Wang, P. Lu, L. Wang, R. Prevedel, J. Qian, “AIE-nanoparticle assisted ultra-deep three-photon microscopy in the in vivo mouse brain under 1300 nm excitation,” Mater. Chem. Front. 5, 3201–3208 (2021). Crossref, Web of Science, Google Scholar
73. D. Wang, S. Xu, P. Pant, E. Redington, S. Soltanian-Zadeh, S. Farsiu, Y. Gong, “Hybrid light-sheet and light-field microscope for high resolution and large volume neuroimaging,” Biomed. Opt. Exp. 10, 6595–6610 (2019). Crossref, Web of Science, Google Scholar
74. J. Lim, K. Lee, K. H. Jin, S. Shin, S. Lee, Y. Park, J. C. Ye, “Comparative study of iterative reconstruction algorithms for missing cone problems in optical diffraction tomography,” Opt. Exp. 23, 16933–16948 (2015). Crossref, Web of Science, Google Scholar
75. J. L. Fan, J. A. Rivera, W. Sun, J. Peterson, H. Haeberle, S. Rubin, N. Ji, “High-speed volumetric two-photon fluorescence imaging of neurovascular dynamics,” Nat. Commun. 11, 1–12 (2020). Crossref, Web of Science, Google Scholar

Vol. 16, No. 01

Metrics

Downloaded 7,428 times

History

Received 5 October 2022

Revised 3 November 2022

Accepted 21 November 2022

Published: 20 January 2023

Information

This is an Open Access article. It is distributed under the terms of the Creative Commons Attribution 4.0 (CC-BY) License. Further distribution of this work is permitted, provided the original work is properly cited.

Keywords

PDF download