Nonlinear scanning structured illumination microscopy based on nonsinusoidal modulation

1. Introduction

Super-resolution microscopy (SRM) enabled the advancement of conventional light microscopy by breaking Abbe’s diffraction limit and realizing bio-imaging at the nanoscale.^{1,2,3,4,5,6,7} There are currently two primary ways to break the diffraction limit: spatial and frequency modulations. The first type, spatial modulation, uses illumination patterns to modulate fluorescence emission and realize the separation of these molecules in different regions. Conventional technologies in the spatial modulation category are stimulated emission depletion (STED) microscopy and reversible saturable optical linear fluorescence transitions (RESOLFT).^8,9,10,11 In STED and RESOLFT, super resolution was achieved by combining a Gaussian excitation beam and a depletion beam similar to a donut shape in the spatially patterned or photoswitching form, effectively emitting fluorescence in the center area of the excitation light spot, and counteracting excitation by a depletion illumination pattern in the area of the donut-shaped spot. Thus, the excitation point spread function (PSF) is constrained. The second technology in the spatial modulation category is stochastic optical reconstruction microscopy (STORM)¹² or photo-activated localization microscopy,^13,14 along with their derivatives. These methods resolve quantum dot fluorescent source distributions based on single-particle localization, in which super resolution is achieved by sequentially activating single emitters and precisely localizing them.^15,16,17 Frequency modulation techniques produce super resolution using structured illumination patterns with which samples are illuminated at high frequencies via the passband of a detection optical transfer function (OTF). The typical method is structured illumination microscopy (SIM) in which a periodic stripe pattern is used to excite samples to produce the Moiré effect^18,19 and the super resolution image is reconstructed by an algorithm.

In comparison with other super-resolution microscopes, SIM has several advantages: (1) It has no special requirements for dyes, and regular fluorescent dyes can meet the imaging requirements. (2) SIM technology requires fewer original images to achieve super resolution, and the imaging speed is faster than those of PLAM and STORM. Currently, for the majority of implementations, SIM is limited to the linear case, which can double the optical resolution in all directions. However, because of the theoretical limitation of linear SIM (L-SIM), the resolution can be increased by a factor of two at most, which makes it impossible to study the precise structure of cells and tissues.

To achieve an even higher resolution than L-SIM, researchers turned to nonlinear SIM (NL-SIM), and there are few reports on improving the resolution using the nonlinear structure of the modulation function. The nonlinearity inherent in either the patterned saturation of fluorescence excitation at high intensity or the patterned depletion of photoswitchable fluorophores^20,21,22 introduces an additional number of harmonics ($H)$, which allows these harmonics to pass through the SIM. If H ≥ 2, the resolution expansion is more than twice the diffraction limit, whereas the traditional linear form of SIM with double resolution improvement has H = 1, and the DC component with uniform illumination has H = 0. The larger the harmonic number, the higher the resolution. However, neither of these technologies can image live cells. Saturation excitation is used at a peak intensity of 8 MW/cm², which is phototoxic and can damage the cell samples and induce strong photobleaching. In the present study, we introduce a novel method that expands the scope of the NL-SIM technology. We address these issues by using nonlinear modulation combined with a scanning two-photon (2P) microscope. 2P excitation is a nonlinear process, and the excitation probability of 2P fluorescence is proportional to the square of the excitation light intensity.^23,24,25 The advantages of 2P fluorescence excitation are first, the excitation light source uses near-infrared light, which has a greater penetration depth²⁶; second, the 2P-excited fluorescent molecules are all located near the focal point, which increases the vertical resolution without the need of a pinhole.²⁷ In addition, in traditional NL-SIM methods, the number of harmonics is based on the intensity of the excitation light. Our 2P-SIM allowed us to generate high-order harmonics through a nonlinear modulation function, leading to higher resolution imaging with low intensities and short exposure times.

2. Theory of 2P-SIM

The resolution of optical microscopic imaging technologies is limited by the system cutoff frequency of the excitation PSF. The OTF and PSF are Fourier transform pairs.²⁸ Unlike other SRM technologies, SIM achieves super resolution by encoding structural details corresponding to high-frequency information in the sample in fundamental frequency information via spatial frequency mixing to generate a Moiré pattern (Fig. 1(a)). A conventional microscope can only resolve the sample structures with information in the OTF. Equivalently, the conventional microscope can only detect information found within a circular region of radius $k_0$ near the observable region (Fig. 1(b)). Essentially, the same circle defines the set of spatial frequency patterns produced through structured illumination. Structured illumination does not alter this physically observable region, but it moves the high-frequency information from outside into the OTF passband, thus making the high-frequency information observable. As a specific example, consider an illumination light structure that consists of two frequency nonsinusoidal stripe patterns. The corresponding Fourier transform has five nonzero points (Fig. 1(c)). One of these points is at the origin, whereas the other four points are offset from the origin in a direction defined by the nonsinusoidal stripe direction of the pattern. When samples are illuminated with this structured illumination light, the image obtained by microscopes contains zero and some high-frequency information that has been mixed into the OTF passband by the Moiré effect (Fig. 1(d)). Owing to the 2P frequency doubling effect, the number of frequencies of the structured illumination pattern in the samples is further increased. Figure 1(e) shows a set of spatial frequency patterns, including one low-frequency information and eight high-frequency information components. The observed image is the sum of these nine frequency contributions, which are not separated using a single image. However, the addition coefficients depend on the phase of the structured illumination pattern. Based on the obtained nine or more images of the sample corresponding to a set of structured-illumination patterns separated by a constant phase, the nine components can be separated through a special algorithm, and the corresponding information can be shifted back to its proper position. The corresponding observable region can be extended in the corresponding direction if the procedure is repeated with other pattern orientations. Figure 1(f) shows the expansion of the multidirectional frequency components.

In this section, we briefly describe the 2P-SIM theoretical mathematical model of the technology. The mathematical model uses the laser-scanning 2P microscopy framework and the harmonic number of the modulation function to obtain super-resolution images.

In laser-scanning 2P microscopy, the intensity of the fluorescent emission $I_{2p\mathrm{\text{em}}}$ described at position ($r)$ in the object plane is determined as follows:

where $I_{2p\mathrm{\text{\hspace{0.17em}ex}}}^{\mathrm{\text{max}}}$ represents the intensity of the excitation light in the specimen. The intensity was normalized in [0,1]. Term $t$ is the time required for any position of the central beam spot ($r)$ during scanning. Assuming that the scanning speed equals unity, $r=t$, $h_{2p\mathrm{\text{\hspace{0.17em}ex}}}$ is the PSF at the excitation wavelength with a cutoff frequency of $k_0$. Here, $s(r)$ represents the sample information. Term ${(I_{2p\mathrm{\text{\hspace{0.17em}ex}}}^{\mathrm{\text{max}}}(t))}^2$ represents the corresponding fluorescence peak intensity excited by $I_{2p\mathrm{\text{\hspace{0.17em}ex}}}^{\mathrm{\text{max}}}(t)$. Term $h_{2p\mathrm{\text{\hspace{0.17em}ex}}}^2$ represents the effective excitation PSF of the 2P microscopy.

As scanning proceeds, time accumulation by the point detector for each image pixel yields the final image frame as follows:

where $\otimes$ denotes the convolution.

To achieve structured light excitation for linear SIM, the sinusoidal structured illumination $I_{2p-L\mathrm{\text{\hspace{0.17em}ex}}}^{\mathrm{\text{max}}}\left(t\right)$ on the specimen can be expressed as follows:

where $k_a$ denotes the temporal modulation frequency. We let $I_{2p-\mathrm{\text{NL ex}}}^{\mathrm{\text{max}}}$ represent the illuminating structured intensity pattern. The sinusoidal modulation function curve and its Fourier transform diagram are shown in Figs. 2(a) and 2(b). It can be observed from the figures that there is only one high-frequency component $k_a$ in the sine modulation function. To produce more frequency patterns, we designed a nonlinear modulation function as follows: where $n$ represents the frequency of the modulation function. Taking $n=2$ as an example, the nonsinusoidal modulation function curve and Fourier transform diagram are shown in Figs. 2(c) and 2(d). It can be observed from the figures that there are two high-frequency components $(k_a,{2k}_a)$ in the non-sine modulation function. The higher the frequency components, the higher the resolution. From Eq. (4), according to the frequency doubling effect of the 2P process, we obtain

In the scanning nonsinusoidal excitation function, a series of harmonics, such as $3k_a,{4k}_a,\dots ,n{k}_a$, are generated through multi-frequency multiplication (Eq. (5)) according to the 2P frequency doubling effect with no requirement of saturation excitation.

Owing to the interaction between the high-order harmonic fringes and the sample structures, the higher undetectable frequency information is moved into the observable passband ($h_{2p\mathrm{\text{\hspace{0.17em}ex}}}^2\left(t\right)h_{\mathrm{\text{em}}}\left(t\right))$ of the 2P microscope; thus, the harmonics further improve the resolution. Moreover, a higher frequency of harmonics increases the resolution. If we tune the modulation frequency $k_a$ to make $k_a=k_0$, the resolution is increased by ($2n+1$) times in comparison with that of a typical 2P microscope. Consider the nonlinear modulation function

as an example. The corresponding microscopy is named 4-harmonic nonlinear SIM (4-NLSIM). The frequency components of the excited fluorescence fringe pattern include direct current, fundamental frequency $k_a$, and second harmonic frequency ${2k}_a$ in the nonlinear modulation function. Thereafter, the 2P effect produces the third and fourth harmonic frequencies (${3k}_a$ and ${4k}_a)$. Thus, the OTF extent reaches its maximum value at ${4k}_a+k_a={5k}_a$, corresponding to a 5-fold increment in the resolution over a typical 2P microscope.

By analyzing the position of the first harmonic peak in the two-dimensional spectrum image, the accurate modulation angle and frequency were decoded. Once the precise modulation frequency and angle are determined, the phase delay can be estimated by shifting and matching the theoretical cosine modes in the spatial domain.²⁹

In addition, for the image reconstruction, we need the $(4n+1)$ equations with a different initial phase $\phi$ to solve the $(4n+1)$ frequency components of $\tilde{s}\left(k\right)$, $\tilde{s}\left(k\pm k_a\right),\dots ,\tilde{s}\left(k\pm 2n{k}_a\right)$ outside the passband of the OTF to achieve a ($2n+1$) time resolution improvement. Considering the different initial phase $\phi$, the normalized peak intensity of the focused excitation light spot can be represented as follows:

where ${\phi }_m=\frac{2\pi m}{4n+1}$, $m=1,2,\dots ,4n+1$. Thus, the image ${\tilde{p}}^{{\phi }_m}(k)$ in the Fourier space can be determined as follows:

where $U_0$, $\dots$, $U_n$ represent constant coefficients, and $\tilde{s}\left(k\right)$ and ${\tilde{h}}_{\mathrm{\text{em}}}\left(k\right)$ denote the Fourier transforms of $s\left(t\right)$ and $h_{\mathrm{\text{em}}}\left(t\right),$ respectively. We can solve the frequency components of $\tilde{s}\left(k\right)$, $\tilde{s}\left(k\pm k_a\right)$, $\dots$ , $\tilde{s}\left(k\pm 2n{k}_a\right)$, based on the $(4n+1)$ equations with different initial phases ${\phi }_m$.

From Eq. (7), to reconstruct the super-resolution image, the high-frequency information of the specimen must be decoded during the data post-processing process. Therefore, structured patterns with different initial phases ${\phi }_m$ are sequentially illuminated on the specimen to produce $n$ independent equations in the following matrix:

3. Results of Resolution Improvement

To demonstrate the resolution improvement of 4-NLSIM, we simulated the imaging process of 2P, linear SIM(L-SIM), 2-harmonic nonlinear SIM (2-NLSIM), and 4-harmonic nonlinear SIM (4-NLSIM) via randomly distributed spots that are infinitely small. In our simulation, an ideal spot sample was generated by randomly selecting one spot from randomly distributed spots. Thereafter, the spot sample was illuminated by sinusoidal and nonsinusoidal patterns. The simulation results are presented in Fig. 3. As expected, 4-NLSIM provides the smallest spot image in comparison with other techniques. According to the intensity profile along the yellow line crossing the bead, the full width at half maximum (FWHM) for 2P is 295nm, L-SIM is 165nm, 2-NLSIM is 100nm, 4-NLSIM is 59nm, which is approximately five times that of conventional 2P. The detectable frequency region was extended to the fourth order (Fig. 3(e)). In addition, this procedure was repeated for seven different pattern orientations separated by $\pi ∕7$ radians to retrieve complete information from a nearly circular region of the frequency space (Fig. 3(f)).

In addition, the ability to distinguish the details of our method was also demonstrated. A pair of the nearest adjacent beads was selected from the image of the randomly distributed spots in the yellow box in Fig. 4(a). The images of two beads for 2P, L-SIM, 2-NLSIM, and 4-NLSIM are shown in Figs. 4(b)–4(e), and based on the intensity profiles along the yellow lines in Fig. 4(f), the 4-NLSIM exhibits the most clearly separated beads.

4. Conclusion

In this paper, we proposed a 2P-SIM method based on point-scanning geometry using the nonsinusoidal modulation of excitation intensity. The nonlinear 4-NLSIM method has broken the resolution limitation of the linear SIM and improves the SIM imaging resolution by a factor of 5 ($\mathrm{\text{FWHM}}=59$nm) in comparison with the conventional 2PM method. The generation of higher order harmonics can be simply controlled by a temporal modulation function, which greatly reduces phototoxicity and photo damage to biosamples in comparison with saturated SIM. In addition, a theoretical model of the image modes and image reconstruction functions is established, and we have demonstrated that the greater the harmonic number, the higher the image resolution. Therefore, the resolution of the reconstructed image can be further improved by increasing the frequency of the nonsinusoidal modulation function.

Conflict of Interest

The authors declare no conflict of interest.

Acknowledgments

This work was supported by National Natural Science Foundation of China (grant nos. 61775148, 61527827, and 61905145); Guangdong Natural Science Foundation and Province Project (2021A1515011916); Shenzhen Science and Technology R&D and Innovation Foundation (grant nos. JCYJ 20200109105608771, JCYJ20180305124754860 and JCYJ 20180228162956597).