Depth perception on fundus images using a single-channel stereomicroscopy

1. Introduction

Binocular human vision a distance of 50–75mm provides three-dimensional (3D) depth perception in object observation. The stereovision is based on the brain translation of two slightly different views.¹ In two-dimensional (2D) imaging, both eyes see the same images that are captured by a camera, although the brain can still achieve a level of depth perception due to pattern recognition such as shadows and size.² Stereoscopic imaging includes two slightly different views for each eye at the same time and provides stereovision via various stereo-display methods.

Stereoscopic imaging methods can be greatly beneficial and sometimes necessary in surgical microscopes, stereo-fundus imaging, and other extra- and intra-ocular purposes in ophthalmology. Stereoscopic imaging in ophthalmology can enhance the depth perception of a physician during visualization in diagnostic applications as well as in therapeutic operations.³ The interior structure of the eye is relativity easy to access by optical imaging modalities that are applied to distinguish various interior eye diseases by providing biometric data such as length and shape. The topography of the blood vessels around the optic nerve head is one of the most important diagnostic parameters, particularly in glaucoma diagnosis.⁴ In this case, the imaging target is not easily accessible, and 2D imaging may not be fully informative. Therefore, optical approaches such as stereoscopic imaging, confocal scanning laser tomography, and optical coherence tomography (OCT) can be used to obtain useful geometric and structural data.⁴ Stereoscopic images of the fundus can be used to assess those blood vessels in glaucoma or other medical diagnostic procedures using popular fundus camera approaches owing to the wide availability of fundus cameras.⁴

Recently, we have introduced a single-channel stereoscopic video imaging modality based on a rotating refractor (RR) that can be customized for various applications.^5,6 In the imaging modality, two slightly different view angles are obtained through the rotating refractor, which is mounted on a step motor and placed inside an optical channel, mainly at the back focal plane (BFP) of objective lens. Therefore, the stereo-angle becomes a function of the refractive index and rotation angle of the rotating refractor. In this study, by optimizing the optical system and the controller unit to achieve a precise real-time control, we investigate the feasibility of a single-channel stereoscopic fundus video imaging modality based on the rotating refractor (SFIM-RR). During imaging of the eye interior and retina, the eye lens and cornea function as a part of the imaging optics.⁷ Therefore, in contrast to the previous location of the rotating refractor,^5,6 which is placed behind the objective lens, the rotating refractor is placed in front of the objective lens at the Fourier plane within a working distance. In this new design, the rotating refractor is used to laterally displace the illumination and the optical paths relative to the pupil center to generate stereo-parallax for image disparity.

Currently, two main approaches are available in stereo-image pairs of the fundus: (i) sequential image acquisition using a single fundus camera at different positions and (ii) simultaneous image acquisition using two stereo-fundus cameras.⁷ The simultaneous imaging approach captures two stereo-parallax images at the same time. The sequential imaging approach captures stereo-parallax images sequentially that minimize equipment requirement.⁸

We provide a rapid sequential automatic image acquisition that can be easily developed with a standard monocular fundus camera, namely the single-channel stereoscopic fundus video imaging modality based on rotating refractor. A ray-tracing simulation (WinLens3D Basic; 1.2.6; 2014-07) is performed to find a preliminary configuration of the optical setup. Mathematical analysis is used to investigate the amount of stereo-base (SB), which is the minimum depth difference that can generate image disparity and cause 3D effect on the stereoscopic depth perception [Fig. 1(a)], and the converging angle between the left and right views as a function of the rotation angle and size of the rotating refractor. Then, the simulation results are evaluated by experiment using an inverse optical pathway. The minimum required torque of the rotating refractor is calculated as a function of the imaging speed, motion profile, and rotation angle. A simple human-eye surgery condition is simulated using an eyeball model to investigate the improvement in depth perception.

2. Materials and Methods

2.1. Single-channel stereoscopic fundus video imaging modality

Figure 1(a) shows the schematic diagram of the experimental setup of the prototype single-channel stereoscopic fundus video imaging modality based on the rotating refractor. A positive achromatic doublet lens ($L_{\mathrm{\text{obj}}}$) with a diameter of 30mm and a focal length of 50mm is used as an objective lens that forms a real intermediate image of the fundus. A beam splitter is employed to separate illumination and observation paths and illuminate the imaging target. The pulsed LED illumination is focused on the beam splitter, and $L_{\mathrm{\text{obj}}}$ images the light beam into the pupil of the subject to illuminate the fundus. An aperture stop (Ø1–12mm) is placed immediately behind the beam splitter in the observation beam path, followed by the relay lenses ($L_{r1}$, f = 40mm and $L_{r2}$, f = 30mm). The second real intermediate image is formed by $L_{r1}$ in front of $L_{r2}$, which is incorporated with a camera lens ($L_c$) to form an image on the camera sensor. The aperture stop is imaged by $L_{\mathrm{\text{obj}}}$ into the pupil plane of the eye model. It is used to control the exit pupil of the observation beam path and thus the optical resolution of the fundus. A combination of a linear polarizer in the illumination beam path, which is used to polarize the illumination beam, and another linear polarizer as an analyzer, which is placed in the observation beam path, suppresses unwanted reflection and light scattering.

2.2. Rotating refractor

The proposed rotating refractor is made from a high refractive index glass (SF10) with a refractive index of 1.73, a polished window of 8 × 12mm², and a length of 20mm. The rotating refractor is placed between $L_{\mathrm{\text{obj}}}$ and the subject eyeball within the working distance, at the BFP of eyeball lens. The rotating refractor is capable of controlling the position of the illumination by changing its orientation [Fig. 1(c)] and the observation beam paths and generates two virtual optical pathways [Fig. 1(a)].

3. Results and Discussion

3.1. Design parameters of optical setup

Figure 2(b) lists the optical parameters (i.e., field of view, front and rear foci, principal planes, magnification, and image and working distances) of the SFIM-RR system that is mathematically simulated and characterized using a WinLens3D Basic software (written for: QIOPTIQ, Germany; by: $\copyright$ Dr. Geoff Adamas, 1994–2016). In the simulation, we use the 20-mm length of rotating refractor. A combination of positive lenses with a focal length of 22 mm is considered as an eyeball lens [Fig. 1(a)]. This eyeball structure is included in the simulation to calculate the field of view and magnification. We find that the field of view of the double-sized eyeball model is approximately 11^∘. The field of view increases as the working distance and the focal length of the objective lens decrease. An increase in the size of the human eyeball results in shorter focal length of the eye lens.

The operation of imaging camera is synchronized with the rotating refractor and the light source that trigger the image acquisition. Figure 1(c) shows the digital signals to control the illumination and the motor (e.g., the direction and the step of rotating refractor). An appropriate step motor and a motor drive with sufficient holding torque are employed to provide smooth motion profile with high resolution for rotating the refractor. Through the digital I/O ports, the motor drive, LED, and CMOS camera are reconnected to a National Instruments (NI) PCI data acquisition (DAQ) system. The NI DAQ is programmed via LabVIEW for synchronization and real-time control of the motion profile, pulsed illumination, and triggering of the camera. Figure 1(b) shows a photograph of the prototype SFIM-RR, which includes an eyeball model. The pulses generated by the controller, including the signals for camera triggering and illumination, motor-drive direction, and motor-drive steps, are evaluated using an oscilloscope to demonstrate their accuracy and precision.

3.2. Relationship between rotation angle, stereo-base, and converging angle

In the SFIM-RR, two stereo-parallax images are obtained by refracting the optical path rotating the rotating refractor, which induce different perspectives. As shown in Figs. 1(a) and 2(a), the imaging axis is laterally shifted after passing through the rotating refractor due to the refraction. Capturing an image at each end of the rotation arc of the rotating refractor leads to the generation of two virtual optic channels with slightly different imaging axes. The distance between the two virtual optic channels is a function of the refractive index, size, and rotation angle of the rotating refractor. In the fundus camera, the diameter of the aperture stop controls the diameter of the exit pupil of the observation beam path ($d_o$), which should be sufficiently small to fit in the subject pupil and to maximize the resolution. Because the effective area of the rotating refractor window is larger than $d_o$, the observation beam is not cropped with the edges of the rotating refractor. Thus, the distance between two virtual imaging axes (i.e., the SB) can be derived as follows :

where $L_{\mathrm{\text{RR}}}$ is the length, $\theta$ is the rotation angle, and $n$ is the refractive index, respectively, of the rotating refractor (SF10, 1.73). To calculate the SB, the rotating refractor displacement should be imaged on the plane of the exit pupil, which is slightly lower than the rotating refractor physical displacement.

To measure the distance between two virtual imaging axes, the illumination part is precisely replaced with a narrow laser beam such that the laser beam represents the position of the imaging axis. Then, the displacement of the center of the laser beam due to the refractor rotation is measured under different RR rotation angles. The calculated stereo-base is compared with the measurements of the laser-beam displacement. In addition, the angular difference between the left and right views is theoretically analyzed to estimate the condition for depth perception of a human vision. To simplify the condition, the cornea and the eyeball lens are considered as positive lenses with a 24-mm focal length. The converging angle ($\beta$) of the left and right laser beams at the focal point is calculated as a function of SB for a given rotating refractor (L = 20mm). $\beta$ can represent the converging angle of the imaging axes and thus may be used to suggest better display condition for the most comfortable observation.

Figure 3(b) shows the measurements and calculations of stereo-base with different lengths of rotating refractor (14, 20, and 26mm). In the experiment, we employ a laser beam, rotate the RR according to the orientation $\theta$, and measure the lateral displacement of laser beam. The measured stereo-base linearly increases as a function of the rotation angle of the rotating refractor. Figure 3(c) shows the converging angle of imaging axis $\beta$ that corresponds to different stereo-bases.

3.3. Characterization of image disparity from stereo-base

The minimum stereoscopic depth perception ($\mathrm{\Delta }{L}_{min}$) that is perceived on the fundus can be expressed as follows^8,9 :

where $f_{\mathrm{\text{eye}}}^{\prime }$ is the image-side focal length of the eyeball [Fig. 3(a)] and $\lambda$ is the wavelength of light. As expressed by Eq. (1), stereo-base is a function of the RR rotation angle ($\theta$). $\mathrm{\Delta }{L}_{min}$ is calculated as a function of $\theta$ for the three different $d_o$ values at a wavelength of 550nm.

$\mathrm{\Delta }{L}_{min}$ is calculated as a function of the rotation angle of the rotating refractor at λ = 550nm for three different exit pupil diameters $d_o$, as shown in Fig. 4. The ability of SFIM-RR to resolve detail over the depth (i.e., depth resolution) improves as the rotation angle or the pupil diameter increases. This implies that the depth resolution can be improved up to where the rotation angle and pupil diameter reach their structural maxima.

3.4. Characterization of imaging speed and motor torque

The motor must be correctly selected to achieve smooth and stable RR rotation during image acquisition. The imaging speed, rotation quality, and even the image quality are affected by an appropriate torque generator for the rotating refractor based on the desired motion profile. After designing the desired motion profile and the rotating refractor, a two-phase hybrid step motor (Applied Motion; HT17-278) is chosen to provide sufficient holding torque and rotation speed. Because the torque is a function of the rotation speed, the required torque and minimum rotation time for each image are estimated for various rotation speeds considering the RR rotation angles that ranged from 6^∘ to 30^∘ [Fig. 5(a)]. The torque can be calculated by

where $I$ is the moment of inertia of the rotating refractor and $A_{\alpha }$ is the angular acceleration, which is expressed as where $t$ is the rotation time.

The torque for controlling the rotating refractor is calculated as a function of the rotation speed for various rotation angles of the rotating refractor [Fig. 5(b)]. In this calculation, the effects from gravitation and air resistance are not considered. Figure 5(c) shows the time to take pair of images as a function of the rotation speed of the rotating refractor for different rotation angles. The rotation time becomes the time delay between the left and right image measurements. The results show that the interval between the left and right images is less than 20ms. This is sufficiently short for realizing a semi-simultaneous stereo-imaging.

3.5. Fundus imaging of eyeball model and rabbit eye

To investigate the feasibility of SFIM-RR, a table tennis ball with a diameter of 40mm is used as an eyeball model to simulate a simple condition of a human eyeball. A section of the ball body with approximately 22-mm diameter is removed to create a window for the lens. A Ø1” achromatic doublets lens (f = 45mm) is used to simulate the cornea and eyeball lens to generate a clear image from an object at infinity on the internal posterior wall. A iris diaphragm is used to limit the entrance size of the eyeball model and to control the diameter and size of the pupil.

Also, we use a rabbit eye to obtain in-vivo stereoscopic fundus image. This study was approved by the institutional review board at Wonju Severance Christian Hospital. We enhance the image quality and resolution of our imaging system by adding polarizer and by optimizing illumination: because the illumination and observation beams pass through the same optical pathway, there may exist some reflection and back-scattering from the optical components that can affect the image quality. In conventional fundus imaging system, this problem is solved using a ring aperture in front of the light source and a pinhole mirror instead of a beam splitter. Thus, the annular light beam can be totally separate from the observation beam path.

The stereovision is created as the rotating refractor modifies optical paths for two optical channels. As the refractor rotates, the observation beam path is laterally shifted to the left- and right-hand sides of the optical axis at the exit pupil. When the camera captures such shifted images, it makes image disparity between the left and right images. Then, the stereovision effect is generated.

The depth difference greater than $\mathrm{\Delta }{L}_{min}$ causes image disparity and provides stereoscopic images in the optimal condition. For depth perception, $\mathrm{\Delta }{L}_{min}$ should be adjusted to match with the depth resolution of eye (i.e., the ability to resolve detail over the depth). The minimum stereoscopic depth perception is determined by the lens and sensor resolution. Also, the maximum values of SB, $d_o$, and $\mathrm{\Delta }{L}_{min}$ are determined by the diameter of the patient pupil $D_{pp}$. As shown in Fig. 1(a), $D_{pp}$ must be almost equal to SB$+d_i$ and larger than SB$+d_o$. Therefore, for the stereoscopic imaging of the fundus, we dilate the subject pupil using mydriatic eye drop.

Figure 6 show the stereo-parallax generated from our fundus imaging system. It clearly shows the image disparity. For such images, considering the pupil diameter and eyeball size, the refractor has the rotation angles of 14^∘ and 32^∘ for the rabbit-eye and eyeball model imaging, respectively. At these rotation angles, the converging angles of imaging axis ($\beta$) are almost 6^∘, which is highly recommended as the comfortable and optimized value for binocular human vision. Here in the eyeball model, considering $d_o=3$mm, $\mathrm{\Delta }{L}_{min}$ is about 0.32mm. The table tennis eyeball phantom is approximately two times bigger than the standard actual human eyeball. Owing to the transparency of the table tennis ball, the eyeball phantom is capable of simple trans-scleral illumination, in addition to the common illumination beam paths. Although this type of illumination is mainly used for special cases, it is particularly useful in our experiment because of the lack of separation of the illumination and observation beam paths. In the case of in-vivo images, the unwanted reflection and back-scattering from optical components and the rabbit eye and the lack of trans-scleral illumination significantly reduced the image quality. However, these limitations can easily be resolved in the commercial version using a sophisticated powerful light source and the separation of the illumination and observation beam paths. In addition, in our study, a smaller eyeball and the difficulty in stable positioning of the rabbit eye compared with the human or eyeball phantom lead to mediocre results.

4. Conclusion

We develop a stereoscopic fundus imaging system. The real-time control of the rotating reflector allows stereoscopic fundus imaging with a single monocular fundus camera. For fundus imaging, we characterize the relation between the rotation angle, stereo-base, and converging angle of rotating reflector. The results show that when sufficient torque and power are applied, the SFIM-RR can provide sequential images for stereovision. One can construct such optical system by using standard monocular fundus camera and stereoscopic surgical video microscope. When augmented and virtual realities are considered, our imaging modality is a simple yet efficient technique to make 3D depth perception of eye model in a virtual space that allows users to access every single point in one’s retina using computer.

Conflict of Interest

The authors declare no conflict of interest.

Acknowledgments

This work was supported by the National Research Foundation of Korea (NRF) grant funded by the Korean Government (2019R1A2C2091068).