In 1908 Gabriel Lippmann reported a new imaging technique for the capture of dense perspective information of 3D scenes [1]. The technique was named as Integral Photography (IP), and was based in inserting an array of microlenses in front of a photographic film. In this way, a collection of small elemental images, each from a different point of view, could be obtained from 3D objects. Later, in 1992, Adelson and Wang [2] introduced the concept of plenoptic function, which describes the radiance of each luminous ray of light as a function of the angle and position in a given point of its trajectory. The IP was applied to microscopy, for the first time, by Jang and Javidi [3], but only with the purpose of displaying microscopic images. It is remarkable the contribution made by Levoy et al. [4] who adapted to microscopy the plenoptic concept. Very recently, the name integral microscopy (iMic) has been coined to refer to this technique. The improvement of the resolution of integral microscopes has aimed many research efforts. In this sense, are noticea-ble the application of ad-hoc deconvolution tools, the use of 4D interpolation techniques, or, more recently [5], the application of physical interpolation based in time multiplexing. The main drawback of iMics is their poor resolution. Factors that favor the loss of resolution are the limited number of pixels, the vignetting in the collected microimages and the comparable sizes of image PSF and microlens-array pitch.

To avoid the later drawbacks, in this contribution we report advances in a novel a novel architecture in which the microlens array is placed at the far-field plane, which in a microscope is just at the aperture stop of the microscope objective. This new realization of the plenoptic architecture is named as Far-field integral Microscopy (FiMic) or alter-natively as Fourier-domain integral Microscopy. As we explain below, The FiMic has some apparent advantages over the iMic: First is that it outputs directly the perspective views. Second is that there is no conflict between the micro-scope PSF and the microlenses sizes, and thus, no extra resolution is lost due to such conflict. Third advantage is the reduction in vignetted pixels. Final advantage is gain in compactness. A preliminary realization of FiMic was reported in [6].

Let us start by showing the typical scheme of an integral microscope, which can be built by performing some minor changes into an optical microscope, which we call hereafter as the native microscope (see Fig. 1). Schematically, optical microscope is transformed into an integral microscope by simply inserting an adequate microlens array (MLA) at the image plane (which is conjugated with the Reference Object Plane −ROP), and displacing axially the CCD, so that it is conjugated with the aperture stop of the objective.

Fig.1.

Schematic layout of an integral microscope.

In order to understand better the characteristics of an iMic, let us consider the parameters of a particular microscopy setup. Let us then consider that we are using a microscope objective (MO) of magnification M = 20 and numerical aperture NA = 0.4. Since the objective is designed for a tube lens of f_L = 180 mm, the focal length of the MO is f_ob = 9.0 mm. Another important parameter is the exit pupil, which is inside the MO, and has diameter ϕ_EP = 7.1 mm. From these parameters we can calculate the resolution limit of the native microscope

which, considering a central wavelength λ = 0.55 μm, gives to ρ_nat = 0.84 μm at the object plane and ρ′_nat = 16.8 μm at the image plane.

Next step is the selection of the optimum MLA. As well known, the resolution limit of calculated views and of rendered depth images is determined by the MLA pitch. Therefore, the smaller the pitch the better the resolution. However, smaller pitch implies stronger diffraction effects. So, in order to minimize such diffraction effects at the MLA plane, and therefore make the ray-optics approach reliable, the pitch should be at least five times larger than diffraction spot. So, an MLA with pitch p = 80 μm is adequate. On the other hand, and in order to make an efficient use of the pixels of the sensor, there must be a good matching between the NA and the microlenses f-number. Specifically, the matching relation is

In our case, the optimum f_# = 25. About the sensor, there is not a big constraint in the selection of pixel size. According to Nyquist, sizes smaller than ρ′_nat/2 = 8.4 μm are acceptable.

Even though the parameters of the iMic were optimized, there are still some remaining important problems. The problems are:

In order to lessen the importance of theses problems, we have proposed a new paradigm in integral microscopy. In our proposal we remove the tube lens, and place directly the MLA at the aperture stop of the MO. Note however that in most commercial MO the aperture plane is not accessible. Even its conjugate, the exit pupil, is neither accessible. Instead it is virtual and placed within the MO. For this reason it is usually necessary the use of a telecentric relay system, as shown in Fig. 2.

Fig.2.

Schematic layout of a far-field integral microscope. In this case the images provided by the microlenses (which we will name as elemental images), are directly perspective views.

Again, in order to understand better the characteristics of FiMic, we consider the parameters of a particular microscopy setup. We use again the same MO (M = 20, NA = 0.4, f_ob = 9.0 mm, and ϕ_EP = 7.1 mm). Another parameter, which now is of great interest, is the angular field number (AFN). The AFN is defined as quotient between the aberration-free field of view (FOV) of the MO, and its focal length;

In our case, AFN = 7.8. Although the lateral (and angular) magnification of the relay system can play an important role in the fine adjustment of the microscope, in this example we consider that it has unit magnification and is formed by two identical lenses of focal length f₁ = f₂ = 50 mm.

Again, the next step is the selection of the optimum MLA. In this case, in order to avoid overlapping and of making an efficient use of the pixels of the sensor, the constraint condition is that the f-number of the microlenses should be equal than the AFN of the objective. The case in which AFN is slightly bigger than the f-number is also acceptable, since the field stop can be used to adjust the fields. Other constraints are: (a) A, relatively, high number of microlenses is needed in order to obtain good angular sampling; (b) Large apertures are needed in order to obtain elemental images with good resolution; and (c) The resolution limit of perspective views and depth rendered images is determined by diffraction, but also by the focal lengths of the elements in the way

In the above equation, ρ_FiMic is the resolution limit at the object space, ρ_EI the resolution limit at the elemental images, and m is the numbers of microlenses along a diameter of the MLA.

Taking into account these constraints, the optimum MLA are arranged in hexagonal way and composed by seven or by nineteen microlenses, see Fig. 3.

Fig. 3.

Optimum microlens arrays for FiMic.

In the case of a MLA composed by nineteen microlenses, and taking into account the parameters of the MO, the pitch should be about p = 1.4 mm, and the f-number about f_# = 7. Note that in fact small changes in the value of f_# only affect the final FOV of the system, but neither the resolution nor the DOF. The EIs are circular with 1.4 mm in diameter, and in which the diffraction spot has radius ρ_EI = 4.7 μm. Note that, contrarily to the case of the iMic, now the diffraction spot 335 times smaller than the EI, and therefore the EIs can be captured sharply. Concerning the CCD, according to Nyquist criterion, an optimum pixel size is about δ = 2.3 μm, which is very close to standard commercial values.

With this system it is possible then to capture directly nineteen orthographic views with resolution limit ρ_FiMiC = 4.2 μm. Depending on the depth position, rendered depth images can have resolution of up to ρ_Rec = 2.1 μm [7]. Finally, the DOF is also enlarged, since it is given by the formula

In the case of the MLA composed by seven microlenses the FiMic parameters are: p = 2.3 mm, f_# = 7, ρ_EI = 4.7 μm, δ = 2.3 μm, ρ_FiMic = 2.5 μm, ρ_Rec = 1.3 μm and DOF = 31.5 μm.

In summary, we present here a new design paradigm in integral microscopy. The main advantages of the new design are: