Open Access
Add to BibSonomy
Abstract
In this work, a high-performance panoramic annular lens system designed with a point-by-point method and variable parameter optimization strategies is proposed. Due to the optimization and design, the system maintains a balance among characteristic parameters and can achieve a ${360}^\circ \times ({40}^\circ {-} {95}^\circ)$ field of view with a 4.34 mm focal length, an ${f}$-number of 4.5, and an $\text{F} {-} \theta$ distortion of less than 0.5%. Experimental results verify the parameters and demonstrate that the distortion is less than 0.8%, except in some areas with mechanical damage on the ogive surface.
© 2020 Optical Society of America under the terms of the OSA Open Access Publishing Agreement
1. INTRODUCTION
Panoramic imaging systems enable the capture of full spatial information and are widely used in visual localization, visual odometry, and biomedicine [1–3]. Due to increasing application needs, several panoramic imaging system structures have been developed, including fisheye lenses, reflective surface panoramic systems, and panoramic annular lens (PAL) systems [4–6], among which PAL systems have been adopted by many researchers due to their high comprehensive performance. In recent decades, researchers have conducted a substantial amount of work to enhance the performance of PAL systems, for example, by compressing the blind area, widening the field of view (FOV), and decreasing the distortion. Zhang et al. achieved a blind area ratio of less than 3.67% due to a special optical structure [7]. Dichroic filters have been employed to achieve PAL systems without blind zones in the middle; Zhang et al. realized a system with a ${360}^\circ \times ({0}^\circ {-} {56}^\circ ,{55}^\circ {-} {110}^\circ)$ FOV based on the principles of fisheye and catadioptric wide-angle lenses [8,9]. No-blind-area PAL systems have a camera utilization rate of 100%, but the sampling density in the central area is very low due to the projection relationship, which results in little effective information. To achieve a larger FOV, two channels of the PAL can be integrated into a single system; Huang et al. integrated seven surfaces into one PAL and achieved a dual-channel panoramic optical system with a ${360}^\circ \times ({38}^\circ {-} {80}^\circ ,\;{102}^\circ {-} {140}^\circ)$ FOV, and Song et al. adopted two tilt-image channels to achieve an ultralarge FOV of ${360}^\circ \times {190}^\circ$ [10,11]. While dual-channel systems enlarge the FOV of panoramic systems, they are difficult to manufacture and achieve low distortion. The distortion of most PAL systems is described as ${F} {-} \theta$ distortion, which is usually larger than 1% [9–12], and Zhou et al. designed a PAL system with a distortion less than 1% [13]. The enhancement of a single parameter of a panoramic system can be applied in certain fields, but comprehensive performance is required in most application scenarios, especially in measurement engineering.
In this paper, a high-performance PAL system with a large FOV, long focal length, and low distortion is realized. In the initial structural design, a point-by-point method is adopted to directly solve several surfaces, which can provide a reasonable input for the optimization process, for which variable parameter strategies are adopted to enable the process to run in a paraxial optics mode. With the proposed design and optimization strategies, the distortion is found to be decreased to 0.5% theoretically and to less than 0.8% in the manufactured system.
2. PRINCIPLE OF PAL AND DESIGN PROCESS
A. Principle of PAL
The structure of a PAL system is presented in Fig. 1 and is mainly composed of a PAL with four surfaces, a stop and relay lens group, and a camera. The rays that originate from an object are refracted by surface 1 and form an “annular entrance pupil” in the PAL, which can create a conjugate relationship with the exit pupil at the aperture stop and enables the design process to be similar to paraxial optics [14]. Then, surfaces 2–4 transfer the rays from the entrance pupil to the exit pupil to make all the chief rays converge at the center of the stop and finally image at the image plane after aberration correction by the relay lenses.
Consider $P$ to be a point on an object plane, $P^{\prime}$ to be the image of $P$, $y^{\prime}$ to be the image height of $P$, and $\theta$ to be the FOV of $P$ as shown in Fig. 1. The adopted image relationship can be described as follows [14]:
where $f$ is the effective focal length of the PAL system; $h$ is a constant and can compress the blind area and improve the effective focal length.B. Design Process
Figure 2 illustrates the design process of the PAL system, which is mainly divided into six steps: (1) analysis of the design demand, including geometric size, FOV, focal length, ${f}$-number, and other specific parameters; (2) determination of the relative position of each surface according to the design demand and calculation of the initial structure of the PAL; (3) design of the relay lens group according to the according to the initial structure of the PAL obtained from step 2 and the image demand; (4) optimization of the system to make the performance meet the design demand and repetition of steps 1–4 if the optimization results cannot satisfy the demand; (5) tolerance analysis and design of the mechanic structure; and (6) verification of the system performance. However, it is difficult to design an appropriate initial structure due to the special unique structure of PALs. The empirical setting of parameters of surfaces in the initial structure may make the system work, but it remains difficult to achieve a high imaging performance. Moreover, the ray overflow phenomenon is also a problem that may interrupt optimization.
3. OPTICAL DESIGN AND OPTIMIZATION
A. Initial Structure Design of PAL
In a PAL panoramic imaging system, distortion and FOV are mutually restricted parameters, and the process of system optimization is to find the optimal solution according to the parameters of the initial structure, that is, the initial structure form determines the upper limit of system performance, so the quality of the initial structure directly affects the final performance of the system. Hence, the initial structure plays an important role in PAL systems. In this paper, an ogive surface is adopted as surface 1 due to its high-performance bending ability, while surface 4 is a flat surface. To obtain a reasonable initial structure, a point-by-point design method similar to the method proposed by Yang et al. is adopted [15]. As shown in Fig. 3, the main principle of the method is divided into two steps, namely, setting up an initial lens group according to the design demand and the iterative solution, and fitting of the surface parameters.
The design process of the PAL consists of five main steps as follows. (1) Determine the relative positions of surfaces 1–4, the entrance pupil, and the exit pupil according to the design requirements. (2) Match the ogive surface to make the chief rays concentrate at the entrance pupil after the refraction of surface 1. (3) Fix surface 2, solve the points on surface 3, and fit them analytically. (4) Fix surface 3, solve the points on surface 2, and fit them analytically. (5) Consider the new surfaces 2–3 as the initial surfaces and repeat steps 3 and 4. Figure 4 illustrates the solution process with surface 3 as an example, and all other surfaces must be fixed in the process, in which the perfect surface is an ideal relay lens and can always refract the entrance ray to the ideal image position.
In the process, $k$ rays that originate from the FOV are adopted to construct the point cloud. Consider ${S_n}$ ($n = {1},\;{2},{\ldots k}$) to be the $n$th ray from the object, ${P_{\textit{mn}}}$ ($m = {1},\;{2},\;{3},\;{4}$) to be the intersection between the $n$th ray and the surface $m$, $r{i_{\textit{mn}}}$ to be the entrance ray at ${P_{\textit{mn}}}$, $r{o_{\textit{mn}}}$ to be the exit ray at ${P_{\textit{mn}}}$, ${P_s}$ to be the center of the exit pupil, ${P_{\textit{pn}}}$ to be the intersection between the $n$th ray and the perfect surface, and ${P_n}$ to be the ideal image position of the $n$th ray. For the $n$th ray, the entrance pupil ${P_{2n}}$ and the exit ray ${\bf r}{{\bf o}_{{\bf 2n}}}$ are certain, while the object and surface 1 are fixed; this means that the entrance ray ${\bf r}{{\bf i}_{{\bf 3n}}}$ is certain. With the distances among fixed flat, perfect surface, and image plane in Fig. 4, the image relationship and Snell’s law $S$ in Eq. (4) can be described as an expression of ${P_{4n}}$. The position of ${P_{4n}}$ can be solved by minimizing Eq. (2) according to Fermat’s principle; thus, the exit ray ${\boldsymbol r}{{\boldsymbol o}_{{ 3\boldsymbol n}}}$ can be solved. With ${\boldsymbol r}{{\boldsymbol i}_{{ 3\boldsymbol n}}}$ and ${\boldsymbol r}{{\boldsymbol o}_{{3\boldsymbol n}}}$, the normal vector of surface $3{{\boldsymbol N}_{{\boldsymbol 3n}}}$ at ${{\boldsymbol P}_{{ 3\boldsymbol n}}}$ can be solved by Eq. (3) according to the vector reflection law of light, which means that the derivative of surface 3 at ${P_{3n}}$ can be solved and the equation of surface 3 can be fitted with the derivatives of all points. In the fit process, seventh-degree polynomial is adopted to fit the data points of derivatives with the least-squares method, and then the analytical equation of surface 3 can be obtained by integral:
where $S$ is the optical path length between ${P_{3n}}$ and ${P_i}$, $n$ is the refractive index of each path, and ${\boldsymbol d}\boldsymbol{s}$ is the length of each path;Similarly, the contour of surface 2 can be obtained and the initial structure can be solved after several iterations.
The target parameters of the PAL system are FOV of ${360}^\circ \times ({40}^\circ {-} {95}^\circ)$, ${f}$-number of 4.5, and image area of ${11.264}\;\text{mm} \times {11.264}\;\text{mm}$. The entrance pupil (stop) is 15 mm away from surface 4, while the image plane is 48 mm away from the stop. Besides, the perfect surface is set 2 mm away from the stop. The calculation process is realized by MATLAB. Figure 5(a) illustrates the initial setup of the PAL, in which both surfaces 2 and 3 are ellipsoidal in consideration of the characteristics of the PAL. Twenty-four equispaced chief rays between 40° and 95° are tracked, and it is evident that the chief rays cannot focus at the center of the stop in the initial setup. Moreover, several rays reflected by surface 3 are intersected with surface 2, and three rays cannot go through the aperture stop, which is not acceptable.
Fig. 5. Layout of the initial structure: (a) the initial setup of the PAL; (b) the iterated structure.
The iteration results are presented in Fig. 5(b), in which all tracked chief rays focus at the center of the aperture stop, which can be used as the initial structure of the PAL. Since the next steps are achieved through Zemax, an even aspheric surface defined by Eq. (4) is adopted to fit surfaces 2 and 3 so as to import the initial structure to Zemax for further design and optimization [16]:
B. Relay Lens Group Matching
Relay lens group transform the rays from the exit pupil to the imaging plane and also play an important role in aberration correction. Hence, the relay lens group is usually composed of many lenses, and aspheric surfaces are adopted to simplify the structure [13]; however, the application of aspheric surfaces increases both the manufacturing difficulty and cost. Considering the feasibility of the mechanical assembly and the mechanical interface of the camera, the first surface is 2 mm away from the top, and the last surface is 7.9 mm away from the image plane. According to the initial structure of the PAL and the ray-tracing results, the FOV of the relay lens group can be determined to be between ${-}{9.267}^\circ$ and 9.267°. With the image relationship of the PAL system, the image height is set as the optimization target and the relay lens group can be matched. As shown in Fig. 6, a simple relay lens group with six spherical surface lenses is matched.
Table 1. Specifications of the PAL System
C. System Optimization
With the initial structure of the PAL and the matched relay lens group, the integrated initial structure of the PAL system is obtained. However, the initial structure cannot satisfy the imaging quality and therefore must be optimized. The optimization process contains two main steps. The first step is aberration correction to make the system meet the design specifications, in which the parameters of the four surfaces of the PAL are fixed and the parameters of the surfaces of the relay lens group are variable to be optimized. The other one is accurate adjustment to enhance the imaging quality, in which the parameters of all surfaces can be set as the variable to be optimized.
Fig. 12. Images captured by the PAL system: (a) picture of the PAL system, (b) raw image captured by the PAL system, and (c) unwrapped image.
Light overflow is a phenomenon that happened in the design process of an imaging system with a large FOV by Zemax, which is shown as incorrect ray position, ray loss, or failure to locate the object in some field of view, and it will interrupt the optimization. To address the problem, a variable parameter optimization strategy is proposed. The strategy is mainly embodied in four aspects: (1) make the aperture size variable, and if overflow occurs in an optimization round, enlarge the size to make the round proceed smoothly and resize it to the design value in the next round; (2) make the FOV variable, and if light overflow occurs at some certain FOV and cannot be avoided, fine-tune the FOV so that the optimization can proceed; (3) if strategies (1) and (2) cannot settle the phenomenon down, discard the overflow light to make the current round proceed smoothly and add it back in the next round; (4) increase the sampling density of the field of view gradually, which can make up for the influence of fine-tuned FOV or discarded lights on the system performance.
After several optimization iterations, the system gradually satisfies the design demand, and the ultimate optical structure of the system is shown in Fig. 7. The total length of the system is 98 mm with 7.9 mm after the intercept. Moreover, the distance between the four surfaces of the PAL and the gaps between relay lenses are sufficiently large, which benefits the manufacturing, optical coating, and assembly.
D. Performance Analysis
The modulation transfer function (MTF), spot diagram, and distortion are the main parameters by which to evaluate a PAL system. The pixel size of the adopted camera is 5.5 µm; hence, the Nyquist frequency should be 91 lp/mm and the MTF value should be greater than 0.3 to achieve a clear image. Figure 8 demonstrates that the MTF value is greater than 0.35 at 91 lp/mm, which indicates that the system has good imaging performance.
Figure 9 illustrates the spot diagram of the system, in which the black circle is an Airy disk and the maximum RMS radius is 3.932 µm at the FOV of 60°, which is smaller than the pixel size of the camera and can therefore image sharply.
Fig. 14. Experimental setup and image captured by the system: (a) experimental device; (b) image captured by the PAL system.
The $\text{F} {-} \theta$ distortion of the system is shown in Fig. 10, which shows that the distortion is less than 0.5% in the entire FOV, and the reference image height is determined by $y^{\prime} = {4.34} \times \theta - {1.691}$.
Moreover, the system is designed as an image space telecentric system, which makes the system a uniform relative illumination within the whole FOV. The specifications of the ultimate system are listed in Table 1.
E. Mechanical Design
As shown in Fig. 11, the mechanical structure of the designed PAL system is composed of a shelter, a PAL column, a relay lens column, and a camera column. The function of the shelter is to protect the PAL while in storage and shelter rays beyond the FOV during use. Manufacturing accuracy has a significant effect on imaging quality, and so the four surfaces of the PAL are manufactured at one time using Freeform L made by Precitech with a surface accuracy of 0.2 µm P-V, while the surfaces of the relay lens group are all spherical and can easily achieve a submicrometer accuracy of submicron easily. In order to ensure assembly precision and simplify the assembly process, all steps for clapping relay lenses are cut at one time.
4. SYSTEM REALIZATION AND PERFORMANCE VERIFICATION
A. Image Results
The assembled system is depicted in Figs. 12(a) and 12(b) and depicts a raw image captured by the system, in which the blind area is a circle with a diameter of 452 pixels and an area equivalent to 3.8% of the sensor. The two buildings in the image are both arc shaped and almost along the same arc, which is quite similar to the imaging form, and the imaging performance can be seen intuitively. The unwrapped image with size of ${6289} \times {757}$ pixels is presented in Fig. 12(c), in which the unwrapped origin position and direction is 0° in Fig. 12(b), and it is evident that the image quality is good and its aberration is very low.
Fig. 15. Distortion measurement results: (a) layout of the corner detection results, (b) imaging relationship results, (c) $\text{F} {-} \theta$ distortion curve by distance, and (d) $\text{F} {-} \theta$ distortion curve by percentage.
B. Performance Verification
To quantitatively verify the image performance, experiments were conducted to measure the distortion and FOV of the PAL system. Figure 13 illustrates the experimental model, which was realized by a standard inner cylinder with a cylindrical checkerboard. Consider $P$ to be a corner on the checkerboard, $P^{\prime}$ to be the image of $P$, $H$ to be the real distance between $P$ and the margin of the image area, $R$ to be the radius of the cylinder, $\theta = g(H,\;R)$ to be the FOV corresponding to ${P}$, and $y^{\prime}$ to be the image height of $P$. $H$ can be calculated by the number of checks, the checkerboard size, and the pixel size, while $f$ can be calculated with the point cloud obtained by tracing object $R$ away from the optical axis via Zemax. Moreover, $y^{\prime}$ can be determined by the pixel size of the camera and the number of pixels between ${P^\prime}$ and the image center. By synthesizing $\theta ,\;y^{\prime}$, and the theoretical image relationship described by Eq. (1), the distortion can be analyzed by Eq. (5):
The distortion measurement results are presented in Fig. 15, in which 100 groups of data along the diameter were chosen and the corners of the checks were sampled data points. The theoretical image relationship is the theoretical line in Fig. 15(b), and it is evident that all data points are in accordance with the theoretical image relationship. The real distortion is presented in Figs. 15(c) and 15(d); the distortion was less than 1.3%, and the trends of all groups were similar. Moreover, the distortion of most groups was less than 0.8%, except for some groups at FOVs of 42.57°, 44.42°, and 93.95°, which was caused by scratches on the ogive surface. Therefore, the distortion results were in accordance with the design parameters while considering the corner detection precision.
5. CONCLUSION
In this work, a PAL system synthesized with large FOV, enough focal length, high f-number, and low distortion was designed and realized. Due to the initial structural design and optimization method, the system can achieve a FOV of ${360}^\circ \times ({40}^\circ {-} {95}^\circ)$ with a theoretical $\text{F} {-} \theta$ distortion of less than 0.5%, and the parameters were verified with experiments. In future research, the PAL system will be calibrated and applied to precision measurement.
Funding
Science Challenging Program, China Academy of Engineering Physics (JCKY2016212A506-0106).
Acknowledgment
The authors thank the Shanghai Engineering Research Center of AI & Robotics and the Engineering Research Center of AI & Robotics, Ministry of Education for their support.
Disclosures
The authors declare no conflicts of interest.
REFERENCES
1. R. Cheng, K. Wang, S. Lin, W. Hu, K. Yang, X. Huang, H. Li, D. Sun, and J. Bai, Panoramic Annular Localizer: Tackling the Variation Challenges of Outdoor Localization Using Panoramic Annular Images and Active Deep Descriptors (IEEE, 2019), pp. 920–925.
2. H. Chen, K. Wang, W. Hu, K. Yang, R. Cheng, X. Huang, and J. Bai, “PALVO: visual odometry based on panoramic annular lens,” Opt. Express 27, 24481–24497 (2019). [CrossRef]
3. B. Paulson, S. Lee, Y. Kim, Y. Moon, and J. K. Kim, “Miniaturized omnidirectional flexible side-view endoscope for rapid monitoring of thin tubular biostructures,” Opt. Express 10, 2264–2274 (2019). [CrossRef]
4. J. Kumler and M. Bauer, “Fish-eye lens designs and their relative performance,” Proc. SPIE 4093, 360–369 (2000). [CrossRef]
5. J. S. Chahl and M. V. Srinivasan, “Reflective surfaces for panoramic imaging,” Appl. Opt. 36, 8275–8285 (1997). [CrossRef]
6. P. Greguss, “Panoramic imaging block for three-dimensional space,” U.S. patent 4566763 (28 January 1986).
7. K. Zhang, X. Zhong, L. Zhang, and T. Zhang, “Design of a panoramic annular lens with ultrawide angle and small blind area,” Appl. Opt. 59, 5737–5744 (2020). [CrossRef]
8. Y. Luo, J. Bai, X. Zhou, X. Huang, Q. Liu, and Y. Yao, “Non-blind area PAL system design based on dichroic filter,” Opt. Express 24, 4913–4923 (2016). [CrossRef]
9. X. Wang, X. Zhong, R. Zhu, F. Gao, and Z. Li, “Extremely wide-angle lens with transmissive and catadioptric integration,” Appl. Opt. 58, 4381–4389 (2019). [CrossRef]
10. Y. Huang, Z. Liu, Y. Fu, and H. Zhang, “Design of a compact two-channel panoramic optical system,” Opt. Express 25, 27691–27705 (2017). [CrossRef]
11. W. Song, X. Liu, P. Lu, Y. Huang, D. Weng, Y. Zheng, Y. Liu, and Y. Wang, “Design and assessment of a 360° panoramic and high-performance capture system with two tiled catadioptric imaging channels,” Appl. Opt. 57, 3429–3437 (2018). [CrossRef]
12. S. Niu, J. Bai, X. Hou, and G. Yang, “Design of a panoramic annular lens with a long focal length,” Appl. Opt. 46, 7850–7857 (2007). [CrossRef]
13. X. Zhou, J. Bai, C. Wang, X. Hou, and K. Wang, “Comparison of two panoramic front unit arrangements in design of a super wide angle panoramic annular lens,” Appl. Opt. 55, 3219–3225 (2016). [CrossRef]
14. J. Wang, Y. Liang, and M. Xu, “Design of panoramic lens based on ogive and aspheric surface,” Opt. Express 23, 19489–19499 (2015). [CrossRef]
15. T. Yang, J. Zhu, and G. Jin, “Design of freeform imaging systems with linear field-of-view using a construction and iteration process,” Opt. Express 22, 3362–3374 (2014). [CrossRef]
16. Radiant Zemax, LLC, ZEMAX Optical Design Program, User’s Guide Version 13 (2014).
