Expand this Topic clickable element to expand a topic
Skip to content
Optica Publishing Group

Automatic underwater polarization imaging without background region or any prior

Open Access Open Access

Abstract

Previous polarization underwater imaging methods based on the physical scattering model usually require background region included in the image and the prior knowledge, which hinders its practical application. In this paper, we analyze and optimize the physically feasible region and propose an improved method by degenerating intermediate variables, which can realize automatic underwater image recovery without background region or any prior. The proposed method does not need to estimate the intermediate variables in the traditional underwater imaging model and is adaptable to the underwater image with non-uniform illumination, which avoids the poor and unstable image recovery performance caused by inaccurate estimation of intermediate parameters due to the improper identification of the background region. Meanwhile, our method is effective for both images without background region and images in which the background region is hard to be identified. In addition, our method solves the significant variation in recovery results caused by the different selection of background regions and the inconsistency of parameter adjustment. The experimental results of different underwater scenes show that the proposed method can enhance image contrast while preserving image details without introducing considerable noise, and the proposed method is effective for the dense turbid medium.

© 2021 Optical Society of America under the terms of the OSA Open Access Publishing Agreement

1. Introduction

Underwater imaging with good performance is a challenging task, in which the scattering and absorption of the incident light wave by particles lead to the distortion of the originally ordered wavefront phase and the attenuation of the amplitude of the light wave. According to the scattering degree of light in the scattering medium, it can be divided into the ballistic regime, the increasing photon scattering and random walk [1]. As the scattering distance and the concentration of the scattering medium increase, the imaging quality in the scattering medium decays sharply. Therefore, suppressing the scattering light is critical for underwater imaging.

Since the scattered light is partially polarized [2,3], polarization imaging has become an effective way of suppressing scattering to achieve a clear underwater vision [412]. The polarization-difference method is served as a common-mode rejection amplifier, which can suppress the effect of backscattered light and amplify the signal from the targets whose polarization-difference magnitude is distinct from the background based on the bionic research by Rowe and Tyo et al. [13,14]. Schechner et al. combined the atmospheric imaging scattering model with polarization and used the backscattering polarization difference between two orthogonal polarization images to achieve underwater imaging [3]. Li et al. greatly improved the quality of underwater imaging by combining the physical scattering model of polarization imaging with histogram stretching [15]. In addition, Alfalou et al. combined polarization with low-pass filtering [16] and correlation technique [17] to achieve polarization-assisted underwater target detection. In general, the method based on the physical scattering model can be used to retrieve the original intensity image more effectively. However, most of the methods that combine the physical scattering model of polarization imaging require the background area to be included in the image to estimate the backscattered light. For the image without background and whose background region is too small to accurately estimate the parameters of the backscattered light, these methods will fail. In addition, these methods usually require manual selection for the background area, which makes it impossible to realize fully automatic image recovery. Meanwhile, the selection of different background regions will greatly influence the results of image recovery. These all limit the practical application of underwater polarization imaging technology. To overcome this problem, Shwartz et al. performed independent component analysis (ICA) over polarization imaging to solve the degree of polarization (DoP) of the airlight, then estimated and separated the airlight [18]. They also proposed the scattering suppression method through similar objects prior and distance prior [19]. Liang et al. realized image recovery by introducing the orientation-angle information from Stokes matrix [20]. However, these methods require prior knowledge or four polarization images.

In this paper, a fully automatic underwater imaging method without background or any prior is proposed. We improve the underwater imaging model based on degenerating intermediate variables and achieve a better scattering suppression effect by automatic optimization of backscattered light intensity information at two orthogonal polarization orientations based on analyzing the physically feasible region of them. The proposed method does not need to estimate any intermediate variables. On one hand, the original image information can directly be utilized to the maximum extent to avoid the inaccurate estimation of intermediate variables. On the other hand, this method can get rid of the limitation of the image background to realize the scattering suppression for the image without background and the image from which the backscattered light information is difficult to be accurately estimated. Meanwhile, it can restrain the non-uniformity of the image to a certain extent, avoid human-computer interaction, and require no prior information to achieve automatic image recovery in a scattering medium, which also solves the large gap of scattering suppression results caused by different selections of background regions and inconsistent parameter adjustment for diverse scenes in some traditional methods.

2. Methodology of image recovery without background or any prior

Based on the traditional polarization image recovery model, the total light intensity $I({x,y} )$ received by the detector can be divided into two parts on the premise that the forward scattering is ignored [17,21]. One part is direct transmission light $D({x,y} )$ from the targets, which is usually called target reflected light. The other part is backscattered light $B({x,y} )$ from scattering media, which is also called background light or veiling light.

$$I({x,y} )= D({x,y} )+ B({x,y} )= L({x,y} )\cdot t({x,y} )+ {A_\infty }[{1 - t({x,y} )} ]\,,$$
where $L({x,y} )$ is the object radiance without scattering or absorption along the line of sight, $t({x,y} )$ represents the transmission coefficient of the scattering medium, and ${A_\infty }$ corresponds to the backscattered light at the infinite distance in the scattering medium according to the physical scattering model. x and y represent the pixel position of the image.

Here, we assume that light travels along the z-axis in Cartesian coordinates. The attenuation coefficient due to absorption and scattering in the scattering medium is distance-dependent which can be represented as $\beta ({\textrm{x},y,z} )$. The transmission coefficient $t({x,y} )$ can be express as

$$t({x,y} )= \exp \left[ { - \int_0^{z^{\prime}} {\beta ({x,y,z} )\textrm{d}z} } \right],$$
where $z^{\prime}$ refers to the distance between the object and the detector.

According to the above formulas, the object radiance $L({x,y} )$ can be derived as

$$L({x,y} )= \frac{{I({x,y} )- {A_\infty }[{1 - t({x,y} )} ]}}{{t({x,y} )}}.$$

The attenuation coefficient $\beta ({\textrm{x},y,z} )$ can be simplified to be a constant $\beta $ when it is position invariant. At this point, Eq. (2) can be simplified as

$$t({x,y} )= \exp [{ - \beta z({x,y} )} ].$$

By combining Eq. (1), Eq. (4) and the physical meaning of ${A_\infty }$, it can be known that $t({x,y} )\to 0$ and $B({x,y} )\to {A_\infty }$ when $z({x,y} )\to \infty $. ${A_\infty }$ is usually replaced by the average of pixel grayscale in the background region $\Omega $ of the image in traditional methods. Therefore, the estimation of the backscattered light from infinity ${\hat{A}_\infty }$ can be calculated by the following equation.

$${\hat{A}_\infty } = {{\sum\limits_\Omega {I({x,y} )} }{\big / }N} = {{\sum\limits_\Omega {{I^{/{/}}}({x,y} )} }{\big / }N} + {{\sum\limits_\Omega {{I^ \bot }({x,y} )} }{\big / }N},$$
where N refers to the number of pixels in the selected background region $\Omega $. The superscripts $/{/}$ and $\bot $ respectively represent the relationship (parallel or perpendicular) between the polarization state of the polarizer in front of the detector and the linear polarization state of the illumination source.

Next, the DoP of backscattered light ${P_{scat}}$ is estimated. There is not only backscattered light but also direct transmission light in the target region. Therefore, compared with using the target region, it is more accurate to use the background region to estimate the DoP of backscattered light.

$${P_{scat}} = \frac{{B_\Omega ^{/{/}}({x,y} )- B_\Omega ^ \bot ({x,y} )}}{{B_\Omega ^{/{/}}({x,y} )+ B_\Omega ^ \bot ({x,y} )}} = \frac{{I_\Omega ^{/{/}}({x,y} )- I_\Omega ^ \bot ({x,y} )}}{{I_\Omega ^{/{/}}({x,y} )+ I_\Omega ^ \bot ({x,y} )}}$$

It is assumed that the scattering medium is uniformly distributed and the DoP of backscattered light does not change with position. In this case, the estimation of the DoP of backscattered light ${\hat{P}_{scat}}$ can be calculated by employing Eq. (7).

$${\hat{P}_{scat}} = \frac{{\sum\limits_\Omega {{I^{/{/}}}({x,y} )} - \sum\limits_\Omega {{I^ \bot }({x,y} )} }}{{\sum\limits_\Omega {{I^{/{/}}}({x,y} )} + \sum\limits_\Omega {{I^ \bot }({x,y} )} }}$$

The transmission coefficient of the background region $\Omega $ can be demonstrated as follows according to Eq. (1) and Eq. (6).

$$\begin{aligned} {t_\Omega }({x,y} )&\textrm{ = 1} - \frac{{B_\Omega ^{/{/}}({x,y} )- B_\Omega ^ \bot ({x,y} )}}{{{{\hat{P}}_{scat}}{{\hat{A}}_\infty }}}\\ & = \textrm{1} - \frac{{I_\Omega ^{/{/}}({x,y} )- I_\Omega ^ \bot ({x,y} )}}{{{{\hat{P}}_{scat}}{{\hat{A}}_\infty }}} \end{aligned}$$

The transmission coefficient of the target region $\Psi $ can be described as

$$\begin{aligned} {t_\Psi }({x,y} )&\textrm{ = 1} - \frac{{B_\Psi ^{/{/}}({x,y} )- B_\Psi ^ \bot ({x,y} )}}{{{{\hat{P}}_{scat}}{{\hat{A}}_\infty }}}\\ & = \textrm{1} - \frac{{I_\Psi ^{/{/}}({x,y} )- I_\Psi ^ \bot ({x,y} )- ({D_\Psi ^{/{/}}({x,y} )- D_\Psi ^ \bot ({x,y} )} )}}{{{{\hat{P}}_{scat}}{{\hat{A}}_\infty }}}. \end{aligned}$$

According to Eq. (8) and Eq. (9), the transmission coefficient of the whole image can degenerate as Eq. (10) when the direct transmission light is unpolarized or it has a very low DoP relative to the backscattered light.

$$t({x,y} )= \textrm{1} - \frac{{{I^{/{/}}}({x,y} )- {I^ \bot }({x,y} )}}{{{{\hat{P}}_{scat}}{{\hat{A}}_\infty }}}$$

We substitute Eq. (5), Eq. (7) and Eq. (10) into Eq. (3) to get the object radiance $L({x,y} )$. Here, $\varDelta I$ means the difference between ${I^{/{/}}}$ and ${I^ \bot }$, and f represents a functional relationship.

$$\begin{aligned} L({x,y} )&= f({I,\Delta I,{I_\varOmega },\Delta {I_\varOmega }} )\\ &\textrm{ } = \frac{{{{\left[ {\sum\limits_\Omega {\Delta I({x,y} )} \cdot I - \sum\limits_\Omega {I({x,y} )} \cdot \Delta I({x,y} )} \right]} / N}}}{{\left[ {{{\sum\limits_\Omega {\Delta I({x,y} )} } / N} - \Delta I({x,y} )} \right]}} \end{aligned}$$

It can be apparently seen that there are two important approximate processing procedures about uniformity in Eq. (5) and Eq. (7). But in fact ${A_\infty }$ and ${P_{scat}}$ are spatially non-uniform variables. In Ref. [22], ${A_\infty }$ and ${P_{scat}}$ are calculated by the method of two-dimensional polynomial fitting, and it is shown that the values of ${A_\infty }$ and ${P_{scat}}$ vary considerably even in the background region. Therefore, the proper estimation of ${A_\infty }$ and ${P_{scat}}$ is significant, and meanwhile, it still affects the estimation of $t({x,y} )$. The key point to estimate ${A_\infty }$ and ${P_{scat}}$ is the selection of $\Omega $ for $I_\Omega ^{/{/}}({x,y} )$ and $I_\Omega ^ \bot ({x,y} )$ in traditional methods. However, on one hand, the background region $\Omega $ needs to be manually selected and cannot be automated for complex scenes. On the other hand, ${I^{/{/}}}({x,y} )$ and ${I^ \bot }({x,y} )$ have a corresponding relationship in pixels. Therefore, ${A_\infty }$ and ${P_{scat}}$ depend on the same background region in the polarized images, which severely limits the estimation range of ${A_\infty }$ and ${P_{scat}}$, and leads to the poor suppressing scattering effect since the optimal ${\hat{A}_\infty }$ and ${\hat{P}_{scat}}$ cannot be obtained.

In order to solve the problem of non-uniformity of the image, we break the pixel correspondence between ${I^{/{/}}}({x,y} )$ and ${I^ \bot }({x,y} )$ to expand the range of ${A_\infty }$ and ${P_{scat}}$ so as to get a better image recovery effect. According to the above ideas, Eq. (11) is transformed into a new function $f^{\prime}$ containing only the original image information (${I^{/{/}}}$ and ${I^ \bot }$) by means of variable substitution. It is further simplified and organized to be Eq. (12), and then the histogram stretching is applied to maximize the use of the grayscale storage range of the digital image to get the final output.

$$\begin{aligned} L({x,y} )&= f^{\prime}({I^{/{/}}},{I^ \bot },{I_\varOmega }^{/{/}},{I_\varOmega }^ \bot )\\ &\textrm{ } = \frac{{2\left[ {{{{I^ \bot }({x,y} )\cdot \sum\limits_\Omega {{I^{/{/}}}({x,y} )} } / N} - {I^{/{/}}}({x,y} ){{ \cdot \sum\limits_\Omega {{I^ \bot }({x,y} )} } / N}} \right]}}{{\left[ {{{\left( {\sum\limits_\Omega {{I^{/{/}}}({x,y} )} - \sum\limits_\Omega {{I^ \bot }({x,y} )} } \right)} / N} - ({{I^{/{/}}}({x,y} )- {I^ \bot }({x,y} )} )} \right]}} \end{aligned}$$

In order to reduce the search range, avoid non-physically feasible points, and get a stable image recovery effect, the physically feasible region of light intensity is determined in the dark gray region in Fig. 1. The physical meanings of the numerator and denominator for $L({x,y} )$ before the simplification represents the direct transmission light and the transmission coefficient, respectively. The signs of the numerator and the denominator are not changed in the whole simplification process. Therefore, the constraints of the following equation set Eq. (13) need to be satisfied according to the denominator and the numerator of Eq. (12).

$$\left\{ {\begin{array}{c} {\arg \;\min [{{I^{/{/}}}({x,y} )} ]\le {{\sum\limits_\Omega {{I^{/{/}}}({x,y} )} }{\big / }N} \le \arg \;\max [{{I^{/{/}}}({x,y} )} ]}\\ {\arg \;\min [{{I^ \bot }({x,y} )} ]\le {{\sum\limits_\Omega {{I^ \bot }({x,y} )} }{\big / }N} \le \arg \;\max [{{I^ \bot }({x,y} )} ]}\\ {{{\sum\limits_\Omega {{I^{/{/}}}({x,y} )} }{\big / }{\sum\limits_\Omega {{I^ \bot }({x,y} )} }} \ge \arg \;\max [{{{{I^{/{/}}}({x,y} )}{\big / }{{I^ \bot }({x,y} )}}} ]}\\ {{{\sum\limits_\Omega {{I^{/{/}}}({x,y} )} }{\big / }N} - {{\sum\limits_\Omega {{I^ \bot }({x,y} )} }{\big / }N} > \arg \;\max [{{I^{/{/}}}({x,y} )- {I^ \bot }({x,y} )} ]} \end{array}} \right.$$

 figure: Fig. 1.

Fig. 1. The physical interpretation of constraints for Eq. (12).

Download Full Size | PDF

We use ${{\sum\limits_\Omega {{I^ \bot }({x,y} )} } / N}$ and ${{\sum\limits_\Omega {{I^{/{/}}}({x,y} )} } / N}$ as the two coordinate axes of the rectangular coordinate system. The physical interpretation of Eq. (13) is shown in Fig. 1, where k represents the slope of the straight line.

Therefore, Eq. (12) can be calculated under the premises that Eq. (13) is satisfied and the object radiance can be obtained by combining the image quality evaluation index under no reference conditions EME as the final evaluation criterion for scattering suppression [23]. EME is short for the value of the measure of enhancement, and under the circumstances without reference, EME can describe image quality accurately in most cases.

$$\begin{aligned} {\left( {{{{{\sum\limits_\Omega {{I^{/{/}}}({x,y} )} } / N},\sum\limits_\Omega {{I^ \bot }({x,y} )} } / N}} \right)_{optimal}} &= \arg \textrm{ max}\{{EME({L({x,y} )} )} \}\\ &\textrm{ } = \arg \textrm{ max}\left\{ {\left|{\frac{1}{{{k_1}{k_2}}}\sum\limits_{l = 1}^{{k_2}} {\sum\limits_{k = 1}^{{k_1}} {20\log \frac{{L_{\max ;k,l}^\omega ({x,y} )}}{{L_{\min ;k,l}^\omega ({x,y} )+ q}}} } } \right|} \right\}, \end{aligned}$$
where q is a small constant to avoid being divided by zero which equals 0.0001. The object radiance image is split up into ${k_1} \times {k_2}$ blocks with the sequence number of (k, l), while $I_{\max ;k,l}^\omega ({x,y} )$ and $I_{\min ;k,l}^\omega ({x,y} )$ are the maximum and minimum value in the block $\omega$ with the sequence number of (k, l) in the image.

The proposed method can restrain the non-uniformity of the image to a certain extent. It directly processes the two original polarization images without estimating intermediate variables such as ${A_\infty }$, ${P_{scat}}$ and $t({x,y} )$, which avoids the poor scattering suppression effect caused by inaccurate estimation of intermediate variables and large difference resulted from different background selections and inconsistent parameter adjustment for images of diverse scenes in traditional methods. Meanwhile, our method does not require any prior and is not limited by the image background or human-computer interaction. It can fully realize automatic underwater image recovery for the image without background and the image whose background is too small to estimate the backscattered light information.

3. Real-world experiments and results

We perform the real-world experiments and obtain the polarization images of various scenes at different concentrations in the turbid water through the experimental setup shown in Fig. 2. The scattering event is influenced by the wavelength of active illumination. In the same detection distance, red light experiences fewer scattering events than visible light at other wavelengths. Meanwhile, the scattering coefficient decreases with the increase of wavelength in the visible light range. Therefore, in order to achieve a better scattering suppression effect, the light source we employed in the experiment is a mounted Light-Emitting Diode (THORLABS M625L3) with a central wavelength of 625 nm. The light emitted by the light source passes through the polarization state generator (PSG) to produce a beam of horizontal linearly polarized light as the active illumination of the underwater scene. The imaging device is a 14-bit digital monochrome CCD camera (AVT Stingray F-033B). The horizontal and vertical linearly polarized images are obtained by placing a rotatable polarization state analyzer (PSA) in front of the camera. PSG and PSA are both composed of a linear polarizer. In future practical applications, a division of focal plane (DoFP) polarization camera can be directly used to obtain two mutually perpendicular linearly polarized images at the same time to achieve automatic scattering suppression.

 figure: Fig. 2.

Fig. 2. The schematic of the experimental setup.

Download Full Size | PDF

We employ a transparent polymethyl methacrylate (PMMA) tank filled with different volumes of water and milk to control the concentration of milk in the water to simulate different scattering intensities. Milk contains spherical particles such as casein molecules with diameters of 0.04∼0.3 µm and fat globules with diameters of 1∼20 µm. The milk we used in the experiments is whole milk with protein content of 3.6 g/100mL and fat content of 4.4 g/100mL. The scattering coefficient of the medium is affected by the milk and solution of microsphere concentrations, which equals 3.00c in $\textrm{c}{\textrm{m}^{ - 1}}$ for whole milk, where c represents the concentration of milk in water [24]. The long-term scattering experiment shows that milk can simulate the scattering characteristics of seawater [17].

First, the scattering suppression effect of images with backgrounds at different concentrations in the same scene is analyzed, and the results compared with other methods are performed, which verify the effectiveness of our proposed method compared to other methods and its adaptability to different concentrations. Secondly, we extract images without background from images of various scenes with background by image segmentation method and make comparison with other methods to verify that our method can automatically realize the image recovery without background. Finally, we perform image recovery by the proposed method on uncropped standard target without background (standard resolution plates) in different concentrations of scattering media to clearly and intuitively prove the advantages of our method in three aspects of fully automatic, background free and adaptable for dense turbid medium.

Figure 3 shows the comparison results of different methods for the wood board scene with background under the conditions of low and high concentration scattering media. In order to maintain the consistency of the experimental results, we did not adjust the experimental parameters such as exposure time and aperture during each set of experiments. Therefore, in the case of high concentration, the grayscale value of the intensity image decreases, and the image darkens due to the enhancement of scattering and absorption. Through comparison, it can be seen that HSI and CLAHE have a good effect on image enhancement at low concentrations. However, at high concentrations, due to the insufficient information of the intensity image, the two digital image processing methods of HSI and CLAHE are limited. In contrast, Schechner’s method and our method can achieve a relatively stable scattering suppression effect at both low and high concentrations. In addition, Schechner’s method needs to identify the background and the target so as to estimate the intermediate parameters through the background. The selection of different background regions and the uniformity of the background will affect the final result of image recovery for Schechner’s method. Compared with Schechner’s method, our method can achieve fully automatic processing without any human-computer interaction and has a more significant scattering suppression effect. In order to make the comparison clearer, we locally enlarged the high-concentration image in Fig. 3. It can be seen that our method has higher contrast than other methods. It is easier to distinguish the pattern details on the wood board and does not introduce noise and distortion.

 figure: Fig. 3.

Fig. 3. The comparison results and their enlarged views of different methods for the scene (wood board) with background under the conditions of low and high concentration scattering media, which respectively are intensity image (original), histogram stretching for intensity image (HSI), and contrast limited adaptive histogram equalization (CLAHE) [25], Schechner’s method [3], and our method.

Download Full Size | PDF

The corresponding histogram of the results for different concentrations and different methods in Fig. 3 is shown in Fig. 4, and the grayscale distribution curve at the dashed line in the enlarged view of Fig. 3 is plotted. It can be seen that in the case of high concentration, the distribution of the grayscale histogram of the intensity image is relatively uniform, and therefore, the effect of HSI is not as obvious as that in the case of low concentration. Due to the higher scattering medium concentration, small ‘tails’ appear on both sides of the histogram of CLAHE, and the histogram distribution becomes significantly narrowed. In order to appropriately compare the grayscale distribution at the dashed line in the enlarged view of Fig. 3, we subtract the minimum grayscale value of each curve without changing their shapes or relative relation so that the lowest point of the curves can be shifted to the horizontal axis of the coordinate axis. According to the corresponding images in Fig. 3, it can be known that the large downward spikes in Fig. 4 represent the signals and the small vibrations are the noises of the images. It can be seen from the grayscale distribution curves in Fig. 4 that our method has the highest signal-to-noise ratio (SNR). Our method has less noise when the signal intensity is similar, and in the case of similar noise amplitudes, it has greater signal strength.

 figure: Fig. 4.

Fig. 4. The histogram of the results for different concentrations and different methods in Fig. 3 and the grayscale distribution curves at the dashed line in the enlarged views of Fig. 3.

Download Full Size | PDF

In order to verify that our method can automatically realize scattering suppression of images without background, we cut the above images to remove their background and directly process the cropped images. The results are shown in Fig. 5. Longitudinal comparisons to the image with backgrounds in Fig. 3 show that the effect of scattering suppression of our method for the image without background is still very significant. As can be seen from Fig. 4, HSI cannot change the distribution shape of the overall grayscale value of the intensity image. The grayscale value of the image after HSI is more widely distributed, so the overall contrast of the image can be enhanced. Although the target of HSI can be highlighted relative to the intensity image, the problem of uneven illumination, pupil cutting and high concentration in the image cannot be solved. Our method breaks the pixel correspondence between ${I^{/{/}}}({x,y} )$ and ${I^ \bot }({x,y} )$, and expands the range of ${A_\infty }$ and ${P_{scat}}$. Thus the shadow caused by uneven illumination and pupil cutting can be eliminated to some extent, and from the results, it can be seen that our method is more competent for high concentrations. We select three parts in the high concentration image to enlarge or convert into a 3D view. For a fair comparison, the spans of ordinate are all set to 120. As can be seen from the enlarged 3D view of the letter ‘optic’ in Fig. 5, it cannot be distinguished in the intensity image without backgrounds. HSI causes the image background trend to be obvious, and the target can be vaguely distinguished. CLAHE can distinguish the target, but the contrast is low. However, our method can clearly distinguish the five letters of ‘optic’, and the contrast and SNR are significantly higher than that of CLAHE from the height and color of the letters in the enlarged 3D view.

 figure: Fig. 5.

Fig. 5. The comparison results and their enlarged views of different methods for the scene without background under the conditions of low and high concentration scattering media.

Download Full Size | PDF

In order to verify the adaptability of our method, we selected a Rubik's Cube with handwriting words “Tianjin University” as the target and enlarge parts of the details. The results are shown in Fig. 6. As can be seen, the backscattered light intensity in the left background part of the intensity image at different concentrations is significantly different, so it can be used to estimate the backscattered light parameter in the traditional physical scattering model. However, the backscattered light cannot be accurately estimated in the image without backgrounds. Therefore, most of the image recovery methods that need to recognize the background will fail. By comparing the results at different concentrations and the details of the enlarged view in Fig. 6, it can be found that our method has an obvious scattering suppression effect and can lead to clear image details, and our method has less noise after image recovery. The recovered image by our method looks more ‘clean’, and the processing effect at low concentration is even close to the visual effect of the scene in clean water.

 figure: Fig. 6.

Fig. 6. The comparison results and their enlarged views of different methods for the scene (Rubik's Cube) without background under the conditions of low and high concentration scattering media.

Download Full Size | PDF

In order to compare the details more clearly and verify the universality of our method, we image the standard resolution plate without background as the target to prove the superiority of our method in the three aspects of fully automatic, background free and high concentration scattering suppression. Different methods at different concentrations and their enlarged views are shown in Fig. 7. From the enlarged view and 3D view, it can be seen that our method can clearly see the details in the bottom right corner which is most difficult to be distinguished, and the grayscale span range of our method is as high as 160, while the grayscale span range of other methods is only 20 to 60.

 figure: Fig. 7.

Fig. 7. The comparison and amplification results of different methods for the scene (standard resolution plate) without background under the conditions of low and high concentration scattering media.

Download Full Size | PDF

To quantify and compare the image quality for different methods in the absence of references, we calculate various evaluation criteria of image quality for the enlarged view in Fig. 7, including standard deviation (STD) [15], RMS contrast (RMSC) [3,26,27], contrast gradient (CG) [28], entropy [29], EME [23], blind-referenceless image spatial quality evaluator (BRISQUE) [30] and natural image quality evaluator (NIQE) [31]. STD, RMSC and CG describe the fluctuation and contrast of the image, respectively. Entropy reflects the amount of information contained in an image. EME describes the severity of changes in local grayscales of an image. BRISQUE and NIQE are two general evaluation criteria that describe the degree of image distortion based on the statistical law of the image spatial domain. Higher values of STD, RMSC, CG, entropy and EME reflect better image quality, while lower values of BRISQUE and NIQUE indicate better image quality. As can be seen in Table 1, most evaluation criteria demonstrate that our method has better image quality, which further proves the effectiveness and superiority of the proposed method.

Tables Icon

Table 1. Quantitative comparison for enlarged view in Fig. 7.

4. Conclusion

In this paper, an automatic underwater image recovery method without background or any prior is realized. We improve the underwater imaging model based on degenerating intermediate variables and achieve a better scattering suppression effect by analyzing and optimizing the physically feasible region of backscattered light intensity information at two orthogonal polarization orientations. The proposed method can restrain the non-uniformity of the image to a certain extent and does not need to estimate intermediate variables, which could avoid the impact of inaccurate estimation of intermediate parameters on image recovery results. Meanwhile, it can get rid of the limitation of the image background to realize the scattering suppression for the image without background and the image whose backscattered light parameters can hardly be accurately estimated for both low and high concentration scattering mediums. This method does not require human-machine interaction, and it also avoids the problem that traditional methods need to manually select the background and adjust the parameters which lead to a significant variation of the recovered image quality. The real-world experimental results show that our method has distinct advantages in scattering suppression, detail recovery and noise introduction in both low and high concentrations of turbid water.

Because of the characteristic of requiring no estimation of the intermediate variables or human-computer interaction for our method, an interesting perspective is to combine the proposed model with deep learning methods which have more computational power to enhance the generalization ability of deep learning scattering suppression imaging when the data set is not enough. We are expected to see the potential application of the proposed method in the field of scattering suppression imaging, such as underwater image recovery, image dehazing and medical tissue imaging.

Funding

National Natural Science Foundation of China (61775163, 62075161); Guangxi Innovation-Driven Development Project (GuikeAA21077008).

Disclosures

The authors declare no conflicts of interest.

Data availability

Data underlying the results presented in this paper are not publicly available at this time but may be obtained from the authors upon reasonable request.

References

1. V. Ntziachristos, “Going deeper than microscopy: The optical imaging frontier in biology,” Nat. Methods 7(8), 603–614 (2010). [CrossRef]  

2. S. Sabbah and N. Shashar, “Light polarization under water near sunrise,” J. Opt. Soc. Am. A 24(7), 2049 (2007). [CrossRef]  

3. Y. Y. Schechner and N. Karpel, “Recovery of underwater visibility and structure by polarization analysis,” IEEE J. Ocean. Eng. 30(3), 570–587 (2005). [CrossRef]  

4. H. H. Jin, L. J. Qian, J. Gao, Z. G. Fan, and J. Chen, “Polarimetric Calculation Method of Global Pixel for Underwater Image Restoration,” IEEE Photonics J. 13(1), 1–15 (2021). [CrossRef]  

5. J. Guan and J. Zhu, “Target detection in turbid medium using polarization-based range-gated technology,” Opt. Express 21(12), 14152 (2013). [CrossRef]  

6. J. G. Guan, J. P. Zhu, and H. Tian, “Polarimetric Laser Range-Gated Underwater Imaging,” Chin. Phys. Lett. 32(7), 074201 (2015). [CrossRef]  

7. L. Yang, J. Liang, W. Zhang, H. Ju, L. Ren, and X. Shao, “Underwater polarimetric imaging for visibility enhancement utilizing active unpolarized illumination,” Opt. Commun. 438, 96–101 (2019). [CrossRef]  

8. F. Liu, Y. Wei, P. Han, K. Yang, L. Bai, and X. Shao, “Polarization-based exploration for clear underwater vision in natural illumination,” Opt. Express 27(3), 3629 (2019). [CrossRef]  

9. F. Liu, P. Han, Y. Wei, K. Yang, S. Huang, X. Li, G. Zhang, L. Bai, and X. Shao, “Deeply seeing through highly turbid water by active polarization imaging,” Opt. Lett. 43(20), 4903 (2018). [CrossRef]  

10. B. Huang, T. Liu, H. Hu, J. Han, and M. Yu, “Underwater image recovery considering polarization effects of objects,” Opt. Express 24(9), 9826 (2016). [CrossRef]  

11. W. Ren, J. Guan, J. Liu, J. Chen, and X. Feng, “Range-gated imaging in turbid conditions using a combination of intensity and polarization information,” Phys. Scr. 94(10), 105505 (2019). [CrossRef]  

12. S. Fang, X. Xia, X. Huo, and C. Chen, “Image dehazing using polarization effects of objects and airlight,” Opt. Express 22(16), 19523 (2014). [CrossRef]  

13. M. P. Rowe, J. S. Tyo, N. Engheta, and E. N. Pugh, “Polarization-difference imaging: a biologically inspired technique for observation through scattering media,” Opt. Lett. 20(6), 608 (1995). [CrossRef]  

14. J. S. Tyo, M. P. Rowe, E. N. Pugh, and N. Engheta, “Target detection in optically scattering media by polarization-difference imaging,” Appl. Opt. 35(11), 1855 (1996). [CrossRef]  

15. X. Li, H. Hu, L. Zhao, H. Wang, Y. Yu, L. Wu, and T. Liu, “Polarimetric image recovery method combining histogram stretching for underwater imaging,” Sci. Rep. 8(1), 12430 (2018). [CrossRef]  

16. K. O. Amer, M. Elbouz, A. Alfalou, C. Brosseau, and J. Hajjami, “Enhancing underwater optical imaging by using a low-pass polarization filter,” Opt. Express 27(2), 621 (2019). [CrossRef]  

17. M. Dubreuil, P. Delrot, I. Leonard, A. Alfalou, C. Brosseau, and A. Dogariu, “Exploring underwater target detection by imaging polarimetry and correlation techniques,” Appl. Opt. 52(5), 997–1005 (2013). [CrossRef]  

18. S. Shwartz, E. Namer, and Y. Y. Schechner, “Blind haze separation,” Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit. 2, 1984–1991 (2006). [CrossRef]  

19. E. Namer, S. Shwartz, and Y. Y. Schechner, “Skyless polarimetric calibration and visibility enhancement,” Opt. Express 17(2), 472 (2009). [CrossRef]  

20. J. Liang, L. Y. Ren, H. J. Ju, E. S. Qu, and Y. L. Wang, “Visibility enhancement of hazy images based on a universal polarimetric imaging method,” J. Appl. Phys. 116(17), 173107 (2014). [CrossRef]  

21. J. S. Jaffe, “Computer Modeling and the Design of Optimal Underwater Imaging Systems,” IEEE J. Ocean. Eng. 15(2), 101–111 (1990). [CrossRef]  

22. H. Hu, L. Zhao, X. Li, H. Wang, and T. Liu, “Underwater image recovery under the nonuniform optical field based on polarimetric imaging,” IEEE Photonics J. 10(1), 1–9 (2018). [CrossRef]  

23. A. M. Grigoryan and S. S. Agaian, “Transform-based image enhancement algorithms with performance measure,” Adv. Imaging Electron Phys. 130, 165–242 (2004). [CrossRef]  

24. Y. Piederrière, F. Boulvert, J. Cariou, B. Le Jeune, Y. Guern, and G. Le Brun, “Backscattered speckle size as a function of polarization: influence of particle-size and- concentration,” Opt. Express 13(13), 5030 (2005). [CrossRef]  

25. K. Zuiderveld, Contrast Limited Adaptive Histogram Equalization (Academic Press, Inc., 1994).

26. J. Liang, L. Ren, H. Ju, W. Zhang, and E. Qu, “Polarimetric dehazing method for dense haze removal based on distribution analysis of angle of polarization,” Opt. Express 23(20), 26146 (2015). [CrossRef]  

27. P. J. Bex, S. G. Solomon, and S. C. Dakin, “Contrast sensitivity in natural scenes depends on edge as well as spatial frequency structure,” J. Vis. 9(10), 1 (2009). [CrossRef]  

28. T. Ishitani and M. Sato, “Contrast-to-gradient method for the evaluation of image resolution taking account of random noise in scanning electron microscopy,” J. Electron Microsc. 53(3), 245–255 (2004). [CrossRef]  

29. R. R. Coifman and M. V. Wickerhauser, “Entropy-based algorithms for best basis selection,” IEEE Trans. Inf. Theory 38(2), 713–718 (1992). [CrossRef]  

30. T. Sun, X. Zhu, J. S. Pan, J. Wen, and F. Meng, “No-reference image quality assessment in spatial domain,” Adv. Intell. Syst. Comput. 21(12), 4695–4708 (2012). [CrossRef]  

31. A. Mittal, R. Soundararajan, and A. C. Bovik, “Making a “completely blind” image quality analyzer,” IEEE Signal Process. Lett. 20(3), 209–212 (2013). [CrossRef]  

Data availability

Data underlying the results presented in this paper are not publicly available at this time but may be obtained from the authors upon reasonable request.

Cited By

Optica participates in Crossref's Cited-By Linking service. Citing articles from Optica Publishing Group journals and other participating publishers are listed here.

Alert me when this article is cited.


Figures (7)

Fig. 1.
Fig. 1. The physical interpretation of constraints for Eq. (12).
Fig. 2.
Fig. 2. The schematic of the experimental setup.
Fig. 3.
Fig. 3. The comparison results and their enlarged views of different methods for the scene (wood board) with background under the conditions of low and high concentration scattering media, which respectively are intensity image (original), histogram stretching for intensity image (HSI), and contrast limited adaptive histogram equalization (CLAHE) [25], Schechner’s method [3], and our method.
Fig. 4.
Fig. 4. The histogram of the results for different concentrations and different methods in Fig. 3 and the grayscale distribution curves at the dashed line in the enlarged views of Fig. 3.
Fig. 5.
Fig. 5. The comparison results and their enlarged views of different methods for the scene without background under the conditions of low and high concentration scattering media.
Fig. 6.
Fig. 6. The comparison results and their enlarged views of different methods for the scene (Rubik's Cube) without background under the conditions of low and high concentration scattering media.
Fig. 7.
Fig. 7. The comparison and amplification results of different methods for the scene (standard resolution plate) without background under the conditions of low and high concentration scattering media.

Tables (1)

Tables Icon

Table 1. Quantitative comparison for enlarged view in Fig. 7.

Equations (14)

Equations on this page are rendered with MathJax. Learn more.

I ( x , y ) = D ( x , y ) + B ( x , y ) = L ( x , y ) t ( x , y ) + A [ 1 t ( x , y ) ] ,
t ( x , y ) = exp [ 0 z β ( x , y , z ) d z ] ,
L ( x , y ) = I ( x , y ) A [ 1 t ( x , y ) ] t ( x , y ) .
t ( x , y ) = exp [ β z ( x , y ) ] .
A ^ = Ω I ( x , y ) / N = Ω I / / ( x , y ) / N + Ω I ( x , y ) / N ,
P s c a t = B Ω / / ( x , y ) B Ω ( x , y ) B Ω / / ( x , y ) + B Ω ( x , y ) = I Ω / / ( x , y ) I Ω ( x , y ) I Ω / / ( x , y ) + I Ω ( x , y )
P ^ s c a t = Ω I / / ( x , y ) Ω I ( x , y ) Ω I / / ( x , y ) + Ω I ( x , y )
t Ω ( x , y )  = 1 B Ω / / ( x , y ) B Ω ( x , y ) P ^ s c a t A ^ = 1 I Ω / / ( x , y ) I Ω ( x , y ) P ^ s c a t A ^
t Ψ ( x , y )  = 1 B Ψ / / ( x , y ) B Ψ ( x , y ) P ^ s c a t A ^ = 1 I Ψ / / ( x , y ) I Ψ ( x , y ) ( D Ψ / / ( x , y ) D Ψ ( x , y ) ) P ^ s c a t A ^ .
t ( x , y ) = 1 I / / ( x , y ) I ( x , y ) P ^ s c a t A ^
L ( x , y ) = f ( I , Δ I , I Ω , Δ I Ω )   = [ Ω Δ I ( x , y ) I Ω I ( x , y ) Δ I ( x , y ) ] / N [ Ω Δ I ( x , y ) / N Δ I ( x , y ) ]
L ( x , y ) = f ( I / / , I , I Ω / / , I Ω )   = 2 [ I ( x , y ) Ω I / / ( x , y ) / N I / / ( x , y ) Ω I ( x , y ) / N ] [ ( Ω I / / ( x , y ) Ω I ( x , y ) ) / N ( I / / ( x , y ) I ( x , y ) ) ]
{ arg min [ I / / ( x , y ) ] Ω I / / ( x , y ) / N arg max [ I / / ( x , y ) ] arg min [ I ( x , y ) ] Ω I ( x , y ) / N arg max [ I ( x , y ) ] Ω I / / ( x , y ) / Ω I ( x , y ) arg max [ I / / ( x , y ) / I ( x , y ) ] Ω I / / ( x , y ) / N Ω I ( x , y ) / N > arg max [ I / / ( x , y ) I ( x , y ) ]
( Ω I / / ( x , y ) / N , Ω I ( x , y ) / N ) o p t i m a l = arg  max { E M E ( L ( x , y ) ) }   = arg  max { | 1 k 1 k 2 l = 1 k 2 k = 1 k 1 20 log L max ; k , l ω ( x , y ) L min ; k , l ω ( x , y ) + q | } ,
Select as filters


Select Topics Cancel
© Copyright 2024 | Optica Publishing Group. All rights reserved, including rights for text and data mining and training of artificial technologies or similar technologies.