1. Introduction

Currently, multi-linear array charge coupled devices (CCDs) are often equipped on optical remote sensing satellites (ORSSs) in order to obtain an image with a wide image swath. In in-orbit operations, each CCD collects a sub-image in a push-broom mode, and the adjacent sub-images have a small overlap. Then, sensor orientation, radiometric correction, geometric stitching, and band registration of all the sub-images are performed in ground processing, and a standard optical remote sensing satellite image (ORSSI) is produced. In order to produce a standard ORSSI with an optimal geometric quality, accurate camera parameters such as the focal length, lens distortions, and camera installation angles, are indispensable.

In-orbit geometric calibration is a widely used method to obtain accurate camera parameters of ORSSs and has been successfully used by many ORSSs, such as SPOT-5 [1], OrbView-3 [2], IRS-P6 [3], ALOS [4], Deimos-2 [5], ZiYuan-3 [6,7], GaoFen-1/2 [8,9], and HaiYang-1C satellites [10]. In the early days of an ORSS’s launch, in-orbit geometric calibration should be performed timely, so that the geometric quality of ORSSIs can be improved. When the ORSS operates in orbit, the status of satellite cameras and the status of satellite position and attitude determination systems may change due to space environment changes and satellite instrument depletions. In-orbit geometric calibration should be performed aperiodically according to the instrument status changes.

Existing in-orbit geometric calibration methods for ORSSs can be mainly classified into two categories: field-dependent methods and field-independent methods. For the former, many ground control points (GCPs) are often taken as absolute controls and used to precisely estimate camera parameters. GCPs can be manually surveyed in a calibration field by a global navigation positioning system (GNSS) or automatically extracted from some reference data. For example, Gachet performed internal calibration of the SPOT-5 HRG and HRS cameras with many GCPs extracted from the reference data [1]. Mulawa performed geometric calibration of the OrbView-3 cameras with thousands of GCPs extracted from the medium-scale aerial images [2]. Wang et al. extracted dense GCPs from the reference digital orthophoto maps (DOMs) and digital elevation models (DEMs) and calibrated external and internal parameters of the ZiYuan-1 02C, ZiYuan-3, and GaoFen-6 satellite cameras [6,11]. Cao et al. performed geometric calibration of the HaiYang-1C satellite cameras with dense GCPs extracted from the reference DOMs and DEMs [10,12]. For the latter, tie constraints are often taken as relative controls and used to estimate camera parameters. Tie constraints are mainly provided by tie points among multiple overlapped images. For example, Greslou et al. and Lebègue et al. calibrated the viewing reference frame biases of the Pleiades-HR camera with a couple of images collected in an auto-reverse mode and calibrated the focal plane with a couple of images collected in a cross mode [13,14]. Cheng et al. employed the tie constraints of three-view stereoscopic images to perform self-calibration of the GaoFen-2 internal parameters [8]. Pi et al. took full advantage of the tie constraints of a cross-image pair and multi-attitude images and presented a self-calibration method for ORSSs [15,16].

With the help of tie constraints, the field-independent calibration methods can theoretically estimate external and internal camera parameters without GCPs. In fact, however, external parameters can be only roughly estimated with tie constraints [13]. From previous studies, we can see that it is actually very difficult to achieve real field-independent calibration. Sparse GCPs are still necessary to achieve an optimal external orientation accuracy [8,17]. Moreover, the field-independent methods usually require that ORSSs have a strong imaging maneuverability (e.g., auto-reverse imaging mode and cross imaging mode). Unfortunately, the majority of in-orbit ORSSs have not such a maneuverability yet. Hence, the field-dependent calibration methods are still popular methods and widely used at present. In the field-dependent methods, an image block of each sub-image should be covered by a calibration field. In practice, in order to reduce the negative influence of satellite position and attitude errors on the camera parameter estimation of different CCDs, field-covered image blocks of all sub-images should be collected at almost the same time, as the valid image block shown in Fig. 1. Meanwhile, valid image blocks of all sub-images should be texture-riched and the collection time difference between image blocks and the field reference data should be as small as possible, so that highly precise GCPs can be extracted. Generally, as long as sufficient evenly distributed GCPs in all sub-image blocks are available, camera parameters of each CCD can be precisely estimated. Specially, the left and the right edges of each sub-image should have GCPs (e.g., sub-image 4 in Fig. 1), so that all the adjacent sub-images can be seamlessly stitched in geometry in the following ground processing.

 

Fig. 1. Sketch map of the field-dependent calibration methods.

Download Full Size | PDF

In practice, due to some uncontrollable factors such as land surface changes, lack of textures, radiometric differences, and ground sampling distance (GSD) differences, we cannot always obtain sufficient evenly distributed GCPs in the valid blocks of all sub-images. Besides, in order to meet ORSSI users’ current application needs for wider image swath and higher resolution, more and more CCDs are placed on a camera focal plane. For example, a panchromatic camera of the Jilin-1 KF 01 satellite has 24 CCDs, and each CCD has 6144 detectors. The GSD of panchromatic images is smaller than 1.0 meter, and the total image swath reaches 136 kilometers [18]. Undoubtedly, the difficulty of in-orbit geometric calibration increases as the number of CCDs increases, because it is very difficult to guarantee that the valid image blocks of all sub-images have sufficient evenly distributed GCPs at almost the same time. In such cases, camera parameters of some CCDs often cannot be precisely estimated by the conventional field-dependent calibration methods. As a result, the geometric quality of standard ORSSIs cannot reach an optimal level.

In this study, we presented a feasible in-orbit geometric calibration method for multiple linear-array ORSSs with tie constraints. In the presented method, GCPs in the valid image blocks are taken as absolute controls, and tie points between the adjacent sub-images are taken as relative controls. With the help of both absolute and relative controls, camera parameters of all CCDs can be precisely and simultaneously estimated. Compared with the conventional field-dependent calibration methods, the major innovation of the presented method is that tie constraints provided by tie points between the adjacent sub-images are introduced. The calibration procedures of multi-linear array ORSSs with both GCPs and tie points are designed. With the introduced tie constraints, all CCDs are logically connected into a complete CCD, and the left and the right edges of all CCDs could be effectively constrained. Consequently, camera parameters of all CCDs can be precisely estimated, even if sufficient evenly distributed GCPs in some sub-images are unavailable.

The remainder of this paper is organized as follows. Section 2 details the establishment of the in-orbit geometric calibration model, the conventional geometric calibration methods, and the presented geometric calibration method. Section 3 describes the use of three GaoFen-6 panchromatic images and two ZiYuan3-02 nadir images to analyze the feasibility and effectiveness of the presented method. Section 4 gives the conclusions.

2. Methodology

2.1 In-orbit geometric calibration model

An in-orbit geometric calibration model of a multiple linear-array ORSS is often established based on a physical sensor model. The physical sensor model should be able to precisely describe the geometric relationship between an image point and the corresponding ground point. In practice, different ORSSs perhaps have different physical sensor models due to different imaging instruments, different definitions of space coordinate systems, and different data processing procedures. Accordingly, different ORSSs perhaps have different in-orbit geometric calibration models. In this study, a look-angle-based in-orbit geometric calibration model is used, and its mathematical expression is as follows [6,10,19].

In in-orbit geometric calibration, (α₀, α₁, α₂, α₃, β₀, β₁, β₂, β₃) are used to represent the look angles (φ_x, φ_y) of each CCD detector and often considered as the internal camera parameters. These internal parameters can describe the comprehensive influences of principal point distance errors, focal length errors, linear-array rotation errors, and lens distortions of a satellite camera on the ORSSI sensor orientation accuracy. ${\textbf R}_{\textrm{Camera}}^{\textrm{ADS}}$ is constructed by the installation angles (pitch, roll, yaw) of the satellite camera, and (pitch, roll, yaw) are often considered as the external camera parameters. These external parameters can describe the comprehensive influences of camera installation errors, ADS installation errors, and GNSS installation errors. It is noted that the look angles of all CCD detectors are defined in a same camera coordinate system O_c-X_cY_cZ_c, as shown in Fig. 2; that is, all CCDs share a same set of external parameters while each CCD has its own internal parameters.

 

Fig. 2. Sketch map of the look angles of CCD detectors.

Download Full Size | PDF

2.2 Conventional geometric calibration

According to the established geometric calibration model in Eq. (1), the main procedures of the conventional field-dependent calibration methods are as follows [6,10,20].

2.3 Presented geometric calibration

In the conventional field-dependent calibration methods, only GCPs are used as control constraints. The geometric calibration accuracy of each sub-image is mainly determined by the GCP extraction accuracy and the internal accuracy of the reference DOMs and DEMs. The geometric stitching accuracy of the adjacent sub-images is mainly determined by the GCP distributions. Specifically, the CCD-detector look angle model in Eq. (1) is mathematically a polynomial model. In theory, GCPs in the narrow image block of a sub-image should be evenly distributed and the left and the right edges of the sub-image should have sufficient GCPs, so that the polynomial model parameters can be effectively constrained and estimated.

Generally, an accurate and robust image matching algorithm can improve the GCP extraction accuracy. Higher-resolution and highly accurate remote sensing images (e.g., aerial images or unmanned aerial vehicle images) can be used to produce the reference DOMs and DEMs, so that the external and internal accuracy of the reference DOMs and DEMs can be improved. Comparatively, we often cannot update the reference DOMs and DEMs in time, because it needs a lot of manpower, material resources, and financial resources. In such a case, many uncontrollable factors such as land surface changes, lack of textures, radiometric differences, and GSD differences, have a negative influence on the GCP extraction. It is thereby very difficult to simultaneously guarantee that all sub-images have satisfied GCPs in the selected narrow image blocks, especially when many CCDs are placed on the focal plane.

When we estimate the internal parameters in Eq. (1), the internal parameters of a certain CCD are actually used to mathematically model the CCD-detector look angles of all the GCPs in the corresponding sub-image. When the sub-image has not sufficient evenly distributed GCPs, perhaps we can achieve a satisfied calibration accuracy. However, the stitching accuracy of the adjacent sub-images is often worse than expected, because the internal parameters are only constrained by the GCPs. In the image areas where lack of GCPs, such as the left and the right edges, the internal parameters will have no effective constraints; that is, the estimated internal parameters are deviated, although the calibration accuracy may be satisfied.

In the field-independent calibration methods, we can see from previous studies that tie constraints provided by tie points are the major geometric constraints to estimate camera parameters [8–16]. Here, tie constraints mean that each pair of tie points in the overlapped images should spatially intersect into a same ground point. In fact, we can also introduce such constraints in the field-dependent methods, since the adjacent sub-images have dense tie points. With the help of tie constraints, the left and the right edges of each sub-image can be theoretically constrained. The stitching accuracy of the adjacent sub-images can be then improved. On basis of this idea, we presented a feasible in-orbit geometric calibration method for multiple linear-array ORSSs with tie constraints. The main procedures are as follows.

It is noted that the matched tie points in each overlapped area are weekly converged. We cannot consider all the three coordinate components (i.e., latitude, longitude, and height) of tie points as unknowns. Otherwise, week convergences will result in large estimation deviations. In order to overcome the week convergence problem, DEM-supported parameter estimation is a preferred method, in which DEMs are used as a height constraint [21,22]. In this study, the main procedures of the DEM-supported internal parameter estimation are as follows.

${\textbf S} = {\left[ {\begin{array}{*{20}{c}} \cdots &{d{\alpha_{0,k}}}&{d{\alpha_{1,k}}}&{d{\alpha_{2,k}}}&{d{\alpha_{3,k}}}&{d{\beta_{0,k}}}&{d{\beta_{1,k}}}&{d{\beta_{2,k}}}&{d{\beta_{3,k}}}& \cdots \end{array}} \right]^\textrm{T}}$ is the correction matrix of the unknown internal parameters; and

${{\textbf L}_g} = \left[ {\begin{array}{*{20}{c}} \vdots \\ {\left( {{\alpha_0} + {\alpha_1}n + {\alpha_2}{n^2} + {\alpha_3}{n^3} - \frac{{{a_{11}}({X - {X_S}} )+ {a_{12}}({Y - {Y_S}} )+ {a_{13}}({Z - {Z_S}} )}}{{{a_{31}}({X - {X_S}} )+ {a_{32}}({Y - {Y_S}} )+ {a_{33}}({Z - {Z_S}} )}}} \right)_{g,k,i}^0}\\ {\left( {{\beta_0} + {\beta_1}n + {\beta_2}{n^2} + {\beta_3}{n^3} - \frac{{{a_{21}}({X - {X_S}} )+ {a_{22}}({Y - {Y_S}} )+ {a_{23}}({Z - {Z_S}} )}}{{{a_{31}}({X - {X_S}} )+ {a_{32}}({Y - {Y_S}} )+ {a_{33}}({Z - {Z_S}} )}}} \right)_{g,k,i}^0}\\ \vdots \end{array}} \right]$ is the constant matrix and can be obtained with the estimated external parameters and the initialized internal parameters. Here, the subscript g denotes GCPs; the subscript k = 1, 2, …, m denotes the k^th sub-image; and the subscript i = 1, 2, …, n_g denotes the i^th GCP in the k^th sub-image.

${{\textbf D}_t} = \left[ {\begin{array}{*{20}{c}} \vdots \\ {\begin{array}{*{20}{c}} \ldots &{\frac{{\partial {F_{x,t,k,j}}}}{{\partial {B_j}}}}&{\frac{{\partial {F_{x,t,k,j}}}}{{\partial {L_j}}}}& \ldots \end{array}}\\ {\begin{array}{*{20}{c}} \ldots &{\frac{{\partial {F_{y,t,k,j}}}}{{\partial {B_j}}}}&{\frac{{\partial {F_{y,t,k,j}}}}{{\partial {L_j}}}}& \ldots \end{array}}\\ \vdots \end{array}} \right]$ is the partial derivative matrix of the unknown latitudes and longitudes of tie points; and ${\textbf T} = {\left[ {\begin{array}{*{20}{c}} \cdots &{d{B_j}}&{d{L_j}}& \cdots \end{array}} \right]^{\textbf T}}$ is the correction matrix of the unknown latitudes and longitudes. Here, the subscript t denotes tie points; the subscript j = 1, 2, …, n_t denotes the j^th tie point in the k^th sub-image.

In the presented calibration method, tie points are matched between the adjacent sub-images. Both the radiometric and geometric differences between the adjacent sub-images are often very small. Dense and highly precise tie points can be easily matched. Moreover, the matched tie points can distribute in the whole overlapped area between the adjacent sub-images rather than just in the selected narrow image block, as shown in Fig. 1. The reason is that satellite position and attitude errors have almost the same influences on a pair of tie points in establishing Eq. (2). With the tie constraints provided by tie points, the left and the right edges of all CCDs can be effectively constrained, and all CCDs can be logically connected into a complete CCD. The internal parameters of all CCDs can then be simultaneously and precisely estimated. It is expected that the estimation deviations of the internal parameters caused by lack of GCPs and uneven GCP distributions can be effectively eliminated. The geometric stitching accuracy of the adjacent sub-images can be accordingly improved.

3. Results and discussion

3.1 Experimental datasets

In this study, three GaoFen-6 panchromatic images and two ZiYuan3-02 nadir images were tested. The general characteristics of the tested images are listed in Table 1. In order to demonstrate the feasibility and effectiveness of the presented calibration method, two sets of reference DOMs and DEMs were used to evaluate the geometric calibration accuracies and the sensor orientation accuracies of images 1, 4, and 5, as listed in Table 1.

Table 1. General characteristics of the tested images.

View Table | View all tables in this article

3.2 Geometric calibration accuracy analysis

In this section, a GaoFen-6 image (i.e., image 1) and a ZiYuan3-02 image (i.e., image 4) in Table 1 were tested. In order to comparatively evaluate the feasibility of the presented calibration method, we designed two experiments as follows.

 

Fig. 3. GCP and tie point distributions in GaoFen-6 (a) sub-image 1, (b) sub-image 2, (c) sub-image 3, (d) sub-image 4, (e) sub-image 5, (f) sub-image 6, (g) sub-image 7, and (h) sub-image 8.

Download Full Size | PDF

 

Fig. 4. GCP and tie point distributions in ZiYuan3-02 (a) sub-image 1, (b) sub-image 2, and (c) sub-image 3.

Download Full Size | PDF

Table 2. Geometric calibration accuracies of the GaoFen-6 sub-images.

View Table | View all tables in this article

Table 3. Geometric calibration accuracies of the ZiYuan3-02 sub-images.

View Table | View all tables in this article

In experiment E1, the geometric calibration accuracies of all the GaoFen-6 and ZiYuan3-02 sub-images achieved by the conventional calibration method reached better than 1.0 pixel, as listed in Tables 2 and 3. On the whole, the conventional calibration method is a mature method. The established geometric calibration model and the designed calibration procedures in the conventional calibration method are already proved to be feasible and effective by many ORSSs. As long as sufficient highly precise and evenly distributed GCPs in all sub-images are available, accurate external and internal parameters of all CCDs can often be estimated. The achieved calibration accuracy is mainly determined by the GCP extraction accuracy and the internal accuracy of the reference DOMs and DEMs. Here, the calibration errors of the GaoFen-6 and the ZiYuan3-02 sub-images caused by the DOM errors were approximately 0.3 pixel. The calibration errors caused by the DEM errors could be ignored. Moreover, land surface changes, lack of textures, radiometric differences, and GSD differences had a large negative influence on the GCP extraction. Generally, the geometric calibration accuracies achieved in experiment E1 were consistent with the expected.

In fact, when we estimate the internal parameters with GCPs in experiment E1, we can see from Eq. (1) that the internal parameters are used to mathematically model all the GCPs’ CCD-detector look angles. The achieved calibration accuracies mainly reflect the GCPs’ accuracies, whilst they are unable to reflect the GCPs’ distributions. When an image area in the selected narrow image block lacks of GCPs, we can often obtain a satisfied calibration accuracy. However, the estimated internal parameters are more likely to be deviated, because the internal parameters cannot be effectively constrained in the GCP-lacked area. Sub-image 3 of the GaoFen-6 image is an example. We could see from Fig. 3(c) that the left edge of sub-image 3 had not GCPs in narrow image block 2. The calibration accuracy of sub-image 3 still reached 0.805 pixel. In such a case, the geometric stitching accuracy except for the geometric calibration accuracy should be used to judge whether the estimated internal parameters were deviated.

In experiment E2, both GCPs and tie points in all the sub-images were used to simultaneously estimate the internal parameters of all the CCDs. The calibration accuracies achieved in experiment E2 were almost the same with those achieved in experiment E1. It demonstrates that the presented method can take full advantage of the absolute constraints provided by GCPs, just like the conventional calibration method. The introduced tie constraints provided by tie points does not bring in negative influences on the absolute constraints. The achieved calibration accuracies thereby reached better than 1.0 pixel. With respect to the effects of tie constraints, we will discuss in the next section.

3.3 Geometric stitching accuracy analysis

In this section, the GaoFen-6 image (i.e., image 1) and the ZiYuan3-02 image (i.e., image 4) used in section 3.2 continued to be tested. The geometric stitching accuracies of the adjacent sub-images achieved with the estimated external and internal parameters in experiments E1 and E2 were comparatively analyzed. Specially, each tie point in the left sub-image of a pair of adjacent sub-images was firstly projected onto the ground, and a ground point was obtained. Then, the ground point was further projected onto the right sub-image, and a projected image point was obtained. Finally, the RMSEs of the image-space coordinate residual errors between the projected image points and the corresponding points were calculated and taken as the geometric stitching accuracy, as listed in Tables 4 and 5.

Table 4. Geometric stitching accuracies of the GaoFen-6 sub-images.

View Table | View all tables in this article

Table 5. Geometric stitching accuracies of the ZiYuan3-02 sub-images.

View Table | View all tables in this article

In experiment E1, the geometric stitching accuracies of the adjacent sub-images achieved by the conventional calibration method differed from approximately 0.1 pixel to 1.1 pixels. It demonstrates that the estimated internal parameters of different CCDs had different levels of deviations, although the calibration accuracies of all the sub-images reached a satisfied level. In Table 4, a representative example is the stitching accuracy of GaoFen-6 sub-image 2 and sub-image 3 in narrow block 2. The left edge of sub-image 3 had no GCPs, as shown in Fig. 3(c). The estimated internal parameters of CCD 3 were not effectively constrained in the left edge, and they were likely to be deviated. The stitching accuracy thereby reached worse than 0.9 pixel, as could be expected. Another example is the stitching accuracy of GaoFen-6 sub-image 6 and sub-image 7 in narrow block 1. The left and the right edges of both sub-images 6 and 7 have sufficient GCPs, but the stitching accuracy reached worse than 1.0 pixel. The residual errors of tie points showed an obvious systematic characteristic, as shown in Fig. 5. Such a result should be associated with the GCP distributions. In fact, we are unable to guarantee that the extracted GCPs are exactly evenly distributed. It is also very difficult for us to theoretically define a standard to judge whether the GCPs are evenly distributed or not. Therefore, in practice, we often select several narrow image blocks in the valid image block shown in Fig. 1, so that an optimal calibration result (i.e., optimal calibration accuracy and optimal stitching accuracy) can be obtained. Such a selection method is often feasible and effective, when an ORSS has fewer linear-array CCDs. When the number of CCDs increases, it is expected that such a selection method will be out of operation. The reason is that it is difficult to guarantee that the calibration accuracies of all the sub-images and the stitching accuracies of all the adjacent sub-images can be simultaneously satisfied in a same selected image block.

 

Fig. 5. Residual error distributions of tie points between GaoFen-6 sub-images 6 and 7 in (a) x and (b) y directions in experiment E1.

Download Full Size | PDF

In experiment E2, tie constraints provided by tie points were introduced in the presented calibration method. With the help of tie points between the adjacent sub-images, all the CCDs could be logically connected into a complete CCD. Essentially, with the introduced tie constraints provided by tie points, the look angle models of all the CCDs were connected into a piecewise polynomial model. Hence, the estimation deviations in the internal parameters could be effectively eliminated, and the geometric stitching accuracies of all the adjacent sub-images were improved to better than 0.2 pixel. To be specific, the stitching accuracy of GaoFen-6 sub-image 2 and sub-image 3 in narrow block 2 was improved from 0.911 pixel to 0.082 pixel. The stitching accuracy of GaoFen-6 sub-image 6 and sub-image 7 in narrow block 1 was improved from 1.055 pixels to 0.105 pixel. It demonstrates that the introduced tie constraints could effectively constrain the left and the right edges of a sub-image, even when the selected narrow image block lacks GCPs or the extracted GCPs are not evenly distributed. Besides, the introduced tie constraints do not have negative influences on the geometric calibration accuracies, as listed in Tables 2 and 3. Optimal calibration accuracies and optimal stitching accuracies of the GaoFen-6 and the ZiYuan3-02 sub-images were thereby achieved in both the selected narrow image blocks.

3.4 Orientation accuracy analysis of stitched images

In this section, in order to comprehensively demonstrate the feasibility of the presented method, the GaoFen-6 image (i.e., image 1) and the ZiYuan3-02 image (i.e., image 4) used in section 3.2 continued to be tested. Firstly, the external and internal parameters estimated with narrow image block 1 were used to geometrically stitch all the sub-images, as performed in [12]. Then, dense GCPs in a narrow image block of the stitched image were extracted from the reference DOMs and DEMs. The extracted GCPs in the stitched GaoFen-6 image were taken as an example and shown in Fig. 6. Finally, the bias compensation based on a rational function model (RFM) in [23] was performed, and the sensor orientation accuracy was achieved. For convenient comparison, the orientation accuracies achieved with the estimated parameters in both experiment E1 and experiment E2 were listed in Table 6. The residual error distributions of GCPs in the stitched GaoFen-6 image were taken as an example and shown in Figs. 7 and 8.

 

Fig. 6. GCP distributions in the stitched GaoFen-6 image in (a) experiment E1 and (b) experiment E2.

Download Full Size | PDF

 

Fig. 7. Residual error distributions of GCPs in the stitched GaoFen-6 image in (a) x and (b) y directions in experiment E1.

Download Full Size | PDF

 

Fig. 8. Residual error distributions of GCPs in the stitched GaoFen-6 image in (a) x and (b) y directions in experiment E2.

Download Full Size | PDF

Table 6. Sensor orientation accuracies of the stitched GaoFen-6 and ZiYuan3-02 images.

View Table | View all tables in this article

From the experimental results in Table 6, we could see that the sensor orientation accuracies of the stitched GaoFen-6 and ZiYuan3-02 images achieved in experiment E1 were almost the same with those in experiment E2. It seems that the geometric stitching errors of the adjacent sub-images did not have any negative influences on the sensor orientation accuracies of the stitched images. Even from the residual error distributions of GCPs in Figs. 7 and 8, it was difficult for us to see the stitching errors. The major reason should be that the GCP errors and the stitching errors were mixed together. Due to the GCP extraction errors, the DOM errors, and the DEM errors, it was inevitable that the extracted GCPs had random errors. The magnitude of these random errors was close to or even larger than the magnitude of the stitching errors, which could be proved by the results in Table 2 to Table 5. In such a case, we were unlikely to separate the systematic stitching errors from the random GCP errors.

In fact, the geometric stitching errors were undoubtedly propagated into the stitched images. Taking the stitched GaoFen-6 image as an example, we could see from Fig. 9 that the stitched image did have stitching errors. In order to clearly evaluate the stitching errors, a narrow image block in the stitched image in experiment E1was firstly selected. Then, dense tie points between the stitched image in experiment E1 and that in experiment E2 were matched. Finally, relative orientation between the two stitched images was performed, and the relative orientation error distributions of tie points were shown in Fig. 10. More specially, each tie point in the stitched image in experiment E1 was first projected onto the ground, and an object point was obtained. Second, the obtained object point was further projected onto the stitched image in experiment E2, and a projected image point was obtained. Then, an affine transformation model was used to describe the geometric relationship between the projected image points and the corresponding image points. The remained errors after the affine transformation compensation were considered as the relative orientation errors.

 

Fig. 9. Partial enlarged map of the stitched GaoFen-6 images in (a) experiment E1 and (b) experiment E2.

Download Full Size | PDF

 

Fig. 10. Residual error distributions of tie points in (a) x and (b) y directions in the stitched GaoFen-6 image.

Download Full Size | PDF

In Fig. 10, we could see that the relative orientation errors in the stitched GaoFen-6 images were obviously discontinuous. These orientation errors were theoretically caused by two factors. One was the stitching errors in the stitched image in experiment E1, and the other was the stitching errors in the stitched image in experiment E2. Comparing the relative orientation errors in Fig. 10 with the stitching accuracies in narrow block 1 in Table 4, we could see that the orientation errors in Fig. 10 were consistent with the stitching accuracies in experiment E1. We could thereby conclude that the vast majority of the relative orientation errors were caused by the stitching errors in experiment E1 rather than those in experiment E2. It demonstrates that the geometric stitching errors of the adjacent sub-images indeed propagated into the stitched images, although it was difficult to clearly detect these stitching errors. The propagated stitching errors will undoubtedly decrease the geometric quality of the stitched images. With the presented method, the geometric stitching errors of the adjacent sub-images could be effectively eliminated. All the sub-images could then be seamlessly stitched into a complete image. To be expected, the geometric quality of the stitched images could be improved.

3.5 Performance analysis of estimated camera parameters

The most important objective of in-orbit geometric calibration is to obtain accurate camera parameters. The estimated external and internal camera parameters should be able to improve the geometric quality of another images. On basis of this, two GaoFen-6 images (i.e., images 2 and 3) and one ZiYuan-3 image (i.e., image 5) in Table 1 were used to analyze the performance of the camera parameters estimated with narrow image block 1 in section 3.2. For convenient comparison, the following experimental results achieved with the estimated camera parameters in experiments E1 and E2 were respectively denoted as scenarios S1 and S2.

In this section, the performance of the estimated camera parameters was evaluated in two aspects: geometric stitching and sensor orientation. For the former, the estimated camera parameters were used to evaluate the geometric stitching accuracies of the tested GaoFen-6 and ZiYuan3-02 sub-images, as listed in Tables 7 and 8. We could see that the geometric stitching accuracies in Tables 7 and 8 were almost the same with those in Tables 4 and 5. It demonstrates that the estimated camera parameters in in-orbit geometric calibration directly affects the stitching accuracies of another sub-images. In experiment E1, the camera parameters estimated with the conventional calibration method were deviated. The stitching accuracies achieved in scenario S1 were thereby worse. Similarly, in experiment E2, the deviations of the estimated camera parameters were effectively eliminated by the presented method. The stitching accuracies achieved in scenario S2 were accordingly improved.

Table 7. Geometric stitching accuracies of the GaoFen-6 sub-images

View Table | View all tables in this article

Table 8. Geometric stitching accuracies of the ZiYuan3-02 sub-images

View Table | View all tables in this article

For the sensor orientation evaluation, the estimated camera parameters were firstly used to geometrically stitch all the sub-images. Dense GCPs in a narrow image block of the stitched image were then extracted from the reference DOMs and DEMs. Finally, the RFM-based sensor orientation was performed. It was a pity that we did not have the reference DOMs and DEMs covering the tested GaoFen-6 images (i.e., images 2 and 3). Only the ZiYuan3-02 image (i.e., image 5) were thereby tested, and the achieved sensor orientation accuracies were listed in Table 9. Similarly, in order to clearly demonstrate the negative influences of the geometric stitching errors, relative orientation between the two stitched images in scenarios S1 and S2 was performed. The residual error distributions of tie points were shown in Fig. 11. We could see from Table 9 that the achieved orientation accuracies in scenario S1 were almost the same with those in scenario S2. Such a conclusion was also consistent with the experimental results in Table 6. From the results in Fig. 11, we could see that the geometric quality of the stitched images was affected by the geometric stitching errors as well. Likewise, the geometric quality decrease was mainly caused by the stitching errors in scenario S1.

 

Fig. 11. Residual error distributions of tie points in (a) x and (b) y directions in the stitched ZiYuan3-02 image.

Download Full Size | PDF

Table 9. Sensor orientation accuracies of the stitched ZiYuan3-02 images

View Table | View all tables in this article

From the above results, we could conclude that in-orbit geometric calibration plays a very important role in the ground processing of ORSSIs. The deviations of the camera parameters estimated in in-orbit geometric calibration will play the part of systematic errors and propagate into another images. To be expected, the geometric quality of these images will be unable to reach an optimal level. Therefore, in order to improve the geometric quality of ORSSIs, we should improve both the geometric calibration accuracy and the geometric stitching accuracy in in-orbit geometric calibration as possible as we can.

4. Conclusion

In in-orbit geometric calibration, the field-dependent geometric calibration methods are popular methods at present and widely used in practice. In such methods, sufficient highly precise and evenly distributed GCPs in all sub-images are often necessary to estimate accurate camera parameters of all CCDs. However, due to some uncontrollable factors such as land surface changes, lack of textures, radiometric differences, and GSD differences, we cannot always obtain sufficient evenly distributed GCPs in the valid image blocks for all sub-images, especially when more and more CCDs are placed on the camera focal plane. In this study, a feasible in-orbit geometric calibration method for multi-linear array ORSSs with tie constraints is presented. In the presented method, both GCPs in all sub-images and tie points between all adjacent sub-images are employed. GCPs are employed to provide absolute constraints, and tie points are employed to provide tie constraints. With the help of tie constraints, the left and the right edges of all sub-images could be effectively constrained. The internal camera parameters of all CCDs can be precisely and simultaneously estimated, even if sufficient evenly distributed GCPs in some sub-images are unavailable.

The presented calibration method was tested on three GaoFen-6 images and two ZiYuan3-02 images. Compared with the conventional method, the experimental results showed that the presented method could effectively constrain the left and the right edges of CCDs when we estimate the internal camera parameters. The deviations of the estimated internal parameters can be then eliminated. Consequently, the geometric stitching accuracy of the adjacent sub-images can be effectively improved. The geometric quality of the stitched images can be accordingly improved. Meanwhile, the introduced tie constraints in the presented method have no negative influences on the absolute constraints provided by GCPs; that is, the geometric calibration accuracy achieved by the presented method remains the same with that achieved by the conventional method. As such, the experimental results demonstrated the feasibility and effectiveness of the presented calibration method.

In this study, the tested GaoFen-6 and ZiYuan3-02 cameras are linearly designed. With the tested GaoFen-6 and ZiYuan3-02 images, the presented calibration method achieved satisfied experimental results. Theoretically, the presented method is also suitable for noncollinear satellite cameras. Of course, more ORSSIs collected by noncollinear cameras are needed to evaluate the feasibility and effectiveness of the presented method in the future.