1. INTRODUCTION

Metamer mismatching [1] refers to the fact that two objects reflecting metameric light under one illumination may reflect nonmetameric light under a second, so two objects having the same color under one illuminant may have different colors under a second. Metamer mismatching has important consequences for human vision and computer vision since the light illuminating an object is frequently changing, for example, as it moves from direct sun to shadow, or when the lights are turned on in a room, or the image is taken at a different time of day, or the object is viewed under fluorescent light at one moment and tungsten light at another.

Foster et al. [2] investigated the frequency with which nonidentical reflectances form metameric pairs under various daylight illuminants and found it to be rare. However, the relative frequency with which two objects reflecting metameric light under one illuminant then reflected nonmetameric light under a second illuminant was much higher. Based on 50 spectral-reflectance images of natural scenes under various phases of daylight ranging from correlated color temperatures of 4000 to 25,000 K, they found that the frequency of occurrence of a metameric pair under one illuminant becoming distinguishable under a second illuminant was ${10}^{-2}$ to ${10}^{-1}$. In a subsequent study, Feng and Foster [3] employed the conditional entropy of colors to predict the frequency of metamerism in natural scenes and again found it to be relatively low. Morovic and Haneishi [4] calculated the probabilities of metamer mismatching in 40 multispectral images with the illuminants changed from D65 to 173 different spectral power distributions and found a similar low frequency of metameric pairs. Prasad and Wenhe [5] consider the issue of metamer mismatching between three digital camera models and the human observer.

In contrast to these studies of the frequency of metamer mismatching in a typical scene, our focus here is not on the frequency of metamer mismatching but rather on the potential amount of metamer mismatching when it occurs. Given only the color signal produced in response to light reflected from an object of unknown reflectance under a given illuminant, we address the issue of what precisely can be said about what the color signal from that same object is likely to be under a different illuminant. Logvinenko et al. [6] addressed this issue in terms of the degree of metamer mismatching (i.e., the volume of the metamer mismatch volumes/bodies) that can arise in theory. Here, we focus on the degree of metamer mismatching that arises in practice.

As Logvinenko et al. [6] argue, metamer mismatching imposes limits on color constancy since even when the full spectra of the two illuminants are known there is an inherent ambiguity in terms what a given color signal (i.e., camera sRGB or CIE XYZ coordinates) under a first illuminant will become under a second illuminant. In the color constancy and computer vision fields, it is generally assumed that the color of an object is an intrinsic property of the object, and hence the focus is on discounting the effects of the illuminant in order to recover the intrinsic color of the object. The intrinsic color is frequently expressed as the color signal that would be obtained from the object under some standard, “canonical” illuminant. However, Logvinenko [7] proves that color cannot be an intrinsic property of an object. His argument is straightforward: If two objects, A and B, are metameric matches (i.e., reflect light that generates an identical color signal) under the first illuminant but do not match under the second illuminant, then which of the two objects is to be considered the carrier of the “intrinsic” color? Clearly, a single color signal that becomes two different color signals cannot possibly map to some unique “intrinsic color” coordinate.

Metamer mismatching means that a color signal under a first light can become any color signal from an infinite convex set of color signals under a second light. This convex set is often called the metamer mismatch volume, or sometimes, the metamer mismatch body. In the present context the latter terminology is preferred because we wish to explore the volumes of metamer mismatch volumes/bodies and the multiple meanings of “volume” in a phrase such as “the volume of the metamer mismatch volume” can become very confusing. We will refer instead to “the volume of the metamer mismatch body” with the body referring to the (three-dimensional) convex set of color signals and the volume being the volume of that convex set.

To establish the extent of metamer mismatching in practice, we examine empirically the metamer mismatch bodies arising under several typical illumination changes for a large set of reflectance spectra obtained from multispectral images and other datasets of reflectances. A preliminary study [8] showed how the empirical mismatch bodies varied systematically with Munsell chroma and value. The present study expands the set of reflectances and illumination conditions used for testing and also compares the empirical metamer mismatch bodies to the theoretical metamer mismatch bodies calculated using the method of Logvinenko et al. [9].

The theoretical metamer mismatch body is based on the premise that the reflectances generating color signals on the boundary of the object color solid are special two-transition reflectances. The reflected values of such two-transition reflectances are either zero or one and make at most two transitions from zero to one or vice versa across the visible spectrum. Clearly, two-transition reflectances with either zero or one values seldom appear in practice, but there is no obvious, nonarbitrary way (e.g., an arbitrary degree of “smoothness”) to constrain the set of reflectances further. The tests reported here show that the average volumes of the empirical and theoretical metamer mismatch bodies are clearly related, with the empirical bodies being substantially smaller than the theoretical ones.

Given a color signal arising from an object under a first illuminant, all that can be said definitely about its color signal under a second illuminant is that it could be any one of the color signals within its metamer mismatch body. Despite this lack of certainty, it is frequently the case (e.g., when white balancing an image) that we need to predict what the object’s color signal is most likely to be under a second illuminant. Of course, any prediction can only be a guess since any of the color signals within the metamer mismatch body is a plausible answer. However, when forced to choose, what is a good choice to make? We explore this issue by making predictions based on several different measures (e.g., mean, median, centroid) of the metamer mismatch body and compare the mean prediction error to that obtained using the CAT02 [10] chromatic adaptation transform that underlies the CIECAM02 [11] color appearance model and to Mirzaei and Funt’s [12] Gaussian metamer method of color signal prediction.

2. REFLECTANCE AND ILLUMINANT SPECTRA

In order to analyze the effects of metamer mismatching in practice, we construct a large dataset of reflectance spectra along with a sampling of illuminant spectra. The reflectance data are divided into disjoint training and test sets. Even though there is no machine learning involved, we use the term “training set” since we will be predicting results for the test data based on a prior set of reflectance data.

A. Dataset of Training Reflectances

A large dataset of spectral reflectances was assembled by gathering spectra from various sources in order to create a representative dataset of the spectral reflectances of natural and man-made objects that are likely to occur in practice. All the spectral reflectances are sampled from 400 to 700 nm at a 10 nm sampling interval.

The dataset was assembled from six main sources. The first group includes 11 multispectral images consisting of rocks, trees, leaves, grass, earth and urban scenes, and medieval and early modern illustrated works [2]. The second group includes 32 multispectral images [13] containing scenes of faces, hair, paints, food, drinks, and some other natural and man-made items. The third group includes 13 multispectral images [14] containing scenes of people, houses, hands, fruits, flowers, and other natural and man-made items. The fourth group includes nine hyperspectral images containing scenes of textile, wood, leaves, painting, paper, and skin [15]. The fifth group includes 21 multispectral images mainly composed of different man-made items [16]. These five groups of images were all acquired with multispectral imaging systems. The sixth group includes spectral reflectances of man-made, natural, and industrial objects, which were measured using a spectrophotometer [17]. In total this leads to a set of 35,420,169 reflectance spectra.

Since many of these reflectance spectra are from multispectral images, it is likely that there will be many extremely similar or duplicate spectra in the datasets. To eliminate these similar/duplicate spectra, the numerical precision of the spectral data is first reduced to integer values 0–50 (i.e., multiplied by 50 and rounded), and then any spectra that are identical at that level of precision are removed. The spectra retained are kept at their full, initial precision. Although a little ad hoc, computationally this method is much faster than computing a distance metric (e.g., angular difference) between the approximately ${10}^{15}$ pairs of spectra. The final dataset contains 25,303,486 distinct reflectance spectra.

B. Dataset of Test Reflectances

For testing, a second, smaller set of reflectance spectra is created by combining the 1600 reflectances of the Munsell glossy edition [18] papers, the 1950 reflectances of the Natural Color System (NCS) [19] samples, along with the 218 reflectances from the “Natural Colors” subset of the University of Eastern Finland’s (UEF) spectral database [20] and 1301 reflectances of natural objects in the ASTER Spectral Library from the Jet Propulsion Laboratory (JPL) [21]. The total test set contains 5,069 reflectances.

C. Chromaticities of the Reflectances

As an indicator of how complete the set of reflectances is we computed the CIE1931 2-deg observer XYZ values under CIE D65 (daylight) of all the spectral reflectances in the training and test sets and plotted them in $xy$-chromaticity space [i.e., $x=X/(X+Y+Z)$, $y=Y/(X+Y+Z)$] as shown in Fig. 1. The plot shows that the full training set (black dots) covers a very significant portion of the $xy$-chromaticity diagram.

 

Fig. 1. Chromaticities of the reflectances in the various datasets under D65 plotted in the $xy$-chromaticity diagram. (a) Black dots indicate the samples from the full training dataset; red (gray in grayscale reproduction) dots are the Munsell papers; and green (white) dots indicate the Finland “Natural Colors” reflectances. (b) Black dots as in (a), bright purple (gray) dots indicate the NCS papers, and cyan (white) dots indicate the JPL reflectances.

Download Full Size | PDF

D. Illuminant Spectra

Eleven illuminants, namely, the CIE standard illuminants A, D50 (5000 K), D65 (6504 K), D100 (10000 K), D150 (15000 K), D200 (20000 K), F4, F8, and F11, along with two cellular/mobile phone LEDs, are used in evaluating the metamer mismatch bodies and color-signal prediction results. They were chosen as a representative test set since A is a typical tungsten light bulb; D50, D65, D100, D150, and D200 are typical daylights with different correlated color temperatures; and F4, F8, and F11 are typical fluorescents with varying degrees of spikiness in their spectra. The two LEDs are typical light sources widely used in cellular/mobile phones. The relative spectral power distributions of these illuminants are shown in Fig. 2. In the calculations described below all the illuminants are first normalized so that CIE Y is 100 for the ideal reflector.

 

Fig. 2. Relative spectral power distributions of the 11 illuminants used for testing. (a) Illuminants D50, D65, D100, D150, and D200; (b) illuminants F4, F8, F11, LED1, and LED2.

Download Full Size | PDF

3. METAMER MISMATCH BODY VOLUMES

Since the range of possible color signals a given color signal under the first illuminant can become under the second illuminant is only limited by the metamer mismatch body, an interesting question is, How does the volume of the metameric mismatch body vary with the initial color signal? To address this question, we computed both the theoretical and empirical metameric mismatch bodies for each of the 1600 reflectances from the Munsell color atlas for a change from each of the 10 other illuminants (Fig. 2) to D65.

A. Theoretical Metamer Mismatch Body Volumes

Using the method and code from Logvinenko et al. [9] we calculated the volumes of the metamer mismatch bodies for the color signals obtained from each of the 1600 Munsell reflectances for the illuminant condition of an illuminant change from D50 to D65 (denoted D50→D65), and similarly each of the conditions D100→D65, D150→D65, D200→D65, F4→D65, F8→D65, F11→D65, A→D65, LED1→D65, and LED2→D65. Figure 3 shows how the theoretical volume varies with the value and chroma of the Munsell samples for the five illuminant conditions D50→D65, D200→D65, A→D65, F4→D65, and LED1→D65. Each red dot is a data point representing the average volume of the metamer mismatch bodies obtained for all hues of the samples having a given Munsell value and chroma.

 

Fig. 3. Average volumes of theoretical metamer mismatch bodies obtained for all Munsell hues plotted as a function of Munsell chroma and value for the illuminant conditions D50, D200, A, F4, and LED1, respectively, changing to D65. Red dots indicate the actual data points. The surface is interpolated through the data points to aid in visualization. The plot colors are those provided by Matlab’s “parula” colormap and are provided simply to aid in visualization. They indicate relative magnitude.

Download Full Size | PDF

Comparing the different panels of Fig. 3, it is clear that the overall shape of the plots is similar across all the illuminant conditions. Each plot clearly peaks for the achromatic (i.e., chroma zero) Munsell paper having value 7.5 and then decreases with increasing chroma. The achromatic sample with value 7.5 actually is the neutral gray with approximately constant reflectance of roughly 50%, as shown in Fig. 4. The results are consistent with those of Logvinenko et al. [9], showing that the theoretical metamer mismatch body is generally larger for color signals near the center of the object color solid (i.e., where the color signal of the light from the ideal 50% reflector resides) and zero for color signals on the boundary of the object color solid.

 

Fig. 4. Spectral reflectances of the neutral gray Munsell papers N 1/, N 7.5/, and N 9/ of value 1, 7.5, and 9, respectively.

Download Full Size | PDF

The figures also show that the average metamer mismatch body volumes decrease smoothly from neutral gray to the highest chroma samples forming the boundary of the Munsell atlas. Although the plot shapes are qualitatively similar, quantitatively the size of the metamer mismatch bodies depends strongly on the illumination condition. As is evident from Table 1, the lights of similar chromaticity can lead to mismatch bodies of very different sizes, with the size more dependent on the type of light than its chromaticity.

Table 1. Comparison of the Mean Volumes of the Theoretical Metamer Mismatch Bodies for the 1600 Munsell Samples for a Change from Each of the Different Illuminants to D65^a

View Table | View all tables in this article

B. Empirical Metamer Mismatch Body Volumes

The size of the theoretical metamer mismatch bodies shows a very distinct dependence on chroma and value but is based on the limiting case of two-transition reflectance functions. Can we expect similar trends in practice? To investigate this question, we calculated metamer mismatch bodies empirically using the large training set of reflectances described in Section 2.

Although the training set contains 25,303,486 distinct reflectances, it is still limited, and for many color signals there are not enough exact metameric matches to compute a metamer mismatch body reliably. Hence, we relaxed the definition of a metameric match slightly and consider any color signal within a small distance $T$ to be a metameric match. In other words, two CIE XYZ color signals, ($X_c,Y_c,Z_c$) and ($X_i,Y_i,Z_i$), will be considered metameric matches whenever

Using this definition of metameric matching, given a color signal ($X_c,Y_c,Z_c$), we find all the reflectances in the training set generating metameric color signals under the first illuminant. Using this set of reflectances, the empirical metameric mismatch body is then determined as the convex hull of the color signals generated by these reflectances under the second illuminant.

Threshold $T$ is chosen so that there are enough approximately metameric samples to compute the convex hull of the metamer mismatch body reliably. In particular, $T$ is chosen so that at least 60 approximately metameric samples are found for 90% of the Munsell samples. The trade-off is that a small $T$ means more accurate metamers and potentially a more accurate metamer mismatch body, but too restrictive a $T$ leads to too few samples, and hence an inaccurate estimate of the metamer mismatch body. $T=1$ is used in the calculations reported in this section.

Figure 5 plots how the volume of the empirical metameric mismatch body varies with the chroma and value of the Munsell samples for each of the illuminant conditions D50→D65, D200→D65, A→D65, F4→D65, and LED1→D65. In comparing the empirical volumes depicted in Fig. 5 to the theoretical volumes from Fig. 3 it is clear that the volumes of the empirical metamer mismatch bodies are generally much smaller but show the same general pattern—peaking for the achromatic Munsell paper having value 7.5 and generally decreasing with increasing chroma.

 

Fig. 5. Volumes (averaged across all Munsell hues) of the empirical metamer mismatch bodies as a function of Munsell chroma and value for the illuminant conditions (a) D50→D65, (b) D200→D65, (c) A→D65, (d) F4→D65, and (e) LED1→D65. Red dots indicate the actual data points with the surface interpolated through the data points to aid in visualization. The plot colors are as in Fig. 3.

Download Full Size | PDF

Table 2 compares—for each of 10 illumination conditions—the mean empirical volume to the mean theoretical volume. The mean in each case is taken over all samples in the test set. The table also lists the mean number of approximately metameric ($T=1$) samples found. For some test samples, too few metamers were found in the training set to compute the empirical metamer mismatch body reliably. In particular, if fewer than 60 metamers were found, then the test sample was excluded from further consideration for the given illumination condition. The numbers excluded in this way are listed in Table 2. The maximum fraction excluded is approximately 22%.

Table 2. Comparison of the Mean Empirical Volumes to the Mean Theoretical Volumes for the 10 Illumination Conditions^a

View Table | View all tables in this article

Based on the data from Table 2, Fig. 6 plots the cube root of the empirical volume as a function of the cube root of the theoretical volume, where a linear fit has R-squared 0.90. Although the metamer mismatch bodies tend to be more ellipsoidal than spherical, the cube root provides an approximate measure of a metamer mismatch body’s “diameter” since volume varies as diameter cubed. The largest color difference between any two samples in a metamer mismatch body can be expected to relate more closely to the body’s diameter than to its volume. The relatively shallow slope (0.15) of the line indicates the “diameter” of the empirical bodies is 15% of that of the corresponding theoretical bodies.

 

Fig. 6. Comparison across 10 different illumination conditions of the mean of the cube roots of the volumes (i.e., mean of the body “diameters”) of the empirical metamer mismatch bodies as a function of mean of the cube roots of the volumes of the theoretical metamer mismatch bodies for the Munsell samples. The linear fit shown has slope of 0.15 with R-squared of 0.90.

Download Full Size | PDF

4. COLOR SIGNAL PREDICTION METHODS AND RESULTS

As mentioned above, given a color signal under one illuminant, all that can be definitively determined about what the color signal will become under a second illuminant is that it will lie within the theoretical metamer mismatch body. Of course, if the reflectance that led to the given color signal is known, then the new color signal can be simply calculated. However, in human vision and color imaging the reflectance is not available, and any prediction must be made based on the color signal alone. We describe a new method of making such a prediction based on the properties of the empirical metamer mismatch body and compare it to existing methods of color-signal prediction.

A. Metamer-Based Prediction Method

Mirzaei and Funt [12] proposed a method of color-signal prediction based on relighting a “wraparound Gaussian metamer.” Given a color signal under the first illuminant, the idea is to find a Gaussian-like (the precise details are irrelevant for the present discussion) reflectance function producing that same color signal under the first illuminant and then to calculate what that reflectance’s color signal would be under the second illuminant. They report excellent results using this Gaussian metamer (GM) method.

The GM method would appear to be limited in that the form of the metameric reflectance is fixed as something Gaussian-like. In comparison, the empirical metamer mismatch body is based on relighting the many reflectances from the training set, producing color signals that are approximately metameric to the given color signal under the first illuminant. The training set also contains only real reflectances, in other words, ones measured in practice rather than an idealized Gaussian-like reflectance function. We hypothesize that basing the color signal prediction on the properties of the empirical metamer mismatch body rather than a single idealized reflectance will lead to more accurate predictions on average.

The empirical metamer mismatch body represents the range of possible color signals that might arise under the second illuminant so the centroid of the body’s convex hull is one method of predicting the color signal under the second illuminant. Other choices we investigate are the mean of the possible color signals under the second illuminant and, similarly, their median.

In the previous section, the threshold $T=1$ was used to ensure there would be enough metameric samples in the training set to estimate the metamer mismatch body accurately. For color signal prediction, we expand the set of reflectances and reduce the threshold to $T=0.3$. We found the lower threshold resulted in better performance. The smaller threshold means that the samples classed as metamers are closer to being true metamers, and this leads to better predictions. In any case, the results do not depend very strongly on the choice of threshold.

We also expanded the set of reflectances by adding scaled versions of the reflectances in the original dataset because inspection of the dataset revealed that the majority of the reflectances $r(\lambda )$ were “dark,” having $\max (r(\lambda ))<0.5$ for $400\mathrm{nm}\le \lambda \le 700\mathrm{nm}$. To increase the total number of spectra and to better represent “brighter” reflectances, for each reflectance in the original dataset we added reflectances $r^{\prime }(\lambda )=2r(\lambda )$ so long as $\max (r^{\prime }(\lambda ))\le 1.0$ for $400\mathrm{nm}\le \lambda \le 700\mathrm{nm}$. The expanded dataset contains 41,941,743 distinct reflectance spectra. Better predictions are obtained with this expanded dataset than the original dataset. In the previous section, we refrained from using the expanded dataset in our analysis of the volumes of the metamer mismatch bodies since there is no guarantee that such scaled reflectances will be found in practice, and we did not want them to affect the volume estimates. In the case of color signal prediction, however, what matters is the accuracy of the resulting predictions.

We also found that outliers were affecting the prediction results. In particular, some of the reflectances in the dataset that were obtained from the multispectral images contained very large spikes at individual wavelengths. Such spikes are likely caused by sensor noise and not from actual scene reflectances. These noise samples lead to outlier CIE XYZ values under the second illuminant that are far from the majority of the samples defining the metamer mismatch body. For color signal prediction, such outliers were removed using the median absolute deviation method [22]. Note that the average empirical volumes reported in the previous section include these spiky spectra since we did not want to prejudge what is and is not a naturally occurring reflectance. In contrast to the situation of color signal prediction where outliers matter because a prediction is made on the basis of a single metamer mismatch body, outliers will have little effect on the final average volumes since the averaging is over thousands of samples.

B. Prediction Results

The spectral reflectances of the Munsell, NCS, UEF Natural, and JPL datasets described above are used for testing. These datasets are distinct from the training set. Predictions are made for a change from illuminants A, D50, D100, D150, D200, F4, F8, F11, and two LEDs to D65 as the “canonical” illuminant. The centroid, mean, and median methods were all tested. The results for all three methods are comparable, but the centroid method generally outperforms the others, so only the results for it are reported here.

For comparison, the GM method and the von Kries–based CAT02 chromatic adaptation transform are tested as well. CAT02 is the chromatic adaptation step underlying the CIECAM02 color appearance model. CAT02 includes a spectral sharpening transform [23]. Chong et al. [24] propose a tensor-based method of choosing the basis for the diagonal transform. The prediction error is measured using the CIEDE2000 [25] color difference measure.

The results for the full test dataset of reflectances are listed in Table 3. The results indicate that the prediction accuracy of the GM method is higher than that of the CAT02 method, which is consistent with the conclusion of Mirzaei and Funt [12]. It is also clear that the proposed centroid method outperforms both the CAT02 and GM predictions in almost all cases. However, evaluating performance based on the mean and median statistics needs to be done with a note of caution. As Hordley and Finlayson [26] point out, these error distributions are generally not normally distributed because they are bounded by zero on the left. As a further way to evaluate the relative performance, the prediction errors for the three methods are histogrammed in Fig. 7. The distribution again shows that the centroid method outperforms the other two methods.

Table 3. Color Signal Prediction Error of the Three Methods Each Applied to the Combined Set of Test Reflectances and Reported in CIEDE2000 (Mean, Median, 95th Percentile, Standard Deviation) for the 10 Illuminant Conditions^a

View Table | View all tables in this article

 

Fig. 7. Histogram of the CIEDE2000 prediction errors for the centroid, GM, and CAT02 methods across the combined set of test reflectances and all 10 illuminant pairs. The height of each bar indicates the number of samples falling within the respective interval, [0,1), [1,2), [2,3), [3,4), [4,5), or [5,∞).

Download Full Size | PDF

In predicting what a color signal will become under a second illuminant, we can expect that the larger its metamer mismatch body, the greater the prediction error is likely to be on average—a larger body represents a wider range of possible answers, and any prediction method is forced to choose only one. Figure 8 shows that the prediction error does increase as expected with increasing size of the empirical metamer mismatch body. Plots (not shown) of the error as a function of the size of the theoretical metamer mismatch bodies are qualitatively similar.

 

Fig. 8. Mean prediction error in CIEDE2000 units as a function of the cube root of the volume of the empirical metamer mismatch body (i.e., body “diameter”) for the three prediction methods: (a) centroid method; (b) GM method; (c) CAT02 method.

Download Full Size | PDF

5. DISCUSSION

Based on a set of over 25 million spectral reflectances of real objects, estimates of the size of the potential metamer mismatch bodies were computed for the color signals generated from 5,069 test reflectances under 10 different illumination conditions. The average volumes of these empirically determined bodies were compared to the average volumes of the corresponding theoretically determined bodies and found to be roughly proportional but significantly smaller. The size of the bodies is important because metamer mismatching imposes a limit on the accuracy with which it is possible to predict the effect a change in illumination will have on a given color signal. The theoretical metamer mismatch body provides an upper limit on the size of a given metamer mismatch body, and the empirical body provides a measure of the lower limit. It is a lower limit since adding a reflectance spectrum to the training set will either lead to a color signal inside the current metamer mismatch body and have no effect, or else lie outside it and therefore increase its size. The empirical bodies also represent a lower limit since the reflectance data is based on a 10 nm sampling interval using samples having a roughly 10 nm bandwidth [2]. In effect, the measured spectra are smoothed versions of the real spectra, and any such smoothing will potentially reduce the calculated amount of metamer mismatching. Since it is real spectral power distributions that enter the eye, the empirical metamer mismatch bodies reported here are likely to underestimate the true amount of metamer mismatching to a certain extent.

As a general rule, the volumes of the metamer mismatch bodies (both empirical and theoretical) were found to decrease with increasing distance of the color signal from mid-gray, as can be seen in Fig. 5. Concomitantly, the average error in predicting how a given color signal will change with a change in illumination was also found to decrease with increasing distance for the color signal from mid-gray. In terms of predicting what a given color signal may change to under a new illuminant, the centroid of the empirical metamer mismatch body performs better overall than the GM method, which in turn outperforms CAT02.