Measurement of the lens optical transfer function using a tartan pattern

Matthew R. Arnison; David P. Morgan-Mar; Chris A. Deller; Peter A. Fletcher; Kieran G. Larkin

doi:10.1364/AO.50.002158

1. Introduction

An important property of high-quality lenses is the ability to produce images with high resolution and strong contrast. The standard objective measurement of the resolution and contrast of a lens is the optical transfer function (OTF). The OTF is defined as the complex ratio between the image contrast and the object contrast for a series of sinusoidal waves with increasing frequency. The modulus of the OTF, which is called the modulation transfer function (MTF), is used more often than the full complex OTF. The MTF is considered a powerful tool for understanding camera lens performance, because it acts as a well-studied bridge between theoretical lens modeling, objective lens measurement, and human perception.

The OTF has found many applications in lens design, prototyping, and production testing. The OTF is very sensitive to the fine details of lens construction, so it is a useful tool for establishing that a lens has been built within tolerance [1, 2]. The OTF is also useful for image restoration and computational photography [3, 4]. However, the OTF is a function of many variables of the imaging geometry, including image position $(x, y)$ , image focus z, transverse magnification $M_{T}$ , spatial frequency $(u, v)$ , lens focal length $f_{L}$ , relative lens aperture $A_{V}$ , and illumination wavelength λ (Fig. 1). In this nine-dimensional (9D) parameter space, the OTF varies smoothly but with complicated behavior, so efficient sampling of the OTF is a crucial issue.

One approach to sampling the OTF is to use the lens under test to image a high-contrast test object. The OTF can then be probed by precise mechanical scanning through the object or image space [5]. However, the test objects that are typically used, such as an illuminated pinhole or a slit, have relatively low signal strength at high spatial frequencies and poor overall optical efficiency. This inherently fragile design can only be made accurate with high illumination power and careful control of stray light. In addition, camera lenses are typically used at high magnifications, which simultaneously places tight constraints on instrument positioning at fine scales of the order of micrometers in the image space, and large scales on the order of meters in the object space (Fig. 1).

Digital sensors enable the second main method of OTF measurement. In this approach, a large test chart is imaged through the lens onto a digital sensor, and the captured image is compared with the ideal test chart image. The problems of positioning are reduced considerably by accurate chart fabrication and the precise layout of pixels across a digital sensor. However, the test patterns typically used for digital OTF measurement continue to have low signal strength at high spatial frequencies, which can be a significant source of noise in measurement results. For example, the International Organization for Standardization (ISO) 12233 spatial frequency response (SFR) method [6, 7] uses a slanted-edge test object (Fig. 2) for which the signal strength is inversely proportional to the spatial frequency.

The low efficiency of these test patterns imposes important constraints on the OTF measurement method. Restrictions may include inflexibility in the sampling of image space positions and spatial frequency orientations. The entire image needs to be well exposed to minimize the effect of sensor noise on the OTF measurement. For color OTF measurement, a convenient method is using a sensor Bayer pattern to filter the results into three color channels. However, this reduces the signal-to-noise ratio (SNR) by up to 50% compared to a monochrome measurement over the same image region. In many important situations the MTF value may be inherently low, for example when measuring lenses with significant aberrations, measuring the OTF in the image corners, or measuring the OTF as a function of defocus.

To improve the accuracy of OTF measurement using a digital camera, we propose a new test pattern that optimizes the signal strength in the spatial frequency domain. The pattern is the sum of multiple sinusoids at different frequencies and orientations. We call this the tartan pattern (Fig. 3). Although sinusoidal patterns have been used since the beginning of OTF measurement [8], including sums of sinusoids [9], we believe our tartan design is novel due to the combination of multiple orientations.

Two key problems in comparing the input test pattern with the captured output are alignment and compensation for the Fourier side lobes caused by the measurement window function. The multiple orientations in the tartan pattern allow accurate alignment, which in turn allows accurate measurement using a novel method we call side-lobe reduction [10].

In this article we introduce the tartan pattern design and the side-lobe reduction algorithm for precise OTF measurement from captured images of a tartan pattern. We use simulations and experiments to compare the accuracy of the tartan method with the ISO standard slanted-edge method. We conclude with an analysis of the trade-offs involved in selecting a particular MTF measurement technique.

2. Slanted-Edge Method

The test pattern for the slanted-edge method is a simple one-dimensional (1D) step function:

g (x, y) = {\begin{matrix} 0, & if x^{'} < 0, \\ 1, & if x^{'} \geq 0, \end{matrix}

where

x^{'}

is defined at an angle ψ with respect to the horizontal x axis,

x^{'} = x \cos ψ + y \sin ψ, y^{'} = - x \sin ψ + y \cos ψ .

The slanted edge is often imaged at a slight angle, such as $ψ = 5 °$ , to improve the superresolution of the detection algorithm. In this paper we will focus on the image corners where the SNR of a test pattern is most critical. For the image corners of a digital single-lens reflex (DSLR) camera with a sensor aspect ratio of $3 ∶ 2$ , the meridional and sagittal angles are

ψ_{M} = \arctan (2 / 3), ψ_{S} = \arctan (2 / 3) + \frac{π}{2} .

An example slanted edge with a meridional orientation in the top-right image corner is shown in Fig. 2.

Because imaging noise is typically white noise, evenly distributed over the spatial frequency spectrum, the signal level of a test pattern as a function of spatial frequency is a useful indicator of the SNR for OTF measurement. The Fourier transform of a 1D edge is

G (u, v) = \int g (x, y) e^{- 2 π i (u x + v y)} = {\begin{matrix} \frac{1}{2}, & if u^{'} = 0, \\ - \frac{i}{2 π u^{'}}, & if u^{'} \neq 0, \end{matrix}

where

(u^{'}, v^{'})

are the Fourier coordinates

(u, v)

rotated by the angle ψ. This shows that the signal strength of an edge is inversely proportional to the spatial frequency, as shown in Fig. 4a. While a binary edge is simple to fabricate, it has the disadvantage that it is impossible to adjust the signal strength in the spatial frequency domain. As the MTF drops at high frequencies, this exacerbates measurement problems at high frequencies.

3. Tartan Pattern Design

Most existing OTF measurement tools tend to cover a continuous range of spatial frequencies. This is the case for spatially discontinuous targets such as pinholes, slits, and edges. The benefit of continuous spatial frequency coverage is obtained through the joint sacrifices of sparse coverage of image positions and low SNR at high spatial frequencies.

Some OTF measurement tools use sine waves at discrete spatial frequencies. A sinusoidal wave in the image plane can be used to measure the OTF at a single spatial frequency. Because imaging noise is typically distributed throughout the spatial frequency domain while the entire input signal is concentrated at a single OTF coordinate, this single spatial frequency measurement achieves a high SNR. If multiple sinusoidal waves are used, then the SNR remains high compared with a spatially discontinuous target, and multiple spatial frequency measurements can be made at each image position. In practice, the OTF tends to vary slowly with spatial frequency, with extrema in the sagittal and meridional orientations. This suggests that a useful strategy is to sample a discrete set of spatial frequencies at two orientations. This is the strategy we followed in designing the tartan pattern (Fig. 3) [11].

The reflectivity of a general continuous tartan pattern $t_{C}$ can be described in the spatial domain using a sum of M cosines:

t (x, y) = 2 \sum_{j = 1}^{M} T_{j} [1 + \cos (2 π u_{j} x + 2 π v_{j} y + φ_{j}^{(T)})],

where

(x, y)

is the image position and each cosine is defined by an amplitude

T_{j}

, a spatial frequency

(u_{j}, v_{j})

, and a phase

φ_{j}

. If we wish to assign relative weights

W_{j}

to each cosine, then we can normalize each cosine amplitude using

T_{j} = W_{j} / (4 n_{t})

, where the normalization factor

n_{t}

ensures that all values of

t (x, y)

are between 0 and 1. If the phases

φ_{j}

are all 0, then

n_{t} = \sum_{j = 1}^{M} W_{j}

, but in general a solution for the value of

n_{t}

needs to be found numerically.

The two-dimensional (2D) Fourier transform of $t (x, y)$ is

T (u, v) = \iint t (x, y) \exp [- 2 π i (u x + v y)] d x d y = 2 δ (u, v) \sum_{j = 1}^{M} T_{j} + \sum_{j = 1}^{M} T_{j} \exp (i φ_{j}^{(T)}) δ (u - u_{j}, v - v_{j}) + \sum_{j = 1}^{M} T_{j} \exp (- i φ_{j}^{(T)}) δ (u + u_{j}, v + v_{j}),

where

δ (u, v)

is the 2D Dirac delta function. The signal strength of a tartan pattern is adjustable using the amplitudes

T_{j}

. If all of the amplitudes are equal, then the signal strength is constant for all of the spatial frequencies sampled. The signal strength at each sample decreases with the number of spatial frequencies that are sampled by the tartan pattern.

An overview of tartan design is shown in Fig. 3. We start the design of an ideal tartan pattern $t_{C}$ by choosing an image region in the top-right image quadrant and selecting six meridional cosines and six sagittal cosines for measurement. The tartan is designed for a sensor pixel size of $5 μm$ , giving a Nyquist frequency of 120 line pairs per millimeter (lppmm) in the meridional and sagittal orientations. The spatial frequencies are at 10, 30, 50, 70, 90, and $110 lppmm$ along each orientation axis. The positions $(u, v)$ for each peak are calculated by rotating frequencies in a meridional/sagittal coordinate system $(u^{'}, v^{'})$ using the relation $(u, v) = (u^{'} \cos ψ_{M} + v^{'} \sin ψ_{M}, u^{'} \sin ψ_{M} - v^{'} \cos ψ_{M})$ .

This creates a tartan pattern that has 25 peaks in the Fourier domain, including a peak at the $(0, 0)$ spatial frequency, which we call the direct current (DC) peak. The 24 non-DC peaks appear in pairs with Hermitian symmetry because the tartan pattern has only real values in the spatial domain. Disregarding the DC peak and the Hermitian peaks, we describe this tartan design as having 12 peaks, because it can be used to measure the OTF at 12 spatial frequencies.

The signal strength of the tartan design $t_{C}$ along the meridional axis in the spatial frequency domain is compared to the slanted-edge signal strength in Fig. 4a. The signal strength G for the slanted edge was calculated using Eq. (4). The signal strength $T_{C}$ for the tartan pattern was calculated using Eqs. (5, 6). In Fig. 4b the ratio $T_{C} / G$ of signal strength between the tartan and edge designs is plotted at the tartan cosine spatial frequencies. The tartan signal is significantly higher than the slanted-edge signal, with the amplitude ratio starting from 1.3 at $10 lppmm$ and climbing as high as 14.4 at $110 lppmm$ . In addition, this tartan design is sampling both meridional and sagittal orientations, while the slanted edge is only sampling the meridional orientation. However, the slanted edge is sampling a continuous range of spatial frequencies, while the tartan design is only sampling 12 spatial frequencies.

The SNR of the tartan pattern can be further increased in several ways: by reducing the number of cosines M, by increasing the cosine weights $W_{j}$ for high-frequency peaks to compensate for low MTF values in the lens, or by randomizing the cosine phases $φ_{j}^{(T)}$ . The phase randomization method for increasing the SNR was first described by Edgar and Bopardikar [12].

When the ideal tartan design $t_{C}$ is rendered to a grayscale image for printing, the pattern will vary continuously across the entire chart. This means that global alignment is required between the image region being analyzed and the ideal test chart to precisely compare the output image region with the input image. To replace this global alignment with a simpler local alignment, we can force the tartan pattern to be tiled. This can be achieved for a specific tile size in sensor pixels by rounding the spatial frequencies to values with an integer number of cycles across the vertical and horizontal tile size.

For example, with a sensor pixel size of $5 μm$ and a tile size of $100 \times 100$ in sensor pixels, the meridional spatial frequency ( $u^{'} = 10 lppmm$ , $v^{'} = 0 lppmm$ , $ψ = ψ_{M}$ ) has pixel axis coordinates of ( $u = 4.16$ , $v = 2.77$ ) in the discrete Fourier transform of a $100 \times 100$ pixel tartan window. To create a tiled tartan pattern, the spatial frequency of this peak would be rounded to ( $u = 4$ , $v = 3$ ). The same rounding operation should be applied to all 12 peaks to create a tiled tartan pattern $t_{T}$ , as shown in Fig. 3. This quantization creates a slight change in the spatial frequency and orientation being measured. However, the effect is deterministic, and it is largest for low spatial frequencies where the OTF has the slowest variation with spatial frequency, so there is no significant impact on OTF measurement accuracy.

The tartan tile $t_{T}$ needs to be scaled for printing. For example, the tile should be scaled 30 times for testing a camera lens at magnification $M_{T} = 1 / 30$ . The scaling should be performed in Fourier space after rounding to maintain the tiling in sensor pixels. The pattern can then be inverse Fourier transformed, tiled over a test chart, and printed. The pattern must be printed with a gamma of 1.0 to maintain the linearity of the cosines sent to the printer. With careful tracking of scaled coordinates, the same printed chart can be used for lens OTF measurements at multiple magnifications and with cameras with different pixel sizes.

When fabricating a test chart with the tartan pattern, it is important to confirm that the printing method is able to accurately reproduce the spatial frequency sampling of the tartan pattern. Also, some fabrication methods, particularly inexpensive ones such as inkjet printing, may result in significant reductions in the cosine amplitudes of the tartan pattern. For this reason, calibration of the printed test chart is necessary to maintain lens MTF measurement accuracy. If a small region of the chart is imaged with a macro lens operating at $1 \times$ magnification, then the tartan frequencies are scaled down by 30 times relative to an image of the test chart taken with a standard camera lens. Even the maximum frequency of $110 lppmm$ in the tartan design $t_{T}$ scales down to $3.7 lppmm$ , for which the OTF of a high- quality macro lens is effectively 1.0. This means that printer OTF measurements made with such a macro lens using our algorithm can be used as a chart calibration.

4. OTF Measurement Using the Tartan Pattern

The spatial frequency spectrum of the tartan pattern $t_{T}$ after being imaged through a lens is

F (u, v) = c H (u, v) T (u, v),

where c is the image gain and

H (u, v)

is the lens OTF we wish to measure. We define the tartan DC as

T_{0} = 2 \sum_{j = 1}^{M} T_{j}

and the values of the lens OTF at each tartan peak as

H (u_{j}, v_{j}) = H_{j} \exp (i φ_{j}^{(H)})

, and note that

H (0, 0) = 1

by the definition of the OTF. Applying these definitions to Eqs. (6, 7) allows us to write

F (u, v) = c T_{0} δ (u, v) + c \sum_{j = 1}^{M} H_{j} T_{j} \exp [i (φ_{j}^{(T)} + φ_{j}^{(H)})] δ (u - u_{j}, v - v_{j}) + c \sum_{j = 1}^{M} H_{j} T_{j} \exp [- i (φ_{j}^{(T)} + φ_{j}^{(H)})] δ (u + u_{j}, v + v_{j}) .

If we denote the values of the imaged tartan spec trum F at each peak location with

F (u_{j}, v_{j}) = F_{j} \exp (i φ_{j}^{(F)})

and

F_{0} = F (0, 0)

, then we can solve for the OTF modulus and phase values at each peak:

H_{j} = \frac{T_{0} F_{j}}{T_{j} F_{0}}, ϕ_{j}^{(H)} = ϕ_{j}^{(F)} - ϕ_{j}^{(T)},

where we have used the DC peak relationship

F_{0} = c T_{0}

to determine the unknown gain c.

While Eq. (9) provides a straightforward recipe for OTF measurement in the continuous imaging case, in practice the image is captured by discrete sampling with a digital sensor, and we wish to measure the OTF over a small region of the image. If the phys ical alignment is not perfect between the tartan pattern in the test chart and the tartan pattern on the image sensor, then the peaks in the captured tartan spectrum P will lie at subsample positions in the Fourier transform of the captured sensor image. In addition, the window function used to extract an image region will affect the shape of the tartan peaks in the Fourier transform of the captured sensor image.

A window operation in the spatial domain corresponds to a convolution with the Fourier transform of the window function in the spatial frequency domain. A simple 1D window function is a square function,

w (x) = {\begin{matrix} 1, & if | x | \leq \frac{a}{2} \\ 0, & if | x | > \frac{a}{2} \end{matrix},

where a is the width of the window. The Fourier transform of a square window is a sinc function,

W (u) = | a | \frac{\sin (a π u)}{a π u} = | a | sinc (a π u) .

The sinc function is oscillatory with a central peak surrounded by significant side lobes.

To understand how this affects our OTF measurement method, we study the convolution of a 2D sinc function with the peaks in the imaged tartan spectrum F. For perfect alignment, samples in the discrete Fourier transform (DFT) coincide exactly with the zeros of the sinc function, which means that each peak appears as a single pixel with the value $F_{j}$ surrounded by zeros. But in practical situations, the alignment of a captured image p is not perfect, so the peak positions in its Fourier transform P no longer coincide with the samples of the DFT. As a result, the sinc function will not be sampled at its zeros, producing DFT samples with rapid oscillations in the neighborhood of each peak as shown in Fig. 5. To avoid significant errors in the OTF measurement, the tartan measurement method needs to be refined to account for the interference effects of the sinc function side lobes.

5. Side-Lobe Reduction

We have developed an algorithm called side-lobe reduction for accurate measurement of the peak heights in the Fourier transform of a windowed tartan pattern [10]. The algorithm is a 2D extension of a 1D spectral peak measurement method by Levin [13]. Our 2D algorithm has the following outline:

Capture an image of a tartan pattern using the lens under test, extract a small image region p using a square window, and Fourier transform to obtain a captured tartan spectrum P.
Detect an affine alignment transform between the ideal tiled tartan spectrum $T_{T}$ and the measured tartan spectrum P, neglecting translation,
Calculate the aligned subpixel peak positions $(u_{A j}, v_{A j})$ within the measured tartan spectrum P using the affine transform,
Estimate the amplitude and phase of P at each peak position $(u_{A j}, v_{A j})$ by interpolation.
Solve for the interaction of the sinc functions associated with each peak to estimate the imaged tartan peak heights $F_{j}$ and phases $φ_{j}^{(F)}$ .
Estimate the OTF using Eq. (9).

We implemented step 2 using subpixel peak finding on the measured tartan spectrum P. The subpixel peak positions are then matched with the ideal tartan peak positions to find an affine transform. Because the interaction of multiple sinc convolutions will perturb the apparent peak positions, it is important to use a combination of peaks to fit the affine transform. This significantly increases the accuracy of the alignment. To achieve a stable affine fit, it is important that there is significant 2D diversity of peak positions [14]. Satisfying this diversity criterion is an important advantage of using a tartan pattern with multiple orientations.

Globally, the test chart alignment is changed by the position and orientation of the camera relative to the test chart, together with lens barrel distortion. However, for a small image region, an affine transform is an acceptable local approximation to the global alignment. Because of the Fourier shift theorem, in the Fourier domain this affine transform has no translation factor. Instead, the tartan peak phases are modified by the addition of a phase ramp.

We then calculate the aligned subpixel positions $(u_{A j}, v_{A j})$ of the tartan peaks in P in step 3 by applying the fitted affine transform to the ideal peak positions in $T_{T}$ . Using high-quality interpolation in step 4 provides an estimate of the amplitude and phase of these subpixel positions in P.

Step 5 relies on an accurate model of the effect of a square window in the spatial domain on the tartan pattern in the Fourier domain. To construct this model, we convolve a 2D version of the sinc function in Eq. (11) with the measured tartan spectrum from Eq. (8), giving

P (u, v) = W (u, v) \otimes F (u, v) = c T_{0} W (u, v) + c \sum_{j = 1}^{M} H_{j} T_{j} \exp [i (φ_{j}^{(T)} + φ_{j}^{(H)})] W (u - u_{j}, v - v_{j}) + c \sum_{j = 1}^{M} H_{j} T_{j} \exp [- i (φ_{j}^{(T)} + φ_{j}^{(H)})] W (u + u_{j}, v + v_{j}) .

This result can be arranged into a matrix representation

P = W \cdot B,

where the left-hand side is

P = (\begin{matrix} P_{0} & P_{1} & \dots & P_{k} & \dots & P_{K} \end{matrix})^{T},

and the right-hand side is the matrix product of

W = (\begin{matrix} W_{00} & W_{01} & .. & W_{0 k} & .. & W_{0 K} \\ W_{10} & W_{11} & .. & W_{1 k} & .. & W_{1 K} \\ .. & .. & .. & W_{.. k} & .. & W_{.. K} \\ W_{j 0} & W_{j 1} & W_{j ..} & W_{j k} & .. & W_{j K} \\ .. & .. & .. & .. & .. & W_{.. K} \\ W_{K 0} & W_{K 1} & W_{K ..} & W_{K k} & W_{K ..} & W_{K K} \end{matrix}),

with

B = (\begin{matrix} B_{0} & B_{1} & \dots & B_{k} & \dots & B_{K} \end{matrix})^{T} .

The DC peak and the Hermitian peak pairs are interleaved into the

K = 2 M + 1

rows of the matrices, using

(u_{H k}, v_{H k}) = {\begin{matrix} (0, 0) & k = 0 \\ (u_{A k}, v_{A k}) & k \in Z_{even}^{+} \\ (- u_{A k}, - v_{A k}) & k \in Z_{odd}^{+} \end{matrix}

for the aligned peak positions,

B_{k} = {\begin{matrix} c T_{0} & k = 0 \\ F_{k / 2} \exp (i φ_{k}^{(F)}) & k \in Z_{even}^{+} \\ F_{(k - 1) / 2} \exp (- i φ_{(k - 1) / 2}^{(F)}) & k \in Z_{odd}^{+} \end{matrix}

for the tartan peak amplitude and phase estimates, and

P_{k} = P (u_{H k}, v_{H k})

for the interpolated pixel values obtained in step 4 from the captured tartan spectrum P. The side-lobe interactions between each aligned peak

(u_{H j}, v_{H j})

and all other aligned peaks

(u_{H k}, v_{H k})

are calculated using

W_{j k} = W (u_{H j} - u_{H k}, v_{H j} - v_{H k}) .

Equation (13) can be solved for K values of

B_{k}

, and these values can be used to solve for the lens OTF H at K discrete positions

(u_{H k}, v_{H k})

using Eq. (9) in step 6. Only

M + 1

discrete samples of the OTF are unique, as the others are Hermitian copies.

The matrix W has small off-diagonal values for well-separated peaks. This means that the solution is likely to be stable and is therefore straightforward. The computation time of steps 1 to 6 of the side-lobe reduction algorithm is about $100 ms$ for a $100 \times 100$ image region p, using a C code implementation running on a $2.5 GHz$ Intel Core 2 Duo processor.

We have compared the results with a Levenberg–Marquardt least-squares fit of $p (x, y)$ in the spatial domain. The results and computation time were broadly similar; however, our Levenberg–Marquardt results were not robust to alignment and noise, which we believe was due to the complexity of joint estimation of cosine frequency and amplitude.

An alternative to a square window function is a Gaussian window. This would broaden the peaks in the spatial frequency domain while attenuating the side lobes. Although we have not tried this method, we believe that broader spectral peaks would complicate peak interpolation and amplitude estimation.

6. Simulation Method

We evaluated the accuracy of lens measurement with the tartan method using a simulation. The previously described tiled tartan pattern $t_{T}$ designed for the top-right image quadrant was used.

We also simulated the slanted-edge method for comparison. The slanted edge had a meridional orientation. The simulation included alignment perturbations, a simulated lens OTF, and multiplicative Gaussian imaging noise. For both our tartan implementation and the slanted-edge implementation that we used, the phase of the OTF was not measured. The following comparison is therefore restricted to the MTF.

The alignment perturbations were chosen from uniform random distributions with the following ranges: $\pm 1 °$ rotation of the image around the optical axis, $\pm 2.5 %$ scale, and $\pm 5 %$ translation relative to the image size. These alignment perturbations were applied to the tartan peak positions and phases to avoid image transformation artifacts.

A finite contrast ratio was applied to each pattern: 10 for the slanted edge [15] and 100 for the tartan pattern. The slanted edge was generated with 8 times oversampling using an edge equation, perturbed by the random alignment, to switch pixels on and off. The edge was reflected fourfold in x and y to reduce Fourier boundary artifacts. We then Fourier transformed both patterns and applied the simulated lens OTF. The edge pattern was scaled down by 16 times by cropping the image in Fourier space after applying the lens OTF.

The images were then inverse Fourier transformed, noise was applied by multiplying the image by a pseudorandom noise image where the noise had a Gaussian distribution with a mean of $1.0 + ε$ and a standard deviation of ε, where ε was either 0.01 or 0.05, and finally each image was windowed using a square function to $100 \times 100$ pixels. The windowed images were used as simulated captured image regions for input into the measurement algorithms.

The slanted-edge measurements were performed using the SFR module of Imatest Master 3.6 [16]. This module implements the ISO 12233 SFR standard. The SFR is an estimate of the MTF. The Imatest algorithm includes enhancements to increase the edge alignment accuracy, including fitting a curved edge to images distorted by lens barrel distortion.

For the simulated lens OTF, we chose a geometrical optics approximation to the defocus OTF by Stokseth [17]. In this model, the normalized focus parameter is

z^{'} = \frac{4 π A_{V} z}{n} \arcsin (\frac{1}{2 A_{V}})^{2} (1 - \frac{z}{f_{L} + z}),

where z is the physical defocus distance,

A_{V}

is the relative aperture, n is the refractive index of the vol ume between the lens and the image plane, and

f_{L}

is the focal length of the lens. The radially symmetric lens OTF is then

H_{G} (q) = \frac{2 J_{1} (z^{'} q)}{z^{'} q},

where

q = (u^{2} + v^{2})^{1 / 2}

is the radial spatial frequency in cycles per length and

J_{1}

is the first-order Bessel function of the first kind. A real camera lens at open aperture will generally have additional aberrations and a lower MTF. However, for medium focus distances and lenses with strong field curvature, defocus is the dominant aberration, so it is useful to explore the accuracy of a lens MTF measurement system for this type of OTF.

For the simulation we used the following param eters: $A_{V} = 1.4$ , $n = 1$ , $f_{L} = 50 mm$ , and a pixel width of $5 μm$ . Because this is a geometrical optics approximation, it does not depend on the illumination wavelength. A significant feature of Eq. (22) is that the function has multiple zero crossings, as shown in Fig. 6. These zero crossings appear as bounces in the MTF.

7. Simulation Results

We tested the MTF measurement performance of the slanted-edge and tartan patterns at defocus distances z of 5, 15, 25, 35, and $45 μm$ . This large defocus range was used to test the accuracy for measurement of the defocus MTF. The simulated imaging noise standard deviation levels were 1% and 5%. A well-exposed image on a DSLR will have a noise of about 1%; however, if a measurement method remains accurate with 5% noise, then it provides significant tolerance to vignetting and variations in lighting conditions.

For each defocus position and noise level, we repeated the simulation for 100 cases, with different random seeds in each case for the alignment perturbation and noise. We discarded the $z = 5 μm$ defocus cases due to a major artifact in the Imatest results that we do not understand. We also discarded 10% of the random seeds due to “zero bin” errors in Imatest. We believe these errors may have been caused by a vulnerability in the slanted-edge algorithm to edge angles with rational slopes.

Simulation results for focus positions of $z = 15$ and $25 μm$ with 5% noise are compared with the ideal MTF in Fig. 7. The modulus of the mean error of the 90 tartan measurements was less than 0.002. Significant uncertainty is clearly visible in the slanted-edge results. The modulus of the mean error was as high as 0.02. This error might be caused by scaling factor differences between the SFR and the MTF. One scaling factor that could cause variations between implementations of the SFR is the difference in the SFR of a discrete derivative compared with a Fourier derivative.

The root mean square (RMS) error of the tartan measurements and edge measurements are compared for the meridional tartan frequencies in Fig. 8. We would consider an MTF error of 0.01 to be significant [18]. The RMS error was often above 0.01 for the slanted-edge measurements and always below 0.01 for the tartan measurements.

The RMS error results in Fig. 8 show a similar trend to the ratios shown in Fig. 4b between the tartan pattern signal strength and the edge pattern signal strength. The RMS error of the tartan measurement does not vary significantly with spatial frequency, while the RMS error of the edge measurement increases approximately linearly with spatial frequency. The average ratio between the tartan and edge errors was 5.0 for 1% imaging noise and 3.0 for 5% imaging noise. The maximum ratio in the simulation results in Fig. 8 was 7.2 at 1% imaging noise, which is half the ratio predicted by the signal strength. This discrepancy could be caused by the effect of the imaging noise on the alignment accuracy in step 2 of the tartan measurement algorithm.

The value of the MTF did not appear to change the nature of the differences between the two measurement methods. For example, the simulated MTF at $70 lppmm$ dropped from 0.42 to 0.05 when the defocus distance z was increased from 15 to $25 μm$ . However, the RMS error of the tartan method with 5% noise was 0.01 for both values of defocus, and the RMS error of the edge method with 5% noise was 0.03 for both values of defocus. This indicates that the uncertainty in both measurement methods was not significantly affected by the value of the simulated MTF. We analyzed the results for the other focus values, and we found the behavior was similar.

The effect of the side-lobe reduction algorithm on the tartan measurement method was measured by comparing side-lobe reduction measurements with direct tartan spectrum peak-finding measurements. The maximum error of these two methods over 90 random cases at $z = 25 μm$ with 1% imaging noise is shown in Fig. 9. The maximum error is significantly lower for the side-lobe reduction results, especially at low spatial frequencies. However, when the imaging noise was increased to 5%, applying side-lobe reduction reduced the maximum error by a much smaller amount.

8. Experimental Results

We tested our tartan algorithm in practice by printing a scaled version of the example tartan pattern [19]. The scaling was designed to measure the frequencies 11, 33, 55, 78, 100, and $122 lppmm$ at a magnification $M_{T} = 1 / 48$ with a Canon EOS 40D DSLR camera with an advanced photo system type-C (APS-C) size sensor with $5.8 μm$ pixels. The scaled tartan frequencies on the printed chart were 0.23, 0.69, 1.15, 1.63, 2.08, and $2.54 lppmm$ . The tartan was tiled to form an A0-sized test chart in order to fill the APS-C sensor at $M_{T} = 1 / 48$ .

The $122 lppmm$ frequency is above the Nyquist frequency of the 40D sensor in the $ψ_{M}$ orientation ( $q_{N} = 104 lppmm$ ). However, at this orientation the aliased spatial frequency does not overlap with any other spatial frequency points in the tartan pattern, so we were able to ignore the $122 lppmm$ frequency during the tartan measurement step.

We used a Canon W8400 wide-format inkjet printer, calibrated to provide a linear output space. The grayscale reproduction of the resulting output was checked and found to be linear to within 1% RMS. The spatial frequency reproduction was checked by printing patterns of specific size and measuring them with an etched glass rule; the scaling was found to be accurate to within 0.2% in both the x and y directions.

The printer MTF was measured from the printed chart using a Canon MP-E $65 mm$ macro lens at $1 \times$ magnification with the relative aperture set to $f / 8$ . At this magnification the highest measured frequency in this tartan design is $2.08 lppmm$ . Assuming this lens has an MTF close to 1.0 at such low spatial frequencies allows us to estimate the mean effective printer MTF at the tartan frequencies. Our results are shown in Table 1.

When measuring the MTF of a lens, the signal strengths measured using our tartan measurement algorithm are divided by the appropriate effective printer MTF to determine the MTF of the lens independent of the printer. The lowest effective MTF for this chart was 0.517. In Fig. 4b the ratio $T_{C} / G$ of signal strength at $100 lppmm$ is 13 times, so even with a 50% drop in signal strength caused during chart printing, the tartan pattern still has a 6 times stronger signal than a slanted edge at $100 lppmm$ . In previous experiments we have measured printer MTFs in multiple locations across a printed chart and shown that they are stable, with an RMS variation of 1%.

The test chart was mounted on a wall and illuminated with tungsten lamps. We captured images with a Canon EF-S $18 - 55 mm$ IS lens mounted on the EOS 40D camera and set to $18 mm$ focal length and relative aperture $f / 3.5$ . The camera was first aligned using a different chart printed with alignment patterns so that the camera sensor was parallel to the chart plane to within $0.1 °$ pan and tilt, using methods reported by us elsewhere [20]. We focused the lens by capturing a focus series and selecting the peak MTF at $33 lppmm$ .

Repeat measurements of best focus images from eight different lenses of the same model with 10 series of photos each, measured at five frequencies each and at 21 image locations distributed across the sensor, showed that our MTF results for each lens were stable to within an RMS of 0.01, with a maximum MTF measurement variation of 0.08.

The tiled nature of the tartan design gives us the opportunity to measure the MTF of a lens across the camera sensor’s entire field of view. We measured the MTF on a set of tiles evenly spaced across the sensor plane. The results at $55 lppmm$ and two orientations at $90 °$ to one another, corrected for the effective printer MTF, are shown as surface plots in Fig. 10.

We compared a subset of our tartan MTF results against MTFs measured by the ISO standard slanted-edge method using Imatest 3.6 Master software. We printed the Imatest “SFRplus” test chart on a Canon W8400 printer at A0 size for measurement at a magnification of $M_{T} = 1 / 48$ . We rotated the chart so that we could measure the same orientation angles $ψ_{M}$ and $ψ_{S}$ as the tartan pattern.

The imaging alignment, focus, and illumination methods were identical to those for the tartan test chart. Regions of interest $128 \times 128$ pixels in size were selected along the lower-left diagonal of the image. The meridional MTF results produced by the Imatest software are shown plotted in Fig. 11 for two regions of interest. Also shown on the same plots are the MTF results obtained from our tartan method at the same image locations. Camera sensor MTF compensation was not applied to the Imatest results or the tartan results. The MTF values compared between the ISO method and the tartan pattern were in reasonably good agreement. The maximum difference over six image locations was 0.11 and the RMS difference was 0.06.

9. Discussion

Maintaining lens OTF measurement accuracy over the 9D geometry of the problem is an ongoing challenge. The diversity of OTF measurement methods reflects different goals for accuracy and 9D sampling.

The advantage of the tartan pattern is in the high signal strength, especially at high spatial frequencies. As shown in the simulation results, this allows accurate measurement over a wide range of sampling conditions. The simulation results show that the tartan measurement method is robust to imaging noise, alignment changes, and low MTF values. This robustness implies that the tartan method will also be robust to large changes in lighting and defocus.

The robustness of the tartan method is important because there are significant sources of measurement error that do not depend on the measurement pattern and detection algorithm. These sources include parallel alignment between the chart and the image sensor [21], focus finding, and chart calibration. Improving the measurement accuracy using the tartan method allows for additional tolerance to variation in the overall measurement conditions.

A significant advantage of the edge method is fine sampling in a single orientation in the spatial frequency domain. However, it is not easy to tune the spatial frequency sampling of an edge pattern without fundamental changes to the algorithm.

In contrast, the tartan pattern is highly tunable. The signal strength, spatial frequency sampling positions, and orientations can all be changed and then measured with the same algorithm [10]. The tartan pattern can also be tiled continuously over the test chart, as demonstrated in Fig. 10 from our experimental results, allowing flexible sampling of image positions and passive alignment tolerance. The main restriction on the tartan design is that the spatial frequency samples should exceed a certain minimum separation to ensure the side-lobe reduction algorithm is stable.

A major difference between the slanted-edge and tartan patterns is chart fabrication. The slanted edge relies on a high-resolution binary print, while the tartan pattern relies on linear grayscale reproduction at high resolution. In both cases, it is desirable to create a large physical test chart, such as A0 poster size for DSLR camera lens tests at $30 \times - 50 \times$ magnification. A binary chart is significantly easier to fabricate than a grayscale chart; however, gray scale reproduction has improved over the past decade with accessible tools such as wide-format inkjet printing and large electronic displays.

While the signal strength of the tartan pattern may be reduced during fabrication, this reduction can be calibrated so that the measurements can be compensated for the reduction. The high initial signal strength of the tartan pattern means that the overall SNR can remain sufficient for accurate measurements, even when the signal strength has been significantly reduced by the printer or display. Our experimental results showed that it is possible to print a tartan chart that has a 6 times higher signal at $100 lppmm$ compared with a slanted-edge chart. An additional chart fabrication factor is that the printer or display may introduce a variation in signal strength across the chart or between different copies of the same chart. This is an additional source of measurement noise for the tartan pattern method compared with the slanted-edge method. In our experiments, this variation in signal strength increased our measurement noise by 1% RMS.

When the OTF is measured using a digital camera, the OTF is affected by the lens, the sensor, and the image processing. This means that digital OTF measurements are actually measuring the system OTF, rather than just the lens OTF. For modern digital cameras, the lens OTF has the most significant contribution to the system OTF. Small pixel sizes mean that the impacts of the pixel fill factor and low-pass filtering are limited and the dominant effects of the sensor on the OTF are predictable. Image processing can be bypassed using raw image files. In contrast, lens design and fabrication constraints typically lead to wavefront aberration limits on camera performance at open aperture, while at narrower apertures the performance is diffraction limited. In addition, the lens OTF is sensitive to the internal alignment achieved during fabrication and to multiple optical parameters at capture time. The relative OTF between two lenses can be compared using a digital camera if the rest of the system remains constant. Measuring the absolute value of the lens OTF requires calibration of the camera OTF. This could be done using a reference lens for which the OTF values have been independently verified.

For our experimental comparison between the tartan and the slanted edge, many important conditions were held constant between the two measurements, including the camera body used for capture and the camera alignment. We also found that repeat lens measurements using the tartan pattern were stable to within an RMS of 0.01. However, we still found an RMS difference of 0.06 between the tartan and slanted-edge measurements. It is possible that a measurement bias was introduced by the tartan method or the slanted-edge method. While differences are not unusual when comparing MTF results between different measurement systems [21], we predict that these differences could be further reduced by calibrating both systems against lens measurements using a stable reference lens.

In this article we have concentrated on measurement of high-quality lenses using DSLR cameras where image processing steps can be disabled. However, the tartan pattern has potential advantages for the measurement of low-cost cameras. Such cameras have strong adaptive processing to reduce noise and sharpen the image. A slanted-edge pattern is more likely to trigger this type of nonlinear processing, which will significantly alter the OTF measurement. A potential disadvantage of the tartan pattern is that strong JPEG compression may filter out the high frequencies.

10. Conclusion

We have proposed a tartan pattern for lens OTF measurement, together with an efficient and robust algorithm for accurate detection of the MTF from captured images of a tartan pattern. The tartan pattern is highly tunable and takes advantage of the flexibility in chart design and detection provided by digital production tools.

We have used simulations to demonstrate that the tartan method has significantly better accuracy than the slanted-edge method for a sparse set of spatial frequencies. With 1% simulated imaging noise, the standard deviation of the tartan method was on average five times smaller than the standard deviation of the slanted-edge method. The strongest improvements in accuracy were at high spatial frequencies.

Our experimental results showed that the tartan method is reproducible to $0.01 RMS$ . We demonstrated the sampling flexibility of the tartan pattern by measuring the MTF on a fine grid of image positions. The tartan method and the slanted-edge method were in reasonable agreement, with an RMS difference of 0.06 for measurements of the same lens.

The authors would like to acknowledge our collaboration with Masatake Kato, Shingo Hayakawa, Hiroshi Saito, and Keiji Ikemori at Canon, Inc. The slanted-edge MTF measurements were performed using Imatest Master 3.6 software (http://www.imatest.com).

Table 1. Measured Effective Printer MTFs of a Printed Tartan Chart

View Table

Fig. 1 Geometry for camera lens OTF measurement. The image position $(x, y)$ and focus z are defined relative to the optical axis at best focus in the image plane. The transverse magnification $M_{T} = z_{i} / z_{o}$ is the ratio between the image size and the object size. For a lens designed for a DSLR camera with an APS-C size sensor, imaging at a transverse magnification of $1 / 50$ , the image width is about $20 mm$ and the object width is about $1 m$ . With the lens aperture open, the focus depth at the image plane is of the order of micrometers. The spatial frequency $(u, v)$ is defined in the image plane. The meridional (M) and sagittal (S) spatial frequency orientations are shown in the top-right object corner. Not shown are the lens focal length $f_{L}$ , the relative lens aperture $A_{V}$ , and the illumination wavelength λ.

Download Full Size | PDF

Fig. 2 Slanted-edge test pattern. A typical slanted-edge pattern has only a small angle relative to the sensor axes. However, in this article we use a slanted edge with a meridional orientation, suitable for OTF measurement in the top-right or bottom-left image corner.

Download Full Size | PDF

Fig. 3 The tiled tartan pattern $t_{T}$ (3) is constructed by combining sinusoidal waves with multiple orientations (1) and frequencies (2). Each sinusoidal wave corresponds to a Hermitian pair of peaks in the Fourier domain, rounded to the nearest integer pixel position. The $(0, 0)$ frequency is in the center of the Fourier domain.

Download Full Size | PDF

Fig. 4 Signal strength of test patterns along the meridional axis in the spatial frequency domain: (a) edge signal strength G and tartan signal strength $T_{C}$ , (b) the ratio $T_{C} / G$ at the tartan peak locations.

Download Full Size | PDF

Fig. 5 Effects of defocus and windowing on the tartan pattern: (a) tiled tartan design $t_{T}$ with (b) Fourier transform $T_{T}$ , (c) imaged tartan p with $z = 45 μm$ of defocus and alignment perturbation with (d) Fourier transform P, and (e) a surface plot of three tartan peaks near the center of the Fourier transform P. The side lobes caused by the spatial domain windowing are clearly visible in (d) and (e). The reduction in peak strength with increasing spatial frequency due to defocus can also be seen. The contrast has been inverted and adjusted in (b) and (d) to clarify the image structure.

Download Full Size | PDF

Fig. 6 Defocus OTF $H_{G}$ used for the simulations, calculated at a range of defocus distances z from 5 to $45 μm$ .

Download Full Size | PDF

Fig. 7 Ideal MTF (thick, dark line) at (a) $z = 15 μm$ and (b) $z = 25 μm$ compared with the simulated meridional MTF measurements from the slanted-edge (multiple thin, light lines) and simulated tartan measurements over 90 random cases with 5% imaging noise. The vertical bars indicate the $\pm 1 σ$ ranges of the slanted-edge method (hollow bars) and the tartan method (solid bars) at each tartan spatial frequency. The tartan frequencies are between each adjoining pair of hollow and solid bars. The slanted-edge results were measured using Imatest Master 3.6.

Download Full Size | PDF

Fig. 8 RMS error of simulated MTF measurements with defocus $z = 25 μm$ over 90 cases of random alignment with (a) 1% imaging noise and (b) 5% imaging noise. The tartan results are shown for the meridional (diamonds) and sagittal (squares) orientations, while the slanted-edge results are meridional only (circles).

Download Full Size | PDF

Fig. 9 Effect of the side-lobe reduction algorithm on the accuracy of the simulated tartan MTF measurements. The maximum error in measurements at the meridional orientation over 90 cases at $z = 25 μm$ with 1% imaging noise is shown for direct peak finding (squares) and side-lobe reduction (circles).

Download Full Size | PDF

Fig. 10 Plots of the MTF of an EF-S $18 - 55 mm$ IS lens at focal length $f_{L} = 18 mm$ measured across the entire image plane at $55 lppmm$ with our tartan pattern. We measured the MTF on a set of evenly spaced tiles, 50 tiles horizontally by 33 tiles vertically. The sampling grid is three times as dense as the grid lines shown on the plots. The plot on the left shows frequencies oriented at angle $ψ_{M}$ , meridionally with respect to the lower-left to top-right diagonal of the sensor. The plot on the right shows frequencies oriented at $ψ_{S}$ , $90 °$ to $ψ_{M}$ . The frequency orientations are fixed across the chart for these measurements, which is in contrast with the standard polar variation of meridional and sagittal orientations.

Download Full Size | PDF

Fig. 11 Plots of the MTF versus spatial frequency of an EF-S $18 - 55 mm$ IS lens at focal length $f_{L} = 18 mm$ as measured by our tartan method (points) and Imatest’s implementation of the ISO standard slanted-edge method (continuous lines). The spatial frequencies are meridional and measured in two different image locations partway between the image center and the bottom left corner, as indicated by the captions on each graph. The maximum difference between the tartan and Imatest MTF results on the plot at 26% bottom left is 0.11, which is the largest difference that we found for this set of measurements. The RMS difference over six image locations was 0.06.

Download Full Size | PDF

1. F. J. P. Consitt and W. Mandler, “OTF techniques in the routine testing of production lenses,” J. Mod. Opt. 18, 123–131 (1971). [CrossRef]

2. H. Kondo, T. Watanabe, and H. Yamaoka, “Criteria for the evaluation of photographic lenses,” J. Mod. Opt. 22, 353–363 (1975). [CrossRef]

3. N. Joshi, R. Szeliski, and D. Kriegman, “PSF estimation using sharp edge prediction,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition 2008 (Institute of Electrical and Electronics Engineers, 2008), pp. 1–8. [CrossRef]

4. A. Levin, S. Hasinoff, P. Green, F. Durand, and W. Freeman, “4D frequency analysis of computational cameras for depth of field extension,” in Proceedings of ACM SIGGRAPH 2009 (ACM, 2009), article 97.

5. P. Lindberg, “Measurement of contrast transmission characteristics in optical image formation,” J. Mod. Opt. 1, 80–89 (1954). [CrossRef]

6. S. E. Reichenbach, S. K. Park, and R. Narayanswarmy, “Characterizing digital image acquisition devices,” Opt. Eng. 30, 170–177 (1991). [CrossRef]

7. International Organization for Standardization, ISO 12233:2000 standard: photography—electronic still-picture cameras—resolution measurements (2000).

8. H. Frieser, “Le Pouvier résolvant des Couches photographiques (The resolving power of photographic coatings),” in IX Congrès International de Photographic Scientifique et Appliquée (Masson SA, 1936), pp. 207–218.

9. Sine Patterns, “Composite Design,” http://web.archive.org/web/20080327091116/www.sinepatterns.com/S_Comp2.htm (2006, accessed June 2010).

10. K. G. Larkin and P. A. Fletcher, “Frequency estimation under affine distortion,” Australian patent application 2008211991 (4 September 2008).

11. M. R. Arnison, S. J. Hardy, and K. G. Larkin, “Optical transfer function measurement system,” Australian patent application 2005203031 (12 July 2005).

12. A. D. Edgar and A. S. Bopardikar, “System and method of testing imaging equipment using transformed patterns,” U.S. patent application 20070230776 (13 June 2006).

13. H. J. Levin, “Discrete Fourier transform (DFT) leakage removal,” U.S. patent 6,882,947 (19 April 2005).

14. D. J. Fleet, D. J. Heeger, T. A. Cass, and D. L. Hecht, “Automatic geometric image transformations using embedded signals,” U.S. patent 5,949,055 (7 September 1999).

15. D. Williams and P. D. Burns, “Applying and extending ISO/TC42 digital camera resolution standards to mobile imaging products,” Proc. SPIE 6494, 64940H (2007). [CrossRef]

16. N. Koren, “Sharpness: what is it and how is it measured?” http://www.imatest.com/docs/sharpness.html (Imatest, 2010, accessed June 2010).

17. P. A. Stokseth, “Properties of a defocused optical system,” J. Opt. Soc. Am. 59, 1314–1321 (1969). [CrossRef]

18. G. C. Brock, A. C. Marchant, and T. L. Williams, “OTF standards for aerial mapping lenses,” J. Mod. Opt. 19, 953–972 (1972). [CrossRef]

19. D. P. Morgan-Mar, M. R. Arnison, C. A. Deller, P. A. Fletcher, and K. G. Larkin, “Two-dimensional measurement of the lens optical transfer function from a digital image,” Proc. SPIE 7876, 78760H (2011). [CrossRef]

20. D. P. Morgan-Mar, M. R. Arnison, C. A. Deller, S. J. Hardy, and K. G. Larkin, “Geometric parameter measurement of an imaging device,” U.S. patent application 20090161945 (21 November 2008).

21. A. C. Marchant, E. A. Ironside, J. F. Attryde, and T. L. Williams, “The reproducibility of MTF measurements,” J. Mod Opt. 22, 249–264 (1975). [CrossRef]

Measurement of the lens optical transfer function using a tartan pattern

Abstract

1. Introduction

2. Slanted-Edge Method

3. Tartan Pattern Design

4. OTF Measurement Using the Tartan Pattern

5. Side-Lobe Reduction

6. Simulation Method

7. Simulation Results

8. Experimental Results

9. Discussion

10. Conclusion

Cited By

Figures (11)

Tables (1)

Equations (22)

Applied Optics

Scaled Frequency (lppmm)	Printed Frequency (lppmm)	Effective MTF
11	0.23	0.956
33	0.69	0.859
55	1.16	0.730
78	1.62	0.614
100	2.08	0.517