1. INTRODUCTION

Pupil size is affected by not only low-level features of stimuli such as luminance, color, and spatial frequencies but also high-level processing including cognitive load, memory, imagination, and attention [1–12]. For example, pupils constrict when people look at a bright stimulus and dilate when they look at a dark stimulus [1]; this is known as pupil light reflex (PLR). When the luminance of the visual stimulus remains unchanged, the dilation of a pupil increases with the level of the task’s difficulty [9]. Moreover, the effects of low-level features and high-level processing can be basically combined to elicit a pupillary response. Since the previous century, pupil measurement has been one of the most popular physiological methods to investigate human functions because it reflects many aspects thereof.

A few studies have revealed the effects of stimulus spatial frequency on pupillary response. In this study, the function is referred to as pupillary spatial frequency response (PSFR). It has been consistently shown that a stimulus with 2–5 c/d of spatial frequencies can result in stronger pupil constriction than that with lower or higher spatial frequencies [4–8]. Although the anatomical pathway of PLR has been well studied [13–15], the pathway of PSFR remains unclear. Previous studies have found a similarity between PSFR and the contrast sensitivity function and have further proposed that the geniculo-striate visual pathway from the retina to V1 is involved in PSFR [4,6]. This hypothesis was supported by investigations of pupillary response in participants with a geniculo-striate lesion [16].

It has been shown that pupils change responding to stimulus luminance at an attended location. Pupils constrict when people attend to a bright stimulus and dilate when they attend to a dark stimulus [17,18]. Visual attention involves directing attention to a spatial location or an object. There are various types of attention, namely, spatial endogenous attention, spatial exogenous attention, feature-based attention, and object-based attention [19,20]. When people willfully attend to information at a given location or automatically attend to a location where an abrupt change occurs, the attention system differs because spatial endogenous and exogenous attention occur separately. Besides attending to distinct spatial locations, people can also attend to specific visual features such as color and orientation or attend to an organized object; the former involves feature-based attention and the latter object-based attention. Spatial attention and object-based attention were examined in this study.

The question of whether high-level processing can modulate the effect of spatial frequency on pupillary response remains. The primary objective of this study was to examine the effect of attention on PSFR. More specifically, we controlled participants’ gaze and attentional direction separately. The secondary objective was to demonstrate the effect of each frequency in PSFR by eliminating possible contamination caused by square waves, sine wave gratings with sharp edges, and checkerboard patterns. Previous studies mostly employed such stimuli [4–8]. Consequently, the pure effect of one spatial frequency could not be examined. Although it is well known that increasing attention can elicit pupil dilation [21], we focused on the effect of attention on PSFR, not of the magnitude of attention on pupillary response.

In this study, we conducted three experiments. The purpose of Experiment 1 was to demonstrate the basic function of PSFR. We compared PSFRs elicited by sine wave grating and a Gabor patch and attempted to find appropriate parameters such as the spatial frequency and eccentricity of the stimulus, which could be used in the following experiments. The focus of Experiment 2 was a study of spatial-based attention. We controlled participants’ gaze and attentional direction while the stimulus remained unchanged. The effects of object-based attention on PSFR were investigated in Experiment 3. We used natural filtered images with complicated spatial frequency components as the stimuli.

According to Young, Han, and Wu [22,23], pupillary response can be organized into two different processes: a transient process and sustained process. The transient process indicates the initial constriction occurring right after the stimulus onset, and the sustained process indicates the steady pupil size during the presentation of a stimulus. The two processes differ from each other, with the former saturating with increasing stimulus contrast whereas the latter varies linearly with increasing contrast. Consequently, in this study, two indices representing each process in the pupillary response were employed.

2. EXPERIMENT 1: THE EFFECTS OF SPATIAL FREQUENCY AND ECCENTRICITY ON PUPILLARY RESPONSE

Previous studies have demonstrated that pupils constricted more when stimulus spatial frequency was in the range of 2–5 c/d [4–8]. However, the spatial frequency contained in the stimulus was not controlled completely because it had sharp edges. The primary objective of this experiment was to present a complete function of pupillary response to spatial frequency by examining whether extra spatial frequency plays a role in PSFR. Therefore, the difference of PSFR by using sine wave grating with a sharp edge and a Gabor patch was examined. Furthermore, in order to understand the effect of eccentricity from the fovea for PSFR, three conditions of the stimulus’s eccentricity were tested. The results in this experiment were used in the subsequent experiments.

A. Participants and Apparatus

The participants included four males (age range: 21–25 years) with corrected-to-normal vision. Informed consent was obtained from each of them. The protocol was approved by the Ethics Committee of Tokyo Institute of Technology.

Eye movements and pupil diameter were measured monocularly (right eye) using an infrared video-based eye tracker (EyeLink 2000, SR Research) with a spatial resolution of 0.1% of diameter for a 5 mm artificial pupil, sampled at 500 Hz. Visual stimuli were presented on a cathode-ray tube (CRT) monitor (Sony GDM-F500R, $1280 \times 1024\,\,{\rm{pixels}}$, 60 Hz refresh rate). The participants sat 57 cm away from the monitor, and they put their chins on a chin rest to prevent their heads from moving. A keyboard was put between the chin rest and monitor to allow the participants to respond.

B. Stimulus and Procedure

The stimulus was either a circular sine wave grating (${\rm{radius}} = 4.5^\circ $) or a Gabor patch ($\sigma = 3^\circ $) with different spatial frequencies. In this experiment, the eccentricity of the stimulus had one of three values (0°, 5°, and 8°). Eight conditions of spatial frequency were used when the eccentricity was 0° and 5°: 0.54, 1.43, 2.32, 3.78, 5.57, 7.8, 10, and 12 c/d. When it was 8°, the spatial frequency of 0.2 c/d was used instead of 12 c/d. The Michelson contrast was set to 1 for all the stimuli. As a result of pixel independence [24], there is a clear luminance difference for sine wave gratings with different orientations when the spatial frequency is high (10 and 12 c/d in this experiment). Therefore, four kinds of clockwise orientations were selected: 60°, 80°, 100°, and 120° (a vertical grating was defined as 0°), which led to a tolerable maximum luminance difference among the four orientations ($0.8\,\,{\rm{cd}}/{{\rm{m}}^2}$ for 10 c/d, $1.2\,\,{\rm{cd}}/{{\rm{m}}^2}$ for 12 c/d). The stimulus was presented against a uniform bright gray background (${{35}}\,\,{\rm{cd}}/{{\rm{m}}^2}$) and was randomly presented in the left or right hemifield for the eccentricity conditions of 5° and 8°. In the condition of 0° eccentricity (fovea), the stimulus was located at the center of the screen.

For each eccentricity and stimulus type, 12 experimental blocks were conducted. Each block contained 72 trials; there were nine trials for each spatial frequency. Each block lasted about 10 min. Twelve blocks were divided into six sessions. A total of 864 trials (${{6}}\,{\rm{sessions}} \times {{2}}\,{\rm{blocks}} \times {{72}}\,{\rm{trials}}$) were conducted for each participant.

The time course of stimulus presentation in Experiment 1 is depicted in Fig. 1. First, during each trial, the participants were instructed to look at a black fixation dot for 2 s. Thereafter, the fixation dot disappeared, and a stimulus with one of eight spatial frequencies and one of four orientations was presented. During the 4 s period of stimulus presentation, the orientation of the stimulus changed either once or twice or did not change. When it changed, it occurred every alternate 1.5 s. This manipulation was employed to hold the participants’ attention at the stimulus location. A large white noise mask was presented right after the stimulus was offset. In order to avoid the influence of unintended spatial frequency information other than the stimulus, we added a Gaussian envelope for the black fixation dot ($\sigma = 0.1^\circ $) and white noise mask ($\sigma = 21^\circ $). After the stimulus presentation, the participants were asked to respond to the number of stimulus orientation changes in the trial, that is, 0, 1, or 2. The participants could rest and blink during this responding period. Once they pressed a button, a new trial began. To avoid fatigue, participants could have rest between blocks and could conduct different sessions on different days.

 

Fig. 1. Time course of stimulus presentation in two trials of Experiment 1. The top panels show an example of the stimulus sequence in a trial of the sine wave grating with a sharp edge, and the bottom panels show an example with a Gaussian envelope (Gabor patch). For illustrative purposes, the stimulus changed twice in this trial. Each trial began with a presentation of a fixation dot for 2 s after which the stimulus with a randomly chosen orientation and spatial frequency was presented. The orientation of the stimulus changed 1.5 s later. Another 1.5 s later, the orientation of the stimulus changed again. Then, after 1 s, the stimulus was replaced by a white noise mask. Note that the white noise mask was reorganized to see it clearly in this paper.

Download Full Size | PDF

C. Data Analysis

For each trial, the horizontal eye position, which averaged over 300 ms prior to the stimulus onset, was defined as the fixation position. The deviation of the horizontal eye position during the stimulus presentation was calculated; if more than 5% of the samples were located 1° from the fixation position, the particular trial was excluded from the analysis. Trials were also discarded when the center of distribution was biased by more than 1° to the stimulus side.

The pupil baseline size was calculated by averaging over 100 ms prior to the stimulus onset for each trial. To maintain the participants’ stable arousal condition, trials where the baseline size deviated more than 2.5 standard deviations from the average baseline size of each block (72 trials) were excluded from the analysis. Pupillary response data were normalized by subtracting the baseline size from the original and then dividing it by the baseline size. Therefore, the data were in percentage values, in which the positive sign represented the dilation percentage and the negative sign the constriction percentage in relation to the baseline size. Subsequently, the blink data were reconstructed by using a cubic Hermite spline to ensure monotonicity during the blink period [25]. However, if there were more than three blinks in a trial, the trial was excluded from the analysis. Finally, pupil trace was smoothed by using a Savitzky–Golay filter (order 2, window length 100 ms) [26].

An example of a typical pupillary response in one trial is presented in Fig. 2. As depicted, a dilation response usually occurs within 1 s after the stimulus onset [15]. Subsequently, the maximum constriction, which is defined as the difference between the baseline size and the smallest value within 2 s after the stimulus onset, occurs. To obtain a stable value for the maximum constriction, the values in the range preceding 50 ms and following 50 ms to the extreme value were averaged. Then, pupil size asymptotically approaches the baseline. We called the asymptotic value the mean pupil change, which may be defined as the average value during the last 2 s in each trial. As shown in Fig. 2, the orientation changes might elicit small pupil constrictions. The effect of orientation changes (rotation) would be cancelled out in the averaged data because the orientation changed twice, once, or did not change at random, and we averaged these three conditions in our final analysis. Furthermore, the maximum constriction occurred usually within 1.5 s from stimulus onset, during which no orientation changes happened. Also, the mean pupil change was an averaged value during the last 2 s of stimulus presentation. In this study, we focused on the latter two parameters, namely, the maximum constriction and mean pupil change, because we were of the view that these two values would reflect the information processing characteristics of the transient and sustained processes of the pupillary response, respectively [22,23].

 

Fig. 2. Typical pupillary response as a function of time. This is one example of averaged pupillary response. A positive value indicates dilation, and a negative value indicates constriction. Two parameters were calculated in Experiment 1: maximum constriction and mean pupil change. Both parameters used in the analysis were normalized by subtracting the baseline size and then dividing it by the baseline. Refer to the text for details.

Download Full Size | PDF

D. Results

A total of 2.4% and 2.3% of the trials for the stimuli of the sine wave grating and the Gabor patch, respectively, were excluded from the analysis. More than 70% of the responses for each participant for the orientation change detection task were correct, even for the stimulus with 10 c/d, with which the participants encountered difficulties identifying the stimulus orientation. The results of both the maximum constriction (panels A and B) and mean pupil change (panels C and D) for both types of stimuli as a function of stimulus spatial frequency are illustrated in Fig. 3. In relation to the stimulus presented at the fovea, 0° of eccentricity, it was evident that the pupillary response was modulated by the spatial frequency as previous studies have revealed [4–8]. With respect to the sine wave grating, the smallest value of pupil size was found at spatial frequencies of approximately 2–8 c/d, and the amplitude of modulation decreased with increasing eccentricity. We found a similar pattern for the Gabor patch; however, the magnitude of spatial frequency modulation for the Gabor patch was smaller than that for the sine wave grating.

 

Fig. 3. Results of Experiment 1. The change of pupil size is plotted along the ordinate (in %) as a function of spatial frequency along the abscissa (in c/d). The line styles and symbol shapes of the plot represents the eccentricity of the stimulus: dashed rectangle—0°, dotted triangle—5°, and solid round—8° of eccentricity. Panels A and B show the results of maximum constriction, and panels C and D show mean pupil change. Panels A and C are the results for the sine wave grating, and panels B and D are the results for the Gabor patch. The error bars represent standard error of the means across four participants.

Download Full Size | PDF

A three-way repeated analysis of variance (ANOVA) was performed to test the significance of the results. The within-subject factors included stimulus type (sine wave grating, Gabor patch), eccentricity (0°, 5°, and 8°), and spatial frequency (0.54, 1.43, 2.32, 3.78, 5.57, 7.8, and 10 c/d). We considered seven conditions of spatial frequency used in all the conditions of eccentricity.

In relation to maximum constriction (Figs. 3A and 3B), the main effects of spatial frequency and eccentricity were significant (spatial frequency: $F_{6,18} = 6.708$, $p = 0.000738$; eccentricity: $F_{2,6} = 52.98$, $p = 0.000154$) even though there was no main effect of stimulus type ($p = 0.0797$). The interaction between spatial frequency and stimulus type was significant ($F_{6,18} = 3.342$, $p = 0.0216$), and the interaction between spatial frequency and eccentricity was also significant ($F_{12,36} = 4.85 $, $p = 0.000108$). There was no interaction between stimulus type and eccentricity ($p = 0.0775$). Finally, there was no significant interaction among spatial frequency, eccentricity, and stimulus type ($p = 0.535$).

With respect to mean pupil change (Figs. 3C and 3D), the main effects of spatial frequency, eccentricity, and stimulus type were significant (spatial frequency: $F_{6,18} = 7.067$, $p = 0.000547$; eccentricity: $F_{2,6} = 5.966$, $p = 0.0375$; and stimulus type: $F_{6,18} = 3.447$, $p = 0.0191$). The interaction between spatial frequency and stimulus type was significant ($F_{6,18} = 3.447$, $p = 0.0191$). Furthermore, that between spatial frequency and eccentricity was significant ($F_{12,36} = 4.509$, $p = 0.000212$). However, the interaction between stimulus type and eccentricity was not significant ($p = 0.0899$). Finally, the interaction among those three factors did not reveal any significant results ($p = 0.65$).

Spatial frequency modulation on pupillary response was found for the Gabor patch, which contained the intended spatial frequency exclusively, even though the effect was weaker than for sine wave grating with a sharp edge. Although it was not strong enough to change the overall pupillary response, the significant interaction between spatial frequency and stimulus type indicated that extra spatial frequency information of the sharp edges influenced the PSFR slightly. Although the perceived stimulus size between the sine wave grating and the Gabor patch was different, this did not affect our results. In a preliminary experiment, we tried to measure the pupillary response to the Gabor patch whose perceived size matched to the sine wave grating. We found that it was comparable with the results when perceived size was not matched.

We also found a significant interaction between spatial frequency and eccentricity. As we can see from Fig. 3, the curve for 8° was a little bit shifted to the left compared with the other two eccentricities. This shift was similar to the contrast sensitivity function (CSF), and it was clearly found in previous research [4,7,27], in which they argued that PSFR and CSF might share at least some common mechanisms. Furthermore, the similarity between the spatial frequency modulations in maximum constriction and mean pupil change suggested that there is also a common mechanism for processing spatial frequency to produce both transient and sustained components of pupillary response.

We confirmed the effect of spatial frequency on pupillary response, even for the Gabor patch. In the middle range of spatial frequency, pupillary responses—both the maximum constriction and mean pupil change—were stronger than the higher and lower frequencies. The participants were also asked to gaze at the fixation dot and count the orientation change times of the peripheral stimulus; a similar result as in the direct viewing condition was found. Although it can be explained by the attentional modulation on PSFR, the possibility that the stimulus itself elicited the pupillary response regardless of attention also exists. In order to examine this possibility, the same stimulus should be utilized and only the condition of attention should be changed. The second and main objective of this study, which was tested in the following experiments, was to examine whether the relationship between pupillary response and spatial frequency is valid when only attention, not eye gaze, is directed to a stimulus.

3. EXPERIMENT 2: THE EFFECTS OF SPATIAL ATTENTION ON PUPILLARY RESPONSE TO SPATIAL FREQUENCY

The aim of Experiment 2 was to examine the effects of spatial attention on the relationship between spatial frequency and pupillary response, which was demonstrated in Experiment 1, by using a stimulus to control a participant’s attentional location. We chose two spatial frequencies, 0.54 and 5.57 c/d, as stimuli based on the results of Experiment 1 because they could evoke the most different pupillary responses. When the two stimuli were presented at the center of the screen (eccentricity of 0°), they would overlap with each other, and when the two stimuli were presented at 8° eccentricity, two of five participants reported that the 5.57 c/d stimulus appeared blurred. Therefore, we chose eccentricity of 5° in this experiment.

A. Participants and Apparatus

Five males (age range: 21–25 years), four of whom also participated in Experiment 1, with corrected-to-normal vision participated in this experiment. Informed consent was obtained from each of them. The protocol was approved by the Ethics Committee of Tokyo Institute of Technology.

The apparatus used in Experiment 1 was employed.

B. Stimulus and Procedure

The stimulus was composed of three items, namely, two circular objects with different spatial frequencies of 0.54 c/d (low frequency) and 5.57 c/d (high frequency), and a small fixation dot. A sine wave grating (${\rm{radius}} = 4.5^\circ $) and a Gabor patch ($\sigma = 3^\circ $) were also used. The grating objects with low and high frequencies were located randomly in the participants’ left or right hemifield to prevent habituation. The eccentricity for the two objects was fixed at 5° from the fixation point to the center of the objects. The contrast and orientation manipulation were the same as in Experiment 1.

Two examples of the trial procedure are illustrated in Fig. 4. First, a fixation dot and an attentional cue were presented. The attentional cue included three types: a left line, a right line, and no line around the fixation dot. The participants should shift their attention to the left hemifield, right hemifield, or remain unchanged based on the cue. Two seconds later, two circular objects of low (0.54 c/d) and high (5.57 c/d) spatial frequencies with random orientations were shown at the left and right hemifield while leaving the fixation dot and attentional cue. During the stimulus presentation, the stimulus orientation (the condition cued to a stimulus) or color of the fixation dot (the condition cued to the fixation) at the cued position could change every 1.5 s. The participants were asked to respond to the number of times (0, 1, or 2) the stimulus orientation or fixation dot color changed after the presentation of the stimulus. This procedure was intended to keep the participants’ attention on the cued location. A large white noise mask was presented immediately after the stimulus was offset to eliminate the stimulus afterimage. After they responded to a number, a new trial began. We assigned 27 trials for each low (0.54 c/d) and high (5.57 c/d) frequency stimulus and 18 trials for the fixation dot, a total of 72 trials in a block. Each participant completed two blocks: one for sine wave grating and the other for the Gabor patch. Each block lasted about 10 min. The same steps followed in Experiment 1 were used to analyze the pupil data.

 

Fig. 4. Time course of stimulus presentation in two trials of Experiment 2. The top panels show a trial using sine wave grating and the bottom panels using the Gabor patch. Each trial began with the presentation of a fixation with an attentional cue for 2 s. The left or right line indicated participants should shift attention leftward or rightward, and no line indicated they should not shift attention. After the cue presentation, stimuli with low (0.54 c/d) and high (5.57 c/d) spatial frequency were presented in the left and right hemifields. In these examples, the orientation of the object in the cued side rotated twice. Finally, the stimuli were replaced by a white noise mask. Note that the white noise mask was reorganized to see it clearly in this paper. Participants were to respond “2” in this example. They could have a break during this period. Once they pressed the keyboard, the next trial began.

Download Full Size | PDF

C. Results

The average percentage of the participants’ correct responses for the orientation change detection task was 98% and 94% for the sine wave grating and Gabor patch, respectively. A total of 2.1% of the trials were excluded from the analysis. The exclusion criteria included pupil baseline size, blink rate, and eye movement; this is noted in Section 2.C.

The results of both the maximum constriction (panels A and B) and mean pupil change (panels C and D) for both types of stimuli are depicted in Fig. 5. The abscissa shows the condition of attentional location (0.54 c/d, 5.57 c/d, and fixation). As shown in Fig. 5, even though the eye was directed to the same position (fixation point) and the visual stimuli were the same, attentional location affected the pupillary response. Covert attention to the location of the object with low spatial frequency (0.54 c/d) caused a larger pupil and less constriction in most cases relative to the location of the object with high spatial frequency (5.57 c/d). This tendency was found for both the sine wave grating and the Gabor patch and for both maximum constriction and mean pupil change.

 

Fig. 5. Results of Experiment 2. The abscissa represents the stimulus property participants were instructed to attend to, low (0.54 c/d) frequency stimulus, high (5.57 c/d) frequency stimulus, and fixation. The ordinate represents the pupil diameter ratio relative to the baseline for maximum constriction (panels A and B) and mean pupil change (panels C and D). Panels A and C are the results for the sine wave grating, and panels B and D are the results for the Gabor patch. The error bars represent standard error of the means across five participants.

Download Full Size | PDF

We performed a two-way repeated ANOVA with stimulus type (sine wave grating, Gabor patch) and attentional location (0.54 c/d, 10 c/d, and fixation dot) as within-subject factors for maximum constriction and mean pupil change. With respect to maximum constriction (Figs. 5A and 5B), the main effect of the stimulus type was significant ($F_{1,4} = 15.51$, $p = 0.017$). However, there was no main effect of attentional location ($p = 0.143$). The interaction between attentional location and stimulus type was not significant ($p = 0.947$). With respect to mean pupil change (Figs. 5C and 5D), the main effect of attentional location was significant ($F_{2,8} = 8.2$, $p = 0.0116$). However, there was no main effect of stimulus type ($p = 0.20$). The interaction between attentional location and stimulus type was not significant ($p = 0.248$). Because we found the main effect of attentional location, a post-hoc test was conducted subsequently. We found that attending to 5.57 c/d elicited a significant smaller pupil compared with attending to 0.54 c/d ($t_{1,4} = 3.22$, $p = 0.032$), whereas there was no appreciable difference of pupil sizes between the condition attending to a fixation dot and attending to 0.54 c/d ($p = 0.0501$) or 5.57 c/d ($p = 0.289$).

The results showed that spatial attention modulated pupillary response to spatial frequency. Furthermore, we found a significant main effect of attentional location for mean pupil change, but no significant main effect for maximum constriction, which thus indicates that attentional modulation might be more robust for the sustained process of the pupillary response. Although it is well known that paying attention in general can elicit pupil dilation [21], that kind of attentional dilation would be smaller than the effects of attentional modulation on PSFR.

Besides spatial attention, previous studies have also investigated the modulation of feature-based attention on PLR. Binda et al. used a stimulus that overlapped both bright and dark dots at the same location [17]. They found that pupil size was smaller when the bright dots rather than the dark dots were attended to. Consequently, by using a similar stimulus setting, we examined the effect of object-based attention on PSFR in Experiment 3.

4. EXPERIMENT 3: THE EFFECTS OF OBJECT-BASED ATTENTION ON PUPILLARY RESPONSE TO SPATIAL FREQUENCY

The purpose of Experiment 3 was to test whether object-based attention modulated the relationship between pupillary response and spatial frequency. We used natural images for the attentional object. The images, which originally contained complex spatial frequency components, were filtered to produce modified images with specific spatial frequency information.

A. Participants and Apparatus

Six males (age range: 21–25 years), five of whom had participated in Experiment 2, with corrected-to-normal vision participated in this experiment. Informed consent was obtained from each of them. The protocol was approved by the Ethics Committee of Tokyo Institute of Technology.

A different eye tracker (iRecHS2 system) from that used in Experiments 1 and 2, with spatial resolution of 10.794 pixel/mm (= 0.093 mm/pixel), was employed to measure pupillary response [28]. Pupil data were recorded at a 390 Hz sampling rate. Stimuli were presented on the liquid crystal display (LCD) monitor (FlexScan SX2762W EIZO, $1920 \times 1200\,\,{\rm{pixels}}$, 60 Hz refresh rate). The participants sat 75 cm away from the monitor and put their chins on the chin rest to prevent any head movements.

B. Stimulus and Procedure

Stimuli containing more complex spatial frequency components were used in Experiment 3. Four types of natural images, namely, a pineapple, rabbit, watermelon, and turtle, were used as the original images. They originally contained complex and multiple spatial frequency components and were low-pass or high-pass filtered (Fig. 6A) to restrict spatial frequency information. Second-order Butterworth filters with a cutoff frequency of 1.5 and 4 c/d were used to create low-pass and high-pass filtered images, respectively. The stimulus (${6^\circ} \times {6^\circ} $) was presented on a uniform bright gray background ($27\,\,{\rm{cd}}/{{\rm{m}}^2}$) and was located at the center of the screen.

 

Fig. 6. Stimuli used in Experiments 3a and 3b. Panel A represents four kinds of single objects: pineapple, rabbit, watermelon, and turtle. Each of them was both low-pass and high-pass filtered. Panel B represents four kinds of combined images. Each of them was a combination of the low-pass and the high-pass filtered images. Refer to the text for details.

Download Full Size | PDF

Experiment 3 comprised two different parts. In Experiment 3a, the stimulus was one of the filtered images with a single object (Fig. 6A). The luminance profiles of eight images were adjusted to be the same by using the SHINE toolbox [29]. The mean overall luminance was $22\,\,{\rm{cd}}/{{\rm{m}}^2}$ with the same luminance histogram.

In Experiment 3b, the stimulus was an image that combined low-pass and high-pass filtered images (Fig. 6B). The stimulus was created by using MATLAB function imadd, which is implemented by adding each element in an array with the corresponding element in another array. Four kinds of images were used: high-pass filtered turtle + low-pass filtered watermelon, high-pass filtered watermelon + low-pass filtered turtle, high-pass filtered watermelon + low-pass filtered rabbit, and high-pass filtered pineapple + low-pass filtered rabbit. The selected images all combined an animal and a plant. The luminance profiles of these four images were adjusted to be the same. Furthermore, the mean luminance was $34\,\,{\rm{cd}}/{{\rm{m}}^2}$, and they had the same luminance histogram.

Examples of trial procedure for Experiments 3a (top panels) and 3b (bottom panels) are depicted in Fig. 7. First, in Experiment 3a, a fixation dot was presented. Two seconds later, one of the eight stimuli was presented. During the stimulus presentation, each participant was asked to scan the shape of the object. Subsequently, a large white noise mask was presented to eliminate the stimulus afterimage until the participant responded. An object identification task was conducted: 1 was pressed for the animal and 0 for the plant. After each response, a new trial began. We assigned nine repetitions for each stimulus, a total of 72 trials in a block. Each block lasted about 10 min.

 

Fig. 7. Time course of stimulus presentation in a trial of Experiments 3a (top panels) and 3b (bottom panels). In both experiments, the fixation or cue was presented for 2 s at first. Then, the stimulus was presented for 4 s. Participants were asked to look at the presented object or attend to the cued object during this period. Finally, the stimulus was replaced by a white noise mask. Note that the cue and white noise mask were reorganized to see them clearly in this paper. Participants were required to respond to the type of the object, animal or plant, and could have a break. Once they pressed the keyboard, the next trial began. Refer to the text for details.

Download Full Size | PDF

In Experiment 3b, a cue, Ob (abbreviation for obscure, indicating a low-pass filtered object) or De (abbreviation for detailed, indicating a high-pass filtered object) was first presented for 2 s. Subsequently, one of the four kinds of stimuli was presented. Participants attended to the cued object. Four seconds later, a white noise mask was presented until the participants responded. Participants had to respond to the object type (animal or plant) they attended to. A total of 72 trials (4 kinds of stimulus × 2 attention instructions × 9 repetitions) were conducted for each participant. This experiment lasted about 10 min.

The analyses of pupil data followed the same steps employed in Experiment 1; however, the eye movement procedure was omitted because the participants were free to move their eyes around.

C. Results

The average percentage of the participants’ correct responses for the object identification task was 97% and 98% for Experiments 3a and 3b, respectively. A total of 2.6% and 2.8% of the trials were excluded from the analysis for Experiments 3a and 3b, respectively. The exclusion criteria included pupil baseline size and blink rate; refer to Section 2.C.

The results of both maximum constriction (top panels) and mean pupil change (bottom panels) for both experiments are presented in Fig. 8. As shown in Fig. 8A, when looking at the isolated high-pass filtered object, pupil size tended to be smaller in comparison with that when looking at the isolated low-pass filtered object. These results were evident for both maximum constriction and mean pupil change. As shown in Fig. 8B, when attending to the high-pass filtered object in the combined image, pupil size tended to be smaller in comparison with that when attending to the low-pass filtered object. These results were evident for both maximum constriction and mean pupil change.

 

Fig. 8. Results of Experiment 3. The averaged proportion of maximum constriction (top panels) and that of mean pupil change (bottom panels): A for every single image presented in Experiment 3a, and B for combined images presented in Experiment 3b. In A, the abscissa represents two types of filters, and in B it represents the filter type of the attended images. The error bars represent standard error of the means across six participants.

Download Full Size | PDF

To examine the significance of the results in Experiment 3a, we performed a two-way repeated ANOVA with the object type (pineapple, rabbit, watermelon, and turtle) and filter type (high-pass filter and low-pass filter) as within-subject factors. The main effect of filter type was significant in both the maximum constriction ($F_{1,5} = 8.759$, $p = 0.0315$) and mean pupil change ($F_{1,5} = 23.12$, $p = 0.00485$). However, the main effect of object type was not significant (maximum constriction: $p = 0.182$; mean pupil change: $p = 0.355$). The interaction between the two factors (maximum constriction: $p = 0.777$; mean pupil change: $p = 0.271$) was not significant.

To examine the significance of the results in Experiment 3b, we performed a one-way repeated ANOVA with filter type (high-pass filter and low-pass filter) as the within-subject factor. Since the object type did not affect the results in Experiment 3a, we omitted the factor of the object for the analysis. The main effect of filter type was significant in both maximum constriction ($F_{1,5} = 9.568$, $p = 0.0271$) and mean pupil change ($F_{1,5} = 7.536$, $p = 0.0405$).

These results indicated that object-based attention could also modulate pupillary response to spatial frequency. The significant effect of stimulus frequency (filter type) was demonstrated for both maximum constriction and mean pupil change. This result was partially inconsistent with the results of Experiment 2, in which a significant effect was observed in the mean pupil change but not in maximum constriction. This difference might be due to the difference in the eye movement instructions given to participants. Eye movements were allowed when observing the stimulus in Experiment 3, but not in Experiment 2. Although the participants gazed at fixation where no stimulus was presented in Experiment 2, they looked at the stimulus directly in Experiment 3. This may have led to different results for maximum constriction in Experiments 2 and 3. Previous research showed that the recorded pupil size varied with gaze position, which is an artifact called the pupil foreshortening error [30]. This might also have caused the different results for maximum constriction.

5. DISCUSSION

We conducted three experiments to examine the effects of attention on the relationship between spatial frequency and pupillary response. Three kinds of stimulus to manipulate spatial frequency components were employed: sine wave grating with a sharp edge, grating with a Gabor envelope, and a spatial frequency filtered natural image. We used two indices of pupillary response, namely, maximum constriction and mean pupil change, which are considered to reflect the transient and sustained components of pupillary response, respectively [22,23].

The results of Experiment 1 confirmed that the difference in the spatial frequency component of the visual stimuli could elicit different pupillary responses even if the mean luminance of the stimulus was the same. Pupil constriction was larger when spatial frequency was between 2 and 8 c/d, and it peaked at 5.57 c/d. This effect of spatial frequency was also evident after the recovery of initial pupil constriction. Previous studies have revealed that the transient maximum constriction of pupillary response was larger when the stimulus spatial frequency was 2–5 c/d [4–8]. Our results were consistent with previous studies; furthermore, it remained through the observation of stimuli (shown in the sustained component of pupillary response).

In Experiment 2, we found that spatially attending to the sine wave grating and Gabor patch with a high spatial frequency (5.57 c/d) elicited a larger pupil constriction relative to that with a low spatial frequency (0.54 c/d). However, there was a significant effect of attentional location for mean pupil change, whereas the significant effect for maximum constriction was not found. Previous studies measured pupillary responses by using achromatic spatial patterns with different contrast and spatial frequencies [23]. They found that the transient component of pupillary response saturated with increasing contrast whereas its sustained component varied linearly with increasing contrast. They argued that these two different components reflected the activities of phasic (M-) and tonic (P-) cells separately. Phasic cells respond quickly to the stimulus onset, whereas tonic cells respond slowly throughout the duration of the stimulus. The different mechanisms for transient and sustained components of pupillary response could explain the difference between maximum constriction and mean pupil change found in the present results, and it is highly likely that the maximum constriction and mean pupil change are differently affected by attention.

We found that the type of spatial attention has different effects on the transient and sustained components of pupillary response. Endogenous attention was dominant in Experiment 2, whereas exogenous attention was dominant in Experiment 1. It has been reported that the time courses of these two kinds of attention are different [31]. Endogenous attention was deployed later relative to exogenous attention, suggesting that participants might, to some extent, have missed the stimulus onset and had a weaker transient pupillary response (phasic). We argue that attentional modulation on PSFR was more clearly reflected in the sustained components. These facts suggested that the effect of attention observed in Experiment 2, namely, the constriction to the middle range of spatial frequency in the sustained component of pupillary response, is induced by the slow process of endogenous attention mediated by the tonic cells.

Previous studies used bright dots and dark dots as stimuli to investigate the effects of feature-based attention on pupillary response [18]. The two kinds of stimuli were separated or overlapped. They found that attention could modulate PLR and argued that pupil size depended on the interaction between the stimulus feature and participants’ attentional state. In Experiment 3, we utilized a similar experimental paradigm; however, the bright dots and dark dots were replaced by high-pass and low-pass filtered images. Participants had a smaller pupil when they attended to the high-pass filtered images compared with when they attended to low-pass filtered images. We concluded that object-based attention could modulate PSFR and supported arguments that pupil size depended on both stimulus feature and participants’ attentional state.

We speculated a possibility in relation to the PSFR. Previous studies have shown that pupils constrict when people shift their eyes from a far location to a near location [32,33]. The coupling of oculomotor responses of vergence, accommodation, and pupil constriction is referred to as the near triad. In Experiment 3b, all the participants gave us feedback about the stimulus appearance. They said that the high-pass filtered object was perceived to be nearer than the low-pass filtered object. This impression is consistent with the results in previous reports that regions with higher spatial frequency (5.65 c/d) appeared closer in depth than those containing lower spatial frequency (0.56 c/d), even when conflictive stereo information existed [34]. Therefore, the perception of different depth accompanied by different spatial frequency leads to accommodation and vergence unconsciously, which results in a smaller pupil size in near vision caused by higher spatial frequency [35–38]. It is recommended that a further experiment to manipulate depth perception be conducted to examine this possibility [39].

Binda et al. showed that attending to a bright disk elicited a smaller pupil compared with attending to a dark disk and proposed that attention might act indirectly in the early visual cortex [17,18]. In this study, we confirmed that attention also modulated PSFR, indicating that pupillary response to luminance and spatial frequency share the same mechanism in the early visual cortex. However, the mechanisms between PLR and PSFR are most likely segregated after the early visual pathway [40–43]. PLR occurs at around 30 weeks’ gestation [40], whereas PSFR becomes reliably visible when infants are 1 month old [41,42]. Furthermore, the latency of PSFR is 40 ms longer than the corresponding PLR latency [3,43]. We speculate a possibility that the visual pathways of PSFR and PLR are the same in the early stage, and attention modulates within the common early stage. The geniculo-striate visual pathway from the retina to V1 might be involved in PSFR, suggesting that the signals from some ganglion cells pass through the superior colliculus to the visual cortex [4,6,16,44]. Because the superior colliculus has been known to be related to attention [45], we speculate that PSFR and PLR share the pathway until signals reach the superior colliculus, and then they become separated.

6. CONCLUSIONS

We provided the first evidence to our knowledge that spatial- and object-based attention can modulate pupillary response to spatial frequency. Furthermore, this study confirmed and broadened the scope of the phenomenon that pupils constrict more when stimulus spatial frequency is within an intermediate range by using a Gabor patch, which contains a specific narrow range of spatial frequency. We also showed the difference in the attentional modulation of transient and sustained components of pupillary response by using two indices, namely, maximum constriction immediately after the stimulus onset and mean pupil change during a sustained period from a few seconds after the stimulus onset.

Based on the present findings, we can provide new thoughts on Human Computer Interaction (HCI). Recently, several methods using the pupillary response for HCI have been proposed. Stoll et al. exploited the pupil dilation elicited by mental effort to get an answer of “yes” or “no” from participants [46]. Mathôt et al., on the other hand, used stimulus with bright or dark background to obtain participants’ selection [47]. We presume it is more practical to use complex natural images than mental load or simple bright/dark images in such an interface because a natural image still contains enough information even it is high-pass or low-pass filtered. As shown in the present study, we can presume their attentional location in natural images with a non-uniform spatial frequency distribution based on the measurements of people’s pupillary response.