Open Access
Add to BibSonomy
Abstract
How brief can a visual stimulus be and still be seen? To answer this question, we developed a digital micromirror device (DMD) based system operating at high speed (22.7 kHz) to control the rapid presentation of visual stimuli and estimated the minimum time required to identify the orientation of tumbling Snellen E letters. Time thresholds were measured in five subjects using a QUEST algorithm to vary the presentation time of the letters subtending either 0.75°, 1.5° and 4.5° on the retina, for two different effective pupil sizes (0.3 and 1 mm). Additionally, to evaluate the effect of defocus on time thresholds, the experiment was repeated with 1.5° letters and induced myopic defocus with 3, 6 and 9 D trial lenses placed in a conjugated pupil plane. We found that subjects were able to identify the orientation of the letters presented as briefly as 5 ms.
© 2018 Optical Society of America under the terms of the OSA Open Access Publishing Agreement
1. Introduction
During the visual process, the image projected on the retina is first sampled by the photoreceptors which convert the information into signals that travel through the optic nerve to the visual cortex where the image is perceived. A long-time interesting topic in vision is the time course of visual perception [1, 2] and the temporal properties that define the visual stimuli. Over the past centuries, time perception, either in terms of succession of images, duration of stimuli or both, has been investigated [2]. To measure the perception time, event related potential (ERP) technics were used to directly measure the brain’s activity on the onset and offset of a visual stimuli [3–7]. Also, psychophysics was used [8] to study visual perception with different tasks such as comprehension, discrimination, detection or identification among other [9–12].
How short can a single visual stimulus be and still be perceived is still an open-question. Different studies have investigated what would be the minimum time that is required for an image to be comprehended [12], a face to be detected or discriminated [10, 13, 14] or a letter to be recognized [11, 15]. It has been shown that to identify a previously or latterly named picture in a rapid serial visual presentation of 6 or 12 images presented between 13 and 80 ms, the image should be presented at least during 13 ms [12]. In facial expression discrimination tasks, subjects were able to discriminate a happy, angry or neutral face when presented during at least 10 ms and a fearful face when presented during at least 20 ms [10]. In these experiments backward masking, i.e. masking the target face before its presentation with a neutral face, was used to prevent awareness of the target faces. Above chance level was found when a target-mask asynchrony of 25 ms was used for discrimination task of a happy, angry and sad faces [14] and 17 ms for detection task of facial expressions [16]. In letter recognition tasks, subjects could surely identify - they were explicitly asked to not guess - a letter among 18 when presented during 16, 19 or 36 ms for the three subjects who performed the experiment. In this task, subjects were fixating a cross and the letters were displayed 2° away from the fixation point [11] and masked with a pattern. In those studies, the time presentation threshold measurements were limited by the display technology as CRT monitors having 75 or 100 Hz refresh rate limited the fastest presentation time to 13 or 10 ms. The question that remains is what would be the result if we are able to present an image much faster, will the subject be able to see and identify it? In other words, what is the shortest time a visual stimulus, such as a letter, can be seen and its orientation identified?
Technology on digital light processing (DLP) devices, developed 30 years ago, offers now the possibility to drive each micromirror of a digital micromirror device (DMD) individually, and present advantages that have been applied to deliver light patterns quickly on the retina [17–19] or to design a programmable aperture in an adaptive optics scanning laser ophthalmoscope (AOSLO) [20].
The fact that the DMD can operate at high frequencies opens the possibility to test the temporal limits of vision by measuring the smallest time interval in which an image can be recognized and identified. The internal memory of these devices allows to accurately control the presentation time by previously uploading a full sequence of images and then displaying them at a known frequency and this precise control over the presentation time to perform psychophysical experiments with control over the time duration of an event. In the current study, we developed and used a DMD-based system operating at high speed (22.7 kHz) placed in a Maxwellian view configuration with a variable pupil size to measure the minimum time required – time threshold – to identify the orientation of briefly presented tumbling E letters.
2. Methods
2.1 System
We developed a new experimental apparatus for a controlled rapid presentation of visual stimuli. The system (Fig. 1) uses a digital micromirror device (DMD) (Vialux, Germany, V7001, controller board V4395, chipset DLP 4100) that consists of a 1024x768 micromirror array with a 13.7 μm mirror pitch for an active mirror area of 14.0x10.5 mm2. .The internal memory of the 64 Gbit on-board SDRAM could store up to 87 thousands of patterns. The micromirrors have two tilt angles ( ± 12° relative to the plane formed by the micromirror array), thus, in our system configuration, each mirror can either guide light into the optical system (ON state) or deflect it outside the system (OFF state), with a nominal micromirror crossover time of 4 μs. All micromirrors can be driven individually and/or synchronously depending on the task. The DMD can operate high contrast black and white (1bit) images with a global switching rate of 22.7 kHz and up to 8 bits images (grayscale) at a speed of 290Hz.
Fig. 1 Optical system and image sequence used to produce a fast visual stimuli presentation. Upper row: Maxwellian view system with the DMD homogenously illuminated with a broadband Xenon lamp and a variable diaphragm, AP1, placed in a conjugated pupil plane to control the effective pupil size. Bottom row: Sequence of images uploaded on the DMD to control the presentation time (t) of the letter E. The sequence of images uploaded on the internal memory of the device contained 100 images, among them 10 white letters on black background. The frequency could be varied between a few images per second and 22.7 kHz which means that, in this configuration, at the highest switching rate of the DMD, the letter E is displayed for 0.44 ms.
A set of telescopes was used to present the patterns on the retina. The relay of lenses provided conjugated pupil (artificial pupil and subject’s pupil) planes and retinal (DMD, R1 and subject’s retina) planes with a magnification of 0.63 from the DMD to the subject’s retina. In the current study, background and test images were presented using the central area of the DMD of 500x500 pixels that subtended 15 degrees on the retina, with a corresponding pixel size of 0.03 degree. We used only 1 bit high contrast black and white images as we wanted to measure the minimum time required for the letter orientation to be identified, using the maximum switching rate of the device.
The DMD was illuminated homogenously with a broadband Xenon lamp (L7810-02 Hamamatsu, Japan) with the UV part of the spectrum blocked. This light source guarantees a stable output over time. To achieve a homogeneous illumination on the DMD, a 100-mm focal length lens was placed in front of the device. The DMD was placed in a Maxwellian view configuration and a variable diaphragm, acting as an artificial pupil, was placed in a conjugated pupil plane in order to control the effective pupil between 0.3 and 1 mm. The luminance at the cornea for a 15x15 degree white uniform field on the DMD was 825 cd/m2 for an effective pupil size of 1mm. A chin rest was used to align the subject on the system while looking at the center of a Maltese cross presented on the DMD.
2.2 Subjects
Five young subjects participated in the experiments. Age ranged from 24 to 34 years old (mean ± SD = 31 ± 4 years). Refractive errors ranged from + 0.5 D to −4 D (mean ± SD = −2.05 ± 1.80 D) and had less than 2 D of cylinder. None of the participants suffered from any systemic disease. Measurements were done monocularly on the right eye of each participant, while the left eye was occluded with an eye patch. Unless otherwise noted, subjects were wearing their corrective spectacles during the experiment except subject S4 with + 0.5 D that did not usually wear any refractive correction. Measurements were performed with the natural pupil of the subjects and controlling the effective pupil size with the diaphragm placed in a conjugated pupil plane of the Maxwellian view system.
A written informed consent was obtained from every subject after a full explanation of the procedure and nature of this study that complied with the tenets of the Declaration of Helsinki and was approved by the Institutional Review Boards of the University of Murcia.
2.3 Psychophysical experiment and protocol
2.3.1 Image generation and DMD control
High contrast Snellen E letters subtending 0.75°, 1.5° and 4.5° on the retina respectively (corresponding to visual acuities of 45, 90 and 270/20 respectively, well above 20/20) were used in the experiment. For each letter size, four sequences of images were generated, consisting of white Snellen E letters – either pointing right, left, up or down – on black background, temporarily sandwiched between white uniform fields.
The image sequences were constructed with 100 images: 10 images containing a Snellen E letter with a given size and orientation sandwiched between 45 white uniform fields (Fig. 1). Choosing 10 successive images of an E letter was arbitrary and seems reasonable to avoid any error due to the mircromirror crossover time.
For each letter size, four image sequences (one for each orientation) were previously uploaded on the internal memory of the DMD, and the displayed time (duration) of the letter E in each sequence was controlled by changing the frequency at which image sequence was shown (i.e. the switching frame rate of the patterns on the DMD). At the maximum switching rate (22.7 kHz) we could show a Snellen E letter during a time of 0.44 milliseconds.
A white uniform field was presented during at least 1 second before and after each sequence. Thus, during the experiment, one trial presentation consisted of a presentation of a white uniform field during one second, then one sequence containing the E letter randomly chosen among the four previously uploaded sequences and again a white uniform field until subject responded to the task.
2.3.2 Psychophysical experiments
A time threshold measurement was used to assess the minimum presentation time of a constant size Snellen E letter to be seen and its orientation identified. Before each measurement, the subject’s head movements were stabilized using a chin rest, while he/she looked at the center of a Maltese cross presented on the DMD. A time threshold measurement was then performed for a constant size of the letter E and consisted of at least 25 trial presentations. In each of them, one of the four sequences was presented and a sound was played, the subject had to report the orientation of the letter in the sequence pressing numerical keypad keys - 8 for up, 4 for left, 2 for down and 6 for right. No response feedback was given. According to subject’s response, the presentation time of the letter E varied from one trial to another by varying the switching rate of the DMD with a QUEST algorithm (Psychtoolbox [21],). The frequency at which the images were presented in each sequence determined the presentation time of the letter E. The maximum switching rate was 22.7 kHz, allowing, in our configuration, presentation times as short as 0.44 ms. Before pressing the response key, subjects were allowed to blink as a white uniform field was displayed until the subject pressed a key, then the next image sequence was displayed while subjects were asked not to blink as exposure time of the letter E was briefer than a blink.
Time thresholds were measured as a function of the size of the letter E. Three different sizes were tested corresponding to the letter E subtending 0.75°, 1.5° and 4.5° on the retina. In this experiment, subjects did not wear their refractive correction. The experiment was performed for a 1 mm effective pupil size and repeated for a 0.3 mm effective pupil diameter, closing the variable diaphragm in the Maxwellian view system.
Additionally, the five subjects repeated the experiment for the intermediate letter size (subtending 1.5° on the retina) and both pupil sizes, with induced myopic defocus of 3, 6 and 9 D. For this experiment, subjects performed the experiment wearing their usual refraction and additional trial lenses were added in a conjugated pupil plane in front of the subject’s eye, so that every subject was exposed to the same amount of myopic defocus.
Time threshold measurements were repeated at least three times and the threshold was calculated averaging the results for each subject and condition.
3. Results
Time threshold as a function of letter size
Figure 2(A)-(B) shows the time threshold measured in the 5 participants and on average across subjects, as a function of letter size, for the two effective pupil sizes. Time threshold decreases with increasing letter size on average across subjects: 7.9 ± 3.5 ms (E letter subtending 0.75°), 6.6 ± 2.3 ms (for 1.5°) and 5.3 ± 1.6 ms (for 4.5°) when using a 1 mm effective pupil diameter, whereas for the 0.3 mm pupil size, the effect of letter size was reduced, with a time threshold on average across subjects of 5.2 ± 1.0 ms (for E letter subtending 0.75°), 4.6 ± 0.4 ms (for 1.5°) and 4.7 ± 0.5ms (for 4.5°). On Fig. 2(A), the upper limit of the data corresponds to the subject with a −4.0 D refraction (threshold = 10.50 ± 2.62 ms on average across the three letter sizes for 1mm effective pupil size) whereas the lower edge of the data corresponds to the subject with a + 0.5 D refraction (threshold = 4.16 ± 0.47 ms on average across the three letter sizes for 1 mm effective pupil size). We thus analyzed the difference in minimum time required to identify a 4.5° letter size versus a 0.75° letter size as a function of subject’s refraction (considering absolute value). Figure 2(C)-(D) shows the difference in threshold for the two different effective pupil sizes. While the difference in time threshold increases with increasing defocus for 1 mm pupil size (slope = 1.36; R2 = 0.98) (Fig. 2(C)), the difference does not seem to depend on the refraction for the 0.3 mm pupil size (Fig. 2(D)).
Fig. 2 Time threshold as a function letter size. Upper row: Time threshold measurements for the 5 participants and on average (green diamonds) across subjects for a 1 mm (A) and 0.3 mm (B) effective pupil size. Error bars stands for standard deviation from at least 3 repeated measurements in each condition. Lower row: Difference in the minimum time required to identify a 4.5° versus a 0.75° letter size as a function of subject’s refraction (absolute values). Data are for the 5 participants (black diamonds) and on average (green diamonds) across subjects for the 1mm (C) and 0.3 mm (D) effective pupil size.
Time threshold as a function of induced defocus
To investigate the effect of defocus on the time threshold for the two effective pupil sizes, we repeated the experiment for the intermediate letter size (1.5° on retina), adding trial lenses of 3, 6 and 9 D in a conjugated pupil plane. Figure 3 shows the time threshold as a function of induced myopic defocus, for each participants (S1 to S5) and on average across subjects (AV) for the two effective pupil sizes. With induced myopic defocus, time threshold increases with an averaged slope of 0.36 ms/D (R2 = 0.99) for 1 mm effective pupil, varying from 4.45 ± 0.57 ms (no defocus, 0 D) to 7.94 ± 1.09 ms (9 D myopic defocus), whereas almost no increase was found for 0.3 mm pupil with an averaged slope of 0.05 ms/D with thresholds varying from 4.39 ± 0.32 ms (0 D) to 5.09 ± 0.68 ms (9 D) across subjects. For each participant, the slope of the increase in time threshold with induced myopic defocus ranged from 0.26 ms/D to 0.46 ms/D for 1 mm effective pupil size and from 0.01 ms/D to 0.1 ms/D for the 0.3 mm effective pupil size.
Fig. 3 Time thresholds as a function of induced myopic defocus with trial lenses of 3, 6 and 9D. Data are for the 5 subjects (S1 to S5) and on average (AV, green diamonds) across subjects for a 1mm (left panels) and 0.3mm (right panels) effective pupil size. Error bars stands for standard deviation from at least 3 repeated measurements in each condition.
4. Discussion
Measurements of very brief (<10 ms) presentation time of an image was possible with the use of a newly developed high speed visual stimuli generator. Five subjects, with different refractive profiles, participated in the experiments and were able to identify the orientation of a single high contrast letter E presented as briefly as 5 ms, much lower than the typical flicker fusion threshold at 50 Hz corresponding to 20 ms [7, 22]. With this relatively low sample size we studied the difference in the minimum time to detect the orientation of the E letter as a function of pupil size and myopic defocus. More complex natural scene –or gray scale images– would have been appropriate to investigate the minimum time required to recognize an image but the switching rate of the digital micromirror device drastically decreases from almost 23 kHz for 1bit image to 290Hz for an 8bits image. Although this rate corresponds to a minimum presented grayscale image of 3.4 ms, in the current configuration, we would not have been able to measure time thresholds with a QUEST algorithm and decided to limit the current study to only high contrast black and white targets.
Differences in time thresholds between subjects –threshold ranging from 3.9 ms to 5.2 ms (mean ± SD = 4.7 ± 0.4 ms) when no defocus is induced (Fig. 3, 0 D induced defocus)– could arise from individual differences in terms of attentional and/or optical factors. When subjects performed the experiment with their corrective refraction and a 1.5° letter size (see Fig. 3, on average data at 0 D Defocus), time threshold was not affected by increasing the effective pupil size –thus increasing the effect of ocular aberrations– with an average threshold across subjects of 4.7 ± 0.5 ms for both pupil sizes (0.3 and 1 mm). When subjects did not wear their spectacles to correct refraction (thus exposed to their own defocus) and performed the experiment for the same 1.5° letter size the time threshold increased from 4.6 ± 0.4 to 6.6 ± 2.3 ms as we increased the effective pupil size from 0.3 to 1 mm. Thus, for this intermediate letter size subtending 1.5°, the ocular aberrations played a role in assessing time threshold and duration is in direct relation with stimulus size as previously suggested [23–25].
In a preliminary experiment, we measured time threshold on subject S2, for 1 mm effective pupil and 1.5° letter on retina, when the sequence of image, either containing a white letter on black background (WoB) or a black letter on white background (BoW), was sandwiched between white or black uniform fields. When using a black uniform field, the orientation of the letter E was accurately identified for each trial even for the smallest presentation time of 0.44 ms and did not depend on the polarity of the letter E (BoW or WoB). The sequence sandwiched between black uniform fields allowed the letter to be seen even for duration as brief as 0.44 ms. With the white uniform field, interestingly, the time threshold did not depend either on the polarity of the letter E, 5.3 ms for the WoB letter and 5.2 ms for the BoW Letter. For this reason, we decided to use, in the current study, a white background before and after WoB letter presentation. This avoided the effect of flashes and minimize visible persistence.
For a faster and faster stimulus, the apparent contrast is reduced as Bloch’s law stipulates: “when the lighting duration varies from almost 2 to 52 ms, the visible light is markedly in inverse proportion to its duration, or for a brief presentation, the product of luminance (or contrast) and duration is constant at the detection threshold” [26–28]. In our system, the subjects never reported that the luminance seemed too low to identify the letter and we did not correct the apparent luminance for each letter presentation because the control over presentation time and luminance of the brief stimulus would have required an automated rotatory neutral density filter wheel synchronized with the DMD switching rate much more complex than the manual wheel used in the experiment.
In the current study, we presented the stimuli only in the fovea although no proper control was performed to make sure that the subject was looking right at the location of the E, though the fixation target provided at the beginning of the experiment showed the position where the letter was going to appear. While we have not studied time threshold for stimulus presented outside the fovea, a previous study [11] has measured the time threshold for single letter recognition where subjects fixated a cross and the letter subtending 0.4° vertically was displayed at 1.8 degrees of eccentricity at 12 possible locations. Thresholds of 16, 19 and 36 ms were measured for the three subjects that were asked not to guess and to respond only when sure about the letter recognition task (among 18 choices – whereas our current experiment only count for four alternative forced choices corresponding to the four orientations of the letter E). These thresholds they report are higher than those measured in the current study when presenting a letter subtending 0.75° in the fovea (7.9 ± 3.5ms on average across subjects). Unfortunately, our current Maxwellian system does not allow to measure time threshold for smaller letter E (due to relay of lenses and retinal magnification considered- which is 0.63 in our case) but with a similar device and lower retinal magnification, the time threshold for smaller letter size could be studied to investigate spatial versus temporal acuity. This experiments can provide information about the time course of the visual stimulus in the brain and the different factors that operate during visual perception.
Funding
European Research Council Advanced Grant (ERC-2013-AdG-339228 (SEECAT)); Secretaría de Estado e Investigación, Desarrollo e Innovación-SEIDI (FIS2016-76163-R); Fundación Séneca-Agencia de Ciencia y Tecnología de la Región de Murcia (19897/GERM/15); European Regional Development Fund (EU-FEDER).
Disclosures
The authors declare that there are no conflicts of interest related to this article.
References and links
1. P. Fraisse, “Influence de la durée du traitement de l’information sur l’estimation d’une durée d’une seconde,” Annee Psychol. 79(2), 495–504 (1979). [CrossRef] [PubMed]
2. P. Fraisse, “Perception and estimation of time,” Annu. Rev. Psychol. 35(1), 1–37 (1984). [CrossRef] [PubMed]
3. O. Hauk, M. H. Davis, M. Ford, F. Pulvermüller, and W. D. Marslen-Wilson, “The time course of visual word recognition as revealed by linear regression analysis of ERP data,” Neuroimage 30(4), 1383–1400 (2006). [CrossRef] [PubMed]
4. P. J. Holcomb and J. Grainger, “On the time course of visual word recognition: an event-related potential investigation using masked repetition priming,” J. Cogn. Neurosci. 18(10), 1631–1643 (2006). [CrossRef] [PubMed]
5. G. F. Woodman, “A brief introduction to the use of event-related potentials in studies of perception and attention,” Atten. Percept. Psychophys. 72(8), 2031–2046 (2010). [CrossRef] [PubMed]
6. S. Kouider and S. Dehaene, “Levels of processing during non-conscious perception: a critical review of visual masking,” Philos. Trans. R. Soc. Lond. B Biol. Sci. 362(1481), 857–875 (2007). [CrossRef] [PubMed]
7. S. Wu, S. A. Burns, and A. E. Elsner, “Effects of flicker adaptation and temporal gain control on the flicker ERG,” Vision Res. 35(21), 2943–2953 (1995). [CrossRef] [PubMed]
8. G. Fechner, Elemente der Psychophysikin (translated to English in 1966, Elements of Psychophysics. Volume I) (Holt, Rinehart and Winston, 1851).
9. L. G. Allan, “Temporal order psychometric functions based on confidence-rating data,” Attention, Perception, \& Psychophysics 18(5), 369–372 (1975). [CrossRef]
10. M. Milders, A. Sahraie, and S. Logan, “Minimum presentation time for masked facial expression discrimination,” Cogn. Emotion 22(1), 63–82 (2008). [CrossRef]
11. C. Bundesen and L. Harms, “Single-letter recognition as a function of exposure duration,” Psychol. Res. 62(4), 275–279 (1999). [CrossRef]
12. M. C. Potter, B. Wyble, C. E. Hagmann, and E. S. McCourt, “Detecting meaning in RSVP at 13 ms per picture,” Atten. Percept. Psychophys. 76(2), 270–279 (2014). [CrossRef] [PubMed]
13. G. Kirouac and F. Y. Doré, “Judgment of facial expressions of emotion as a function of exposure time,” Percept. Mot. Skills 59(1), 147–150 (1984). [CrossRef] [PubMed]
14. M. Calvo and F. Esteves, “Detection of emotional faces: Low perceptual threshold and wide attentional span,” Vis. Cogn. 12(1), 13–27 (2005). [CrossRef]
15. A. Petersen and T. S. Andersen, “The effect of exposure duration on visual character identification in single, whole, and partial report,” J. Exp. Psychol. Hum. Percept. Perform. 38(2), 498–514 (2012). [CrossRef] [PubMed]
16. J. Maxwell and R. Davidson, “Unequally masked: Indexing differences in the perceptual salience of” unseen” facial expressions,” Cogn. Emotion\& Emotion 18(8), 1009–1026 (2004). [CrossRef]
17. K. V. Vienola, M. Damodaran, B. Braaf, K. A. Vermeer, and J. F. de Boer, “Parallel line scanning ophthalmoscope for retinal imaging,” Opt. Lett. 40(22), 5335–5338 (2015). [CrossRef] [PubMed]
18. B. Lochocki, A. Gambín, S. Manzanera, E. Irles, E. Tajahuerce, J. Lancis, and P. Artal, “Single pixel camera ophthalmoscope,” Optica 3(10), 1056–1059 (2016). [CrossRef]
19. J. Lu, B. Gu, X. Wang, and Y. Zhang, “Adaptive optics parallel near-confocal scanning ophthalmoscopy,” Opt. Lett. 41(16), 3852–3855 (2016). [CrossRef] [PubMed]
20. K. A. Sapoznik, T. Luo, A. de Castro, L. Sawides, R. L. Warner, and S. A. Burns, “Enhanced retinal vasculature imaging with a rapidly configurable aperture,” Biomed. Opt. Express 9(3), 1323–1333 (2018). [CrossRef] [PubMed]
21. D. H. Brainard, “The Psychophysics Toolbox,” Spat. Vis. 10(4), 433–436 (1997). [CrossRef] [PubMed]
22. A. O. Holcombe, “Seeing slow and seeing fast: two limits on perception,” Trends Cogn. Sci. (Regul. Ed.) 13(5), 216–221 (2009). [CrossRef] [PubMed]
23. E. A. C. Thomas and W. B. Weaver, “Cognitive processing and time perception,” Atten. Percept. Psychophys. 17(4), 363–367 (1975). [CrossRef]
24. E. A. C. Thomas and N. E. Cantor, “Simultaneous time and size perception,” Percept.& Psychophys. 19(4), 353–360 (1976). [CrossRef]
25. G. M. Long and R. J. Beaton, “The contribution of visual persistence to the perceived duration of brief targets,” Percept. & Psychophys. 28(5), 422–430 (1980). [CrossRef] [PubMed]
26. A. Gorea, “A Refresher of the Original Bloch’s Law Paper (Bloch, July 1885),” Iperception 6(4), 2041669515593043 (2015). [CrossRef] [PubMed]
27. M. A.-M. Bloch, “Expériences sur la vision,” Société de biologie (France). C. R. Seances Soc. Biol. Fil. 1885, 493–495 (1885).
28. E. Baumgardt and B. Hillmann, “Duration and size as determinants of peripheral retinal response,” J. Opt. Soc. Am. 51(3), 340–344 (1961). [CrossRef] [PubMed]
