When Eyes and Ears Offer Different Words: Timing Attention

A written colour name on a screen can be irrelevant to a task and still influence the answer. If the instruction is to respond only to a spoken word, the writing should be ignored. Yet matching information might help, while a competing word might slow the response. Even that distinction is incomplete until we know what serves as the comparison condition. A large difference between matching and conflicting information need not consist entirely of damage caused by distraction.

In 2013, Sarah E. Donohue, Lawrence G. Appelbaum, Christina J. Park, Kenneth C. Roberts and Marty G. Woldorff examined interactions between spoken and written colour words while varying their timing. Their design differed from the familiar Stroop task in which people name the ink colour of a conflicting word. Here, the writing was black. Participants identified either a spoken word or a written word by pressing a key. The question concerned how meaning from another sensory channel influenced that response, depending on when it arrived. [1]

Changing the task without changing the kind of input

The final analysis included fifteen healthy, right-handed volunteers with a mean age of about twenty-three. They were native English speakers with normal visual acuity and colour vision. Three additional participants did not return for the second session, and five were excluded under the study’s response-proportion criterion. Numerous trials allowed repeated comparisons within individuals, but did not turn this small participant sample into a large population survey.

Four words corresponded to the response keys: red, green, blue and yellow. Two different key mappings were used across participants to reduce confounding between particular words and particular fingers. During the auditory-attention portions, the participant responded to the spoken word and ignored the written one. During visual-attention portions, the instruction was reversed. This allowed the researchers to compare the direction of cross-sensory influence using the same kinds of words, rather than comparing unrelated tasks with unrelated materials. [1]

Congruent trials paired matching words across channels. Incongruent trials paired different words, with the irrelevant word also belonging to the set of possible responses. Neutral trials used a different irrelevant colour word that was not mapped to any of the four response keys. For example, brown could accompany a red target without offering a brown response key. Neutral therefore did not mean meaningless. The distractor remained a recognizable word, unlike silence or an empty screen.

Moving the irrelevant word through time

The researchers varied the onset difference between target and distractor from four hundred milliseconds before the target to four hundred milliseconds after it, in steps of one hundred milliseconds. Simultaneous presentation was one point within that range. This design asked whether input must begin together to interact, and whether an irrelevant word still matters when it arrives after the relevant information has already begun.

Participants completed two sessions on separate days. One mixed the different onset relationships within blocks; the other kept the same onset difference throughout a block. Session order was counterbalanced. The interval from the beginning of one trial to the next was also varied, so a constant target-distractor relationship did not make every event in the experiment perfectly predictable. Additional trials presented the target alone in either modality, supplying a reference without any cross-channel word. [1]

Response-time means were calculated from correct trials. Responses had to fall between two hundred and twelve hundred milliseconds after target onset to qualify for the analysis; approximately two percent of otherwise correct responses were outside that window. Accuracy was analysed separately. This is an important measurement boundary: a reported mean response time describes selected successful responses under specified timing criteria, not every action a participant made.

What made the larger difference larger?

Responses to written targets were generally faster than responses to spoken targets. The overall incongruent-minus-congruent difference was also larger when participants attended to sound: roughly fifty-six milliseconds, compared with twenty-four milliseconds when they attended to writing. A visual distractor therefore had a larger total congruency-related influence on the auditory task than the reverse. But calling that entire difference stronger interference would obscure the components of the result. [1]

Relative to the neutral word condition, facilitation from matching information was substantially larger than slowing from conflicting information. The smaller incongruent-minus-neutral component did not differ significantly between the two attended modalities. Much of the asymmetry in the total effect was associated with greater benefit from matching words during auditory attention. The experiment thus reveals assistance from task-irrelevant information as well as competition from it.

The choice of neutral reference makes a practical difference to interpretation. A word that is not assigned a response can still have a meaning that must be processed. A target presented alone removes that additional word entirely. The authors discuss how taking the unimodal condition as the reference would produce a different balance between apparent facilitation and interference. Neither reference is merely a label: each describes a different experimental situation. To understand the numerical effect, the contents of the comparison condition must remain visible.

Early information and an unfinished answer

Congruency-related effects were largest when irrelevant information preceded the target and declined as it arrived later. Effects at some positive onset differences indicate that processing did not end instantly when the target appeared. Later information could still influence the response being formed. This does not identify the timing of every neural step. A key press is a behavioural endpoint after several processes, rather than a direct recording of their locations in the brain.

Earlier neutral distractors were also associated with faster responses, consistent with a general alerting contribution. A preceding event might prepare someone for an approaching target even without matching its meaning. The authors distinguish this possible contribution from the extra benefit associated with the same word appearing across channels. Matching content and advance notice are not interchangeable explanations, although both can support performance within this arrangement. [1]

Another boundary comes from the physical form of the stimuli. A written word is disclosed as a whole when it appears; a spoken word unfolds during articulation. The faster visual responses cannot therefore support a universal claim that vision is always faster or more important than hearing. The study also normalized response times using unimodal baselines, and the asymmetry in overall congruency and facilitation remained. That analysis argues against explaining the entire pattern by baseline speed alone, without removing the importance of the particular linguistic materials.

A specific result, not a personal diagnosis

The main analyses found no significant overall effects or interactions of blocked versus mixed onset arrangements. This does not establish that temporal expectations are ineffective in every setting. The article discusses additional analyses at an extreme onset difference separately. Likewise, neither behavioural timing nor the authors’ interpretation should be converted into a claim that a particular brain region was measured in this experiment. Suggestions for later neural research are not results already obtained.

The supported conclusion is that cross-sensory influence depends jointly on the attended channel, input timing and the meaning of the reference condition. This experiment does not supply an attention-disorder diagnosis, a general intelligence score or a universal rule for working with multimedia. Its more useful lesson concerns how to read an apparent distraction effect: ask which information was relevant, which arrived first, and whether the measured difference represents lost performance, gained assistance or both. Attention is expressed not only in excluding information, but in forming the required response while several meaningful inputs are available.

Sources

[1] Donohue, S. E., Appelbaum, L. G., Park, C. J., Roberts, K. C., and Woldorff, M. G. (2013). “Cross-Modal Stimulus Conflict: The Behavioral Effects of Stimulus Input Timing in a Visual-Auditory Stroop Task.” PLOS ONE, 8(4), e62802. Open access, Creative Commons Attribution. Used with independently written exposition and interpretation. https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0062802