Visual Counterpart Research Articles

Non solo Roma. Torino, i Savoia e le Fiandre nei Campeggiamenti di Emanuele Tesauro (1639)

Non solo Roma. Torino, i Savoia e le Fiandre nei Campeggiamenti di Emanuele Tesauro (1639) This essay focuses on the siege of Saint-Omer, an episode of the Thirty Years War, as seen through the eyes of several contemporaries. The first part, written by Blythe Alice Raviola, is dedicated to Emanuele Tesauro’s Sant’Omero assediato (1639). Tesauro followed prince Thomas of Savoy as court historiographer during his Flemish campaign; therefore this book should be read within the political and diplomatic context of the Savoy court. Sant’Omero assediato is considered not only as an account of the military operations, but also as a plaidoyer for Thomas’ political and military activity. The second part of the article, by Luca Bianco, establishes several connections with art historical aspects of the siege and of Tesauro’s mission. Bianco examines the subsequent frontispieces of his book and Tesauro’s possible knowledge of Karel Van Mander’s Le livre des Peintres. The final part of the article sheds light on two little-known visual representations of the siege: an artfully drawn map by Pieter Snayers and an etching by Stefano Della Bella. Whereas the first is probably produced for military purposes, Bianco proposes Tuscan Ottavio Piccolomini, Thomas’ fellow condottiero and a noteworthy collector, as the hypothetic commissioner of Della Bella’s etching, which could be considered as a visual counterpart to Tesauro’s literary account.

Feeling form: the neural basis of haptic shape perception.

The tactile perception of the shape of objects critically guides our ability to interact with them. In this review, we describe how shape information is processed as it ascends the somatosensory neuraxis of primates. At the somatosensory periphery, spatial form is represented in the spatial patterns of activation evoked across populations of mechanoreceptive afferents. In the cerebral cortex, neurons respond selectively to particular spatial features, like orientation and curvature. While feature selectivity of neurons in the earlier processing stages can be understood in terms of linear receptive field models, higher order somatosensory neurons exhibit nonlinear response properties that result in tuning for more complex geometrical features. In fact, tactile shape processing bears remarkable analogies to its visual counterpart and the two may rely on shared neural circuitry. Furthermore, one of the unique aspects of primate somatosensation is that it contains a deformable sensory sheet. Because the relative positions of cutaneous mechanoreceptors depend on the conformation of the hand, the haptic perception of three-dimensional objects requires the integration of cutaneous and proprioceptive signals, an integration that is observed throughout somatosensory cortex.

Portrait of My Teacher, Fräulein Sophie von Prieser: Florine Stettheimer.

The dreamlike spectacle that can pop from a painting byFlorineStettheimer (1871-1944)defies easydescription. What to make, for example, of the merry group in SundayAfternoon in the Country that includes a guest gleefully performing a handstand under a pink cotton candy–like tree, and another lying lopsided on his back with a precariously balanced parasol? There seems to be little place for the douranddismalhere, andpartof the fun is the stories that suggest themselves in these sometimespsychedelic scenarios. It’s all in the spirit of savoring the joys of life. Stettheimer was born in Rochester, New York, but she lived in Stuttgart and Berlin, Germany, at times during childhood. Florine’s father Joseph Stettheimer did not remain with the family after the birth of the youngest child, and mother Rosetta and children moved to Germany. Of the 5 children, the 3 youngest were sisters Carrie, Florine, and Ettie, none of whom ever married. What could be little guessed by the sisters in these early years is that they would someday host and mingle with the glittering avant garde of artistic innovation in their New York City salon. As a young girl, Florine was a student at the Priesersches Institut in Stuttgart, and later in life portrayed its directress in Portrait of My Teacher, Fraulein Sophie von Prieser (Muhling M, Althaus K, Boller S, eds. Florine Stettheimer. Munich, Germany: Stadtische Galerie im Lenbachhaus und Kunstbau/Hirmer Verlag GmbH; 2014:1-192). InPortrait, adignifiedwoman inadarkdress trimmedwith lace at the wrists jauntily holds aloft a lorgnette. Stettheimer painted this likenessof vonPrieserplaced ina settingof awellappointed room bounded by a balcony overlooking a palace. An image of a youngFlorine inwhite frock andbrunette locks is reflected in a mirror hanging above the mantel ledge (http://www.portlandartmuseum.us/mwebcgi/mweb.exe ?request=record;id=11685;type=101). Her teacher’s name is worked into the balcony railing behindwhich children appear to be playing a game on the lawn. A filigree chair with plush rosy quilted cushion allows for repose while gazing upon the scenery and goings-on. Florine’s sister Ettie described the classical head resting on themantel as a bust of the goddess Juno (Bloemink BJ. The Life and Art of Florine Stettheimer. NewHaven, CT/London, UK: Yale University Press; 1995:i-303). Posies in a blue and white vase sit atop the tasseledwhite table linen, echoing the pattern of the wall covering. Given the conventions of the era, von Prieser was a woman of some distinction and accomplishment, and exemplifies the contributions and wisdom to be gained from elders. Back in New York City in the 1890s, Stettheimer received tutelage from teachers at the Art Students League who included Kenyon Cox, James Carroll Beckwith, and Harry SiddonsMowbray.Subsequent travel inEuropewith familyprovided an opportunity for extensive exposure to art and culture, but at the onset of World War I, the Stettheimers returned to NewYork, and Florinewould not see Europe again. The use of clear, brilliant pigments by the French artist Henri Matisse was a guiding principle for Stettheimer in her own vivid renderings andwas reflected in her paintings even when her energetic personal style began tomove beyond the early influences. She experiencedher inaugural (itwould also serve as a finale) solo show in 1916; thereafter, she would exhibit in group venues. A turning point in her career occurred in following years that marked a transformation of her work to that of intriguing originality. Indeed, it has been suggested that much of her more important art appeared inmid to later life (Bloemink, pp ix-x). One source of inspiration for her later pictures was to be found in the hubbub and hilarity of family and friends, perhaps exemplified by the antics of beachgoers in her exuberant Lake Placid. The dynamic character of her paintings paralleled the vibe of the 1920s, and the writer Carl Van Vechten suggested that her work could be considered to be the visual counterpart of jazz. OperaalsobecamepartofStettheimer’sartisticoeuvre,and she was asked to participate in designing costumes and sets for Four Saints in Three Acts,which featured a Gertrude Stein libretto. Her contribution to the opera was favorably received, and thus itwasaroundage63 that sheexperienced this measure of public acclaim. Another honor occurred with the contribution of her work to an American art retrospective to be held in 1938 at the Jeu de Paume in Paris. It was not only through the personalities in her paintings that Stettheimerwas able to conveyher enjoyment of life; she also expressed her emotions through poetry. In her posthumously published Crystal Flowers, an entry reads as follows: “Our Parties/Our Picnics/Our Banquets/Our Friends/Have at last a raison d’etre/Seen in color and design/It amusesme/To recreate them/To paint them.”

Less is More, and More is Less, More or Less: The Historical Progression, Aesthetic Characteristics, and Physical Limitations of Minimalism

Since its emergence as an aesthetic category in the mid-twentieth century, minimalism has been contentious amongst scholars of all forms of art. It has been alternately celebrated, questioned, and condemned by not only its critics, but also the artists whose works have been given the historical title “minimalist.” This article explores the emergence of minimalist music, examining its relation to the earlier “avant-garde” works of John Cage and other eclectic influences, such as jazz and Eastern music. In doing so, this article attempts to establish a broad understanding of the elements integral to minimalist music, with a special focus on the composers La Monte Young, Terry Riley, and Steve Reich. The works of Riley and Reich are compared to the works of visual artists Barnett Newman and Sol LeWitt in order to highlight the pivotal elements of the minimalist aesthetic, including repetition, simplicity, and, to borrow Cage’s term, “Unfixity.” This article concludes that the minimalist compositions of the aforementioned composers ultimately demonstrate the integral characteristics of minimalism better than their visual counterparts, due to the temporal nature of music. However, the article seeks to demonstrate the importance of contemplating visual and musical interpretations of minimalism together, as they are complimentary windows into this modern movement.

Smooth pursuit eye and gaze movements to moving auditory targets: Evidence for velocity-specific processing in the auditory system

Auditory motion perception remains poorly understood, in contrast with its well-established visual counterpart. Visual smooth pursuit (SP), a velocity-specific behavior, has been well-quantified during both head-fixed (ocular SP) and head-free (eye + head, or gaze SP) conditions. In contrast, auditory SP tracking has received little attention, despite its potential for demonstrating a motion-specific process in audition. We presented constant-velocity (10–40°/s), free-field auditory (0.2–20 kHz white noise), and visual (LED) targets to head-fixed or head-free subjects while recording ocular and gaze SP responses, respectively. To control for possible SP in the absence of a target, subjects were asked to recreate auditory trajectories after priming with a set of auditory ramps (practiced SP). We found that ocular auditory SP is consistently higher in gain than practiced SP, but variable and lower than visual SP. Further, the gain of auditory gaze SP exceeds ocular auditory SP. Finally, SP of periodic (triangular) motion trajectories revealed that auditory, like visual, SP improves rapidly over time, indicating predictive behavior. In sum, auditory SP closely mimics visual SP but with reduced gain. We propose that auditory motion processing exists, is robust, and recruits a velocity-dependent neural process shared with vision.

The neural basis of tactile motion perception.

The manipulation of objects commonly involves motion between object and skin. In this review, we discuss the neural basis of tactile motion perception and its similarities with its visual counterpart. First, much like in vision, the perception of tactile motion relies on the processing of spatiotemporal patterns of activation across populations of sensory receptors. Second, many neurons in primary somatosensory cortex are highly sensitive to motion direction, and the response properties of these neurons draw strong analogies to those of direction-selective neurons in visual cortex. Third, tactile speed may be encoded in the strength of the response of cutaneous mechanoreceptive afferents and of a subpopulation of speed-sensitive neurons in cortex. However, both afferent and cortical responses are strongly dependent on texture as well, so it is unclear how texture and speed signals are disambiguated. Fourth, motion signals from multiple fingers must often be integrated during the exploration of objects, but the way these signals are combined is complex and remains to be elucidated. Finally, visual and tactile motion perception interact powerfully, an integration process that is likely mediated by visual association cortex.

Audio-Visual Speech Recognition for People with Speech Disorders

recognition of disorder people is a difficult task due to the lack of motor-control of the speech articulators. Multimodal speech recognition can be used to enhance the robustness of disordered speech. This paper introduces an automatic speech recognition system for people with dysarthria speech disorder based on both speech and visual components. The Mel-Frequency Cepestral Coefficients (MFCC) is used as features representing the acoustic speech signal. For the visual counterpart, the Discrete Cosine Transform (DCT) Coefficients are extracted from the speaker's mouth region. Face and mouth regions are detected using the Viola-Jones algorithm. The acoustic and visual input features are then concatenated on one feature vector. Then, the Hidden Markov Model (HMM) classifier is applied on the combined feature vector of acoustic and visual components. The system is tested on isolated English words spoken by disorder speakers from UA-Speech data. Results of the proposed system indicate that visual features are highly effective and can improve the accuracy to reach 7.91% for speaker dependent experiments and 3% for speaker independent experiments.

Evaluation of the attention network test using vibrotactile stimulations.

We report a vibrotactile version of the attention network test (ANT)-the tactile ANT (T-ANT). It has been questioned whether attentional components are modality specific or not. The T-ANT explores alertness, orienting, cognitive control, and their relationships, similar to its visual counterpart, in the tactile modality. The unique features of the T-ANT are in utilizing stimuli on a single plane-the torso-and replacing the original imperative flanker task with a tactile Simon task. Subjects wore a waist belt mounted with two vibrotactile stimulators situated on the back and positioned to the right and left of the spinal column. They responded by pressing keys with their right or left hand in reaction to the type of vibrotactile stimulation (pulsed/continuous signal). On a single trial, an alerting tone was followed by a short tactile (informative/noninformative) peripheral cue and an imperative tactile Simon task target. The T-ANT was compared with a variant of the ANT in which the flanker task was replaced with a visual Simon task. Experimental data showed effects of orienting over control only when the peripheral cues were informative. In contrast to the visual task, interactions between alertness and control or alertness and orienting were not found in the tactile task. A possible rationale for these results is discussed. The T-ANT allows examination of attentional processes among patients with tactile attentional deficits and patients with eyesight deficits who cannot take part in visual tasks. Technological advancement would enable implementation of the T-ANT in brain-imaging studies.

Political Posters for the Twenty-First Century: A Spotlight on the Justseeds Artists' Cooperative

Political Posters for the Twenty-First Century: A Spotlight on the Justseeds Artists' Cooperative

Temporal integration of consecutive tones into synthetic vowels demonstrates perceptual assembly in audition.

Temporal integration is the perceptual process combining sensory stimulation over time into longer percepts that can span over 10 times the duration of a minimally detectable stimulus. Particularly in the auditory domain, such "long-term" temporal integration has been characterized as a relatively simple function that acts chiefly to bridge brief input gaps, and which places integrated stimuli on temporal coordinates while preserving their temporal order information. These properties are not observed in visual temporal integration, suggesting they might be modality specific. The present study challenges that view. Participants were presented with rapid series of successive tone stimuli, in which two separate, deviant target tones were to be identified. Critically, the target tone pair would be perceived as a single synthetic vowel if they were interpreted to be simultaneous. During the task, despite that the targets were always sequential and never actually overlapping, listeners frequently reported hearing just one sound, the synthetic vowel, rather than two successive tones. The results demonstrate that auditory temporal integration, like its visual counterpart, truly assembles a percept from sensory inputs across time, and does not just summate time-ordered (identical) inputs or fill gaps therein. This finding supports the idea that temporal integration is a universal function of the human perceptual system.

Understanding visual metaphor

Using a history documentary ‘The Kinta Story (1949)’, this article explores the ‘grammar’ of visual metaphor. Numerous images can be found in history documentary, while many more are being planned, which can be accessed by people all around the world. These images technically represent producers’ ideas. They construct connotation and meaning for audiences to read as what the readers want. The visuals are highly posed and set in descript locations to make them usable across the globe. They represent actual places or tragedies and they document witness, which symbolically represents moods such as ‘contentment’ and ‘freedom’. It is argued that visual metaphors cannot be described adequately in formal terms only. Rather, they must be considered as visual representations of metaphorical thoughts or concepts and the changing of time and mass. A cognitive definition of metaphor must not, however, distract from potential variations in meaning and impact which arise from the mode of communication through which metaphors are expressed. This study suggests that many of the dissimilarities between verbal metaphor and its visual counterpart are results from the differences regarding what the two modes are able to express easily and efficiently.

On the learnability of auditory concepts

The field of categorization and concept learning research has been dominated by findings involving visual stimuli. Among these findings is the learning difficulty ordering of the family of category structures associated with visual categorical stimuli consisting of four objects defined over three dimensions. This ordering has been observed numerous times in several rigorous studies (Kruschke, 1992; Love et al., 2004; Nosofsky et al., 1994; Shepard et al., 1961; Vigo, 2011a, 2013a. 2013b) and has been influential in shaping current theories of conceptual behavior. In recent work, we have freed the field from this visual bias by examining the learnability of auditory categorical stimuli that are instances of the aforementioned structures. We found that, in general, for auditory categorical stimuli the learning difficulty ordering of these structures is somewhat different from that of their visual counterparts. However, we also found that this difference may be explained and accurately predicted by a simple encoding mechanism proposed in generalized invariance structure theory (GIST; Vigo, 2013b). We view this result as evidence in support of the proposition that a single basic conceptual system underlies the acquisition of both auditory and visual concepts.

The Romantic Prospects of the Duke of Richmond's Moose

George Stubbs's painting The Duke of Richmond's First Bull Moose (1770) offers a visual counterpart to an arresting account addressed to the naturalist Thomas Pennant in Gilbert White's Natural History of Selborne of his 1768 visit to inspect the Duke of Richmond's recently deceased female “moose-deer.” The Duke was one of several aristocrats who acquired moose from Canada; his animals were studied by White, by Pennant, and by the physician and anatomist William Hunter who commissioned Stubbs's work. While Stubbs depicted the animal accurately, he had no knowledge of its natural habitat; the moose is incongruously placed in a mountainous lake landscape during an approaching storm. This romanticized landscape prospect offers a metaphor for the story of the moose in late eighteenth-century Britain. While looking forward and outward to understand the natural world, these depictions of the moose are equally subjective and imaginative and contributed to debates on extinction and the idea of species. The unsuccessful introduction of the moose to Britain defeated prospects for their domestication and cross-breeding. Close reading of Stubbs's painting, White's narrative, Pennant's Arctic Zoology, and Hunter's unpublished scientific paper on the moose suggest the kinds of wonder which mark the efforts on the part of science and the arts to understand this puzzling and mysterious animal.

Musique Concrète Thinking in Visual Music Practice: Audiovisual silence and noise, reduced listening and visual suspension

This article is based on my creative practice as an electroacoustic composer who has developed a practice of audiovisual composition broadly sited within the field of visual music. A brief contextual survey sites my work by first presenting a personal definition of visual music and of a set of conceptual approaches to work in this field. My practice is framed as an attempt to apply ideas and principles taken from musique concrete in an audiovisual domain. I discuss in particular the idea of reduced listening and propose a visual equivalent, visual suspension. I discuss the problems around reduced listening when applied to concrete 'real-world' sounds, and propose that two audio archetypes, silence (or tending-to-silence) and noise (or tending-to-noise), exhibit unique physical and phenomenological properties which sidestep these issues. Observing a similar set of problems around visual suspension, I propose visual counterparts to silence and noise-by relating both to the idea of self-similarity, both temporal and spatial-which exhibit similar properties. In my own work I have found these audiovisual territories to be especially fertile, and to open up avenues for new kinds of sound-image relationships with great creative potential.

Auditory and Visual Mismatch Negativity in Psychiatric Disorders: A Review

Event-related potentials (ERPs) provide an objective index of neurocognitive abnormality or dysfunction in neuropsychiatric disorders. Auditory mismatch negativity (aMMN), which was discovered in the 1980s, is one unique type of ERP. Several lines of evidence suggest that it reflects auditory preattentive (automatic) information processing and the function of glutamatergic N-methyl-D-aspartate receptors. Many studies also have suggested that aMMN is a promising biomarker for some psychiatric disorders such as schizophrenia. Prospective studies combining analyses of several biomarkers, including aMMN, in schizophrenia patients from the prodromal period, are underway. Meanwhile, the existence of a visual counterpart of aMMN (vMMN) has been disputed, and a genuine vMMN was only confirmed in the last decade. Thus, there have been a relatively small number of vMMN studies compared with studies of aMMN, and the mechanism underlying vMMN is still unclear. However, it is well known that abnormal auditory and visual sensory information processing is present in psychiatric disorders, and vMMN is an important component of ERPs studied in psychiatric patients. The aim of this review is to provide an overview of recent aMMN and vMMN findings in several psychiatric disorders, including schizophrenia, major depression, Alzheimer’s disease, and developmental disorders. In doing so, we highlight their potential use as biomarkers of psychiatric disorders.

Exploiting the Auditory Modality in Decision Support: Beneficial "Warning" Effects and Unavoidable Costs

The rate at which technology continues to develop and permeate our lives is such that it has become increasingly easier, and thus more likely, for information to be presented to us via different modalities simultaneously. But to what extent does this confluence of information affect our subsequent judgment and performance? Furthermore, what are the implications for system design when this information is critical to saving our lives and others? This study uses a visual ‘microworld’ simulation of a naval anti-air warfare to investigate whether the content and priority of audio messages that accompany changes in the visual modality assist or hinder performance in the task (identification of change and evaluation of threats). Results indicate that although helping critical change detection, a critical warning in the auditory modality is not as efficient as its visual counterpart. Moreover, audio messages tended to bias threat evaluation towards perceiving objects as more hostile than they were in reality. Such findings have clear implications in regard to the costs and benefits of further exploiting the auditory modality in dynamic visual environments.

Dual pathways for haptic and visual perception of spatial and texture information

Segregation of information flow along a dorsally directed pathway for processing object location and a ventrally directed pathway for processing object identity is well established in the visual and auditory systems, but is less clear in the somatosensory system. We hypothesized that segregation of location vs. identity information in touch would be evident if texture is the relevant property for stimulus identity, given the salience of texture for touch. Here, we used functional magnetic resonance imaging (fMRI) to investigate whether the pathways for haptic and visual processing of location and texture are segregated, and the extent of bisensory convergence. Haptic texture-selectivity was found in the parietal operculum and posterior visual cortex bilaterally, and in parts of left inferior frontal cortex. There was bisensory texture-selectivity at some of these sites in posterior visual and left inferior frontal cortex. Connectivity analyses demonstrated, in each modality, flow of information from unisensory non-selective areas to modality-specific texture-selective areas and further to bisensory texture-selective areas. Location-selectivity was mostly bisensory, occurring in dorsal areas, including the frontal eye fields and multiple regions around the intraparietal sulcus bilaterally. Many of these regions received input from unisensory areas in both modalities. Together with earlier studies, the activation and connectivity analyses of the present study establish that somatosensory processing flows into segregated pathways for location and object identity information. The location-selective somatosensory pathway converges with its visual counterpart in dorsal frontoparietal cortex, while the texture-selective somatosensory pathway runs through the parietal operculum before converging with its visual counterpart in visual and frontal cortex. Both segregation of sensory processing according to object property and multisensory convergence appear to be universal organizing principles.

Acoustic Event Detection Based on Feature-Level Fusion of Audio and Video Modalities

Acoustic event detection (AED) aims at determining the identity of sounds and their temporal position in audio signals. When applied to spontaneously generated acoustic events, AED based only on audio information shows a large amount of errors, which are mostly due to temporal overlaps. Actually, temporal overlaps accounted for more than 70% of errors in the real-world interactive seminar recordings used in CLEAR 2007 evaluations. In this paper, we improve the recognition rate of acoustic events using information from both audio and video modalities. First, the acoustic data are processed to obtain both a set of spectrotemporal features and the 3D localization coordinates of the sound source. Second, a number of features are extracted from video recordings by means of object detection, motion analysis, and multicamera person tracking to represent the visual counterpart of several acoustic events. A feature-level fusion strategy is used, and a parallel structure of binary HMM-based detectors is employed in our work. The experimental results show that information from both the microphone array and video cameras is useful to improve the detection rate of isolated as well as spontaneously generated acoustic events.

Neural Mechanisms of Tactile Motion Integration in Somatosensory Cortex

How are local motion signals integrated to form a global motion percept? We investigate the neural mechanisms of tactile motion integration by presenting tactile gratings and plaids to the fingertips of monkeys, using the tactile analogue of a visual monitor and recording the responses evoked in somatosensory cortical neurons. The perceived directions of the gratings and plaids are measured in parallel psychophysical experiments. We identify a population of somatosensory neurons that exhibit integration properties comparable to those induced by analogous visual stimuli in area MT and find that these neural responses account for the perceived direction of the stimuli across all stimulus conditions tested. The preferred direction of the neurons and the perceived direction of the stimuli can be predicted from the weighted average of the directions of the individual stimulus features, highlighting that the somatosensory system implements a vector average mechanism to compute tactile motion direction that bears striking similarities to its visual counterpart.

Translating Non-Figuration: Heather Dohollau's Poems on 'Pure Visuality'

This article explores the dialogue between poems by Heather Dohollau and art forms that seem particularly alien to verbal ordering, namely so-called ‘abstract’ canvases by Joan Mitchell ( No Daisies ) and Genevieve Asse ( Sans titre , 1988). Dohollau’s ekphrastic texts relate to the painting’s verbal element — its title — which teases us into identifying objects in the painting even as it forbids us to do so. Representational instability is reflected in the free circulation of metaphors set in motion. Words do not aim to dominate their visual counterpart, thus challenging views of ekphrasis as rivalry that are traced from Lessing to the critical rhetoric of W. J. T. Mitchell and his followers. Rather, I argue that Dohollau’s poems seek to translate the painting’s very resistance to language through specifically poetic resources. Words also aim to recreate art perception in its sensorial, preverbal fabric by emphasizing the haptic involvement of sight. Thus Dohollau’s poems gesture towards a visual–verbal bilingualism, transcending the limits of what a single semiotic system can express. In the process, both the form and the ethos of ekphrasis are reinvented.