Multimodal Regions Research Articles

Extracting data from human manipulation of objects towards improving autonomous robotic grasping

Humans excel in manipulation tasks, a basic skill for our survival and a key feature in our manmade world of artefacts and devices. In this work, we study how humans manipulate simple daily objects, and construct a probabilistic representation model for the tasks and objects useful for autonomous grasping and manipulation by robotic hands. Human demonstrations of predefined object manipulation tasks are recorded from both the human hand and object points of view. The multimodal data acquisition system records human gaze, hand and fingers 6D pose, finger flexure, tactile forces distributed on the inside of the hand, colour images and stereo depth map, and also object 6D pose and object tactile forces using instrumented objects. From the acquired data, relevant features are detected concerning motion patterns, tactile forces and hand-object states. This will enable modelling a class of tasks from sets of repeated demonstrations of the same task, so that a generalised probabilistic representation is derived to be used for task planning in artificial systems. An object centred probabilistic volumetric model is proposed to fuse the multimodal data and map contact regions, gaze, and tactile forces during stable grasps. This model is refined by segmenting the volume into components approximated by superquadrics, and overlaying the contact points used taking into account the task context. Results show that the features extracted are sufficient to distinguish key patterns that characterise each stage of the manipulation tasks, ranging from simple object displacement, where the same grasp is employed during manipulation (homogeneous manipulation) to more complex interactions such as object reorientation, fine positioning, and sequential in-hand rotation (dexterous manipulation). The framework presented retains the relevant data from human demonstrations, concerning both the manipulation and object characteristics, to be used by future grasp planning in artificial systems performing autonomous grasping.

Ultracompact 2$\,\times\,$2 Photonic Crystal Waveguide Power Splitter Based on Self-Imaging Effect Realized by Asymmetric Interference

The properties of two-fold images, based on asym metric interference in multimode interference-based photonic crystal waveguides, are numerically studied with finite-difference time-domain simulations. With these image properties, 2 × 2 power splitters (PSs) with an ultracompact size are designed. Ar bitrary splitting ratio can be achieved by modifying the sizes (the effective refractive indices) of one pair of rods in the multimode region. As an example, a 3-dB 2 × 2 PS with a size of 24 × 8 μm <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> is presented with an output efficiency as high as 98%. The method can be extended for the design of M × N PSs for photonic integrated circuits applications.

Probabilistic fibre tract analysis of cytoarchitectonically defined human inferior parietal lobule areas reveals similarities to macaques.

The human inferior parietal lobule (IPL) is a multimodal brain region, subdivided in several cytoarchitectonic areas which are involved in neural networks related to spatial attention, language, and higher motor processing. Tracer studies in macaques revealed differential connectivity patterns of IPL areas as the respective structural basis. Evidence for comparable differential fibre tracts of human IPL is lacking. Here, anatomical connectivity of five cytoarchitectonic human IPL areas to 64 cortical targets was investigated using probabilistic tractography. Connection likelihood was assessed by evaluating the number of traces between seed and target against the distribution of traces from that seed to voxels in the same distance as the target. The main fibre tract pattern shifted gradually from rostral to caudal IPL: Rostral areas were predominantly connected to somatosensory and superior parietal areas while caudal areas more strongly connected with auditory, anterior temporal and higher visual cortices. All IPL areas were strongly connected with inferior frontal, insular and posterior temporal areas. These results showed striking similarities with connectivity patterns in macaques, providing further evidence for possible homologies between these two species. This shift in fibre tract pattern supports a differential functional involvement of rostral (higher motor functions) and caudal IPL (spatial attention), with probable overlapping language involvement. The differential functional involvement of IPL areas was further supported by hemispheric asymmetries of connection patterns which showed left-right differences especially with regard to connections to sensorimotor, inferior frontal and temporal areas.

Modality-Specific Neural Effects of Selective Attention to Taste and Odor

The insular cortex is implicated in general attention and in taste perception. The effect of selective attention to taste on insular responses may therefore reflect a general effect of attention or it may be (taste) modality specific. To distinguish between these 2 possibilities, we used functional magnetic resonance imaging to evaluate brain response to tastes and odors while subjects passively sampled the stimuli or performed a detection task. We found that trying to detect a taste (attention to taste) resulted in activation of the primary taste cortex (anterior and mid-dorsal insula) but not in the primary olfactory cortex (piriform). In contrast, trying to detect an odor (attention to odor) increased activity in primary olfactory but not primary gustatory cortex. However, we did identify a region of far anterior insular cortex that responded to both taste and odor "searches." These results demonstrate modality-specific activation of primary taste cortex by attention to taste and primary olfactory cortex by attention to odor and rule out the possibility that either response reflects a general effect of attentional deployment. The findings also support the existence of a multimodal region in far anterior insular cortex that is sensitive to directed attention to taste and smell.

High-performance 90° hybrid based on a silicon-on-insulator multimode interference coupler

We propose a multimode interference coupler (MMI) design for high-index-contrast technologies based on a shallowly etched multimode region, which is, for the first time to our knowledge, directly coupled to deeply etched input and output waveguides. This reduces the phase errors associated with the high-index contrast, while still allowing for a very compact layout. Using this structure, we fabricate a 2 × 4 MMI operating as a 90° hybrid, with a footprint of only 0.65 mm × 0.53 mm, including all the structures necessary to couple light to a fiber array. We experimentally demonstrate a common mode rejection ratio better than -20 dBe and phase errors better than ±5° in a ~50 nm bandwidth.

Sustained Spatial Attention in Touch: Modality-Specific and Multimodal Mechanisms

Sustained attention to a body location results in enhanced processing of tactile stimuli presented at that location compared to another unattended location. In this paper, we review studies investigating the neural correlates of sustained spatial attention in touch. These studies consistently show that activity within modality-specific somatosensory areas (SI and SII) is modulated by sustained tactile-spatial attention. Recent evidence suggests that these somatosensory areas may be recruited as part of a larger cortical network,also including higher-level multimodal regions involved in spatial selection across modalities. We discuss, in turn, the following multimodal effects in sustained tactile-spatial attention tasks. First, cross-modal attentional links between touch and vision, reflected in enhanced processing of task-irrelevant visual stimuli at tactuallyattended locations, are mediated by common (multimodal) representations of external space. Second, vision of the body modulates activity underlying sustained tactile-spatial attention, facilitating attentional modulation of tactile processing in between-hand (when hands are sufficiently far apart) and impairing attentional modulation in within-hand selection tasks. Finally, body posture influences mechanisms of sustained tactile-spatial attention, relying, at least partly, on remapping of tactile stimuli in external, visuallydefined, spatial coordinates. Taken together, the findings reviewed in this paper indicate that sustained spatial attention in touch is subserved by both modality-specific and multimodal mechanisms. The interplay between these mechanisms allows flexible and efficient spatial selection within and across sensory modalities.

Imaging of Microglial Activation and Amyloid Deposition in Dementia with PET

Positron emission tomography (PET) is a well-established imaging modality. Measurement of regional cerebral glucose metabolism (rCMRglc) using PET and [18F]-2- fluoro-2-deoxy-D-glucose (FDG) has become a standard technique in both dementia research and in a diagnostic setting. When measuring rCMRglc in Alzheimer's disease (AD), characteristic reductions in rCMRglc are found in neocortical association areas including the posterior cingulate, precuneus, temporoparietal and frontal multimodal association regions; the primary visual cortex, sensorimotor cortex, basal ganglia and cerebellum are relatively unaffected. FDG-PET has also been used in the study of mild cognitive impairment (MCI) to accurately predict the subsequent decline to AD.

PeakAnalyzer: genome-wide annotation of chromatin binding and modification loci.

BackgroundFunctional genomic studies involving high-throughput sequencing and tiling array applications, such as ChIP-seq and ChIP-chip, generate large numbers of experimentally-derived signal peaks across the genome under study. In analyzing these loci to determine their potential regulatory functions, areas of signal enrichment must be considered relative to proximal genes and regulatory elements annotated throughout the target genome Regions of chromatin association by transcriptional regulators should be distinguished as individual binding sites in order to enhance downstream analyses, such as the identification of known and novel consensus motifs.ResultsPeakAnalyzer is a set of high-performance utilities for the automated processing of experimentally-derived peak regions and annotation of genomic loci. The programs can accurately subdivide multimodal regions of signal enrichment into distinct subpeaks corresponding to binding sites or chromatin modifications, retrieve genomic sequences encompassing the computed subpeak summits, and identify positional features of interest such as intersection with exon/intron gene components, proximity to up- or downstream transcriptional start sites and cis-regulatory elements. The software can be configured to run either as a pipeline component for high-throughput analyses, or as a cross-platform desktop application with an intuitive user interface.ConclusionsPeakAnalyzer comprises a number of utilities essential for ChIP-seq and ChIP-chip data analysis. High-performance implementations are provided for Unix pipeline integration along with a GUI version for interactive use. Source code in C++ and Java is provided, as are native binaries for Linux, Mac OS X and Windows systems.

Surface-emitting dye-doped polymer laser coupled with stimulated resonant Raman scattering

Surface-emitting polymer laser was fabricated with 1,4-bis[2-[4-[N,N-di(p-totyl)amino]phenyl]vinyl]benzene-doped poly(vinyl-pyrrolidone) thin films sandwiched between two distributed Bragg reflector (DBR) mirrors. Under pulsed optical pumping, Fabry–Perot (FP) type resonation resulted in multi-mode laser oscillations depending upon the active film thickness. With increasing excitation wavelengths, an emission peak based on stimulated resonant Raman scattering (SRRS) was superimposed on the multimode band region. When the SRRS peak just overlapped with one of the FP modes, the emission intensity was enhanced and the line width was considerably narrowed. Such SRRS-coupled FP oscillations can be applied to realize a tunable single-mode surface-emitting polymer laser.

Once an island, now the focus of attention

The insula (originally called the ‘‘island of Reil’’) is emerging from its hiding place inside of the human brain. It is easy to find articles and textbooks which show the lateral aspect of the brain but barely mention the insula, if at all (http://www.ninds.nih.gov/disorders/brain_basics/ know_your_brain.htm) or treat it as a deep brain structure, like the amygdala (http://www.scientificamerican.com/ article.cfm?id=faulty-circuits). In fact, in Brodmann’s famous map of cortical cytoarchitectonic areas, it was not even worthy of a number! [See Kurth et al. 2009, PMID: 19822572; Brodmann (1909) described only a posterior granular and an anterior agranular region in the human insular cortex.] Older neuroscientists remember the insula as a portion of the visceral brain, based on prominent writings by Penfield, Mesulam, Saper, and others (Penfield and Faulk 1955; Mesulam and Mufson 1982; Saper 2002); some investigators simply call it a multi-modal region and cite the brief reviews by Augustine (1985, 1996); but for many new investigators who find it unexpectedly activated in their functional imaging study, it is simply an enigma. I came upon the insula by following a neural pathway for pain and temperature using functional anatomical methods (Craig et al. 1994). Eventually, I realized that the cortical terminus of this pathway in the posterior insula of primates provides a homeostatic representation of the physiological condition of the body (Craig 2002). When my collaborators and I performed an imaging study designed to document this interoceptive pathway in humans, we found that an extension of this pathway to the anterior insula correlated with subjective feelings from the body, consonant with the well-known predictions of the James-Lange theory of emotion and the ‘‘somatic marker’’ hypothesis (Craig et al. 2000). At that time, functional imaging was still emerging, and few studies had observed insular activation. Today, a PubMed search for ‘‘insula AND imaging’’ limited to English and Humans pulls up 30,415 references! In order to provide an overview of this vast literature for a 2009 opinion article (Craig 2009), I compiled reports from disparate and unfamiliar branches of neuroscience. To my mind, this burgeoning literature compelled the hypothesis that the anterior insula engenders human awareness, yet only one article had directly addressed this possibility (Klein et al. 2007), and an astonishing number of authors had reported strong activation of the insula without comment. The immediate need for an anthology became obvious, in which leading primary investigators from these disparate fields could re-appraise the role of the insula in light of the new perspective provided by this extraordinary convergence of evidence. I shared this idea with Antoine Bechara, Hugo Critchley, and Steve Petersen, who graciously agreed to serve as Co-Editors. We elected to generate a Special Issue in a leading journal, to enable rapid on-line publication and broad accessibility. Our excitement was validated when every investigator queried agreed to contribute to this Special Issue on the insula. The first six articles of this set are contributed by clinical investigators, and while each focuses on particular aspects of human behavior, each one also emphasizes the multiplicity of functions attributable to the insula (Ibanez et al. 2010; Karnath and Baier 2010; Ackermann and Riecker 2010; Naqvi and Bechara 2010; Paulus and Stein 2010; Seeley 2010). The next three articles address neuroanatomical issues that are significant for our appreciation of how special the human insula really is, and these articles underscore fertile opportunities for neurogenetic and functional analyses (Butti and Hof 2010; Allman et al. 2010; Kurth et al. A. D. Craig (&) Barrow Neurological Institute, Phoenix, AZ, USA e-mail: Bud.Craig@CHW.edu

Cross-modal interactions between human faces and voices involved in person recognition

Faces and voices are key features of human recognition but the way the brain links them together is still unknown. In this study, we measured brain activity using functional magnetic resonance imaging (fMRI) while participants were recognizing previously learned static faces, voices and voice–static face associations. Using a subtraction method between bimodal and unimodal conditions, we observed that voice–face associations activated both unimodal visual and auditory areas, and specific multimodal regions located in the left angular gyrus and the right hippocampus. Moreover, a functional connectivity analysis confirmed the connectivity of the right hippocampus with the unimodal areas. These findings demonstrate that binding faces and voices rely on a cerebral network sustaining different aspects of integration such as sensory inputs processing, attention and memory.

Spectral and electro-optic response of UV-written waveguides in LiNbO_3 single crystals

An experimental study of the spectral and electro-optic response of direct UV-written waveguides in LiNbO3 is reported. The waveguides were written using c.w. laser radiation at 275, 300.3, 302, and 305 nm wavelengths with various writing powers (35-60 mW) and scan speeds (0.1-1.0 mm/sec). Spectral analysis was used to determine the multimode and single mode wavelength regions and, the cut-off point of the fabricated waveguides. Electro-optic characterization of these waveguides reveals that the electro-optic coefficient (r33) decreases for longer writing wavelengths, with a maximum of 31 pm/V for 275 nm and, is reduced to 14 pm/V for waveguides written with 305 nm.

Morphometry of human insular cortex and insular volume reduction in Williams syndrome

Functional imaging in humans and anatomical data in monkeys have implicated the insula as a multimodal sensory integrative brain region. The topography of insular connections is organized by its cytoarchitectonic regions. Previous attempts to measure the insula have utilized either indirect or automated methods. This study was designed to develop a reliable method for obtaining volumetric magnetic resonance imaging (MRI) measurements of the human insular cortex, and to validate that method by examining the anatomy of insular cortex in adults with Williams syndrome (WS) and healthy age-matched controls. Statistical reliability was obtained among three raters for this method, supporting its reproducibility not only across raters, but within different software packages. The procedure described here utilizes native-space morphometry as well as a method for dividing the insula into connectivity-based sub-regions estimated from cytoarchitectonics. Reliability was calculated in both ANALYZE ( N = 3) and BrainImageJava ( N = 10) where brain scans were measured once in each hemisphere by each rater. This highly reliable method revealed total, anterior, and posterior insular volume reduction bilaterally (all p’s < .002) in WS, after accounting for reduced total brain volumes in these participants. Although speculative, the reduced insular volumes in WS may represent a neural risk for the development of hyperaffiliative social behavior with increased specific phobias, and implicate the insula as a critical limbic integrative region. Native-space quantification of the insula may be valuable in the study of neurodevelopmental or neuropsychiatric disorders related to anxiety and social behavior.

Part VI Introduction

Emerging research over the last decade has shown that long-term music training and skill learning can be a strong stimulator for neuroplastic changes in the developing as well as adult brain. Making music places unique demands on the nervous system, leading to strong coupling of perception and action mediated by sensory, motor, and multimodal integrative regions distributed throughout the brain. Furthermore, listening to music and making music (“musicking”) provokes motion, improves and increases between-subject communication and interaction, and is considered to be and experienced as a joyous and rewarding activity. Several reports in this volume have now shown that listening to music and musicking may have the power to make rehabilitation processes not only more enjoyable, but also possibly more effective. Why is music so special and how does listening to and making music achieve its rehabilitative effects? Music is a strong multimodal stimulus that simultaneously transmits visual, auditory, and motoric information to a specialized brain network consisting of frontotemporo-parietal regions whose components are also part of the putative human mirrorneuron system. Among other functions, this system might support the coupling between perceptual events (visual or auditory) and motor actions (leg, arm/hand, or vocal/articulatory actions). Music might be a special vehicle to engage components of this mirror-neuron system. Furthermore, music might also provide an alternative entry point into a “broken” brain system to remediate impaired neural processes or neural connections by engaging and linking up brain centers that would otherwise not be engaged or linked with each other. The chapters in this section will demonstrate several music-based experimental interventions whose effectiveness in clinical populations is demonstrated and whose underlying neurobiological mechanisms are being examined. These neurologically based music therapies are grounded in specific effects on brain networks involved in listening to and making music and on the effects of music on emotion, perception-action mediation, and social cognition networks. Using an instrument or using one’s own voice alone or within a group involves, in a unique way, a feed-forward and feedback circuit in the brain that connects sensory and motor regions. Training-induced and use-dependent plastic changes within the nodal points of these networks and in the fiber bundles connecting these regions might underlie effects that outlast the duration of the actual intervention. Nevertheless, the gold standard of proving the efficacy of an intervention will be a randomized clinical trial (RCT), in which participants are randomly assigned and a new music-based intervention is tested against a gold standard or an established intervention. Having a scientific basis for the interventions and obtaining evidence from RCTs are the most important steps in establishing neurologically based music therapies as accepted interventions. Thus, listening to and making music, including singing, if used appropriately and based on a neurobiological understanding of how music is processed in the brain and what brain regions are active in music making (“musicking”), can serve as a powerful and engaging treatment modality that can enhance brain recovery processes and neuroplasticity in general as shown in the following chapters.

Modulation of activation in multimodal sensory-motor regions by the observation of meaningful touch: an fMRI study

Modulation of activation in multimodal sensory-motor regions by the observation of meaningful touch: an fMRI study

Ultra-Rapid Sensory Responses in the Human Frontal Eye Field Region

Most of what we know about the human frontal eye field (FEF) is extrapolated from studies in animals. There is ample evidence that this region is crucial for eye movements. However, evidence is accumulating that this region also plays a role in sensory processing and that it belongs to a "fast brain" system. We set out to investigate these issues in humans, using intracerebral recordings in patients with drug-refractory epilepsy. Event-related potential recordings were obtained from 11 epileptic patients from within the FEF region while they passed a series of visual and auditory perceptual tests. No eye movement was required. Ultra-rapid responses were observed, with mean onset latencies at 24 ms after stimulus to auditory stimuli and 45 ms to visual stimuli. Such early responses were compatible with cortical routes as assessed with simultaneous recordings in primary auditory and visual cortices. Components were modulated very early by the sensory characteristics of the stimuli, in the 30-60 ms period for auditory stimuli and in the 45-60 ms period for visual stimuli. Although the frontal lobes in humans are generally viewed as being involved in high-level cognitive processes, these results indicate that the human FEF is a remarkably quickly activated multimodal region that belongs to a network of low-level neocortical sensory areas.

Duration matters: Dissociating neural correlates of detection and evaluation of social gaze

The interpretation of interpersonal gaze behavior requires the use of complex cognitive processes and guides social interactions. Among a variety of different gaze characteristics, gaze direction and gaze duration modulate crucially the meaning of the “social gaze”. Nevertheless, prior neuroimaging studies disregarded the relevance of gaze duration by focusing on gaze direction only.The present functional magnetic resonance imaging (fMRI) study focused on the differentiation of these two gaze parameters. Therefore direct gaze displayed by virtual characters was contrasted with averted gaze and, additionally, systematically varied with respect to gaze duration (i.e., 1, 2.5 or 4 s). Consistent with prior findings, behavioral data showed that likeability was higher for direct than for averted gaze and increased linearly with increasing direct gaze duration. On the neural level, distinct brain regions were associated with the processing of gaze direction and gaze duration: (i) the comparison between direct and averted gaze revealed activations in bilateral occipito-temporal regions including the posterior superior temporal sulcus (pSTS); (ii) whereas increasing duration of direct gaze evoked differential neural responses in the medial prefrontal cortex (MPFC) including orbitofrontal and paracingulate regions.The results suggest two complementary cognitive processes related to different gaze parameters. On the one hand, the recruitment of multimodal sensory regions in the pSTS indicates detection of gaze direction via complex visual analysis. On the other hand, the involvement of the MPFC associated with outcome monitoring and mentalizing indicates higher-order social cognitive processes related to evaluation of the ongoing communicational input conveyed by direct gaze duration.

A neural network model of semantic memory linking feature-based object representation and words

Recent theories in cognitive neuroscience suggest that semantic memory is a distributed process, which involves many cortical areas and is based on a multimodal representation of objects. The aim of this work is to extend a previous model of object representation to realize a semantic memory, in which sensory-motor representations of objects are linked with words. The model assumes that each object is described as a collection of features, coded in different cortical areas via a topological organization. Features in different objects are segmented via γ-band synchronization of neural oscillators. The feature areas are further connected with a lexical area, devoted to the representation of words. Synapses among the feature areas, and among the lexical area and the feature areas are trained via a time-dependent Hebbian rule, during a period in which individual objects are presented together with the corresponding words. Simulation results demonstrate that, during the retrieval phase, the network can deal with the simultaneous presence of objects (from sensory-motor inputs) and words (from acoustic inputs), can correctly associate objects with words and segment objects even in the presence of incomplete information. Moreover, the network can realize some semantic links among words representing objects with shared features. These results support the idea that semantic memory can be described as an integrated process, whose content is retrieved by the co-activation of different multimodal regions. In perspective, extended versions of this model may be used to test conceptual theories, and to provide a quantitative assessment of existing data (for instance concerning patients with neural deficits).

The Cortical Dynamics of Intelligible Speech

An important and unresolved question is how the human brain processes speech for meaning after initial analyses in early auditory cortical regions. A variety of left-hemispheric areas have been identified that clearly support semantic processing, although a systematic analysis of directed interactions among these areas is lacking. We applied dynamic causal modeling of functional magnetic resonance imaging responses and Bayesian model selection to investigate, for the first time, experimentally induced changes in coupling among three key multimodal regions that were activated by intelligible speech: the posterior and anterior superior temporal sulcus (pSTS and aSTS, respectively) and pars orbitalis (POrb) of the inferior frontal gyrus. We tested 216 different dynamic causal models and found that the best model was a "forward" system that was driven by auditory inputs into the pSTS, with forward connections from the pSTS to both the aSTS and the POrb that increased considerably in strength (by 76 and 150%, respectively) when subjects listened to intelligible speech. Task-related, directional effects can now be incorporated into models of speech comprehension.

Emotional Context Enhances Auditory Novelty Processing in Superior Temporal Gyrus

Visualizing emotionally loaded pictures intensifies peripheral reflexes toward sudden auditory stimuli, suggesting that the emotional context may potentiate responses elicited by novel events in the acoustic environment. However, psychophysiological results have reported that attentional resources available to sounds become depleted, as attention allocation to emotional pictures increases. These findings have raised the challenging question of whether an emotional context actually enhances or attenuates auditory novelty processing at a central level in the brain. To solve this issue, we used functional magnetic resonance imaging to first identify brain activations induced by novel sounds (NOV) when participants made a color decision on visual stimuli containing both negative (NEG) and neutral (NEU) facial expressions. We then measured modulation of these auditory responses by the emotional load of the task. Contrary to what was assumed, activation induced by NOV in superior temporal gyrus (STG) was enhanced when subjects responded to faces with a NEG emotional expression compared with NEU ones. Accordingly, NOV yielded stronger behavioral disruption on subjects' performance in the NEG context. These results demonstrate that the emotional context modulates the excitability of auditory and possibly multimodal novelty cerebral regions, enhancing acoustic novelty processing in a potentially harming environment.