Multisensory Cues Research Articles

Multisensory Effects on Illusory Self-Motion (Vection): the Role of Visual, Auditory, and Tactile Cues.

A critical component to many immersive experiences in virtual reality (VR) is vection, defined as the illusion of self-motion. Traditionally, vection has been described as a visual phenomenon, but more recent research suggests that vection can be influenced by a variety of senses. The goal of the present study was to investigate the role of multisensory cues on vection by manipulating the availability of visual, auditory, and tactile stimuli in a VR setting. To achieve this, 24 adults (Mage = 25.04) were presented with a rotating stimulus aimed to induce circular vection. All participants completed trials that included a single sensory cue, a combination of two cues, or all three cues presented together. The size of the field of view (FOV) was manipulated across four levels (no-visuals, small, medium, full). Participants rated vection intensity and duration verbally after each trial. Results showed that all three sensory cues induced vection when presented in isolation, with visual cues eliciting the highest intensity and longest duration. The presence of auditory and tactile cues further increased vection intensity and duration compared to conditions where these cues were not presented. These findings support the idea that vection can be induced via multiple types of sensory inputs and can be intensified when multiple sensory inputs are combined.

Immersive Digital Tourism: The Role of Multisensory Cues in Digital Museum Experiences

Facilitated by emerging technologies, the immersive digital museum reflects disruptive innovation in today’s tourism experience and offers a multidimensional experience different from traditional museums. To better understand how visitors respond to this innovative form of digital tourism, the current research investigates visitor experiences at the digital museum and achieves a threefold goal. First, this research delineates a three-dimensional digital museum visitor experience, namely, joviality, personal escapism, and localness. Built on this experiential framework, the present research further affirms that visual and auditory cues are the most powerful multisensory cue combination in enhancing a holistic visitor experience at the digital museum. This study also finds that emotional state and sense of presence mediate the relations between the multisensory cues and visitors’ digital museum experiences. This research contributes to the conceptualization of a digital museum experience, and provides a foundation for the future research endeavor of the new generation of digital tourism.

Temporal binding as multisensory integration: Manipulating perceptual certainty of actions and their effects

It has been proposed that statistical integration of multisensory cues may be a suitable framework to explain temporal binding, that is, the finding that causally related events such as an action and its effect are perceived to be shifted towards each other in time. A multisensory approach to temporal binding construes actions and effects as individual sensory signals, which are each perceived with a specific temporal precision. When they are integrated into one multimodal event, like an action-effect chain, the extent to which they affect this event’s perception depends on their relative reliability. We test whether this assumption holds true in a temporal binding task by manipulating certainty of actions and effects. Two experiments suggest that a relatively uncertain sensory signal in such action-effect sequences is shifted more towards its counterpart than a relatively certain one. This was especially pronounced for temporal binding of the action towards its effect but could also be shown for effect binding. Other conceptual approaches to temporal binding cannot easily explain these results, and the study therefore adds to the growing body of evidence endorsing a multisensory approach to temporal binding.

On the Use of Multi-sensory Cues in Symmetric and Asymmetric Shared Collaborative Virtual Spaces

Physical face-to-face collaboration with someone gives a higher-quality experience compared to mediated communication options, such as a phone- or video-based chat. Participants can share rich sensory cues to multiple human senses in a physical space. Also, the perceptual sensing of the surrounding environment including other peoples' reactions can influence human communication and emotion, and thus collaborative performance. Shared spaces in virtual environments provide degraded sensory experiences because most commercial virtual reality systems typically provide only visual and audio feedback. The impact of richer, multi-sensory feedback on joint decision-making tasks in VR is still an open area of research. Two independent studies exploring this topic are presented in this paper. We implemented a multi-sensory system that delivers vision, audio, tactile, and smell feedback, and we compared the system to a typical VR system. The scenario placed two users in a virtual theme-park safari ride with a number of non-player character (NPC) passengers to simulate realistic scenarios compared to the real-world and we varied the type and complexity of NPCs reactions to participants. In Experiment 1, we provided both users with either multi-sensory or typical sensory feedback symmetrically as a between-subjects factor, and used NPC reaction type as a within-subjects factor. In Experiment 2, we provided sensory feedback asymmetrically to each user (i.e., one had multi-sensory cues and the other had typical sensory cues) as a between-subjects factor, and used NPC reaction type as a within-subjects factor. We found that the number of sensory channels and NPC reactions did not influence user perception significantly under either symmetric or asymmetric sensory feedback conditions. However, after accounting for individual personality traits (e.g., assertive, passive), as well as any existing relationship between the pairs, we found that increasing the number of sensory channels can significantly improve subjective responses.

Effects of Interfering Cue Separation Distance and Amplitude on the Haptic Detection of Skin Stretch.

Multi-sensory haptic cues, which contain several types of tactile stimuli that are presented concurrently to the user, have been shown to be useful for conveying information-rich cues. One limitation of multi-sensory cues is that user perception of individual cue components can be hindered by more salient components of the composite cue. In this article, we investigate how amplitude and distance between cues affect the perception of multi-sensory haptic cues. Specifically, participants' absolute threshold perception of stretch cues was measured in the presence of interfering squeeze cues using a modular testbed. We evaluated ten conditions of varying interference amplitude and distance between cues. We found that interference cue amplitude and distance between cues both have a statistically significant effect on the absolute perception of stretch cues. As interference cue amplitude increases, and as distance between cues decreases, absolute perception of stretch cues worsens. These results inform design considerations for future wearable multi-sensory haptic devices, so that cue salience can be maximized and interference effects minimized.

Transfer of training—Virtual reality training with augmented multisensory cues improves user experience during training and task performance in the real world

Virtual reality (VR) can create safe, cost-effective, and engaging learning environments. It is commonly assumed that improvements in simulation fidelity lead to better learning outcomes. Some aspects of real environments, for example vestibular or haptic cues, are difficult to recreate in VR, but VR offers a wealth of opportunities to provide additional sensory cues in arbitrary modalities that provide task relevant information. The aim of this study was to investigate whether these cues improve user experience and learning outcomes, and, specifically, whether learning using augmented sensory cues translates into performance improvements in real environments. Participants were randomly allocated into three matched groups: Group 1 (control) was asked to perform a real tyre change only. The remaining two groups were trained in VR before performance was evaluated on the same, real tyre change task. Group 2 was trained using a conventional VR system, while Group 3 was trained in VR with augmented, task relevant, multisensory cues. Objective performance, time to completion and error number, subjective ratings of presence, perceived workload, and discomfort were recorded. The results show that both VR training paradigms improved performance for the real task. Providing additional, task-relevant cues during VR training resulted in higher objective performance during the real task. We propose a novel method to quantify the relative performance gains between training paradigms that estimates the relative gain in terms of training time. Systematic differences in subjective ratings that show comparable workload ratings, higher presence ratings and lower discomfort ratings, mirroring objective performance measures, were also observed. These findings further support the use of augmented multisensory cues in VR environments as an efficient method to enhance performance, user experience and, critically, the transfer of training from virtual to real environment scenarios.

Attention and sensory integration for postural control in young adults with autism spectrum disorders.

Postural control impairments have been reported in adults with autism spectrum disorders (ASD). Balance relies on the integration of multisensory cues, a process that requires attention. The purpose of this study was to determine if the influence of attention demands on sensory integration abilities relevant for balance partially contributes to postural control impairments in ASD. Young adults with ASD (N = 24) and neurotypical participants (N = 24) were exposed to sensory perturbations during standing. An established dual-task paradigm was used, requiring participants to maintain balance in these sensory challenging environments and to perform auditory information processing tasks (simple reaction time task and choice reaction time task). Balance was assessed using sway magnitude and sway speed, and attention demands were evaluated based on the response time in the auditory tasks. While young adults with ASD were able to maintain balance in destabilizing sensory conditions, they were more challenged (greater sway speed) than their neurotypical counterparts. Additionally, when exposed for an extended amount of time (3min) to the most challenging sensory condition included in this study, adults with ASD exhibited a reduced ability to adapt their postural control strategies (sway speed was minimally reduced), demonstrating a postural inflexibility pattern in ASD compared to neurotypical counterparts. Finally, the impact of performing an auditory information processing task on balance and the dual-task cost on information processing (response time) was similar in both groups. ASD may disrupt temporal adaptive postural control processes associated with sensory reweighting that occurs in neurotypicals.

Specular Reflection Detection and Inpainting in Transparent Object through MSPLFI

Multispectral polarimetric light field imagery (MSPLFI) contains significant information about a transparent object’s distribution over spectra, the inherent properties of its surface and its directional movement, as well as intensity, which all together can distinguish its specular reflection. Due to multispectral polarimetric signatures being limited to an object’s properties, specular pixel detection of a transparent object is a difficult task because the object lacks its own texture. In this work, we propose a two-fold approach for determining the specular reflection detection (SRD) and the specular reflection inpainting (SRI) in a transparent object. Firstly, we capture and decode 18 different transparent objects with specularity signatures obtained using a light field (LF) camera. In addition to our image acquisition system, we place different multispectral filters from visible bands and polarimetric filters at different orientations to capture images from multisensory cues containing MSPLFI features. Then, we propose a change detection algorithm for detecting specular reflected pixels from different spectra. A Mahalanobis distance is calculated based on the mean and the covariance of both polarized and unpolarized images of an object in this connection. Secondly, an inpainting algorithm that captures pixel movements among sub-aperture images of the LF is proposed. In this regard, a distance matrix for all the four connected neighboring pixels is computed from the common pixel intensities of each color channel of both the polarized and the unpolarized images. The most correlated pixel pattern is selected for the task of inpainting for each sub-aperture image. This process is repeated for all the sub-aperture images to calculate the final SRI task. The experimental results demonstrate that the proposed two-fold approach significantly improves the accuracy of detection and the quality of inpainting. Furthermore, the proposed approach also improves the SRD metrics (with mean F1-score, G-mean, and accuracy as 0.643, 0.656, and 0.981, respectively) and SRI metrics (with mean structural similarity index (SSIM), peak signal-to-noise ratio (PSNR), mean squared error (IMMSE), and mean absolute deviation (MAD) as 0.966, 0.735, 0.073, and 0.226, respectively) for all the sub-apertures of the 18 transparent objects in MSPLFI dataset as compared with those obtained from the methods in the literature considered in this paper. Future work will exploit the integration of machine learning for better SRD accuracy and SRI quality.

Does Movement Amplitude of a Co-performer Affect Individual Performance in Musical Synchronization?

Interpersonal coordination in musical ensembles often involves multisensory cues, with visual information about body movements supplementing co-performers’ sounds. Previous research on the influence of movement amplitude of a visual stimulus on basic sensorimotor synchronization has shown mixed results. Uninstructed visuomotor synchronization seems to be influenced by amplitude of a visual stimulus, but instructed visuomotor synchronization is not. While music performance presents a special case of visually mediated coordination, involving both uninstructed (spontaneously coordinating ancillary body movements with co-performers) and instructed (producing sound on a beat) forms of synchronization, the underlying mechanisms might also support rhythmic interpersonal coordination in the general population. We asked whether visual cue amplitude would affect nonmusicians’ synchronization of sound and head movements in a musical drumming task designed to be accessible regardless of musical experience. Given the mixed prior results, we considered two competing hypotheses. H1: higher amplitude visual cues will improve synchronization. H2: different amplitude visual cues will have no effect on synchronization. Participants observed a human-derived motion capture avatar with three levels of movement amplitude, or a still image of the avatar, while drumming along to the beat of tempo-changing music. The moving avatars were always timed to match the music. We measured temporal asynchrony (drumming relative to the music), predictive timing, ancillary movement fluctuation, and cross-spectral coherence of ancillary movements between the participant and avatar. The competing hypotheses were tested using conditional equivalence testing. This method involves using a statistical equivalence test in the event that standard hypothesis tests show no differences. Our results showed no statistical differences across visual cues types. Therefore, we conclude that there is not a strong effect of visual stimulus amplitude on instructed synchronization.

Multisensory Integration and Sense Modalism

The Bayesian model of multisensory cue integration proposed by Ernst and Banks ([2002]) provides an attractive model for understanding a way that our sensory systems may interact. Moreover, it has been suggested that the process of multisensory integration that it models underpins conscious experiences with multisensory representational contents merged across modalities (de Vignemont [2014b]). Should we therefore take empirical support for the Bayesian model as evidence of the multimodality of perception? Focusing on evidence of integration across vision and touch, I argue that apparent support for the model does not warrant the rejection of the view that each of our conscious perceptual experiences is associated with one and only one sense modality.

Adaptive weighting of taste and odor cues during flavor choice.

Colloquially referred to as "taste," flavor is in reality a thoroughly multisensory experience. Yet, a mechanistic understanding of the multisensory computations underlying flavor perception and food choice is lacking. Here, we used a multisensory flavor choice task in rats to test specific predictions of the statistically optimal integration framework, which has previously yielded much insight into cue integration in other multisensory systems. Our results confirm three key predictions of this framework in the unique context of flavor choice behavior, providing novel mechanistic insight into multisensory flavor processing.NEW & NOTEWORTHY The authors demonstrate that rats make choices about which flavor solution (i.e., taste-odor mixture) to consume by weighting the individual taste and odor components according to the reliability of the information they provide about which solution is the preferred one. A similar weighting operation underlies multisensory cue combination in other domains and offers novel insight into the computations underlying multisensory flavor perception and food choice behavior.

An immersive virtual reality game to train spatial attention orientation after stroke: A feasibility study

Immersive virtual reality (IVR) may boost neglect recovery, as it can provide an engaging experience in a 3D environment. We designed an IVR rehabilitation game for neglect patients using the Oculus Rift. Multisensory cues were presented in the neglected visual field in a patient-tailored way. We acquired pilot data in 15 neurologically healthy controls and 7 stroke patients. First, we compared cybersickness before and after VR exposure. Second, we assessed the user experience through a questionnaire. Third, we tested whether neglect symptoms corresponded between the VR game and a computerized cancelation task. Fourth, we evaluated the effect of the multisensory cueing on target discrimination. Last, we tested two algorithms to tailor the game to the characteristics of the neglected visual field. Cybersickness significantly reduced after VR exposure in six stroke patients and was low in healthy controls. Patients rated the user experience neutral to positive. In addition, neglect symptoms were consistent between a computerized cancelation and VR rehabilitation task. The multisensory cue positively affected target discrimination in the game and we successfully presented sensory stimulation to the neglected visual field in a patient-tailored way. Our results show that it is promising to use gamified patient-tailored immersive VR for neglect rehabilitation.

Multisensory cueing facilitates naming in aphasia

BackgroundImpaired naming is a ubiquitous symptom in all types of aphasia, which often adversely impacts independence, quality of life, and recovery of affected individuals. Previous research has demonstrated that naming can be facilitated by phonological and semantic cueing strategies that are largely incorporated into the treatment of anomic disturbances. Beneficial effects of cueing, whereby naming becomes faster and more accurate, are often attributed to the priming mechanisms occurring within the distributed language network.ObjectiveWe proposed and explored two novel cueing techniques: (1) Silent Visuomotor Cues (SVC), which provided articulatory information of target words presented in the form of silent videos, and (2) Semantic Auditory Cues (SAC), which consisted of acoustic information semantically relevant to target words (ringing for “telephone”). Grounded in neurophysiological evidence, we hypothesized that both SVC and SAC might aid communicative effectiveness possibly by triggering activity in perceptual and semantic language regions, respectively.MethodsTen participants with chronic non-fluent aphasia were recruited for a longitudinal clinical intervention. Participants were split into dyads (i.e., five pairs of two participants) and required to engage in a turn-based peer-to-peer language game using the Rehabilitation Gaming System for aphasia (RGSa). The objective of the RGSa sessions was to practice communicative acts, such as making a request. We administered SVCs and SACs in a pseudorandomized manner at the moment when the active player selected the object to be requested from the interlocutor. For the analysis, we compared the times from selection to the reception of the desired object between cued and non-cued trials.ResultsNaming accuracy, as measured by a standard clinical scale, significantly improved for all stimuli at each evaluation point, including the follow-up. Moreover, the results yielded beneficial effects of both SVC and SAC cues on word naming, especially at the early intervention sessions when the exposure to the target lexicon was infrequent.ConclusionsThis study supports the efficacy of the proposed cueing strategies which could be integrated into the clinic or mobile technology to aid naming even at the chronic stages of aphasia. These findings are consistent with sensorimotor accounts of language processing, suggesting a coupling between language, motor, and semantic brain regions.Trial registrationNCT02928822. Registered 30 May 2016.

Deploying attention to the target location of a pointing action modulates audiovisual processes at nontarget locations.

The current study examined how the deployment of spatial attention at the onset of a pointing movement influenced audiovisual crossmodal interactions at the target of the pointing action and at nontarget locations. These interactions were quantified by measuring the susceptibility to the fission (i.e., reporting two visual flashes under one flash and two auditory beep pairings) and fusion (i.e., reporting one flash under two flashes and one beep pairing)audiovisual illusions. At movement onset, unimodal, or auditory and visual bimodal stimuli were either presented at the target of the pointing action or in an adjacent, nontarget location. In Experiment 1, perceptual accuracy within the unimodal and bimodal conditions was lower in the nontarget relative to the target condition. The fission illusion was uninfluenced by target condition. However, the fusion illusion was more likely to be reported at the target relative to the nontarget location. In Experiment 2, the stimulifrom Experiment 1 were further presented at a location near where the eyes were fixated (i.e., congruent condition), where the hand was aiming (i.e., target), or in a location where neither the eyes were fixated nor the hand was aiming. The results yielded the greatest susceptibility to the fusion illusion when the visual location and movement end points were congruent relative to when either movement or fixation was incongruent. Although attention may facilitate the processing of unisensoryand multisensory cues in general, attention might have the strongest influence on the audiovisual integration mechanisms that underlie the sound-induced fusion illusion.

Somatosensory-visual effects in visual biological motion perception.

Social cognition is dependent on the ability to extract information from human stimuli. Of those, patterns of biological motion (BM) and in particular walking patterns of other humans, are prime examples. Although most often tested in isolation, BM outside the laboratory is often associated with multisensory cues (i.e. we often hear and see someone walking) and there is evidence that vision-based judgments of BM stimuli are systematically influenced by motor signals. Furthermore, cross-modal visuo-tactile mechanisms have been shown to influence perception of bodily stimuli. Based on these observations, we here investigated if somatosensory inputs would affect visual BM perception. In two experiments, we asked healthy participants to perform a speed discrimination task on two point light walkers (PLW) presented one after the other. In the first experiment, we quantified somatosensory-visual interactions by presenting PLW together with tactile stimuli either on the participants' forearms or feet soles. In the second experiment, we assessed the specificity of these interactions by presenting tactile stimuli either synchronously or asynchronously with upright or inverted PLW. Our results confirm that somatosensory input in the form of tactile foot stimulation influences visual BM perception. When presented with a seen walker's footsteps, additional tactile cues enhanced sensitivity on a speed discrimination task, but only if the tactile stimuli were presented on the relevant body-part (under the feet) and when the tactile stimuli were presented synchronously with the seen footsteps of the PLW, whether upright or inverted. Based on these findings we discuss potential mechanisms of somatosensory-visual interactions in BM perception.

The interplay between multisensory integration and perceptual decision making

Facing perceptual uncertainty, the brain combines information from different senses to make optimal perceptual decisions and to guide behavior. However, decision making has been investigated mostly in unimodal contexts. Thus, how the brain integrates multisensory information during decision making is still unclear. Two opposing, but not mutually exclusive, scenarios are plausible: either the brain thoroughly combines the signals from different modalities before starting to build a supramodal decision, or unimodal signals are integrated during decision formation. To answer this question, we devised a paradigm mimicking naturalistic situations where human participants were exposed to continuous cacophonous audiovisual inputs containing an unpredictable signal cue in one or two modalities and had to perform a signal detection task or a cue categorization task. First, model-based analyses of behavioral data indicated that multisensory integration takes place alongside perceptual decision making. Next, using supervised machine learning on concurrently recorded EEG, we identified neural signatures of two processing stages: sensory encoding and decision formation. Generalization analyses across experimental conditions and time revealed that multisensory cues were processed faster during both stages. We further established that acceleration of neural dynamics during sensory encoding and decision formation was directly linked to multisensory integration. Our results were consistent across both signal detection and categorization tasks. Taken together, the results revealed a continuous dynamic interplay between multisensory integration and decision making processes (mixed scenario), with integration of multimodal information taking place both during sensory encoding as well as decision formation.

A dual‐process model of how incorporating audio‐visual sensory cues in video advertising promotes active attention

AbstractCapturing consumers’ attention to video advertising is a key marketing challenge. Using real video advertising viewed by a sample of consumers matching the US population on age and gender, the authors examined attention to audio‐visual and visual sensory cues identifying the brand using a biometric measure, phasic heart rate. The findings show differences in attention type depending on which senses are stimulated by the sensory cue. This research uncovers a mechanism (internal processing) for how consumers process multisensory cues (audio‐visual) in video advertising and confirms visual sensory cues can elicit automatic attention through the orienting response. In addition to the dual‐coding explanation for the superiority of audio‐visual sensory cues, the present research suggests a second reason: that audio‐visual sensory cues can elicit additional internal processing of the brand name, resulting in active attention and better storage of the brand in memory. This has implications for the limited capacity model of motivated mediated message processing, as well as for marketers relating to the type of content recommended to follow an audio‐visual or visual sensory cue in video advertising.

The Physical-Virtual Patient Simulator: A Physical Human Form With Virtual Appearance and Behavior.

We introduce a new type of patient simulator referred to as the Physical-Virtual Patient Simulator (PVPS). The PVPS combines the tangible characteristics of a human-shaped physical form with the flexibility and richness of a virtual patient. The PVPS can exhibit a range of multisensory cues, including visual cues (eg, capillary refill, facial expressions, appearance changes), auditory cues (eg, verbal responses, heart sounds), and tactile cues (eg, localized temperature, pulse). We describe the implementation of the technology, technical testing with healthcare experts, and an institutional review board-approved pilot experiment involving 22 nurse practitioner students interacting with a simulated child in 2 scenarios: sepsis and child abuse. The nurse practitioners were asked qualitative questions about ease of use and the cues they noticed. Participants found it easy to interact with the PVPS and had mixed but encouraging responses regarding realism. In the sepsis scenario, participants reported the following cues leading to their diagnoses: temperature, voice, mottled skin, attitude and facial expressions, breathing and cough, vitals and oxygen saturation, and appearance of the mouth and tongue. For the child abuse scenario, they reported the skin appearance on the arms and abdomen, perceived attitude, facial expressions, and inconsistent stories. We are encouraged by the initial results and user feedback regarding the perceived realism of visual (eg, mottling), audio (eg, breathing sounds), and tactile (eg, temperature) cues displayed by the PVPS, and ease of interaction with the simulator.

Are two cues always better than one? The role of multiple intra-sensory cues compared to multi-cross-sensory cues in children's incidental category learning

Simultaneous presentation of multisensory cues has been found to facilitate children's learning to a greater extent than unisensory cues (e.g., Broadbent, White, Mareschal, & Kirkham, 2017). Current research into children's multisensory learning, however, does not address whether these findings are due to having multiple cross-sensory cues that enhance stimuli perception or a matter of having multiple cues, regardless of modality, that are informative to category membership. The current study examined the role of multiple cross-sensory cues (e.g., audio-visual) compared to multiple intra-sensory cues (e.g., two visual cues) on children's incidental category learning. On a computerized incidental category learning task, children aged six to ten years (N = 454) were allocated to either a visual-only (V: unisensory), auditory-only (A: unisensory), audio-visual (AV: multisensory), Visual-Visual (VV: multi-cue) or Auditory-Auditory (AA: multi-cue) condition. In children over eight years of age, the availability of two informative cues, regardless of whether they had been presented across two different modalities or within the same modality, was found to be more beneficial to incidental learning than with unisensory cues. In six-year-olds, however, the presence of multiple auditory cues (AA) did not facilitate learning to the same extent as multiple visual cues (VV) or when cues were presented across two different modalities (AV). The findings suggest that multiple sensory cues presented across or within modalities may have differential effects on children's incidental learning across middle childhood, depending on the sensory domain in which they are presented. Implications for the use of multi-cross-sensory and multiple-intra-sensory cues for children's learning across this age range are discussed.

Effects of Multisensory Cues, Self-Enhancing Imagery and Self Goal-Achievement Emotion on Purchase Intention

This research aimed at studying the role of self-enhancing imagery and self goal-achievement emotion in the effect of characteristics perceived at advertisements using multisensory cues on purchase intention. Sports shoes advertisement was selected as an empirical research object. Questionnaire survey method was used to collect data. 'WenJuanXing' site was used to make the questionnaire in Chinese, and it was loaded on WeChat and QQ. 260 participants from different regions of China participated in online questionnaire survey. The results of testing the hypotheses by structural equation model in Amos 21.0 program are summarized as followings. The congruency between multisensory cues and self-discrepancy awareness positively evoked the self-enhancing imagery and the self goal-achievement emotion. The object relevance between the consumer and the product advertised did not induce the emotion, but evoked the self-enhancing imagery. Both of the self-enhancing imagery and the self goal-achievement emotion had positive effects on the product purchase intention. When developing advertisement, marketers should focus on multisensory cues' characteristics to enhance the self-enhancing imageries as well as to help feel the goal-achievement emotion. They should pay attention to the ways by which the multisensory cues' characteristics used to develop advertisement can be perceived to be congruent with each other by consumers.