Visual Feature Integration Research Articles

Hierarchical visual semantic guidance for enhanced relationship recognition in domain knowledge graphs

Multi-modal entity relationship recognition is a crucial foundation for constructing accurate and comprehensive domain knowledge graphs. However, due to the sparse and diverse nature of textual semantic information and the semantic differences exhibited by multi-modal data, existing methods often suffer from issues such as information loss and noise features. In this paper, the Hierarchical Visual Semantic Guidance (HVSG) Network is proposed, which utilizes images to furnish visual semantic information for text, facilitating the identification of implicit relationships between entities. Specifically, salient local instance objects are initially extracted from the global image, followed by the use of a hierarchical visual semantic construction module to establish multi-level visual semantics. The integration of multi-level visual and textual features is then achieved through a visual semantic guidance module. This work constructed a Chinese multi-modal entity relation classification dataset in the domain of unmanned vessels and conducted experiments on two datasets. Compared to state-of-the-art (SOTA) models, HVSG achieved the best performance, with F1 scores improved by 1.05% and 0.58% respectively, indicating that HVSG can offer better performance in exploring implicit relationships between textual entities.

Enhancing prediction of supraspinatus/infraspinatus tendon complex injuries through integration of deep visual features and clinical information: a multicenter two-round assessment study

ObjectiveDevelop and evaluate an ensemble clinical machine learning–deep learning (CML-DL) model integrating deep visual features and clinical data to improve the prediction of supraspinatus/infraspinatus tendon complex (SITC) injuries.MethodsPatients with suspected SITC injuries were retrospectively recruited from two hospitals, with clinical data and shoulder x-ray radiographs collected. An ensemble CML-DL model was developed for diagnosing normal or insignificant rotator cuff abnormality (NIRCA) and significant rotator cuff tear (SRCT). All patients suspected with SRCT were confirmed by arthroscopy examination. The model’s performance was evaluated using sensitivity, specificity, accuracy, and area under the curve (AUC) metrics, and a two-round assessment was conducted to authenticate its clinical applicability.ResultsA total of 974 patients were divided into three cohorts: the training cohort (n = 828), the internal validation cohort (n = 89), and the external validation cohort (n = 57). The CML-DL model, which integrates clinical and deep visual features, demonstrated superior performance compared to individual models of either type. The model’s sensitivity, specificity, accuracy, and area under curve (95% confidence interval) were 0.880, 0.812, 0.836, and 0.902 (0.858–0.947), respectively. The CML-DL model exhibited higher sensitivity and specificity compared to or on par with the physicians in all validation cohorts. Furthermore, the assistance of the ensemble CML-DL model resulted in a significant improvement in sensitivity for junior physicians in all validation cohorts, without any reduction in specificity.ConclusionsThe ensembled CML-DL model provides a solution to help physicians improve the diagnosis performance of SITC injury, especially for junior physicians with limited expertise.Critical relevance statementThe ensembled clinical machine learning–deep learning (CML-DL) model integrating deep visual features and clinical data provides a superior performance in the diagnosis of supraspinatus/infraspinatus tendon complex (SITC) injuries, particularly for junior physicians with limited expertise.Key points1. Integrating clinical and deep visual features improves diagnosing SITC injuries.2. Ensemble CML-DL model validated for clinical use in two-round assessment.3. Ensemble model boosts sensitivity in SITC injury diagnosis for junior physicians.Graphical

Visual and cognitive processes contribute to age-related improvements in visual selective attention.

Children (N = 103, 4-9 years, 59 females, 84% White, c. 2019) completed visual processing, visual feature integration (color, luminance, motion), and visual search tasks. Contrast sensitivity and feature search improved with age similarly for luminance and color-defined targets. Incidental feature integration improved more with age for color-motion than luminance-motion. Individual differences in feature search ( = .11) and incidental feature integration ( = .06) mediated age-related changes in conjunction visual search, an index of visual selective attention. These findings suggest that visual selective attention is best conceptualized as a series of developmental trajectories, within an individual, that vary by an object's defining features. These data have implications for design of educational and interventional strategies intended to maximize attention for learning and memory.

Processing load, and not stimulus evidence, determines the duration of unconscious visual feature integration

Integration across space and time is essential for the analysis of motion, low contrast, and many more stimuli. A crucial question is what determines the duration of integration. Based on classical models of decision-making, one might expect that integration terminates as soon as sufficient evidence about a stimulus is accumulated and a threshold is crossed. However, there is very little research on this question as most experimental paradigms cannot monitor processing following stimulus presentation. In particular, it is difficult to determine when processing terminates. Here, using the sequential metacontrast paradigm (SQM), in which information is mandatorily integrated along motion trajectories, we show that the processing load determines the extent of integration but that evidence accumulation does not. Further, the extent of integration is determined by absolute time instead of the number of elements presented. These results have important implications for understanding the time course and mechanisms of temporal integration.

PVL-Cartographer: Panoramic Vision-Aided LiDAR Cartographer-Based SLAM for Maverick Mobile Mapping System

The Mobile Mapping System (MMS) plays a crucial role in generating accurate 3D maps for a wide range of applications. However, traditional MMS that utilizes tilted LiDAR (light detection and ranging) faces limitations in capturing comprehensive environmental data. We propose the “PVL-Cartographer” SLAM (Simultaneous Localization And Mapping) approach for MMS to address these limitations. This proposed system incorporates multiple sensors to yield dependable and precise mapping and localization. It consists of two subsystems: early fusion and intermediate fusion. In early fusion, range maps are created from LiDAR points within a panoramic image space, simplifying the integration of visual features. The SLAM system accommodates both visual features with and without augmented ranges. In intermediate fusion, camera and LiDAR nodes are merged using a pose graph, with constraints between nodes derived from IMU (Inertial Measurement Unit) data. Comprehensive testing in challenging outdoor settings demonstrates that the proposed SLAM system can generate trustworthy outcomes even in feature-scarce environments. Ultimately, our suggested PVL-Cartographer system effectively and accurately addresses the MMS localization and mapping challenge.

Multiple Visual Feature Integration Based Automatic Aesthetics Evaluation of Robotic Dance Motions

Imitation of human behaviors is one of the effective ways to develop artificial intelligence. Human dancers, standing in front of a mirror, always achieve autonomous aesthetics evaluation on their own dance motions, which are observed from the mirror. Meanwhile, in the visual aesthetics cognition of human brains, space and shape are two important visual elements perceived from motions. Inspired by the above facts, this paper proposes a novel mechanism of automatic aesthetics evaluation of robotic dance motions based on multiple visual feature integration. In the mechanism, a video of robotic dance motion is firstly converted into several kinds of motion history images, and then a spatial feature (ripple space coding) and shape features (Zernike moment and curvature-based Fourier descriptors) are extracted from the optimized motion history images. Based on feature integration, a homogeneous ensemble classifier, which uses three different random forests, is deployed to build a machine aesthetics model, aiming to make the machine possess human aesthetic ability. The feasibility of the proposed mechanism has been verified by simulation experiments, and the experimental results show that our ensemble classifier can achieve a high correct ratio of aesthetics evaluation of 75%. The performance of our mechanism is superior to those of the existing approaches.

Microstructural Anomalies Evaluated by Neurite Orientation Dispersion and Density Imaging Are Related to Deficits in Facial Emotional Recognition via Perceptual-Binding Difficulties in Autism Spectrum Disorder.

The integration of visual features is important for recognizing objects as a coherent whole, a key domain of difficulty in autism spectrum disorder (ASD). We tested the hypothesis that ASD patients exhibit difficulties in facial emotional recognition via perceptual binding difficulties due to weak coherence. We assessed 18 ASD and 27 typically developing individuals for their ability to identify emotional expressions from faces in pictures moving behind a narrow vertical and horizontal slit. In this task, only a single local piece of facial information was provided at any one time through the slit. Using a voxel-based analysis of neurite-orientation dispersion and density imaging (NODDI), we examined the relationship between NODDI index values at each voxel and the behavioral performance of ASD patients in the slit-viewing paradigm. ASD patients demonstrated impaired recognition of facial emotional expression only in horizontal slit-viewing. This deficit was associated with deficits in communication ability. Voxel-based analysis revealed significant negative correlations between behavioral deficits in horizontal slit-viewing and NODDI index values in clusters including the ventral occipital complex region, superior temporal/parietal association areas, and forceps major of the corpus callosum. Our results indicated deficits for the first time in perceptual integration of facial expression across hemispheres in ASD patients due to microstructural disturbances in the corpus callosum and areas related to viewing of the human face. This may underscore the difficulties faced by ASD patients in understanding the emotions of other people, contributing to impairments in communication ability in ASD patients. Autism Res 2020, 13: 729-740. © 2020 International Society for Autism Research, Wiley Periodicals, Inc. LAY SUMMARY: We assessed ASD and typically developing individuals for their ability to identify emotional expressions from faces in pictures moving behind a narrow vertical and horizontal slit. ASD patients demonstrated impaired recognition of facial emotional expression only in horizontal slit-viewing. Voxel-based analysis revealed significant negative correlations between behavioral deficits and NODDI index values in clusters including the corpus callosum. Our results indicated deficits in perceptual integration of facial expression across hemispheres in ASD patients potentially resulting from microstructural disturbances.

Complex environment perception and positioning based visual information retrieval

The biological vision model is devoted to provide a novel technology approach by merging new cognitive visual features with inspired nerve cells cognitive intelligence cortex and try to relate with real worlds object recognition. To perceive an arbitrary natural scene from complex environment perception and sensing in robotic mobility and manipulation on unstructured random natural scene understanding is a challenging problem in the visual image processing. This paper has considered neural network (NN) which is nothing but the grid of “neurons like” nodes. Based on the NN technique,the authors have proposed a new scheme for the scene understanding and recognition. In addition, the significant intellectual visual features are also incorporated for scene expression; those are very crucial and provide cognitive intelligence to robot vision. Due to the dynamic nature of artificial neural network intelligence, we have adapted the attributes of the Gabor filter and Laplacian of Gaussian filter; those play the significant role in the robot visual perception. Through the study of perceptual ability of the natural scene image from complex environment for robot vision is enhanced with the integration of cognitive visual features and the scene expression.

Image Segmentation Based on Weakly Supervised MKL on Mixed Visual Features

Weakly supervised learning has outstanding ability to solve classification tasks, and multiformity middle-level visual features provide more abundant discriminant information for meaningful regions. In this paper, we study the integration of the middle-level visual features including homogeneity of superpixels, region objectness and texture map for segmentation. Then, three kernels are exploited to map visual features to high-dimensional space. A few labeled pixels are chosen for training support vector machines(SVMs) in a single image with hybrid kernels. On this basis, the remaining pixels are labeled with classified results of SVMs and refined the segmentation results by merging pre-segments of mean-shift. We perform sufficient experiments on Berkeley datasets and compared them with several excellent segmentation algorithms. Extensive experimental results of the proposed method show superior segmentation performance and expanded tests on PASCAL VOC datasets further validate the effectiveness of the algorithm.

What underlies visual selective attention development? Evidence that age-related improvements in visual feature integration influence visual selective attention performance

Visual selective attention (VSA) improves across childhood. Conjunction search tasks require integrating multiple visual features in order to find a target among distractors and are often used to measure VSA. Motivated by the visual system’s architecture and developmental changes in neural connectivity, we predicted that feature integration across separate visual pathways (e.g., color and motion) should develop later than feature integration within the same visual pathways (e.g., luminance and motion). A total of 89 4- to 10-year-old children completed a visual search task that manipulated whether feature integration was between separate parallel visual pathways or within the same visual pathway. We first examined whether color–motion integration was associated with a performance cost relative to luminance–motion integration across childhood. We found that color–motion integration was worse than luminance–motion integration in early childhood but that this difference decreased with age. We also examined whether luminance–motion and color–motion visual search performance developed differently across childhood. Reaction time (RT) visual search slopes for the luminance–motion condition were both stable across childhood and steeper overall than those for the color–motion condition. In contrast, RT search slopes for the color–motion condition became steeperincrease across childhood. Finally, we found that age-related improvements in color–motion integration, relative to luminance–motion integration, were associated with longer color–motion search rates across childhood. These data suggest that age-related improvements in color–motion feature integration may increase competition between color–motion targets and distractors, thereby increasing the amount of time needed to process distractors as nontargets during the selection process.

An image-text consistency driven multimodal sentiment analysis approach for social media

Social media users are increasingly using both images and text to express their opinions and share their experiences, instead of only using text in the conventional social media. Consequently, the conventional text-based sentiment analysis has evolved into more complicated studies of multimodal sentiment analysis. To tackle the challenge of how to effectively exploit the information from both visual content and textual content from image-text posts, this paper proposes a new image-text consistency driven multimodal sentiment analysis approach. The proposed approach explores the correlation between the image and the text, followed by a multimodal adaptive sentiment analysis method. To be more specific, the mid-level visual features extracted by the conventional SentiBank approach are used to represent visual concepts, with the integration of other features, including textual, visual and social features, to develop a machine learning sentiment analysis approach. Extensive experiments are conducted to demonstrate the superior performance of the proposed approach.

Multisensory feature integration in (and out) of the focus of spatial attention.

Anne Treisman transformed the way in which we think about visual feature integration. However, that does not mean that she was necessarily right, nor that she looked much beyond vision when considering how features might be bound together into perceptual objects. While such a narrow focus undoubtedly makes sense, given the complexity of human multisensory information processing, it is nevertheless somewhat surprising to find that Treisman herself never extended her feature integration theory outside of the visual modality. After all, she first cut her 'attentional teeth' thinking about problems of auditory and audiovisual selective attention. In this article, we review the literature concerning feature integration beyond the visual modality, concentrating, in particular, on the integration of features from different sensory modalities. We highlight a number of the challenges, as far as any straightforward attempt to extend feature integration to the non-visual (i.e. auditory and tactile) and cross-modal (or multisensory) cases, is concerned. These challenges include the problem of how basic features should be defined, the question of whether it even makes sense to talk of objects of perception in the auditory and olfactory modalities, the possibility of integration outside of the focus of spatial attention, and the integration of features from different sensory modalities in the control of action. Nevertheless, despite such limitations, Treisman's feature integration theory still stands as the standard approach against which alternatives are assessed, be it in the visual case or, increasingly, beyond.

Visual Feature Integration of Three Attributes in Stimulus-Response Mapping Is Distinct From That of Two.

In the human visual system, different attributes of an object are processed separately and are thought to be then temporarily bound by attention into an integrated representation to produce a specific response. However, if such representations existed in the brain for arbitrary multi-attribute objects, a combinatorial explosion problem would be unavoidable. Here, we show that attention may bind features of different attributes only in pairs and that bound feature pairs, rather than integrated object representations, are associated with responses for unfamiliar objects. We found that in a mapping task from three-attribute stimuli to responses, presenting three attributes in pairs (two attributes in each window) did not significantly complicate feature integration and response selection when the stimuli were not very familiar. We also found that repeated presentation of the same triple conjunctions significantly improved performance on the stimulus-response task when the correct responses were determined by the combination of three attributes, but this familiarity effect was not observed when the response could be determined by two attributes. These findings indicate that integration of three or more attributes is a distinct process from that of two, requiring long-term learning or some serial process. This suggests that integrated object representations are not formed or are formed only for a limited number of very familiar objects, which resolves the computational difficulty of the binding problem.

Sparsely encoded distinctive visual features for object recognition

An image feature descriptor named “sparsely encoded distinctive visual features (SEDVF)” is proposed for object recognition. SEDVF is built with the integration of local and global visual features. Visual information on edge orientation, magnitude, color, and pixel intensity is sparsely encoded by a bit plane slicing technique. Distinctive features are obtained using winner-takes-all principle. Edge gradient multiorientation detector method (EGMOD) is proposed to obtain the gradient orientations. EGMOD extracts four-directional (horizontal, vertical, and both diagonal) edge information with the proposed multioriented Scharr operator in YIQ color space. Magnitude features are extracted by incorporating chromatic information along horizontal and vertical directions in the RGB color space. SEDVF can be used as a color feature descriptor that has good discriminative power of visual features. The proposed descriptor is extensively tested for performance evaluation using K-nearest neighbor classifier on three standard datasets, including Columbia object image library, Amsterdam library of object images, and PVOC 2007, respectively. Experimental results reveal outperformance of SEDVF as compared to the state-of-the-art object recognition methods.

MULTIMEDIA COMPONENT IN THE EDUCATIONAL TECHNOLOGY GAME-TRAINING

Multimedia technologies play a significant role in improving the quality of educational process in conditions of profile training of senior pupils. Under the media we understand didactic multi environment, which offers educational information in aesthetically designed interactive form with the integration of sound and visual features that will positively affect the efficient flow of perceptual-mnemonic processes and contributes to the formation and development of competence of the individual. The purpose of this article is to identify the didactic features of the multimedia component in the structure of pedagogical technologies of education and the consideration of specific practical examples of the application of multimedia visualizations in the composition of the educational technology game-training. The study is based on techniques such as: disclosure of the nature of pedagogical patterns to the nomination and the formulation of General principles, methodological approaches, methods and didactic conditions for the implementation of multimedia educational technologies education, which will contribute to the tasks of special education. Main didactic elements multimedia visualization of the educational information is visual-sound image, or multimedia illustration. In this context, the main objective of creating a visual and sound image is to assist the student during the process of perception and of mental creation in your own mind a clear mental image of a phenomenon or process that is studied, which in turn facilitates the organization of independent work of pupils during the educational activities. The availability of multimedia software allows also to compensate for the failure of laboratory facilities, due to the possibility of modelling the processes and phenomena of nature, which is important for professional self-determination of senior pupils in the conditions of profile training. This requires a choice of the most effective elements of computer technology for solving specific tasks in a specific lesson. Multimedia visualization effectively enables the implementation of diverse methods, techniques and technician training, which can be used in the classroom for the invariant of subjects and course selection and courses.

Measuring Integration Processes in Visual Symmetry with Frequency-Tagged EEG

Symmetry is a highly salient feature of the natural world which requires integration of visual features over space. The aim of the current work is to isolate dynamic neural correlates of symmetry-specific integration processes. We measured steady-state visual evoked potentials (SSVEP) as participants viewed symmetric patterns comprised of distinct spatial regions presented at two different frequencies (f1 and f2). We measured intermodulation components, shown to reflect non-linear processing at the neural level, indicating integration of spatially separated parts of the pattern. We generated a wallpaper pattern containing two reflection symmetry axes by tiling the plane with a two-fold reflection symmetric unit-pattern and split each unit-pattern diagonally into separate parts which could be presented at different frequencies. We compared SSVEPs measured for wallpapers and control patterns for which both images were equal in terms of translation and rotation symmetry but reflection symmetry could only emerge for the wallpaper pattern through integration of the image-pairs. We found that low-frequency intermodulation components differed between the wallpaper and control stimuli, indicating the presence of integration mechanisms specific to reflection symmetry. These results showed that spatial integration specific to symmetry perception can be isolated through a combination of stimulus design and the frequency tagging approach.

Preliminary Evidence for the Psychophysiological Effects of a Technological Atmosphere in E-Commerce

As information and communication technologies (ICTs) become more advanced, consumers are able to experience retailing activities such as searching for products and services in online retail shops and for Internet-exclusive branded contents. Specifically, fashion retailers are facing the need to develop more novel experiential design than one another to maximize customers’ experience in Internet websites and secure sustainable competency. Confirming methods of organic integration of experiential and visual features of both online and mobile channels is an important aspect of the study of extended consumers’ interfaces of retail channels. Mehrabian and Russell’s stimulusorganism- response (S-O-R) paradigm and Sugiyama and Andree’s attention, interest, search, action, and share (AISAS) model were used for this research. Specifically, the present study considered the effect of e-commerce website features on consumers’ emotional reactions (pleasure and arousal) and the consequent impact on online consumer behaviors (search, action, and share). Hence, plus the self-reported survey methods, each subject’s psychophysiological indicators (i.e., pleasure and arousal) were measured to obtain more objective and reliable data and to redeem the results of the self-reported survey. Findings revealed the implications of the e-commerce website feature by comprehending the S-O-R paradigm and AISAS model and extending the understanding of the role of variables associated with comprehended frameworks based on psychophysiological data.

A low‐complexity virtual reality modeling method based on ontology grid sparsity

AbstractThis paper focuses on an innovative low‐complexity modeling method for virtual reality. Voxels are the basic unit of pixels in 3D (three dimensions), and when stored and processed in a computer, they contain more data redundancy than the demand for actual human vision presentations. This excess is known as voxel redundancy. The reason for the redundancy is that infinite‐dimensional expansion and heterogeneous voxel features have merged in modeling methods. The major voxel compressions and human factors correlations are important to consider. Therefore, we propose a novel low‐complexity virtual reality based on ontology grid sparsity. The redundancy performance of the existing virtual reality modeling methods is first analyzed. The high‐dimension modeling method is designed to describe the base elements that are needed for humans to distinguish what is actually perceived. Based on the ontology theory, grid sparsity is exploited to construct the correlation model. We set up the relevant degree and sensitivity level calculations to classify the correlations between ontology features and other features. The features with weak sensitivity are ignored or merged, which successfully reduces the dimensions and redundancy that are needed to represent virtual items. Visual feature integration combines human factors into a voxel model. Finally, we build the virtual laboratory as a test case in the Unity 3D and 3DMax environment. The tests demonstrate that the new modeling method offers 21.2% and 34.7% of presentation delay and redundancy improvement, respectively.

Chinese-character crowding-I. Effects of structural similarity.

Crowding impedes the identification of flanked objects in peripheral vision. Prior studies have shown crowding strength decreases with target-flanker similarity. Research on crowding in Chinese-character recognition has been scarce in the literature. We aimed to fill the research gap by examining the effects of structural similarity on Chinese-character crowding. Regularity in within-character configuration, i.e., orthographic legality, of flankers was manipulated in Experiment 1. Target-flanker similarity in orthographic legality did not affect crowding strength, measured as contrast threshold elevation. Crowding weakened only when the strokes in the flankers were scrambled. Contour integrity of flankers was manipulated by randomly perturbing the phase spectra of the stimulus images in Experiments 2a and 2b. Crowding by perturbed-phase flankers remained robust but was weaker compared with intact-phase flankers. Target-flanker similarity in contour integrity modulated crowding strength. Our findings were consistent with the postulation that faulty integration of low-level visual features contributed to crowding of Chinese characters. Studies on Chinese-character recognition and crowding can provide important insights into how the visual system processes complex daily objects.

Measuring Integration Processes in Visual Symmetry with Frequency-tagged EEG

Symmetry is a highly salient feature of the natural world which requires integration of visual features over space. The aim of the current work is to isolate dynamic neural correlates of symmetry-specific integration processes. We measured steady-state visual evoked potentials (SSVEP) as participants viewed symmetric patterns comprised of distinct spatial regions presented at two different frequencies (f1 and f2). We measured intermodulation components, shown to reflect non-linear processing at the neural level, indicating integration of spatially separated parts of the pattern. We generated a wallpaper pattern containing two reflection symmetry axes by tiling the plane with a two-fold reflection symmetric unit-pattern and split each unit-pattern diagonally into separate parts which could be presented at different frequencies. We compared SSVEPs measured for wallpapers and control patterns for which both images were equal in terms of translation and rotation symmetry but reflection symmetry could only emerge for the wallpaper pattern through integration of the image-pairs. We found that low-frequency intermodulation components differed between the wallpaper and control stimuli, indicating the presence of integration mechanisms specific to reflection symmetry. These results showed that spatial integration specific to symmetry perception can be isolated through a combination of stimulus design and the frequency tagging approach.