High-level Descriptors Research Articles

Emotion recognition for human–computer interaction using high-level descriptors

Recent research has focused extensively on employing Deep Learning (DL) techniques, particularly Convolutional Neural Networks (CNN), for Speech Emotion Recognition (SER). This study addresses the burgeoning interest in leveraging DL for SER, specifically focusing on Punjabi language speakers. The paper presents a novel approach to constructing and preprocessing a labeled speech corpus using diverse social media sources. By utilizing spectrograms as the primary feature representation, the proposed algorithm effectively learns discriminative patterns for emotion recognition. The method is evaluated on a custom dataset derived from various Punjabi media sources, including films and web series. Results demonstrate that the proposed approach achieves an accuracy of 69%, surpassing traditional methods like decision trees, Naïve Bayes, and random forests, which achieved accuracies of 49%, 52%, and 61% respectively. Thus, the proposed method improves accuracy in recognizing emotions from Punjabi speech signals.

VisionDeep-AI: Deep learning-based retinal blood vessels segmentation and multi-class classification framework for eye diagnosis

Eye problems can lead to vision loss and have a significant impact on daily life, underscoring the critical importance of early diagnosis and treatment to prevent further damage and complications. A comprehensive segmentation-classification framework – VisionDeep-AI is developed in the proposed work for retinal vessel segmentation and multi-class classification on given fundus images. A weighted bi-directional feature pyramid network and U-Net backbone architecture-based customized deep-learning model is built to segment blood vessels which enhances feature extraction and multi-scale feature fusion. The architectural design comprises an end-to-end encoder-decoder network featuring six-depth layers with varying resolutions, enabling the extraction of high-level descriptors and lower-level, fine-grained characteristics. Furthermore, a multi-modal deep feature fusion architecture is developed for the multi-class classification of fundus images in four different categories by integrating features from segmented vessel images and raw fundus images, thereby accommodating more diversified information. A thorough analysis of VisionDeep-AI has been done on a comprehensive colour fundus image dataset. To ensure reliable results, data augmentation was done to prevent the over-fitting of the model and to enhance its generalization capability. The segmentation model performed exceptionally well and achieved a high accuracy of 97.73% and a dice coefficient of 89.90% in blood vessel segmentation from fundus images. Furthermore, multi-modal deep feature-fused classification architecture achieved a test accuracy of 81.50%, with a specificity of 93.83%. These findings showcase the robustness, generalizability and efficiency of the proposed VisionDeep-AI framework for fundus image segmentation and retinal disease classification, contributing towards advanced medical technologies and diagnosis precision.

Semantic Image Retrieval Analysis Based on Deep Learning and Singular Value Decomposition

The exponential growth in the total quantity of digital images has necessitated the development of systems that are capable of retrieving these images. Content-based image retrieval is a technique used to get images from a database. The user provides a query image, and the system retrieves those photos from the database that are most similar to the query image. The image retrieval problem pertains to the task of locating digital photographs inside extensive datasets. Image retrieval researchers are transitioning from the use of keywords to the utilization of low-level characteristics and semantic features. The push for semantic features arises from the issue of subjective and time-consuming keywords, as well as the limitation of low-level characteristics in capturing high-level concepts that users have in mind. The main goal of this study is to examine how convolutional neural networks can be used to acquire advanced visual features. These high-level feature descriptors have the potential to be the most effective compared to the handcrafted feature descriptors in terms of image representation, which would result in improved image retrieval performance. The (CBIR-VGGSVD) model is an ideal solution for content-based image retrieval that is based on the VGG-16 algorithm and uses the Singular Value Decomposition (SVD) technique. The suggested model incorporates the VGG-16 model for the purpose of extracting features from both the query images and the images kept in the database. Afterwards, the dimensionality of the features retrieved from the VGG-16 model is reduced using SVD. Then, we compare the query photographs to the dataset images using the cosine metric to see how similar they are. When all is said and done, images that share a high degree of similarity will be successfully extracted from the dataset. A validation of the retrieval performance of the CBIR-VGGSVD model is performed using the Corel-1K dataset. When the VGG-16 standard model is the sole one used, the implementation will produce an average precision of 0.864. On the other hand, when the CBIR-VGGSVD model is utilized, this average precision is revealed to be (0.948). The findings of the retrieval ensured that the CBIR-VGGSVD model provided an improvement in performance on the test pictures that were utilized, surpassing the performance of the most recent approaches.

An Image Retrieval Pipeline in a Medical Data Integration Center.

Medical images need annotations with high-level semantic descriptors, so that domain experts can search for the desired dataset among an enormous volume of visual media within a Medical Data Integration Center. This article introduces a processing pipeline for storing and annotating DICOM and PNG imaging data by applying Elasticsearch, S3 and Deep Learning technologies. The proposed method processes both DICOM and PNG images to generate annotations. These image annotations are indexed in Elasticsearch with the corresponding raw data paths, where they can be retrieved and analyzed.

Blind-Spot Collision Detection System for Commercial Vehicles Using Multi Deep CNN Architecture.

Buses and heavy vehicles have more blind spots compared to cars and other road vehicles due to their large sizes. Therefore, accidents caused by these heavy vehicles are more fatal and result in severe injuries to other road users. These possible blind-spot collisions can be identified early using vision-based object detection approaches. Yet, the existing state-of-the-art vision-based object detection models rely heavily on a single feature descriptor for making decisions. In this research, the design of two convolutional neural networks (CNNs) based on high-level feature descriptors and their integration with faster R-CNN is proposed to detect blind-spot collisions for heavy vehicles. Moreover, a fusion approach is proposed to integrate two pre-trained networks (i.e., Resnet 50 and Resnet 101) for extracting high level features for blind-spot vehicle detection. The fusion of features significantly improves the performance of faster R-CNN and outperformed the existing state-of-the-art methods. Both approaches are validated on a self-recorded blind-spot vehicle detection dataset for buses and an online LISA dataset for vehicle detection. For both proposed approaches, a false detection rate (FDR) of 3.05% and 3.49% are obtained for the self recorded dataset, making these approaches suitable for real time applications.

A Computational Probe into the Behavioral and Neural Markers of Atypical Facial Emotion Processing in Autism.

Despite ample behavioral evidence of atypical facial emotion processing in individuals with autism spectrum disorder (ASD), the neural underpinnings of such behavioral heterogeneities remain unclear. Here, I have used brain-tissue mapped artificial neural network (ANN) models of primate vision to probe candidate neural and behavior markers of atypical facial emotion recognition in ASD at an image-by-image level. Interestingly, the image-level behavioral patterns of the ANNs better matched the neurotypical subjects 'behavior than those measured in ASD. This behavioral mismatch was most remarkable when the ANN behavior was decoded from units that correspond to the primate inferior temporal (IT) cortex. ANN-IT responses also explained a significant fraction of the image-level behavioral predictivity associated with neural activity in the human amygdala (from epileptic patients without ASD), strongly suggesting that the previously reported facial emotion intensity encodes in the human amygdala could be primarily driven by projections from the IT cortex. In sum, these results identify primate IT activity as a candidate neural marker and demonstrate how ANN models of vision can be used to generate neural circuit-level hypotheses and guide future human and nonhuman primate studies in autism.SIGNIFICANCE STATEMENT Moving beyond standard parametric approaches that predict behavior with high-level categorical descriptors of a stimulus (e.g., level of happiness/fear in a face image), in this study, I demonstrate how an image-level probe, using current deep-learning-based ANN models, allows identification of more diagnostic stimuli for autism spectrum disorder enabling the design of more powerful experiments. This study predicts that IT cortex activity is a key candidate neural marker of atypical facial emotion processing in people with ASD. Importantly, the results strongly suggest that ASD-related atypical facial emotion intensity encodes in the human amygdala could be primarily driven by projections from the IT cortex.

Research on Chorus Emotion Recognition and Intelligent Medical Application Based on Health Big Data.

In chorus activities, the conductor leads chorus members to recreate music works. If you want to interpret music works perfectly with sound, emotion and emotional expression are particularly important. In this paper, a cloud HBD (health big data) integration system based on ensemble learning is designed to realize the high-efficiency and high-precision integration of HBD. An emotional speech database containing three emotions such as pleasure, calmness, and boredom is established, and the corpus problems such as emotional feature analysis and extraction needed for chorus emotion recognition research are solved. It also studies the classification and decision-making in emotional changes, and a DBN (deep belief network) chorus emotion recognition algorithm based on multiple emotional features is proposed. Feature DBN (Deep Belief Network) Chorus Emotion Recognition Algorithm This paper extracts various robust low-level features according to different features' ability to describe emotions and then feeds them into the DBN network to extract high-level feature descriptors. Then, the classification results of ELM (extreme learning machine) are voted and fused with the idea of ensemble learning, and the effectiveness of the algorithm is proved on three public datasets.

Plausible blockers of Spike RBD in SARS-CoV2-molecular design and underlying interaction dynamics from high-level structural descriptors.

COVID-19 is characterized by an unprecedented abrupt increase in the viral transmission rate (SARS-CoV-2) relative to its pandemic evolutionary ancestor, SARS-CoV (2003). The complex molecular cascade of events related to the viral pathogenicity is triggered by the Spike protein upon interacting with the ACE2 receptor on human lung cells through its receptor binding domain (RBDSpike). One potential therapeutic strategy to combat COVID-19 could thus be limiting the infection by blocking this key interaction. In this current study, we adopt a protein design approach to predict and propose non-virulent structural mimics of the RBDSpike which can potentially serve as its competitive inhibitors in binding to ACE2. The RBDSpike is an independently foldable protein domain, resilient to conformational changes upon mutations and therefore an attractive target for strategic re-design. Interestingly, in spite of displaying an optimal shape fit between their interacting surfaces (attributed to a consequently high mutual affinity), the RBDSpike–ACE2 interaction appears to have a quasi-stable character due to a poor electrostatic match at their interface. Structural analyses of homologous protein complexes reveal that the ACE2 binding site of RBDSpike has an unusually high degree of solvent-exposed hydrophobic residues, attributed to key evolutionary changes, making it inherently “reaction-prone.” The designed mimics aimed to block the viral entry by occupying the available binding sites on ACE2, are tested to have signatures of stable high-affinity binding with ACE2 (cross-validated by appropriate free energy estimates), overriding the native quasi-stable feature. The results show the apt of directly adapting natural examples in rational protein design, wherein, homology-based threading coupled with strategic “hydrophobic ↔ polar” mutations serve as a potential breakthrough.Graphical Supplementary InformationThe online version contains supplementary material available at 10.1007/s00894-021-04779-0.

Making use of our data

Engineers are acquiring data at an ever-increasing rate: data from computational design studies; measurements data from manufacturing processes, development tests, and products in service; contemporary data and legacy data. In this paper, two recommendations are made to allow engineers to make better use of these expanding databases. First, we should build on the hierarchical nature of our data; we can navigate and filter the database using high level descriptors such as design specifications and performance metrics, and then request comparative plots of detailed data such as line, contour and surface plots. Second, we can speed up the rate at which we learn from data by making the visualisations dynamic; in so doing, we enable virtual experiments to be performed that highlight connections between input parameters, output metrics and physical mechanisms. The embodiment of these two principles in the open source project, dbslice, is described. Three example applications (an aerodynamic design study for a compressor stator; the application of machine learning to aid navigation of large databases; and visualisation of a database of snapshots from an unsteady simulation) are presented. In each case, the hierarchical data and dynamic visualisations allow the user to explore the database and experience the connections and patterns within it. By Making Use of Our Data to interactively navigate existing and new design spaces in this way, engineers can accelerate their response to the challenges of future products.

Automatic Extraction and Labelling of Memorial Objects From 3D Point Clouds

This research addresses the problem of automatic extraction of memorial objects from cultural heritage sites represented as scenes of 3D point clouds. Point clouds provide a fine spatial resolution and accurate proxy of the real world. However, how to use them directly is not always obvious. This is especially true for applications where extensive training data or computational resources are not available. In this paper, we present a methodology for automatic segmentation and labelling of cultural heritage objects from 3D point cloud scenes. The proposed methodology is based on machine learning techniques and, in particular, makes use of the concept of transfer learning. Memorial objects are segmented from the scene based on their geometric shape characteristic through a conditional multi-scale partitioning scheme. Then, high-level latent feature descriptors are extracted by a convolutional neural network pre-trained on different 3D object models from a standard dataset (e.g., ModelNet). Based on these descriptors, a classification model (multilayer perceptron) is trained and applied to obtain semantic labels. Experiments demonstrated that the proposed methodology is effective for the extraction and labelling of grave marker objects from cultural heritage sites.

High-Level Descriptors for Fall Event Detection Supported by a Multi-Stream Network

The need for assertive video classification has been increasingly in demand. Especially for detecting endangering situations, it is crucial to have a quick response to avoid triggering more serious problems. During this work, we target video classification concerning falls. Our study focuses on the use of high-level descriptors able to correctly characterize the event. These descriptor results will serve as inputs to a multi-stream architecture of VGG-16 networks. Therefore, our proposal is based on the analysis of the best combination of high-level extracted features for the binary classification of videos. This approach was tested on three known datasets, and has proven to yield similar results as other more consuming methods found in the literature.

Relating molecular descriptors to frontier orbital energy levels, singlet and triplet excited states of fused tricyclics using machine learning

Fused tricyclic organic compounds are an important class of organic electronic materials. In designing molecules for organic electronics, knowing what chemical structure that be used to tune the molecular property is one of the keys that can help to improve the material performance. In this research, we applied machine learning and data analytic approaches in addressing this problem. The energy states (Lowest Unoccupied Molecular Orbital (HOMO), Highest Occupied Molecular Orbitals (LUMO), singlet (Es) and triplet (ET) energy) of more than 10 thousand fused tricyclics are calculated. Corresponding descriptors are also generated. We find that the Coulomb matrix is a poorer descriptor than high-level descriptors in a multilayer perceptron neural network. Correlations as high as 0.95 is obtained using a multilayer perceptron neural network with Mean Absolute Error as low as 0.08 eV. The descriptors that are important in tuning the energy levels are revealed using the Random Forest algorithm. Correlations of such descriptors are also plotted. We found that the higher the number of tertiary amines, the deeper are the HOMO and LUMO levels. The presence of NN in the aromatic rings can be used to tune the ES. However, there is no single dominant descriptor that can be correlated with the ET. A collection of descriptors is found to give a far better correlation with ET. This research demonstrated that machine learning and data analytics in guiding how certain chemical substructures correlate with the molecule energy states.

Hierarchical support vector machine for facial micro-expression recognition

The sample category distribution of spontaneous facial micro-expression datasets is unbalanced, due to the experimental environment, collection equipment, and individualization of subjects, which brings great challenges to micro-expression recognition. Therefore, this paper introduces a micro-expression recognition model based on the Hierarchical Support Vector Machine (H-SVM) to reduce the interference of sample category distribution imbalance. First, we calculated the position of the apex frame in the micro-expression image sequence. To keep micro-expression frames balanced, we sparsely sample the images sequence according to the apex frame. Then, the Low-level Descriptors of the region of interest of the micro-expression image sequence and the High-level Descriptors of apex frame are extracted. Finally, the H-SVM model is used to classify the fusion features of different levels. The experimental results on SMIC, CAMSE2, SAMM, and their composite datasets show that our method can achieve superior performance in micro-expression recognition.

In the frame, road map for Australian sport on an uncertain journey through COVID-19

In the frame, road map for Australian sport on an uncertain journey through COVID-19

Moving vehicle detection in aerial infrared image sequences via fast image registration and improved YOLOv3 network

ABSTRACTOwing to its powerful adaptability and robustness to the weak light, the infrared camera equipped on the unmanned aerial vehicle is more and more applied for aerial photography. Nowadays, how to make full use of the aerial infrared image sequences for the moving vehicle detection has attracted widespread attention. However, due to the low-resolution, low contrast, and few texture features of the infrared image, it is extremely difficult to detect the moving vehicles. In our work, an effective and efficient moving vehicle detection approach in the aerial infrared image sequences is proposed via fast image registration and You Only Look Once Version 3 (YOLOv3) network. First, to compensate for the motion of the aerial infrared camera, a fast infrared image registration method is put forward. To improve the accuracy and efficiency of image registration, we construct a multi-screening based mechanism for screening out the incorrect and redundant feature points. For feature description, the low-level and high-level descriptors are combined to further improve the registration accuracy. Then, the 2-frame difference and image masking are introduced to acquire the frame mask images, where only the region-of-interest is reserved, and the remaining regions are masked. Next, we construct a new structure of an improved YOLOv3 network with only 23 layers. Due to the insufficiency of the infrared vehicle samples, transfer learning is introduced to train the improved YOLOv3 network. Finally, the proposed approach is evaluated on the Defence Advanced Research Projects Agency (DARPA) Video verification of Identify (VIVID) and Northwest Polytechnical University (NPU) data sets. Experiments and comprehensive analyses show that the proposed approach can achieve satisfactory and competitive moving vehicle detection results.

Classification of Barking Context of Domestic Dog using High-Level Descriptors

Classification of Barking Context of Domestic Dog using High-Level Descriptors

An Insight Framework for Content based Medical Image Retrieval using Deep Convolutional Neural Network

Content Based Image Retrieval system is essentially proven techniques for retrieving the DICOM based images. DICOM format is use to acquire, store and shared medical images. Typically, retrieval of image under query upon the medical image databases is performed by fusing the low-level and high-level descriptors along with DCNN features. In this paper, DICOM image meta data are extracted by using dicom function and create the local database for reserve the information. To accomplish the initial search and retrieve the images by using extraction of Semantic information. The DICOM tags are extracted from the DICOM images and use DCNN feature to build a feature vector database. Subsequently prediction is done based on the by leveraging the convolution layers based on the meta data along with DCNN image features. This paper attempts to implement pre filter method to all DICOM images which further decrease the searching no of images, searching time and ultimately gives fast processing. DCNN based prediction model was constructed and finest results are accomplished. Average accuracy of precision and recall up to0.80 and 0.87 respectively is achieved based on precision value which would be suitable for high quality image retrieval based on semantic information confined in the image.

CNN-BASED FEATURE-LEVEL FUSION OF VERY HIGH RESOLUTION AERIAL IMAGERY AND LIDAR DATA

Abstract. Land-cover classification of Remote Sensing (RS) data in urban area has always been a challenging task due to the complicated relations between different objects. Recently, fusion of aerial imagery and light detection and ranging (LiDAR) data has obtained a great attention in RS communities. Meanwhile, convolutional neural network (CNN) has proven its power in extracting high-level (deep) descriptors to improve RS data classification. In this paper, a CNN-based feature-level framework is proposed to integrate LiDAR data and aerial imagery for object classification in urban area. In our method, after generating low-level descriptors and fusing them in a feature-level fusion by layer-stacking, the proposed framework employs a novel CNN to extract the spectral-spatial features for classification process, which is performed using a fully connected multilayer perceptron network (MLP). The experimental results revealed that the proposed deep fusion model provides about 10% improvement in overall accuracy (OA) in comparison with other conventional feature-level fusion techniques.

CETAF Collection Dashboard: Mapping natural history collections diversity

Several initiatives aim to map the diversity of Natural History (NH) collections and standardise their descriptions. The Global Registry of Biodiversity Repositories (GRBio) is the most recent global registry. Unfortunately the server has been down since mid-2018 but the Global Biodiversity Information Facility (GBIF) recently "rescued" this data. In addition to this, the One World Collection exercise is a set of high-level collection descriptors (size, group coverage and geographic distribution) supporting a common strategy between the largest world institutions. Despite these efforts, a large part of the NH collections remains digitally unavailable and digitisation at the specimen level will take several decades. A new NH collections dashboard is needed in order to harmonise the efforts of the institutions. The Consortium of European Taxonomic Facilities (CETAF) is a good place to introduce this excercise. CETAF’s members hold over half of the world’s NH collections, representing 80% of the world's bio- and geo-diversity. Most of these collections are now engaged in the preparation for the common process of the Distributed System of Scientific Collections (DiSSCo, European Strategy Forum on Research Infrastructure). Additionally in Belgium, the Royal Belgian Institute of Natural Sciences (RBINS), Royal Museum of Central Africa (RMCA), Meise Botanic Garden (MBG) and CETAF have joined efforts to set up a common research portal (Natural Heritage, BRAIN-be project). The goal is to link together several collection management systems (CMS) and to (re)create links between isolated collection items. The CETAF collections dashboard splits the information into small metadata units related to topics relevant to the collections (taxonomy, geographic distribution, digitisation strategy and coverage, stratigraphy, etc.). The model allows for the creation of new units without a complete modification of the database structure. All units are defined by the Dublin Core and by fields derived from the Innovation and consolidation for large scale digitisation of natural heritage (ICEDIG) d2.3 deliverable (van Egmond et al. 2019). The object hierarchy allows for the creation of sub-collections and preserves the unity of the information. The CMS has an internal object database with a full index and a faceted search interface. It also has web services and XLS (Microsoft® Excel®) import/export functionalities. The collection dashboard also includes a complete workflow and access rights management at the object level. This is important for the information that is protected by the EU General Data Protection Regulation (GDPR). The dashboard is now being evaluated with the collections hosted by the partners of Natural Heritage. The system will be proposed to CETAF members and connections will be established with the international portals such as the GBIF or the future DiSSCo portal.

On the influence of low-level visual features in film classification.

BackgroundIn this paper we present a model of parameters to aesthetically characterize films using a multi-disciplinary approach: by combining film theory, visual low-level video descriptors (modeled in order to supply aesthetic information) and classification techniques using machine and deep learning.MethodsFour different tests have been developed, each for a different application, proving the model's usefulness. These applications are: aesthetic style clustering, prediction of production year, genre detection and influence on film popularity.ResultsThe results are compared against high-level information to determine the accuracy of the model to classify films without knowing such information previously. The main difference with other film characterization approaches is that we are able to isolate the influence of high-level descriptors to really understand the relevance of low-level features and, accordingly propose a useful set of low-level visual descriptors for that purpose. This model has been tested with a representative number of films to prove that it can be used for different applications.