Representation Ability Research Articles

Multimodal Sentiment Analysis Based on Attentional Temporal Convolutional Network and Multi-Layer Feature Fusion

Multimodal sentiment analysis aims to extract and integrate information from different modalities to accurately identify the sentiment expressed in multimodal data. How to effectively capture the relevant information within a specific modality and how to fully exploit the complementary information among multiple modalities are two major challenges in multimodal sentiment analysis. Traditional approaches fail to obtain the global contextual information of long time-series data when extracting unimodal temporal features, and they usually fuse the features from multiple modalities with the same method and ignore the correlation between different modalities when modeling inter-modal interactions. In this paper, we first propose an Attentional Temporal Convolutional Network (ATCN) to extract unimodal temporal features for enhancing the feature representation ability, then introduce a Multi-layer Feature Fusion (MFF) model to improve the effectiveness of multimodal fusion, which fuses the different-level features by different methods according to the correlation coefficient between the features, and cross-modal multi-head attention is used to fully explore the potential relationship between the low-level features. The experimental results on SIMS and CMU-MOSI datasets show that the proposed model achieves superior performance on sentiment analysis tasks compared to state-of-the-art baselines.

Billion-Scale Bipartite Graph Embedding: A Global-Local Induced Approach

Bipartite graph embedding (BGE), as the fundamental task in bipartite network analysis, is to map each node to compact low-dimensional vectors that preserve intrinsic properties. The existing solutions towards BGE fall into two groups: metric-based methods and graph neural network-based (GNN-based) methods. The latter typically generates higher-quality embeddings than the former due to the strong representation ability of deep learning. Nevertheless, none of the existing GNN-based methods can handle billion-scale bipartite graphs due to the expensive message passing or complex modelling choices. Hence, existing solutions face a challenge in achieving both embedding quality and model scalability. Motivated by this, we propose a novel graph neural network named AnchorGNN based on global-local learning framework, which can generate high-quality BGE and scale to billion-scale bipartite graphs. Concretely, AnchorGNN leverages a novel anchor-based message passing schema for global learning, which enables global knowledge to be incorporated to generate node embeddings. Meanwhile, AnchorGNN offers an efficient one-hop local structure modelling using maximum likelihood estimation for bipartite graphs with rational analysis, avoiding large adjacency matrix construction. Both global information and local structure are integrated to generate distinguishable node embeddings. Extensive experiments demonstrate that AnchorGNN outperforms the best competitor by up to 36% in accuracy and achieves up to 28 times speed-up against the only metric-based baseline on billion-scale bipartite graphs.

A practical investigation of cultivating students’ art innovation ability based on word vector model

Abstract In this paper, a multi-branch deep network architecture (JESF-MBF) based on joint embedded semantic features is proposed. In order to construct the structural consistency between visual image space and label semantic space, as well as to ensure the correspondence between sketch or image data and labels, the JESF-MBF model uses cross-entropy-based multivariate classification loss and cosine distance-based semantic similarity loss for model optimization. The optimized model improves the ability of label semantic information representation and image visual information representation, alleviates the discrepancy of data domain distribution between sketch and image, and constructs the mapping relationship between visual embedding space and semantic embedding space, so the sketch-based zero-sample image retrieval task is realized based on the intermediate medium. In the comparative model analysis, the average evaluation index of JESF-MBF is 0.412, better than that of Doodle2search at 0.344. In terms of the teaching effectiveness of the teaching platform constructed in this paper, the average time spent for the experimental class to complete each subject work is 9.4 class periods, while the average time required for the experimental class is 19.7 class periods. The results show that the art teaching platform based on the JESF-MBF model is very effective in terms of performance as well as improving the art level.

Analisis Kemampuan Representasi Matematis Pada Materi Barisan Dan Deret

This research aims to describe the level of ability of 32 students in class XII - MIPA 1 at SMA Negeri 1 Jatiwangi. The question instrument used in this research is 5 written test questions which refer to indicators of students' mathematical representation abilities. The data analysis technique used in this research is an interactive analysis technique which consists of three components, namely data reduction, data presentation, and drawing conclusions. The results of this research show that (1) The level of students' mathematical representation ability has an average score of 39.32 in the medium category. (2) Errors in solving statistics problems which include conceptual errors (wrong understanding of the meaning of the problem), principle errors (wrong conclusions in determining the final answer to the problem, wrong use of rules in mathematical formulas), and operational errors (wrong in using operations in mathematics such as addition, subtraction, division and multiplication). students' mathematical representations, and students' errors in solving sequences and series problems based on indicators of students' mathematical representation abilities, in class XII - MIPA 1 at SMA Negeri 1 Jatiwangi. This research uses descriptive research with a qualitative approach. The subject of this research.

Research on multi-granularity imbalanced knowledge condition monitoring for mechanical equipment based on hierarchical ELM in multi-entropy space

Condition monitoring is essentially a complex multi-classification task that requires struggling with multiple conditions and imbalanced data. To address this issue, this study proposes a hierarchical cognition method that utilizes multi-entropy feature representation to capture the state of equipment under complex working conditions. Firstly, this study proposes a feature modeling method for mechanical equipment based on multi-entropy space, utilizing the cloud model as a space component. Secondly, the study uses multi-entropy feature data to develop a hierarchical Extreme Learning Machine (HELM) fault identification model for complex working conditions and data imbalance inspired by human hierarchical cognitive behavior. The feature data are decomposed into four levels, namely speed, load, abnormality, and fault, respectively. Additionally, a hierarchical feature selection mechanism is designed to improve the efficiency of HELM. Finally, comparative experiments are carried out on two platforms. Experiments show that the If factors of the two datasets in the multi-entropy space reach 0.6888 and 10.4881, respectively. At the same time, in the classification experiment for the imbalanced data, the classification accuracy reaches 90.68 when the fault samples are 10. The results show that the multi-entropy feature created in this study has a specific representation ability for mechanical states. Furthermore, the proposed hierarchical extreme learning method can effectively identify faults under multiple conditions and data imbalance backgrounds.

The use of graphic design elements in commercial architectural design in the context of intelligence

Abstract In this paper, firstly, in the context of intelligence, using techniques such as hierarchical features, local perceptual field, weight sharing, and spatial subsampling make the convolutional neural network with representational learning ability, certain translation, scaling, and deformation invariance, which can accurately extract the features of target objects in graphic design elements. Next, the network structure, loss function, and optimization methods of MaskR-CNN are analyzed, and the MaskR-CNN convolutional neural network is implemented based on the TensorFlow framework, and the network is applied to the extraction of buildings from graphic design elements. Then the borrowed landscape technique in graphic design is used to enrich the combination form of elements of architectural design through the mirror effect of water landscape. The selected IALD dataset is composed of 180 data images and 180 corresponding labeled images, and the effect of high-resolution building extraction of graphic design elements based on the MaskR-CNN model is verified by simulation experiments. The simulation results show that Mask R-CNN improves the accuracy of building extraction from graphic design elements by 5.11%, mAP also improves by about 10.09%, and the inference time is less than 1 second; Mask R-CNN has good robustness and accuracy for building extraction from graphic design elements. This study has important research significance for the depth and development of the commercial architectural design.

Development of Pisa Model Math Problems with Central Java Cultural Context

This research aims to develop PISA model mathematics questions with a Central Javanese cultural context that are valid, reliable and have a potential effect on students' mathematical literacy abilities. The research method used in this research is design research type development studies, which consists of two stages: preliminary and formative evaluation. The resulting test instrument was then validated by experts consisting of experienced Mathematics Education lecturers and Mathematics Teachers. The respondents in this research were 29 students at SMP IT Harapan Ummat Purbalingga. The product of this research is PISA model Mathematics test questions with a Central Javanese cultural context that are valid, reliable and have a potential effect on students' literacy abilities based on the PISA framework. The validation results were analyzed using the Aiken V index by calculating the correlation coefficient ( ) and the Aiken V coefficient value was between 0.583 to 0.917, which means that for all questions, 4 questions had high criteria and 3 questions had medium criteria. Then reliability was analyzed using IBM SPSS Statistics 2022 by determining the Cronbach Alpha coefficient and obtained a value of 0.904. Then, from the results of the analysis at the field test stage, it can be seen that several basic mathematical abilities are emerging which are based on mathematical literacy abilities such as reasoning and argumentation abilities of 44.3%, communication skills of 34%, representation abilities of 41.25%, ability to formulate strategies to solve problems. Problems amounted to 70.3%, the ability to use symbolic language, formats, and techniques and operations amounted to 57.67%, and the ability to use mathematical tools amounted to 23.5%.

Towards self-explainable graph convolutional neural network with frequency adaptive inception

Graph convolutional neural networks (GCNs) have demonstrated powerful representing ability of irregular data, e.g., skeletal data and graph-structured data, providing the effective mechanism to fuse the neighbor nodes. However, inheriting from the deep learning, GCN also lacks interpretability, which hinders its application to scenarios that have high demand for transparency. Although, there have been many efforts on the interpretability of deep learning, they mainly concentrate on i.i.d data that is hard to be deployed to GCNs, which involve not only the node feature, but also the graph structure. There are few works that attempt to explain it with post-hoc manner, which can be biased, resulting in mis-representation of the true explanation. Therefore, in this paper, we propose a framework, namely ExpFiGCN, that reveals explainability of the GCNs from the perspective of graph structure and mathematical analysis. Specifically, ExpFiGCN can find the most intrinsically relevant node to the central node and obtain the informative and discriminative signals while performing denoising. For the graph structure, we find K-nearest nodes; for the mathematical analysis, every channel of a node and its neighborhoods contribute dynamically to the final channel signal, which can capture the inherent difference of different channels and neighbor nodes. Meanwhile, it can enhance the representation ability of nodes and ameliorate the over-smoothing problem. On the other hand, our model can dynamically adjust the importance of neighborhoods to the central vertex. We empirically validate the effectiveness of the proposed framework ExpFiGCN on various benchmark datasets. Experimental results show that our method achieves substantial improvements and outperforms the state-of-the-art performance strikingly.

Dual-Branch Fourier-Mixing Transformer Network for Hyperspectral Target Detection

In recent years, transformers have shown great potential in hyperspectral image processing and have also been gradually applied in hyperspectral target detection (HTD). Nonetheless, applying a typical transformer to HTD remains challenging. The heavy computation burden of the multi-head self-attention (MSA) in transformers limits its efficient HTD, while the limited ability to extract local spectral features can reduce the discrimination of the learned spectral features. To further explore the potential of transformers for HTD, for balance of representation ability and computational efficiency, we propose a dual-branch Fourier-mixing transformer network for hyperspectral target detection (DBFTTD). First, this work explores a dual-branch Fourier-mixing transformer network. The transformer-style network replaces the MSA sublayer in the transformer with a Fourier-mixing sublayer, which shows advantages in improving computational efficiency and learning valuable spectral information effectively for HTD. Second, this work proposes learnable filter ensembles in the Fourier domain that are inspired by ensemble learning to improve detection performance. Third, a simple but efficient dropout strategy is proposed for data augmentation. Sufficient and balanced training samples are constructed for training the dual-branch network, and training samples for balanced learning can further improve detection performance. Experiments on four data sets indicate that our proposed detector is superior to the state-of-the-art detectors.

Estimation of LFM signal parameters using RD compressed sampling and the DFRFT dictionary

In this paper, a method combining random demodulator (RD) and discrete fractional Fourier transform (DFRFT) dictionary is suggested to directly estimate the parameters of linear frequency modulation (LFM) signals from compressed sampling data. First, the RD system parameters are modified in accordance with the characteristics of the LFM signal to produce effective compressed sampling data. Next, a DFRFT dictionary is built using the fractional Fourier transform theory, and sparse representation coefficients are obtained by reconstructing the compressed sampling data using the recovery algorithm and DFRFT dictionary. The signal exhibits characteristics that make it pulse under the best fractional transform order, so the problem of signal parameter estimation can be reduced to searching for the location of the maximum value of sparse representation coefficients. The location is determined by a parameter optimization algorithm, and from there, the initial frequency and Chirp rate of the LFM signal can be estimated. Lastly, simulation and real data tests are performed to confirm that the suggested method can directly be utilized to estimate the parameter of LFM signals using compressed sampling data in addition to having high sparse representation ability for LFM signals.

MiPEPPred-FRL: A Novel Method for Predicting Plant MiRNA-Encoded Peptides Using Adaptive Feature Representation Learning.

MicroRNAs (miRNAs) are an essential type of small molecule RNAs that play significant regulatory roles in organisms. Recent studies have demonstrated that small open reading frames (sORFs) harbored in primary miRNAs (pri-miRNAs) can encode small peptides, known as miPEPs. Plant miPEPs can increase the abundance and activity of cognate miRNAs by promoting the transcription of their corresponding pri-miRNAs, thereby modulating plant traits. Biological experiments are the most effective way to accurately identify miPEPs; however, they are time-consuming and expensive. Hence, an efficient computational method for the identification of miPEPs on a large scale is highly desirable. Up to now, there have been no specialized computational tools for identifying miPEPs. In this work, a novel predictor named miPEPPred-FRL based on an adaptive feature representation learning framework that consists of the feature transformation module and the cascade architecture has been proposed. The feature transformation module integrating a newly designed feature selection method and classifier selection rule is developed to convert sequence-based features into primary class and probabilistic features, which are then fed into the improved cascade architecture to obtain more stable and discriminative augmented features. Finally, the augmented features are utilized to construct the final predictor. Cross-validation experiments illustrate that the novel feature selection method and classifier selection rule contribute to boosting the feature representation ability of the framework. Furthermore, the high accuracy of miPEPPred-FRL on independent testing data suggests that it is a trustworthy and valuable tool for the identification of miPEPs.

MacJNet: weakly-supervised multimodal image deformable registration using joint learning framework and multi-sampling cascaded MIND

Deformable multimodal image registration plays a key role in medical image analysis. It remains a challenge to find accurate dense correspondences between multimodal images due to the significant intensity distortion and the large deformation. macJNet is proposed to align the multimodal medical images, which is a weakly-supervised multimodal image deformable registration method using a joint learning framework and multi-sampling cascaded modality independent neighborhood descriptor (macMIND). The joint learning framework consists of a multimodal image registration network and two segmentation networks. The proposed macMIND is a modality-independent image structure descriptor to provide dense correspondence for registration, which incorporates multi-orientation and multi-scale sampling patterns to build self-similarity context. It greatly enhances the representation ability of cross-modal features in the registration network. The semi-supervised segmentation networks generate anatomical labels to provide semantics correspondence for registration, and the registration network helps to improve the performance of multimodal image segmentation by providing the consistency of anatomical labels. 3D CT-MR liver image dataset with 118 samples is built for evaluation, and comprehensive experiments have been conducted to demonstrate that macJNet achieves superior performance over state-of-the-art multi-modality medical image registration methods.

A swin transformer based bird nest detection approach with unmanned aerial vehicle images for power distribution and pole towers

AbstractThe authors propose a novel object detection algorithm for identifying bird nests in medium voltage power line aerial images, which is crucial for ensuring the safe operation of the power grid. The algorithm utilises an improved Swin Transformer as the main feature extraction network of Fast R‐CNN, further enhanced with a channel attention and modified binary self‐attention mechanism to improve the feature representation ability. The proposed algorithm is evaluated on a newly constructed image dataset of medium voltage transmission lines containing bird nests, which are annotated and classified. Experimental results show that the proposed algorithm achieves satisfied accuracy and robustness in recognising bird nests compared to traditional algorithms.

Intelligent recognition and behavior tracking of sea cucumber infected with Vibrio alginolyticus based on machine vision

The outbreak of aggregative diseases in the process of sea cucumber cultivation has brought huge economic losses to aquaculture farmers. It is of positive significance to realize intelligent detection of abnormal behavior to avoid the outbreak of aggregative diseases. Therefore, this paper researches the approaches of intelligent recognition and behavior tracking of sea cucumbers. Fusing the Coordinated Attention and Bi-directional Feature Pyramid Network, the DT-YOLOv5 intelligent recognition model is proposed to enhance the representation ability and feature extraction ability. A multi-object behavior tracking approach is presented based on the automatic frame-matching coordinates, which can track multiple objects and calculate the volumes of exercise. The experimental results show that the precision, recall and AP50:95 are 99.43%, 98.91% and 84.89%, respectively. This research provides a theoretical support for the detection of abnormal behavior of aquatic animals during intensive aquaculture and has potential practical application value for protecting the welfare of sea cucumbers and improving the intelligence level of aquaculture.

Causal-ViT: Robust Vision Transformer by causal intervention

Artificial intelligence based on deep learning is better at improving the representation ability of models from data. However, due to the limitation of fixed receptive field, these agents are not able to provide a correct response outside the fixed receptive field. To address this problem, this paper provides a new perspective with improving the Image Recognition tasks. This study firstly constructs two extended receptive fields using structural causal model. Then, an approximate intervention method that changes the traditional likelihood prediction to predict the result of causal intervention is proposed. Finally, this study formulates the objective function to adapt the proxy training, which makes the whole model work well. Above all of these, a new Vision Transformer variant named Causal-ViT is proposed. Furthermore, rich experimental results of different tasks are reported. These results show that the proposed perspective makes a significant improvement in Image Recognition tasks. By simply plugging Causal-ViT to different sub-tasks, all of them bring the new benchmarks of themselves field, which proves our method is flexible.

Spatial weighted graph-driven fault diagnosis of complex process industry considering technological process flow

Each chemical process industry system possesses unique process knowledge, which serves as a representation of the system’s state. As graph-theory based methods are capable of embedding process knowledge, they have become increasingly crucial in the field of process industry diagnosis. The fault representation ability of the diagnosis model is directly associated with the quality of the graph. Unfortunately, simple fully connected graphs fail to strengthen the internal connections within the same process but weaken the interactive connections between different processes. Moreover, each node in the graph is considered equally important, making it impossible to prioritize crucial system monitoring indicators. To address the above shortcomings, this paper presents a spatial weighted graph (SWG)-driven fault diagnosis method of complex process industry considering technological process flow. Initially, the physical space sensor layout of the technological process flow is mapped into the spatial graph structure, where each sensor is regarded as a node and these nodes are connected by the k nearest neighbor algorithm. Subsequently, according to the mechanism knowledge, the sensors in the process are divided into different importance categories and weight coefficients are assigned to their nodes. The similarities between these weighted nodes are calculated, and the resulting edge information are used to construct the SWGs. Finally, the SWGs are input to a graph convolutional network, facilitating fault representation learning for fault diagnosis of complex process industry. Validation experiments are conducted using public industrial datasets, and the results demonstrate that the proposed method can effectively integrate the process knowledge to improve the fault diagnosis accuracy of the model.

The effectiveness of Problem-Based Learning physics pocketbook integrating Augmented Reality with the local wisdom of catapults in improving mathematical and graphical representation abilities

This research aimed to reveal the effectiveness of using a Problem-Based Learning (PBL) physics pocketbook integrating augmented reality (AR) with the local wisdom of catapults in improving the mathematical and graphical representation abilities of high school students. This study is a quasi-experiment with a pretest-posttest control group design. The data collection was carried out in two stages, namely empirical testing and effectiveness testing stages. The empirical test was conducted on 276 grade XII students, while the effectiveness test was conducted on three science classes of grade XI students of State Senior High School (SMAN) 10 Yogyakarta. The three classes consisted of 36 students in the experimental class, 36 students in the Contrast Class 1, and 36 students in the Contrast 2 Class. The mathematical and graphical representation abilities were measured through a test instrument that passes the analysis of the empirical test using the QUEST program. General Linear Model (GLM) was utilized to analyze the effectiveness test, which was supported by the SPSS software. The result of the General Linear Model (GLM) analysis in the experimental class showed a significant value of 0.000 on the mathematical and graphical representation ability variables. The findings of this study indicate that the use of the developed Problem-Based Learning physics pocketbook integrating augmented reality with local wisdom is effective in increasing the mathematical and graphical representation abilities of grade XI students, with an effective contribution of 82.6% and 84.4% each.

SFPFusion: An Improved Vision Transformer Combining Super Feature Attention and Wavelet-Guided Pooling for Infrared and Visible Images Fusion.

The infrared and visible image fusion task aims to generate a single image that preserves complementary features and reduces redundant information from different modalities. Although convolutional neural networks (CNNs) can effectively extract local features and obtain better fusion performance, the size of the receptive field limits its feature extraction ability. Thus, the Transformer architecture has gradually become mainstream to extract global features. However, current Transformer-based fusion methods ignore the enhancement of details, which is important to image fusion tasks and other downstream vision tasks. To this end, a new super feature attention mechanism and the wavelet-guided pooling operation are applied to the fusion network to form a novel fusion network, termed SFPFusion. Specifically, super feature attention is able to establish long-range dependencies of images and to fully extract global features. The extracted global features are processed by wavelet-guided pooling to fully extract multi-scale base information and to enhance the detail features. With the powerful representation ability, only simple fusion strategies are utilized to achieve better fusion performance. The superiority of our method compared with other state-of-the-art methods is demonstrated in qualitative and quantitative experiments on multiple image fusion benchmarks.

CNN and Transformer Fusion for Remote Sensing Image Semantic Segmentation

Semantic segmentation of remote sensing images has been widely used in environmental protection, geological disaster discovery, and natural resource assessment. With the rapid development of deep learning, convolutional neural networks (CNNs) have dominated semantic segmentation, relying on their powerful local information extraction capabilities. Due to the locality of convolution operation, it can be challenging to obtain global context information directly. However, Transformer has excellent potential in global information modeling. This paper proposes a new hybrid convolutional and Transformer semantic segmentation model called CTFuse, which uses a multi-scale convolutional attention module in the convolutional part. CTFuse is a serial structure composed of a CNN and a Transformer. It first uses convolution to extract small-size target information and then uses Transformer to embed large-size ground target information. Subsequently, we propose a spatial and channel attention module in convolution to enhance the representation ability for global information and local features. In addition, we also propose a spatial and channel attention module in Transformer to improve the ability to capture detailed information. Finally, compared to other models used in the experiments, our CTFuse achieves state-of-the-art results on the International Society of Photogrammetry and Remote Sensing (ISPRS) Vaihingen and ISPRS Potsdam datasets.

MPCNet: Compressed multi-view video restoration via motion-parallax complementation network

The performance in restoring compressed multi-view video (MVV) of the existing learning-based methods is limited because they only utilize information of temporally adjacent frames or parallax neighboring views. However, the compression artifacts caused by multi-view coding (MVC) may be related to the reference errors of intra-frame, inter-frame, and inter-view. In this paper, with delicately utilizing the stereo information from both temporal and parallax domains, a motion-parallax complementation network (MPCNet) is proposed to restore the quality of compressed MVV more efficiently. First, we introduce a motion-parallax complementation strategy consisting of a coarse stage and a fine stage. By mutually compensating the feature extracted from multiple domains, useful multi-frame information can be efficiently preserved and aggregated step by step. Second, an attention-based feature filtering and modulation module (AFFM) is proposed, which provides an efficient fusion method for two features by suppressing misleading information. By deploying it in most submodules of the proposed approach, the representational ability of MPCNet can be improved, resulting in a more substantial restoration performance. Experimental results prove the effectiveness of MPCNet by an average increase of 1.978 dB in PSNR, and 0.0282 in MS-SSIM. The BD-rate reduction can reach 47.342% on average. The subjective quality is greatly improved and lots of compression distortions are eliminated. Meanwhile, this work also benefits the accuracy improvement for high-level vision tasks, e.g., mIoU of semantic segmentation and mAP of object detection achieve 0.352 and 51.71, respectively. Quantitative and qualitative analyses demonstrate that MPCNet outperforms state-of-the-art approaches.