Multimodal Propagation Research Articles

Social Media Sentiment Analysis

Social media sentiment analysis is the computational detection and extraction of human subjective evaluation of objects embedded on social media. Previous sentiment analysis was conducted on isolated written texts, and typically classified sentiment into positive, negative, and neutral states. Social media sentiment analysis has included multi-modal texts, temporal dynamics, interactions, network relationships, and sentiment propagation. Specific emotions and sentiment intensity are also detected.

Multimodal uncertainty propagation analysis for the morphing wings of cross-domain variant aircraft

Multimodal uncertainty propagation analysis for the morphing wings of cross-domain variant aircraft

Spectral-temporal-spatial customization via modulating multimodal nonlinear pulse propagation

Multimode fibers (MMFs) are gaining renewed interest for nonlinear effects due to their high-dimensional spatiotemporal nonlinear dynamics and scalability for high power. High-brightness MMF sources with effective control of the nonlinear processes would offer possibilities in many areas from high-power fiber lasers, to bioimaging and chemical sensing, and to intriguing physics phenomena. Here we present a simple yet effective way of controlling nonlinear effects at high peak power levels. This is achieved by leveraging not only the spatial but also the temporal degrees of freedom during multimodal nonlinear pulse propagation in step-index MMFs, using a programmable fiber shaper that introduces time-dependent disorders. We achieve high tunability in MMF output fields, resulting in a broadband high-peak-power source. Its potential as a nonlinear imaging source is further demonstrated through widely tunable two-photon and three-photon microscopy. These demonstrations provide possibilities for technology advances in nonlinear optics, bioimaging, spectroscopy, optical computing, and material processing.

Directional Multi-Frequency Guided Waves Communications Using Discrete Frequency-Steerable Acoustic Transducers.

A novel directional transducer based on Guided Waves (GW) is introduced in this paper, designed for use in structural health monitoring (SHM) and acoustic data communication applications, i.e., systems in which the elastic medium serves as a transmission channel and information is conveyed through the medium via elastic waves. Such systems can overcome difficulties associated with traditional communication methods like wire-based or radio frequency (RF), which can be complex and have limitations in harsh environments or hard-to-reach places. However, the development of these techniques is hampered by GW dispersive and multi-modal propagation and by multi-path interference. The shortcomings can be effectively addressed by employing Frequency Steerable Acoustic Transducers (FSATs), which leverage their inherent directional capabilities. This can be achieved through the exploitation of a frequency-dependent spatial filtering effect, yielding to a direct correlation between the frequency content of the transmitted or received signals and the direction of propagation. The proposed transducer is designed to actuate or sense the A0 Lamb wave propagating in three orientations using varying frequencies, and has three channels with distinct frequencies for each direction, ranging from 50 kHz to 450 kHz. The transducer performance was verified through Finite Element (FE) simulations, accompanied by experimental testing using a Scanning Laser Doppler Vibrometer (SLDV). The unique frequency-steering capability of FSATs is combined with the On-Off Keying (OOK) modulation scheme to achieve frequency directivity in hardware, similar to ongoing research in 5G communications. The MIMO capabilities of the transducer were finally tested over a thin aluminum plate, showing excellent agreement with the FE simulation results.

Learning modality feature fusion via transformer for RGBT-tracking

RGB-T tracking can be seen as multi-view fusion tracking, and in this study, we propose a network with transformer structure, Multi-Modal Mutual Propagation Tracker (MMMPT). In order to obtain robust appearance model from multi-modal data, we adopt encoder–decoder architecture for extract information. In the encoding stage, the template features of multiple frames enhance the common features across them through the self-attention mechanism to obtain time-invariant target representation. At the same time, it also interacts with multi-modal data through cross-modal propagation, resulting in a modal-invariant representation of the target. The transformer decoder transfers useful information from the template to search areas through a similarity matrix. We experiment on the RGBT234, GTOT, VTUAV and LasHeR datasets to assess the RGBT-transformer tracker. Extensive experiments indicate that our proposed framework is not inferior to the state-of-the-art trackers in terms of robustness and accuracy.

A Comparison of the Performance of Four Acoustic Modulation Techniques for Robot Communication in Pipes

Autonomous robotics is an emerging technology for the inspection of a vast network of underground pipes. Communication between autonomous robots is essential to optimise their efficiency and network coverage. However, sending a message acoustically is not a well-researched topic because most of the existing literature is devoted to the study of the acoustic properties of the pipe for the purpose of sensing rather than communication. In particular, the influence of multi-modal propagation and background noise on the quality of acoustic communication in pipes has not been well understood. This paper studies the performance of four standard acoustic communication techniques in a dry drainage pipe to fill this knowledge gap. The noise resistance, communication range and data rate of these techniques are estimated through numerical simulations and laboratory experiments. It has been found that the techniques based on shift keying requires at least 5--10 dB signal to noise ratio (SNR) to function properly while the chirp linear frequency modulation technique can operate reliably with SNR = 0 dB or event lower SNR. The results also suggest that the multi-modal propagation in the pipe has significant effects on the package error rate. The frequency dependent sound attenuation in the pipe also affects the communication range and data rate. In particular, for a 150 mm diameter dry pipe the maximum robot operation distance is likely to be limited to 50--100 m with the highest carrier frequency of around 10 kHz and data rates below 6300 bps. The results of this work pave the way to the development of acoustic communication modules to be deployed on tetheless robots designed to inspect a buried pipe network autonomously and collaboratively.

Array ultrasonic guided wave spiral focusing detection method for inner damage of messenger cable in covered area

The messenger cable damage detection of catenary system is one of the important safeguard to ensure the safe operation of electrified railways. Due to the multi-layer and opposite spiral direction structure of the messenger cable, the ultrasonic guided wave dispersion curve is difficult to obtain, and the inner layer damage signal is too weak to identify. A novel array ultrasonic guided wave spiral focusing detection method is proposed for the inner layer damage of messenger cable in covered area. Firstly, the dispersion curves of ultrasonic guided waves in the messenger cable were calculated by semi-analytical finite element based on twisted coordinate system, and the multi-modal propagation characteristics were analyzed. Secondly, the path differences from guided wave array elements to inner layer were studied to get the adjustment matrix of spiral focusing. Thirdly, the inner damage information was extracted from spiral focusing signal by cross sparse representation based on dispersion dictionary, and the damage autofocus imaging was realized by bi-directional time reversal imaging. Finally, the spiral focusing enhancement effect and identification imaging effect of inner layer damage were analyzed and verified through simulation and experiments. The results show that, compared with the normal superimposed method, the damage signal indexes of sub-outer layer and the center layer are increased by 32.2% and 40.8% respectively after spiral focusing. The inner layer damage signal of the spiral focusing method has a higher recognition rate and good anti-interference ability, and the damage imaging has low sensitivity to threshold value and has no damage artifacts.

Multi-Modal Scattering and Propagation Through Several Close Periodic Grids

The presented method describes reflection and transmission of electromagnetic waves at multiple closely stacked metal grids using multimodal S-parameter propagation. For that, the periodic surface admittance curve of every metal layer was determined through a novel semianalytic approach that uses simple Fourier transformations instead of solving an integral equation. The modal components of this surface admittance were used to express the generalized scattering matrices of the individual grids. By applying multimodal propagation techniques to the resulting scattering parameters, it was possible to model the electromagnetic interactions within a multilayer stack of periodic impedance sheets. Resulting reflection and transmission parameters perfectly matched the corresponding full-wave simulations even above the grating lobe regime. In the end, the universal mode propagation method enabled modeling of a layer stack, connected to lumped components.

Global–Local Transformer Network for HSI and LiDAR Data Joint Classification

Hyperspectral images (HSI) contain rich spatial and spectral detail information, while light detection and ranging (LiDAR) data can provide the elevation information. Thus, the fusion of HSI and LiDAR data can help for more accurate image classification, which becomes a hot research topic. However, it is difficult to capture complex local and global spatial-spectral associations, meanwhile, how to build an effective interaction between multi-modal data is another important issue. To this end, a novel global-local transformer network (GLT-Net) is proposed for the joint classification of HSI and LiDAR data, in this paper. The main idea is to fully exploit the advantage of the convolution operator in characterizing locally correlated features and the promising capability of transformer architecture in learning long-range dependencies. Moreover, multi-scale feature fusion and probabilistic decision fusion strategies are also designed in one framework, in order to further improve classification performance. Here, the proposed GLT-Net mainly consists of multi-scale local spatial feature learning, global spectral feature learning, and global-local feature fusion classification. In specific, multi-modal image cubes of different sizes are firstly extracted and sent into convolutional neural networks (CNNs) to learn local spatial features, which is followed by multi-modal information propagation and spatial-attention guided multi-scale feature fusion. Afterwards, by considering spectral feature channels from a sequential perspective, vision transformers are introduced to model the global spectral dependencies. Finally, multiple class estimations based on local and global features are integrated via a probabilistic decision fusion strategy. In this way, complementary information of multi-modal data as well as local/global spectral-spatial information can be fully mined and jointly utilized. Extensive experiments on three popular HSI and LiDAR datasets demonstrate that the proposed method performs superiority over state-of-the-art methods. The source code of the proposed method will be made publicly available at https://github.com/Ding-Kexin/GLT-Net.

Multi-Modal Context Propagation for Person Re-Identification With Wireless Positioning

Existing person re-identification methods mainly rely on thevisualappearance captured by camerasfor identity matching.However, due to the sensitivity of visual data to occlusion, blur, clothing change, etc.,existing methodsstruggleto distinguish pedestriansin challenging scenarios. Inspired by the fact that most pedestrians carry around smart wireless devices, e.g. mobile phones that can be sensed by WiFi or cellular networks as wireless positioning signals, we propose to exploit the free yet informative wireless signalsto assist person re-identification. It is well recognized thatwireless signals are robust to visual noisesmentioned above, which perform as a good complement to the visual data. To make full use of these multi-modal clues for person re-identification, we propose a multi-modal context propagation framework MCPF that contains a recurrent context propagation module RCPM and an unsupervised multi-modal cross-domain method UMM-ReID. RCPM enables context information to be continuously propagatedand fusedbetween visual data and wireless data. UMM-ReID utilizes wireless signals to constrain the estimation of pseudo labels. To evaluate our approach, we contribute a new wireless positioning person re-identification (WP-ReID) dataset. Extensive experimentsdemonstrate the effectiveness of the proposed method. Benefitingfrom the collaboration of RCPM and UMM-ReID, the proposed framework MCPF achieves a significant performance improvement over existing methods.

Active noise control of higher modes in a duct using near field compensation and a ring of harmonic acoustic pneumatic sources

The motivation of this work was to develop an efficient method of tonal active control of the sound power radiated by turbofans. To develop and validate a concept with a multimodal propagation model, the simplified geometry of a duct cylinder is considered for the problem statement. Two design constraints are imposed: i) an array of Harmonic Acoustic Pneumatic Source (HAPS) must be used as secondary sources instead of loudspeakers ii) an array of error microphones must be implemented close to the exhaust and secondary sources. The requirement of error microphones in the near-field of control sources leads to an original theoretical multivariable control strategy by compensation. It is practically implemented with a preliminary learning step based on the measurement of the optimal control inputs and the residual sound field at the near-field microphones resulting from the direct minimization of the radiated sound power. Numerical studies allow the validation of this control strategy and the analysis of the active compensation mechanisms. A dedicated controller is designed to provide highly accurate magnitude and phase of the HAPS acoustic response. Sound power attenuations larger than 20 dB are measured on an experimental test bench using a ring of 6 HAPS and a ring of 6 near-field microphones in situations where the primary sound power is up to 130 dB-SWL. All the experimental results are consistent with the numerical analysis of the compensation matrix.

An uncertainty propagation analysis method considering multimodal random distribution

<p indent=0mm>In practical engineering applications, random variables may follow a multimodal random distribution, such as damping characteristics of train shock absorber after long-term use and fatigue life of safety cut specimens after repair. Traditional uncertainty propagation analysis methods are mainly used for random variables following a unimodal distribution. Therefore, large errors will be produced when random variables follow a multimodal distribution. In this study, an uncertainty propagation analysis method considering a multimodal random distribution is proposed. First, the probability density function of input multimodal random variables is defined by the Gaussian mixture model; then, the high-order moments of response function are calculated using the efficient trivariate dimension reduction method, which effectively improves the computational efficiency of uncertainty propagation. Once again, the maximum entropy method is used to compute the probability density function of response, whereas an adaptive method based on entropy is used to determine the optimal order of statistical moments. Finally, four numerical examples are demonstrated to analyze the accuracy and efficiency of the proposed method in handling multimodal uncertainty propagation analysis.

Efficient Experimental and Data-Centered Workflow for Microstructure-Based Fatigue Data

BackgroundEarly fatigue mechanisms for various materials are yet to be unveiled for the (very) high-cycle fatigue (VHCF) regime. This can be ascribed to a lack of available data capturing initial fatigue damage evolution, which continues to adversely affect data scientists and computational modeling experts attempting to derive microstructural dependencies from small sample size data and incomplete feature representations.ObjectiveThe aim of this work is to address this lack and to drive the digital transformation of materials such that future virtual component design can be rendered more reliable and more efficient. Achieving this relies on fatigue models that comprehensively capture all relevant dependencies.MethodsTo this end, this work proposes a combined experimental and data post-processing workflow to establish multimodal fatigue crack initiation and propagation data sets efficiently. It evolves around fatigue testing of mesoscale specimens to increase damage detection sensitivity, data fusion through multimodal registration to address data heterogeneity, and image-based data-driven damage localization.ResultsA workflow with a high degree of automation is established, that links large distortion-corrected microstructure data with damage localization and evolution kinetics. The workflow enables cycling up to the VHCF regime in comparatively short time spans, while maintaining unprecedented time resolution of damage evolution. Resulting data sets capture the interaction of damage with microstructural features and hold the potential to unravel a mechanistic understanding.ConclusionsThe proposed workflow lays the foundation for future data mining and data-driven modeling of microstructural fatigue by providing statistically meaningful data sets extendable to a wide range of materials.

Passive Shallow Water Automated Target Recognition using Deep Convolutional Bi directional Long Short Term Memory

The extremely challenging nature of passive acoustic surveillance makes it a key area of research in NavalNon-Co-operative Target Recognition especially in Anti-Submarine Warfare systems. In shallow waters, thecomplex acoustics due to the highly varying ambient background noise as well as the multi-modal propagation in the surface-bottom bounded channel makes surveillance even difficult. In this work, an ensemble of Convolutional Neural Networks and Bidirectional Long Short Term Memory stages employing soft attention is used to effectively capture the spectro-temporal dynamics of the target signature. In order to alleviate the overall computational cost associated with the optimal model search in the extensive hyperparameter space, a recursive model elimination scheme, making frugal use of the available resources, is also proposed. Experimental analysis on acoustic target records, collected from the shallows of Arabian Sea, has yielded encouraging results in terms of model accuracy, precision and recall.

High frequency guided wave ultrasound measurements using low-cost equipment

Guided wave ultrasound can be applied to inspect large volumes of structures from a single transducer location and is particularly effective in one-dimensional structures such as rods, pipes and rail. An application, which is of interest in South Africa, is the inspection rock bolts in the mining industry. The difficulty in inspecting embedded rods is that there is rapid attenuation of the signals with distance. While guided wave systems generally operate at low frequencies (20 kHz to 200 kHz), high frequency operation (1 MHz to 10 MHz) can offer some advantages when inspecting rock bolts. Guided wave ultrasound testing is complicated by multi-modal propagation and dispersion, which require the use of sophisticated excitation signals, which are not available in equipment designed for conventional ultrasonic testing. It was therefore necessary to develop a measurement setup and the approach taken was to use standard lab instruments wherever possible and to only purchase the items that are unique to this measurement, using the very limited budget available. The measurement setup was used to perform measurements on a rod and confirmed theoretical predictions. These measurements identified the frequencies where modes propagate with low attenuation and allow the attenuation to be quantified.

Improved multimodal formulation of the wave propagation in a 3D waveguide with varying cross-section and curvature.

An efficient method is proposed to solve the multimodal wave propagation within a three-dimensional waveguide bounded by a hard wall with varying cross section and curvature. This is achieved by first turning the original problem, in a complex-shaped waveguide, into a cylindrical waveguide with unit radius, by means of an adapted and flexible geometrical transformation. Then supplementary modes are defined to enrich the standard modal basis that is usually considered in such methods and to help restore the right boundary condition. It is shown through various numerical applications that the introduction of these supplementary modes, whatever the complexity of the waveguide geometry, significantly enhances the multimodal method, notably by increasing its convergence rate, whether one's aim is to solve the wavefield or the scattering problem.

Self-Supervised Ultrasound to MRI Fetal Brain Image Synthesis.

Fetal brain magnetic resonance imaging (MRI) offers exquisite images of the developing brain but is not suitable for second-trimester anomaly screening, for which ultrasound (US) is employed. Although expert sonographers are adept at reading US images, MR images which closely resemble anatomical images are much easier for non-experts to interpret. Thus in this article we propose to generate MR-like images directly from clinical US images. In medical image analysis such a capability is potentially useful as well, for instance for automatic US-MRI registration and fusion. The proposed model is end-to-end trainable and self-supervised without any external annotations. Specifically, based on an assumption that the US and MRI data share a similar anatomical latent space, we first utilise a network to extract the shared latent features, which are then used for MRI synthesis. Since paired data is unavailable for our study (and rare in practice), pixel-level constraints are infeasible to apply. We instead propose to enforce the distributions to be statistically indistinguishable, by adversarial learning in both the image domain and feature space. To regularise the anatomical structures between US and MRI during synthesis, we further propose an adversarial structural constraint. A new cross-modal attention technique is proposed to utilise non-local spatial information, by encouraging multi-modal knowledge fusion and propagation. We extend the approach to consider the case where 3D auxiliary information (e.g., 3D neighbours and a 3D location index) from volumetric data is also available, and show that this improves image synthesis. The proposed approach is evaluated quantitatively and qualitatively with comparison to real fetal MR images and other approaches to synthesis, demonstrating its feasibility of synthesising realistic MR images.

Multi-modal constraint propagation via compatible conditional distribution reconstruction

In many applications, the pairwise constraint is a kind of weaker supervisory information which can be collected more efficiently than the label information. The constraint propagation is a constrained clustering approach, which has been proved to be a success of exploiting such side-information. Recent years have witnessed many methods of multi-modal constraint propagation. However, the problem of reasonably fusing different modalities under the framework of constraint propagation remains unaddressed. In this paper, we first identify a necessary and sufficient condition for compatible conditional distributions under a specific assumption and address the problem of Compatible Conditional Distributions Reconstruction (CCDR). With the help of CCDR, we propose a multi-modal constraint propagation method dubbed Instance Level Multi-Modal Constraint Propagation (ILMCP). ILMCP fuses the affinity of different modalities at the data instance level instead of the modality level and constructs a unified affinity matrix. Extensive experiments on two publicly available multi-modal datasets show the superior performance of the proposed method.

Reconstruction of angle-resolved backscattering through a multimode fiber for cell nuclei and particle size determination

We demonstrate reconstruction of angle-resolved optical backscattering after transmission through a multimode fiber. Angle-resolved backscattering is an important tool for particle sizing, and has been developed as a diagnostic modality for detecting epithelial precancer. In this work, we fully characterized the transfer function of a multimode fiber using a plane-wave illumination basis across two dimensions. Once characterized, angle-resolved scattering information which has been scrambled by multimodal propagation can be easily and accurately reconstructed. Our technique was validated using a Mie theory-based inverse light scattering analysis (ILSA) algorithm on polystyrene microsphere phantoms of known sizes. To demonstrate the clinical potential of this approach, nuclear morphology was determined from the reconstructed angular backscattering from MCF-10A human mammary epithelial cell samples and validated against quantitative image analysis (QIA) of fluorescence microscopy images.

A general frame for uncertainty propagation under multimodally distributed random variables

Uncertainty propagation under multimodally distributed random variables, called multimodal distribution propagation for short, is a challenging problem due to the complicate probability density function of the random variables. In this paper, a general frame based on a new finite mixture model (λMM) constructed by derivative lambda probability density function and polynomial chaos expansion method is put forward to efficiently solve the multimodal distribution propagation problem. First, λMM with high accuracy and extensive applicability is proposed to represent the arbitrary unimodal distributions and multimodal distributions. Second, a pseudo EM method proved strictly is proposed to estimate the parameters of this λMM. Third, the statistical moments of the response by polynomial chaos expansion method is derived mathematically. Since the new mixture model can be decomposed into several unimodal probability distributions, the multimodal distribution propagation can be successfully converted into several unimodal distribution propagations, which further can be easily solved by polynomial chaos expansion method. Finally, the maximum entropy principle is adopted to evaluate the probability density function of the result of every unimodal distribution propagation, which is then superimposed into the final probability density function of the system response. The proposed frame only requires the first four statistical moments to evaluate the probability density function and avoid calculating the higher statistical moments, especially for the situation that the probability distribution of the system response is multimodal. Four examples are presented to verify the high accuracy and efficiency of the proposed general frame for uncertainty propagation.