Rich Detail Research Articles

HyperGAN: A Hyperspectral Image Fusion Approach Based on Generative Adversarial Networks

The objective of hyperspectral pansharpening is to fuse low-resolution hyperspectral images (LR-HSI) with corresponding panchromatic (PAN) images to generate high-resolution hyperspectral images (HR-HSI). Despite advancements in hyperspectral (HS) pansharpening using deep learning, the rich spectral details and large data volume of HS images place higher demands on models for effective spectral extraction and processing. In this paper, we present HyperGAN, a hyperspectral image fusion approach based on Generative Adversarial Networks. Unlike previous methods that deepen the network to capture spectral information, HyperGAN widens the structure with a Wide Block for multi-scale learning, effectively capturing global and local details from upsampled HSI and PAN images. While LR-HSI provides rich spectral data, PAN images offer spatial information. We introduce the Efficient Spatial and Channel Attention Module (ESCA) to integrate these features and add an energy-based discriminator to enhance model performance by learning directly from the Ground Truth (GT), improving fused image quality. We validated our method on various scenes, including the Pavia Center, Eastern Tianshan, and Chikusei. Results show that HyperGAN outperforms state-of-the-art methods in visual and quantitative evaluations.

Physical prior-guided deep fusion network with shading cues for shape from polarization

Shape from polarization (SfP) is a powerful passive three-dimensional imaging technique that enables the reconstruction of surface normal with dense textural details. However, existing deep learning-based SfP methods only focus on the polarization prior, which makes it difficult to accurately reconstruct targets with rich texture details under complicated scenes. Aiming to improve the reconstruction accuracy, we utilize the surface normal estimated from shading cues and the innovatively proposed specular confidence as shading prior to provide additional feature information. Furthermore, to efficiently combine the polarization and shading priors, a novel deep fusion network named SfPSNet is proposed for the information extraction and the reconstruction of surface normal. SfPSNet is implemented based on a dual-branch architecture to handle different physical priors. A feature correction module is specifically designed to mutually rectify the defects in channel-wise and spatial-wise dimensions, respectively. In addition, a feature fusion module is proposed to fuse the feature maps of polarization and shading priors based on an efficient cross-attention mechanism. Our experimental results show that the fusion of polarization and shading priors can significantly improve the reconstruction quality of surface normal, especially for objects or scenes illuminated by complex lighting sources. As a result, SfPSNet shows state-of-the-art performance compared with existing deep learning-based SfP methods benefiting from its efficiency in extracting and fusing information from different priors.

Transformer- and joint learning-based dual-domain networks for undersampled MRI segmentation.

Recently, magnetic resonance imaging (MRI) has become a crucial medical imaging technology widely used in clinical practice. However, MRI faces challenges such as the lengthy acquisition time for k-space data and the need for time-consuming manual annotation by radiologists. Traditionally, these challenges have been addressed individually through undersampled MRI reconstruction and automatic segmentation algorithms. Whether undersampled MRI segmentation can be enhanced by treating undersampled MRI reconstruction and segmentation as an end-to-end task, trained simultaneously, rather than as serial tasks should beexplored. We introduce a novel Transformer- and Joint Learning-based Dual-domain Network (TJLD-Net) for undersampled MRIsegmentation. This method significantly enhances feature recognition in the segmentation process by fully utilizing the rich detail obtained during the image reconstruction phase. Consequently, the method can achieve precise and reliable image segmentation even with undersampled k-space data. Additionally, it incorporates an attention mechanism for feature enhancement, which improves the representation of shared features by learning the contextual information in MRimages. Simulation experiments demonstrate that the segmentation performance of TJLD-Net on three datasets is significantly higher than that of the joint model (RecSeg) and six baseline models (where reconstruction and segmentation are regarded as serial tasks). On the CHAOS dataset, the Dice scores of TJLD-Net are, on average, 9.87%, 2.17%, 1.90%, 1.80%, 9.60%, 0.80%, and 6.50% higher than those of the seven compared models. On the ATLAS challenge dataset, the average Dice scores of TJLD-Net improve by 4.23%, 5.63%, 2.30%, 1.53%, 3.57%, 0.93%, and 6.60%. Similarly, on the SKM-TEA dataset, the average Dice scores of TJLD-Net improve by 4.73%, 12.80%, 14.83%, 8.67%, 4.53%, 11.60%, and 12.10%. The novel TJLD-Net model provides a promising solution for undersampled MRI segmentation, overcoming the poor performance issues encountered by automated segmentation algorithms in low-quality acceleratedimaging.

Feature fusion and context interaction for RGB-D indoor semantic segmentation

Depth information in indoor scenes is crucial to understanding the spatial and occluding relationships of objects. Depth information contains rich geometric details and is unaffected by lighting, texture, and color, significantly enhancing traditional color images. RGB images and depth images have been widely applied in various image analysis. However, effectively leveraging the complementary information from both modalities remains a challenge. To address this issue, we propose a unified and effective feature fusion and context interaction network (FCINet). It includes three modules. Firstly, we construct a multi-modal feature fusion module (MFFM), which can achieve the aggregation of two modalities along spatial and channel dimensions. Secondly, we construct a global and local information interaction context module (GLCM) to encode rich semantic information. Finally, we construct a feature alignment and fusion module (FAFM), which integrates upsampled low-level features with high-level features to mitigate spatial information loss. Experimental results indicate that the model achieved favorable outcomes on both the NYU Depth v2 dataset and the more complex SUN RGB-D dataset.

Precomputed low-frequency lighting in cinematic volume rendering.

Cinematic Rendering (CR) employs physical models such as ray tracing and global illumination to simulate real-world light phenomena, producing high-quality images with rich details. In the medical field, CR can significantly aid doctors in accurate diagnosis and preoperative planning. However, doctors require efficient real-time rendering when using CR, which presents a challenge due to the substantial computing resources demanded by CR's ray tracing and global illumination models. Precomputed lighting can enhance the efficiency of real-time rendering by freezing certain scene variables. Typically, precomputed methods freeze geometry and materials. However, since the physical rendering of medical images relies on volume data rendering of transfer functions, the CR algorithm cannot utilize precomputed methods directly. To improve the rendering efficiency of the CR algorithm, we propose a precomputed low-frequency lighting method. By simulating the lighting pattern of shadowless surgical lamps, we adopt a spherical distribution of multiple light sources, with each source capable of illuminating the entire volume of data. Under the influence of these large-area multi-light sources, the precomputed lighting adheres to physical principles, resulting in shadow-free and uniformly distributed illumination. We integrated this precomputed method into the ray-casting algorithm, creating an accelerated CR algorithm that achieves more than twice the rendering efficiency of traditional CR rendering.

Indigenous Spirituality, Health, and Well-Being in the Young: Yarns With the Victorian Aboriginal Community.

The extant literature has scant detail about everyday spiritual practices that aid Indigenous young people. This paper systematically explores Indigenous Spirituality, health, and well-being through Elder-governed yarns conducted via Zoom with 44 Aboriginal Elders, Healers, and Senior and Junior people involved in health and well-being of the Victorian Aboriginal community. These yarns were analyzed through an innovative, constructivist, multi-perspectival discursive grounded theory method. Key findings are that Spirituality is crucial for health and well-being, leading to a clear mind and at-peace "center" in a person. Aboriginal spiritual practices reflect the unique characteristics and essential rhythms of Country. Spiritual development is incremental and increases the obligations and responsibilities a person has to community and Country and leads to increased caring for Country. This paper provides rich detail about practical spiritual techniques to aid Indigenous young people and their kinship networks. It has the potential to shape future policy.

A multimodal learning machine framework for Alzheimer’s disease diagnosis based on neuropsychological and neuroimaging data

Alzheimer’s disease (AD) is the most prevalent form of dementia, with no current cure. Early screening and intervention are vital. In multimodal AD data, besides neuroimaging dimensions, neuropsychological tests based on cognitive domains also provide the clinical information for diagnosing AD. However, previous multimodal methods often fuse these neuropsychological test scores with other data, losing the rich clinical details inherent in each test, including videos, speech, images, and text. To address this, we propose a novel framework with an entropy-based polynomial dimension expansion function that restores this critical information by accurately calculating the optimal polynomial degree. Additionally, the proposed framework offers a series of cognitive-based Extreme Learning Machine (ELM) models to better utilize the detailed clinical insights from neuropsychological tests, reducing diagnostic redundancy and noise. Finally, we design a boosting ensemble strategy that combines diagnostic models from various dimensions and cognitive domains, automatically optimizing weights to enhance diagnostic accuracy. Tested on the Alzheimer’s Disease Neuroimaging Initiative (ADNI) dataset, our approach achieves over 98% accuracy and F1 scores, with no observed bias between mild cognitive impairment (MCI) and AD groups. Therefore, our framework can offer clinicians more logical recommendations for diagnosing and managing the disease.

Two-stage deep image restoration network with application to single image shadow removal

In this paper, we introduce a two-stage deep learning-based image restoration network and its application to remove shadow information from a single image, named by ESCNet (Encoder-decoder based Shadow removal with Colorization Network). Most existed single image-based shadow removal methods may suffer from that the shadow contains multiple regions of different colors or rich image details. To tackle with the problems, our key idea is to first remove shadow(s) from an image followed by repainting the shadow-removed region(s) in this image. To accomplish this, we present a deep two-stage network, cascading a shadow removal network (SRN) and a colorization network (CN). The presented encoder-decoder-based SRN with fusion of global and local feature information is used to remove the shadow(s) in the grayscale domain of the input image while recovering the image details for the shadow-removed region(s). Then the proposed CN aims at repainting the removed shadow region(s) via re-colorization. The proposed deep model has been well trained and well evaluated on the two well-known public datasets, i.e., ISTD (Image Shadow Triplets Dataset) and SRD (Shadow Removal Dataset). Experimental results have shown that the proposed method outperforms the compared state-of-the-art (SOTA) shadow removal approaches quantitatively and qualitatively.

Overview of High-Dynamic-Range Image Quality Assessment.

In recent years, the High-Dynamic-Range (HDR) image has gained widespread popularity across various domains, such as the security, multimedia, and biomedical fields, owing to its ability to deliver an authentic visual experience. However, the extensive dynamic range and rich detail in HDR images present challenges in assessing their quality. Therefore, current efforts involve constructing subjective databases and proposing objective quality assessment metrics to achieve an efficient HDR Image Quality Assessment (IQA). Recognizing the absence of a systematic overview of these approaches, this paper provides a comprehensive survey of both subjective and objective HDR IQA methods. Specifically, we review 7 subjective HDR IQA databases and 12 objective HDR IQA metrics. In addition, we conduct a statistical analysis of 9 IQA algorithms, incorporating 3 perceptual mapping functions. Our findings highlight two main areas for improvement. Firstly, the size and diversity of HDR IQA subjective databases should be significantly increased, encompassing a broader range of distortion types. Secondly, objective quality assessment algorithms need to identify more generalizable perceptual mapping approaches and feature extraction methods to enhance their robustness and applicability. Furthermore, this paper aims to serve as a valuable resource for researchers by discussing the limitations of current methodologies and potential research directions in the future.

Ship detection in reefs and deep-sea with medium-high resolution images

ABSTRACT Accurate and efficient ship detection is crucial for ocean monitoring and management, especially in reefs and deep-sea, where fishing and illegal activities threaten sustainability of ecosystems. Obtaining the size of ships in reefs and deep-sea helps to identify ship types and to assess the impact of ships on marine ecosystems quantitatively. Ship detections were mainly applied at coast or river of the inland region using high spatial resolution remote sensing data due to its rich details and frequent coverage. However, the ships located in reefs and deep-sea regions were rarely studied because of price, and availability of high spatial resolution remote sensing data. Global covered medium-high resolution (10–30 m) remote sensing data (such as Sentinel-2) makes it possible to obtain the spatial-temporal distribution of ships, especially small ships. This study developed a deep learning ship detection algorithm – an enhanced Rotated-Ship Detector (RShipDet) to detect ships in reefs and deep-sea regions. RShipDet was applied to Nansha Islands based on two medium-high resolution ship detection datasets (Sentinel2-Ship and SDGSAT-Ship). These two datasets include various backgrounds of reefs and deep-sea, and complex scenarios such as cloud cover and parallel ships, adapting RShipDet to ship detection under complex circumstances in reefs and deep-sea. The results showed that: (1) Small gains: gains of 3.3% and 8.7% Average-Precision (AP) compared to Rotation-equivariant Detector (ReDet) and Faster-RCNN on Sentinel2-Ship dataset; (2) Strong generalization capabilities: 77.4% AP on SDGSAT-Ship dataset; (3) Better performance under complex conditions: RShipDet obtained more accurate ship detection results over regions with cloud cover, islands and reefs, deep-sea, and ports compared to other classical detectors. Our algorithm could be applied for better management of ocean resources and activities in reefs and deep-sea.

Thermodynamic efficiency of membrane separation of dilute gas: Estimation for CO2 direct air capture application

Gas separation technology is crucial for addressing environmental issues like CO2 capture to mitigate climate change. While membrane separation is often cited for its efficiency, accurate estimations are scarce. We present estimations based on classical thermodynamics for very lean CO2 composition (400 ppm), revealing rich details in simple systems and deriving guiding principles. Our main conclusion emphasizes the critical necessity of a high membrane separation ratio, and we discuss candidates for achieving this goal.

Explainable fashion compatibility Prediction: An Attribute-Augmented neural framework

Grasping complementary relationships between fashion product pairings is gaining increasing attention in the e-commerce field. Current methods primarily utilize visual cues to assess compatibility, which, despite their efficacy, often lack sufficient explainability. Meanwhile, the rich semantic details embedded in product attributes remain largely unexplored. To tackle this, we propose a novel framework called Explainable Attribute-augmented Neural framework (EAN), which integrates comprehensive attribute and visual data, enabling explainability in fashion product compatibility modeling. We conduct quantitative and qualitative experiments to demonstrate the effectiveness and explainability of our proposed framework. The practical significance of our research is twofold. Firstly, it helps consumers understand the underlying reasons for fashion item pairings, thereby assisting them in refining their dressing combinations. Secondly, it provides novel perspectives for product design and assists e-commerce platforms in creating more effective product marketing combinations.

Illumination-aware divide-and-conquer network for improperly-exposed image enhancement

Improperly-exposed images often have unsatisfactory visual characteristics like inadequate illumination, low contrast, and the loss of small structures and details. The mapping relationship from an improperly-exposed condition to a well-exposed one may vary significantly due to the presence of multiple exposure conditions. Consequently, the enhancement methods that do not pay specific attention to this issue tend to yield inconsistent results when applied to the same scene under different exposure conditions. In order to obtain consistent enhancement results for various exposures while restoring rich details, we propose an illumination-aware divide-and-conquer network (IDNet). Specifically, to address the challenge of directly learning a sophisticated nonlinear mapping from an improperly-exposed condition to a well-exposed one, we utilize the discrete wavelet transform (DWT) to decompose the image into the low-frequency (LF) component, which primarily captures brightness and contrast, and the high-frequency (HF) components that depict fine-scale structures. To mitigate the inconsistency in correction across various exposures, we extract a conditional feature from the input that represents illumination-related global information. This feature is then utilized to modulate the dynamic convolution weights, enabling precise correction of the LF component. Furthermore, as the co-located positions of LF and HF components are highly correlated, we create a mask to distill useful knowledge from the corrected LF component, and integrate it into the HF component to support the restoration of fine-scale details. Extensive experimental results demonstrate that the proposed IDNet is superior to several state-of-the-art enhancement methods on two datasets with multiple exposures.

MEEAFusion: Multi-Scale Edge Enhancement and Joint Attention Mechanism Based Infrared and Visible Image Fusion.

Infrared and visible image fusion can integrate rich edge details and salient infrared targets, resulting in high-quality images suitable for advanced tasks. However, most available algorithms struggle to fully extract detailed features and overlook the interaction of complementary features across different modal images during the feature fusion process. To address this gap, this study presents a novel fusion method based on multi-scale edge enhancement and a joint attention mechanism (MEEAFusion). Initially, convolution kernels of varying scales were utilized to obtain shallow features with multiple receptive fields unique to the source image. Subsequently, a multi-scale gradient residual block (MGRB) was developed to capture the high-level semantic information and low-level edge texture information of the image, enhancing the representation of fine-grained features. Then, the complementary feature between infrared and visible images was defined, and a cross-transfer attention fusion block (CAFB) was devised with joint spatial attention and channel attention to refine the critical supplemental information. This allowed the network to obtain fused features that were rich in both common and complementary information, thus realizing feature interaction and pre-fusion. Lastly, the features were reconstructed to obtain the fused image. Extensive experiments on three benchmark datasets demonstrated that the MEEAFusion proposed in this research has considerable strengths in terms of rich texture details, significant infrared targets, and distinct edge contours, and it achieves superior fusion performance.

A Semi-Supervised Lie Detection Algorithm Based on Integrating Multiple Speech Emotional Features

When people tell lies, they often exhibit tension and emotional fluctuations, reflecting a complex psychological state. However, the scarcity of labeled data in datasets and the complexity of deception information pose significant challenges in extracting effective lie features, which severely restrict the accuracy of lie detection systems. To address this, this paper proposes a semi-supervised lie detection algorithm based on integrating multiple speech emotional features. Firstly, Long Short-Term Memory (LSTM) and Auto Encoder (AE) network process log Mel spectrogram features and acoustic statistical features, respectively, to capture the contextual links between similar features. Secondly, the joint attention model is used to learn the complementary relationship among different features to obtain feature representations with richer details. Lastly, the model combines the unsupervised loss Local Maximum Mean Discrepancy (LMMD) and supervised loss Jefferys multi-loss optimization to enhance the classification performance. Experimental results show that the algorithm proposed in this paper achieves better performance.

High-Precision Dichotomous Image Segmentation With Frequency and Scale Awareness.

Dichotomous image segmentation (DIS) with rich fine-grained details within a single image is a challenging task. Despite the plausible results achieved by deep learning-based methods, most of them fail to segment generic objects when the boundary is cluttered with the background. In fact, the gradual decrease in feature map resolution during the encoding stage and the misleading texture clue may be the main issues. To handle these issues, we devise a novel frequency-and scale-aware deep neural network (FSANet) for high-precision DIS. The core of our proposed FSANet is twofold. First, a multimodality fusion (MF) module that integrates the information in spatial and frequency domains is adopted to enhance the representation capability of image features. Second, a collaborative scale fusion module (CSFM) which deviates from the traditional serial structures is introduced to maintain high resolution during the entire feature encoding stage. In the decoder side, we introduce hierarchical context fusion (HCF) and selective feature fusion (SFF) modules to infer the segmentation results from the output features of the CSFM module. We conduct extensive experiments on several benchmark datasets and compare our proposed method with existing state-of-the-art (SOTA) methods. The experimental results demonstrate that our FSANet achieves superior performance both qualitatively and quantitatively. The code will be made available at https://github.com/chasecjg/FSANet.

Infrared and Harsh Light Visible Image Fusion Using an Environmental Light Perception Network.

The complementary combination of emphasizing target objects in infrared images and rich texture details in visible images can effectively enhance the information entropy of fused images, thereby providing substantial assistance for downstream composite high-level vision tasks, such as nighttime vehicle intelligent driving. However, mainstream fusion algorithms lack specific research on the contradiction between the low information entropy and high pixel intensity of visible images under harsh light nighttime road environments. As a result, fusion algorithms that perform well in normal conditions can only produce low information entropy fusion images similar to the information distribution of visible images under harsh light interference. In response to these problems, we designed an image fusion network resilient to harsh light environment interference, incorporating entropy and information theory principles to enhance robustness and information retention. Specifically, an edge feature extraction module was designed to extract key edge features of salient targets to optimize fusion information entropy. Additionally, a harsh light environment aware (HLEA) module was proposed to avoid the decrease in fusion image quality caused by the contradiction between low information entropy and high pixel intensity based on the information distribution characteristics of harsh light visible images. Finally, an edge-guided hierarchical fusion (EGHF) module was designed to achieve robust feature fusion, minimizing irrelevant noise entropy and maximizing useful information entropy. Extensive experiments demonstrate that, compared to other advanced algorithms, the method proposed fusion results contain more useful information and have significant advantages in high-level vision tasks under harsh nighttime lighting conditions.

An Effective and Lightweight Full-Scale Target Detection Network for UAV Images Based on Deformable Convolutions and Multi-Scale Contextual Feature Optimization

Currently, target detection on unmanned aerial vehicle (UAV) images is a research hotspot. Due to the significant scale variability of targets and the interference of complex backgrounds, current target detection models face challenges when applied to UAV images. To address these issues, we designed an effective and lightweight full-scale target detection network, FSTD-Net. The design of FSTD-Net is based on three principal aspects. Firstly, to optimize the extracted target features at different scales while minimizing background noise and sparse feature representations, a multi-scale contextual information extraction module (MSCIEM) is developed. The multi-scale information extraction module (MSIEM) in MSCIEM can better capture multi-scale features, and the contextual information extraction module (CIEM) in MSCIEM is designed to capture long-range contextual information. Secondly, to better adapt to various target shapes at different scales in UAV images, we propose the feature extraction module fitting different shapes (FEMFDS), based on deformable convolutions. Finally, considering low-level features contain rich details, a low-level feature enhancement branch (LLFEB) is designed. The experiments demonstrate that, compared to the second-best model, the proposed FSTD-Net achieves improvements of 3.8%, 2.4%, and 2.0% in AP50, AP, and AP75 on the VisDrone2019, respectively. Additionally, FSTD-Net achieves enhancements of 3.4%, 1.7%, and 1% on the UAVDT dataset. Our proposed FSTD-Net has better detection performance compared to state-of-the-art detection models. The experimental results indicate the effectiveness of the FSTD-Net for target detection in UAV images.

A General Image Super-Resolution Reconstruction Technique for Walnut Object Detection Model

Object detection models are commonly used in yield estimation processes in intelligent walnut production. The accuracy of these models in capturing walnut features largely depends on the quality of the input images. Without changing the existing image acquisition devices, this study proposes a super-resolution reconstruction module for drone-acquired walnut images, named Walnut-SR, to enhance the detailed features of walnut fruits in images, thereby improving the detection accuracy of the object detection model. In Walnut-SR, a deep feature extraction backbone network called MDAARB (multilevel depth adaptive attention residual block) is designed to capture multiscale information through multilevel channel connections. Additionally, Walnut-SR incorporates an RRDB (residual-in-residual dense block) branch, enabling the module to focus on important feature information and reconstruct images with rich details. Finally, the CBAM (convolutional block attention module) attention mechanism is integrated into the shallow feature extraction residual branch to mitigate noise in shallow features. In 2× and 4× reconstruction experiments, objective evaluation results show that the PSNR and SSIM for 2× and 4× reconstruction reached 24.66 dB and 0.8031, and 19.26 dB and 0.4991, respectively. Subjective evaluation results indicate that Walnut-SR can reconstruct images with richer detail information and clearer texture features. Comparative experimental results of the integrated Walnut-SR module show significant improvements in mAP50 and mAP50:95 for object detection models compared to detection results using the original low-resolution images.

Speaker-independent speech inversion for recovery of velopharyngeal port constriction degreea).

For most of his illustrious career, Ken Stevens focused on examining and documenting the rich detail about vocal tract changes available to listeners underlying the acoustic signal of speech. Current approaches to speech inversion take advantage of this rich detail to recover information about articulatory movement. Our previous speech inversion work focused on movements of the tongue and lips, for which "ground truth" is readily available. In this study, we describe acquisition and validation of ground-truth articulatory data about velopharyngeal port constriction, using both the well-established measure of nasometry plus a novel technique-high-speed nasopharyngoscopy. Nasometry measures the acoustic output of the nasal and oral cavities to derive the measure nasalance. High-speed nasopharyngoscopy captures images of the nasopharyngeal region and can resolve velar motion during speech. By comparing simultaneously collected data from both acquisition modalities, we show that nasalance is a sufficiently sensitive measure to use as ground truth for our speech inversion system. Further, a speech inversion system trained on nasalance can recover known patterns of velopharyngeal port constriction shown by American English speakers. Our findings match well with Stevens' own studies of the acoustics of nasal consonants.