Large-scale Image Research Articles

DESAT: A Distance-Enhanced Strip Attention Transformer for Remote Sensing Image Super-Resolution

Transformer-based methods have demonstrated impressive performance in image super-resolution tasks. However, when applied to large-scale Earth observation images, the existing transformers encounter two significant challenges: (1) insufficient consideration of spatial correlation between adjacent ground objects; and (2) performance bottlenecks due to the underutilization of the upsample module. To address these issues, we propose a novel distance-enhanced strip attention transformer (DESAT). The DESAT integrates distance priors, easily obtainable from remote sensing images, into the strip window self-attention mechanism to capture spatial correlations more effectively. To further enhance the transfer of deep features into high-resolution outputs, we designed an attention-enhanced upsample block, which combines the pixel shuffle layer with an attention-based upsample branch implemented through the overlapping window self-attention mechanism. Additionally, to better simulate real-world scenarios, we constructed a new cross-sensor super-resolution dataset using Gaofen-6 satellite imagery. Extensive experiments on both simulated and real-world remote sensing datasets demonstrate that the DESAT outperforms state-of-the-art models by up to 1.17 dB along with superior qualitative results. Furthermore, the DESAT achieves more competitive performance in real-world tasks, effectively balancing spatial detail reconstruction and spectral transform, making it highly suitable for practical remote sensing super-resolution applications.

Remote Assistance for Bone-Fractured Patients using Deep Learning Models

ABSTRACT Remote diagnosis enables healthcare professionals to evaluate and diagnose patients from a distance using telecommunication technologies, enhancing healthcare delivery by improving accessibility, especially for those in remote or underserved areas. One of the significant sustainability challenges in remote medical diagnostics is offering timely assistance to vulnerable groups like the elderly, disabled, mentally impaired individuals, and wounded military personnel in combat zones. This becomes particularly difficult in emergencies when rapid analysis of medical records is needed, especially if the data is stored on secure blockchain networks. The proposed work addresses these challenges by deploying a comprehensive framework for large-scale analysis, utilizing both document and image classification for dual validation. It integrates advanced techniques such as Inception V3, VGG-16, VGG-19, RESNET-50, and Densenet-201 for bone fracture detection, with Inception V3 achieving the highest accuracy of 95.1%. In addition, a Document Classification Analysis (DCA) method is proposed, which automatically classifies the severity of fractures. Object detection techniques are also introduced for detecting minor fractures using region-based image segmentation, ensuring precise diagnosis even for subtle injuries. This pioneering integration of technologies provides a holistic solution for remote medical diagnostics.

Measuring Cell Dimensions in Fission Yeast Using Machine Learning.

In fission yeast (Schizosaccharomyces pombe), cell length is a crucial indicator of cell cycle progression. Microscopy screens that examine the effect of agents or genotypes suspected of altering genomic or metabolic stability and thus cell size are crucial for studying disruptions to cell cycle dynamics. This method is based on using an automated cell segmentation algorithm to measure S. pombe cells imaged by brightfield (BF) microscopy methods. PhotoPhenosizer (PP) is a machine learning-based tool designed for automated cell measuring and dimensional analysis of morphology frequency distributions. Integration of this method into large-scale pipelines for tracking cell dimension change streamlines morphological measurements, which facilitates the examination of cellular responses to genomic and metabolic stresses. In this protocol, we use PP to observe the effect of genomic instability on cell size dynamics over a 12-day chronological lifespan assay. Our results show that relative to wild-type cells, a replication stress mutant shows larger cells during chronological aging in excess glucose media. Our results are consistent with activation of checkpoints that regulate cell morphology in response to DNA damage. This method's application highlights the relevance of its incorporation in experimental routines that require large-scale image processing and its adoption by users with routine needs in S. pombe molecular research projects.

Brain neurovascular coupling in amyotrophic lateral sclerosis: Correlations with disease progression and cognitive impairment.

'Neurovascular coupling' (NVC) alterations, assessing the interplay between local cerebral perfusion and neural activity within a given brain region or network, may reflect neurovascular unit impairment in amyotrophic lateral sclerosis (ALS). The aim was to explore NVC as a correlation between the functional connectivity and cerebral blood flow within the large-scale resting-state functional magnetic resonance imaging brain networks in a sample of ALS patients compared to healthy controls (HCs). Forty-eight ALS patients (30 males; mean age 60.64 ± 9.62 years) and 32 HC subjects (14 males; mean age 55.06 ± 16 years) were enrolled and underwent 3 T magnetic resonance imaging. ALS patients were screened by clinical and neuropsychological scales and were retrospectively classified as very fast progressors (VFPs), fast progressors and slow progressors (SPs). Neurovascular coupling reduction within the default mode network (DMN) (p = 0.005) was revealed in ALS patients compared to HCs, observing, for this network, significant NVC differences between VFP and SP groups. Receiver operating characteristic curve analysis showed that impaired NVC in the DMN at baseline best discriminated VFPs and SPs (area under the curve 75%). Significant correlations were found between NVC and the executive (r = 0.40, p = 0.01), memory (r = 0.32, p = 0.04), visuospatial ability (r = 0.40, p = 0.01) and non-ALS-specific (r = 0.40, p = 0.01) subscores of the Edinburgh Cognitive and Behavioural ALS Screen. The reduction of brain NVC in the DMN may reflect largely distributed abnormalities of the neurovascular unit. NVC alterations in the DMN could play a role in anticipating a faster clinical progression in ALS patients, aiding patient selection and monitoring during clinical trials.

Real-time non-line-of-sight computational imaging using spectrum filtering and motion compensation.

Non-line-of-sight (NLOS) imaging aims at recovering the shape and albedo of hidden objects. Despite recent advances, real-time video of complex and dynamic scenes remains a major challenge owing to the weak signal of multiply scattered light. Here we propose and demonstrate a framework of spectrum filtering and motion compensation to realize high-quality NLOS video for room-sized scenes. Spectrum filtering leverages a wave-based model for denoising and deblurring in the frequency domain, enabling computational image reconstruction with a small number of sampling points. Motion compensation tailored with an interleaved scanning scheme can compute high-resolution live video during the acquisition of low-quality image sequences. Together, we demonstrate live NLOS videos at 4 fps for a variety of dynamic real-life scenes. The results mark a substantial stride toward real-time, large-scale and low-power NLOS imaging and sensing applications.

Kinetic inductance current sensor for visible to near-infrared wavelength transition-edge sensor readout

Single-photon detectors based on the superconducting transition-edge sensor are used in a number of visible to near-infrared applications, particularly for photon-number-resolving measurements in quantum information science. To be practical for large-scale spectroscopic imaging or photonic quantum computing applications, the size of visible to near-infrared transition-edge sensor arrays and their associated readouts must be increased from a few pixels to many thousands. In this manuscript, we introduce the kinetic inductance current sensor, a scalable readout technology that exploits the nonlinear kinetic inductance in a superconducting resonator to make sensitive current measurements. Kinetic inductance current sensors can replace superconducting quantum interference devices for many applications because of their ability to measure fast, high slew-rate signals, their compatibility with standard microwave frequency-division multiplexing techniques, and their relatively simple fabrication. Here, we demonstrate the readout of a visible to near-infrared transition-edge sensor using a kinetic inductance current sensor with 3.7 MHz of bandwidth. We measure a readout noise of 1.4pA/Hz\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$1.4\\,{{{\\rm{pA}}}}/\\sqrt{{{{\\rm{Hz}}}}}$$\\end{document}, considerably below the detector noise at frequencies of interest, and an energy resolution of (0.137 ± 0.001) eV at 0.8 eV, comparable to resolutions observed with non-multiplexed superconducting quantum interference device readouts.

Semantic layout-guided diffusion model for high-fidelity image synthesis in ‘The Thousand Li of Rivers and Mountains’

“The Thousand Li of Rivers and Mountains” (TLRM) is one of the most famous traditional paintings. Even experienced artists face significant challenges in replicating TLRM. Utilizing semantic layout-guided diffusion models offers a novel approach to this task; however, synthesizing high-quality images with consistent semantics and layout is difficult due to TLRM’s unique characteristics. In this paper, we propose a novel diffusion model-based framework guided by semantic layout to generate images tailored to TLRM’s distinct features. We introduce the layout enhanced map and latent layout injection strategies, which improve semantic fidelity and color distribution. These innovations are integrated into our semantic latent diffusion model for effective semantically-guided image generation. To address the training challenges posed by a single large-scale image, we employ specialized data augmentation techniques, facilitating the generation of one-hot semantic layout representations and ensuring continuous training until convergence. Additionally, we developed the TLRM-dataset for semantic image synthesis, which enhances visual quality and semantic consistency. Experimental results and user surveys demonstrate that our model can produce high-fidelity and diversified results, offering significant potential for art education, digital preservation, and cultural heritage. Our code is publicly available at https://github.com/rane7/TLRM-SLDM.

Hollowed-Out Icon Colorization with Pre-trained Large-Scale Image Generation Model

Hollowed-Out Icon Colorization with Pre-trained Large-Scale Image Generation Model

Water-Adapter: adapting the segment anything model for surface water extraction in optical very-high-resolution remotely sensed imagery

ABSTRACT Surface water extraction (SWE) from very-high-resolution optical remote sensing images is crucial yet challenging due to the complex spectral variability of water bodies. To address this, we propose Water-Adapter, a novel method that enhances SWE by leveraging the Segment Anything Model (SAM), a large-scale image segmentation framework. Our approach introduces a task-specific input module that utilizes explicit visual prompting by focusing the tunable parameters on the visual content of each image, specifically leveraging features from frozen patch embeddings and low-frequency components. By freezing most of SAM’s image encoder parameters and incorporating a few domain-specific trainable adapters, Water-Adapter effectively integrates remote sensing knowledge into SAM, significantly improving segmentation performance with minimal computational overhead. Extensive experiments on the GLH-Water dataset demonstrate that Water-Adapter outperforms state-of-the-art methods in both accuracy and efficiency.

A Heatmap-Supplemented R-CNN Trained Using an Inflated IoU for Small Object Detection

Object detection architectures struggle to detect small objects across applications including remote sensing and autonomous vehicles. Specifically, for unmanned aerial vehicles, poor detection of small objects directly limits this technology’s applicability. Objects both appear smaller than they are in large-scale images captured in aerial imagery and are represented by reduced information in high-altitude imagery. This paper presents a new architecture, CR-CNN, which predicts independent regions of interest from two unique prediction branches within the first stage of the network: a conventional R-CNN convolutional backbone and an hourglass backbone. Utilizing two independent sources within the first stage, our approach leads to an increase in successful predictions of regions that contain smaller objects. Anchor-based methods such as R-CNNs also utilize less than half the number of small objects compared to larger ones during training due to the poor intersection over union (IoU) scores between the generated anchors and the groundtruth—further reducing their performance on small objects. Therefore, we also propose artificially inflating the IoU of smaller objects during training using a simple, size-based Gaussian multiplier—leading to an increase in the quantity of small objects seen per training cycle based on an increase in the number of anchor–object pairs during training. This architecture and training strategy led to improved detection overall on two challenging aerial-based datasets heavily composed of small objects while predicting fewer false positives compared to Mask R-CNN. These results suggest that while new and unique architectures will continue to play a part in advancing the field of object detection, the training methodologies and strategies used will also play a valuable role.

Imperial Study on Convolutional Layer in CNN for Data Mining

Convolutional Neural Networks (CNNs) have become a cornerstone in the field of data mining, particularly for tasks involving large-scale image and pattern recognition. This paper presents an imperial study on the efficacy of convolutional layers within CNN architectures in data mining applications. We analyze the impact of varying convolutional layer configurations, such as kernel size, stride, and depth, on performance metrics including accuracy, computational cost, and feature extraction quality. By conducting a series of experiments across different datasets, we explore how these configurations influence the model's ability to generalize and detect complex patterns in structured and unstructured data. Our findings indicate that optimizing the convolutional layers significantly improves the efficiency of CNNs for data mining tasks, offering insights into best practices for architectural design in such applications. This study provides valuable guidelines for leveraging CNNs in the realm of data mining, pushing the boundaries of automated data analysis and knowledge discovery

GMDIC: a digital image correlation measurement method based on global matching for large deformation displacement fields

The digital image correlation method is a non-contact optical measurement method, which has the advantages of full-field measurement, simple operation, and high measurement accuracy. The traditional DIC method can accurately measure displacement and strain fields, but there are still many limitations. (i) In the measurement of large displacement deformations, the calculation accuracy of the displacement field and strain field needs to be improved due to the unreasonable setting of parameters such as subset size and step size. (ii) It is difficult to avoid under-matching or over-matching when reconstructing smooth displacement or strain fields. (iii) When processing large-scale image data, the computational complexity will be very high, resulting in slow processing speeds. In recent years, deep-learning-based DIC has shown promising capabilities in addressing the aforementioned issues. We propose a new, to the best of our knowledge, DIC method based on deep learning, which is designed for measuring displacement fields of speckle images in complex large deformations. The network combines the multi-head attention Swin-Transformer and the high-efficient channel attention module ECA and adds positional information to the features to enhance feature representation capabilities. To train the model, we constructed a displacement field dataset that conformed to the real situation and contained various types of speckle images and complex deformations. The measurement results indicate that our model achieves consistent displacement prediction accuracy with traditional DIC methods in practical experiments. Moreover, our model outperforms traditional DIC methods in cases of large displacement scenarios.

CLAP. I. Resolving miscalibration for deep learning-based galaxy photometric redshift estimation

Obtaining well-calibrated photometric redshift probability densities for galaxies without a spectroscopic measurement remains a challenge. Deep learning discriminative models, typically fed with multi-band galaxy images, can produce outputs that mimic probability densities and achieve state-of-the-art accuracy. However, several previous studies have found that such models may be affected by miscalibration, an issue that would result in discrepancies between the model outputs and the actual distributions of true redshifts. Our work develops a novel method called the C ontrastive L earning and A daptive KNN for P hotometric Redshift (CLAP) that resolves this issue. It leverages supervised contrastive learning (SCL) and $k$-nearest neighbours (KNN) to construct and calibrate raw probability density estimates, and implements a refitting procedure to resume end-to-end discriminative models ready to produce final estimates for large-scale imaging data, bypassing the intensive computation required for KNN. The harmonic mean is adopted to combine an ensemble of estimates from multiple realisations for improving accuracy. Our experiments demonstrate that CLAP takes advantage of both deep learning and KNN, outperforming benchmark methods on the calibration of probability density estimates and retaining high accuracy and computational efficiency. With reference to CLAP, a deeper investigation on miscalibration for conventional deep learning is presented. We point out that miscalibration is particularly sensitive to the method-induced excessive correlations among data instances in addition to the unaccounted-for epistemic uncertainties. Reducing the uncertainties may not guarantee the removal of miscalibration due to the presence of such excessive correlations, yet this is a problem for conventional methods rather than CLAP. These discussions underscore the robustness of CLAP for obtaining photometric redshift probability densities required by astrophysical and cosmological applications. This is the first paper in our series on CLAP.

The aortic paradox: a nationwide analysis of 523,994 individual echocardiograms exploring aortic dissection

Abstract Background Increasing aortic dilation increases risk of aortic dissection. Nevertheless, dissection occurs below guideline-directed cut-offs for prophylactic surgery. There is no current large-scale population imaging data assessing aortic dimensions before dissection. Purpose To examine the proportion of aortic dissections occurring in people with aortic diameters below guidelines for prophylactic intervention. Methods Patients within the National Echo Database of Australia (NEDA) were stratified according to absolute, height-indexed and body surface area (BSA)-indexed aortic dimensions. Fatal thoracic aortic dissections (ICD-10-AM code I79) were identified via linkage with the National Death Index. Results 524,994 individuals were assessed, comprising patients with normal aortic dimensions (n=460,992), mild dilation (n=53,402), moderate dilation (n=10,029) and severe dilation (n=572). 274,992 (52.4%) were male, with median age 64 years and median follow-up time 6.9 years. 899 fatal aortic dissections occurred (normal diameter=610, mildly-dilated aorta=215, moderately-dilated =53 and severely-dilated=21). Using normal aortas as the reference population, odds of fatal dissection increased with aortic diameter (mild=OR 3.05, 95% confidence interval (CI) 2.61-3.56; moderate=OR 4.0, 95% CI 3.02-5.30; severe=OR 28.72, 95% CI 18.44-44.72). Due to the much larger number of patients without severe aortic dilation, 97.7% of fatal aortic dissections occurred in non-severely dilated aortas. Following sensitivity analysis, severe aortic dilation was responsible for at most 24.4% of fatal aortic dissections. Results were robust for absolute, height-indexed or BSA-indexed aortic measurements. Conclusion Although severe aortic dilatation is associated with a thirty-fold increase in fatal dissection, severely dilated aortas are implicated in only 2.3-24.4% of fatal dissections. This highlights the ‘aortic paradox’ and limitations of current guidelines. Future studies should seek to refine risk predictors in patients without severe aortic dilation.

Artificial Intelligence and Linguistic Landscape research

Abstract This article explores applications of artificial intelligence (AI) technologies in Linguistic Landscape research. Traditionally, LL research has relied on manual data collection and analysis, often involving photographs of public signage, advertisements, and other visual language displays. However, this manual approach can present challenges, including time-consuming data collection, inconsistent data quality, and potential researcher bias. Two AI technologies in particular hold promise for addressing these challenges in LL research: computer vision (CV) and large language models (LLMs). CV automates the identification and extraction of text from images, improving data accuracy and enabling large-scale image analysis. LLMs, based on natural language processing, can detect, translate, and interpret multilingual text. This article explores the affordances and challenges of using AI technologies in LL research and discusses methods to improve data collection, enhance accuracy, and support the analysis of multilingual environments. It also raises ethical issues and limitations of the technologies.

CellSAM: Advancing Pathologic Image Cell Segmentation via Asymmetric Large-Scale Vision Model Feature Distillation Aggregation Network.

Segment anything model (SAM) has attracted extensive interest as a potent large-scale image segmentation model, with prior efforts adapting it for use in medical imaging. However, the precise segmentation of cell nucleus instances remains a formidable challenge in computational pathology, given substantial morphological variations and the dense clustering of nuclei with unclear boundaries. This study presents an innovative cell segmentation algorithm named CellSAM. CellSAM has the potential to improve the effectiveness and precision of disease identification and therapy planning. As a variant of SAM, CellSAM integrates dual-image encoders and employs techniques such as knowledge distillation and mask fusion. This innovative model exhibits promising capabilities in capturing intricate cell structures and ensuring adaptability in resource-constrained scenarios. The experimental results indicate that this structure effectively enhances the quality and precision of cell segmentation. Remarkably, CellSAM demonstrates outstanding results even with minimal training data. In the evaluation of particular cell segmentation tasks, extensive comparative analyzes show that CellSAM outperforms both general fundamental models and state-of-the-art (SOTA) task-specific models. Comprehensive evaluation metrics yield scores of 0.884, 0.876, and 0.768 for mean accuracy, recall, and precision respectively. Extensive experiments show that CellSAM excels in capturing subtle details and complex structures and is capable of segmenting cells in images accurately. Additionally, CellSAM demonstrates excellent performance on clinical data, indicating its potential for robust applications in treatment planning and disease diagnosis, thereby further improving the efficiency of computer-aided medicine.

Efficient labeling of french mammogram reports with MammoBERT

Recent advances in deep learning and natural language processing (NLP) have broadened opportunities for automatic text processing in the medical field. However, the development of models for low-resource languages like French is challenged by limited datasets, often due to legal restrictions. Large-scale training of medical imaging models often requires extracting labels from radiology text reports. Current methods for report labeling primarily rely on sophisticated feature engineering based on medical domain knowledge or manual annotations by radiologists. These methods can be labor-intensive. In this work, we introduce a BERT-based approach for the efficient labeling of French mammogram image reports. Our method leverages both the expansive scale of existing rule-based systems and the precision of radiologist annotations. Our experimental results showcase the superiority of the proposed approach. It was initially fine-tuned on a limited dataset of radiologist annotations. Then, it underwent training on annotations generated by a rule-based labeler. Our findings reveal that our final model, MammoBERT, significantly outperforms the rule-based labeler while simultaneously reducing the necessity for radiologist annotations during training. This research not only advances the state of the art in medical image report labeling but also offers an efficient and effective solution for large-scale medical imaging model development.

Post-earthquake inspection of high-speed railway viaducts with multi-scale task interaction deep learning strategy

Automated computer vision-based inspections of railway infrastructures, such as component type, damaged status, and location, have been investigated actively by resorting to task-specific deep learning models. However, task-specific models that fulfill these separate inspection tasks encountered bottlenecks in improving inference accuracy, and bring huge computational costs. Multi-task deep learning, which can fulfill these inspection tasks concurrently, has yet to be fully investigated in the context of structural inspection. In this study, a multi-scale task interaction deep learning strategy is presented towards component recognition, damage segmentation, and depth estimation for a comprehensive post-earthquake inspection of high-speed railway viaducts. Three modules for multi-scale task interaction were proposed to modify the multi-task deep neural network, taking full advantage of task commonalities at multiple scales. The proposed method was validated with a large-scale image dataset of high-speed railway viaducts. Component recognition and depth estimation were incorporated to implement multi-task learning since they have higher pattern affinities at multiple scales. Results reported that mean Intersection over Unions of testing samples of component and damage tasks were 91.2% and 72.1%, RMSE of depth estimation was 1.54 m. Compared with single-task cases, training time, inference duration, and FLOPs of the multi-task model were reduced by 23%, 30%, 27% respectively. Results showed improvement in both inference accuracy and training efficiency, substantiating the superiority of the proposed strategy.

Linearly programmable two-dimensional halide perovskite memristor arrays for neuromorphic computing.

The exotic properties of three-dimensional halide perovskites, such as mixed ionic-electronic conductivity and feasible ion migration, have enabled them to challenge traditional memristive materials. However, the poor moisture stability and difficulty in controlling ion transport due to their polycrystalline nature have hindered their use as a neuromorphic hardware. Recently, two-dimensional (2D) halide perovskites have emerged as promising artificial synapses owing to their phase versatility, microstructural anisotropy in electrical and optoelectronic properties, and excellent moisture resistance. However, their asymmetrical and nonlinear conductance changes still limit the efficiency of training and accuracy of inference. Here we achieve highly linear and symmetrical conductance changes in Dion-Jacobson 2D perovskites. We further build a 7 × 7 crossbar array based on analogue perovskite synapses, achieving a high device yield, low variation with synaptic weight storing capability, multi-level analogue states with long retention, and moisture stability over 7 months. We explore the potential of such devices in large-scale image inference via simulations and show an accuracy within 0.08% of the theoretical limit. The excellent device performance is attributed to the elimination of gaps between inorganic layers, allowing the halide vacancies to migrate homogeneously regardless of grain boundaries. This was confirmed by first-principles calculations and experimental analysis.

Quantitative Analysis of Acquisition Speed of High-Precision FLIM Technologies via Simulation and Modeling

In practical applications such as cancer diagnosis and industrial detection, there is a critical demand for fast fluorescence lifetime imaging (Fast-FLIM). The Fast-FLIM systems suitable for complex environments are typically achieved by enhancing the hardware performance of time-correlated single-photon counting (TCSPC), with an acquisition speed of about a few frames per second (fps). However, due to the limitation of single-photon acquisition, the imaging speed is still far from the demand of practical application. The synchroscan streak camera (SC) maps signals from the temporal dimension to the spatial dimension, effectively overcoming the long acquisition time caused by single-photon acquisition. This paper constructs a method to calculate the acquisition time for the TCSPC-FLIM and SC-FLIM systems, and it quantitatively compares the speed. The research demonstrates that the main factors limiting the acquisition speed of the FLIM systems are the photon emission rate, the photon counting rate, the required SNR, the dwell time, and the number of parallel channels. In high-quality and large-scale lifetime imaging, the acquisition speed of the SC-FLIM is at least 104 times faster than that of the TCSPC-FLIM. Therefore, the synchroscan streak camera has more significant potential to promote Fast-FLIM.