Large-scale Image Research Articles

Distributed training of CosPlace for large-scale visual place recognition.

Visual place recognition (VPR) is a popular computer vision task aimed at recognizing the geographic location of a visual query, usually within a tolerance of a few meters. Modern approaches address VPR from an image retrieval standpoint using a kNN on top of embeddings extracted by a deep neural network from both the query and images in a database. Although most of these approaches rely on contrastive learning, which limits their ability to be trained on large-scale datasets (due to mining), the recently reported CosPlace proposes an alternative training paradigm using a classification task as the proxy. This has been shown to be effective in expanding the potential of VPR models to learn from large-scale and fine-grained datasets. In this work, we experimentally analyze CosPlace from a continual learning perspective and show that its sequential training procedure leads to suboptimal results. As a solution, we propose a different formulation that not only solves the pitfalls of the original training strategy effectively but also enables faster and more efficient distributed training. Finally, we discuss the open challenges in further speeding up large-scale image retrieval for VPR.

Research On the Application of Improved Models Based on Senet in Large-Scale Image Retrieval Algorithms

Research On the Application of Improved Models Based on Senet in Large-Scale Image Retrieval Algorithms

BIFROST: a method for registering diverse imaging datasets of the Drosophila brain.

Large-scale functional imaging experiments in Drosophila have given us new insights into neural activity in various sensory and behavioral contexts. However, precisely registering volumetric images from different studies has proven challenging, limiting quantitative comparisons of data across experiments. Here, we address this limitation by developing BIFROST, a registration pipeline robust to differences across experimental setups and datasets. We benchmark this pipeline by genetically labeling cell types in the fly brain and demonstrate sub-10 micron registration precision, both across specimens and across laboratories. We further demonstrate accurate registration between in-vivo brain volumes and ultrastructural connectomes, enabling direct structure-function comparisons in future experiments.

Addressing incomplete tile phenomena in image tiling: Introducing the grid six-intersection model

Abstract With the continuous development of Earth Observation technology, resolution of imagery and gridded data has significantly increased, leading to a rapid increase in data volume. To efficiently acquire and analyze these vast amounts of imagery and gridded data, image tiling technology has been developed to effectively access data of interested areas. Tiling technology divides large-scale image data into smaller tiles, providing fast, accurate, and efficient access support for imagery and gridded data. The spatial grid model, as the foundational framework of the new generation of geographic spatial information, plays a critical role in the retrieval, integration, services, and applications of imagery and gridded data resources. When tiling image data based on the spatial grid model, it always generates both complete and incomplete tiles. Particularly, when conducting image tile retrieval using the same rule-based grid in the retrieval area, incomplete tile phenomena along the boundary regions of the retrieved images often occur, resulting in gaps within the retrieval area. To tackle this issue, in this study, we present a new topological model called the Grid Six-Intersection Model (G-6IM), specifically designed for regular rectangular grids, to accurately represent boundary issues in image tiling. Through a practical case study, we demonstrate the effectiveness and practical application potential of the proposed G-6IM model, providing new insights and guidance for the improvement and optimization of imagery and gridded data tiling technology.

Efficient Feature Matching for Large-scale Images based on Cascade Hash and Local Geometric Constraint

Abstract. Feature matching plays a crucial role in 3D reconstruction to provide correspondences between overlapped images. The accuracy and efficiency of feature matching significantly impact the performance of 3D reconstruction. The widely used framework with the exhaustive nearest neighbor searching (NNS) between descriptors and RANSAC-based geometric estimation is, however, low-efficient and unreliable for large-scale UAV images. Inspired by indexing-based NNS, this paper implements an efficient feature matching method for large-scale images based on Cascade Hashing and local geometric constraints. Our proposed method improves upon traditional feature matching approaches by introducing a combination of image retrieval, data scheduling, and GPU-accelerated Cascade Hashing. Besides, it utilizes a local geometric constraint to filter matching results within a matching framework. On the one hand, the GPU-accelerated Cascade Hashing technique generates compact and discriminative hash codes based on image features, facilitating the rapid completion of the initial matching process, and significantly reducing the search space and time complexity. On the other hand, after the initial matching is completed, the method employs a local geometric constraint to filter the initial matching results, enhancing the accuracy of the matching results. This forms a three-tier framework based on data scheduling, GPU-accelerated Cascade Hashing, and local geometric constraints. We conducted experiments using two sets of large-scale UAV image data, comparing our method with SIFTGPU to evaluate its performance in initial matching, outlier rejection, and 3D reconstruction. The results demonstrate that our method achieves a feature matching speed 2.0 times that of SIFTGPU while maintaining matching accuracy and producing comparable reconstruction results. This suggests that our method holds promise for efficiently addressing large-scale image matching.

Coastal Tidal Flat And Tidal Current Observation Based On Satellite Remote Sensing

Satellite remote sensing has been an effective way to acquire surface image sequences of costal field, and the satellite video at a frame rate of multiple frames per second could provide chance for short-time observations at high temporal resolution such as the observation of surface tidal currents. The observation of surface tidal currents has been facilitated by advanced image preprocessing methods and large-scale image velocimetry, while the space-time volume velocimetry (STVV) was used in this study. STVV can simultaneously obtain the direction and magnitude of the flow velocity through the surface texture without any artificial tracers, hence it is considered as a promising method. Since it has only been proposed in recent years, this study evaluated the accuracy of the STVV algorithm and tested the robustness of the STVV to environmental disturbance factors firstly. The simulated image sequences were used so that the exact accuracy of STVV can be known by comparing the measured and ground truth values since the ground truth of the pixel flow velocity is predictable and modifiable. The effects of the two main STVV parameters, the size of the search area and the selected frames of images, on the results were analyzed, and the robustness against image noise and fog was examined. The size of the search area and selected frames of images were found having the similar effects on the calculation results that the larger both are, the higher the calculation accuracy will be. Although the lower Signal-to-noise ratios lead to more inaccurate results, STVV is robust to image noises from the results in the given cases. Additionally, the accuracy of STVV is acceptable for a small amount of mist, but thicker fog results in large errors. Further research to develop image processing methods to improve the accuracy of STVV in the future is highly necessary. Therefore, based on one satellite video with the frame rate of 10fps and the resolution of 0.95m shooting on September 15th, 2017, surface tidal current observation of the New York Harbor during the flood tide using STVV was achieved. The flow velocity magnitude can reach 3m/s, and the flow direction is consistent with the actual situation. The feasibility of surface tidal current observation using STVV from the satellite video was proved

Non-telecentric photoacoustic microscopy for multi-scale imaging.

Optical-resolution photoacoustic microscopy (OR-PAM) excels in precisely imaging a biological tissue based on absorption contrast. However, existing OR-PAMs are confined by fixed compromises between spatial resolution and field of view (FOV), preventing the integration of large FOV and local high-resolution within one system. Here, we present a non-telecentric OR-PAM (nTC-PAM) that empowers efficient adaptation of FOV and spatial resolution to match the multi-scale requirement of diverse biological imaging. Our method allows for a large-scale transformation in FOV and even surpassing the nominal FOV of the objective with minimal marginal degradation of the lateral resolution. We demonstrate the advantage of nTC-PAM through multi-scale imaging of the leaf phantom, mouse ear, and cortex. The results reveal that nTC-PAM can switch the FOV and spatial resolution to meet the requirements of different biological tissues, such as large-scale imaging of the whole cerebral cortex and high-resolution imaging of microvascular structures in local brain regions.

Implementing Inception v3, VGG-16 and VGG-19 Architectures of CNN for Medicinal Plant leaves Identification and Disease Detection

Medicinal plants have been a valuable source of healing and healthcare for centuries, serving as a crucial component of traditional medicine systems worldwide. The identification of these plants is a fundamental step in harnessing their therapeutic potential and ensuring their sustainable use. Accurate identification of these plants and the early detection of diseases affecting their leaves are crucial for ensuring a consistent supply of high-quality medicinal resources. In this paper, three types of CNN architectures of deep learning are used to develop robust classification models for distinguishing between healthy and diseased medicinal plant leaves. Inception v3 is known for its versatility in handling various input image sizes without the need for extensive pre-processing, while VGG-19 exhibits high accuracy, robust feature extraction, and suitability for complex image patterns. VGG-16, known for its simplicity and ease of training, provided competitive results, particularly where computational resources were limited. These networks have been pre-trained on large-scale image datasets and fine-tuned using them. These different approaches provide a varied solution to this problem by comparing their accuracy levels, feasibility and to select which one suits the best in this study.

Early diagnosis of greenhouse cucumber downy mildew in seedling stage using chlorophyll fluorescence imaging technology

Cucumber downy mildew, caused by Pseudoperonospora cubensis, typically remains hidden from view during its initial infection stage. To address this issue, this study proposed an early diagnosis model for greenhouse cucumber downy mildew. This study was conducted under controlled conditions, utilising a large-scale mobile chlorophyll fluorescence imaging system to monitor the samples during their seedling stage on a daily basis from the first day of inoculation. A total of 98 sets of fluorescence parameter values and corresponding fluorescence images were collected. Machine learning methods, such as recursive feature elimination (RFE) and Least Absolute Shrinkage and Selection Operator (LASSO) regression with L1 regularisation, were used to screen the chlorophyll fluorescence parameter Ft_D3, whose corresponding the chlorophyll fluorescence images were used as inputs to the proposed model, following by employing a convolutional neural network (CNN) transfer learning method to the early detection task of cucumber downy mildew in fluorescence images. The study improved the topology structure of ResNet50, the network model with a learning rate of 0.001 and 16 cycles as the optimal feature extractor. The results indicated that the enhanced network displayed improved performance in early detection of cucumber downy mildew compared with other CNNs. Infected leaves were distinguished from healthy leaves in the early stages of infection, specifically 3 days before the appearance of symptoms. The accuracy of the model in the task of early diagnosis of downy mildew was 94.76%. This study presents an efficient approach for the photosynthetic characterisation and early identification of cucumber downy mildew.

Assessing Face Image Quality: A Large-Scale Database and a Transformer Method.

The amount of face images has been witnessing an explosive increase in the last decade, where various distortions inevitably exist on transmitted or stored face images. The distortions lead to visible and undesirable degradation on face images, affecting their quality of experience (QoE). To address this issue, this paper proposes a novel Transformer-based method for quality assessment on face images (named as TransFQA). Specifically, we first establish a large-scale face image quality assessment (FIQA) database, which includes 42,125 face images with diversifying content at different distortion types. Through an extensive crowdsource study, we obtain 712,808 subjective scores, which to the best of our knowledge contribute to the largest database for assessing the quality of face images. Furthermore, by investigating the established database, we comprehensively analyze the impacts of distortion types and facial components (FCs) on the overall image quality. Accordingly, we propose the TransFQA method, in which the FC-guided Transformer network (FT-Net) is developed to integrate the global context, face region and FC detailed features via a new progressive attention mechanism. Then, a distortion-specific prediction network (DP-Net) is designed to weight different distortions and accurately predict final quality scores. Finally, the experiments comprehensively verify that our TransFQA method significantly outperforms other state-of-the-art methods for quality assessment on face images.

PRESERVING HUMAN LARGE-SCALE BRAIN CONNECTIVITY FINGERPRINT IDENTIFIABILITY WITH RANDOM PROJECTIONS.

The complex etiology of various neurodegenerative diseases and psychiatric disorders, especially at the individual level, has posed unmatched challenges to the advancement of personalized medicine. Recent technical advancements in functional magnetic resonance imaging has enabled researchers to map brain large-scale connectivity at an unprecedented level of subject precision. Nonetheless, along with the early dawn of promises in personalized medicine using various neuroimaging modalities rose the challenge of clinical utility of brain connectomics (e.g., functional connectomes). Besides many established challenges of functional connectome utility such as edge reliability, there exists an easily overlooked challenge that does not get the same level of attention: computationality of functional connectome. To improve clinical utility of functional connectomics, we propose a random projection method that would preserve a practically similar level of subject identifiability while sampling and retaining only a proportion of functional edges in subjects' functional connectome. Our work pave a way towards computational improvements, hence clinical utility, of functional connectomes while not compromising the integrity of biomarkers learnt from whole-brain large-scale functional connectivity imaging modality.

Dual-attention-transformer-based semantic reranking for large-scale image localization

Dual-attention-transformer-based semantic reranking for large-scale image localization

Dynamic range issues and image information content when visualizing seismic data

The purpose of this review is to discuss the problem of seismic data visualization in relation to human vision capabilities based on the seismic signal dynamic range assessment. The ways of increasing the information content and efficiency of visualization methods in terms of increasing their resolution are considered. The conducted analysis has shown that the dynamic range of seismic recordings exceeds the capabilities of the human eye by 2–3 orders of magnitude and the capabilities of technical visualization means by 4–5 orders of magnitude. Color vision models are considered on example of the most versatile hardware additive RGB color model. The model was chosen based on the idea of the human vision system as a set of photosensitive cones that register red, green and blue colors and the human brain, which processes and forms a color image in one’s consciousness. The relationships between monochrome and color images perceived by the human eye based on several types of sensitive receptors have been established. It is concluded that the use of a color image significantly expands information capabilities. Thus, 8 grey steps are enough for the monochrome method of information output, whereas the use of color devices allows to increase the number of steps up to 14. The methodology for solving this problem consists in the application of new technologies based on the use of large-scale and three-dimensional images providing a significant increase in the values of luminous flux, resolution and contrast based on the expansion of the dynamic range taking into consideration the limits of human vision.

Efficient preconditioned least-squares wave-equation migration

Since the appearance of wave-equation migration (WEM), many have tried to improve the resolution and effectiveness of this technology. Least-squares wave-equation migration is one of those attempts that try to fill the gap between migration assumptions and reality in an iterative manner. However, these iterations do not come cheap. A proven solution to limit the number of least-squares iterations is to correct the gradient direction within each iteration via the action of a preconditioner that approximates the inverse Hessian. However, the Hessian computation, or even the Hessian approximation computation, in large-scale seismic imaging problems involves an expensive computational bottleneck, making it unfeasible. Therefore, we develop an efficient computation of the Hessian approximation operator in the context of one-way WEM in the space-frequency domain. We build the Hessian approximation operator depth by depth, considerably reducing the operator size each time it is calculated. We prove the validity of our method with two numerical examples. We then extend our proposal to the framework of full-wavefield migration, which is based on WEM principles but includes interbed multiples. Finally, this efficient preconditioned least-squares full-wavefield migration is successfully applied to a data set with strong interbed multiple scattering.

Automated void detection in high resolution x-ray printed circuit boards (PCBs) images with deep segmentation neural network

Defect detection and quantification on printed circuit board (PCB) assemblies are crucial for ensuring the quality of electronic products. X-ray inspection is commonly used to detect internal defects, in which high-resolution gray-scale images are captured to reveal the internal structure, particularly the defect features, of an assembled PCB. To detect defect features, conventional computer vision techniques require hand-crafted feature filters and usually have a narrow application scope. This study introduces an automated method for detecting voids using supervised machine learning. The method employs end-to-end segmentation models to identify pixels that belong to the void and integrated circuit (IC) regions in X-ray PCB images. The segmentation results are then used to compute the percentage of voids within each IC, which is a key measure of the quality of the inspected PCB. Accurate predictions with an intersection over union of over 0.95 have achieved. A challenge in this development is the large size of the images, which can reach to millions of pixels, and the small scale of defect features. Training a deep segmentation model for large-scale images using conventional methods requires a lot of memory. Downscaling high-resolution images result in the loss of defect features. In this study, a unique training method utilizing the local nature of the defects is proposed, which reduces memory consumption and speeds up the training process. Additionally, to overcome the limited training data, various augmentation methods are applied and transfer learning is employed to construct segmentation models for different types of PCBs.

Cortex-wide transcranial localization microscopy with fluorescently labeled red blood cells

Large-scale imaging of brain activity with high spatio-temporal resolution is crucial for advancing our understanding of brain function. The existing neuroimaging techniques are largely limited by restricted field of view, slow imaging speed, or otherwise do not have the adequate spatial resolution to capture brain activities on a capillary and cellular level. To address these limitations, we introduce fluorescence localization microscopy aided with sparsely-labeled red blood cells for cortex-wide morphological and functional cerebral angiography with 4.9 µm spatial resolution and 1 s temporal resolution. When combined with fluorescence calcium imaging, the proposed method enables extended recordings of stimulus-evoked neuro-vascular changes in the murine brain while providing simultaneous multiparametric readings of intracellular neuronal activity, blood flow velocity/direction/volume, and vessel diameter. Owing to its simplicity and versatility, the proposed approach will become an invaluable tool for deciphering the regulation of cortical microcirculation and neurovascular coupling in health and disease.

Experimental investigations on the effects of longitudinal groove fabric for drag reductions on a track cyclist

The previous study has shown that the longitudinal groove fabric can reduce drag forces on a circular cylinder by forcing a drag crisis (Zheng et al. 2021). In this study, the effects of the longitudinal groove fabric are investigated on a full-scale track cycling mannequin. The force measurement results show that the longitudinal groove fabric on the upper arms can achieve a maximum drag reduction of about 7% at a flow speed of 17 m/s, and its control effects depend on flow speeds. Large-scale particle image velocimetry measurements further show that the drag reductions on the upper arm are characterized by diminished streamwise velocity deficits. The control effects also vary on different spanwise locations of the arm, where the flow behaves distinctively. The measurements also reveal the distinct flow dynamics at different heights, i.e., wake interactions and swirling motions, showing the complexity of reducing drag forces from a track cyclist.

Improving cross-site generalisability of vision-based solar forecasting models with physics-informed transfer learning

Forecasting solar energy from cloud cover observations is crucial to truly anticipate future changes in power supply. On an intra-hour timescale, ground-level sky cameras located near a solar site offer the most valuable source of information on incoming clouds. In the literature, the analysis of these hyperlocal cloud cover observations for solar modelling is increasingly performed by deep learning algorithms trained and tested on years’ worth of local data. However, this approach is not suitable for industrial applications since solar energy producers cannot wait for years of local data collection to start generating reliable solar forecasts. However, they might own relevant multi-location data collected from other solar sites over time. This study thus explores the capability of such algorithms to generalise beyond their training location in two data scarce conditions: zero-shot learning (i.e. direct application of a trained model to a new location without local fine-tuning) and few-shot learning (i.e. calibration of a pre-trained model based on very limited local data such as a day of observations). Zero-shot learning results show that using local clear-sky models to normalise output variables (e.g. solar irradiance or solar energy production values) facilitates cross-dataset transfer learning. Compared to previous methods, the resulting forecast skill increases by close to 25% in cloudy conditions and by more than 700% in clear-sky conditions. An additional gain is observed when local data collected in overcast weather conditions are used for model calibration via few-shot learning. The corresponding neural networks trained in data scarce conditions achieve comparable performance to expert local models based on years of training data. These promising results shed light on the potential of large-scale and multi-location sky image datasets to improve the generalisation skills of solar forecasting algorithms.

Smart grading of diabetic retinopathy: an intelligent recommendation-based fine-tuned EfficientNetB0 framework.

Diabetic retinopathy is a condition that affects the retina and causes vision loss due to blood vessel destruction. The retina is the layer of the eye responsible for visual processing and nerve signaling. Diabetic retinopathy causes vision loss, floaters, and sometimes blindness; however, it often shows no warning signals in the early stages. Deep learning-based techniques have emerged as viable options for automated illness classification as large-scale medical imaging datasets have become more widely available. To adapt to medical image analysis tasks, transfer learning makes use of pre-trained models to extract high-level characteristics from natural images. In this research, an intelligent recommendation-based fine-tuned EfficientNetB0 model has been proposed for quick and precise assessment for the diagnosis of diabetic retinopathy from fundus images, which will help ophthalmologists in early diagnosis and detection. The proposed EfficientNetB0 model is compared with three transfer learning-based models, namely, ResNet152, VGG16, and DenseNet169. The experimental work is carried out using publicly available datasets from Kaggle consisting of 3,200 fundus images. Out of all the transfer learning models, the EfficientNetB0 model has outperformed with an accuracy of 0.91, followed by DenseNet169 with an accuracy of 0.90. In comparison to other approaches, the proposed intelligent recommendation-based fine-tuned EfficientNetB0 approach delivers state-of-the-art performance on the accuracy, recall, precision, and F1-score criteria. The system aims to assist ophthalmologists in early detection, potentially alleviating the burden on healthcare units.

System-level time computation and representation in the suprachiasmatic nucleus revealed by large-scale calcium imaging and machine learning

The suprachiasmatic nucleus (SCN) is the mammalian central circadian pacemaker with heterogeneous neurons acting in concert while each neuron harbors a self-sustained molecular clockwork. Nevertheless, how system-level SCN signals encode time of the day remains enigmatic. Here we show that population-level Ca2+ signals predict hourly time, via a group decision-making mechanism coupled with a spatially modular time feature representation in the SCN. Specifically, we developed a high-speed dual-view two-photon microscope for volumetric Ca2+ imaging of up to 9000 GABAergic neurons in adult SCN slices, and leveraged machine learning methods to capture emergent properties from multiscale Ca2+ signals as a whole. We achieved hourly time prediction by polling random cohorts of SCN neurons, reaching 99.0% accuracy at a cohort size of 900. Further, we revealed that functional neuron subtypes identified by contrastive learning tend to aggregate separately in the SCN space, giving rise to bilaterally symmetrical ripple-like modular patterns. Individual modules represent distinctive time features, such that a module-specifically learned time predictor can also accurately decode hourly time from random polling of the same module. These findings open a new paradigm in deciphering the design principle of the biological clock at the system level.