Search Image Research Articles

The Dark Energy Survey Supernova Program: Light Curves and 5 Yr Data Release

We present griz photometric light curves for the full 5 yr of the Dark Energy Survey Supernova (DES-SN) program, obtained with both forced point-spread function photometry on difference images (DiffImg) performed during survey operations, and scene modelling photometry (SMP) on search images processed after the survey. This release contains 31,636 DiffImg and 19,706 high-quality SMP light curves, the latter of which contain 1635 photometrically classified SNe that pass cosmology quality cuts. This sample spans the largest redshift (z) range ever covered by a single SN survey (0.1 < z < 1.13) and is the largest single sample from a single instrument of SNe ever used for cosmological constraints. We describe in detail the improvements made to obtain the final DES-SN photometry and provide a comparison to what was used in the 3 yr DES-SN spectroscopically confirmed Type Ia SN sample. We also include a comparative analysis of the performance of the SMP photometry with respect to the real-time DiffImg forced photometry and find that SMP photometry is more precise, more accurate, and less sensitive to the host-galaxy surface brightness anomaly. The public release of the light curves and ancillary data can be found at github.com/des-science/DES-SN5YR and doi:10.5281/zenodo.12720777.

A novel deformation measurement method for rotating blade based on PSO-ILS image correlation matching and mismatch correction

Abstract Aiming at the problem of small measurement range and difficult measurement of traditional contact sensor method in the rotating state of aero-engine blades, this paper proposes a novel deformation measurement method for rotating blade based on image correlation matching and mismatch correction. Firstly, a Particle Swarm Optimization-Iterative Local Search image intelligent matching algorithm is proposed, which effectively balances global search and local optimization, and the image matching displacement accuracy reaches 10−3 pixel. Secondly, a mismatch point detection method based on multi-scale local Root Mean Square is proposed, and the high-precision detection of mismatch points is realized by considering the influence of local structure. Finally, through the hierarchical refinement of the sub-pixel level mismatch point correction method, the mismatch point is corrected in the two iterative stages of global search and local optimization. The corrected mismatch point response value K is reduced by 99% compared with that before correction, which further improves the accuracy of deformation field calculation. In the experiment, the deformation of the rotating blade of 4500 RPM is measured, which proves that the image intelligent matching algorithm and the deformation field calculation method proposed in this paper can provide new methods and technical support for the accurate measurement of the blade surface deformation.

Experiments Reveal That Search Image Might Be Responsible for Seasonal Variation in Nest-Predation Rates

Experiments Reveal That Search Image Might Be Responsible for Seasonal Variation in Nest-Predation Rates

Совершенствование медико-социальных мер по охране психического здоровья подростков в системе первичной медико-санитарной помощи (обзор литературы).

Actuality. To develop effective tools for protecting the mental health of adolescents, medical workers need to clarify the leading reasons for its formation in modern conditions, predictors and criteria for the risk of its deterioration. Currently, there are no integrated medicalpsychological- social monitoring studies of adolescent mental health, including dynamic monitoring of psychological, pedagogical, social and medical indicative indicators. The purpose of the study is to review scientific publications devoted to the analysis of the state of mental health of children and adolescents in the Russian Federation. Research methods. Using the search image: («mental health of children and adolescents», «organization of psychiatric care for children and adolescents», «addictive behavior of children and adolescents», «risk factors for mental health of children and adolescents»), a collection of 63 relevant articles published by domestic and foreign authors over the past 15 years (2013–2023) have been formed, indexed in the bibliometric databases WoS, Scopus, RSCI, and in the search engines PubMed, Google Scholar. Research results. The performed analysis allows us to note that such issues as the manifestation of clinical mental disorders, their predictors with age-related detail, and the development of integral indicators of mental health deterioration have not been sufficiently studied. Studies indicating the presence of problems in the system of mental health protection of minors during their studies in general education organizations do not provide specific recommendations for their solution.. Conclusion. To develop effective tools for the protection of adolescent mental health, medical professionals need to clarify the leading causes of its formation in modern conditions, predictors and criteria for the risk of its deterioration. This will make it possible to fill the existing deficit in the algorithms of medical support for adolescents during their studies in general education organizations and medical, psychological and pedagogical integration.

Highly compact adaptive network based on transformer for RGBT tracking

RGBT tracking is a challenging task that requires robust fusion of visible images (RGB) and thermal infrared images (TIR) to handle various scenarios, such as illumination changes, occlusions, and camouflage. The current popular RGBT trackers mostly based on two stream Siamese trackers. They tend to separately extracts template and search images region features, which neglects relationship between target and background. Moreover, they do not fuse RGB and TIR features properly, limiting their ability to utilize complementary features. To address these issues, we introduce a highly compact adaptive transformer-based network that unifies the process of feature extracting and correlation between template and search images. In this way, the compact network has dual-branches. They can combine feature extraction and correlation for different modalities of template images and search images. Meanwhile, we introduce a cross-modal weight redistribution module (CMWR) for multi-modal fusion. This adaptive fusion scheme learns discriminative features of RGB and TIR data and assigns weights to them, enabling them to complement each other. Furthermore, to address the issue of tracking targets of different scales, we design scale-adaptive optimization pyramid module (SAOP) that adapt to objects of different sizes. Our method achieves exceptional performance on the GTOT, RGBT234 and LasHeR datasets, surpassing most of the existing methods. The results are consistent across multiple datasets, demonstrating the effectiveness and superiority of our approach. And our code is released at: https://github.com/ELOESZHANG/HCANet.

Color polymorphism and mating trends in a population of the alpine leaf beetle Oreina gloriosa.

The bright colors of Alpine leaf beetles (Coleoptera, Chrysomelidae) are thought to act as aposematic signals against predation. Within the European Alps, at least six species display a basal color of either blue or green, likely configuring a classic case of müllerian mimicry. In this context, intra-population color polymorphism is paradoxical as the existence of numerous color morphs might hamper the establishment of a search image in visual predators. Assortative mating may be one of the main factors contributing to the maintenance of polymorphic populations. Due to the marked iridescence of these leaf beetles, the perceived color may change as the viewing or illumination angle changes. The present study, conducted over three years, involved intensive sampling of a population of Oreina gloriosa from the Italian Alps and applied colorimetry and a decision tree method to identify the color morphs in an objective manner. The tertiary sex ratio of the population was biased in favor of males, suggesting that viviparous females hide to give birth. Seven color morphs were identified, and their frequencies varied significantly over the course of the study. Three different analyses of mating (JMating, QInfomating, and Montecarlo simulations) recognized a general trend for random mating which coexists with some instances of positive and negative assortative mating. This could help explain the pre-eminence of one morph (which would be favored because of positive selection due to positive assortative mating) in parallel with the persistence of six other morphs (maintained due to negative assortative mating).

SEER: Backdoor Detection for Vision-Language Models through Searching Target Text and Image Trigger Jointly

This paper proposes SEER, a novel backdoor detection algorithm for vision-language models, addressing the gap in the literature on multi-modal backdoor detection. While backdoor detection in single-modal models has been well studied, the investigation of such defenses in multi-modal models remains limited. Existing backdoor defense mechanisms cannot be directly applied to multi-modal settings due to their increased complexity and search space explosion. In this paper, we propose to detect backdoors in vision-language models by jointly searching image triggers and malicious target texts in feature space shared by vision and language modalities. Our extensive experiments demonstrate that SEER can achieve over 92% detection rate on backdoor detection in vision-language models in various settings without accessing training data or knowledge of downstream tasks.

Sequential Fusion Based Multi-Granularity Consistency for Space-Time Transformer Tracking

Regarded as a template-matching task for a long time, visual object tracking has witnessed significant progress in space-wise exploration. However, since tracking is performed on videos with substantial time-wise information, it is important to simultaneously mine the temporal contexts which have not yet been deeply explored. Previous supervised works mostly consider template reform as the breakthrough point, but they are often limited by additional computational burdens or the quality of chosen templates. To address this issue, we propose a Space-Time Consistent Transformer Tracker (STCFormer), which uses a sequential fusion framework with multi-granularity consistency constraints to learn spatiotemporal context information. We design a sequential fusion framework that recombines template and search images based on tracking results from chronological frames, fusing updated tracking states in training. To further overcome the over-reliance on the fixed template without increasing computational complexity, we design three space-time consistent constraints: Label Consistency Loss (LCL) for label-level consistency, Attention Consistency Loss (ACL) for patch-level ROI consistency, and Semantic Consistency Loss (SCL) for feature-level semantic consistency. Specifically, in ACL and SCL, the label information is used to constrain the attention and feature consistency of the target and the background, respectively, to avoid mutual interference. Extensive experiments have shown that our STCFormer outperforms many of the best-performing trackers on several popular benchmarks.

DPT‐tracker: Dual pooling transformer for efficient visual tracking

AbstractTransformer tracking always takes paired template and search images as encoder input and conduct feature extraction and target‐search feature correlation by self and/or cross attention operations, thus the model complexity will grow quadratically with the number of input images. To alleviate the burden of this tracking paradigm and facilitate practical deployment of Transformer‐based trackers, we propose a dual pooling transformer tracking framework, dubbed as DPT, which consists of three components: a simple yet efficient spatiotemporal attention model (SAM), a mutual correlation pooling Transformer (MCPT) and a multiscale aggregation pooling Transformer (MAPT). SAM is designed to gracefully aggregates temporal dynamics and spatial appearance information of multi‐frame templates along space‐time dimensions. MCPT aims to capture multi‐scale pooled and correlated contextual features, which is followed by MAPT that aggregates multi‐scale features into a unified feature representation for tracking prediction. DPT tracker achieves AUC score of 69.5 on LaSOT and precision score of 82.8 on TrackingNet while maintaining a shorter sequence length of attention tokens, fewer parameters and FLOPs compared to existing state‐of‐the‐art (SOTA) Transformer tracking methods. Extensive experiments demonstrate that DPT tracker yields a strong real‐time tracking baseline with a good trade‐off between tracking performance and inference efficiency.

SiamRAAN: Siamese Residual Attentional Aggregation Network for Visual Object Tracking

The Siamese network-based tracker calculates object templates and search images independently, and the template features are not updated online when performing object tracking. Adapting to interference scenarios with performance-guaranteed tracking accuracy when background clutter, illumination variation or partial occlusion occurs in the search area is a challenging task. To effectively address the issue with the abovementioned interference and to improve location accuracy, this paper devises a Siamese residual attentional aggregation network framework for self-adaptive feature implicit updating. First, SiamRAAN introduces Self-RAAN into the backbone network by applying residual self-attention to extract effective objective features. Then, we introduce Cross-RAAN to update the template features online by focusing on the high-relevance parts in the feature extraction process of both the object template and search image. Finally, a multilevel feature fusion module is introduced to fuse the RAAN-enhanced feature information and improve the network’s ability to perceive key features. Extensive experiments conducted on benchmark datasets (GOT-10K, LaSOT, OTB-50, OTB-100 and UAV123) demonstrated that our SiamRAAN delivers excellent performance and runs at 51 FPS in various challenging object tracking tasks. Code is available at https://github.com/MallowYi/SiamRAAN.

Habitat heterogeneity limits prey colour polymorphism maintained via negative frequency-dependent selection.

The persistence of non-neutral trait polymorphism is enigmatic because stabilizing selection is expected to deplete variation. In cryptically coloured prey, negative frequency-dependent selection due to search image formation by predators has been proposed to favour rare variants, promoting polymorphism. However, in a heterogeneous environment, locally varying disruptive selection favours patch type-specific optima, resulting in spatial segregation of colour variants. Here, we address whether negative frequency-dependent selection can overcome selection posed by habitat heterogeneity to promote local polymorphism using an individual-based model. In addition, we compare how prey and predator mobility may modify the outcome. Our model revealed that frequency-dependent predation could strongly promote local prey polymorphism, but only when differences between morphs in patch-specific fitness were small. The effect of frequency-dependent predation depended on the predator adjustment of search image and was hampered by the prey population structure. Gene flow due to prey movement counteracted local selection, promoted local polymorphism to some extent, and relaxed the conditions for polymorphism due to frequency-dependent predation. Importantly, abrupt spatial changes in morph frequencies decreased the probability that mobile frequency-dependent predators could maintain local prey polymorphism. Overall, our study suggests that in a spatially heterogeneous environment, negative frequency-dependent selection may help maintain local polymorphism but only under a limited range of conditions.

Human Detection From Unmanned Aerial Vehicles’ Images for Search and Rescue Missions: A State-of-the-Art Review

Human Detection From Unmanned Aerial Vehicles’ Images for Search and Rescue Missions: A State-of-the-Art Review

Statistical similarity matching and filtering for clinical image retrieval by machine learning approach

Clinical image retrieval plays a pivotal role in modern healthcare for diagnostics and research, but prior research has grappled with the challenge of achieving high accuracy due to limited filtering techniques. The proposed method includes statistical distance measurements for similarity comparison and a machine learning technique for image filtering. Throughout this framework, the search area for similarity matching is reduced by first filtering away irrelevant images using the probabilistic outcomes of the Support Vector Machine (SVM) classification as class predictions of search and database images. Resizing is done as part of the preprocessing. Then, using Principal Component Analysis (PCA), the preprocessed data’s textural features, visual characteristics, and low-level features are extracted. The study also suggested an adaptive similarity matching method centered on a linear integration of feature-level similarities on the individual-level level. The precision and ranking order details of the most appropriate images retrieved and predicted by SVMs are considered when calculating the feature weights. The system continually alters weights for every distinctive search to generate beneficial outcomes. The supervised and unsupervised learning strategies are studied to link low-level global image features in the generated PCA-based Eigen Space using their high-level semantic and visual classifications to reduce the semantic gap and enhance retrieval effectiveness. The ground-truth database used in experiments has 1594 unique medical images with 3 different databases. Our method significantly improves the precision and recall rates in image retrieval tasks by combining sophisticated feature extraction, data-driven algorithms, and deep learning models. Research obtained an impressive accuracy of 0.99, demonstrating the effectiveness of our approach. This novel methodology addresses the limitations of prior research and provides a robust and reliable solution for clinicians and researchers in the medical field seeking to access and analyze relevant clinical images.

Mutually exclusive feeding yields Holling type III functional response

Abstract Many generalist predators, including fish, insects and mammals, temporarily focus on their search behaviour and can switch between prey species. Flexible and rapid behavioural plasticity enables a predator to forage optimally, as guided by search image formation and prey switching in response to changing prey abundances. More than half a century of studies have used Holling type III functional response to model the impact of this feeding behaviour on population dynamics, yet a mechanistic understanding for the pathway between individual behaviour and feeding rates remains poorly understood. To understand this common feeding behaviour, we developed three mechanistic models using first‐principles to provide direct derivations for the relationship between observed individual behaviour and feeding rates. The derived models yield three novel, generalised functional responses for predators characterised by prey switching and mutually exclusive feeding (i.e. feeding is limited to one prey species at a time). We show how these functional responses act as Holling type III response, and how they can be used to predict predators' diet compositions. Our results demonstrate that several behavioural forms of prey switching, as displayed in a wide range of predator species, consistently produce type III conforming functional responses. Read the free Plain Language Summary for this article on the Journal blog.

Exploring reliable infrared object tracking with spatio-temporal fusion transformer

Currently, most thermal infrared (TIR) trackers rely on feature matching between the search image and a fixed template cropped from the first frame. Some Siam-based TIR trackers with a template update mechanism introduce historical prediction information in the temporal dimension through correlation filters. However, their feature characterization capability is inadequate to resist target scale variations, appearance changes, and occlusion. To address this challenge, we explore a novel spatio-temporal fusion Transformer (STFT) model to realize robust TIR object tracking. Our approach involves a Transformer-based encoder–decoder that fuses spatio-temporal information. Specifically, we design a dynamic template update strategy based on salient points feature(SPF) representation, which allows the model to leverage the most powerful spatio-temporal information by retrieving multiple salient points on the target image. To further fortify the dynamic template update strategy, we propose an IoU-Aware target state estimation head that utilizes the joint representation of target classification and localization. An IoU-Aware criterion is developed for quality estimation of the dynamic template. The proposed STFT-Net approach has been put to the evaluation on three challenging benchmarks, with extensive experimental results showcasing its superior performance in contrast to acclaimed tracking algorithms. The code is available at https://github.com/qinxin-wh/STFT-Net.

RETRACTED ARTICLE: Dual Siamese Anchor Points Adaptive Tracker with Transformer for RGBT Tracking

Due to environmental conditions, such as rainy days, foggy days, and dim lighting, objects in visible light images are not prominently displayed, leading to an easy loss of targets during tracking. In recent years, many RGB visible light trackers have achieved significant success in addressing visual tracking challenges. However, these trackers perform poorly when tracking targets under special conditions, such as occlusions and low-light scenarios. In contrast, objects in thermal infrared images are more distinct in poor lighting conditions. Given this characteristic, researchers have shown increased interest in the development of trackers that combine thermal infrared and visible light imagery. However, some mainstream RGBT (red–green–blue and thermal) algorithms, such as MANET and ADNET, are based on the anchor-based theory, requiring consideration of anchor box sizes and introducing a substantial number of hyperparameters. This can lead to suboptimal performance when tracking dynamically changing targets. Moreover, these models rely on convolutional neural networks for feature extraction, which have limitations in capturing global features. In this paper, we introduce a novel training network model called DAPAT, which combines the anchor-free concept with Transformer theory. DAPAT differs from previous models in several ways. Specifically, we have designed a straightforward model to extract precise global features from template and search images. We have also incorporated two enhancement modules into the model to improve template and search images of different sizes while suppressing the impact of non-target images. We employ a dual-stream feature fusion network to reduce the loss of image feature information due to feature correlation operations. Finally, we compare the performance of the tracking model proposed in this paper with some advanced RGBT trackers on three data sets (RGBT234, RGBT210, and GTOT). The test results demonstrate that our tracker exhibits improvements in robustness and success rate, among other performance aspects.

Visible-infrared image matching based on parameter-free attention mechanism and target-aware graph attention mechanism

Image matching involves identifying the template region in the search image after selecting it in the template image. Image matching, as a basic task, is a pre-order step of many image processing. Because of the different imaging principles of visible and infrared images, the difference between visible and infrared images is large. Compared with the matching of visible images, it is more difficult to match visible and infrared images. Due to the characteristics of infrared images, many targets in infrared images do not have clear outlines, which makes it difficult to distinguish targets and backgrounds in infrared images during the matching process. Regarding the issue above, we propose a network named VINet for visible infrared image matching, and design a feature extraction network AM-Net based on Inception-v3. In order to improve the feature expression ability of the network, we added a parameter-free attention mechanism to the AM-Net network, which improved the network expression ability without introducing new parameters. In addition, we added DropBlock to the AM-Net network, which can achieve better regularization through the DropBlock. To accurately differentiate the background from the target and obtain the target position, we integrate target-aware graph attention in VINet and employ the CIoU loss during training. To address the limited number of visible and infrared image matching datasets, we repurpose existing datasets by relabeling them to obtain new training and testing sets. The experimental results indicate that our method can achieve better matching results compared with other state-of-the-art methods.

Surprise! Draw the scene: Visual recall reveals poor incidental working memory following visual search in natural scenes.

Searching within natural scenes can induce incidental encoding of information about the scene and the target, particularly when the scene is complex or repeated. However, recent evidence from attribute amnesia (AA) suggests that in some situations, searchers can find a target without building a robust incidental memory of its task relevant features. Through drawing-based visual recall and an AA search task, we investigated whether search in natural scenes necessitates memory encoding. Participants repeatedly searched for and located an easily detected item in novel scenes for numerous trials before being unexpectedly prompted to draw either the entire scene (Experiment 1) or their search target (Experiment 2) directly after viewing the search image. Naïve raters assessed the similarity of the drawings to the original information. We found that surprise-trial drawings of the scene and search target were both poorly recognizable, but the same drawers produced highly recognizable drawings on the next trial when they had an expectation to draw the image. Experiment 3 further showed that the poor surprise trial memory could not merely be attributed to interference from the surprising event. Our findings suggest that even for searches done in natural scenes, it is possible to locate a target without creating a robust memory of either it or the scene it was in, even if attended to just a few seconds prior. This disconnection between attention and memory might reflect a fundamental property of cognitive computations designed to optimize task performance and minimize resource use.

Colletes hederae bees are equally attracted by visual and olfactory cues of inconspicuous Hedera helix flowers

Wild bees are heavily declining worldwide except for a few species, such as Colletes hederae, which is spreading in its distribution throughout Europe. Colletes hederae mainly forages on ivy (Hedera helix) which is widespread in Europe and the plants’ availability is thought to contribute to the successful spread of C. hederae. A rapid location of the plants using visual and/or olfactory floral cues would allow the bee to efficiently forage. Beside bee visitors, the flowers attract a high variety of other insects, such as Vespula wasps that were recently investigated regarding their floral-cue preferences. The aim of this study was to investigate the communication between C. hederae and its H. helix host flowers, and to compare the results with that previously obtained with V. germanica wasps. We identified headspace compounds detectable by the bees using gas chromatography coupled to electroantennography (GC-EAD) and performed behavioral experiments to both compare the attractiveness of visual and olfactory floral cues and to determine the attractiveness of a synthetic mixture composed of physiologically active compounds. In the GC-EAD analyses, bees responded to 15 flower-specific compounds of various chemical classes, of which 4-oxoisophorone, (E)-linalool-oxide furanoid, and acetophenone were the most abundant in the floral scent. In the bioassays, visual and olfactory flower cues were equally attractive for bees, but a combination of both cues was needed to elicit not only approach responses but also landings. A synthetic mixture of the EAD-active compounds was attractive to the bees, but to a lesser extent than the natural scent of H. helix flowers. The bees’ integrations of different floral-cue modalities in its search image and its strong antennal responses elicited by various floral scent compounds make C. hederae highly effective in finding its host flowers. In comparison to V. germanica wasps, the bees relied stronger on visual cues than the wasps do, but both species showed the highest attraction when presented with a combination of the cues.

Target-Aware Transformer Tracking

Object tracking is aimed at locating a specific object in the image sequence, such as pedestrians, vehicles, and so on. The existing algorithms based on siamese neural network predict the target through similarity matching. Although these algorithms have achieved satisfactory performance, in the process of similarity calculation between template image and search image, only local information is often concerned, which makes the algorithms difficult to obtain the optimal solution. To deal with the abovementioned problems, we propose a model based on Transformer, named TaTrack. Specifically, we first use the encoders to enhance the features. Then, the dependency between template features and search features is established through the target-aware module. Finally, we utilize the classification regression network to locate the target, and use the classification score to adapt to update the template image. Experiments show that our model can achieve great performance on GOT-10k, LaSOT, and TrackingNet datasets.