Current Frame Research Articles

Multi-frame network feature fusion model and self-attention mechanism for vehicle lane line detection

The traditional lane detection networks mainly use independent single frame images to extract features first and then detect them, which cannot deal with the scene with complex background well. Therefore, this paper proposes a lane parallel detection network based on multi-frame network feature fusion model and self-attention mechanism according to the scene characteristics that vehicles can obtain continuous images during normal driving. Firstly, a parallel feature extraction structure is designed. On the one hand, a single frame network with high precision is used to extract the features of the current frame. On the other hand, a lightweight multi-frame network is designed to extract features of low-resolution multi-frame temporal images. And the recurrent neural network module is used to fuse the extracted temporal features and obtain multi-frame features. Self-attention mechanism can effectively capture the relevant information of internal features. Then the fusion module of single frame feature, multi-frame feature and self-attention feature is designed. The feature map of lane line is output by up-sampling network. The experimental results show that the network in this paper has significant improvement in both objective detection accuracy and subjective effect compared with other methods.

Exploring Multi-modal Spatial-Temporal Contexts for High-performance RGB-T Tracking.

In RGB-T tracking, there exist rich spatial relationships between the target and backgrounds within multi-modal data as well as sound consistencies of spatial relationships among successive frames, which are crucial for boosting the tracking performance. However, most existing RGB-T trackers overlook such multi-modal spatial relationships and temporal consistencies within RGB-T videos, hindering them from robust tracking and practical applications in complex scenarios. In this paper, we propose a novel Multi-modal Spatial-Temporal Context (MMSTC) network for RGB-T tracking, which employs a Transformer architecture for the construction of reliable multi-modal spatial context information and the effective propagation of temporal context information. Specifically, a Multi-modal Transformer Encoder (MMTE) is designed to achieve the encoding of reliable multi-modal spatial contexts as well as the fusion of multi-modal features. Furthermore, a Quality-aware Transformer Decoder (QATD) is proposed to effectively propagate the tracking cues from historical frames to the current frame, which facilitates the object searching process. Moreover, the proposed MMSTC network can be easily extended to various tracking frameworks. New state-of-the-art results on five prevalent RGB-T tracking benchmarks demonstrate the superiorities of our proposed trackers over existing ones.

NeRFBuff: Fast Neural Rendering via Inter-frame Feature Buffering.

Neural radiance fields (NeRF) have demonstrated impressive performance in novel view synthesis, but are still slow to render complex scenes at a high resolution. We introduce a novel method to boost the NeRF rendering speed by utilizing the temporal coherence between consecutive frames. Rather than computing features of each frame entirely from scratch, we reuse the coherent information (e.g., density and color) computed from the previous frames to help render the current frame, which significantly boosts rendering speed. To effectively manage the coherent information of previous frames, we introduce a history buffer with a multiple-plane structure, which is built online and updated from old frames to new frames. We name this buffer as multiple plane buffer (MPB). With this MPB, a new frame can be efficiently rendered using the warped features from previous frames. Extensive experiments on the NeRF-Synthetic, LLFF, and Mip-NeRF-360 datasets demonstrate that our method significantly boosts rendering efficiency and achieves 4× speedup on real-world scenes compared to the baseline methods while preserving competitive rendering quality.

АВТОМАТИЗАЦИЯ ПРОЦЕДУР ОБНАРУЖЕНИЯ ЛЕСНЫХ ПОЖАРОВ ПО РЕЗУЛЬТАТАМ ОБРАБОТКИ ВИДЕО

The results of developing a proposal for automating forest fire detection procedures based on pattern recognition methods are presented. The mathematical problem of constructing a feature space and the problem of pattern recognition based on the results of comparing feature vectors have been formulated and posed. As part of the solution to the problem under consideration, it is proposed to form a feature space from two vectors characterizing the image of the current and previous frames. The choice of histograms characterizing the distribution of image brightness intensity as feature vectors is justified. Experimental results are presented that confirm the validity of the theoretical conclusions and recommendations for automating detection procedures 
 are formulated.

Sparse regularized correlation filter for UAV object tracking with adaptive contextual learning and keyfilter selection

Recently, correlation filter has been widely applied in unmanned aerial vehicle (UAV) tracking due to its high frame rates, robustness and low calculation resources. However, it is fragile because of two inherent defects, i.e., boundary effect and filter corruption. To handle them, in this work, we propose a novel ℓ1 regularization correlation filter with adaptive contextual learning and keyfilter selection for UAV tracking. Firstly, we adaptively detect the positions of effective contextual distractors by the aid of the distribution of local maximum values on the response map of current frame which is generated by using the previous correlation filter model. Next, we eliminate inconsistent labels for the tracked target by removing one on each distractor and develop a new score scheme for each distractor. Then, we can select the keyfilter from the filters pool by finding the maximal similarity between the target at the current frame and the target template corresponding to each filter in the filters pool. Finally, quantitative and qualitative experiments on three authoritative UAV datasets show that the proposed method is superior to the state-of-the-art tracking methods based on correlation filter framework.

Definite-like meaning of bare classifiers in Nung

Abstract Nung (Tai, Kra-Dai) is a numeral-classifier language that contains the Classifier-Noun (cl-n, or ‘bare classifier’) construction. Drawing on Langacker’s Cognitive Grammar (Langacker 1987, 2004, 2008, 2017) (CG), I argue that the use of a Nung cl-n phrase is only possible when it refers to a unique instance in the current discourse frame of the current discourse space. This explains why cl-n phrases show a definite-like meaning contrast with other types of nominal phrases (NPs) in Nung, as well as why Nung cl-n phrases can be interpreted as either specific or non-specific indefinite. This paper makes two theoretical contributions. Firstly, it shows that the existing theory of numeral classifiers within CG, which is based on the unitization function of numeral classifiers, is not sufficient to account for the meaning contrast between cl-n and other types of NP in Nung. Secondly, given that a referent that exists in the current discourse frame need not exist in the actual world or pre-exist in the discourse, this paper illustrates how a referent can satisfy a presupposition of uniqueness without satisfying a presupposition of existence in the actual world or in the discourse.

Scene-Based Nonuniformity Correction Method Using Principal Component Analysis for Infrared Focal Plane Arrays

In this paper, principal component analysis is introduced to form a scene-based nonuniformity correction method for infrared focal plane arrays. The estimation of the gain and offset of the infrared detector and the correction of nonuniformity based on the neural network method with a novel estimation of desired target value are achieved concurrently. The current frame and several adjacent registered frames are decomposed onto a set of principal components, and then the first principal component is extracted to construct the desired target value. It is practical, forms fewer ghosting artifacts, and considerably promotes correction precision. Numerical experiments demonstrate that the proposed method presents excellent performance when dealing with clean infrared data with synthetic pattern noise as well as the real infrared video sequence.

A novel method for real-time object-based copy-move tampering localization in videos using fine-tuned YOLO V8

The research community faces challenges for video forgery detection techniques as advancements in multimedia technology have made it easy to alter the original video content and share it on electronic and social media with false propaganda. The copy-move attack is the most commonly practiced type of attack in videos/images, where an object is copied and moved into the current frame or any other frame of the video. Hence an illusion of recreation can be created to forge the content. It is very difficult to differentiate to uncover the forgery traces by the naked eye. Hence, a passive method-based algorithm is proposed to scientifically investigate the statistical properties of the video by normalizing the median difference of the frames at the pixel level, and graphical analysis successfully shows the clear peak in the forged region. After that, a new deep learning approach, “You Only Look at Once”, the latest eighth version of YOLO, is tuned and trained for the localization of forged objects in the real-time domain. The validation and testing results obtained from the trained YOLO V8 are successfully able to detect and localize the forged objects in the videos with mean average precision (mAP) of 0.99, recall is 0.99, precision is 0.99, and highest confidence score. The proposed YOLO V8 is fine-tuned in three different ways, and the performance of the proposed method outperforms existing state-of-the-art techniques in terms of inference speed, accuracy, precision, recall, testing, and training time.

Robust RGB: D-SLAM in highly dynamic environments based on probability observations and clustering optimization

Visual simultaneous localization and mapping (SLAM) is the underlying support of unmanned systems. Currently, most visual SLAM methods are based on the static environment assumption so that dynamic objects in the camera’s field of view will seriously disrupt its working performance. In view of this, an RGB-D SLAM approach based on probability observations and clustering optimization for highly dynamic environments is proposed, which can effectively eliminate the influence of dynamic objects and accurately estimate the ego-motion of an RGB-D camera. The method contains a dual static map point detection strategy, which is carried out simultaneously in the current and previous frames. First, to enhance tracking robustness in highly dynamic environments, the probabilities of map points being static, calculated by both reprojection deviation and intensity deviation, are used to weight the cost function for pose estimation. Meanwhile, by taking the previous frames as a reference, a static velocity probability based on sparse scene flow is acquired to preliminarily recognize static map points and further improve the tracking accuracy. Then, an improved map point optimization strategy based on K-means clustering is designed, which effectively takes advantage of the clustering algorithm to refine the static map point labels while alleviating its stubborn problem. Finally, the experimental results on the TUM dataset and real scenes compared with the state-of-the-art visual SLAM methods illustrate that the proposed method achieves an extremely robust and accurate performance for estimating camera pose in highly dynamic environments.

Playing by the rules: formalisation in the agroecology sector during the COVID-19 pandemic

ABSTRACT In Spain, small-scale agroecological production has emerged in response to the current environmental and food crisis. However, the lack of a protective legal framework, multiple sanctions and the political inertia of economic growth are deeply constraining and altering the very essence of such initiatives. In particular, new regulations introduced during the COVID-19 pandemic have accelerated the formalisation and institutionalization of the agroecology sector. Based on ethnographic fieldwork in the region of Madrid, the article explores the logics and processes through which agroecological producers adjust (or not) to current regulative frames, and the implications of the decisions for their work.

Neural Temporal Denoising for Indirect Illumination.

Various temporal denoising methods have been proposed to clean up the noise for real-time ray tracing (RTRT). These methods rely on the temporal correspondences of pixels between the current and previous frames, i.e. per-pixel screen-space motion vectors. However, the state-of-the-art temporal reuse methods with traditional motion vectors cause artifacts in motion occlusions. We accordingly propose a novel neural temporal denoising method for indirect illumination of Monte Carlo (MC) ray tracing at 1 sample per pixel. Based on end-to-end multi-scale kernel-based reconstruction, we apply temporally reliable dual motion vectors to facilitate better reconstruction of the occlusions, and also introduce additional motion occlusion loss to reduce ghosting artifacts. Experiments show that our method significantly reduces the over-blurring and ghosting artifacts while generating high-quality images at real-time rates.

An Advanced Software Platform and Algorithmic Framework for Mobile DBH Data Acquisition

Rapid and precise tree Diameter at Breast Height (DBH) measurement is pivotal in forest inventories. While the recent advancements in LiDAR and Structure from Motion (SFM) technologies have paved the way for automated DBH measurements, the significant equipment costs and the complexity of operational procedures continue to constrain the ubiquitous adoption of these technologies for real-time DBH assessments. In this research, we introduce KAN-Forest, a real-time DBH measurement and key point localization algorithm utilizing RGB-D (Red, Green, Blue-Depth) imaging technology. Firstly, we improved the YOLOv5-seg segmentation module with a Channel and Spatial Attention (CBAM) module, augmenting its efficiency in extracting the tree’s edge features in intricate forest scenarios. Subsequently, we devised an image processing algorithm for real-time key point localization and DBH measurement, leveraging historical data to fine-tune current frame assessments. This system facilitates real-time image data upload via wireless LAN for immediate host computer processing. We validated our approach on seven sample plots, achieving bbAP50 and segAP50 scores of: 90.0%(+3.0%), 90.9%(+0.9%), respectively with the improved YOLOv5-seg model. The method exhibited a DBH estimation RMSE of 17.61∼54.96 mm (R2=0.937), and secured 78% valid DBH samples at a 59 FPS. Our system stands as a cost-effective, portable, and user-friendly alternative to conventional forest survey techniques, maintaining accuracy in real-time measurements compared to SFM- and LiDAR-based algorithms. The integration of WLAN and its inherent scalability facilitates deployment on Unmanned Ground Vehicles (UGVs) to improve the efficiency of forest inventory. We have shared the algorithms and datasets on Github for peer evaluations.

Cross-stage recurrent feature sharing network for video dehazing

Hazy weather conditions feed serious haze effects, such as color and visibility degradations in the captured videos. Extensive research on video dehazing has been carried out. However, they lack to balance the spatial and high-level contextual information during restoration. Being a complex task in this paper, we propose a cross-stage recurrent feature sharing network to balance the competing objective. In the proposed network, a cross-stage feature merging module operates on learned features of the previous video frame using the proposed encoder decoder based generator architecture. Learned features fed into the same encoder decoder framework for the current video frame feature learning. This module benefit the network while restoring the current video frame. It shares the feature information between two consecutive frames as they have a very minute information change. Meanwhile, the design of our encoder decoder generator architecture is based on a multi-receptive resolution module, feature fusion module, and pixel-wise spatial attention module. These modules mainly focus on acquiring broad contextual information, fusing intermediate-stage features to increase the learning ability of the network, and capturing robust features while discarding unnecessary features. Experimental evaluation results and ablation study show that the proposed network is superior to existing state-of-the-art methods for video dehazing.

Evaluation of a novel biomechanics-informed walking frame, developed through a Knowledge Transfer Partnership between biomechanists and design engineers.

Walking aids such as walking frames offer support during walking, yet paradoxically, people who self-report using them remain more likely to fall than people who do not. Lifting of walking frames when crossing door thresholds or when turning has shown to reduce stability, and certain design features drive the need to lift (e.g. small, non-swivelling wheels at the front). To overcome shortfalls in design and provide better stability, biomechanists and industrial engineers engaged in a Knowledge Transfer Partnership to develop a novel walking frame that reduces the need for lifting during everyday tasks. This paper presents the results for the final prototype regarding stability, safety and other aspects of usability. Four studies were conducted that explored the prototype in relation to the current standard frame: a detailed gait lab study of 9 healthy older adults performing repeated trials for a range of everyday tasks provided mechanical measures of stability, a real-world study that involved 9 users of walking frames provided measures of body weight transfer and lifting events, two interview studies (5 healthcare professionals and 7 users of walking frames) elicited stakeholder perceptions regarding stability, safety and usability. Analysis of healthy older adults using a standard walking frame and the prototype frame demonstrated that the prototype increases stability during performance of complex everyday tasks (p < 0.05). Similarly, gait assessments of walking frame users in their home environment showed that the prototype facilitated safer usage patterns and provided greater and more continuous body weight support. Interviews with healthcare professionals and users showed that the prototype was perceived to be safe and effective and hence more usable. The outcomes of the separate studies all support the same conclusion: the prototype is an improvement on the status quo, the typical front-wheeled Zimmer frame for indoor use which has not changed in design for decades. The significance of this work lies in the success of the Knowledge Transfer Partnership and in biomechanics-informed design leading to improvements, which in future may be applied to other walking aids, to benefit walking aid users by promoting safer, more stable use of their aid.

A CMOS Image Sensor Dark Current Compensation Using In-Pixel Temperature Sensors.

This paper presents a novel technique for dark current compensation of a CMOS image sensor (CIS) by using in-pixel temperature sensors (IPTSs) over a temperature range from -40 °C to 90 °C. The IPTS makes use of the 4T pixel as a temperature sensor. Thus, the 4T pixel has a double functionality, either as a pixel or as a temperature sensor. Therefore, the dark current compensation can be carried out locally by generating an artificial dark reference frame from the temperature measurements of the IPTSs and the temperature behavior of the dark current (previously calibrated). The artificial dark current frame is subtracted from the actual images to reduce/cancel the dark signal level of the pictures. In a temperature range from -40 °C to 90 °C, results show that the temperature sensors have an average temperature coefficient (TC) of 1.15 mV/°C with an inaccuracy of ±0.55 °C. Parameters such as conversion gain, gain of the amplifier, and ADC performance have been analyzed over temperature. The dark signal can be compensated in the order of 80% in its median value, and the nonuniformity is reduced in the order of 55%.

Graph Neural Networks for Distributed Power Allocation in Wireless Networks: Aggregation Over-the-Air

Distributed power allocation is important for interference-limited wireless networks with dense transceiver pairs. In this paper, we aim to design low signaling overhead distributed power allocation schemes by using graph neural networks (GNNs), which are scalable to the number of wireless links. We first apply the message passing neural network (MPNN), a unified framework of GNN, to solve the problem. We show that the signaling overhead grows quadratically as the network size increases. Inspired from the over-the-air computation (AirComp), we then propose an Air-MPNN framework, where the messages from neighboring nodes are represented by the transmit power of pilots and can be aggregated efficiently by evaluating the total interference power. The signaling overhead of Air-MPNN grows linearly as the network size increases, and we prove that Air-MPNN is permutation invariant. To further reduce the signaling overhead, we propose the Air message passing recurrent neural network (Air-MPRNN), where each node utilizes the graph embedding and local state in the previous frame to update the graph embedding in the current frame. Since existing communication systems send a pilot during each frame, Air-MPRNN can be integrated into the existing standards by adjusting pilot power. Simulation results validate the scalability of the proposed frameworks, and show that they outperform the existing power allocation algorithms in terms of sum-rate for various system parameters.

A fuzzy decision-making system for video tracking with multiple objects in non-stationary conditions

Computer vision remains challenged by tracking multiple objects in motion frames, despite efforts to improve surveillance, healthcare, and human-machine interaction. This paper presents a method for monitoring several moving objects in non-stationary settings for autonomous navigation. Additionally, at each phase, movement information between successive frames, including the new frame and the previous frame, is employed to determine the location of moving objects inside the camera's field of view, and the background in the new frame is determined. With the help of a matching algorithm, the Kanade–Lucas–Tomasi (KLT) feature tracker for each frame is determined. To get the new frame, we access the matching feature points between two subsequent frames, calculate the movement size of the feature points and the camera movement, and subtract the previous frame of moving objects from the current frame. Every moving object within the camera's field of view is captured at every moment and location. The moving items are categorized and segregated using fuzzy logic based on their mass center and length-to-width ratio. Our algorithm was implemented to investigate autonomous navigation surveillance of three types of moving objects, such as a vehicle, a pedestrian, a bicycle, or a motorcycle. The results indicate high accuracy and an acceptable time requirement for monitoring moving objects. It has a tracking and classification accuracy of around 75 % and processes 43 frames per second, making it superior to existing approaches in terms of speed and accuracy.

Anchor-free deep convolutional neural network for tracking and counting cotton seedlings and flowers

Accurate counting of plants and their organs in natural environments is essential for breeders and growers. For breeders, counting plants during the seedling stage aids in selecting genotypes with superior emergence rates, while for growers, it informs decisions about potential replanting. Meanwhile, counting specific plant organs, such as flowers, forecasts yields for different genotypes, offering insights into production levels. The overall goal of this study was to investigate a deep convolutional neural network-based tracking method, CenterTrack, for cotton seedling and flower counting from video frames. The network is extended from a customized CenterNet, which is an anchor-free object detector. CenterTrack predicts the detections of the current frame and displacements of detections between the previous frame and the current frame, which are used to associate the same object in consecutive frames. The modified CenterNet detector achieved high accuracy on both seedling and flower datasets with an overall AP50 of 0.962. The video tracking hyperparameters were optimized for each dataset using orthogonal tests. Experimental results showed that seedling and flower counts with optimized hyperparameters highly correlated with those of manual counts (R2 = 0.98 andR2 = 0.95) and the mean relative errors of 75 cotton seedling testing videos and 50 flower testing videos were 5.5 % and 10.8 %, respectively. An average counting speed of 20.4 frames per second was achieved with an input resolution of 1920 × 1080 pixels for both seedling and flower videos. The anchor-free deep convolution neural network-based tracking method provides automatic tracking and counting in video frames, which will significantly benefit plant breeding and crop management.

DETECTION OF VOID NUCLEATION, COALESCENCE AND COLLAPSE IN ATOMISTIC SIMULATIONS

Extraction of required information from atomic configurations is crucial for efficient use of molecular dynamics (MD) simulations, and automating this process allows the collection of statistical data to develop and validate physical models and data-driven approaches. Here we update the previously developed pore searching algorithm to automatically detect the events of void nucleation, collapse and coalescence by comparing pairs of atomic configurations. We calculate an intersection matrix, which shows common regions between voids of current and previous time frames, and analysis of this intersection matrix gives a numerical criterion for determination of the considered events. The updated algorithm is verified in the case of small MD system and further used to analyze a large MD system with many hundreds of voids. The predominance of void collapse is confirmed in the case of uniform triaxial tension of solid aluminum, while void coalescence instead of collapse is detected in the case of a more complex loading with axial compression and lateral tension. This dependence of the predominant mechanism on the loading path correlates with previous finite element analysis in the literature. Another interesting finding is a continuing nucleation and collapse of voids far beyond the fracture beginning at a level of negative pressure much lower than the spall strength, which can be attributed to a developed defect structure of material produced by plastic growth of voids.

A spatial–temporal contexts network for object tracking

Although there have been significant advancements and developments in visual object tracking in recent years, most trackers have failed to adapt to the deterioration of object appearance in complex scenes. Typically, they utilize only spatial information or simple temporal networks. The fusion of spatial and temporal contexts among consecutive frames can hypothetically capture historical information to boost tracking performance but inevitably pollutes the model with noisy samples. To this end, we proposed a novel end-to-end ConvLSTM-based tracking framework called STCTrack, which uses spatial and temporal information from each frame and adapts to noisy samples. Specifically, a multilayer residual ConvLSTM-based spatial–temporal context network (STCN) was proposed in STCTrack to retain the target’s past information and consequently guide the tracker to focus on the most informative regions of the current frame. Furthermore, a multi-similarity map fusion model was proposed to calculate the pixel-level similarity map, allowing STCTrack to adaptively retrieve historical target information from different times and be resilient to partial occlusions and nonrigid deformations. Extensive empirical studies were conducted on benchmarks, including OTB2015, GOT-10K, TrackingNet, LaSOT, UAV123, and VOT2018. The empirical results suggest that the proposed STCTrack achieves state-of-the-art performance compared with existing schemes. The code and models used in this study are publicly available at www.github.com/Kevoen/STCTrack to encourage further research on this topic.