Unconstrained Scenarios Research Articles

Efficient license plate recognition in unconstrained scenarios

Automatic license plate recognition (ALPR) is a critical technology for intelligent transportation systems. Most existing ALPR methods are focused on specific application scenarios. Although there are a few methods that focus on unconstrained scenarios, they are very time-consuming. In this work, we propose an efficient ALPR (EALPR) framework, where we can handle distorted license plates (LP) caused by perspective problems with high efficiency. We design a light LPD structure based on efficient object detection methods and use anchor-free strategies for LPD to alleviate the problem of expensive costs. Benefitting from these optimizations and a united framework structure, the proposed EALPR has real-time efficiency. We evaluate our method on five datasets and the results show that our method achieves state-of-the-art accuracy: 98.15% on OpenALPR(EU), 95.61% on OpenALPR(BR), 99.51% on AOLP(RP), 88.81% on SSIG, 79.41% on CD-HARD. Additionally, our method achieves an impressive speed of 74.9 FPS (Frames Per Second), outperforming existing approaches and demonstrating its efficiency. Our source code can be accessed at https://github.com/wechao18/Efficient-alpr-unconstrained.

Enhancing fruit and vegetable detection in unconstrained environment with a novel dataset

Automating the detection of fruits and vegetables using computer vision is essential for modernizing agriculture, improving efficiency, ensuring food quality, and contributing to sustainable and technologically advanced farming practices. This paper presents an end-to-end pipeline for detecting and localizing fruits and vegetables in real-world scenarios. To achieve this, a dataset named FRUVEG67 was curated that includes images of 67 classes of fruits and vegetables captured in unconstrained scenarios, with only a few manually annotated samples per class. A semi-supervised data annotation algorithm (SSDA) was developed that generates bounding boxes for objects to label the remaining nonannotated images. For detection, Fruit and Vegetable Detection Network (FVDNet) was proposed, an ensemble version of YOLOv8n featuring three distinct grid configurations. In addition, an averaging approach for the prediction of the bounding box and a voting mechanism for the prediction of the classes was implemented. Finally Jensen–Shannon Divergence (JSD) in conjunction with focal loss was integrated as the overall loss function for better detection of smaller objects. Experimental results highlight the superiority of FVDNet compared to recent versions of YOLO, showcasing remarkable improvements in detection and localization performance. An impressive mean average precision (mAP) score of 0.82 across all classes was achieved. Furthermore, the efficacy of FVDNet on open-category refrigerator images were evaluated, where it demonstrates promising results.

A novel successive convexification framework with symplectic pseudospectral solutions for nonlinear optimal control problems of constrained pantograph delay systems

This paper proposes a novel symplectic pseudospectral iteration framework for the nonlinear optimal control problem (OCP) of a pantograph delay system with inequality constraints. Traditional numerical algorithms for OCPs with pantograph delay systems have predominantly focused on unconstrained scenarios, neglecting constrained problems. Therefore, we propose a novel symplectic pseudospectral iteration framework for constrained problems. First, we employ the successive convexification (SCvx) method to transform the original problem into a series of linear quadratic (LQ) problems, addressing nonlinearity. Then, leveraging time-scale transformations and parametric variational principles, we derive the first-order necessary conditions (FONCs) for these convexified problems, including a Hamiltonian two-point boundary value problem (HTBVP) with stretching terms and a linear complementarity problem. To handle pantograph time delays effectively, we employ a local Legendre-Gauss-Lobatto (LGL) pseudospectral method with proportional grid discretization. Finally, employing a symplectic indirect algorithm, we develop a symplectic pseudospectral method (SPM) for solving the transformed LQ problems, converting them into a system of sparse linear algebraic equations and a linear complementarity problem. Validation of the framework is performed using diverse illustrative examples, demonstrating strong agreement between SCvx and SPM in computational efficiency. Numerical experiments confirm the sparsity of the core matrix and robustness to initial guesses.

ALP-Net: a segmentation-free approach for license plate recognition in unconstrained scenarios

ALP-Net: a segmentation-free approach for license plate recognition in unconstrained scenarios

HDNet: Human-like discrimination with visual key for few-shot cross-domain object detection

Most existing few-shot object detection (FSD) methods implicitly assume that the target domain data with few samples conform to the same statistical distribution as the source domain. However, this assumption is impractical, especially when dealing with unconstrained scenarios. Also, the decline of fine-grained hidden samples caused by scene switching, significant morphological changes, etc. has brought great challenges to object detection. To solve the above few-shot cross-domain detection (FS-CDD) problems, in this work, we propose a novel and flexible Human-like Discrimination Network (HDNet), which is composed of four modules. Firstly, multi-level key generation (MKG) fully mining multi-level comprehensive abstract representation. The precious knowledge that contains diverse low and high levels is obtained by three parallel branches. After decoupling the hidden space, the rich patches from each pair of target and source domains are closely matched in the Embedded Space Implicit Association (ESIA) using degree description. In order to enhance the discriminative capability of the model for a few samples, the instances are additionally encoded and corrected at the classification end using the Prediction head with instance embedding (PHIE). Finally, the Adaptive Reweighted Module (ARM) is redesigned to determine coefficient of multiple loss functions, which avoids the improper operation of setting coefficients according to experience in the past. Extensive experiments demonstrate that despite the dual challenges of limited samples and cross-domain scenarios, the proposed HDNet exhibits remarkable performance.

Cognitive effort detection for tele-robotic surgery via personalized pupil response modeling.

Gaze tracking and pupillometry are established proxies for cognitive load, giving insights into a user's mental effort. In tele-robotic surgery, knowing a user's cognitive load can inspire novel human-machine interaction designs, fostering contextual surgical assistance systems and personalized training programs. While pupillometry-based methods for estimating cognitive effort have been proposed, their application in surgery is limited by the pupil's sensitivity to brightness changes, which can mask pupil's response to cognitive load. Thus, methods considering pupil and brightness conditions are essential for detecting cognitive effort in unconstrained scenarios. To contend with this challenge, we introduce a personalized pupil response model integrating pupil and brightness-based features. Discrepancies between predicted and measured pupil diameter indicate dilations due to non-brightness-related sources, i.e., cognitive effort. Combined with gaze entropy, it can detect cognitive load using a random forest classifier. To test our model, we perform a user study with the da Vinci Research Kit, where 17 users perform pick-and-place tasks in addition to auditory tasks known to generate cognitive effort responses. We compare our method to two baselines (BCPD and CPD), demonstrating favorable performance in varying brightness conditions. Our method achieves an average true positive rate of 0.78, outperforming the baselines (0.57 and 0.64). We present a personalized brightness-aware model for cognitive effort detection able to operate under unconstrained brightness conditions, comparing favorably to competing approaches, contributing to the advancement of cognitive effort detection in tele-robotic surgery. Future work will consider alternative learning strategies, handling the difficult positive-unlabeled scenario in user studies, where only some positive and no negative events are reliably known.

Integration of Modified Classical Conjugate Gradient Methods for Unconstrained Optimization

The integration of modified classical conjugate gradient methods (CGMs) for unconstrained optimization represents a crucial and evolving area of research within the field of optimization algorithms. Over time, numerous studies have put forth diverse modifications and novel approaches to enhance the effectiveness of classical CGMs. These modifications aim to address specific challenges and improve the overall performance of optimization algorithms in unconstrained scenarios. In order to tackle unconstrained optimization challenges and improve our understanding of their synergies, this ongoing study aims to unify different modified classical CGMs. Conventional CGMs have proven effective for optimization tasks, and a range of different approaches have been produced by carefully modifying these techniques. The main goal of this paper is to combine these modified versions, with particular attention to those that have similar numerators. The integration process involves systematically merging the advantageous aspects of these modified methods to develop not only innovative but also more resilient approaches to unconstrained optimization problems. The ultimate goal of this unification effort is to capitalize on the strengths inherent in different approaches to create a cohesive framework that significantly improves overall optimization performance. To thoroughly assess the efficacy of the integrated methods, a series of comprehensive performance tests are conducted. These tests include a meticulous comparison of outcomes with those of classical CGMs, providing valuable insights into the relative strengths and weaknesses of the modified approaches across diverse optimization scenarios. The evaluation criteria encompass convergence rates, solution accuracy, and computational efficiency. The conclusive outcome demonstrates that the unified approaches consistently outperform individual methods across all three crucial evaluation criteria.

Optimization of the end‐of‐life tire repartition within the European treatment system to minimize its environmental impacts

AbstractThis study contrasts two different approaches to inform European‐scale decision‐making to mitigate the environmental impacts of the end‐of‐life tires (ELT) management system. The first analysis is a traditional life cycle assessment (LCA) that compares the environmental performances of the 12 main available European end‐of‐life (EOL) technologies in ELT processing while restricting the boundaries to the EOL stage. The second analysis has a broader scope, addressing the optimization of the ELT distribution within the 12 considered pathways to minimize the environmental impacts of the total tire use in Europe under present capacity and constraints. The results of the traditional LCA show that, except for landfill, all the tested EOL routes present environmental benefits. Material recovery pathways bring the most environmental credits, whereas civil engineering pathways are the least promising. The LCA results that emerged from the optimization of ELT management technologies yield two optimal technological mixes that maximize the quantity of ELT recycled in molded objects production: such results represent a hypothetical case with no constraints. When considering constraints, that is, limitations on maximum quantities of ELT that can undergo retreading, pyrolysis, or recycling in synthetic turfs, in molded objects and in production, the number of optimal technology mixes increases to five. The type of technologies favored depends on the minimized impact categories (climate change, fossil and nuclear energy use, human health, and ecosystem quality). A comparison between constrained and unconstrained scenarios shows that achieving the best environmental performances is conditional to the accessibility of the EOL technologies as well as their individual environmental impacts.

Advancements in Deep-Learning-Based Object Detection in Challenging Environments

This article focuses on object detection in challenging environments, where objects of interest need to be detected in images captured under unconstrained conditions. These environments can include outdoor scenes with varying lighting, weather conditions, and background clutter. Object detection in such scenarios is crucial for applications like autonomous driving, surveillance, and industrial production inspection. The paper explores existing techniques for object detection in challenging environments and proposes novel solutions to enhance its performance. One key aspect emphasized in the paper is the need for relevant datasets to validate models under unconstrained scenarios. While several datasets already exist, the research identifies gaps in specific situations and addresses them by proposing new datasets, such as the Indian Lane Dataset for autonomous vehicles. The proposed object detection framework leverages spatial information to detect objects in challenging environments, and its performance is evaluated against benchmark datasets and the newly proposed datasets. A real-world application demonstrates significant improvements in object detection performance in natural environments.

Aerial Surveillance Leveraging Delaunay Triangulation and Multiple-UAV Imaging Systems

In aerial surveillance systems, achieving optimal object detection precision is of paramount importance for effective monitoring and reconnaissance. This article presents a novel approach to enhance object detection accuracy through the integration of Delaunay triangulation with multi-unmanned aerial vehicle (UAV) systems. The methodology involves positioning multiple UAVs at pre-specified locations using the Delaunay triangulation algorithm with performance of O (n log n). This is compared with the conventional single UAV approach at a near distance. Our findings reveal that the collaborative efforts of multiple UAVs, guided by Delaunay triangulation, significantly improves object detection accuracy, especially when compared to a single UAV operating in close proximity. This research employs advanced image processing techniques to identify objects in the area under surveillance. Results indicate a substantial enhancement in the collective surveillance capabilities of the multi-UAV system, demonstrating its efficacy in unconstrained scenarios. This research not only contributes to the optimization of aerial surveillance operations but also underscores the potential of spatially informed UAV networks for applications demanding heightened object detection accuracy. The integration of Delaunay triangulation with multi-UAV systems emerges as a promising strategy for advancing the capabilities of aerial surveillance in scenarios ranging from security and emergency response to environmental monitoring.

A discriminative multiscale feature extraction network for facial expression recognition in the wild

Driven by advancements in deep learning technologies, substantial progress has been achieved in the field of facial expression recognition over the past decade, while challenges remain brought about by occlusions, pose variations and subtle expression differences in unconstrained (wild) scenarios. Therefore, a novel multiscale feature extraction method is proposed in this paper, that leverages convolutional neural networks to simultaneously extract deep semantic features and shallow geometric features. Through the mechanism of channel-wise self-attention, prominent features are further extracted and compressed, preserving advantageous features for distinction and thereby reducing the impact of occlusions and pose variations on expression recognition. Meanwhile, inspired by the large cosine margin concept used in face recognition, a center cosine loss function is proposed to avoid the misclassification caused by the underlying interclass similarity and substantial intra-class feature variations in the task of expression recognition. This function is designed to enhance the classification performance of the network through making the distribution of samples within the same class more compact and that between different classes sparser. The proposed method is benchmarked against several advanced baseline models on three mainstream wild datasets and two datasets that present realistic occlusion and pose variation challenges. Accuracies of 89.63%, 61.82%, and 91.15% are achieved on RAF-DB, AffectNet and FERPlus, respectively, demonstrating the greater robustness and reliability of this method compared to the state-of-the-art alternatives in the real world.

Enhancing Face Recognition with Detachable Self-Supervised Bypass Networks.

Attributed to the development of deep networks and abundant data, automatic face recognition (FR) has quickly reached human-level capacity in the past few years. However, the FR problem is not perfectly solved in case of large poses and uncontrolled occlusions. In this paper, we propose a novel bypass enhanced representation learning (BERL) method to improve face recognition under unconstrained scenarios. The proposed method integrates self-supervised learning and supervised learning together by attaching two auxiliary bypasses, a 3D reconstruction bypass and a blind inpainting bypass, to assist robust feature learning for face recognition. Among them, the 3D reconstruction bypass enforces the face recognition network to encode pose independent 3D facial information, which enhances the robustness to various poses. The blind inpainting bypass enforces the face recognition network to capture more facial context information for face inpainting, which enhances the robustness to occlusions. The whole framework is trained in end-to-end manner with two self-supervised tasks above and the classic supervised face identification task. During inference, the two auxiliary bypasses can be detached from the face recognition network, avoiding any additional computational overhead. Extensive experimental results on various face recognition benchmarks show that, without any cost of extra annotations and computations, our method outperforms state-of-the-art methods. Moreover, the learnt representations can also well generalize to other face-related downstream tasks such as the facial attribute recognition with limited labeled data.

Attention Analysis in Robotic-Assistive Therapy for Children With Autism.

Children with Autism Spectrum Disorder (ASD) show severe attention deficits, hindering their capacity to acquire new skills. The automatic assessment of their attention response would provide the therapists with an important biomarker to better quantify their behaviour and monitor their progress during therapy. This work aims to develop a quantitative model, to evaluate the attention response of children with ASD, during robotic-assistive therapeutic sessions. Previous attempts to quantify the attention response of autistic subjects during human-robot interaction tasks were limited to restrained child movements. Instead, we developed an accurate quantitative system to assess the attention of ASD children in unconstrained scenarios. Our approach combines gaze extraction (Gaze360 model) with the definition of angular Areas-of-Interest, to characterise periods of attention towards elements of interest in the therapy environment during the session. The methodology was tested with 12 ASD children, achieving a mean test accuracy of 79.5 %. Finally, the proposed attention index was consistent with the therapists' evaluation of patients, allowing a meaningful interpretation of the automatic evaluation. This encourages the future clinical use of the proposed system.

YOLO-MS: Multispectral Object Detection via Feature Interaction and Self-Attention Guided Fusion

Object detection is essential for an autonomous driving sensing system. Since the light condition is changed in unconstrained scenarios, the detection accuracy based on visible images can be greatly degraded. Although the detection accuracy can be improved by fusing visible and infrared images, existing multispectral object detection (MOD) algorithms suffer from inadequate inter-modal interaction and a lack of global dependence in the fusion approach. Thus, we propose a multispectral object detection framework called YOLO-MS by designing a Feature Interaction and Self-Attention Fusion Network (FISAFN) as the backbone network. Within the FISAFN, correlations between two modalities are extracted by the Feature Interaction Module (FIM) for reconstructing the information components of each modality and enhancing capability of information exchange. To filter redundant features and enhance complementary features, long-range information dependence between two modalities are established by using a Self-Attention Feature Fusion Module (SAFFM). Thus, a better information richness of the fused features can be achieved. Experimental results on the FLIR-aligned dataset and the MFD dataset demonstrate that the proposed YOLO-MS performs favorably against state-of-the-art approaches, including feature-level fusion and pixel-level fusion. And further, the proposed YOLO-MS possesses good detection performance under diverse scene conditions.

SwipeFormer: Transformers for mobile touchscreen biometrics

The growing number of mobile devices over the past few years brings a large amount of personal information, which needs to be properly protected. As a result, several mobile authentication methods have been developed. In particular, behavioural biometrics has become one of the most relevant methods due to its ability to extract the uniqueness of each subject in a secure, non-intrusive, and continuous way. This article presents SwipeFormer, a novel Transformer-based system for mobile subject authentication by means of swipe gestures in an unconstrained scenario (i.e., subjects could use their personal devices freely, without restrictions on the direction of swipe gestures or the position of the device). Our proposed system contains two modules: (i) a Transformer-based feature extractor, and (ii) a similarity computation module. Mobile data from the touchscreen and different background sensors (accelerometer and gyroscope) have been studied, including in the analysis both Android and iOS operating systems. A complete analysis of SwipeFormer is carried out using an in-house large-scale database acquired in unconstrained scenarios. In these operational conditions, SwipeFormer achieves Equal Error Rate (EER) values of 6.6% and 3.6% on Android and iOS respectively, outperforming the state of the art. In addition, we evaluate SwipeFormer on the popular publicly available databases Frank DB and HuMIdb, achieving EER values of 11.0% and 5.0% respectively, outperforming previous approaches under the same experimental setup.

Simulating Generalised Locational Marginal Pricing for Power Markets in East Africa

There is an enormous opportunity for adopting a wholesale electricity market under locational marginal pricing. Besides ensuring competition and accounting for network congestion, the price signal reflects a space-time electricity economy. This paper examines the prospects of introducing a single East African electricity market in five selected partner states. It reports on a pioneering effort to simulate the coupling of power markets, estimate locational marginal prices, and determine the economic benefits of cross-border power trade and competition across these five countries. The nodes representing the primary power systems in these five countries are simulated in an optimal power flow model using optimisation software GAMS. The total welfare of electric power market integration is estimated at $ 4.8 million/hr, representing a welfare gain of $ 2.6 million/hr, thus, net total welfare increases by 118%. The simulation also investigates the implications for locational prices and power flows under constrained and unconstrained scenarios. To secure the benefits of an integrated electricity market, it is vital to pursue significant institutional restructuring, a robust transmission infrastructure, and the harmonisation of energy policies.

Pattern recognition of grooves in human lips for improved authentication in cyber-physical systems using U-Net architecture

Latent Lip groove application is been a notable topic in forensic applications like crime and other investigations. The detection of lip movement is been a challenging task since it is a smaller integral part of the human face. The conventional models operate on the available public or private dataset but it is constrained to the large population and unconstrained environment. The study aims at developing a deep learning model in a multimodal system using the deep U-Net Convolutional Neural Network architecture. It also aims at improving biometric authentication through a deep pattern recognition that involves the feature extraction of grooves present in the human lips. An examination of grooves present in the input lip image is conducted by the present system to check the authenticity of the person entering the cyber-physical systems. The lip images are collected from the public security cameras via high-definition cameras in crowded areas that help the proposed method in forensic investigation and further, it considers various unconstrained scenarios to improve the efficacy of the system. The study involves initially pre-processing of lip image, and feature extraction of lip grooves to improve the efficacy of the lip trait. The simulation is conducted on the MATLAB tool to examine the efficacy of the model against various existing methods. Further, the study does not take into account the datasets available on the websites and lip images are only collected from a large set population in a real-time environment. The results of the simulation show that the proposed method achieves a higher degree of accuracy in extracting the grooves from the input lip images.

On the Fine Resolution of Antenna Topologies

The emergence of massive multiple-input-multiple-output, ray-based channel models and millimeter wave systems makes it timely to take a new look at the resolution of different antenna topologies. Hence, we derive a novel approach to evaluate the fine resolution of large antenna arrays. The approach is general, tractable for arbitrary layouts and is based on a bivariate Taylor series expansion of the normalized interference between two closely separated rays. The technique is applied to several topologies in both space-constrained and unconstrained scenarios. The resulting expressions and corresponding results give insight into the relative ability of the arrays to resolve rays and provide a simple performance hierarchy for typical scenarios.

A multi-channel soft biometrics framework for seamless border crossings

As the number of passengers at border entry points such as airports and rail stations increases, so does the demand for seamless, secure, and fast biometric technologies for verification purposes. Although fingerprints are currently useful biometric technologies, they are intrusive and slow down the end-to-end verification process, increasing the chances of tampering. Emerging as an alternative technology, soft biometrics have proven successful for non-intrusive and rapid verification. Soft biometrics consists of a large set of features from three different modalities of the human body, including the face, body, and essential & auxiliary attachments. This paper proposes a multi-channel soft biometrics framework that leverages soft biometrics technology over traditional biometrics. The framework encapsulates four distinct components: ApparelNet, which verifies essential and auxiliary attachments; A-Net, which measures anthropometric soft biometrics; OneDetect, which predicts global soft biometrics; and RSFS, which develops a set of highly relevant and supportive soft biometrics for verification. The proposed framework addresses several critical limitations of existing biometrics technologies during the verification process at border entry points, such as intrusive behavior, response time, biometric tampering, and privacy issues. The proposed multi-channel soft biometrics framework has been evaluated using several benchmark datasets in the field, such as Front-view Gait (FVG), Pedestrian Attribute Recognition At Far Distance (PETA), and Multimedia and Vision (MMV) Pedestrian. Using heterogeneous datasets enables the testing of each framework component or channel against numerous constrained and unconstrained scenarios. The outcome of the envisioned multi-channel soft biometrics framework is presented based on distinct outcomes from each channel, but it remains focused on determining a single cumulative verification score for verification at border control. In addition, this multi-channel soft biometrics framework has extended applications in several fields, including crowd surveillance, the fashion industry, and e-learning.

An Ultra-Fast Automatic License Plate Recognition Approach for Unconstrained Scenarios

Recently, with the development of deep learning, Automatic License Plate Recognition (ALPR) has made great progress, However, there are still many challenges to accomplish license plate (LP) recognition under various traffic scenarios. One of them is the detection speed and recognition speed, and the other is the difficulty to recognize the low resolution and highly tilted LP images. In this paper, we present a two-stage ALPR framework to achieve efficient LP detection and recognition in unconstrained scenarios. Our LP detector is based on improved Yolov3-tiny, and we also propose a lightweight recognition network MRNet based on multi-scale features. In order to improve the inference speed, we abandoned the rectification of LP images, and RNNs that are difficult to compute in parallel. Additionally, we also propose a license plate data augmentation method, which achieves more effective augmentation and improves the generalization ability of the network through secondary random and hyperparametric search. As an additional contribution, we provide a challenging dataset collected from real-world driving recorders. The dataset is for multiple LPs in a single image, which makes up for the lack of public datasets in multiple LP scenarios. We evaluated the results on five datasets and showed that we achieved the best performance on almost all test sets, achieving 99.8% accuracy on CCPD with more than 180,000 license plate test sets. In terms of speed, the inference speed for detecting license plates reaches 751 FPS, and the fastest inference time for recognizing a single license plate takes only 2.9 ms.