Handcrafted Feature Methods Research Articles

Real-Time Hand Gesture Recognition: A Comprehensive Review of Techniques, Applications, and Challenges

Abstract Real-time Hand Gesture Recognition (HGR) has emerged as a vital technology in human-computer interaction, offering intuitive and natural ways for users to interact with computer-vision systems. This comprehensive review explores the advancements, challenges, and future directions in real-time HGR. Various HGR-related technologies have also been investigated, including sensors and vision technologies, which are utilized as a preliminary step in acquiring data in HGR systems. This paper discusses different recognition approaches, from traditional handcrafted feature methods to state-of-the-art deep learning techniques. Learning paradigms have been analyzed such as supervised, unsupervised, transfer, and adaptive learning in the context of HGR. A wide range of applications has been covered, from sign language recognition to healthcare and security systems. Despite significant developments in the computer vision domain, challenges remain in areas such as environmental robustness, gesture complexity, computational efficiency, and user adaptability. Lastly, this paper concludes by highlighting potential solutions and future research directions trying to develop more robust, efficient, and user-friendly real-time HGR systems.

MSTCN: A multiscale temporal convolutional network for user independent human activity recognition.

Background: In recent years, human activity recognition (HAR) has been an active research topic due to its widespread application in various fields such as healthcare, sports, patient monitoring, etc. HAR approaches can be categorised as handcrafted feature methods (HCF) and deep learning methods (DL). HCF involves complex data pre-processing and manual feature extraction in which the models may be exposed to high bias and crucial implicit pattern loss. Hence, DL approaches are introduced due to their exceptional recognition performance. Convolutional Neural Network (CNN) extracts spatial features while preserving localisation. However, it hardly captures temporal features. Recurrent Neural Network (RNN) learns temporal features, but it is susceptible to gradient vanishing and suffers from short-term memory problems. Unlike RNN, Long-Short Term Memory network has a relatively longer-term dependency. However, it consumes higher computation and memory because it computes and stores partial results at each level. Methods: This work proposes a novel multiscale temporal convolutional network (MSTCN) based on the Inception model with a temporal convolutional architecture. Unlike HCF methods, MSTCN requires minimal pre-processing and no manual feature engineering. Further, multiple separable convolutions with different-sized kernels are used in MSTCN for multiscale feature extraction. Dilations are applied to each separable convolution to enlarge the receptive fields without increasing the model parameters. Moreover, residual connections are utilised to prevent information loss and gradient vanishing. These features enable MSTCN to possess a longer effective history while maintaining a relatively low in-network computation. Results: The performance of MSTCN is evaluated on UCI and WISDM datasets using a subject independent protocol with no overlapping subjects between the training and testing sets. MSTCN achieves accuracies of 97.42 on UCI and 96.09 on WISDM. Conclusion: The proposed MSTCN dominates the other state-of-the-art methods by acquiring high recognition accuracies without requiring any manual feature engineering.

HOG-ShipCLSNet: A Novel Deep Learning Network With HOG Feature Fusion for SAR Ship Classification

Ship classification in synthetic aperture radar (SAR) images is a fundamental and significant step in ocean surveillance. Recently, with the rise of deep learning (DL), modern abstract features from convolutional neural networks (CNNs) have hugely improved SAR ship classification accuracy. However, most existing CNN-based SAR ship classifiers overly rely on abstract features, but uncritically abandon traditional mature hand-crafted features, which may incur some challenges for further improving accuracy. Hence, this article proposes a novel DL network with histogram of oriented gradient (HOG) feature fusion (HOG-ShipCLSNet) for preferable SAR ship classification. In HOG-ShipCLSNet, four mechanisms are proposed to ensure superior classification accuracy, that is, 1) a multiscale classification mechanism (MS-CLS-Mechanism); 2) a global self-attention mechanism (GS-ATT-Mechanism); 3) a fully connected balance mechanism (FC-BAL-Mechanism); and 4) an HOG feature fusion mechanism (HOG-FF-Mechanism). We perform sufficient ablation studies to confirm the effectiveness of these four mechanisms. Finally, our experimental results on two open SAR ship datasets (OpenSARShip and FUSAR-Ship) jointly reveal that HOG-ShipCLSNet dramatically outperforms both modern CNN-based methods and traditional hand-crafted feature methods.

MSTCN: A multiscale temporal convolutional network for user independent human activity recognition.

Background: In recent years, human activity recognition (HAR) has been an active research topic due to its widespread application in various fields such as healthcare, sports, patient monitoring, etc. HAR approaches can be categorised as handcrafted feature methods (HCF) and deep learning methods (DL). HCF involves complex data pre-processing and manual feature extraction in which the models may be exposed to high bias and crucial implicit pattern loss. Hence, DL approaches are introduced due to their exceptional recognition performance. Convolutional Neural Network (CNN) extracts spatial features while preserving localisation. However, it hardly captures temporal features. Recurrent Neural Network (RNN) learns temporal features, but it is susceptible to gradient vanishing and suffers from short-term memory problems. Unlike RNN, Long-Short Term Memory network has a relatively longer-term dependency. However, it consumes higher computation and memory because it computes and stores partial results at each level. Methods: This work proposes a novel multiscale temporal convolutional network (MSTCN) based on the Inception model with a temporal convolutional architecture. Unlike HCF methods, MSTCN requires minimal pre-processing and no manual feature engineering. Further, multiple separable convolutions with different-sized kernels are used in MSTCN for multiscale feature extraction. Dilations are applied to each separable convolution to enlarge the receptive fields without increasing the model parameters. Moreover, residual connections are utilised to prevent information loss and gradient vanishing. These features enable MSTCN to possess a longer effective history while maintaining a relatively low in-network computation. Results: The performance of MSTCN is evaluated on UCI and WISDM datasets using subject independent protocol with no overlapping subjects between the training and testing sets. MSTCN achieves F1 scores of 0.9752 on UCI and 0.9470 on WISDM. Conclusion: The proposed MSTCN dominates the other state-of-the-art methods by acquiring high recognition accuracies without requiring any manual feature engineering.

RISTDnet: Robust Infrared Small Target Detection Network

The infrared (IR) small target detection algorithm with a high detection rate, low false alarm rate, and high real-time performance has significant application value in the field of IR remote sensing. IR small targets in complex backgrounds have low contrast and low signal-to-noise ratio (SNR). Therefore, small target detection is more difficult. Traditional IR small target detection is generally implemented by local contrast methods (LCM), nonlocal autocorrelation methods (NAM), and adaptive segmentation. In this letter, a robust infrared small target detection network (RISTDnet) is proposed based on deep learning. In RISTDnet, a feature extraction framework combining handcrafted feature methods and convolutional neural networks is constructed, a mapping network between feature maps and the likelihood of small targets in the image is established, and a threshold is applied on the likelihood map to segment real targets. Experimental results show that the RISTDnet can detect small targets with different sizes and low SNRs in complex backgrounds and have better effectiveness and robustness against existing algorithms.

A novel approach with dual-sampling convolutional neural network for ultrasound image classification of breast tumors

Breast cancer is one of the leading causes of female cancer deaths. Early diagnosis with prophylactic may improve the patients’ prognosis. So far ultrasound (US) imaging has been a popular method in breast cancer diagnosis. However, its accuracy is bounded to traditional handcrafted feature methods and expertise. A novel method, named dual-sampling convolutional neural networks (DSCNNs), was proposed in this paper for the differential diagnosis of breast tumors based on US images. Combining traditional convolutional and residual networks, DSCNN prevented gradient disappearance and degradation. The prediction accuracy was increased by the parallel dual-sampling structure, which can effectively extract potential features from US images. Compared with other advanced deep learning methods and traditional handcrafted feature methods, DSCNN reached the best performance with an accuracy of 91.67% and an area under curve of 0.939. The robustness of the proposed method was also verified by using a public dataset. Moreover, DSCNN was compared with evaluation from three radiologists utilizing US-BI-RADS lexicon categories for overall breast tumors assessment. The result demonstrated that the prediction sensitivity, specificity and accuracy of the DSCNN were higher than those of the radiologist with 10 year experience, suggesting that the DSCNN has the potential to help doctors make judgements in clinic.

Disease named entity recognition using long-short dependencies.

The automatic extraction of disease named entity is a challenging research problem that has attracted attention from the biomedical text mining community. Handcrafted feature methods were employed for this task given a little success since they are limited by the scope of the expert. Lately, deep learning-based methods have been employed to solve this issue. However, most architectures used for this task take into consideration long dependencies only. The proposed method is a two-stage deep neural network model. We start by discovering local dependencies and creating high-level features from word embedding inputs using a deep convolutional neural network. Then we identify long dependencies using a bi-directional recurrent neural network. To solve the problem of unbalanced dataset given by the BMEWO tagging schema and to enforce sequence modeling, we developed a new POS-based tagging schema that subdivides the dominant class into smaller more balanced units. The proposed system was trained and tested on NCBI and achieved an [Formula: see text]-score of 85.59 outperforming the current state-of-the-art methods. Our research results show the effectiveness of using both long and short dependencies. The results also illustrate the benefits of combining different word embedding techniques and the incorporation of morphological features in this task.

AGSF: Adaptive Graph Formulation and Hand-Crafted Graph Spectral Features for Shape Representation

Addressing intra-class variation in high similarity shapes is a challenging task in shape representation due to highly common local and global shape characteristics. Therefore, this paper proposes a new set of hand-crafted features for shape recognition by exploiting spectral features of the underlying graph adaptive connectivity formed by the shape characteristics. To achieve this, the paper proposes a new method for formulating an adaptively connected graph on the nodes of the shape outline. The adaptively connected graph is analysed in terms of its spectral bases followed by extracting hand-crafted adaptive graph spectral features (AGSF) to represent both global and local characteristics of the shape. Experimental evaluation using five 2D shape datasets and four challenging 3D shape datasets shows improvements with respect to the existing hand-crafted feature methods up to 9.14% for 2D shapes and up to 14.02% for 3D shapes. Also for 2D datasets, the proposed AGSF has outperformed the deep learning methods by 17.3%.

The achievement of higher flexibility in multiple-choice-based tests using image classification techniques

In spite of the high accuracy of the existing optical mark reading (OMR) systems and devices, a few restrictions remain existent. In this work, we aim to reduce the restrictions of multiple-choice questions (MCQ) within tests. We use an image registration technique to extract the answer boxes from answer sheets. Unlike other systems that rely on simple image processing steps to recognize the extracted answer boxes, we address the problem from another perspective by training a machine learning classifier to recognize the class of each answer box (i.e., confirmed, crossed out or blank answer). This gives us the ability to deal with a variety of shading and mark patterns, and distinguish between chosen (i.e., confirmed) and canceled answers (i.e., crossed out). All existing machine learning techniques require a large number of examples in order to train a model for classification; therefore, we present a dataset including six real MCQ assessments with different answer sheet templates. We evaluate two strategies of classification: a straightforward approach and a two-stage classifier approach. We test two handcrafted feature methods and a convolutional neural network. At the end, we present an easy-to-use graphical user interface of the proposed system. Compared with existing OMR systems, the proposed system has the least constraints and achieves a high accuracy. We believe that the presented work will further direct the development of OMR systems toward reducing the restrictions of the MCQ tests.

Fingerprint Liveness Detection Using an Improved CNN With Image Scale Equalization

Due to the lack of pre-judgment of fingerprints, fingerprint authentication systems are frequently vulnerable to artificial replicas. Anonymous people can impersonate authorized users to complete various authentication operations, thereby disrupting the order of life and causing tremendous economic losses to society. Therefore, to ensure that authorized users' fingerprint information is not used illegally, one possible anti-spoofing technique, called fingerprint liveness detection (FLD), has been exploited. Compared with the hand-crafted feature methods, the deep convolutional neural network (DCNN) can automatically learn the high-level semantic detail via supervised learning algorithm without any professional background knowledge. However, one disadvantage of most CNNs models is that fixed scale images (e.g., 227 × 227 ) are essential in the input layer. Although the scale problem can be handled by cropping or scaling operations via transforming an image of any scale into a fixed scale, they can easily cause some key texture information loss and image resolution degradation, which will weaken the generalization performance of the classifier model. In this paper, a novel FLD method called an improved DCNN with image scale equalization, has been proposed to preserve texture information and maintain image resolution. Besides, an adaptive learning rate method has been used in this paper. In the performance evaluation, the confusion matrix is applied into FLD for the first time as a performance indicator. The amounts of the experimental results based on the LivDet 2011 and LivDet 2013 data sets also verify that the detection performance of our method is superior to other methods.