Image-based Feature Research Articles

A Robust Hybrid Approach for Malware Detection: Leveraging CNN and LSTM for Encrypted Traffic Analysis

The rapid growth in Internet usage and advancements in network technologies have escalated the risk of network attacks. As the adoption of encryption protocols increases, so does the difficulty in identifying malware within encrypted traffic. Malware represents a significant danger in cyberspace, as it compromises personal data and harms computer systems. Network attacks involve unauthorized access to networks, often aiming to disrupt or damage them, with potentially severe consequences. To counter these threats, researchers, developers, and security experts are constantly innovating new malware detection techniques. Recently, deep learning has gained traction in network security and intrusion detection systems (IDSs), with models such as Convolutional Neural Networks (CNN) and Long Short-Term Memory (LSTM) showing promise in detecting malicious traffic. Despite these advancements, extracting relevant features from diverse malware types remains a challenge. Current solutions demand substantial computational resources and are often inefficient for large datasets. Additionally, existing image-based feature extraction methods consume significant resources. This study tackles these issues by employing a 1D CNN alongside LSTM for the detection and classification of encrypted malicious traffic. Using the Malware Analysis benchmark dataset, which consists of 42,797 malware and 1,079 goodware API call sequences, the proposed model achieved an accuracy of 99.2%, surpassing other state-of-the-art models

DeepAVP-TPPred: identification of antiviral peptides using transformed image-based localized descriptors and binary tree growth algorithm.

Despite the extensive manufacturing of antiviral drugs and vaccination, viral infections continue to be a major human ailment. Antiviral peptides (AVPs) have emerged as potential candidates in the pursuit of novel antiviral drugs. These peptides show vigorous antiviral activity against a diverse range of viruses by targeting different phases of the viral life cycle. Therefore, the accurate prediction of AVPs is an essential yet challenging task. Lately, many machine learning-based approaches have developed for this purpose; however, their limited capabilities in terms of feature engineering, accuracy, and generalization make these methods restricted. In the present study, we aim to develop an efficient machine learning-based approach for the identification of AVPs, referred to as DeepAVP-TPPred, to address the aforementioned problems. First, we extract two new transformed feature sets using our designed image-based feature extraction algorithms and integrate them with an evolutionary information-based feature. Next, these feature sets were optimized using a novel feature selection approach called binary tree growth Algorithm. Finally, the optimal feature space from the training dataset was fed to the deep neural network to build the final classification model. The proposed model DeepAVP-TPPred was tested using stringent 5-fold cross-validation and two independent dataset testing methods, which achieved the maximum performance and showed enhanced efficiency over existing predictors in terms of both accuracy and generalization capabilities. https://github.com/MateeullahKhan/DeepAVP-TPPred.

Multistep validation of a post-ERCP pancreatitis prediction system integrating multimodal data: a multicenter study

Background and study aimsThe impact of various categories of information on the prediction of Post Endoscopic Retrograde Cholangiopancreatography Pancreatitis (PEP) remains uncertain. We aimed to comprehensively investigate the risk factors associated with PEP by constructing and validating a model incorporating multi-modal data through multiple steps. Patients and MethodsA total of 1,916 cases underwent ERCP were retrospectively collected from multiple centers for model construction. Through literature research, 49 electronic health record (EHR) features and one image feature related to PEP were identified. The EHR features were categorized into baseline, diagnosis, technique, and prevent strategies, covering pre-ERCP, intra-ERCP, and peri-ERCP phases. We first incrementally constructed models 1-4 incorporating these four feature categories, then added the image feature into models 1-4 and developed models 5-8. All models underwent testing and comparison using both internal and external test sets. Once the optimal model was selected, we conducted comparison among multiple machine learning algorithms. ResultsCompared with model 2 incorporating baseline and diagnosis features, adding technique and prevent strategies (model 4) greatly improved the sensitivity (63.89% vs 83.33%, p<0.05) and specificity (75.00% vs 85.92%, p<0.001). Similar tendency was observed in internal and external tests. In model 4, the top three features ranked by weight were previous pancreatitis, NSAIDS, and difficult cannulation. The image-based feature has the highest weight in model 5-8. Lastly, model 8 employed Random Forest algorithm showed the best performance. ConclusionsWe firstly developed a multi-modal prediction model for identifying PEP with clinical-acceptable performance. The image and technique features are crucial for PEP prediction.

Optimization Analysis of AI Intelligent English Teaching Strategies Based on Computer Virtual Reality Technology

The Internet of Things and other technology breakthroughs have a big impact on how English is taught (IoTV). The investigation of the actual English teaching process and the identification of student features come first. In order to use the interpolation technique with the student image-based feature detection method, we investigate the IoTV reform. As a result, we find a clever algorithm for IoTV that recognises student features more effectively. Artificial intelligence (AI) is the design technique used to create the intelligence algorithm for pupil feature recognition. A comprehensive multifunctional human-computer interaction system utilizes a variety of input and output streams. In addition to the standard computer keyboard, cursor clicking, and screen touching, the most recent speech and facial recognition technology can be employed for data input. Students learned to orally interact with the robot and act as a guide to various destinations. The Multimodal Interaction System for English Education (MMIEE) is an investigation into the use of network and artificial intelligence (AI) in teaching. Recurrent neural network (Rein RNN)-based Reinforced learning is utilized for the perpectual evaluation model to categorize the questions posed by teachers in terms of their content and kind, and to conduct experimental investigation. The results show that the proposed Rein_RNN model achieves 98.6% of accuracy in 4.3sec of mean evaluation time.

Crack _ PSTU: Crack detection based on the U-Net framework combined with Swin Transformer

Cracks serve as a significant indicator of aging infrastructure. While the surfaces of traditional infrastructures are visually inspected manually, this approach is labor-intensive, highly subjective, and often surpasses the capabilities of available inspection personnel. Although some researchers have employed traditional image processing and machine learning techniques to address these challenges, the presence of irregular crack shapes, complex lighting conditions, and limitations in detecting only a single type of surface crack complicate automatic crack detection. Recent research indicates that deep learning methods are increasingly leading in image-based feature extraction, object detection, and pixel attribute analysis. This paper introduces a method, Crack_PSTU (Pre-trained Swin Transformer U-Net), which leverages the U-Net framework and Swin Transformer model for infrastructure crack classification and detection tasks. Specifically, we utilize a pre-trained Swin Transformer network as the encoder and implement the decoder using convolution, pooling, and other operations, creating a "U"-shaped model architecture. Our experiments encompassed 11,298 crack images from varied scenes, with 9603 designated for training and 1695 for validation. The findings reveal that our approach surpasses other algorithms for this dataset.

DeLiVoTr: Deep and light-weight voxel transformer for 3D object detection

The image-based backbone (feature extraction) networks downsample the feature maps not only to increase the receptive field but also to efficiently detect objects of various scales. The existing feature extraction networks in LiDAR-based 3D object detection tasks follow the feature map downsampling similar to image-based feature extraction networks to increase the receptive field. But, such downsampling of LiDAR feature maps in large-scale autonomous driving scenarios hinder the detection of small size objects, such as pedestrians. To solve this issue we design an architecture that not only maintains the same scale of the feature maps but also the receptive field in the feature extraction network to aid for efficient detection of small size objects. We resort to attention mechanism to build sufficient receptive field and we propose a Deep and Light-weight Voxel Transformer (DeLiVoTr) network with voxel intra- and inter-region transformer modules to extract voxel local and global features respectively. We introduce DeLiVoTr block that uses transformations with expand and reduce strategy to vary the width and depth of the network efficiently. This facilitates to learn wider and deeper voxel representations and enables to use not only smaller dimension for attention mechanism but also a light-weight feed-forward network, facilitating the reduction of parameters and operations. In addition to model scaling, we employ layer-level scaling of DeLiVoTr encoder layers for efficient parameter allocation in each encoder layer instead of fixed number of parameters as in existing approaches. Leveraging layer-level depth and width scaling we formulate three variants of DeLiVoTr network. We conduct extensive experiments and analysis on large-scale Waymo and KITTI datasets. Our network surpasses state-of-the-art methods for detection of small objects (pedestrians) with an inference speed of 20.5 FPS.

Mudslide susceptibility assessment based on a two-channel residual network

In response to the challenges posed by rugged terrain in Yunnan, hindering large-scale mudslide screening efforts, this article introduces a dual-channel Convolutional Neural Network (CNN) constructed using elevation data from historical mudslide-prone valleys (Digital Elevation Model, DEM) and remote sensing imagery. The network is designed to facilitate the comprehensive assessment of potential mudslide hazards in gullies, serving as a crucial tool for early mudslide disaster warning. The model initially employs an enhanced residual structure to extract fundamental features from both types of data. Subsequently, it leverages the SE module and deep separable structure to emphasize the importance of relevant features and expedite model convergence. Finally, the model classifies the gullies under evaluation based on their similarity to gullies where mudslides have previously occurred. Experimental results demonstrate the model’s robust performance in assessing mudflow-prone gullies, achieving an impressive precision rate of up to 81.10% and a recall rate of 82.76%. When applied to evaluate the potential hazard of mudslide gullies across the entirety of Nujiang Prefecture, the model predicts that 87.80% of the mudslide locations are at an extremely high risk. These findings underscore the viability of utilizing image-based gully feature analysis for assessing the hazard levels of mudslide-prone gullies.

A novel multi-biometric technique for verification of secure e-document

Extracting unique and distinctive traits is one of the most important challenges that researchers face, who rely on biometrics to extract exceptional traits for an individual. A large amount of biometric evidence that can be identified and found in various research has been done. In this paper, a biometrics system is proposed that combines the benefits of fingerprinting and uses a novel strategy to combine it with the image-based fingerprint vein feature set. The proposed system is fast and performs effective personal identification by combining both features. The features extracted from the venous print and fingerprint are matched to the nearest neighbors of the authorized person forms to verify the identity of the person. Several experiments have been performed on selected datasets to evaluate the performance of the new biometrics system. The obtained results prove that our proposed system is superior to biometric systems that use the feature of single biometrics. However, our goal is to set up an algorithm that is inexpensive in terms of time complexity while keeping it at the required security levels.

Fault Diagnosis Method for Railway Turnout with Pinball Loss-Based Multiclass Support Matrix Machine

The intelligent maintenance of railway equipment plays a pivotal role in advancing the sustainability of transportation and manufacturing. Railway turnouts, being an essential component of railway infrastructure, often encounter various faults, which present operational challenges. Existing fault diagnosis methods for railway turnouts primarily utilize vectorized monitoring data, interpreted either through vector-based models or distance-based measurements. However, these methods exhibit limited interpretability or are heavily reliant on standard curves, which impairs their performance or restricts their generalizability. To address these limitations, a railway turnouts fault diagnosis method with monitoring signal images and support matrix machine is proposed herein. In addition, a pinball loss-based multiclass support matrix machine (PL-MSMM) is designed to address the noise sensitivity limitations of the multiclass support matrix machine (MSMM). First, the time-series monitoring signals in one dimension are transformed into images in two dimensions. Subsequently, the image-based feature matrix is constructed. Then, the PL-MSMM model is trained using the feature matrix to facilitate the fault diagnosis. The proposed method is evaluated using a real-world operational current dataset, achieving a fault identification accuracy rate of 98.67%. This method outperforms the existing method in terms of accuracy, precision, and F1-score, demonstrating its superiority.

Image-based feature extraction for inline quality assurance and wear classification in high-speed blanking processes

Wear is one of the key factors that determine the efficiency of multi-stage processes that include blanking operations. Since wear in these processes not only causes unplanned downtime but also directly affects product quality, inline detection of wear and its effect on product quality is of major importance. However, current quality assurance (QA) methods are limited to manual offline inspection by operators at predefined intervals, so that 100% inspection of the product and description of the state of wear is not found in industrial practice. The aim of this work is therefore to develop an optical system that enables in-line acquisition of product images and the associated control of blanking-specific quality features up to stroke rates of 300 strokes per minute (spm). In order to make the system attractive to small- and medium-sized enterprises (SME), the system is designed to minimize integration and investment costs using commercially available components. By combining the system with a methodology for extracting blanking-specific features, so-called key performance parameters (KPPs), the condition of the blanked surface as a relevant quality parameter is derived directly from the workpiece image. To demonstrate the transferability of the methodology to industrial applications, two use cases are investigated. In the first case, the KPPs are used directly to determine the quality of the blanked workpiece and are compared with reference measurements. Here, the KPPs are quantified with a mean absolute error of 18 μm compared to a ground truth. In the second case, the KPPs are used to build a machine learning (ML) model to estimate the wear of the blanking tool. Here, an accuracy of 92% is achieved in classifying the actual wear state.

Streamline video-based automatic fabric pattern recognition using Bayesian-optimized convolutional neural network

Examining fabric weave patterns (FWPs) is connected to image-based surface texture feature (STF) acquisition, which can be difficult due to the structural complexity of woven fabrics. Randomly capturing static images may not correlate with the entire STF of a fabric. Traditionally, FWPs analysis is conducted by human vision, which causes an intensive cognitive load. Ultimately, the human vision-based cognitive load leads to ineffective quality inspection and error-prone FWPs analysis results. Given the above challenges, this study proposes a new streamlined video-based FWPs recognition method by incorporating Bayesian-optimized convolutional neural network (Bayes Opt-CNN). Essentially, this method is capable of leveraging the spatiotemporal features of the fabric’s intricate surface structure. In this study, to validate the effectiveness of the proposed method, seven types of fabric structures were captured as streamline videos, which were then converted into sequences of image frames. Subsequently, the Bayesian optimization process was introduced to select the best hyperparameters during CNN-based supervised learning for pattern recognition. The evaluation demonstrates that the proposed method outperforms the benchmark method for identifying FWPs.

Rapid label-free detection of early-stage lung adenocarcinoma and tumor boundary via multiphoton microscopy.

Lung cancer is the most commonly diagnosed cancer and the leading cause of cancer-related deaths in China. Rapid and precise evaluation of tumor tissue during lung cancer surgery can reduce operative time and improve negative-margin assessment, thus increasing disease-free and overall survival rates. This study aimed to explore the potential of label-free multiphoton microscopy (MPM) for imaging adenocarcinoma tissues, detecting histopathological features, and distinguishing between normal and cancerous lung tissues. We showed that second harmonic generation (SHG) signals exhibit significant specificity for collagen fibers, enabling the quantification of collagen features in lung adenocarcinomas. In addition, we developed a collagen score that could be used to distinguish between normal and tumor areas at the tumor boundary, showing good classification performance. Our findings demonstrate that MPM imaging technology combined with an image-based collagen feature extraction method can rapidly and accurately detect early-stage lung adenocarcinoma tissues. This article is protected by copyright. All rights reserved.

HoLoCo: Holistic and local contrastive learning network for multi-exposure image fusion

Multi-exposure image fusion (MEF) targets to integrate multiple shots with different exposures and generates a single higher dynamic image than each. Existing deep learning-based MEF approaches only adopt reference high dynamic images (HDR) as positive samples to guide the training of fusion networks. However, simply relying on these positive samples are difficult to find the optimal parameters for the network as a whole. Thus, the structure or texture information is blurred or missed in the generated HDR results. Moreover, few approaches attempted to prevent illumination degeneration during the fusion process, resulting in poor color saturation on their fused results. To address such limitations, in this paper, we introduce a novel holistic and local constraint built upon contrastive learning, namely HoLoCo, to discover the intrinsic information of both source LDR images and the reference HDR one. In this manner, the generated fused images are pulled to the HDR image and pushed away from the LDR source images in both image-based and path-based latent feature spaces. Besides, inspired by Retinex theory, we propose a color correction module (CCM) to refine illumination features. CCM involves dual streams that can collaborate to ensure the nature of color information and details consistency. Extensive experiments on two datasets show that our HoLoCo can continuously generate visual-appealing HDR results with precise detail and vivid color rendition, performing favorably against the state-of-the-art MEF approaches. Source code is available in github https://github.com/JinyuanLiu-CV/HoLoCo.

An efficient pattern-based approach for insider threat classification using the image-based feature representation

Insider threats are expensive, difficult to detect, and sadly, on the rise. Despite significant research efforts, existing approaches are inadequate in accuracy and precision. They also suffer from a high false-positive rate in detecting insider attacks due to the heterogeneous nature of available insider threat data. Researchers have attempted an image-based approach for detecting insider threats to overcome the challenges from existing techniques. Most existing image-based approaches utilized Convolutional neural networks (CNN) to detect insider threats. Moreover, the CNN-based model lost the important user behavioral features due to the pooling operation. Also, they are often unsuitable for predictive modeling with features that lack spatial correlations. Aiming to address this issue, the Wavelet convolutional neural network (WCNN) is proposed. The WCNN model takes advantage of spectral and spatial analysis to classify insider threats using image-based feature representations. The proposed approach combines the scenario-specific single-day features from the user activity logs into a one-dimensional feature vector. It is then represented as images that reveal visual patterns effectively to identify malicious insiders using WCNN. In addition, the proposed approach adopts the SMOTEENN sampling technique to solve the class imbalance problem. The performance of the proposed approach is evaluated on the benchmark dataset. Experimental results show the improvement of the proposed approach over the current state-of-the-art techniques in terms of classification accuracy (97.19%), AUC (97.30%), and low false positives to identify malicious insiders.

An AI-Aided Diagnostic Framework for Hematologic Neoplasms Based on Morphologic Features and Medical Expertise

A morphologic examination is essential for the diagnosis of hematological diseases. However, its conventional manual operation is time-consuming and laborious. Herein, we attempt to establish an artificial intelligence (AI)-aided diagnostic framework integrating medical expertise. This framework acts as a virtual hematological morphologist (VHM) for diagnosing hematological neoplasms. Two datasets were established as follows: An image dataset was used to train the Faster Region-based Convolutional Neural Network to develop an image-based morphologic feature extraction model. A case dataset containing retrospective morphologic diagnostic data was used to train a support vector machine algorithm to develop a feature-based case identification model based on diagnostic criteria. Integrating these 2 models established a whole-process AI-aided diagnostic framework, namely, VHM, and a 2-stage strategy was applied to practice case diagnosis. The recall and precision of VHM in bone marrow cell classification were 94.65% and 93.95%, respectively. The balanced accuracy, sensitivity, and specificity of VHM were 97.16%, 99.09%, and 92%, respectively, in the differential diagnosis of normal and abnormal cases, and 99.23%, 97.96%, and 100%, respectively, in the precise diagnosis of chronic myelogenous leukemia in chronic phase. This work represents the first attempt, to our knowledge, to extract multimodal morphologic features and to integrate a feature-based case diagnosis model for designing a comprehensive AI-aided morphologic diagnostic framework. The performance of our knowledge-based framework was superior to that of the widely used end-to-end AI-based diagnostic framework in terms of testing accuracy (96.88% vs 68.75%) or generalization ability (97.11% vs 68.75%) in differentiating normal and abnormal cases. The remarkable advantage of VHM is that it follows the logic of clinical diagnostic procedures, making it a reliable and interpretable hematological diagnostic tool.

Spatial patterns of the cap-binding complex eIF4F in human melanoma cells

As a central node of protein synthesis, the cap-binding complex, eukaryotic translation initiation factor 4 F (eIF4F), is involved in cell homeostasis, development and tumorigenesis. A large body of literature exists on the regulation and function of eIF4F in cancer cells, however the intracellular localization patterns of this complex are largely unknown. Since different subsets of mRNAs are translated in distinct subcellular compartments, understanding the distribution of translation initiation factors in the cell is of major interest. Here, we developed an in situ detection method for eIF4F at the single cell level. By using an image-based spot feature analysis pipeline as well as supervised machine learning, we identify five distinct spatial patterns of the eIF4F translation initiation complex in human melanoma cells. The quantity of eIF4F complex per cell correlated with the global mRNA translation activity, and its variation is dynamically regulated by cell state or extracellular stimuli. In contrast, the spatial patterns of eIF4F complexes at the single cell level could distinguish melanoma cells harboring different oncogenic driver mutations. This suggests that different tumorigenic contexts differentially regulate the subcellular localization of mRNA translation, with specific localization of eIF4F potentially associated with melanoma cell chemoresistance.

Few-Shot Learning Enables Population-Scale Analysis of Leaf Traits in Populus trichocarpa.

Plant phenotyping is typically a time-consuming and expensive endeavor, requiring large groups of researchers to meticulously measure biologically relevant plant traits, and is the main bottleneck in understanding plant adaptation and the genetic architecture underlying complex traits at population scale. In this work, we address these challenges by leveraging few-shot learning with convolutional neural networks to segment the leaf body and visible venation of 2,906 Populus trichocarpa leaf images obtained in the field. In contrast to previous methods, our approach (a) does not require experimental or image preprocessing, (b) uses the raw RGB images at full resolution, and (c) requires very few samples for training (e.g., just 8 images for vein segmentation). Traits relating to leaf morphology and vein topology are extracted from the resulting segmentations using traditional open-source image-processing tools, validated using real-world physical measurements, and used to conduct a genome-wide association study to identify genes controlling the traits. In this way, the current work is designed to provide the plant phenotyping community with (a) methods for fast and accurate image-based feature extraction that require minimal training data and (b) a new population-scale dataset, including 68 different leaf phenotypes, for domain scientists and machine learning researchers. All of the few-shot learning code, data, and results are made publicly available.

HISMD: A Novel Immune Subtyping System for HNSCC

Immune subtyping is an important way to reveal immune heterogeneity, which may contribute to the diversity of the progression and treatment in head and neck squamous cell carcinoma (HNSCC). However, reported immune subtypes mainly focus on levels of immune infiltration and are mostly based on a mono-omics profile. This study aimed to identify a comprehensive immune subtype for HNSCC via multi-omics clustering and build a novel subtype prediction system for clinical application. Data were obtained from The Cancer Genome Atlas database and our independent multicenter cohort. Multi-omics clustering was performed to identify 3 clusters of 499 patients in The Cancer Genome Atlas based on immune-related gene expression and somatic mutations. The immune characteristics and biological features of the obtained clusters were revealed by bioinformatics, and 3 immune subtypes were identified: 1) adaptive immune activation subtype predominantly enriched in T cells, 2) innate immune activation subtype predominantly enriched in macrophages, and 3) immune desert subtype. Subsequently, the clinical implications of each subtype were analyzed per clinical epidemiology. We found that adaptive immune activation showed better survival outcomes and had a similar response to chemotherapy with innate immune activation, whereas immune desert might be relatively resistant to chemotherapy. Moreover, a subtype prediction system was developed by deep learning with whole slide images and named HISMD: HNSCC Immune Subtypes via Multi-omics and Deep Learning. We endowed HISMD with interpretability through image-based key feature extraction. The clinical implications, biological significances, and predictive stability of HISMD were successfully verified by using our independent multicenter cohort data set. In summary, this study revealed the immune heterogeneity of HNSCC and obtained a novel, highly accurate, and interpretable immune subtyping prediction system. For clinical implementation in the future, additional validation and utility studies are warranted.

High-content phenotyping of Parkinson's disease patient stem cell-derived midbrain dopaminergic neurons using machine learning classification.

SummaryCombining multiple Parkinson's disease (PD) relevant cellular phenotypes might increase the accuracy of midbrain dopaminergic neuron (mDAN) in vitro models. We differentiated patient-derived induced pluripotent stem cells (iPSCs) with a LRRK2 G2019S mutation, isogenic control, and genetically unrelated iPSCs into mDANs. Using automated fluorescence microscopy in 384-well-plate format, we identified elevated levels of α-synuclein (αSyn) and serine 129 phosphorylation, reduced dendritic complexity, and mitochondrial dysfunction. Next, we measured additional image-based phenotypes and used machine learning (ML) to accurately classify mDANs according to their genotype. Additionally, we show that chemical compound treatments, targeting LRRK2 kinase activity or αSyn levels, are detectable when using ML classification based on multiple image-based phenotypes. We validated our approach using a second isogenic patient-derived SNCA gene triplication mDAN model which overexpresses αSyn. This phenotyping and classification strategy improves the practical exploitability of mDANs for disease modeling and the identification of novel LRRK2-associated drug targets.

The visual measurement system in the parachute used on the Tianwen-1 Mars exploration mission

ABSTRACT The parachute of the probe plays an important role in the deceleration of the Mars landing mission. This paper provides an overview of the visual measurement system of the parachute in Tianwen-1 exploration mission. Using two monitoring cameras, the system has captured stereo images during the parachute deployment. The morphological and motion parameters of the parachute are reconstructed by image-based feature tracking and 3D reconstruction methods. To improve the image quality, a super-resolution (SR) reconstruction method based on sparse dictionary coding is proposed. In addition, we use a multi-scale feature tracking method to track the feature points and update the motion parameters. The system has been verified by the test on the Earth and successfully applied on the onboard images. The reconstructed parachute parameters are important in the post-flight performance assessment, and the research results can be used as a regression analysis means for the structural design.