Pre-trained Detector Research Articles

A human-in-the-loop method for pulmonary nodule detection in CT scans

Automated pulmonary nodule detection using computed tomography scans is vital in the early diagnosis of lung cancer. Although extensive well-performed methods have been proposed for this task, they suffer from the domain shift issue between training and test images. Unsupervised domain adaptation (UDA) methods provide a promising means to mitigate the domain variance; however, their performance is still limited since no target domain supervision is introduced. To make the pulmonary nodule detection algorithm more applicable in clinical practice and further boost the performance across domains, we propose a human-in-the-loop method in a semi-supervised fashion to enhance the model generalization ability when transferred from source domain to target domain. Specifically, we first train a detector model on source domain, and then the pre-trained detector is utilized with our proposed uncertainty-guided sample selection scheme (USSS) to find a few target domain samples worth annotating most and obtain their human annotations. Finally, the annotated and the rest unlabeled target domain samples are used together to refine the pre-trained model via our proposed zoom-in and zoom-out constraint (ZZC) strategy. We evaluate our method on the Nodule Analysis 2016 (LUNA16) and TianChi datasets. Experimental results show that our method surpasses recent competitive methods on source domain and also achieves surprising performance on target domain.

Beyond the Label Itself: Latent Labels Enhance Semi-supervised Point Cloud Panoptic Segmentation

As the exorbitant expense of labeling autopilot datasets and the growing trend of utilizing unlabeled data, semi-supervised segmentation on point clouds becomes increasingly imperative. Intuitively, finding out more ``unspoken words'' (i.e., latent instance information) beyond the label itself should be helpful to improve performance. In this paper, we discover two types of latent labels behind the displayed label embedded in LiDAR and image data. First, in the LiDAR Branch, we propose a novel augmentation, Cylinder-Mix, which is able to augment more yet reliable samples for training. Second, in the Image Branch, we propose the Instance Position-scale Learning (IPSL) Module to learn and fuse the information of instance position and scale, which is from a 2D pre-trained detector and a type of latent label obtained from 3D to 2D projection. Finally, the two latent labels are embedded into the multi-modal panoptic segmentation network. The ablation of the IPSL module demonstrates its robust adaptability, and the experiments evaluated on SemanticKITTI and nuScenes demonstrate that our model outperforms the state-of-the-art method, LaserMix.

Domain Adaptive Subterranean 3D Pedestrian Detection via Instance Transfer and Confidence Guidance

With the exploration of subterranean scenes, determining how to ensure the safety of subterranean pedestrians has gradually become a hot research topic. Considering the poor illumination and lack of annotated data in subterranean scenes, it is essential to explore the LiDAR-based domain adaptive detectors for localizing the spatial location of pedestrians, thus providing instruction for evacuation and rescue. In this paper, a novel domain adaptive subterranean 3D pedestrian detection method is proposed to adapt pre-trained detectors from the annotated road scenes to the unannotated subterranean scenes. Specifically, an instance transfer-based scene updating strategy is designed to update the subterranean scenes by transferring instances from the road scenes to the subterranean scenes, aiming to create sufficient high-quality pseudo labels for fine-tuning the pre-trained detector. In addition, a pseudo label confidence-guided learning mechanism is constructed to fully utilize pseudo labels of different qualities under the guidance of confidence scores. Extensive experiments validate the superiority of our proposed domain adaptive subterranean 3D pedestrian detection method.

Cybersecurity Threat Detection using Machine Learning and Network Analysis

Cybercriminals continually develop innovative strategies to confound and frustrate their victims, necessitating constant vigilance to protect the availability, confidentiality, and integrity of digital systems. Machine learning (ML) has emerged as a powerful technique for intelligent cyber analysis, enabling proactive defenses by studying recurring patterns of successful attacks. However, two significant drawbacks hinder the widespread adoption of ML in security analysis: high computing overheads and the need for specialized frameworks. This study aims to quantify the extent to which a hub can enhance ecosystem safety. Typical cyberattacks were executed on an Internet of Things (IoT) network within a smart house to validate the hub's efficacy. Furthermore, the resistance of the intrusion detection system (IDS) to adversarial machine learning (AML) attacks was investigated, where models are targeted with adversarial samples exploiting weaknesses in the pre-trained detector.

Cybersecurity Threat Detection using Machine Learning and Network Analysis

Cybercriminals continually develop innovative strategies to confound and frustrate their victims, necessitating constant vigilance to protect the availability, confidentiality, and integrity of digital systems. Machine learning (ML) has emerged as a powerful technique for intelligent cyber analysis, enabling proactive defenses by studying recurring patterns of successful attacks. However, two significant drawbacks hinder the widespread adoption of ML in security analysis: high computing overheads and the need for specialized frameworks. This study aims to quantify the extent to which a hub can enhance ecosystem safety. Typical cyberattacks were executed on an Internet of Things (IoT) network within a smart house to validate the hub's efficacy. Furthermore, the resistance of the intrusion detection system (IDS) to adversarial machine learning (AML) attacks was investigated, where models are targeted with adversarial samples exploiting weaknesses in the pre-trained detector.

Mutually Improved Endoscopic Image Synthesis and Landmark Detection in Unpaired Image-to-Image Translation.

The CycleGAN framework allows for unsupervised image-to-image translation of unpaired data. In a scenario of surgical training on a physical surgical simulator, this method can be used to transform endoscopic images of phantoms into images which more closely resemble the intra-operative appearance of the same surgical target structure. This can be viewed as a novel augmented reality approach, which we coined Hyperrealism in previous work. In this use case, it is of paramount importance to display objects like needles, sutures or instruments consistent in both domains while altering the style to a more tissue-like appearance. Segmentation of these objects would allow for a direct transfer, however, contouring of these, partly tiny and thin foreground objects is cumbersome and perhaps inaccurate. Instead, we propose to use landmark detection on the points when sutures pass into the tissue. This objective is directly incorporated into a CycleGAN framework by treating the performance of pre-trained detector models as an additional optimization goal. We show that a task defined on these sparse landmark labels improves consistency of synthesis by the generator network in both domains. Comparing a baseline CycleGAN architecture to our proposed extension (DetCycleGAN), mean precision (PPV) improved by +61.32, mean sensitivity (TPR) by +37.91, and mean F1 score by +0.4743. Furthermore, it could be shown that by dataset fusion, generated intra-operative images can be leveraged as additional training data for the detection network itself.

Joint Commonsense and Relation Reasoning for Image and Video Captioning

Exploiting relationships between objects for image and video captioning has received increasing attention. Most existing methods depend heavily on pre-trained detectors of objects and their relationships, and thus may not work well when facing detection challenges such as heavy occlusion, tiny-size objects, and long-tail classes. In this paper, we propose a joint commonsense and relation reasoning method that exploits prior knowledge for image and video captioning without relying on any detectors. The prior knowledge provides semantic correlations and constraints between objects, serving as guidance to build semantic graphs that summarize object relationships, some of which cannot be directly perceived from images or videos. Particularly, our method is implemented by an iterative learning algorithm that alternates between 1) commonsense reasoning for embedding visual regions into the semantic space to build a semantic graph and 2) relation reasoning for encoding semantic graphs to generate sentences. Experiments on several benchmark datasets validate the effectiveness of our prior knowledge-based approach.

A novel data augmentation scheme for pedestrian detection with attribute preserving GAN

Recently pedestrian detection has progressed significantly. However, detecting pedestrians of small scale or in heavy occlusions is still notoriously difficult. Besides, the generalization ability of pre-trained detectors across different datasets remains to be improved. Both of these issues can be attributed to insufficient training data coverage. To cope with this, we present an efficient data augmentation scheme by transferring pedestrians from other datasets into the target scene with a novel Attribute Preserving Generative Adversarial Networks (APGAN). The proposed methodology consists of two steps: pedestrian embedding and style transfer. The former step can simulate pedestrian images of various scale and occlusion, in any pose or background, thus greatly promoting the data variation. The latter step aims to make the generated samples more realistic while guarantee the data coverage. To achieve this goal, we propose APGAN, which pursues both good visual quality and attribute preserving after style transfer. With the proposed method, we can make effective sample augmentations to improve the generalization ability of the trained detectors and enhance its robustness to scale change and occlusions. Extensive experiment results validate the effectiveness and advantages of our method.

Action recognition of earthmoving excavators based on sequential pattern analysis of visual features and operation cycles

Abstract This paper proposes a vision-based action recognition framework that considers the sequential working patterns of earthmoving excavators for automated cycle time and productivity analysis. The sequential patterns of visual features and operation cycles are incorporated into the action recognition framework, which includes three main processes: excavator detection, excavator tracking, and excavator action recognition. The first process uses a pre-trained detector to localize earthmoving excavators on sites from all of captured images. Next, the detection results are associated and the excavators are tracked by Tracking-Learning-Detection algorithm. Lastly, to recognize the operation types of the excavators, their sequential patterns of visual features and operation cycles are modeled and trained with a hybrid deep learning algorithm, i.e., Convolutional Neural Networks and Double-layer Long Short Term Memory. Three experiments were performed to validate the proposed framework and confirm the positive effects of sequential modeling: (1) an experiment without sequential pattern analysis, (2) an experiment with the sequential pattern analysis of visual features, and (3) an experiment with the sequential pattern analysis of visual features and operation cycles. In the experiments, the research team used a total of 72,365 images collected from actual earthmoving sites. The average accuracies of the excavator action recognition in the three cases were 79.8%, 90.9%, and 93.8% respectively. The results demonstrated the applicability of the proposed framework and the significant positive impacts of sequential pattern modeling on the recognition performance. This research contributed to the identification of critical elements that explain sequential working patterns and to the development of a novel vision-based action recognition framework. In addition, the findings of this study can help to automate cycle time analysis and productivity monitoring of earthmoving excavators.

Cascade of Boolean detector combinations

This paper considers a scenario when we have multiple pre-trained detectors for detecting an event and a small dataset for training a combined detection system. We build the combined detector as a Boolean function of thresholded detector scores and implement it as a binary classification cascade. The cascade structure is computationally efficient by providing the possibility to early termination. For the proposed Boolean combination function, the computational load of classification is reduced whenever the function becomes determinate before all the component detectors have been utilized. We also propose an algorithm, which selects all the needed thresholds for the component detectors within the proposed Boolean combination. We present results on two audio-visual datasets, which prove the efficiency of the proposed combination framework. We achieve state-of-the-art accuracy with substantially reduced computation time in laughter detection task, and our algorithm finds better thresholds for the component detectors within the Boolean combination than the other algorithms found in the literature.

Detection guided deconvolutional network for hierarchical feature learning

Deep learning models have gained significant interest as a way of building hierarchical image representation. However, current models still perform far behind human vision system because of the lack of selective property, the lack of high-level guidance for learning and the weakness to learn from few examples. To address these problems, we propose a detection-guided hierarchical learning algorithm for image representation. First, we train a multi-layer deconvolutional network in an unsupervised bottom-up scheme. During the training process, we use each raw image as an input, and decompose an image using multiple alternating layers of non-negative convolutional sparse coding and max-pooling. Inspired from the observation that the filters in top layer can be selectively activated by different high-level structures of images, i.e., one or partial filters should correspond to a particular object class, we update the filters in network by minimizing the reconstruction errors of the corresponding feature maps with respect to certain object detection maps obtained by a set of pre-trained detectors. With the fine-tuned network, we can extract the features of given images in a purely unsupervised way with no need of detectors. We evaluate the proposed feature representation on the task of object recognition, for which an SVM classifier with spatial pyramid matching kernel is used. Experiments on the datasets of PASCAL VOC 2007, Caltech-101 and Caltech-256 demonstrate that our approach outperforms some recent hierarchical feature descriptors as well as classical hand-crafted features.

An efficient human detection method for multi-pedestrian tracking

Traditional human detection using pre-trained detectors tends to be computationally intensive for time-critical tracking tasks, and the detection rate is prone to be unsatisfying when occlusion, motion blur and body deformation occur frequently. A spatial-confidential proposal filtering method (SCPF) is proposed for efficient and accurate human detection. It consists of two filtering phases: spatial proposal filtering and confidential proposal filtering. A compact spatial proposal is generated in the first phase to minimize the search space to reduce the computation cost. The human detector only estimates the confidence scores of the candidate search regions accepted by the spatial proposal instead of global scanning. At the second phase, each candidate search region is assigned with a supplementary confidence score according to their reliability estimated by the confidential proposal to reduce missing detections. The performance of the SCPF method is verified by extensive tests on several video sequences from available public datasets. Both quantitatively and qualitatively experimental results indicate that the proposed method can highly improve the efficiency and the accuracy of human detection.

Scene transformation for detector adaptation

This paper focuses on detecting vehicles in different target scenes with the same pre-trained detector which is very challenging due to view variations. To address this problem, we propose a novel approach for detection adaptation based on scene transformation, which contributes in both view transformation and automatic parameter estimation. Instead of modifying the pre-trained detectors, we transform scenes into frontal/rear view handling with pitch and yaw view variations. Without human interactions but only some general prior knowledge, the transformation parameters are automatically initialized, and then online optimized with spatial–temporal voting, which guarantees that the transformation matches the pre-trained detector. Since there is no need of labeling new samples and manual camera calibration, our approach can considerably reduce manual interactions. Experiments on challenging real-world videos demonstrate that our approach achieves significant improvements over the pre-trained detector, and it is even comparable to the performance of the detector trained on fully labeled sequences.