Single-shot Detection Research Articles

Real-time embedded object detection and tracking system in Zynq SoC

With the increasing application of computer vision technology in autonomous driving, robot, and other mobile devices, more and more attention has been paid to the implementation of target detection and tracking algorithms on embedded platforms. The real-time performance and robustness of algorithms are two hot research topics and challenges in this field. In order to solve the problems of poor real-time tracking performance of embedded systems using convolutional neural networks and low robustness of tracking algorithms for complex scenes, this paper proposes a fast and accurate real-time video detection and tracking algorithm suitable for embedded systems. The algorithm combines the object detection model of single-shot multibox detection in deep convolution networks and the kernel correlation filters tracking algorithm, what is more, it accelerates the single-shot multibox detection model using field-programmable gate arrays, which satisfies the real-time performance of the algorithm on the embedded platform. To solve the problem of model contamination after the kernel correlation filters algorithm fails to track in complex scenes, an improvement in the validity detection mechanism of tracking results is proposed that solves the problem of the traditional kernel correlation filters algorithm not being able to robustly track for a long time. In order to solve the problem that the missed rate of the single-shot multibox detection model is high under the conditions of motion blur or illumination variation, a strategy to reduce missed rate is proposed that effectively reduces the missed detection. The experimental results on the embedded platform show that the algorithm can achieve real-time tracking of the object in the video and can automatically reposition the object to continue tracking after the object tracking fails.

Single-shot detection of 8 unique monochrome fringe patterns representing 4 distinct directions via multispectral fringe projection profilometry

Spatial resolution in three-dimensional fringe projection profilometry is determined in large part by the number and spacing of fringes projected onto an object. Due to the intensity-based nature of fringe projection profilometry, fringe patterns must be generated in succession, which is time-consuming. As a result, the surface features of highly dynamic objects are difficult to measure. Here, we introduce multispectral fringe projection profilometry, a novel method that utilizes multispectral illumination to project a multispectral fringe pattern onto an object combined with a multispectral camera to detect the deformation of the fringe patterns due to the object. The multispectral camera enables the detection of 8 unique monochrome fringe patterns representing 4 distinct directions in a single snapshot. Furthermore, for each direction, the camera detects two π-phase shifted fringe patterns. Each pair of fringe patterns can be differenced to generate a differential fringe pattern that corrects for illumination offsets and mitigates the effects of glare from highly reflective surfaces. The new multispectral method solves many practical problems related to conventional fringe projection profilometry and doubles the effective spatial resolution. The method is suitable for high-quality fast 3D profilometry at video frame rates.

All-optical sampling of few-cycle infrared pulses using tunneling in a solid

Recent developments in ultrafast laser technology have resulted in novel few-cycle sources in the mid-infrared. Accurately characterizing the time-dependent intensities and electric field waveforms of such laser pulses is essential to their applications in strong-field physics and attosecond pulse generation, but this remains a challenge. Recently, it was shown that tunnel ionization can provide an ultrafast temporal “gate” for characterizing high-energy few-cycle laser waveforms capable of ionizing air. Here, we show that tunneling and multiphoton excitation in a dielectric solid can provide a means to measure lower-energy and longer-wavelength pulses, and we apply the technique to characterize microjoule-level near- and mid-infrared pulses. The method lends itself to both all-optical and on-chip detection of laser waveforms, as well as single-shot detection geometries.

Comparative analysis of deep learning image detection algorithms

A computer views all kinds of visual media as an array of numerical values. As a consequence of this approach, they require image processing algorithms to inspect contents of images. This project compares 3 major image processing algorithms: Single Shot Detection (SSD), Faster Region based Convolutional Neural Networks (Faster R-CNN), and You Only Look Once (YOLO) to find the fastest and most efficient of three. In this comparative analysis, using the Microsoft COCO (Common Object in Context) dataset, the performance of these three algorithms is evaluated and their strengths and limitations are analysed based on parameters such as accuracy, precision and F1 score. From the results of the analysis, it can be concluded that the suitability of any of the algorithms over the other two is dictated to a great extent by the use cases they are applied in. In an identical testing environment, YOLO-v3 outperforms SSD and Faster R-CNN, making it the best of the three algorithms.

CNN Training Using 3D Virtual Models for Assisted Assembly with Mixed Reality and Collaborative Robots

The assisted assembly of customized products supported by collaborative robots combined with mixed reality devices is the current trend in the Industry 4.0 concept. This article introduces an experimental work cell with the implementation of the assisted assembly process for customized cam switches as a case study. The research is aimed to design a methodology for this complex task with full digitalization and transformation data to digital twin models from all vision systems. Recognition of position and orientation of assembled parts during manual assembly are marked and checked by convolutional neural network (CNN) model. Training of CNN was based on a new approach using virtual training samples with single shot detection and instance segmentation. The trained CNN model was transferred to an embedded artificial processing unit with a high-resolution camera sensor. The embedded device redistributes data with parts detected position and orientation into mixed reality devices and collaborative robot. This approach to assisted assembly using mixed reality, collaborative robot, vision systems, and CNN models can significantly decrease assembly and training time in real production.

Fast COVID-19 Detection of Chest X-Ray Images Using Single Shot Detection MobileNet Convolutional Neural Networks

COVID-19 is a new disease with a very rapid and tremendous spread. The most important thing needed now is a COVID-19 early detection system that is fast, easy to use, portable, and affordable. Various studies on desktop-based detection using Convolutional Neural Networks have been successfully conducted. However, no research has yet applied mobile-based detection, which requires low computational cost. Therefore, this research aims to produce a COVID-19 early detection system based on chest X-ray images using Convolutional Neural Network models to be deployed in mobile applications. It is expected that the proposed Convolutional Neural Network models can detect COVID-19 quickly, economically, and accurately. The used architecture is MobileNet's Single Shot Detection. The advantage of the Single Shot Detection MobileNet models is that they are lightweight to be applied to mobile-based devices. Therefore, these two versions will also be tested, which one is better. Both models have successfully detected COVID-19, normal, and viral pneumonia conditions with an average overall accuracy of 93.24% based on the test results. The Single Shot Detection MobileNet V1 model can detect COVID-19 with an average accuracy of 83.7%, while the Single Shot Detection MobileNet V2 Single Shot Detection model can detect COVID-19 with an average accuracy of 87.5%. Based on the research conducted, it can be concluded that the approach to detecting chest X-rays of COVID-19 can be detected using the MobileNet Single Shot Detection model. Besides, the V2 model shows better performance than the V1. Therefore, this model can be applied to increase the speed and affordability of COVID-19 detection.

Measuring loblolly pine crowns with drone imagery through deep learning

In modeling forest stand growth and yield, crown width, a measure for stand density, is among the parameters that allows for estimating stand timber volumes. However, accurately measuring tree crown size in the field, in particular for mature trees, is challenging. This study demonstrated a novel method of applying machine learning algorithms to aerial imagery acquired by an unmanned aerial vehicle (UAV) to identify tree crowns and their widths in two loblolly pine plantations in eastern Texas, USA. An ortho mosaic image derived from UAV-captured aerial photos was acquired for each plantation (a young stand before canopy closure, a mature stand with a closed canopy). For each site, the images were split into two subsets: one for training and one for validation purposes. Three widely used object detection methods in deep learning, the Faster region-based convolutional neural network (Faster R-CNN), You Only Look Once version 3 (YOLOv3), and single shot detection (SSD), were applied to the training data, respectively. Each was used to train the model for performing crown recognition and crown extraction. Each model output was evaluated using an independent test data set. All three models were successful in detecting tree crowns with an accuracy greater than 93%, except the Faster R-CNN model that failed on the mature site. On the young site, the SSD model performed the best for crown extraction with a coefficient of determination (R2) of 0.92, followed by Faster R-CNN (0.88) and YOLOv3 (0.62). As to the mature site, the SSD model achieved a R2 as high as 0.94, follow by YOLOv3 (0.69). These deep leaning algorithms, in particular the SSD model, proved to be successfully in identifying tree crowns and estimating crown widths with satisfactory accuracy. For the purpose of forest inventory on loblolly pine plantations, using UAV-captured imagery paired with the SSD object detention application is a cost-effective alternative to traditional ground measurement.

Single-shot weakly-supervised object detection guided by empirical saliency model

Even though weakly-supervised object detection (WSOD) has become an effective method to relieve the heavy work of labeling, there are still difficult problems to be solved. WSOD method represented by a Multiple Instance Learning (MIL) have some common problems including running slowly and focusing on discriminative parts rather than the whole object, which will lead to false detection. To improve the efficiency and accuracy, we propose a single-shot weakly-supervised object detection model guided by empirical saliency model (SSWOD). As human vision always focuses on the most attracting parts of the image, saliency maps can usually guide our model to locate the most promising object areas. By this way, our model takes the saliency areas as pseudo ground-truths to realize the WSOD task with only class labels. Moreover, empirical saliency is designed to refine the pseudo ground-truth and improve the detection. Our new framework not only realizes a one-step detection without region proposals, but also reduces computational consumption. Experiments on PASCAL VOC 2007 & 2012 benchmarks demonstrate that SSWOD is 8 times faster and 5 times smaller than previous approaches, surpassing the state-of-the-art WSOD methods by 6.1% mean average precision (mAP).

Single-shot number-resolved detection of microwave photons with error mitigation

Single-photon detectors are ubiquitous and integral components of photonic quantum cryptography, communication, and computation. Many applications, however, require not only detecting the presence of any photons, but distinguishing the number present with a single shot. Here, we implement a single-shot, high-fidelity photon number-resolving detector of up to 15 microwave photons in a cavity-qubit circuit QED platform. This detector functions by measuring a series of generalized parity operators which make up the bits in the binary decomposition of the photon number. Our protocol consists of successive, independent measurements of each bit by entangling the ancilla with the cavity, then reading out and resetting the ancilla. Photon loss and ancilla readout errors can flip one or more bits, causing nontrivial errors in the outcome, but these errors have a traceable form which can be captured in a simple hidden Markov model. Relying on the independence of each bit measurement, we mitigate biases in ensembles of measurements, showing good agreement with the predictions of the model. The mitigation improves the average total variation distance error of Fock states from $13.5\%$ to $1.1\%$. We also show that the mitigation is efficiently scalable to an $M$-mode system provided that the errors are independent and sufficiently small. Our work motivates the development of new algorithms that utilize single-shot, high-fidelity PNR detectors.

Single-shot detection of four-level phase modulated signals using inter-pixel crosstalk for holographic data storage

We report a simple and stable phase detection method for holographic data storage systems. Our method needs only one image acquisition, and no additional light waves or interferometric measurements are required. The phase information is retrieved from intensities at pixel boundaries where inter-pixel crosstalk occurs owing to the spatial filtering in the Fourier plane of the input image. We numerically and experimentally demonstrate that our method can retrieve the phase signal with a sufficiently low error rate for holographic storage. We also compare the storage density of the proposed method to the conventional phase detection method.

Single-Shot Detection Based on Cyclic Attention

The shallow feature map of the single-shot detector (SSD) is not always conducive to enhancing the recognition precision for a small object because of the lack of contextual information. In this research, a single-shot detection algorithm based on cyclic attention (CA-SSD) is proposed to construct a fast and accurate detector that efficiently obtains full-image contextual information. Our network is constructed by integrating ResNet-34 and proposed novel cyclic attention blocks. This type of building block aggregates different transformations, one of which includes an attention module that uses a long but narrow pooling kernel to acquire horizontal and vertical contextual information for each pixel of all pixels. Each pixel eventually captures the full-image dependencies by following an even further cyclic operation. Our design considers the variability of the gradient, which not only improves the reliability of the cyclic attention block but also cuts the number of parameters for computation. Additionally, by exploring the effects of the stem block and its stride on the performance of ResNet-based SSD algorithms, our network retains more detailed information. For an input size of 300 × 300, CA-SSD attained 82.5% mAP on PASCAL VOC 2007 test, 78.4% mAP on PASCAL VOC 2012 test, and 32.7% mAP on MS COCO. Experimental results achieved with CA-SSD surpass the best results achieved with the traditional SSD and other advanced object detection algorithms while real-time speed is maintained.

Parallel Residual Bi-Fusion Feature Pyramid Network for Accurate Single-Shot Object Detection.

This paper proposes the Parallel Residual Bi-Fusion Feature Pyramid Network (PRB-FPN) for fast and accurate single-shot object detection. Feature Pyramid (FP) is widely used in recent visual detection, however the top-down pathway of FP cannot preserve accurate localization due to pooling shifting. The advantage of FP is weakened as deeper backbones with more layers are used. In addition, it cannot keep up accurate detection of both small and large objects at the same time. To address these issues, we propose a new parallel FP structure with bi-directional (top-down and bottom-up) fusion and associated improvements to retain high-quality features for accurate localization. We provide the following design improvements: 1) parallel bifusion FP structure with a bottom-up fusion module (BFM) to detect both small and large objects at once with high accuracy; 2) concatenation and re-organization (CORE) module provides a bottom-up pathway for feature fusion, which leads to the bi-directional fusion FP that can recover lost information from lower-layer feature maps; 3) CORE feature is further purified to retain richer contextual information. Such CORE purification in both top-down and bottom-up pathways can be finished in only a few iterations; 4) adding of a residual design to CORE leads to a new Re-CORE module that enables easy training and integration with a wide range of deeper or lighter backbones. The proposed network achieves state-of-the-art performance on the UAVDT17 and MS COCO datasets.

Adaptive Fusion of Multi-Scale YOLO for Pedestrian Detection

Although pedestrian detection technology is constantly improving, pedestrian detection remains challenging because of the uncertainty and diversity of pedestrians in different scales and of occluded pedestrian modes. This study followed the common framework of single-shot object detection and proposed a divide-and-rule method to solve the aforementioned problems. The proposed model introduced a segmentation function that can split pedestrians who do not overlap in one image into two subimages. By using a network architecture, multiresolution adaptive fusion was performed on the output of all images and subimages to generate the final detection result. This study conducted an extensive evaluation of several challenging pedestrian detection data sets and finally proved the effectiveness of the proposed model. In particular, the proposed model achieved the most advanced performance on data sets from Visual Object Classes 2012 (VOC 2012), the French Institute for Research in Computer Science and Automation, and the Swiss Federal Institute of Technology in Zurich and obtained the most competitive results in a triple-width VOC 2012 experiment carefully designed by the present study.

Jet Single Shot Detection

We apply object detection techniques based on Convolutional Neural Networks to jet reconstruction and identification at the CERN Large Hadron Collider. In particular, we focus on CaloJet reconstruction, representing each event as an image composed of calorimeter cells and using a Single Shot Detection network, called Jet-SSD. The model performs simultaneous localization and classification and additional regression tasks to measure jet features. We investigate TernaryWeight Networks with weights constrained to {-1, 0, 1} times a layer- and channel-dependent scaling factors. We show that the quantized version of the network closely matches the performance of its full-precision equivalent.

Non-Anchor-Based Vehicle Detection for Traffic Surveillance Using Bounding Ellipses

Cameras for traffic surveillance are usually pole-mounted and produce images that reflect a birds-eye view. Vehicles in such images, in general, assume an ellipse form. A bounding box for the vehicles usually includes a large empty space when the vehicle orientation is not parallel to the edges of the box. To circumvent this problem, the present study applied bounding ellipses to a non-anchor-based, single-shot detection model (CenterNet). Since this model does not depend on anchor boxes, non-max suppression (NMS) that requires computing the intersection over union (IOU) between predicted bounding boxes is unnecessary for inference. The SpotNet that extends the CenterNet model by adding a segmentation head was also tested with bounding ellipses. Two other anchor-based, single-shot detection models (YOLO4 and SSD) were chosen as references for comparison. The model performance was compared based on a local dataset that was doubly annotated with bounding boxes and ellipses. As a result, the performance of the two models with bounding ellipses exceeded that of the reference models with bounding boxes. When the backbone of the ellipse models was pretrained on an open dataset (UA-DETRAC), the performance was further enhanced. Several data augmentation schemes also improved the performance of the proposed models. As a result, the best mAP score of a CenterNet exceeds 0.95 when augmenting heatmaps with bounding ellipses.

Single-electron operations in a foundry-fabricated array of quantum dots

Silicon quantum dots are attractive for the implementation of large spin-based quantum processors in part due to prospects of industrial foundry fabrication. However, the large effective mass associated with electrons in silicon traditionally limits single-electron operations to devices fabricated in customized academic clean rooms. Here, we demonstrate single-electron occupations in all four quantum dots of a 2 x 2 split-gate silicon device fabricated entirely by 300-mm-wafer foundry processes. By applying gate-voltage pulses while performing high-frequency reflectometry off one gate electrode, we perform single-electron operations within the array that demonstrate single-shot detection of electron tunneling and an overall adjustability of tunneling times by a global top gate electrode. Lastly, we use the two-dimensional aspect of the quantum dot array to exchange two electrons by spatial permutation, which may find applications in permutation-based quantum algorithms.

Experimental demonstration of the plasma electron density diagnostics using the single-shot THz detection method

We report the first experimental demonstration of a novel plasma diagnostics method using the single-shot THz detection method. In this experiment, a broadband single THz pulse is generated by focusing two-color (400 nm and 800 nm in wavelength) laser pulses in air and the THz pulse is sent through an inductively-coupled plasma with a density of 1013–1014 cm−3 which is in the modern tokamak plasma density range. Phase shift of the THz pulse is measured by the single-shot THz detection method and the instant plasma density information is obtained from the phase shift. The result is compared with that of the existing THz time domain spectroscopy method and it is found that both results are in good agreement. It implies that the proposed method in this paper can provide a new plasma diagnostics tool which can be used for dynamically changing plasmas.

Single-photon pulse induced giant response in N\u2009>\u2009100 qubit system

The temporal dynamics of large quantum systems perturbed weakly by a single excitation can give rise to unique phenomena at the quantum phase boundaries. Here, we develop a time-dependent model to study the temporal dynamics of a single photon interacting with a defect within a large system of interacting spin qubits (N > 100). Our model predicts a quantum resource, giant susceptibility, when the system of qubits is engineered to simulate a first-order quantum phase transition (QPT). We show that the absorption of a single-photon pulse by an engineered defect in the large qubit system can nucleate a single shot quantum measurement through spin noise read-out. This concept of a single-shot detection event (“click”) is different from parameter estimation, which requires repeated measurements. The crucial step of amplifying the weak quantum signal occurs by coupling the defect to a system of interacting qubits biased close to a QPT point. The macroscopic change in long-range order during the QPT generates amplified magnetic noise, which can be read out by a classical device. Our work paves the way for studying the temporal dynamics of large quantum systems interacting with a single-photon pulse.

Charge Detection in an Array of CMOS Quantum Dots

The recent development of arrays of quantum dots in semiconductor nanostructures highlights the progress of quantum devices toward large scale. However, how to realize such arrays on a scalable platform such as silicon is still an open question. One of the main challenge resides in the detection of charges within the array. It is a prerequisite functionality to initialize a desired charge state and readout spins through spin-to-charge conversion mechanisms. In this paper, we use two methods based on either a single-lead charge detector, or a reprogrammable single electron transistor. Thanks to these methods, we study the charge dynamics and sensitivity by performing single shot detection of the charge. Finally, we can probe the charge stability at any node of a linear array and assess the Coulomb disorder in the structure. We find an electrochemical potential fluctuation induced by charge noise comparable to that reported in other silicon quantum dots.

Widefield, Spatiotemporal Mapping of Spontaneous Activity of Mouse Cultured Neuronal Networks Using Quantum Diamond Sensors

Quantum diamond sensors containing Nitrogen Vacancy (NV) centers were integrated into the culture of spontaneously electrically active cultures of live mouse primary cortical neurons. Two diamond formats were used enabling extracellular studies of cells cultured directly on a planar diamond plate and intracellular studies via endocytosis mediated integration of nanoscale diamond particles within cells. Over a 14 day culture period functional neuronal networks formed as confirmed by endogenous calcium transients detected via fluorescent imaging of a calcium probe (Fluo-5 AM). Spatially correlated images of photoluminescence from NV centers were also recorded demonstrating successful acquisition of optically detected magnetic resonance from regions of endogenously firing neuronal cultures. Further, a fast and simple measurement protocol based on acquisition of NV photoluminescence without and with the application of near resonant microwaves is presented. This protocol was applied to live signaling cells and dead cells. The relative photoluminescence from NVs with and without microwaves varied for the case of live as compared to dead cells warranting future investigation. Overall this work presents a protocol for integration of NV diamond sensors in a spontaneously active network of neuronal cells of relevance to the field of neuroscience. Photoluminescence from NV centers was detected with high spatiotemporal resolution using measurement protocols potentially capable of single shot detection of neuronal activity. It is anticipated that the methodologies introduced in this work will underpin the establishment of NV diamond sensors as a radically new measurement platform capable of rapid, non-destructive functional studies of cells.