Adaptive Downsampling Research Articles

An efficient detector for detecting surface defects on cold-rolled steel strips

Surface-defect inspection is vital in cold-rolled steel-strip manufacturing, given the complexities of production environments and the high speeds involved. Further, the defects on cold-rolled steel strips are often characterized by their small size, diversity of types, and similarities among different types, posing significant challenges in balancing detection accuracy and efficiency. To address the challenges, we designed a detector based on You Only Look Once version 5 (YOLOv5) to achieve precise detection of surface defects on cold-rolled steel strips. First, a dataset containing seven types of defects was curated, named the Cold-Rolled Steel Defect Dataset (CR7-DET). Next, a feature-extraction network based on residual-like connections within a single residual block (Res2net) was developed to enhance the model’s feature-extraction capability, alongside introducing a multi-head attention module to focus on key information features. To reduce the information loss during feature fusion, we established an adaptive feature-fusion Path Aggregation Network (aff-PAN), which was optimized by designing a lightweight adaptive down-sampling module (LAD) to increase the sensory-field implementation of feature fusion. The ghost convolution effectively reduced the number of parameters and increased the speed without affecting the model’s performance. Finally, experiments were conducted on our CR7-DET and a public dataset (GC10-DET). With a reduced parameter count of 6.85 million, our model achieved a mean average precision(mAP) of 87.6% on CR7-DET and 79.7% on GC10-DET. The experimental results demonstrated that our model achieved a balance between detection accuracy and inference efficiency. The model has the potential to reduce scrap rates caused by defects and improve the overall surface quality of cold-rolled steel strips.

Research on Parallel Reading and Drawing Techniques for Chemical Mechanical Polishing Simulation Data Based on Multi-Thread

In advanced integrated circuit manufacturing, the quality of chemical mechanical polishing (CMP) is a key factor affecting chip performance and yield. Designers need to use CMP simulation tools to locate and analyze the defects in the layout after the CMP process. However, the advancement of process nodes and the increase in data volume presents a great challenge to the speed of graphical display of CMP simulation data. To solve this issue, we propose a solution that uses multi-threading technology to optimize both data reading and drawing. In the process of data reading, we employ OpenMP and memory mapping (Mmap) technology to achieve parallel reading of file segmentation and propose a fast-string conversion algorithm based on the properties of simulation data. In the process of data drawing, we propose an adaptive downsampling method for data graphical display that combines multi-threading and double buffering technology to enable the parallel drawing of layouts. The effectiveness of this method is verified by testing CMP simulation data of various scales. Compared to traditional methods, this approach improves reading efficiency by over 8 times and drawing efficiency by more than 10 times. Furthermore, it enhances the smoothness of interaction with the CMP simulation tool.

A Two-Stage Registration Strategy for Thermal–Visible Images in Substations

The analysis of infrared video images is becoming one of the methods used to detect thermal hazards in many large-scale engineering sites. The fusion of infrared thermal imaging and visible image data in the target area can help people to identify and locate the fault points of thermal hazards. Among them, a very important step is the registration of thermally visible images. However, the direct registration of images with large-scale differences may lead to large registration errors or even failure. This paper presents a novel two-stage thermal–visible-image registration strategy specifically designed for exceptional scenes, such as a substation. Firstly, the original image pairs that occur after binarization are quickly and roughly registered. Secondly, the adaptive downsampling unit partial-intensity invariant feature descriptor (ADU-PIIFD) algorithm is proposed to correct the small-scale differences in details and achieve finer registration. Experiments are conducted on 30 data sets containing complex power station scenes and compared with several other methods. The results show that the proposed method exhibits an excellent and stable performance in thermal–visible-image registration, and the registration error on the entire data set is within five pixels. Especially for multimodal images with poor image quality and many detailed features, the robustness of the proposed method is far better than that of other methods, which provides a more reliable image registration scheme for the field of fire safety.

Content adaptive downsampling for low bitrate video coding

Content adaptive downsampling for low bitrate video coding

SCA-Net: Spatial and channel attention-based network for 3D point clouds

A key challenge in data processing of 3D point cloud data is how to identify key regions in the point cloud from geometric relationships. Previous work has attempted to overcome this deficiency by slicing and dicing data. Such an approach limits the model’s ability to effectively learn global information. In this paper, in order to better extract features and obtain geometric information we propose a Point Attention (PointAT) model and propose Attention Value (AT value) model for feature fusion to apply geometric relationship to the data. Then, we propose a new Spatial and Channel Attention-based network (SCA). The SCA is the overall structure of the network, and the main purpose is to connect PointAT and AT value model, then capturing meaningful geometric information by applying the geometric relationship between point clouds patches to the model, then propose a auto pooling framework to extract to global features. In this work, we concentrate on learning geometric relationship between point cloud data. For this purpose, we introduce a point attention model based on spatial and channel attention to learn the geometric relationship between point clouds, and further combine the geometric relationship with the point cloud data by the AT Value Model. Finally, we introduce a adaptive downsampling structure, Autopooling. This downsampling structure consider each point’s importance weight and picking key points adaptively, which can be used with convolutional networks. Extensive experiments conducted on two benchmark datasets (ModelNet40 and ShapeNet) clearly demonstrate the effectiveness of our SCA and SCA-Auto (SCAA with Auto pooling) methods.

Fault diagnosis of driving gear in rack and pinion drives based on multi-scale local binary pattern extraction and sparse representation

Rack and pinion drives (RPDs) are key components of battery-swapping systems (BSSs) used in electric heavy trucks; the faults occurring in these drives reduce the efficiency, accuracy, quality of battery swapping, and even pose potential safety risks. The operating conditions of RPD driving gear in BSSs are characterized by speed fluctuations, relatively low speeds, and reciprocating motion. To assess the driving gear fault characteristics under these conditions, based on the solution of image recognition under complex and low illumination conditions, this study proposes a fault diagnosis framework that includes adaptive down-sampling, three-dimensional acceleration data fusion, multi-scale local binary pattern (MS-LBP) extraction, and sparse representation. First, adaptive down-sampling is used to smooth out the speed fluctuation. Subsequently, MS-LBP extraction is employed to obtain efficient fault features at low speed. Finally, dictionary learning and sparse representations are conducted on the MS-LBP features. The effectiveness and advantages of the proposed diagnosis approach are demonstrated using monitoring data acquired from a BSS. Moreover, comparative studies demonstrate that the proposed fault diagnosis method yields improved performance.

A single-stage point cloud cleaning network for outlier removal and denoising

As a simple, flexible and effective representation for objects, 3D point cloud has attracted more and more attention in recent years. However, raw point clouds obtained from 3D scanners or image-based reconstruction techniques are often contaminated with noise and outliers, which hinders downstream tasks such as object classification, surface reconstruction, and so on. Therefore, point cloud cleaning, i.e., removing noisy points and outliers from raw point cloud, is a prior step of most geometry processing workflows. The exiting techniques for point cloud cleaning usually include two stages, that is, discarding outliers at first, and then denoising the resulting point cloud. This two-stage process usually requires two different models, which is cumbersome to train and use. To solve this problem, a novel data driven method, named SSPCN (single-stage point cloud cleaning network), is proposed in this paper. SSPCN can simultaneously remove outliers and denoise a point cloud in a single model. Specifically, SSPCN is consisted of adaptive downsampling module, feature compensation module, upsampling module and coordinate reconstruction module. Given a raw point cloud as input, the downsampling module is first used to obtain a prefiltered point cloud subset and learn initial features of the subset. The feature compensation module is then utilized to learn accurate features from initial features. Next, the upsampling module upsamples the features to restore the original size of the point cloud. Last, the coordinate reconstruction module generates a cleaned point cloud from upsampled features. SSPCN is validated both on synthetic and real scanned data. Extensive experiments demonstrate that SSPCN outperforms state-of-the-art point cloud cleaning techniques in terms of quantitative metric and visual quality.

Densely attention mechanism based network for COVID-19 detection in chest X-rays

Automatic COVID-19 detection using chest X-ray (CXR) can play a vital part in large-scale screening and epidemic control. However, the radiographic features of CXR have different composite appearances, for instance, diffuse reticular-nodular opacities and widespread ground-glass opacities. This makes the automatic recognition of COVID-19 using CXR imaging a challenging task. To overcome this issue, we propose a densely attention mechanism-based network (DAM-Net) for COVID-19 detection in CXR. DAM-Net adaptively extracts spatial features of COVID-19 from the infected regions with various appearances and scales. Our proposed DAM-Net is composed of dense layers, channel attention layers, adaptive downsampling layer, and label smoothing regularization loss function. Dense layers extract the spatial features and the channel attention approach adaptively builds up the weights of major feature channels and suppresses the redundant feature representations. We use the cross-entropy loss function based on label smoothing to limit the effect of interclass similarity upon feature representations. The network is trained and tested on the largest publicly available dataset, i.e., COVIDx, consisting of 17,342 CXRs. Experimental results demonstrate that the proposed approach obtains state-of-the-art results for COVID-19 classification with an accuracy of 97.22%, a sensitivity of 96.87%, a specificity of 99.12%, and a precision of 95.54%.

Fourier Single-Pixel Imaging Based on Adaptive Down-Sampling in Frequency Domain

Fourier Single-Pixel Imaging Based on Adaptive Down-Sampling in Frequency Domain

To Drop or to Select: Reduce the Negative Effects of Disturbance Features for Point Cloud Classification From an Interpretable Perspective

The perturbation features limit the performance of point cloud classification models both for clean point cloud and point cloud obtained from real-world. In this paper, we propose two methods to enhance models by reducing the negative impact of the nuisance features from the perspective of interpretability, namely, dropping the nuisance points before inputting the point cloud into models and adaptively selecting the important features during the training process. The former is achieved by saliency analysis of models, the perturbation points are those with low contribution to models. For each sample, dropping the low contribution points based on the saliency scores is equivalent to filtering the perturbation features. We design a generic framework for generating saliency maps for various models and datasets, and obtain empirical values for the number of dropped points on each set of them. Then we apply the unsupervised dropping process to improve the robustness of models. The latter is achieved by adaptive downsampling, and we design a multi-stage learnable class-attention-based downsampling module to replace the commonly used Farthest Point Sampling (FPS). As the training progresses, the downsampling module tends to select the common features for each category, thus eliminating the nuisance features to improve the learning efficiency of the model. For dropping points (DP), we generate saliency maps for PointNet&++, DGCNN and PointMLP on ModelNet40 and ScanObjectNN, PointNet+DP reaches an overall accuracy (OA) of 92.5% and 72% on ModelNet40 and ScanObjectNN, surpassing the original model by 3.4% and 5.3%, the OA of PointNet++ with DP on ModelNet40 object classification can be raised from 91.8% to 93.7%. For adaptive feature selection (AFS), PointMLP-elite+AFS reaches an OA of 92.5% and a mean accuracy (mAcc) of 72% on ScanObjectNN, surpassing the original model by 0.8% and 1%. It reaches the level of PointMLP with 6.3% of the number of parameters. Considering the difficulty of the deployment of deep models, PointMLP-elite+AFS is the most cost-effective classification model known on ScanObjectNN.

An Adaptive Downsampling FPGA-Based TDC Implementation for Time Measurement Improvement

In this work, we present a compact “adaptive downsampling” method that mitigates the nonlinearity problems associated with FPGA-based TDCs that utilize delay lines. Additionally, this generic method allows for trade-offs between resolution, linearity, and resource utilization. Since nonlinearity is one of the predominant issues regarding delay lines in FPGA-based TDCs, combined with the fact that delay lines are utilized for a wide range of TDC architectures (not limited to the delay-line TDC), other implementations (e.g., Vernier or wave union TDCs), also in different FPGA devices, can directly benefit from the proposed adaptive method, with no need for either custom routing or complex tuning of the converter. Furthermore, implementation-related challenges regarding clock skew, measurement uncertainty, and the placement of the TDC are discussed and we also propose an experimental setup that utilizes only FPGA resources in order to characterize the converter. Although the TDC in this work was implemented in a Xilinx Virtex-6 device and was characterized under different operational modes, we successfully optimized the converter’s nonlinearity and resource utilization while retaining single-shot precision. The best performing (in terms of linearity) implementation reached DNLrms and INLrms values of 0.30 LSB and 0.45 LSB, respectively, and the single-shot precision (σ) was 9.0 ps.

Toward Real-World Super-Resolution via Adaptive Downsampling Models.

Most image super-resolution (SR) methods are developed on synthetic low-resolution (LR) and high-resolution (HR) image pairs that are constructed by a predetermined operation, e.g., bicubic downsampling. As existing methods typically learn an inverse mapping of the specific function, they produce blurry results when applied to real-world images whose exact formulation is different and unknown. Therefore, several methods attempt to synthesize much more diverse LR samples or learn a realistic downsampling model. However, due to restrictive assumptions on the downsampling process, they are still biased and less generalizable. This study proposes a novel method to simulate an unknown downsampling process without imposing restrictive prior knowledge. We propose a generalizable low-frequency loss (LFL) in the adversarial training framework to imitate the distribution of target LR images without using any paired examples. Furthermore, we design an adaptive data loss (ADL) for the downsampler, which can be adaptively learned and updated from the data during the training loops. Extensive experiments validate that our downsampling model can facilitate existing SR methods to perform more accurate reconstructions on various synthetic and real-world examples than the conventional approaches.

Unsupervised Domain Adaptive 3-D Detection With Data Adaption From LiDAR Point Cloud

Existing unsupervised domain adaptive (UDA) 3D detection methods only address the domain gap caused by the prior size of 3D bounding boxes between different datasets, which ignore the difference in the distribution of point clouds. To address this challenge, we propose an unsupervised domain adaptive 3D detection by data adaption, which trains the model by transferring the source domain instances into the target domain scenes by adaptive point distribution. First, an instance transferring method is proposed for selecting and transferring suitable instances from the source domain into the target domain scene; Second, we propose an adaptive downsampling method to adjust the point cloud distribution of the transferred instances to approximate the points distribution of the target domain. Finally, our method trains the randomly initialized detector with the pseudo-instances in the target domain. To the best of our knowledge, we first address the UDA problem of the 3D detectors from the perspective of data. Extensive experiments on several popular datasets show that the proposed method outperforms the existing state-of-the-art methods by a large margin. Further experiments also show our approach is detector-agnostic and achieves consistent and significant gains on all types of 3D detectors.

Elastic Downsampling: An Adaptive Downsampling Technique to Preserve Image Quality

This paper presents a new adaptive downsampling technique called elastic downsampling, which enables high compression rates while preserving the image quality. Adaptive downsampling techniques are based on the idea that image tiles can use different sampling rates depending on the amount of information conveyed by each block. However, current approaches suffer from blocking effects and artifacts that hinder the user experience. To bridge this gap, elastic downsampling relies on a Perceptual Relevance analysis that assigns sampling rates to the corners of blocks. The novel metric used for this analysis is based on the luminance fluctuations of an image region. This allows a gradual transition of the sampling rate within tiles, both horizontally and vertically. As a result, the block artifacts are removed and fine details are preserved. Experimental results (using the Kodak and USC Miscelanea image datasets) show a PSNR improvement of up to 15 dB and a superior SSIM (Structural Similarity) when compared with other techniques. More importantly, the algorithms involved are computationally cheap, so it is feasible to implement them in low-cost devices. The proposed technique has been successfully implemented using graphics processors (GPU) and low-power embedded systems (Raspberry Pi) as target platforms.

Automated Feature-Based Down-Sampling Approaches for Fine Registration of Irregular Point Clouds

The integration of three-dimensional (3D) data defined in different coordinate systems requires the use of well-known registration procedures, which aim to align multiple models relative to a common reference frame. Depending on the achieved accuracy of the estimated transformation parameters, the existing registration procedures are classified as either coarse or fine registration. Coarse registration is typically used to establish a rough alignment between the involved point clouds. Fine registration starts from coarsely aligned point clouds to achieve more precise alignment of the involved datasets. In practice, the acquired/derived point clouds from laser scanning and image-based dense matching techniques usually include an excessive number of points. Fine registration of huge datasets is time-consuming and sometimes difficult to accomplish in a reasonable timeframe. To address this challenge, this paper introduces two down-sampling approaches, which aim to improve the efficiency and accuracy of the iterative closest patch (ICPatch)-based fine registration. The first approach is based on a planar-based adaptive down-sampling strategy to remove redundant points in areas with high point density while keeping the points in lower density regions. The second approach starts with the derivation of the surface normals for the constituents of a given point cloud using their local neighborhoods, which are then represented on a Gaussian sphere. Down-sampling is ultimately achieved by removing the points from the detected peaks in the Gaussian sphere. Experiments were conducted using both simulated and real datasets to verify the feasibility of the proposed down-sampling approaches for providing reliable transformation parameters. Derived experimental results have demonstrated that for most of the registration cases, in which the points are obtained from various mapping platforms (e.g., mobile/static laser scanner or aerial photogrammetry), the first proposed down-sampling approach (i.e., adaptive down-sampling approach) was capable of exceeding the performance of the traditional approaches, which utilize either the original or randomly down-sampled points, in terms of providing smaller Root Mean Square Errors (RMSE) values and a faster convergence rate. However, for some challenging cases, in which the acquired point cloud only has limited geometric constraints, the Gaussian sphere-based approach was capable of providing superior performance as it preserves some critical points for the accurate estimation of the transformation parameters relating the involved point clouds.

Real-Time Image Super-Resolution Using Recursive Depthwise Separable Convolution Network

In recent years, deep convolutional neural networks (CNNs) have been widely used for image super-resolution (SR) to achieve a range of sophisticated performances. Despite the significant advancement made in CNNs, it is still difficult to apply CNNs to practical SR applications due to enormous computations of deep convolutions. In this paper, we propose two lightweight deep neural networks using depthwise separable convolution for the real-time image SR. Specifically, depthwise separable convolution divides the standard convolution into depthwise convolution and pointwise convolution to significantly reduce the number of model parameters and multiplication operations. Moreover, recursive learning is adopted to increase the depth and receptive field of the network in order to improve the SR quality without increasing the model parameters. Finally, we propose a novel technique called Super-Sampling (SS) to learn more abundant high-resolution information by over-sampling the output image followed by adaptive down-sampling. The proposed two models, named SSNet-M and SSNet, outperform the existing state-of-the-art real-time image SR networks, including SRCNN, FSRCNN, ESPCN, and VDSR, in terms of model complexity, and subjective and PSNR/SSIM evaluations on Set5, Set14, B100, Urban100, and Manga109.

ZUMIs - A fast and flexible pipeline to process RNA sequencing data with UMIs.

BackgroundSingle-cell RNA-sequencing (scRNA-seq) experiments typically analyze hundreds or thousands of cells after amplification of the cDNA. The high throughput is made possible by the early introduction of sample-specific bar codes (BCs), and the amplification bias is alleviated by unique molecular identifiers (UMIs). Thus, the ideal analysis pipeline for scRNA-seq data needs to efficiently tabulate reads according to both BC and UMI.Findings zUMIs is a pipeline that can handle both known and random BCs and also efficiently collapse UMIs, either just for exon mapping reads or for both exon and intron mapping reads. If BC annotation is missing, zUMIs can accurately detect intact cells from the distribution of sequencing reads. Another unique feature of zUMIs is the adaptive downsampling function that facilitates dealing with hugely varying library sizes but also allows the user to evaluate whether the library has been sequenced to saturation. To illustrate the utility of zUMIs, we analyzed a single-nucleus RNA-seq dataset and show that more than 35% of all reads map to introns. Also, we show that these intronic reads are informative about expression levels, significantly increasing the number of detected genes and improving the cluster resolution.Conclusions zUMIs flexibility makes if possible to accommodate data generated with any of the major scRNA-seq protocols that use BCs and UMIs and is the most feature-rich, fast, and user-friendly pipeline to process such scRNA-seq data.

Image compression using JPEG with reduced blocking effects via adaptive down-sampling and self-learning image sparse representation

Blocking is an annoying effect in image compression using JPEG especially at low bit-rates. In this paper, a two-phase method is proposed for reducing JPEG blocking effects. In the first phase, the image is adaptively down-sampled via assessing the DCT coefficients of the image blocks. Since the blocks have a fixed size, independent to the size of the image, down-sampling can reduce the number of blocks, and hence reduce the blocking effects. Appropriate down-sampling before compression can improve coding performance, especially at low bit-rate. The decoder, after decompression, up-samples the image to its original resolution. Although, down-sampling alleviates the blocking effects, yet some blocking effects are remained in the result and the image is damaged due to resizing. Hence, self-learning sparse representation is applied in the second phase for a better deblocking and also for alleviating the degradation due to resizing. Experimental results demonstrate that the proposed method can efficiently improve the subjective and objective quality of JPEG compressed images at low bit-rates and outperforms the existing methods.

Omnidirectional video coding using latitude adaptive down‐sampling and pixel rearrangement

Projection of omnidirectional videos in the spherical domain onto the planar domain is required to utilise existing video compression techniques. Due to its simplicity, the equirectangular projection (ERP) method has been widely adopted for sphere-to-plane mapping. However, the coding efficiency of the ERP method is inefficient because the ERP video includes visual redundancies in the high latitude areas of the sphere. To solve this problem, this Letter presents an efficient data compression scheme that employs latitude adaptive down-sampling followed by pixel rearrangement. Experimental results show that the proposed method can achieve a bitrate reduction of up to 28.9%, as compared with the ERP.

Planar-Based Adaptive Down-Sampling of Point Clouds

Derived point clouds from laser scanners and image-based dense-matching techniques usually include tremendous number of points. Processing (e.g., segmenting) such huge dataset is time-consuming and might not be necessary. For example, a planar surface just needs few points to be defined. In contrast, linear/cylindrical and rough features require more points for reliable modeling since during the data acquisition process, only a portion of linear/cylindrical features is present in the point cloud.This paper introduces an adaptive down-sampling strategy for removing redundant points from high density planar regions while retaining points in planar areas with sparse points and all the points within linear/cylindrical and rough neighborhoods. To demonstrate the feasibility and performance of the proposed procedure, a comparison of segmentation results using original laser and image-based point clouds as well as the adaptively, uniformly, and point-spacing-based down-sampled point clouds are presented while commenting on the computational efficiency and the segmentation quality.