Skip-layer Connections Research Articles

A lightweight deep learning method to identify different types of cervical cancer

Cervical cancer is the second most common cancer in women’s bodies after breast cancer. Cervical cancer develops from dysplasia or cervical intraepithelial neoplasm (CIN), the early stage of the disease, and is characterized by the aberrant growth of cells in the cervix lining. It is primarily caused by Human Papillomavirus (HPV) infection, which spreads through sexual activity. This study focuses on detecting cervical cancer types efficiently using a novel lightweight deep learning model named CCanNet, which combines squeeze block, residual blocks, and skip layer connections. SipakMed, which is not only popular but also publicly available dataset, was used in this study. We conducted a comparative analysis between several transfer learning and transformer models such as VGG19, VGG16, MobileNetV2, AlexNet, ConvNeXT, DeiT_tiny, MobileViT, and Swin Transformer with the proposed CCanNet. Our proposed model outperformed other state-of-the-art models, with 98.53% accuracy and the lowest number of parameters, which is 1,274,663. In addition, accuracy, precision, recall, and the F1 score were used to evaluate the performance of the models. Finally, explainable AI (XAI) was applied to analyze the performance of CCanNet and ensure the results were trustworthy.

AID-YOLO: An Efficient and Lightweight Network Method for Small Target Detector in Aerial Images

The progress of object detection technology is crucial for obtaining extensive scene information from aerial perspectives based on computer vision. However, aerial image detection presents many challenges, such as large image background sizes, small object sizes, and dense distributions. This research addresses the specific challenges relating to small object detection in aerial images and proposes an improved YOLOv8s-based detector named Aerial Images Detector-YOLO(AID-YOLO). Specifically, this study adopts the General Efficient Layer Aggregation Network (GELAN) from YOLOv9 as a reference and designs a four-branch skip-layer connection and split operation module Re-parameterization-Net with Cross-Stage Partial CSP and Efficient Layer Aggregation Networks (RepNCSPELAN4) to achieve a lightweight network while capturing richer feature information. To fuse multi-scale features and focus more on the target detection regions, a new multi-channel feature extraction module named Convolutional Block Attention Module with Two Convolutions Efficient Layer Aggregation Net-works (C2FCBAM) is designed in the neck part of the network. In addition, to reduce the sensitivity to position bias of small objects, a new function, Normalized Weighted Distance Complete Intersection over Union (NWD-CIoU_Loss) weight adaptive loss function, was designed in this study. We evaluate the proposed AID-YOLO method through ablation experiments and comparisons with other advanced models on the VEDAI (512, 1024) and DOTAv1.0 datasets. The results show that compared to the Yolov8s baseline model, AID-YOLO improves the mAP@0.5 metric by 7.36% on the VEDAI dataset. Simultaneously, the parameters are reduced by 31.7%, achieving a good balance between accuracy and parameter quantity. The Average Precision (AP) for small objects has improved by 8.9% compared to the baseline model (YOLOv8s), making it one of the top performers among all compared models. Furthermore, the FPS metric is also well-suited for real-time detection in aerial image scenarios. The AID-YOLO method also demonstrates excellent performance on infrared images in the VEDAI1024 (IR) dataset, with a 2.9% improvement in the mAP@0.5 metric. We further validate the superior detection and generalization performance of AID-YOLO in multi-modal and multi-task scenarios through comparisons with other methods on different resolution images, SODA-A and the DOTAv1.0 datasets. In summary, the results of this study confirm that the AID-YOLO method significantly improves model detection performance while maintaining a reduced number of parameters, making it applicable to practical engineering tasks in aerial image object detection.

DAGAN: A GAN Network for Image Denoising of Medical Images Using Deep Learning of Residual Attention Structures

Medical images are susceptible to noise and artifacts, so denoising becomes an essential pre-processing technique for further medical image processing stages. We propose a medical image denoising method based on dual-attention mechanism for generative adversarial networks (GANs). The method is based on a GAN model with fused residual structure and introduces a global skip-layer connection structure to balance the learning ability of the shallow and deep networks. The generative network uses a residual module containing channel and spatial attention for efficient extraction of CT image features. The mean square error loss and perceptual loss are introduced to construct a composite loss function to optimize the model loss function, which helps to improve the image generation effect of the model. Experimental results on the LUNA dataset and “the 2016 Low-Dose CT Grand Challenge” dataset show that DAGAN achieves the best results in root mean square error (RMSE), structural similarity (SSIM) and peak signal-to-noise ratio (PSNR) when compared to the state-of-the-art methods. In particular, PSNR reaches 31.2308 dB and 27.5265 dB, SSIM reaches 0.9115 and 0.7895, while RMSE is 0.0082 and 0.0112, respectively. This indicates that our method performs better than the state-of-the-art methods in the task of CT image denoising.

Dual-Gradients Localization Framework With Skip-Layer Connections for Weakly Supervised Object Localization

Dual-Gradients Localization Framework With Skip-Layer Connections for Weakly Supervised Object Localization

A multi-scale interactive U-Net for pulmonary vessel segmentation method based on transfer learning

Pulmonary vessel segmentation is the key application of AI in lung disease diagnosis and surgical planning. Compared with manual labeling, automatic labeling of pulmonary vessels using an AI-based medical image segmentation method has the advantages of low cost, high accuracy, and efficiency, which is the development trend of medical images. In terms of pulmonary vessel segmentation, FCN and U-Net are the most widely used pulmonary vessel segmentation methods based on deep learning. However, the precision of pulmonary vessel segmentation, especially the small vessels, tends to be poor by such methods. Therefore, to solve the above problem, Multi-Scale Interactive U-Net (MSI-U-Net) is proposed. In MSI-U-Net, three decoder branches are used to extract small-scale, middle-scale and large-scale vessels respectively, which can improve the accuracy of small vessel segmentation by enhancing the representational ability of small vessels. In addition, to solve the problem of small vessel information loss caused by down-sampling, we introduce the attention mechanism into the skip-layer connection and propose a cross-layer aggregation module (CLA). Among the three decoder branches, a multi-scale information interaction strategy (MSIIS) is proposed based on transfer learning, which can effectively enhance the correlation of multi-scale vessels in lung CT images. In the training stage, we propose a scale-induced supervision strategy (SISS). This strategy uses the idea of fusion first and then supervision, which effectively solves the problem of inconsistency in multi-scale vessels classification, thereby reducing the segmentation errors. Finally, we use feature transmission instead of convolution parameter sharing to realize the multi-scale information interaction strategy, and propose an extension scheme called Multi-Level Cascade Interactive U-Net (MLCI-U-Net). The experimental results indicate that our MSI-U-Net and MLCI-U-Net have better performance than other state-of-the-art methods on pulmonary vessel segmentation. Specifically, the best Dice similarity coefficient (DSC), Sensitivity and Precision are obtained by the proposed methods to segment pulmonary vessels.

Multiscale Progressive Segmentation Network for High-Resolution Remote Sensing Imagery

Semantic segmentation of high-resolution remote sensing imageries (HRSIs) is a critical task for a wide range of applications, such as precision agriculture and urban planning. Although convolutional neural networks (CNNs) have made great progress in accomplishing this task recently, there still exist some challenges to address, one of which is simultaneously segmenting objects with large scale variations in a HRSI. Targeting at this challenge, previous CNNs often adopt multiple convolution kernels in one layer or skip-layer connections between different layers to extract multiscale representations. However, due to the limited learning capacity of each CNN, it tends to make trade-offs in segmenting different-scale objects. This would lead to unsatisfactory segmentation results for some objects, especially the small or the large ones. In this paper, we propose a multiscale progressive segmentation network to address this issue. Instead of forcing one network to deal with all scales of objects, our network attempts to cascade three subnetworks for gradually segmenting objects with small scales, large scales, and other scales. In order to make the subnetwork focus on the specific scale objects, a scale guidance module is designed. It takes advantage of segmentation results from the preceding subnetwork to guide the feature learning of the succeeding one. Additionally, to acquire the final segmentation results, we propose a position sensitive module for adaptively combining the outputs of the three subnetworks. This module is capable of assigning combination weights of different subnetworks according to their importance. Experiments on two benchmark datasets named Vaihingen and Potsdam indicate that our proposed network can achieve considerable improvements in comparison with several state-of-the-art segmentation models.

Identifying Oceanic Eddy With an Edge-Enhanced Multiscale Convolutional Network

Identifying oceanic eddy from remotely sensed sea surface height (SSH) data is challenging, mainly because of its large-size variations. This article proposes an automatic identification model upon convolutional neural networks for dealing with this issue. The proposed network is comprised of two branches: an eddy identification branch and an edge extraction branch. Both of them adopt encoder–decoder frameworks, and the encoder is shared with each other. The eddy identification branch simultaneously uses multiscale convolution modules in the encoder and skip-layer connections between the encoder and the decoder to learn multiscale features, thus effectively identifying eddies with different sizes. Differently, the edge extraction branch is designed to learn the edge information of eddies, which is not fully captured by the eddy identification branch. To sufficiently evaluate the identification performance of our proposed model, several experiments are conducted on a public eddy identification dataset named SCSE-Eddy, and the results indicate that the proposed model is capable of achieving higher performance than those of the existing identification models.

Attention-Based Convolutional Denoising Autoencoder for Two-Lead ECG Denoising and Arrhythmia Classification

This paper presents a fast and accurate electrocardiogram (ECG) denoising and classification method for low-quality ECG signals. To achieve this, a novel attention-based convolutional denoising autoencoder model is proposed that utilizes skip-layer and attention module for reliable reconstruction of ECG signals from extreme noise conditions. Skip-layer connections are used to reduce information loss while reconstructing the original signal, and a lightweight, efficient channel attention module is used to update relevant features retrieved via cross-channel interaction efficiently. The model is trained and tested using four widely available databases. For evaluation, the signals are mixed with simulated additive white Gaussian noise ranging from -20 to 20 decibels (dB) and MIT-BIH Noise Stress Test Database (NSTDB) noise ranging from -6 to 24 dB. The model outperformed the most cited published works, achieving an average Signal-to-noise ratio (SNR) improvement of 19.07±1.67 and percentage-root-mean-square difference (PRD) of 11.0% at 0 dB SNR noise. The model classification performance on 60, 000 beats are 98.76 ± 0.44% precision, 98.48 ± 0.58% recall, and 98.88 ± 0.42% accuracy, respectively using a stratified five-fold cross-validation strategy.

Forecasting in Big Data Environments: An Adaptable and Automated Shrinkage Estimation of Neural Networks (AAShNet)

This paper considers improved forecasting in possibly nonlinear dynamic settings, with high-dimension predictors (“big data” environments). To overcome the curse of dimensionality and manage data and model complexity, we examine shrinkage estimation of a back-propagation algorithm of a neural net with skip-layer connections. We expressly include both linear and nonlinear components. This is a high-dimensional learning approach including both sparsity \(L_1\) and smoothness \(L_2\) penalties, allowing high-dimensionality and nonlinearity to be accommodated in one step. This approach selects significant predictors as well as the topology of the neural network. We estimate optimal values of shrinkage hyperparameters by incorporating a gradient-based optimization technique resulting in robust predictions with improved reproducibility. The latter has been an issue in some approaches. This is statistically interpretable and unravels some network structure, commonly left to a black box. An additional advantage is that the nonlinear part tends to get pruned if the underlying process is linear. In an application to forecasting equity returns, the proposed approach captures nonlinear dynamics between equities to enhance forecast performance. It offers an appreciable improvement over current univariate and multivariate models by actual portfolio performance.

Conveyor-Belt Detection of Conditional Deep Convolutional Generative Adversarial Network

In underground mining, the belt is a critical component, as its state directly affects the safe and stable operation of the conveyor. Most of the existing non-contact detection methods based on machine vision can only detect a single type of damage and they require pre-processing operations. This tends to cause a large amount of calculation and low detection precision. To solve these problems, in the work described in this paper a belt tear detection method based on a multi-class conditional deep convolutional generative adversarial network (CDCGAN) was designed. In the traditional DCGAN, the image generated by the generator has a certain degree of randomness. Here, a small number of labeled belt images are taken as conditions and added them to the generator and discriminator, so the generator can generate images with the characteristics of belt damage under the aforementioned conditions. Moreover, because the discriminator cannot identify multiple types of damage, the multi-class softmax function is used as the output function of the discriminator to output a vector of class probabilities, and it can accurately classify cracks, scratches, and tears. To avoid the features learned incompletely, skip-layer connection is adopted in the generator and discriminator. This not only can minimize the loss of features, but also improves the convergence speed. Compared with other algorithms, experimental results show that the loss value of the generator and discriminator is the least. Moreover, its convergence speed is faster, and the mean average precision of the proposed algorithm is up to 96.2%, which is at least 6% higher than that of other algorithms.

Multi-Channel Fetal ECG Denoising With Deep Convolutional Neural Networks.

Non-invasive fetal electrocardiography represents a valuable alternative continuous fetal monitoring method that has recently received considerable attention in assessing fetal health. However, the non-invasive fetal electrocardiogram (ECG) is typically severely contaminated by a considerable amount of various noise sources, rendering fetal ECG denoising a very challenging task. This work employs a deep learning approach for removing the residual noise from multi-channel fetal ECG after the maternal ECG has been suppressed. We propose a deep convolutional encoder-decoder network with symmetric skip-layer connections, learning end-to-end mappings from noise-corrupted fetal ECG signals to clean ones. Experiments on simulated data show an average signal-to-noise ratio (SNR) improvement of 9.5 dB for fetal ECG signals with input SNR ranging between −20 and 20 dB. The method is additionally evaluated on a large set of real signals, demonstrating that it can provide significant quality improvement of the noisy fetal ECG signals. We further show that employment of multi-channel signal information by the network provides superior and more reliable performance as opposed to its single-channel network counterpart. The presented method is able to preserve beat-to-beat morphological variations and does not require any prior information on the power spectra of the noise or the pulse location.

A Multi-Scale Feature Fusion Method Based on U-Net for Retinal Vessel Segmentation.

Computer-aided automatic segmentation of retinal blood vessels plays an important role in the diagnosis of diseases such as diabetes, glaucoma, and macular degeneration. In this paper, we propose a multi-scale feature fusion retinal vessel segmentation model based on U-Net, named MSFFU-Net. The model introduces the inception structure into the multi-scale feature extraction encoder part, and the max-pooling index is applied during the upsampling process in the feature fusion decoder of an improved network. The skip layer connection is used to transfer each set of feature maps generated on the encoder path to the corresponding feature maps on the decoder path. Moreover, a cost-sensitive loss function based on the Dice coefficient and cross-entropy is designed. Four transformations—rotating, mirroring, shifting and cropping—are used as data augmentation strategies, and the CLAHE algorithm is applied to image preprocessing. The proposed framework is tested and trained on DRIVE and STARE, and sensitivity (Sen), specificity (Spe), accuracy (Acc), and area under curve (AUC) are adopted as the evaluation metrics. Detailed comparisons with U-Net model, at last, it verifies the effectiveness and robustness of the proposed model. The Sen of 0.7762 and 0.7721, Spe of 0.9835 and 0.9885, Acc of 0.9694 and 0.9537 and AUC value of 0.9790 and 0.9680 were achieved on DRIVE and STARE databases, respectively. Results are also compared to other state-of-the-art methods, demonstrating that the performance of the proposed method is superior to that of other methods and showing its competitive results.

Classification of Hyperspectral Images via Multitask Generative Adversarial Networks

Deep learning has shown its huge potential in the field of hyperspectral image (HSI) classification. However, most of the deep learning models heavily depend on the quantity of available training samples. In this article, we propose a multitask generative adversarial network (MTGAN) to alleviate this issue by taking advantage of the rich information from unlabeled samples. Specifically, we design a generator network to simultaneously undertake two tasks: the reconstruction task and the classification task. The former task aims at reconstructing an input hyperspectral cube, including the labeled and unlabeled ones, whereas the latter task attempts to recognize the category of the cube. Meanwhile, we construct a discriminator network to discriminate the input sample coming from the real distribution or the reconstructed one. Through an adversarial learning method, the generator network will produce real-like cubes, thus indirectly improving the discrimination and generalization ability of the classification task. More importantly, in order to fully explore the useful information from shallow layers, we adopt skip-layer connections in both reconstruction and classification tasks. The proposed MTGAN model is implemented on three standard HSIs, and the experimental results show that it is able to achieve higher performance than other state-of-the-art deep learning models.

Deep Learning for Multilabel Remote Sensing Image Annotation With Dual-Level Semantic Concepts

Multilabel remote sensing (RS) image annotation is a challenging and time-consuming task that requires a considerable amount of expert knowledge. Most existing RS image annotation methods are based on handcrafted features and require multistage processes that are not sufficiently efficient and effective. An RS image can be assigned with a single label at the scene level to depict the overall understanding of the scene and with multiple labels at the object level to represent the major components. The multiple labels can be used as supervised information for annotation, whereas the single label can be used as additional information to exploit the scene-level similarity relationships. By exploiting the dual-level semantic concepts, we propose an end-to-end deep learning framework for object-level multilabel annotation of RS images. The proposed framework consists of a shared convolutional neural network for discriminative feature learning, a classification branch for multilabel annotation and an embedding branch for preserving the scene-level similarity relationships. In the classification branch, an attention mechanism is introduced to generate attention-aware features, and skip-layer connections are incorporated to combine information from multiple layers. The philosophy of the embedding branch is that images with the same scene-level semantic concepts should have similar visual representations. The proposed method adopts the binary cross-entropy loss for classification and the triplet loss for image embedding learning. The evaluations on three multilabel RS image data sets demonstrate the effectiveness and superiority of the proposed method in comparison with the state-of-the-art methods.

End-to-end trained encoder–decoder convolutional neural network for fetal electrocardiogram signal denoising

Objective: Non-invasive fetal electrocardiography has the potential to provide vital information for evaluating the health status of the fetus. However, the low signal-to-noise ratio of the fetal electrocardiogram (ECG) impedes the applicability of the method in clinical practice. Quality improvement of the fetal ECG is of great importance for providing accurate information to enable support in medical decision-making. In this paper we propose the use of artificial intelligence for the task of one-channel fetal ECG enhancement as a post-processing step after maternal ECG suppression. Approach: We propose a deep fully convolutional encoder–decoder framework, learning end-to-end mappings from noise-contaminated fetal ECGs to clean ones. Symmetric skip-layer connections are used between corresponding convolutional and transposed convolutional layers to help recover the signal details. Main results: Experiments on synthetic data show an average improvement of 7.5 dB in the signal-to-noise ratio (SNR) for input SNRs in the range of −15 to 15 dB. Application of the method with real signals and subsequent ECG interval analysis demonstrates a root mean square error of 9.9 and 14 ms for the PR and QT intervals, respectively, when compared with simultaneous scalp measurements. The proposed network can achieve substantial noise removal on both synthetic and real data. In cases of highly noise-contaminated signals some morphological features might be unreliably reconstructed. Significance: The presented method has the advantage of preserving individual variations in pulse shape and beat-to-beat intervals. Moreover, no prior knowledge on the power spectra of the noise or the pulse locations is required.

High-Level Semantic Networks for Multi-Scale Object Detection

To better solve scale variance problem, deep multi-scale methods usually detect objects of different scales by different in-network layers. However, the semantic levels of features from different layers are usually inconsistent. In this paper, we propose a multi-branch and high-level semantic network by gradually splitting a base network into multiple different branches. As a result, the different branches have same depth and the output features of different branches have similarly high-level semantics. Due to the difference of receptive fields, the different branches are suitable to detect objects of different scales. Meanwhile, the multi-branch network does not introduce additional parameters by sharing the convolutional weights of different branches. To further improve detection performance, skip-layer connections are used to add context to the branch of relatively small receptive field, and dilated convolution is incorporated to enlarge the resolutions of output feature maps. When they are embedded into Faster RCNN architecture, the weighted scores of proposal generation network and proposal classification network are further proposed. Experiments on three pedestrian datasets (i.e., the KITTI dataset, the Caltech dataset, and the Citypersons dataset), one face dataset (i.e., the WIDER FACE dataset), and two general object datasets (i.e., the COCO benchmark and the PASCAL VOC dataset) demonstrate the effectiveness and generality of proposed method. On these datasets, our method achieves state-of-the-art performance.

Retinal Vessels Segmentation Based on Dilated Multi-Scale Convolutional Neural Network

Accurate segmentation of retinal vessels is a basic step in diabetic retinopathy (DR) detection. Most methods based on deep convolutional neural network (DCNN) have small receptive fields, and hence they are unable to capture global context information of larger regions, with difficult to identify pathological. The final segmented retina vessels contain more noise with low classification accuracy. Therefore, in this paper, we propose a DCNN structure named as D-Net. In the encoding phase, we reduced the loss of feature information by reducing the downsampling factor, which reduced the difficulty of tiny thin vessels segmentation. We use the combined dilated convolution to effectively enlarge the receptive field of the network and alleviate the “grid problem” that exists in the standard dilated convolution. In the proposed multi-scale information fusion module (MSIF), parallel convolution layers with different dilation rates are used, so that the model can obtain more dense feature information and better capture retinal vessel information of different sizes. In the decoding module, the skip layer connection is used to propagate context information to higher resolution layers, so as to prevent low-level information from passing the entire network structure. Finally, our method was verified on DRIVE, STARE, and CHASE dataset. The experimental results show that our network structure outperforms some state-of-art method, such as N 4 -fields, U-Net, and DRIU in terms of accuracy, sensitivity, specificity, and ${AUC_{ROC}}$ . Particularly, D-Net outperforms U-Net by 1.04 %, 1.23 %, and 2.79% in DRIVE, STARE, and CHASE dataset, respectively.

Hyper-Siamese network for robust visual tracking

Matching-based tracking has drawn increasingly interest in the object tracking field, among which SiamFC tracker shows great potentials in achieving high accuracy and efficiency. However, the feature representations of target in SiamFC are extracted by the last layer of convolutional neural networks and mainly capture semantic information, which makes SiamFC drift easily in presence of similar distractors. Considering that the different layers of convolutional neural networks characterize the target from different perspectives and the lower-level feature maps of SiamFC are computed beforehand, in this paper we design a skip-layer connection network named Hyper-Siamese to aggregate the hierarchical feature maps of SiamFC and constitute the hyper-feature representations of the target. Hyper-Siamese network is trained end-to-end offline on the ILSVRC2015 dataset and later utilized for online tracking. By visualizing the outputs of different layers and comparing the tracking results under various concatenation mode of layers, we prove that different convolutional layers are all useful for object tracking. Experimental results on the OTB100 and TC128 benchmarks demonstrate that our proposed algorithm performs favorably against not only the foundation tracker SiamFC (2.9% gain in OS rate and 2.8% gain in DP rate on OTB100) but also many state-of-the-art trackers. Meanwhile, our proposed tracker can achieve a real-time tracking speed (25 fps).

A Multi-strategy Region Proposal Network

The Faster Region-based Convolutional Network (Faster R-CNN) was recently proposed achieving outstanding performance for object detection. Specially, a Region Proposal Network (RPN) is designed to efficiently predict region proposals with a wide range of scales and aspect ratios in Faster R-CNN. Nevertheless, once the number and quality of region proposals generated by RPN are not ideal the object detection performance of Faster R-CNN is affected. In this paper, multiple strategies are applied to address these limitations and improve RPN. Hence, a novel architecture for region proposal generation is presented which is named as Multi-strategy Region Proposal Network (MSRPN). Four improvements are presented in MSRPN. Firstly, a novel skip-layer connection network is designed for combining multi-level features and boosting the ability of pooling layers. Thereupon, the quality of region proposals is strengthened. Secondly, improved anchor boxes are introduced with adaptive aspect ratio and evenly distributed interval of selected scales. In this way, the number of predicted region proposals for detection is seriously reduced and the efficiency of object localization is increased. Particularly, the capability of small object detection is enhanced by applying the first and second improvements. Thirdly, classification layer and regression layer are unified as a single convolutional layer. Furthermore, the model complexity of output layer is reduced. Thus, the speed of training and testing is accelerated. Fourthly, the bounding box regression part of multi-task loss function in RPN is improved. Consequently, the performance of bounding box regression is promoted.In the experiment, MSRPN is compared with the Fast Region-based Convolutional Network (Fast R-CNN), Faster R-CNN, Inside-Outside Net (ION), Multi-region CNN (MR-CNN) and HyperNet approaches. MSRPN achieves the state-of-the-art mean average precision (mAP) of 78.9%, 74.8% and 32.1% on PASCAL VOC 2007, 2012 and MS COCO data sets with the deep VGG-16 model, surpassing other five object detection methods. Simultaneously, the above experiment results are obtained by MSRPN with only 150 region proposals per image. Additionally, MSRPN gets excellent performance on small object detection. Furthermore, MSRPN runs at 6 fps which is faster than other methods. In conclusion, the MSRPN method can provide important support for the intelligent object detection systems.

Neural networks, linear functions and neglected non-linearity

The multiplicity of approximation theorems for Neural Networks do not relate to approximation of linear functions per se. The problem for the network is to construct a linear function by superpositions of non-linear activation functions such as the sigmoid function. This issue is important for applications of NNs in statistical tests for neglected nonlinearity, where it is common practice to include a linear function through skip-layer connections. Our theoretical analysis and evidence point in a similar direction, suggesting that the network can in fact provide linear approximations without additional ‘assistance’. Our paper suggests that skip-layer connections are unnecessary, and if employed could lead to misleading results.