Simple Training Process Research Articles

ContraFeat: Contrasting Deep Features for Semantic Discovery

StyleGAN has shown strong potential for disentangled semantic control, thanks to its special design of multi-layer intermediate latent variables. However, existing semantic discovery methods on StyleGAN rely on manual selection of modified latent layers to obtain satisfactory manipulation results, which is tedious and demanding. In this paper, we propose a model that automates this process and achieves state-of-the-art semantic discovery performance. The model consists of an attention-equipped navigator module and losses contrasting deep-feature changes. We propose two model variants, with one contrasting samples in a binary manner, and another one contrasting samples with learned prototype variation patterns. The proposed losses are computed with pretrained deep features, based on our assumption that the features implicitly possess the desired semantic variation structure including consistency and orthogonality. Additionally, we design two metrics to quantitatively evaluate the performance of semantic discovery methods on FFHQ dataset, and also show that disentangled representations can be derived via a simple training process. Experimentally, we show that our models achieve state-of-the-art semantic discovery results without relying on layer-wise manual selection, and these discovered semantics can be used to manipulate real-world images.

Robust deep semi-supervised learning with label propagation and differential privacy

Semi-supervised learning (SSL) methods provide a powerful tool for utilizing abundant unlabeled data to strengthen standard supervised learning. Traditional graph-based SSL methods prevail in classical SSL problems for their intuitional implementation and effective performance. However, they encounter troubles when applying to image classification followed by modern deep learning, since the diffusion algorithms face the curse of dimensionality. In this study, we propose a simple and efficient SSL method, combining a graph-based SSL paradigm with differential privacy. We aim at developing coherent latent feature space of deep neural networks so that the diffusion algorithm in the latent space can give more precise predictions for unlabeled data. Our approach achieves state-of-the-art performance on the Cifar10, Cifar100, and Mini-imagenet benchmark datasets and obtains an error rate of 18.56% on Cifar10 using only 1% of all labels. Furthermore, our approach inherits the benefits of graph-based SSL methods with a simple training process and can be easily combined with any network architecture.

Virtual Grid Mapping for Visual Size Measurements

This paper proposes a novel, more robust method for single-image visual size measurements, named Virtual Grid Mapping (VGM). VGM requires just a single image as input; however, it does not require any prior information concerning the scene like the horizon line or reference objects. Unlike current methods, the VGM approach uses a grid of virtual 3D points projected to the 2D image plane, along with the fusion of probable depth values indicated by 2D-3D point correspondences, it manages to reduce the uncertainty originating from the calibration and the positioning of the camera; thus, providing more accurate measurements. Given the geometric properties of the camera, a novel approach of VGM is that it automatically generates and projects a grid of virtual 3D points to the 2D image plane, enabling the establishment of approximative correspondences between 3D points of the real world and 2D points of the image plane. Then, by considering these initial 3D-2D point correspondences as known, a range of possible depth values is estimated, and through an adaptive fusion process, the size of an object of interest can be accurately approximated. Another advantage of VGM over the state-of-the-art deep learning -based methods, is that it requires a simple training process. Experiments performed on simulated and real image datasets, captured in controlled conditions and <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">in-the-wild</i> , show that the mean absolute percentage error (MAPE) of the proposed method ranges between 3.13% and 11.66%.

An improved deep echo state network inspired by tissue-like P system forecasting for non-stationary time series

As a recurrent neural network, ESN has attracted wide attention because of its simple training process and unique reservoir structure, and has been applied to time series prediction and other fields. However, ESN also has some shortcomings, such as the optimization of reservoir and collinearity. Many researchers try to optimize the structure and performance of deep ESN by constructing deep ESN. However, with the increase of the number of network layers, the problem of low computing efficiency also follows. In this paper, we combined membrane computing and neural network to build an improved deep echo state network inspired by tissue-like P system. Through analysis and comparison with other classical models, we found that the model proposed in this paper has achieved great success both in predicting accuracy and operation efficiency.

EasyVRModeling

The latest innovations of VR make it possible to construct 3D models in a holographic immersive simulation environment. In this paper, we develop a user-friendly mid-air interactive modeling system named EasyVRModeling. We first prepare a dataset consisting of diverse components and precompute the discrete signed distance function (SDF) for each component. During the modeling phase, users can freely design complicated shapes with a pair of VR controllers. Based on the discrete SDF representation, any CSG-like operation (union, intersect, subtract) can be performed voxel-wise. Throughout the modeling process, we maintain one single dynamic SDF for the whole scene so that the zero-level set surface of the SDF exactly encodes the up-to-date constructed shape. Both SDF fusion and surface extraction are implemented via GPU to allow for smooth user experience. We asked 34 volunteers to create their favorite models using EasyVRModeling. With a simple training process for several minutes, most of them can create a fascinating shape or even a descriptive scene very quickly.

Determining the defect locations and sizes in elastic plates by using the artificial neural network and boundary element method

In this work, a new approach for solving the inverse problem of the defect location in an elastic plate is developed by combining the artificial neural network (ANN) and the boundary element method (BEM). The inverse problem is formulated as a regression problem, which extracts the solution of highest probability through machine learning (ML) from a large amount of data. The efficiency and accuracy of data generation is guaranteed by using the BEM, which solves for strain values on the boundaries of plates with circular defects and creates the ML datasets. The ANNs, which are constructed by a fully connected multi-layer perceptron, are trained using these datasets to predict the center coordinates and radii of the circular defects and can achieve about 98% accuracy in the detection. Compared with the Lamb wave based structural health monitoring (SHM) techniques, which in general require signal generation, collection and processing, the proposed approach only requires the data of boundary strains of the considered structures as the input data and a simple training process. Therefore, it is much easier to implement and has great potential in applications of SHM.

Wi-Wheat+: Contact-free wheat moisture sensing with commodity WiFi based on entropy

In this paper, we propose a contact-free wheat moisture monitoring system, termed Wi-Wheat+, to address the several limitations of the existing grain moisture detection technologies, such as time-consuming process, expensive equipment, low accuracy, and difficulty in real-time monitoring. The proposed system is based on Commodity WiFi and is easy to deploy. Leveraging WiFi CSI data, this paper proposes a feature extraction method based on multi-scale and multi-channel entropy. The feasibility and stability of the system are validated through experiments in both Line-Of-Sight (LOS) and Non-Line-Of-Sight (NLOS) scenarios, where ten types of wheat moisture content are tested using multi-class Support Vector Machine (SVM). Compared with the Wi-Wheat system proposed in our prior work, Wi-Wheat+ ​has higher efficiency, requiring only a simple training process, and can sense more wheat moisture content levels.

Low-power-consumption physical reservoir computing model based on overdamped bistable stochastic resonance system

The physical implementation of reservoir computing (RC), a brain-inspired computing framework, is attracting increasing attention in various research fields owing to its capability of quick learning and the relatively simple training process. Although several physical RC models have been envisaged and realized, most of them require additional peripherals; it leads to an increase in the power consumption of the system. In this study, we propose a novel RC model that is based on an overdamped bistable system and exhibits a counter-intuitive phenomenon called stochastic resonance, through which it can transfer noise energy to the information-carrying signal to realize learning with a comparatively low power consumption. The proposed model also possesses the functional capability of filtering and amplification and thus does not require additional peripheral equipment. In order to prove the feasibility and determine the desired operation mode of the system, two basic benchmark tests, namely short-term memory and parity check tasks, were employed to assess the proposed model. The results verify that this work will potentially act as a stepping stone towards realizing even better low-power-consumption physical RC.

Detection of the location of pneumothorax in chest X-rays using small artificial neural networks and a simple training process

The purpose of this study was to evaluate the diagnostic performance achieved by using fully-connected small artificial neural networks (ANNs) and a simple training process, the Kim-Monte Carlo algorithm, to detect the location of pneumothorax in chest X-rays. A total of 1,000 chest X-ray images with pneumothorax were taken randomly from NIH (the National Institutes of Health) public image database and used as the training and test sets. Each X-ray image with pneumothorax was divided into 49 boxes for pneumothorax localization. For each of the boxes in the chest X-ray images contained in the test set, the area under the receiver operating characteristic (ROC) curve (AUC) was 0.882, and the sensitivity and specificity were 80.6% and 83.0%, respectively. In addition, a common currently used deep-learning method for image recognition, the convolution neural network (CNN), was also applied to the same dataset for comparison purposes. The performance of the fully-connected small ANN was better than that of the CNN. Regarding the diagnostic performances of the CNN with different activation functions, the CNN with a sigmoid activation function for fully-connected hidden nodes was better than the CNN with the rectified linear unit (RELU) activation function. This study showed that our approach can accurately detect the location of pneumothorax in chest X-rays, significantly reduce the time delay incurred when diagnosing urgent diseases such as pneumothorax, and increase the effectiveness of clinical practice and patient care.

Joint architecture and knowledge distillation in CNN for Chinese text recognition

The distillation technique helps transform cumbersome neural networks into compact networks so that models can be deployed on alternative hardware devices. The main advantage of distillation-based approaches include a simple training process, supported by most off-the-shelf deep learning software and no special hardware requirements. In this paper, we propose a guideline for distilling the architecture and knowledge of pretrained standard CNNs. The proposed algorithm is first verified on a large-scale task: offline handwritten Chinese text recognition (HCTR). Compared with the CNN in the state-of-the-art system, the reconstructed compact CNN can reduce the computational cost by >10×and the model size by >8×with negligible accuracy loss. Then, by conducting experiments on two additional classification task datasets: Chinese Text in the Wild (CTW) and MNIST, we demonstrate that the proposed approach can also be successfully applied on mainstream backbone networks.

Style Transfer Using Whitening and Coloring to Realize Feature Transformation

This paper proposes a general image style transfer method based on VGG network. This method does not need to learn and pre-train any kind of style in advance. The algorithm mainly implements the style transfer task by reconstructing the automatic decoder. In the process of feature extraction, the algorithm matches the content features and style features through whitening and coloring. We conducted a series of experiments with different photographic images. The experimental results show that our method has a simple training process, low computational complexity and high computational speed and excellent generalization ability.

Discriminative Autoencoding Framework for Simple and Efficient Anomaly Detection

In this paper, a discriminative autoencoding framework is proposed for semi-supervised anomaly detection using reconstruction errors. The framework only consists of a generator and a discriminative encoder, and the output of the latter is a vector. In the training process, the framework is trained as a generative adversarial network based on quadratic potential divergence. An extra loss added in the objective function enforces the discriminative encoder to use the mean value of the output vector for discrimination, which also empowers it with encoding ability. In the testing process, the trained framework can be used as an autoencoder to reconstruct test samples, where the trained discriminative encoder works as an encoder, and samples with reconstruction errors above a predefined threshold are determined as anomalies. The properties of quadratic potential divergence ensure a simple training process with comparable performance, meanwhile the discriminative encoder with two functions makes full use of training resources and reduces required network structures. Comparisons on benchmark datasets also show the efficiency and superiorities of the proposed methods.

Orchard Free Space and Center Line Estimation Using Naive Bayesian Classifier for Unmanned Ground Self-Driving Vehicle

In the case of autonomous orchard navigation, researchers have developed algorithms that utilize features, such as trunks, canopies, and sky in orchards, but there are still various difficulties in recognizing free space for autonomous navigation in a changing agricultural environment. In this study, we applied the Naive Bayesian classification to detect the boundary between the trunk and the ground and propose an algorithm to determine the center line of free space. The naïve Bayesian classification requires a small number of samples for training and a simple training process. In addition, it was able to effectively classify tree trunk’s points and noise points of the orchard, which are problematic in vision-based processing, and noise caused by small branches, soil, weeds, and tree shadows on the ground. The performance of the proposed algorithm was investigated using 229 sample images obtained from an image acquisition system with a Complementary Metal Oxide Semiconductor (CMOS) Image Sensor (CIS) camera. The center line detected by the unaided-eye manual decision and the results extracted by the proposed algorithm were compared and analyzed for several parameters. In all compared parameters, extracted center line was more stable than the manual center line results.

Stokes Space Modulation Format Identification for Optical Signals Using Probabilistic Neural Network

A Stokes space modulation format identification (MFI) method using probabilistic neural network (PNN) is proposed for coherent optical receivers. According to amplitude histograms obtained by the distribution of Stokes vectors on the s <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> axis, the incoming signals are first classified into PDM-mPSK, PDM-16QAM, and PDM-64 QAM signals based on PNN. To further identify PDM-mPSK signals, the constellation feature of Stokes vectors on s <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sub> -s <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">3</sub> plane is extracted by image processing techniques and then processed by PNN. The high identification accuracy is demonstrated via numerical simulations with 28-Gbaud PDM-QPSK, PDM-8PSK, PDM-16QAM, and PDM-64QAM signals over a wide optical signal-to-noise ratio range. Owing to the characteristic of PNN, the proposed MFI method has simple training process, small training data size and a small number of required symbols. Proof-of-concept experiments have been implemented to verify the effectiveness of the proposed MFI method among 28-Gbaud PDM-QPSK, PDM-8PSK, and PDM-16QAM signals.

Contactor Modeling Technology Based on an Artificial Neural Network

We propose a new contactor modeling method that incorporates the back propagation (BP) neural network to map the complex nonlinear electromechanical coupling relation of the contactor to build its model. First, the artificial neural network (ANN) model collects the actual operational data of the contactor, including the coil voltage, coil current and moving core displacement, and then uses the strong nonlinear fitting ability of the BP neural network to perform the model training. When the training is completed, the ANN model can output the precise displacement according to the input data of the coil voltage and the coil current. Through a simple training process, this method can complete the modeling of any electromagnetic contactor. This method avoids the need to solve the complex magnetic circuit equation of the contactor and thus provides a simple and universal method for calculating the displacement of the electromagnetic switch. The co-simulation system is used to model, train, and analyze the contactor ANN model. Finally, a relevant experiment is conducted to confirm the effectiveness of the ANN model.

Slope stability evaluation using Gaussian processes with various covariance functions

This paper presents a stability evaluation method for slopes based on Gaussian processes (GPs), which is a popular machine learning technique for nonlinear system modeling. Covariance function is one of the most critical parts in GPs modeling, because it determines the properties of sample functions drawn from the Gaussian process prior. Sixteen covariance functions are tested on several datasets for slope stability evaluation problems.Experimental: results show that GPs models can reflect the complex relationship between input and output variables. The obtained results are better or similar to the results obtained by several other existing methods, such as artificial neural networks, support vector machines, etc. The other important attractions of GPs include a simple training process and a predictive distribution of the system output.

Extreme learning machine based spatiotemporal modeling of lithium-ion battery thermal dynamics

Due to the overwhelming complexity of the electrochemical related behaviors and internal structure of lithium ion batteries, it is difficult to obtain an accurate mathematical expression of their thermal dynamics based on the physical principal. In this paper, a data based thermal model which is suitable for online temperature distribution estimation is proposed for lithium-ion batteries. Based on the physics based model, a simple but effective low order model is obtained using the Karhunen–Loeve decomposition method. The corresponding uncertain chemical related heat generation term in the low order model is approximated using extreme learning machine. All uncertain parameters in the low order model can be determined analytically in a linear way. Finally, the temperature distribution of the whole battery can be estimated in real time based on the identified low order model. Simulation results demonstrate the effectiveness of the proposed model. The simple training process of the model makes it superior for onboard application.

Label Transfer by Measuring Compactness

This paper presents a new automatic image annotation algorithm. First, we introduce a new similarity measure between images: compactness. This uses low level visual descriptors for determining the similarity between two images. Compactness shows how close test image features lie to training image feature cluster centers. The measure provides the core for a k-nearest neighbor type image annotation method. Afterward, a formalism for defining different transfer techniques is devised and several label transfer techniques are provided. The method as whole is evaluated on four image annotation benchmarks. The results on these sets validate the accuracy of the approach, which outperforms many state-of-the-art annotation methods. The method presented here requires a simple training process, efficiently combines different feature types and performs better than complex learning algorithms, even in this incipient form. The main contributions of this paper are the usage of compactness as a similarity measure that enables efficient low level feature comparison and an annotation algorithm based on label transfer.

Analysis of prediction performance in wavelet minimum complexity echo state network

Echo state network (ESN) has become one of the most popular recurrent neural networks (RNN) for its good prediction performance of non-linear time series and simple training process. But several problems still prevent ESN from becoming a widely used tool. The most prominent problem is its high complexity with lots of random parameters. Aiming at this problem, a minimum complexity ESN model (MCESN) was proposed. In this paper, we proposed a new wavelet minimum complexity ESN model (WMCESN) to improve the prediction accuracy and increase the practical applicability. Our new model inherits the characters of minimum complexity ESN model using the fixed parameters and simple circle topology. We injected wavelet neurons to replace the original neurons in internal reservoir and designed a wavelet parameter matrix to reduce the computing time. By using different datasets, our new model performed better than the minimum complexity ESN model with normal neurons, but only utilized tiny time cost. We also used our own packets of transmission control protocol (TCP) and user datagram protocol (UDP) dataset to prove that our model can deal with the data packet bit prediction problem well.