Neural Net Approach Research Articles

Kernel methods are competitive for operator learning

We present a general kernel-based framework for learning operators between Banach spaces along with a priori error analysis and comprehensive numerical comparisons with popular neural net (NN) approaches such as Deep Operator Networks (DeepONet) [46] and Fourier Neural Operator (FNO) [45]. We consider the setting where the input/output spaces of target operator G†:U→V are reproducing kernel Hilbert spaces (RKHS), the data comes in the form of partial observations ϕ(ui),φ(vi) of input/output functions vi=G†(ui) (i=1,…,N), and the measurement operators ϕ:U→Rn and φ:V→Rm are linear. Writing ψ:Rn→U and χ:Rm→V for the optimal recovery maps associated with ϕ and φ, we approximate G† with G¯=χ∘f¯∘ϕ where f¯ is an optimal recovery approximation of f†:=φ∘G†∘ψ:Rn→Rm. We show that, even when using vanilla kernels (e.g., linear or Matérn), our approach is competitive in terms of cost-accuracy trade-off and either matches or beats the performance of NN methods on a majority of benchmarks. Additionally, our framework offers several advantages inherited from kernel methods: simplicity, interpretability, convergence guarantees, a priori error estimates, and Bayesian uncertainty quantification. As such, it can serve as a natural benchmark for operator learning.

Noninvasive monitoring system forTenebrio molitor larvae based on image processing with a watershed algorithm and a neural net approach

Due to the increase of the human world population, modern-day research is looking for new methods of protein exploitation. Therefore the authors conducted a joint research project with the goal to automate the breeding ofTenebrio molitor as a novel protein source. An important task is to monitor the size of larvae in order to control the rearing process. In this work, a suitable algorithm is presented to measure the size distribution of the population. It is a combination of classical image processing functions and a neural net to enhance the dataset for a more reliable result. The output can be used to determine the most efficient time for harvesting. First, a grayscale picture of the insects in one box is taken and binarised by a threshold algorithm. The connected objects in this image are separated by an irregular watershed algorithm that delivers separate segments of larvae. Not all single segments can be used for measuring the size distribution; therefore, an artificial neural network is used for a classification. In the end, the algorithm separates the segments given by the watershed and categorises them into four categories: good segments, medium segments, bad segments, and artefacts. The good segments have a recall rate of 91.4%. In the end, the identified segments can be used to establish a method for determining the size distribution and, thus, to document the growth of the larvae.

Prediction of straight tooth milling of Scots pine wood by shank cutter based on neural net computations and regression analysis

Regression models and a neural net approach were used to predict the cutting performance during milling of Scots pine (Pinus sylvestris L.) by shank cutter. The influence of rake angle, spindle speed, and milling depth on surface roughness of the workpiece, as well as the connection between the milling force and the surface roughness, were thoroughly considered. Four approaches were used to predict the workpiece’s surface roughness based on the experimental data: Back Propagation Neural Network (BPNN), Radial Basis Function Neural Network (RBFNN), Support Vector Machines (SVM), and multiple linear regression. The comparative analysis of the predictive models showed that Neural Network (NN) had preferable performance for prediction of machined surface roughness, with an R2 of 0.98. The SVM had certain fluctuations and the R2 of the multiple linear regression was just 0.87, indicating that they did not fit well for prediction machined surface roughness. In summary, the effective trend of milling parameters on the machined surface roughness of Scots pine was similar to multiple nonlinear regression, and the accurate prediction by BPNN model can provide technical support for the surface roughness of the Scots Pine and enhance shank cutter performance.

Neural Net Approach For Image Thinning

Thinning is the operation that seeks to reduce a connected region of pixels of a given property set to a small size. One of the most applications of thinning is the image recognission. In this paper, we proposed a new algorithm for thinning an image using Neural Net approach called (Thinning Back- propagation algorithm, TBP), the results show that the operation considered including Zhang Suen algorithm, Hilditch's algorithm. Applying Zhang Suen and Hilditch's algorithms showed that the output of the Hilditch's algorithm is better from the presence of shape accuracy point of view. The proposed Thinning Back-propagation (TBP) algorithm is superior to other algorithms; the results show that the Hilditch's algorithm gives somewhat better results than Zhang Suen algorithm, therefore the proposed TBP has been implemented depending on the Hilditch's algorithm. The accuracy of the TBP is equivalent to Hilditch's algorithm but it is a little bit faster than Hilditch's algorithm. Experimental results are presented to compare the performance of the algorithms.

Learning Accurate and Interpretable Decision Rule Sets from Neural Networks

This paper proposes a new paradigm for learning a set of independent logical rules in disjunctive normal form as an interpretable model for classification. We consider the problem of learning an interpretable decision rule set as training a neural network in a specific, yet very simple two-layer architecture. Each neuron in the first layer directly maps to an interpretable if-then rule after training, and the output neuron in the second layer directly maps to a disjunction of the first layer rules to form the decision rule set. Our representation of neurons in this first rules layer enables us to encode both the positive and the negative association of features in a decision rule. State-of-the-art neural net training approaches can be leveraged for learning highly accurate classification models. Moreover, we propose a sparsity-based regularization approach to balance between classification accuracy and the simplicity of the derived rules. Our experimental results show that our method can generate more accurate decision rule sets than other state-of-the-art rule-learning algorithms with better accuracy-simplicity trade-offs. Further, when compared with uninterpretable black-box machine learning approaches such as random forests and full-precision deep neural networks, our approach can easily find interpretable decision rule sets that have comparable predictive performance.

Systematic Review of Digital Phenotyping and Machine Learning in Psychosis Spectrum Illnesses.

Digital phenotyping is the use of data from smartphones and wearables collected in situ for capturing a digital expression of human behaviors. Digital phenotyping techniques can be used to analyze both passively (e.g., sensor) and actively (e.g., survey) collected data. Machine learning offers a possible predictive bridge between digital phenotyping and future clinical state. This review examines passive digital phenotyping across the schizophrenia spectrum and bipolar disorders, with a focus on machine-learning studies. A systematic review of passive digital phenotyping literature was conducted using keywords related to severe mental illnesses, data-collection devices (e.g., smartphones, wearables, actigraphy devices), and streams of data collected. Searches of five databases initially yielded 3312 unique publications. Fifty-one studies were selected for inclusion, with 16 using machine-learning techniques. All studies differed in features used, data pre-processing, analytical techniques, algorithms tested, and performance metrics reported. Across all studies, the data streams and other study factors reported also varied widely. Machine-learning studies focused on random forest, support vector, and neural net approaches, and almost exclusively on bipolar disorder. Many machine-learning techniques have been applied to passively collected digital phenotyping data in schizophrenia and bipolar disorder. Larger studies, and with improved data quality, are needed, as is further research on the application of machine learning to passive digital phenotyping data in early diagnosis and treatment of psychosis. In order to achieve greater comparability of studies, common data elements are identified for inclusion in future studies.

Evaluation and comparison of a machine learning cloud identification algorithm for the SLSTR in polar regions

A Feed Forward Neural Net (NN) approach to distinguish between clouds and the surface has been applied to the Sea and Land Surface Temperature Radiometer in polar regions. The masking algorithm covers the Arctic, Antarctic and regions typically classified as the cryosphere such as northern hemisphere permafrost. The mask has been trained using collocations with the CALIOP active lidar, which in narrow strips provide more accurate detection of cloud, and was subsequently evaluated as a function of cloud type and surface type. The mask was compared with the existing operational Bayesian and Empirical cloud masks by eye and also statistically using CALIOP data. It was found to perform exceptionally well in the polar regions. The Kuiper skill score improved from 0.28, for the operational Bayesian and 0.17 for the Empirical masks to 0.77 for the NN. The NN algorithm also has a much more homogeneous performance over all surface types. The key improvement came from better identification of clear scenes; for the NN mask, the same performance in terms of contamination of cloudy pixels in the sample of identified clear pixels can be achieved while retaining 40% of the clear pixels compared with 10% for the operational cloud identification. The algorithm performed with almost the same skill over sea and land. The best performance was achieved for opaque clouds while transparent and broken clouds showed slightly reduced accuracy.

Combined land surface emissivity and temperature estimation from Landsat 8 OLI and TIRS

Remote sensing of Land Surface Temperature (LST) generally requires atmospheric parameters and the emissivity (∊) of the target to be estimated. The atmospheric up- and downwelling radiances and transmittance can be accurately modelled using radiative transfer models and profiles of relative humidity and temperature, either measured by radiosonde probes or retrieved from assimilating weather models. The estimation of ∊ is a large source of uncertainty in the resulting LST product, and there are various approaches using multi-angle observations, multispectral optical or multispectral thermal infrared imagery. In this paper, the estimation of LST from the Thermal InfraRed Sensor (TIRS) on board Landsat 8 is evaluated using more than 6 years of in situ temperature measurements from a network of 14 Autonomous Weather Stations (AWS) in Belgium. ∊ is estimated from concomitant atmospherically corrected imagery from the Operational Land Imager (OLI) using two new neural network approaches trained on ECOSTRESS spectra, and an established NDVI based method. Results are compared to using ∊=1 and the ASTER Global Emissivity Dataset. LST retrievals from L8/TIRS perform well for all emissivity data sources for >500 matchups with AWS subsoil temperature measurements: Mean Differences 0.8–3.7 K and unbiased Root Mean Squared Differences of 2.9–3.5 K for both B10 and B11. The use of unity emissivity gives the best results in terms of MD (0.8 K) and unb-RMSD (3 K). Similar ranges of unb-RMSD are found for >500 matchups with broadband radiometer temperatures (2.6–3.1 K), that have lower absolute MD values (−2.2–0.6 K). For the radiometer temperatures, both the neural net approaches gave lowest MD, in the best case ±0.1 K. The present investigation can hence recommend the neural nets to derive ∊ for the retrieval of LST over the AWS in Belgium. Using published matchup results from other authors however, no single source of ∊ data performed better than ∊=1, but this could be due to their low number of matchups. Further efforts for estimating representative pixel average emissivities are needed, and establishing a denser in situ measurement network over varied land use, with rather homogeneous land cover within a TIRS pixel, may aid further validation of a per pixel and per scene ∊ estimates from multispectral imagery. AWS data seems valuable for evaluation of satellite LST, with the advantage of a much lower cost and higher potential matchup density compared to conventional radiometers.

Simulation of hydrodynamics and water quality in Xi’an Yanming Lake China

In order to facilitate works of water environment management in Yanming Lake Xi’an Shaanxi, the MIKE 21 hydrodynamic model coupled with the advection-dispersion equation was employed to develop a water quality model to investigate the characteristics of the flow field on the conditions of different wind and to simulate the variations of pollutants, which were Chemical Oxygen Demand (COD), Total Nitrogen (TN) and Total Phosphate (TP). The Back propagation Neural Net (BPPN) approach was introduced to utilize the results of simulation to evaluate water quality in Yanming Lake. Additionally, based on the distribution characteristic of pollutants, three scenarios for external pollutants’ load reduction were proposed. Response Characteristics of water quality after implementation of different scenarios was conducted. The results showed that the time-average velocity value ranged from 0 to 0.01 m/s in Yanming Lake on the condition of both no wind and southwest wind. Six circulations could be induced by southwest wind in the flow field. The time-average COD concentration field could be divided into three regions, whose concentration was 12-13 mg/L, 13-14 mg/L, and 14-15 mg/L respectively. Time-average TN concentration field consisted of two regions, whose concentration was 2.85-3.00 mg/L and 3.00-3.15 mg/L separately, and TN concentration exceeded the standard of Grade V (GB3838-2002). Time-average TP concentration field had no significant zoned phenomenon and its concentration ranged from 0.040 to 0.048 mg/L. Results from evaluation of water quality indicated that water quality was in the standard of inferior Grade V. Additionally, water quality will be better improved after implementation of Scenario 2 rather than Scenario 1 and 3.

New combinative method for solving one-dimensional fractional Bratu differential equation

Indeed, the special advantages of artificial neural networks in modelling and solving complex real-world phenomena, have made them powerful computational and mathematical tools in applied sciences and engineering. The main purpose of this paper is to derive an iterative method based on a combination of neural nets approach and power-series method for an approximate treatment of the fractional Bratu-type equations. Supposedly, the problem considered has a solution in terms of the series expansion of unknown function, the proper implementation of a suitable neural architecture leads to estimate the unknown series coefficients, systematically. So as to show practical applicability and robustness of this technique, Bratu's boundary value problem in one-dimensional planar is solved as numerical example. Achieved numerical and simulative results reveal that the method is very effective and powerful.

Evaluation of multisource data for glacier terrain mapping: a neural net approach

Spectrally similar nature of land covers in a glacierized terrain hampers their automated mapping from multispectral satellite data, which may be overcome by using multisource data. In the present study, an artificial neural network (ANN)-based information extraction approach was applied for mapping the Kolahoi glacier and adjoining areas, using Landsat TM (Thematic Mapper) data and several ancillary layers such as image transformations and topographic attributes. Results reveal that ANN (highest overall accuracy (OA): 83.74%) outperforms maximum likelihood classifier (highest OA: 66.90%) and the incorporation of ancillary data into the classification process significantly enhances the mapping accuracy (>9%), particularly the addition of Near Infrared Red/Short Wave Infrared (NIR/SWIR) data to the spectral data. A nine-band combination dataset (spectral data, slope, Red/NIR and decorrelation stretch) was found to be the best multisource dataset. Results of the Z-tests (at 95% confidence level) also corroborate and statistically validate the above findings.

Semantic similarity measurement based on knowledge mining: an artificial neural net approach

This article presents a new approach to automatically measure semantic similarity between spatial objects. It combines a description logic based knowledge base (an ontology) and a multi-layer neural network to simulate the human process of similarity perception. In the knowledge base, spatial concepts are organized hierarchically and are modelled by a set of features that best represent the spatial, temporal and descriptive attributes of the concepts, such as origin, shape and function. Water body ontology is used as a case study. The neural network was designed and human subjects' rankings on similarity of concept pairs were collected for data training, knowledge mining and result validation. The experiment shows that the proposed method achieves good performance in terms of both correlation and mean standard error analysis in measuring the similarity between neural network prediction and human subject ranking. The application of similarity measurement with respect to improving relevancy ranking of a semantic search engine is introduced at the end.

The Research of Text Mining Based on Self-Organizing Maps

New methods that are user-friendly and efficient are needed for guidance among the masses of textual information available in the Internet and the World Wide Web. This paper describes text mining has been gaining popularity in the knowledge discovery field, particularity with the increasing availability of digital documents in various languages from all around the world. In this work, we attempt to develop a language-neutral method to tackle the linguistics difficulties in the text mining process. Using a variation of automatic clustering techniques, which apply a neural net approach, namely the Self-Organizing Maps (SOM). The SOM is used to generate two maps, namely the word cluster map and the document cluster map, which reveal the relationships among words and documents respectively. The search process incorporates these two maps and effectively finds the relevant documents according to the keywords specified in the query. The conceptually associated web documents are found not only by the specific keywords but the relevant words found by the word cluster map.

Neural Network Approaches to Grade Adult Depression

Depression is a common but worrying psychological disorder that adversely affects one's quality of life. It is more ominous to note that its incidence is increasing. On the other hand, screening and grading of depression is still a manual and time consuming process that might be biased. In addition, grades of depression are often determined in continuous ranges, e.g., 'mild to moderate' and 'moderate to severe' instead of making them more discrete as 'mild', 'moderate', and 'severe'. Grading as a continuous range is confusing to the doctors and thus affecting the management plan at large. Given this practical issue, the present paper attempts to differentiate depression grades more accurately using two neural net learning approaches-'supervised', i.e., classification with Back propagation neural network (BPNN) and Adaptive Network-based Fuzzy Inference System (ANFIS) classifiers, and 'unsupervised', i.e., 'clustering' technique with Self-organizing map (SOM), built in MATLAB 7. The reason for using the supervised and unsupervised learning approaches is that, supervised learning depends exclusively on domain knowledge. Supervision may induce biasness and subjectivities related to the decision-making. Finally, the performance of BPNN and ANFIS are compared and discussed. It was observed that ANFIS, being a hybrid system, performed much better compared to the BPNN classifier. On the other hand, SOM-based clustering technique is also found useful in constructing three distinct clusters. It also assists visualizing the multidimensional data clusters into 2-D.

An Instance-Based Algorithm With Auxiliary Similarity Information for the Estimation of Gait Kinematics From Wearable Sensors

Wearable human movement measurement systems are increasingly popular as a means of capturing human movement data in real-world situations. Previous work has attempted to estimate segment kinematics during walking from foot acceleration and angular velocity data. In this paper, we propose a novel neural network [GRNN with Auxiliary Similarity Information (GASI)] that estimates joint kinematics by taking account of proximity and gait trajectory slope information through adaptive weighting. Furthermore, multiple kernel bandwidth parameters are used that can adapt to the local data density. To demonstrate the value of the GASI algorithm, hip, knee, and ankle joint motions are estimated from acceleration and angular velocity data for the foot and shank, collected using commercially available wearable sensors. Reference hip, knee, and ankle kinematic data were obtained using externally mounted reflective markers and infrared cameras for subjects while they walked at different speeds. The results provide further evidence that a neural net approach to the estimation of joint kinematics is feasible and shows promise, but other practical issues must be addressed before this approach is mature enough for clinical implementation. Furthermore, they demonstrate the utility of the new GASI algorithm for making estimates from continuous periodic data that include noise and a significant level of variability.

An artificial neural net assisted approach to editing edges in petrographic images collected with the rotating polarizer stage

A practical methodology to edit edges within petrographic images is presented. The procedure uses the existing output of a standard segmentation routine as input. Edges are skeletonized and converted into segments which separate two grains and join at nodal points. As series of colour and texture parameters are extracted from all grains. For each edge segment, an artificial neural net (ANN) evaluates differences in the parameters for the grains separated by the segment. ANN output is used to classify segments as true or false edges and can be thresholded at different levels. A manual procedure allows for the final classification of the segments. This method significantly improves the speed with which edges can be edited in preparation for other studies.

Time of searching for similar binary vectors in associative memory

Times of searching for similar binary vectors in neural-net and traditional associative memories are investigated and compared. The neural-net approach is demonstrated to surpass the traditional ones even if it is implemented on a serial computer when the entropy of a space of signals is of order of several hundreds and the number of stored vectors is vastly larger than the entropy.

A Multilingual Text Mining Approach Based on Self-Organizing Maps

This paper describes our work on developing a language-independent technique for discovery of implicit knowledge from multilingual information sources. Text mining has been gaining popularity in the knowledge discovery field, particularity with the increasing availability of digital documents in various languages from all around the world. However, currently most text mining tools mainly focus only on processing monolingual documents (particularly English documents): little attention has been paid to apply the techniques to handle the documents in Asian languages, and further extend the mining algorithms to support the aspects of multilingual information sources. In this work, we attempt to develop a language-neutral method to tackle the linguistics difficulties in the text mining process. Using a variation of automatic clustering techniques, which apply a neural net approach, namely the Self-Organizing Maps (SOM), we have conducted several experiments to uncover associated documents based on a Chinese corpus, Chinese-English bilingual parallel corpora, and a hybrid Chinese-English corpus. The experiments show some interesting results and a couple of potential paths for future work in the field of multilingual information discovery. Besides, this work is expected to act as a starting point for exploring the impacts on linguistics issues with the machine-learning approach to mining sensible linguistics elements from multilingual text collections.

A neural net approach to the automated analysis of crystallization trials

A neural net approach to the automated analysis of crystallization trials

A flexible brand choice model based on neural net methodology A comparison to the linear utility multinomial logit model and its latent class extension

Brand choice models as a rule have a linear (deterministic) utility function, i.e. they conceive utility as linear combination of predictors like price, sales promotion variables, brand name and other product attributes. To discover nonlinear effects on brands' utilities in a flexible way we specify deterministic utility by means of a certain type of neural net. This feedforward multilayer perceptron is able to approximate any continuous multivariate function and its derivatives with the desired level of precision. In an empirical study the neural net based choice model leads to better out-of-sample results than homogeneous and heterogeneous versions of linear utility MNL models. On the other hand the latent class variant of the linear utility MNL model attains better fit values for estimation data than the neural net model. The neural net approach implies different choice elasticities for most predictors and identifies nonlinear effects (like interaction effects, thresholds, saturation effects).