Parsimonious Network Research Articles

Evolving large-scale data stream analytics based on scalable PANFIS

The main challenge in large-scale data stream analytics lies in the ability of machine learning to generate large-scale data knowledge in reasonable timeframe without suffering from a loss of accuracy. Many distributed machine learning frameworks have recently been built to speed up the large-scale data learning process. However, most distributed machine learning used in these frameworks still uses an offline algorithm model which cannot cope with the data stream problems. In fact, large-scale data are mostly generated by the non-stationary data stream where its pattern evolves over time. To address this problem, we propose a novel Evolving Large-scale Data Stream Analytics framework based on a Scalable Parsimonious Network based on Fuzzy Inference System (Scalable PANFIS), where the PANFIS evolving algorithm is distributed over the worker nodes in the cloud to learn large-scale data stream. Scalable PANFIS framework incorporates the active learning (AL) strategy and two model fusion methods. The AL accelerates the distributed learning process to generate an initial evolving large-scale data stream model (initial model), whereas the two model fusion methods aggregate an initial model to generate the final model. The final model represents the update of current large-scale data knowledge which can be used to infer future data. Extensive experiments on this framework are validated by measuring the accuracy and running time of four combinations of Scalable PANFIS and other Spark-based built in algorithms. The results indicate that Scalable PANFIS with AL improves the training time to be almost two times faster than Scalable PANFIS without AL. The results also show both rule merging and the voting mechanisms yield similar accuracy in general among Scalable PANFIS algorithms and they are generally better than Spark-based algorithms. In terms of running time, the Scalable PANFIS training time outperforms all Spark-based algorithms when classifying a multi-class label dataset.

Parsimonious Network Based on a Fuzzy Inference System (PANFIS) for Time Series Feature Prediction of Low Speed Slew Bearing Prognosis

In recent years, the utilization of rotating parts, e.g., bearings and gears, has been continuously supporting the manufacturing line to produce a consistent output quality. Due to their critical role, the breakdown of these components might significantly impact the production rate. Prognosis, which is an approach that predicts the machine failure, has attracted significant interest in the last few decades. In this paper, the prognostic approaches are described briefly and advanced predictive analytics, namely a parsimonious network based on a fuzzy inference system (PANFIS), is proposed and tested for low speed slew bearing data. PANFIS differs itself from conventional prognostic approaches, supporting online lifelong prognostics without the requirement of a retraining or reconfiguration phase. The PANFIS method is applied to normal-to-failure bearing vibration data collected for 139 days to predict the time-domain features of vibration slew bearing signals. The performance of the proposed method is compared to some established methods, such as ANFIS, eTS, and Simp_eTS. From the results, it is suggested that PANFIS offers an outstanding performance compared to those methods.

A tutorial on regularized partial correlation networks.

Recent years have seen an emergence of network modeling applied to moods, attitudes, and problems in the realm of psychology. In this framework, psychological variables are understood to directly affect each other rather than being caused by an unobserved latent entity. In this tutorial, we introduce the reader to estimating the most popular network model for psychological data: the partial correlation network. We describe how regularization techniques can be used to efficiently estimate a parsimonious and interpretable network structure in psychological data. We show how to perform these analyses in R and demonstrate the method in an empirical example on posttraumatic stress disorder data. In addition, we discuss the effect of the hyperparameter that needs to be manually set by the researcher, how to handle non-normal data, how to determine the required sample size for a network analysis, and provide a checklist with potential solutions for problems that can arise when estimating regularized partial correlation networks. (PsycINFO Database Record (c) 2018 APA, all rights reserved).

Grid-based computational methods for the design of constraint-based parsimonious chemical reaction networks to simulate metabolite production: GridProd

BackgroundConstraint-based metabolic flux analysis of knockout strategies is an efficient method to simulate the production of useful metabolites in microbes. Owing to the recent development of technologies for artificial DNA synthesis, it may become important in the near future to mathematically design minimum metabolic networks to simulate metabolite production.ResultsWe have developed a computational method where parsimonious metabolic flux distribution is computed for designated constraints on growth and production rates which are represented by grids. When the growth rate of this obtained parsimonious metabolic network is maximized, higher production rates compared to those noted using existing methods are observed for many target metabolites. The set of reactions used in this parsimonious flux distribution consists of reactions included in the original genome scale model iAF1260. The computational experiments show that the grid size affects the obtained production rates. Under the conditions that the growth rate is maximized and the minimum cases of flux variability analysis are considered, the developed method produced more than 90% of metabolites, while the existing methods produced less than 50%. Mathematical explanations using examples are provided to demonstrate potential reasons for the ability of the proposed algorithm to identify design strategies that the existing methods could not identify.ConclusionWe developed an efficient method for computing the design of minimum metabolic networks by using constraint-based flux balance analysis to simulate the production of useful metabolites. The source code is freely available, and is implemented in MATLAB and COBRA toolbox.

Propagation of Local Demand Shocks: Local Specificity and Supply Chain Network Channel

This paper investigates a role of supply chain network in transmitting housing market disruptions during the Great Recession. We build up a unique micro-level data that combines local housing market condition, firms' sales in each local market, and firm-level supply chain network information. Exploiting firm-specific demand shock stemming from cross-market variation in house price changes and an initial difference in firms' local sales, we find that such shock not only affects downstream firms but also transmits to their suppliers. The estimated supplier-level elasticity is quantitatively large, reflecting larger role of downstream firms with higher elasticity in the network structure. To quantify such propagation at the aggregate level, we build up a parsimonious network model calibrated to match the micro-level data. Our counterfactual analysis shows that approximately 18% of the observed drop in the aggregate output can be attributed to the propagating role of the supply chain network.

Abstract 1310: Systematic evaluation of gene networks for discovery of disease genes

Abstract Gene networks are rapidly growing in size and number, raising the question of which networks are most appropriate for a particular application. Here, we evaluate 21 human genome-wide interaction networks for their ability to recover gene sets associated with 446 different diseases and 9 cancer hallmarks. While all networks have some ability in these recovery tasks, we observe a wide range of performance with STRING, GeneMANIA and GIANT networks having the best performance overall. A general tendency is that performance scales with network size, suggesting that new interaction discovery currently outweighs the detrimental effects of false positives. Correcting for size, we find that the DIP network provides the highest efficiency (value per interaction). Based on these results we create a parsimonious composite network with both high efficiency and absolute performance, which outperforms any single resource. This work provides a benchmark for selection of molecular networks in human disease research. Citation Format: Justin K. Huang, Daniel E. Carlin, Michael K. Yu, Wei Zhang, Jason F. Kreisberg, Pablo Tamayo, Trey Ideker. Systematic evaluation of gene networks for discovery of disease genes [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2018; 2018 Apr 14-18; Chicago, IL. Philadelphia (PA): AACR; Cancer Res 2018;78(13 Suppl):Abstract nr 1310.

Systematic Evaluation of Molecular Networks for Discovery of Disease Genes

Gene networks are rapidly growing in size and number, raising the question of which networks are most appropriate for particular applications. Here, we evaluate 21 human genome-wide interaction networks for their ability to recover 446 disease gene sets identified through literature curation, gene expression profiling, or genome-wide association studies. While all networks have some ability to recover disease genes, we observe a wide range of performance with STRING, ConsensusPathDB, and GIANT networks having the best performance overall. A general tendency is that performance scales with network size, suggesting that new interaction discovery currently outweighs the detrimental effects of false positives. Correcting for size, we find that the DIP network provides the highest efficiency (value per interaction). Based on these results, we create a parsimonious composite network with both high efficiency and performance. This work provides a benchmark for selection of molecular networks in human disease research.

Big Data Analytics based on PANFIS MapReduce

In this paper, a big data analytic framework is introduced for processing high-frequency data stream. This framework architecture is developed by combining an advanced evolving learning algorithm namely Parsimonious Network Fuzzy Inference System (PANFIS) with MapReduce parallel computation, where PANFIS has the capability of processing data stream in large volume. Big datasets are learnt chunk by chunk by processors in MapReduce environment and the results are fused by rule merging method, that reduces the complexity of the rules. The performance measurement has been conducted, and the results are showing that the MapReduce framework along with PANFIS evolving system helps to reduce the processing time around 22 percent in average in comparison with the PANFIS algorithm without reducing performance in accuracy.

Parsimonious Kernel Recursive Least Squares Algorithm for Aero-Engine Health Diagnosis

Kernel adaptive filtering (KAF) has gained widespread popularity among the machine learning community for online applications due to its convexity, simplicity, and universal approximation ability. However, the network generated by KAF keeps growing with the accumulation of the training samples, which leads to the increasing memory requirement and computational burden. To address this issue, a pruning approach that attempts to restrict the network size to a fixed value is incorporated into a kernel recursive least squares (KRLS) algorithm, yielding a novel KAF algorithm called parsimonious KRLS (PKRLS). The basic idea of the pruning technique is to remove the center with the least importance from the existing dictionary. The importance of a center is quantified by its contribution to minimizing the cost function. The calculation of the importance measure is formulated in an efficient manner, which facilitates its implementation in online settings. Experimental results on the benchmark tasks show that PKRLS obtains a parsimonious network structure with the satisfactory prediction accuracy. Finally, a multi-sensor health diagnosis approach based on PKRLS is developed for identifying the health state of a degraded aero-engine in real time. A case study in a turbofan engine degradation data set demonstrates that PKRLS provides an effective and efficient candidate for modeling the performance deterioration of real complex systems.

Learning stratified 3D reconstruction

Stratified 3D reconstruction, or a layer-by-layer 3D reconstruction upgraded from projective to affine, then to the final metric reconstruction, is a well-known 3D reconstruction method in computer vision. It is also a key supporting technology for various well-known applications, such as streetview, smart3D, oblique photogrammetry. Generally speaking, the existing computer vision methods in the literature can be roughly classified into either the geometry-based approaches for spatial vision or the learning-based approaches for object vision. Although deep learning has demonstrated tremendous success in object vision in recent years, learning 3D scene reconstruction from multiple images is still rare, even not existent, except for those on depth learning from single images. This study is to explore the feasibility of learning the stratified 3D reconstruction from putative point correspondences across images, and to assess whether it could also be as robust to matching outliers as the traditional geometry-based methods do. In this study, a special parsimonious neural network is designed for the learning. Our results show that it is indeed possible to learn a stratified 3D reconstruction from noisy image point correspondences, and the learnt reconstruction results appear satisfactory although they are still not on a par with the state-of-the-arts in the structure-from-motion community due to largely its lack of an explicit robust outlier detector such as random sample consensus (RANSAC). To the best of our knowledge, our study is the first attempt in the literature to learn 3D scene reconstruction from multiple images. Our results also show that how to implicitly or explicitly integrate an outlier detector in learning methods is a key problem to solve in order to learn comparable 3D scene structures to those by the current geometry-based state-of-the-arts. Otherwise any significant advancement of learning 3D structures from multiple images seems difficult, if not impossible. Besides, we even speculate that deep learning might be, in nature, not suitable for learning 3D structure from multiple images, or more generally, for solving spatial vision problems.

Parsimonious random vector functional link network for data streams

The majority of the existing work on random vector functional link networks (RVFLNs) is not scalable for data stream analytics because they work under a batch learning scenario and lack a self-organizing property. A novel RVLFN, namely the parsimonious random vector functional link network (pRVFLN), is proposed in this paper. pRVFLN adopts a fully flexible and adaptive working principle where its network structure can be configured from scratch and can be automatically generated, pruned and recalled from data streams. pRVFLN is capable of selecting and deselecting input attributes on the fly as well as capable of extracting important training samples for model updates. In addition, pRVFLN introduces a non-parametric type of hidden node which completely reflects the real data distribution and is not constrained by a specific shape of the cluster. All learning procedures of pRVFLN follow a strictly single-pass learning mode, which is applicable for online time-critical applications. The advantage of pRVFLN is verified through numerous simulations with real-world data streams. It was benchmarked against recently published algorithms where it demonstrated comparable and even higher predictive accuracies while imposing the lowest complexities.

Stacking PCANet +: An Overly Simplified ConvNets Baseline for Face Recognition

The principal component analysis network (PCANet) is asserted as a parsimonious stacking-based convolutional neural networks (CNNs) instance for generic object recognition including face. However, to be regarded a CNN resemblance, PCANet lacks a nonlinearity in between two successive convolutional layers. The multilayer PCANet (by neglecting the nonlinearity pre-requisite) is also deemed far-fetched for the network depth beyond two, due to feature dimensionality explosion. We thus devise a PCANet alternative, dubbed PCANet+ in this letter, to untangle these constraints. To be more precise, conforming to the CNN essentials, PCANet+ conveys a mean-pooling unit manipulating each feature map. On top of that, we streamline the PCANet topology to permit a deep construction with an expanded PCA filter ensemble. We scrutinize the PCANet+ performance using face recognition technology and other two faces in the wild datasets, namely, labeled faces in the wild and YouTube faces. The experimental results reveal that the PCANet+ descriptor prevails over its predecessor and other stacking-based descriptors in face identification and verification, serving a baseline for ConvNets.

Block-Diagonal Covariance Selection for High-Dimensional Gaussian Graphical Models

ABSTRACTGaussian graphical models are widely used to infer and visualize networks of dependencies between continuous variables. However, inferring the graph is difficult when the sample size is small compared to the number of variables. To reduce the number of parameters to estimate in the model, we propose a nonasymptotic model selection procedure supported by strong theoretical guarantees based on an oracle type inequality and a minimax lower bound. The covariance matrix of the model is approximated by a block-diagonal matrix. The structure of this matrix is detected by thresholding the sample covariance matrix, where the threshold is selected using the slope heuristic. Based on the block-diagonal structure of the covariance matrix, the estimation problem is divided into several independent problems: subsequently, the network of dependencies between variables is inferred using the graphical lasso algorithm in each block. The performance of the procedure is illustrated on simulated data. An application to a real gene expression dataset with a limited sample size is also presented: the dimension reduction allows attention to be objectively focused on interactions among smaller subsets of genes, leading to a more parsimonious and interpretable modular network. Supplementary materials for this article are available online.

A neural network model for familiarity and context learning during honeybee foraging flights.

How complex is the memory structure that honeybees use to navigate? Recently, an insect-inspired parsimonious spiking neural network model was proposed that enabled simulated ground-moving agents to follow learned routes. We adapted this model to flying insects and evaluate the route following performance in three different worlds with gradually decreasing object density. In addition, we propose an extension to the model to enable the model to associate sensory input with a behavioral context, such as foraging or homing. The spiking neural network model makes use of a sparse stimulus representation in the mushroom body and reward-based synaptic plasticity at its output synapses. In our experiments, simulated bees were able to navigate correctly even when panoramic cues were missing. The context extension we propose enabled agents to successfully discriminate partly overlapping routes. The structure of the visual environment, however, crucially determines the success rate. We find that the model fails more often in visually rich environments due to the overlap of features represented by the Kenyon cell layer. Reducing the landmark density improves the agents route following performance. In very sparse environments, we find that extended landmarks, such as roads or field edges, may help the agent stay on its route, but often act as strong distractors yielding poor route following performance. We conclude that the presented model is valid for simple route following tasks and may represent one component of insect navigation. Additional components might still be necessary for guidance and action selection while navigating along different memorized routes in complex natural environments.

Phylogeography, genetic variability and structure of Acanthamoeba metapopulations in Iran inferred by 18S ribosomal RNA sequences: A systematic review and meta-analysis.

To verify phylogeography and genetic structure of Acanthamoeba populations among the Iranian clinical isolates and natural/artificial environments distributed in various regions of the country. We searched electronic databases including Medline, PubMed, Science Direct, Scopus and Google Scholar from 2005 to 2016. To explore the genetic variability of Acanthamoeba sp, 205 sequences were retrieved from keratitis patients, immunosuppressed cases and environmental sources as of various geographies of Iran. T4 genotype was the predominant strain in Iran, and the rare genotypes belonged to T2, T3, T5 (Acanthamoeba lenticulata), T6, T9, T11, T13 and T15 (Acanthamoeba jacobsi). A total of 47 unique haplotypes of T4 were identified. A parsimonious network of the sequence haplotypes demonstrated star-like feature containing haplogroups IR6 (34.1%) and IR7 (31.2%) as the most common haplotypes. In accordance with the analysis of molecular variance, the high value of haplotype diversity (0.612-0.848) of Acanthamoeba T4 represented genetic variability within populations. Neutrality indices of the 18S ribosomal RNA demonstrated negative values in all populations which represented a considerable divergence from neutrality. The majority of genetic diversity belonged to the infected contact lens and dust samples in immunodeficiency and ophthalmology wards, which indicated potential routes for exposure to a pathogenic Acanthamoeba sp. in at-risk individuals. A pairwise fixation index (FST) was from low to high values (0.02433-0.41892). The statistically FST points out that T4 is genetically differentiated between north-west, north-south and central-south metapopulations, but not differentiated between west-central, west-south, central-south, and north-central isolates. An occurrence of IR6 and IR7 displays that possibly a gene flow of Acanthamoeba T4 occurred after the founder effect or bottleneck experience through ecological changes or host mobility. This is the first systematic review and meta-analysis providing new approaches into gene migration and transmission patterns of Acanthamoeba sp, and targeting at the high-risk individuals/sources among the various regions of Iran.

On the quirks of maximum parsimony and likelihood on phylogenetic networks

Maximum parsimony is one of the most frequently-discussed tree reconstruction methods in phylogenetic estimation. However, in recent years it has become more and more apparent that phylogenetic trees are often not sufficient to describe evolution accurately. For instance, processes like hybridization or lateral gene transfer that are commonplace in many groups of organisms and result in mosaic patterns of relationships cannot be represented by a single phylogenetic tree. This is why phylogenetic networks, which can display such events, are becoming of more and more interest in phylogenetic research. It is therefore necessary to extend concepts like maximum parsimony from phylogenetic trees to networks. Several suggestions for possible extensions can be found in recent literature, for instance the softwired and the hardwired parsimony concepts. In this paper, we analyze the so-called big parsimony problem under these two concepts, i.e. we investigate maximum parsimonious networks and analyze their properties. In particular, we show that finding a softwired maximum parsimony network is possible in polynomial time. We also show that the set of maximum parsimony networks for the hardwired definition always contains at least one phylogenetic tree. Lastly, we investigate some parallels of parsimony to different likelihood concepts on phylogenetic networks.

Network Receptive Field Modeling Reveals Extensive Integration and Multi-feature Selectivity in Auditory Cortical Neurons.

Cortical sensory neurons are commonly characterized using the receptive field, the linear dependence of their response on the stimulus. In primary auditory cortex neurons can be characterized by their spectrotemporal receptive fields, the spectral and temporal features of a sound that linearly drive a neuron. However, receptive fields do not capture the fact that the response of a cortical neuron results from the complex nonlinear network in which it is embedded. By fitting a nonlinear feedforward network model (a network receptive field) to cortical responses to natural sounds, we reveal that primary auditory cortical neurons are sensitive over a substantially larger spectrotemporal domain than is seen in their standard spectrotemporal receptive fields. Furthermore, the network receptive field, a parsimonious network consisting of 1–7 sub-receptive fields that interact nonlinearly, consistently better predicts neural responses to auditory stimuli than the standard receptive fields. The network receptive field reveals separate excitatory and inhibitory sub-fields with different nonlinear properties, and interaction of the sub-fields gives rise to important operations such as gain control and conjunctive feature detection. The conjunctive effects, where neurons respond only if several specific features are present together, enable increased selectivity for particular complex spectrotemporal structures, and may constitute an important stage in sound recognition. In conclusion, we demonstrate that fitting auditory cortical neural responses with feedforward network models expands on simple linear receptive field models in a manner that yields substantially improved predictive power and reveals key nonlinear aspects of cortical processing, while remaining easy to interpret in a physiological context.

Online gradient method with smoothing ℓ0 regularization for feedforward neural networks

ℓp regularization has been a popular pruning method for neural networks. The parameter p was usually set as 0<p≤2 in the literature, and practical training algorithms with ℓ0 regularization are lacking due to the NP-hard nature of the ℓ0 regularization problem; however, the ℓ0 regularization tends to produce the sparsest solution, corresponding to the most parsimonious network structure which is desirable in view of the generalization ability. To this end, this paper considers an online gradient training algorithm with smoothing ℓ0 regularization (OGTSL0) for feedforward neural networks, where the ℓ0 regularizer is approximated by a series of smoothing functions. The underlying principle for the sparsity of OGTSL0 is provided, and the convergence of the algorithm is also theoretically analyzed. Simulation examples support the theoretical analysis and illustrate the superiority of the proposed algorithm.

Gene flow for Echinococcus granulosus metapopulations determined by mitochondrial sequences: A reliable approach for reflecting epidemiological drift of parasite among neighboring countries

In genetic diversity and population structure of Echinococcus granulosus, the gene flow can illustrate how the Echinococcus isolates have epidemiologically drifted among endemic neighboring countries. 51 isolates of hydatid cysts were collected from human, dog, cattle and sheep in northwest Iran, where placed co-border with Turkey. DNA samples were extracted, amplified and subjected to sequence analysis of NADH dehydrogenase subunit 1 (nad1) and cytochrome oxidase subunit 1 (cox1) genes. As well, sequences of Echinococcus at east to the southeast regions of Turkey were retrieved from GenBank database for the cox1 gene. The confirmed isolates were grouped as G1 (n = 74) and G3 (n = 6) genotypes. 31 unique haplotypes were identified inferred by the analyzed sequences of cox1 among two distinct populations. A parsimonious network of the sequence haplotypes displayed star-like features in the overall population containing TUR1, IR15 and IR22 as the most common haplotypes. According to AMOVA test, the high value of haplotype diversity (0.94758–0.98901) of E. granulosus was reflected the total genetic variability within populations while nucleotide diversity was low (0.00727–0.01046) in Iranian and Turkish metapopulations. Neutrality indices of the cox1 were shown negative values (−15.078 to −10.057) in Echinococcus populations which indicating a significant divergence from neutrality. A pairwise fixation index (Fst) as a degree of gene flow was partially high value for all populations (0.151). The statistically Fst value indicates that E. granulosus sensu stricto (G1-G3) are genetically moderate differentiated among Iranian and Turkish isolates. The occurrence of TUR1 and IR15 elucidate that there is possibly the dawn of domestication due to transfer of alleles between populations through the diffusion of stock raising or anthropogenic movements. To evaluate the hypothetical evolutionary scenario, further exploration is necessitated to analyze isolates from various host species in rest Middle East countries.

Gene migration for re-emerging amebiasis in Iran's northwest-Iraq borders: a microevolutionary scale for reflecting epidemiological drift of Entamoeba histolytica metapopulations.

In the microevolutionary scales of Entamoeba isolates, the gene migration shows how Entamoeba spp. has epidemiologically drifted among border countries. Five hundred fecal samples were taken from patients suffering gastrointestinal disorders, abdominal pain, and diarrhea at Saggez, northwest Iran located within the border Iraq country. Following parasitological techniques, DNA samples were extracted and amplified by polymerase chain reaction (PCR) of 18S rRNA region to identify Entamoeba infections. To distinguish the Entamoeba spp., a multiplex PCR was conducted. Amplicons were sequenced to reconfirm their heterogeneity traits and phylogenetic analysis. Additionally, Entamoeba histolytica sequences of Iraq were retrieved from GenBank database. The suspected isolates were diagnosed as E. histolytica (2.2%), Entamoeba moshkovskii (1%), and Entamoeba dispar (0.4%). Mixed Entamoeba infections did not detect among isolates. A parsimonious network of the sequence haplotypes displayed star-like features in the overall isolates containing E.h1, E.d2, and E.m3 as the most common haplotypes. According to analysis of molecular variance (AMOVA) test, high partial value of haplotype diversity (0.700 to 0.800) of E. histolytica was shown the total genetic variability within populations while nucleotide diversity was low among Iranian and Iraqi metapopulations. Neutrality indices of the 18S rRNA were shown negative values in E. histolytica populations which indicating significant deviations from neutrality. A pairwise fixation index (F-statistics [Fst]) as a degree of gene flow had a low value for all populations (0.001) while the number of migrants was 2.48. The statistically Fst value indicates that E. histolytica isolates are not genetically differentiated among shared isolates of Iran and Iraq. Occurrence of E.h1 between two regional populations indicates that there is dawn of Entamoeba flow due to transfer of alleles from one population to another population through host mobility and ecological alterations. To evaluate the hypothetical evolutionary scenario, further study is required to analyze Entamoeba spp. in the neighboring Middle East countries.