Journal Of Machine Learning Research Research Articles

Signature-based validation of real-world economic scenarios

AbstractMotivated by insurance applications, we propose a new approach for the validation of real-world economic scenarios. This approach is based on the statistical test developed by Chevyrev and Oberhauser ((2022) Journal of Machine Learning Research, 23(176), 1–42.) and relies on the notions of signature and maximum mean distance. This test allows to check whether two samples of stochastic processes paths come from the same distribution. Our contribution is to apply this test to a variety of stochastic processes exhibiting different pathwise properties (Hölder regularity, autocorrelation, and regime switches) and which are relevant for the modelling of stock prices and stock volatility as well as of inflation in view of actuarial applications.

Exploring Current Trends in Artificial Intelligence Technology An Extensive Review

Artificial intelligence (AI) has become increasingly pervasive across various domains, including smartphones, social media platforms, search engines, and autonomous vehicles, among others. This study undertakes a scoping review of the current landscape of AI technologies, following the PRISMA framework, with the aim of identifying the most advanced technologies utilized in different domains of AI research. Three reputable journals within the artificial intelligence and machine learning domain, namely the Journal of Artificial Intelligence Research, the Journal of Machine Learning Research, and Machine Learning, were selected for this review. Articles published in 2022 were scrutinized against certain criteria: the technology must be tested against comparable solutions, employ commonly approved or well-justified datasets, and demonstrate improvements over comparable solutions. A crucial aspect of technology development identified in this review is the processing and exploitation of data collected from diverse sources. Given the highly unstructured nature of data, technological solutions should minimize the need for manual intervention by humans. The review indicates that creating labeled datasets is a labor-intensive process, leading to increased research focus on solutions leveraging unsupervised or semi-supervised learning technologies. Efficient updating of learning algorithms and the interpretability of predictions emerge as key considerations in the development of AI technologies. Moreover, in real-world applications, ensuring safety and providing explainable predictions are imperative before widespread adoption can be achieved. Thus, this review underscores the importance of addressing these factors to facilitate the responsible and effective integration of AI technologies into various domains.

A fast asynchronous Markov chain Monte Carlo sampler for sparse Bayesian inference

Abstract We propose a very fast approximate Markov chain Monte Carlo sampling framework that is applicable to a large class of sparse Bayesian inference problems. The computational cost per iteration in several regression models is of order O(n(s+J)), where n is the sample size, s is the underlying sparsity of the model, and J is the size of a randomly selected subset of regressors. This cost can be further reduced by data sub-sampling when stochastic gradient Langevin dynamics are employed. The algorithm is an extension of the asynchronous Gibbs sampler of Johnson et al. [(2013). Analyzing Hogwild parallel Gaussian Gibbs sampling. In Proceedings of the 26th International Conference on Neural Information Processing Systems (NIPS’13) (Vol. 2, pp. 2715–2723)], but can be viewed from a statistical perspective as a form of Bayesian iterated sure independent screening [Fan, J., Samworth, R., & Wu, Y. (2009). Ultrahigh dimensional feature selection: Beyond the linear model. Journal of Machine Learning Research, 10, 2013–2038]. We show that in high-dimensional linear regression problems, the Markov chain generated by the proposed algorithm admits an invariant distribution that recovers correctly the main signal with high probability under some statistical assumptions. Furthermore, we show that its mixing time is at most linear in the number of regressors. We illustrate the algorithm with several models.

Mini-batch stochastic subgradient for functional constrained optimization

In this paper, we consider finite sum composite optimization problems with many functional constraints. The objective function is expressed as a finite sum of two terms, one of which admits easy computation of (sub)gradients while the other is amenable to proximal evaluations. We assume a generalized bounded gradient condition on the objective which allows us to simultaneously tackle both smooth and nonsmooth problems. We also consider the cases of both with and without a quadratic functional growth property. Further, we assume that each constraint set is given as the level set of a convex but not necessarily a differentiable function. We reformulate the constrained finite sum problem into a stochastic optimization problem for which the stochastic subgradient projection method from Necoara and Singh [Stochastic subgradient projection methods for composite optimization with functional constraints; 2022 Journal of Machine Learning Research, 23, 1–35] specializes in a collection of mini-batch variants, with different mini-batch sizes for the objective function and functional constraints, respectively. More specifically, at each iteration, our algorithm takes a mini-batch stochastic proximal subgradient step aimed at minimizing the objective function and then a subsequent mini-batch subgradient projection step minimizing the feasibility violation. By specializing in different mini-batching strategies, we derive exact expressions for the stepsizes as a function of the mini-batch size and in some cases we also derive insightful stepsize-switching rules which describe when one should switch from a constant to a decreasing stepsize regime. We also prove sublinear convergence rates for the mini-batch subgradient projection algorithm which depend explicitly on the mini-batch sizes and on the properties of the objective function. Numerical results also show a better performance of our mini-batch scheme over its single-batch counterpart.

Thin-Film Batteries Failure Prediction from a Short Pulsed Test Using Machine Learning

We experimentally demonstrated for the first time in 2019 LiCoO2 all solid-state thin-film batteries with high electrochemical density storage of 0.89 mA.h.cm-2 [1]. Despite these excellent performances, some devices may be defective and fail during cycling. Therefore, in the context of industrial production of such devices, there is a need to reliably assess the functionality of the cells on the wafer scale. The most straightforward approach consists in simply cycling the batteries as well as performing complementary short tests such as Open-Circuit Voltage measurement (OCV) and Electrochemical Impedance Spectroscopy (EIS). However, with 14 000+ cells per wafer said approach is too time consuming. Moreover tested batteries would not be suitable for further commercialization as the electrodes have already been solicited, which could affect their shelf life.We therefore have investigated on the ability of a short – i.e. with a duration in the order of the minute – pulsed test to predict the failure of our dies, using machine learning. This Electrical Wafer Sorting (EWS) or screening protocol consists in inducing a partial charge for 1 min and then to observe the relaxation by measuring the OCV for 10 s. OCV and EIS are also measured before and after the pulse. Certain features are extracted from profiles collected during the EWS protocol (shown in Figure a)), such as the mean OCV or the real part of the impedance at 1 kHz. 1122 devices were screened using this protocol and then cycled for 1 cycle (1 full charge 1 full discharge). The batteries were then classified in binary categories, according to the outcome of the cycling test: fail or pass. Fail category encompasses all failure modes: (e.g. open-circuit, short-circuit). The features extracted during the EWS were used to build the prediction model, whose target is the cycling categorization. Foremost, the data was split in a training set and a test set. In a first step, variable selection was achieved using a simple decision tree model. It enabled us to choose the combination of parameters leading to a maximization of the f1-scores (from 68 to 73 %). However, the learning curves revealed that this simple model clearly overfitted and was unable to generalize. In a second step, a variety of models were trained using the selected features, including Random Forest, Support Vector Machines, and Logistic Regression. The three models leading to best f1-scores were kept and their hyperparameters have been further optimized following a grid search approach. The algorithm giving the highest scores is the Random Forest classifier. Although it was not possible to increase its accuracy (still at 73%) the optimization step helped to significantly reduce the overfitting, reinforcing the ability of the model to be equally accurate on new data. The corresponding confusion matrix and learning curves are displayed on Figure b). Figure c) shows respectively predicted and actual wafermaps for data present in the dataset used to build the model. An example of predicted wafermap on a complete wafer is presented on Figure d).Achieved performance provide an interesting base for investigations in a research context. It is shown that the functionality of the thin film batteries can be predicted with reasonable confidence. Nevertheless, far better scores (above 90%) are required for production application. Thus further work will focus on collecting new data to check on generalizability and if it helps to improve the model accuracy when used for training. All the work presented in this abstract has been carried out using scikit-learn Python package [2].[1] S. Oukassi et al., “Millimeter scale thin film batteries for integrated high energy density storage,” 2019, vol. 2019-December. doi: 10.1109/IEDM19573.2019.8993483.[2] F. Pedregosa et al., “Scikit-learn: Machine Learning in Python,” Journal of Machine Learning Research, vol. 12, no. 85, pp. 2825–2830, 2011. Figure 1

102 The deep phenotype characterization of ‘Off-the-Shelf’ CD19-chimeric antigen receptor (CAR) T cells allows to identify their subset complexity and to optimize their manufacturing

BackgroundUmbilical cord blood (UCB) represents a promising source of T cells for the generation of ‘off-the-shelf’ T cells engineered to express a chimeric antigen receptor (CAR). This study is aimed at understanding the composition of T cell subsets within UCB-CAR-T cells.MethodsT cells, either from UCB or peripheral mononuclear cells (PBMCs) of healthy donors, were activated in vitro with CD3/CD28 mAbs either conjugated to magnetic beads (Dynabeads) or to a colloidal polymeric nanomatrix (TransAct; Miltenyi Biotec). T cells were then transduced with lentiviral vectors encoding for CD19-CD28z or CD19-4-1BBz CARs. The deep phenotype analyses of the CD19-CAR-T cells (N=32) was performed through a multidimensional flow cytometry to assess the expression/co-expression of T cell-associated markers (N=29). The NGFR was utilized as probe for the expression of CD19-CAR. To select the pertinent markers characterising the different groups, we applied a machine learning technique called L0-regularized logistic regression,1 2 and implemented in the R packageL0Learn. 5-fold cross-validation (CV) was used to select the optimal values of the tuning parameters. CD19-CAR-T cells have been also characterized for the transcriptomic profile by parallel quantitative PCR using the high throughput BioMark HD platform and for cytokines, perforin and granzyme B release upon the co-culture with CD19 expressing or not target cells.ResultsT lymphocytes UCB showed efficient expression of the CARs (40–70% of positive cells). Different T cell subsets could discriminate the composition of T cells activated with either Beads or TranAct. CD4+NGFR+CD45RA+ or CD8+NGFR+CD45RA+ T cells associated with different combinations of CCR7, CD62L, LAG3, CD57, CD56 could discriminate between cells activated with Beads vs. TranAct (figures 2–3). CD8+NGFR+CD45RO+CD279−CD152+ T cells were also differentially expressed in TranAct vs. Beads. The PCA analyses also highlighted differences in terms of CD19-CAR-T cell subsets (such as CD8+NGFR+CD45RO+CD62L+, CD8+NGFR+CD45RO+CCR7+, CD8+NGFR+CD45RO+CD272+TIM−3+, CD8+NGFR+CD45RO+CD272+TIM−3+, CD8+NGFR+CD45RA+CD272+TIM−3− and CD4+NGFR+CD45RA+CD272−TIM−3+) in PBMCs vs. UCBs (figure 1). In addition, bystander T cells with different phenotype not expressing the CARs were also detected within the populations of T cells with different origins. Similarly, different T subsets were found in relationship with the sources of T cells. These CD19-CAR-T cells were also characterized for the anti-tumor activity and transcriptomic profiling.Abstract 102 Figure 1PCA of CAR-T cells from UCB vs. PBMCsAbstract 102 Figure 2PCA of CAR-T cells from UCB to compare TransAct vs. beadsAbstract 102 Figure 3PCA of CD19-CAR-T cells to compare TransAct vs. Beads irrespective of the source of the T cellsConclusionsThe combination of deep phenotype characterization with novel statistical tools allowed to identify the complexity of subsets in the engineered T cells in relationship with the starting material and the methods for the activation of the lymphocytes. These findings have important implications for the optimization of the manufacturing of CD19-CAR-T cells.ReferencesAntoine Dedieu, Hussein Hazimeh, and Rahul Mazumder. Learningsparse classifiers: Continuous and mixed integer optimization perspectives. Journal of Machine Learning Research 2021.Hussein Hazimeh and Rahul Mazumder. Fast best subset selection: Coordinatedescent and local combinatorial optimization algorithms. Operations Research 2020;68(5):1517–1537.Ethics ApprovalSidra Medicine’s Ethics Board approval, #1812044429

A visualized bibliometric analysis of mapping research trends of machine learning in engineering (MLE)

In this work, we conducted a visualized bibliometric analysis to map the research trends of machine learning in engineering (MLE) based on articles indexed in the Web of Science Core Collection published between 2000 and 2019. The research distributions, knowledge bases, research hotspots, and research frontiers for MLE studies are revealed by using VOSviewer software and visualization technology. The growth of the literature related to MLE averaged 24.3% in the past two decades. A total of 3057 peer-reviewed papers from 96 countries published in 1299 different journals were identified. The USA was the most productive country, with 23.73% of the overall articles and 32.25% of the overall citations. The most active research organization was MIT, with 41 publications and 1079 citations, and the Journal of Machine Learning Research had the largest number of citations in the field of MLE. In particular, our findings indicate that the research issues of “random forests”, “support vector machine”, “extreme learning machine”, “deep learning”, “statistical learning theory”, and “Python machine learning” formed the knowledge bases of MLE from 2000 to 2019, while the research hotspots focused on applications of machine learning benchmark algorithms. Burst detection analysis results showed that more burst keywords emerged and had a higher frequency of change after 2010. This study provides an insight view of the overall research trends of MLE and may help researchers better understand this research field and predict its dynamic directions.

Error Prediction of Air Quality at Monitoring Stations Using Random Forest in a Total Error Framework.

Instead of a flag valid/non-valid usually proposed in the quality control (QC) processes of air quality (AQ), we proposed a method that predicts the p-value of each observation as a value between 0 and 1. We based our error predictions on three approaches: the one proposed by the Working Group on Guidance for the Demonstration of Equivalence (European Commission (2010)), the one proposed by Wager (Journal of , 15, 1625–1651 (2014)) and the one proposed by Lu (Journal of , 22, 1–41 (2021)). Total Error framework enables to differentiate the different errors: input, output, structural modeling and remnant. We thus theoretically described a one-site AQ prediction based on a multi-site network using Random Forest for regression in a Total Error framework. We demonstrated the methodology with a dataset of hourly nitrogen dioxide measured by a network of monitoring stations located in Oslo, Norway and implemented the error predictions for the three approaches. The results indicate that a simple one-site AQ prediction based on a multi-site network using Random Forest for regression provides moderate metrics for fixed stations. According to the diagnostic based on predictive qq-plot and among the three approaches used in this study, the approach proposed by Lu provides better error predictions. Furthermore, ensuring a high precision of the error prediction requires efforts on getting accurate input, output and prediction model and limiting our lack of knowledge about the “true” AQ phenomena. We put effort in quantifying each type of error involved in the error prediction to assess the error prediction model and further improving it in terms of performance and precision.

Optimality of singular vector perturbation under maximum norm

Fan, Wang, and Zhong estimate the difference between the singular vectors of a matrix and those of a perturbed matrix in terms of the maximum norm. Their estimations are used effectively to establish the asymptotic properties of robust covariance estimators (see Journal of Machine Learning Research, 2018;18:1‐42). In this paper, we give the corresponding lower bound estimates, which show Fan‐Wang‐Zhong's estimations optimal.

A model-free soft classification with a functional predictor

Class probability is a fundamental target in classification that contains complete classification information. In this article, we propose a class probability estimation method when the predictor is functional. Motivated by Wang et al. (Biometrika, 95, 149-167, 2007), our estimator is obtained by training a sequence of functional weighted support vector machines (FWSVM) with different weights, which can be justified by the Fisher consistency of the hinge loss. The proposed method can be extended to multiclass classification via pairwise coupling proposed by Wu et al. (Journal of Machine Learning Research, 5, 975-1005, 2004). The use of FWSVM makes our method model-free as well as computationally efficient due to the piecewise linearity of the FWSVM solutions as functions of the weight. Numerical investigation to both synthetic and real data show the advantageous performance of the proposed method.

Creating and Testing Specialized Dictionaries for Text Analysis

Practitioners in many domains–e.g., clinical psychologists, college instructors, researchers–collect written responses from clients. A well-developed method that has been applied to texts from sources like these is the computer application Linguistic Inquiry and Word Count (LIWC). LIWC uses the words in texts as cues to a person’s thought processes, emotional states, intentions, and motivations. In the present study, we adopt analytic principles from LIWC and develop and test an alternative method of text analysis using naïve Bayes methods. We further show how output from the naïve Bayes analysis can be used for mark up of student work in order to provide immediate, constructive feedback to students and instructors.
 References
 
 Blei, D. M., Ng, A. Y., & Jordan, M. I. (2003). Latent Dirichlet Allocation. Journal of Machine Learning Research 3, 993-1022.
 Boot, P., Zijlstra, H., & Geenen, R. (2017). The Dutch translation of the Linguistic Inquiry and Word Count (LIWC) 2007 dictionary. Dutch Journal of Applied Linguistics, 6(1), 65-76.
 Chung, C. K., & Pennebaker, J. W. (2008). Revealing dimensions of thinking in open-ended self-descriptions: An automated meaning extraction method for natural language. Journal of research in personality, 42(1), 96-132.
 Hsieh, H-F., & Shannon, S. E. (2005).Three approaches to qualitative content analysis. Qualitative health research, 15(9), 277-1288.
 Kintsch, W. (1998). Comprehension: A paradigm for cognition. New York: Cambridge University Press.
 Landauer, T. K., Foltz, P. W., & Laham, D. (1998). An introduction to latent semantic ana­lysis. Discourse processes, 25(2-3), 259-284.
 Lund, K., & Burgess, C. (1996). Producing high-dimensional semantic spaces from lexical co-occurrence. Behavior Research Methods, Instruments, & Computers, 28(2), 203-208.
 Massó, G., Lambert, P., Penagos, C. R., & Saurí, R. (2013, December). Generating New LIWC Dictionaries by Triangulation. In Asia Information Retrieval Symposium (pp. 263-271). Springer, Berlin, Heidelberg.
 Newman, M., Groom, C.J., Handelman, L.D., & Pennebaker, J.W. (2008). Gender differences in language use: An analysis of 14,000 text samples. Discourse Processes, 45(3), 211-236.
 Pennebaker, J.W., Boyd, R.L., Jordan, K., & Blackburn, K. (2015). The development and psychometric properties of LIWC 2015. Austin, TX: University of Texas at Austin.
 Tausczik, Y. R., & Pennebaker, J. W. (2010). The psychological meaning of words: LIWC and computerized text analysis methods. Journal of language and social psychology, 29(1), 24-54.
 Van Wissen, L., & Boot, P. (2017, September). An Electronic Translation of the LIWC Dictionary into Dutch. In: Electronic lexicography in the 21st century: Proceedings of eLex 2017 Conference. (pp. 703-715). Lexical Computing.

Machine Learning Research in India: A Scientometric Assessment of Papers During 2006–17

The article provides a quantitative and qualitative description of machine learning research in India using measures such as growth rate, global publication share, national-level share of international collaborative papers, citations per paper, and highly cited papers. The data for the study was sourced from Scopus database covering 12-year period 2006–17. The study finds that the United States is the world leader in the machine learning research, followed by China, United Kingdom, Germany, and India. India registered 52.46% annual growth in the subject, more than twice that of the world (22.03%). India's citation impact was 5.34 citations per paper and its national share of highly cited papers was 0.68% (27 papers). The leading Indian research organizations in machine learning research include Vellore Institute of Technology, Anna University, Chennai, Indian Institute of Technology, New Delhi, Indian Institute of Science (IIS), Bangalore, and so on. The top 15 journals that accounted for 31.46% of India's output include International Journal of Machine Learning and Cybernetics (with 82 papers), International Journal of Applied Engineering Research (77 papers), Journal of Machine Learning Research (46 papers), Neurocomputing (44 papers), and so on.

Finding a Common Ground in Human and Machine-Based Text Processing

Language makes human communication possible. Apart from everyday applications, language can provide insights into individuals’ thinking and reasoning. Machine-based analyses of text are becoming widespread in business applications, but their utility in learning contexts are a neglected area of research. Therefore, the goal of the present work is to explore machine-assisted approaches to aid in the analysis of students’ written compositions. A method for extracting common topics from written text is applied to 78 student papers on technology and ethics. The primary tool for analysis is the Latent Dirichlet Allocation algorithm. The results suggest that this machine-based topic extraction method is effective and supports a promising prospect for enhancing classroom learning and instruction. The method may also prove beneficial in other applied applications, like those in clinical and counseling practice.
 References
 
 Blei, D. M., Ng, A. Y., & Jordan, M. I. (2003). Latent Dirichlet Allocation. Journal of Machine Learning Research 3, 993-1022.
 Bruner, J. (1990). Acts of meaning. Cambridge, MA: Harvard University Press.
 Chen, K. Y. M., & Wang, Y. (2007). Latent dirichlet allocation. http://acsweb.ucsd.edu/~yuw176/ report/lda.pdf.
 Chung, C. K., & Pennebaker, J. W. (2008). Revealing dimensions of thinking in open-ended self-descriptions: An automated meaning extraction method for natural language. Journal of research in personality, 42(1), 96-132.
 Feldman, S. (1999). NLP meets the Jabberwocky: Natural language processing in information retrieval. Online Magazine, 23, 62-73. Retrieved from: http://www.onlinemag.net/OL1999/ feldmann5.html
 Mishlove, J. (2010). https://www.youtube.com/watch?v=0XTDLq34M18 (Accessed June 12, 2018).
 Ostrowski, D. A. (2015). Using latent dirichlet allocation for topic modelling in twitter. In Semantic Computing (ICSC), 2015 IEEE International Conference (pp. 493-497). IEEE.
 Pennebaker, J. W. (2004). Theories, therapies, and taxpayers: On the complexities of the expressive writing paradigm. Clinical Psychology: Science and Practice, 11(2), 138-142.
 Pennebaker, J.W., Boyd, R.L., Jordan, K., & Blackburn, K. (2015). The development and psychometric properties of LIWC 2015. Austin, TX: University of Texas at Austin.
 Pennebaker, J. W., Chung, C. K., Frazee, J., Lavergne, G. M., & Beaver, D. I. (2014). When small words foretell academic success: The case of college admissions essays. PLoS ONE, 9(12), e115844.
 Pennebaker, J. W., & King, L. A. (1999). Linguistic styles: Language use as an individual difference. Journal of Personality and Social Psychology, 77(6), 1296-1312.
 Recchia, G., Sahlgren, M., Kanerva, P., & Jones, M. N. (2015). Encoding sequential information in semantic space models: Comparing holographic reduced representation and random permutation. Computational intelligence and neuroscience, 2015, 1-18.
 Salzmann, Z. (2004). Language, Culture, and Society: An Introduction to Linguistic Anthropology (3rd ed). Westview Press.
 Schank, R. C., Goldman, N. M., Rieger III, C. J., & Riesbeck, C. (1973). MARGIE: Memory analysis response generation, and inference on English. In IJCAI, 3, 255-261.
 Taraban, R., Marcy, W. M., LaCour Jr., M. S., & Burgess II, R. A. (2017). Developing machine-assisted analysis of engineering students’ ethics course assignments. Proceedings of the American Society of Engineering Education (ASEE) Annual Conference, Columbus, OH. https://www.asee.org/public/conferences/78/papers/19234/view.
 Taraban, R., Marcy, W. M., LaCour, M. S., Pashley, D., & Keim, K. (2018). Do engineering students learn ethics from an ethics course? Proceedings of the American Society of Engineering Education – Gulf Southwest (ASEE-GSW) Annual Conference, Austin, TX. http://www.aseegsw18.com/papers.html.
 Taraban, R., & Marshall, P. H. (2017). Deep learning and competition in psycholinguistic research. East European Journal of Psycholinguistics, 4(2), 67-74.
 Weizenbaum, J. (1966). ELIZA—a computer program for the study of natural language communication between man and machine. Communications of the ACM, 9(1), 36-45.
 Winograd, T. (1972). Understanding natural language. New York: Academic Press.

Call for Papers: Special Issue on Profit-Driven Analytics.

Call for Papers: Special Issue on Profit-Driven Analytics.

Multi-modal parcellation of the frontal lobe

[1] Skudlarski, P. (2008). Neuroimage 43, 554-561. [2] Uddin, L.Q. (2013). Trends in Cognitive Sciences 17, 600-602. [3] Nooner, K.B. (2012). Frontiers in Neuroscience 6. [4] Yarkoni, K. (2011). Frontiers in Neuroinformatics. doi: 10.3389/conf. fninf. [5] Gorgolewski, K. (2011). Frontiers in Neuroinformatics 5. [6] Kelly, C. (2012). Neuroimage 61, 1129/1142. [7] Moreno-Dominguez, D. (2014). Human Brain Mapping. doi: 10.1002/hbm.22528. [8] Toro, R. (2008). Cerebral Cortex 18, 2553/2559. [9] Pedregosa, F. (2011). The Journal of Machine Learning Research 12, 2825-2830. Clustered cortical parcellations offer a medium for comparing the different methods for characterizing the connectome. By integrating neuroimaging software via Nipype, we have written a modular pipeline used to cluster subregions of the frontal lobe. The pipeline allows us to explore various combinations of analytic and preprocessing strategies, and offers the possibility of flexible investigation of cortical parcellation across modalities and datasets. 2 Institut Pasteur, Paris, France

Advanced Machine Learning for classification of EEG traits as Parkinson’s biomarker

We aim to develop non-­‐invasive, low-­‐cost preclinical markers for synucleinopathies (Parkinson’s Disease -­‐ PD or Dementia with Lewy Bodies -­‐ DLB) with impact on neuroprotection. In a first stage we have applied Machine Learning techniques to the analysis of the spontaneous, waking EEG of REM Sleep Behavior Disorder (RBD) patients. We know that patients with RBD may evolve towards PD and other synucleinopathies (i.e. DLB) providing a useful guide in the search for biomarkers. EEG has proven to be sensitive to brain alterations related to both RBD and PD patients, and it is known that patients with either RBD or de novo PD without obvious cognitive alterations show similar EEG/MEG alterations (mainly slowing of the EEG) while awake. In the work described herein we achieve the classification of patients according to their ultimate diagnosis, differentiating among different patient groups. We have analyzed EEG data from a study of RBD patients at the Center for Advanced Research in Sleep Medicine of Montreal. The EEG was recorded from 8 patients who years later evolved to PD, 8 patients who later evolved to LBD, 10 patients with RBD who did not convert, and 17 healthy controls. It is worth mentioning that 80% of the PD and DLB patients developed disease at a follow-­‐up of 8 years. Support Vector Machines were applied to different feature types. Absolute Band Power features extracted each 4 seconds outperform the other ones. After non-­‐parametric feature selection of the 5 most discriminative channel-­‐band combinations, classification was applied. A procedure for performance estimation close to operational conditions, which is denoted as leave-­‐pair-­‐subjects-­‐ out, has been employed for evaluation of the implemented system. Excellent performance levels, AUC 93.75-­‐99%, were obtained for all realized group comparisons. In conclusion, we believe that these results support the idea that classification of EEG shows great potential as a preclinical biomarker. Our results confirm the existent literature, where EEG slowing characterizes RBD and PD/DLB patients. Furthermore we have attained this characterization at the individual subject level for the first time by employing machine learning. Given that EEG was acquired 8 years before disease development in a majority of cases, we believe that the utilization of the developed system for conversion prognosis of synucleinopathies (PD or DLB) will have real impact on both early treatment and drug development.

Unsupervised feature selection with ensemble learning

In this paper, we show that the way internal estimates are used to measure variable importance in Random Forests are also applicable to feature selection in unsupervised learning. We propose a new method called Random Cluster Ensemble (RCE for short), that estimates the out-of-bag feature importance from an ensemble of partitions. Each partition is constructed using a different bootstrap sample and a random subset of the features. We provide empirical results on nineteen benchmark data sets indicating that RCE, boosted with a recursive feature elimination scheme (RFE) (Guyon and Elisseeff, Journal of Machine Learning Research, 3:1157---1182, 2003), can lead to significant improvement in terms of clustering accuracy, over several state-of-the-art supervised and unsupervised algorithms, with a very limited subset of features. The method shows promise to deal with very large domains. All results, datasets and algorithms are available on line (http://perso.univ-lyon1.fr/haytham.elghazel/RCE.zip).

Comment on "Robustness and regularization of support vector machines" by H. Xu et al. (Journal of machine learning research, vol. 10, pp. 1485-1510, 2009)

This paper comments on the published work dealing with robustness and regularization of support vector machines (Journal of Machine Learning Research, Vol. 10, pp. 1485-1510, 2009) by H. Xu et al. ...

Evidence of intertextuality: investigating Paul the Deacon's Angustae Vitae

1. J. Kristeva, “Word, Dialog and Novel,” in Toril Moi, ed., The Kristeva Reader, New York: Columbia University Press, 1986, p. 34-61. 2. G. Conte, Latin Literature: A History, Transl. by J. Solodow, rev. by D. Fowler and G. Most, Johns Hopkins University Press, 1994. 3. C. Forstall and W. Scheirer, “Features from Frequency: Authorship and Stylistic Analysis Using Repetitive Sound,” Chicago Colloquium on Digital Humanities and Computer Science, 2009. 4. L. Manevit and M. Yousef, “One-Class SVMs for Document Classification,” Journal of Machine Learning Research (2), pp. 139-154, 2001. 5. M. Platnauer, Latin Elegiac Verse: A Study of the Metrical Usages of Tibullus, Propertius & Ovid. Cambridge University Press, 1951. See for example pp. 36-39. 6. E. Dummler, ed., Poetae Latini Aevi Carolini: Tomus. I. Berlin 1881, pp. 35-86. 7. K. Harrington, J. Pucci, and A.G. Elliott, Medieval Latin (2nd ed.), University of Chicago Press, 1997. 8. N. Coffee, J. Koenig, S. Poornima, and C. Forstall, The Tesserae Project, http:// tesserae.caset.buffalo.edu The study of intertextuality, the shaping of a text’s meaning by other texts, remains a laborious process for the literary critic. Kristeva1 suggests that “Any text is constructed as a mosaic of quotations; any text is the absorption and transformation of another.” The nature of these mosaics is widely varied, from direct quotations representing a simple and overt intertextuality, to more complex transformations that are intentionally or subconsciously absorbed into a text. Since, in many cases, the problem is one of pattern recognition, it is a good candidate for automated assistance by computers. As a case study for our computational analysis of intertextuality, we turn to Paul the Deacon’s 8th century poem Angustae Vitae, which we suggest has a strong connection to the poetry of Catullus.

Copula techniques in wireless communications

Copula techniques were originally developed as a method for modelling data dependence in financial applications and are proving useful in many other fields. We show how the Copula concept may be exploited to model dependence in wireless communications problems. In particular we consider multipath correlation, with signal fading, in a wireless propagation medium. The Copula approach is also considered for the purpose of separating signals that have become dependent in such propagation scenarios and we investigate methods for Blind Source Separation that provide alternatives to the popular Independent Component Analysis approach. Our approach to the Blind Source Separation problem forms an objective function based on the copula parameter of the dependence structure, then a transformation is sought which inverts the function producing the dependence and which yields an independent copula. This approach has the potential to provide a robust and easily applied technique for isolating wireless communications signals in a wide range of propagation scenarios. References M.-S. Alouini, A. Abdi, and M. Kaveh. Sum of gamma variates and performance of wireless communication systems over Nakagami-fading channels. Vehicular Technology, IEEE Transactions on, 50(6):1471--1480, November 2001. doi:10.1109/25.966578 N. C. Beaulieu and C. Cheng. an efficient procedure for Nakagami-m fading simulation. In Global Telecommunications Conference, 2001. GLOBECOM '01. IEEE, volume 6, pages 3336--3342, November 2001. doi:10.1109/GLOCOM.2001.966304 Ray-Bing Chen, Meihui Guo, Wolfgang Hardle, and Shih-Feng Huang. Independent component analysis via copula techniques. SFB 649 Discussion Papers SFB649DP2008-004, Sonderforschungsbereich 649, Humboldt University, Berlin, Germany, January 2008. http://ideas.repec.org/p/hum/wpaper/sfb649dp2008-004.html David Christensen. Fast algorithms for the calculation of Kendall's $\tau $. Computational Statistics, 20(1):51--62, March 2005. doi:10.1007/BF02736122 P. Dita. Factorization of unitary matrices. Journal of Physics A: Mathematical and General, 36(11):2781--2789, March 2003. doi:10.1088/0305-4470/36/11/309 A. Hyvarinen. Fast and robust fixed-point algorithms for independent component analysis. Neural Networks, IEEE Transactions on, 10(3):626--634, May 1999. doi:10.1109/72.761722 Jian Ma and Zengqi Sun. Copula component analysis. CoRR, 2007. http://arxiv.org/abs/cs/0703095 Erik G. Miller and John W. Fisher III. ICA using spacings estimates of entropy. Journal of Machine Learning Research, 4:1271--1295, December 2003. http://jmlr.csail.mit.edu/papers/v4/learned-miller03a.html R. B. Nelsen. An introduction to copulas. Springer-Verlag New York, Inc., 1999. doi:10.1007/0-387-28678-0 P. Tichavsky, Z. Koldovsky, and E. Oja. Performance analysis of the FastICA algorithm and Cramer--Rao bounds for linear independent component analysis. IEEE Trans. on Signal Processing, April 2006. doi:10.1109/TSP.2006.870561 M. D. Yacoub, G. Fraidenraich, and J. C. S. Santos Filho. Nakagami-m phase-envelope joint distribution. Electronics Letters, 41(5):259--261, March 2005. doi:10.1049/el:20057014 Q. T. Zhang. A decomposition technique for efficient generation of correlated nakagami fading channels. Selected Areas in Communications, IEEE Journal on, 18(11):2385--2392, November 2000. doi:10.1109/49.895043