Challenges In Machine Learning Research Articles

Micro-Knowledge Embedding for Zero-shot Classification

Zero-shot learning is one of the most challenging machine learning tasks, in which learning stable and transferable knowledge from seen classes plays a pivotal role. To improve the currently unsatisfactory performance of zero-shot object recognition, this paper proposes a novel image representation method, namely, micro-knowledge. In our method, the segmentation of micro-regions and the consequent learning of micro-knowledge are unified by the introduction of a self-attention mechanism. A zero-shot classification framework is carefully designed based on micro-knowledge of images. Under this framework, multiple micro-region descriptions are first obtained by embedding micro-knowledge and then merged to carry out the final classification of unseen objects. Finally, a capsule-unified framework is employed as a graphical programming tool to accomplish the aforementioned tasks. Experiments on public datasets show that the proposed framework can generally achieve competitive results for the classification of unseen objects. Specifically, these results verify that the micro-knowledge learned from one dataset can be directly applied to others without complicated adjustments and demonstrate that using visual features instead of semantic features can result in a decrease in classification error. This research will bring new ideas into the field of zero-shot learning and will serve as an appealing option when addressing the problem of domain shift.

Mathematical and Circuit Level Analysis Interpretation and Recommendations on Neuron Models

Contrary to von Neumann computer architecture, neuromorphic computing is a biologically inspired method for building several sorts of brain-inspired computers. This computer technology uses silicon neurons and synapses to solve difficult machine learning and AI challenges. The goal of neuromorphic computing is to build a brain-like ability to compute, learn, and adapt. Building an appropriate neuroscience model, establishing a new architecture, modeling new devices, finding new materials for the devices, programming framework, and applications for these neuromorphic devices are major technological challenges. This study covers numerous neuron models from Hodgkin–Huxley to I&F, mathematical equations, circuit level analysis and motivations for neuromorphic computing. Here some of the most useful silicon neuron models are discussed in terms of biological plausibility, computational efficiency, and temperature dependency. This survey shows that more than 52 transistors make up the silicon HH neuron, which occupies less than 0.01[Formula: see text][Formula: see text] and consumes 60[Formula: see text]uW of power. On the other hand, an I&F neuron needs less than 20 transistors, occupies 442[Formula: see text][Formula: see text] and consumes 40[Formula: see text]pW of power.

Gradient Boosting Hybridized with Exponential Natural Evolution Strategies for Estimating the Strength of Geopolymer Self-Compacting Concrete

The current global demand to minimize carbon dioxide (CO2$) emissions from Portland cement manufacturing processes has led to the use of environmentally friendly additives in cement products. The so-called green cementitious composites have become increasingly essential in the design of cementitious composite mixtures, providing the environmental compatibility of concrete as a building material. Engineers face a difficult problem in predicting the mechanical properties of green composites due to their changing nature under various circumstances. Machine learning models then emerge as surrogate models to perform this difficult task. The very design of such models has become a challenge for machine learning. This study presents a gradient boosting algorithm hybridized with Natural Exponential Evolution Strategies inspired by nature to predict the mechanical properties of geopolymeric self-compacting concrete. The hybrid model is used to evolve the parameters, automating the selection of the best set of internal parameters to estimate the strength properties of geopolymer self-compacting concrete. Results show the predictive ability superiority of machine learning models and optimization algorithms hybridization compared to manually tuned models. In addition, this approach can minimize laboratory work, potentially optimize experimental design, and reduce sample production time and associated activity burden

An improved and random synthetic minority oversampling technique for imbalanced data

Imbalanced data learning has become a major challenge in data mining and machine learning. Oversampling is an effective way to re-achieve the balance by generating new samples. However, most oversampling methods cannot perform well in the presence of noises and complicated distribution structures, very easy to generate redundant/unsafe/outlier samples. To handle this problem, we endeavor to propose a novel oversampling method, namely Improved and Random Synthetic Minority Oversampling Technique (IR-SMOTE). The core idea of IR-SMOTE is three-fold: (1) by applying an ascending operation to sort the majority class samples, noise samples in each cluster of minority class after k-means clustering are successfully removed; (2) the number of synthetic samples is adaptively assigned to each cluster in minority class by means of the kernel density estimation technique; and (3) based on the obtained attributes of the temporary synthetic samples in terms of random-SMOTE, a new synthesizing method is developed to generate new samples with a guaranteed diversity. Finally, many comparison experiments have been carried out on 18 well-known data sets, which illustrate the effectiveness and universal applicability of the proposed IR-SMOTE method for imbalanced data classification.

ESPA+: Scalable Entropy-Optimal Machine Learning Classification for Small Data Problems

Classification problems in the small data regime (with small data statistic T and relatively large feature space dimension D) impose challenges for the common machine learning (ML) and deep learning (DL) tools. The standard learning methods from these areas tend to show a lack of robustness when applied to data sets with significantly fewer data points than dimensions and quickly reach the overfitting bound, thus leading to poor performance beyond the training set. To tackle this issue, we propose eSPA+, a significant extension of the recently formulated entropy-optimal scalable probabilistic approximation algorithm (eSPA). Specifically, we propose to change the order of the optimization steps and replace the most computationally expensive subproblem of eSPA with its closed-form solution. We prove that with these two enhancements, eSPA+ moves from the polynomial to the linear class of complexity scaling algorithms. On several small data learning benchmarks, we show that the eSPA+ algorithm achieves a many-fold speed-up with respect to eSPA and even better performance results when compared to a wide array of ML and DL tools. In particular, we benchmark eSPA+ against the standard eSPA and the main classes of common learning algorithms in the small data regime: various forms of support vector machines, random forests, and long short-term memory algorithms. In all the considered applications, the common learning methods and eSPA are markedly outperformed by eSPA+, which achieves significantly higher prediction accuracy with an orders-of-magnitude lower computational cost.

Ten years of image analysis and machine learning competitions in dementia

Machine learning methods exploiting multi-parametric biomarkers, especially based on neuroimaging, have huge potential to improve early diagnosis of dementia and to predict which individuals are at-risk of developing dementia. To benchmark algorithms in the field of machine learning and neuroimaging in dementia and assess their potential for use in clinical practice and clinical trials, seven grand challenges have been organized in the last decade: MIRIAD (2012), Alzheimer’s Disease Big Data DREAM (2014), CADDementia (2014), Machine Learning Challenge (2014), MCI Neuroimaging (2017), TADPOLE (2017), and the Predictive Analytics Competition (2019). Based on two challenge evaluation frameworks, we analyzed how these grand challenges are complementing each other regarding research questions, datasets, validation approaches, results and impact.The seven grand challenges addressed questions related to screening, clinical status estimation, prediction and monitoring in (pre-clinical) dementia. There was little overlap in clinical questions, tasks and performance metrics. Whereas this aids providing insight on a broad range of questions, it also limits the validation of results across challenges. The validation process itself was mostly comparable between challenges, using similar methods for ensuring objective comparison, uncertainty estimation and statistical testing. In general, winning algorithms performed rigorous data pre-processing and combined a wide range of input features.Despite high state-of-the-art performances, most of the methods evaluated by the challenges are not clinically used. To increase impact, future challenges could pay more attention to statistical analysis of which factors (i.e., features, models) relate to higher performance, to clinical questions beyond Alzheimer’s disease, and to using testing data beyond the Alzheimer’s Disease Neuroimaging Initiative. Grand challenges would be an ideal venue for assessing the generalizability of algorithm performance to unseen data of other cohorts. Key for increasing impact in this way are larger testing data sizes, which could be reached by sharing algorithms rather than data to exploit data that cannot be shared. Given the potential and lessons learned in the past ten years, we are excited by the prospects of grand challenges in machine learning and neuroimaging for the next ten years and beyond.

Analysis of High Efficient Perovskite Solar Cells Using Machine Learning

Affordable manufacturing along with high efficiency perovskite solar cell in photovoltaic technology has everyone's attention. Perovskite, which is in the lead role in solar cells, is full of characteristics such as high absorption coefficient, low exciton binding energy, charge carrier capable of having better mobility as well as more diffusion length and availability in suitable energy band. The application of machine learning technology is proving to be a boon to ensure optimum implementation with different properties in photovoltaic device, design, simple construction process and low-cost price. Machine learning is a branch of artificial intelligence which includes large data aggregation, precise structure property installation, demonstration and final model after model validation. The most of the source of database is the simulation and experimental results, calculations and related literature surveys which have a comprehensive compilation of the performance of hybrid perovskite device, collection of structures and properties of elements. Structure-property relationship installation comes under feature engineering which establishes a clear relationship between structure and the properties. In other demonstration process, proper algorithms are selected, data is generated and tested as well as pure estimated values are taken. This article contains a detailed discussion on the involvement of machine learning technology to build high-performance Perovskite solar cells. Proper selection as well as designing of active perovskite absorbent layer by machine learning successfully establishes results by including other parts such as non-toxic (lead free) and stability. Mature machine learning technology becomes a very essential method in determining the solvent combination of hybrid perovskite and in estimating design of the entire solar cell to ensure optimum implementation in the sector of perovskite solar technology. Finally, a phased concept has been briefly discussed to meet the challenges of machine learning and potential future compatibilities related to the prevalence.

Applications of Machine Learning in Alloy Catalysts: Rational Selection and Future Development of Descriptors.

At present, alloys have broad application prospects in heterogeneous catalysis, due to their various catalytic active sites produced by their vast element combinations and complex geometric structures. However, it is the diverse variables of alloys that lead to the difficulty in understanding the structure‐property relationship for conventional experimental and theoretical methods. Fortunately, machine learning methods are helpful to address the issue. Machine learning can not only deal with a large number of data rapidly, but also help establish the physical picture of reactions in multidimensional heterogeneous catalysis. The key challenge in machine learning is the exploration of suitable general descriptors to accurately describe various types of alloy catalysts, which help reasonably design catalysts and efficiently screen candidates. In this review, several kinds of machine learning methods commonly used in the design of alloy catalysts is introduced, and the applications of various reactivity descriptors corresponding to different alloy systems is summarized. Importantly, this work clarifies the existing understanding of physical picture of heterogeneous catalysis, and emphasize the significance of rational selection of universal descriptors. Finally, the development of heterogeneous catalytic descriptors for machine learning are presented.

Conceptual challenges for interpretable machine learning

As machine learning has gradually entered into ever more sectors of public and private life, there has been a growing demand for algorithmic explainability. How can we make the predictions of complex statistical models more intelligible to end users? A subdiscipline of computer science known as interpretable machine learning (IML) has emerged to address this urgent question. Numerous influential methods have been proposed, from local linear approximations to rule lists and counterfactuals. In this article, I highlight three conceptual challenges that are largely overlooked by authors in this area. I argue that the vast majority of IML algorithms are plagued by (1) ambiguity with respect to their true target; (2) a disregard for error rates and severe testing; and (3) an emphasis on product over process. Each point is developed at length, drawing on relevant debates in epistemology and philosophy of science. Examples and counterexamples from IML are considered, demonstrating how failure to acknowledge these problems can result in counterintuitive and potentially misleading explanations. Without greater care for the conceptual foundations of IML, future work in this area is doomed to repeat the same mistakes.

Supervised outlier detection for classification and regression

Outlier detection, i.e., the task of detecting points that are markedly different from the data sample, is an important challenge in machine learning. When a model is built, these special points can skew the model training and result in less accurate predictions. Due to this fact, it is important to identify and remove them before building any supervised model and this is often the first step when dealing with a machine learning problem. Nowadays, there exists a very large number of outlier detector algorithms that provide good results, but their main drawbacks are their unsupervised nature together with the hyperparameters that must be properly set for obtaining good performance. In this work, a new supervised outlier estimator is proposed. This is done by pipelining an outlier detector with a following a supervised model, in such a way that the targets of the later supervise how all the hyperparameters involved in the outlier detector are optimally selected. This pipeline-based approach makes it very easy to combine different outlier detectors with different classifiers and regressors. In the experiments done, nine relevant outlier detectors have been combined with three regressors over eight regression problems as well as with two classifiers over another eight binary and multi-class classification problems. The usefulness of the proposal as an objective and automatic way to optimally determine detector hyperparameters has been proven and the effectiveness of the nine outlier detectors has also been analyzed and compared.

Multiparametric Oncologic Hybrid Imaging: Machine Learning Challenges and Opportunities.

Machine learning (ML) is considered an important technology for future data analysis in health care. The inherently technology-driven fields of diagnostic radiology and nuclear medicine will both benefit from ML in terms of image acquisition and reconstruction. Within the next few years, this will lead to accelerated image acquisition, improved image quality, a reduction of motion artifacts and - for PET imaging - reduced radiation exposure and new approaches for attenuation correction. Furthermore, ML has the potential to support decision making by a combined analysis of data derived from different modalities, especially in oncology. In this context, we see great potential for ML in multiparametric hybrid imaging and the development of imaging biomarkers. In this review, we will describe the basics of ML, present approaches in hybrid imaging of MRI, CT, and PET, and discuss the specific challenges associated with it and the steps ahead to make ML a diagnostic and clinical tool in the future. · ML provides a viable clinical solution for the reconstruction, processing, and analysis of hybrid imaging obtained from MRI, CT, and PET.. · Küstner T, Hepp T, Seith F. Multiparametric Oncologic Hybrid Imaging: Machine Learning Challenges and Opportunities. Fortschr Röntgenstr 2022; 194: 605 - 612.

A comprehensive survey of clustering algorithms: State-of-the-art machine learning applications, taxonomy, challenges, and future research prospects

Clustering is an essential tool in data mining research and applications. It is the subject of active research in many fields of study, such as computer science, data science, statistics, pattern recognition, artificial intelligence, and machine learning. Several clustering techniques have been proposed and implemented, and most of them successfully find excellent quality or optimal clustering results in the domains mentioned earlier. However, there has been a gradual shift in the choice of clustering methods among domain experts and practitioners alike, which is precipitated by the fact that most traditional clustering algorithms still depend on the number of clusters provided a priori. These conventional clustering algorithms cannot effectively handle real-world data clustering analysis problems where the number of clusters in data objects cannot be easily identified. Also, they cannot effectively manage problems where the optimal number of clusters for a high-dimensional dataset cannot be easily determined. Therefore, there is a need for improved, flexible, and efficient clustering techniques. Recently, a variety of efficient clustering algorithms have been proposed in the literature, and these algorithms produced good results when evaluated on real-world clustering problems. This study presents an up-to-date systematic and comprehensive review of traditional and state-of-the-art clustering techniques for different domains. This survey considers clustering from a more practical perspective. It shows the outstanding role of clustering in various disciplines, such as education, marketing, medicine, biology, and bioinformatics. It also discusses the application of clustering to different fields attracting intensive efforts among the scientific community, such as big data, artificial intelligence, and robotics. This survey paper will be beneficial for both practitioners and researchers. It will serve as a good reference point for researchers and practitioners to design improved and efficient state-of-the-art clustering algorithms.

Modeling Drosophila vision neural pathways to detect weak moving targets from cluttered backgrounds

In target detection and recognition, low intensity and small volume characterize weak targets and pose serious challenges to their detection. Cluttered backgrounds pose another difficulty for target detection by generating a large amount of interfering noise. Detecting weak moving targets from cluttered backgrounds remains a challenging task; however, this topic has been extensively investigated for some time. The vision of Drosophila flies provides an excellent paradigm to address this issue because it can accurately and efficiently detect weak moving targets during high-speed flight. This type of biological behavior provides valuable inspiration for weak target detection methods; however, it has not been fully analyzed in the relevant literature thus far. In this study, we propose a neural vision pathway model based on the visual system of Drosophila to functionally simulate neural activation for the detection of weak moving targets. Specifically, we mathematically modeled a weak target motion detector to generate a group of responses to small motions in cluttered backgrounds. We then fused these responses to model the consequent sensitivity of the Drosophila neural vision pathways. The proposed model functionally interprets the advantages of the neural vision architecture of the species to perform the target-detection task. Experimental results demonstrated the effectiveness of the proposed bionic model in detecting weak moving targets embedded in cluttered backgrounds. Moreover, we systematically analyzed the sensitivity preferences of the model for different target sizes. These findings are qualitatively and quantitatively presented to demonstrate the neurobiological advantages of the neural architecture of Drosophila flies on a conventional machine learning challenge.

Joint Patch Weighting and Moment Matching for Unsupervised Domain Adaptation in Micro-Expression Recognition

Unsupervised domain adaptation (DA) is a challenging machine learning problem since the labeled training (source) and unlabeled testing (target) sets belong to different domains and then have different feature distributions, which has recently attracted wide attention in micro-expression recognition (MER). Although some well-performing unsupervised DA methods have been proposed, these methods cannot well solve the problem of unsupervised DA in MER, a. k. a., cross-domain MER. To deal with such a challenging problem, in this letter we propose a novel unsupervised DA method called Joint Patch weighting and Moment Matching (JPMM). JPMM bridges the source and target micro-expression feature sets by minimizing their probability distribution divergence with a multi-order moment matching operation. Meanwhile, it takes advantage of the contributive facial patches by the weight learning such that a domain-invariant feature representation involving micro-expression distinguishable information can be learned. Finally, we carry out extensive experiments to evaluate the proposed JPMM method is superior to recent state-of-the-art unsupervised DA methods in dealing with cross-domain MER.

Vec2image: an explainable artificial intelligence model for the feature representation and classification of high-dimensional biological data by vector-to-image conversion.

Feature representation and discriminative learning are proven models and technologies in artificial intelligence fields; however, major challenges for machine learning on large biological datasets are learning an effective model with mechanistical explanation on the model determination and prediction. To satisfy such demands, we developed Vec2image, an explainable convolutional neural network framework for characterizing the feature engineering, feature selection and classifier training that is mainly based on the collaboration of principal component coordinate conversion, deep residual neural networks and embedded k-nearest neighbor representation on pseudo images of high-dimensional biological data, where the pseudo images represent feature measurements and feature associations simultaneously. Vec2image has achieved better performance compared with other popular methods and illustrated its efficiency on feature selection in cell marker identification from tissue-specific single-cell datasets. In particular, in a case study on type 2 diabetes (T2D) by multiple human islet scRNA-seq datasets, Vec2image first displayed robust performance on T2D classification model building across different datasets, then a specific Vec2image model was trained to accurately recognize the cell state and efficiently rank feature genes relevant to T2D which uncovered potential T2D cellular pathogenesis; and next the cell activity changes, cell composition imbalances and cell–cell communication dysfunctions were associated to our finding T2D feature genes from both population-shared and individual-specific perspectives. Collectively, Vec2image is a new and efficient explainable artificial intelligence methodology that can be widely applied in human-readable classification and prediction on the basis of pseudo image representation of biological deep sequencing data.

Challenges for machine learning in RNA-protein interaction prediction.

RNA-protein interactions have long being recognised as crucial regulators of gene expression. Recently, the development of scalable experimental techniques to measure these interactions has revolutionised the field, leading to the production of large-scale datasets which offer both opportunities and challenges for machine learning techniques. In this brief note, we will discuss some of the major stumbling blocks towards the use of machine learning in computational RNA biology, focusing specifically on the problem of predicting RNA-protein interactions from next-generation sequencing data.

Hybrid Symmetrical Uncertainty and Reference Set Harmony Search Algorithm for Gene Selection Problem

Selecting the most miniature possible set of genes from microarray datasets for clinical diagnosis and prediction is one of the most challenging machine learning tasks. A robust gene selection technique is required to identify the most significant subset of genes by removing spurious or non-predictive genes from the original dataset without sacrificing or reducing classification accuracy. Numerous studies have attempted to address this issue by implementing either a filter or a wrapper. Although the filter approaches are computationally efficient, they are completely independent of the induction algorithm. On the other hand, wrapper approaches outperform filter approaches but are computationally more expensive. Therefore, this study proposes an enhanced gene selection method that uses a hybrid technique that combines the Symmetrical Uncertainty (SU) filter and Reference Set Harmony Search Algorithm (RSHSA) wrapper method, known as SU-RSHSA. The framework to develop the proposed SU-RSHSA includes numerous stages: (1) investigate a novel gene selection method based on the HSA and will then determine appropriate values for the HSA’s parameters, (2) enhance the construction process of the initial harmony memory while satisfying the diversity of the solution by embedding a reference set within the HSA (RSHSA), and (3) investigates the effect of integrating Symmetrical Uncertainty (SU) as a filter and RSHSA as a wrapper (SU-RSHSA) to maximize classification accuracy by leveraging their respective advantages. The results demonstrate that the SU-RSHSA outperforms the original HSA and SU-HSA in terms of classification accuracy, a small number of selected relevant genes, and reduced computational time. More importantly, the proposed SU-RSHSA gene selection method effectively generates a small subset of salient genes with high classification accuracy.

Data Integration Challenges for Machine Learning in Precision Medicine.

A main goal of Precision Medicine is that of incorporating and integrating the vast corpora on different databases about the molecular and environmental origins of disease, into analytic frameworks, allowing the development of individualized, context-dependent diagnostics, and therapeutic approaches. In this regard, artificial intelligence and machine learning approaches can be used to build analytical models of complex disease aimed at prediction of personalized health conditions and outcomes. Such models must handle the wide heterogeneity of individuals in both their genetic predisposition and their social and environmental determinants. Computational approaches to medicine need to be able to efficiently manage, visualize and integrate, large datasets combining structure, and unstructured formats. This needs to be done while constrained by different levels of confidentiality, ideally doing so within a unified analytical architecture. Efficient data integration and management is key to the successful application of computational intelligence approaches to medicine. A number of challenges arise in the design of successful designs to medical data analytics under currently demanding conditions of performance in personalized medicine, while also subject to time, computational power, and bioethical constraints. Here, we will review some of these constraints and discuss possible avenues to overcome current challenges.

Optimization methods for improved efficiency and performance of Deep Q-Networks upon conversion to neuromorphic population platforms

Deep Spiking Neural Networks (SNNs) with event-driven dynamics become increasingly popular in many challenging Machine Learning applications, based on their cheap and efficient computations. The discontinuity of the SNN dynamics, however, leads to problems in the learning process, resulting in performance loss, as the dominant gradient-based training approaches are not easily adaptable to the discontinuous SNN activation domain. One promising approach develops SNNs by converting trained Deep Neural Networks to SNNs, which has been very successful in classification applications. Recently, the scope of the conversion studies has been extended to Deep Q-Networks (DQNs), and highly competitive performance has been achieved on many challenging Atari games. The present work provides a comprehensive description of the DQN to SNN conversion algorithm and evaluates the causes of the potential performance loss during the conversion process. We analyze three key factors which allow practical implementations without loss of generality for a large class of highly demanding Q-learning problems, including robust conversion rate, threshold percentile, and simulation time. Our results are not only competitive to DQN in terms of performance but also highly efficient, which is extremely beneficial upon implementations on neuromorphic platforms.

RDPVR: Random Data Partitioning with Voting Rule for Machine Learning from Class-Imbalanced Datasets

Since most classifiers are biased toward the dominant class, class imbalance is a challenging problem in machine learning. The most popular approaches to solving this problem include oversampling minority examples and undersampling majority examples. Oversampling may increase the probability of overfitting, whereas undersampling eliminates examples that may be crucial to the learning process. We present a linear time resampling method based on random data partitioning and a majority voting rule to address both concerns, where an imbalanced dataset is partitioned into a number of small subdatasets, each of which must be class balanced. After that, a specific classifier is trained for each subdataset, and the final classification result is established by applying the majority voting rule to the results of all of the trained models. We compared the performance of the proposed method to some of the most well-known oversampling and undersampling methods, employing a range of classifiers, on 33 benchmark machine learning class-imbalanced datasets. The classification results produced by the classifiers employed on the generated data by the proposed method were comparable to most of the resampling methods tested, with the exception of SMOTEFUNA, which is an oversampling method that increases the probability of overfitting. The proposed method produced results that were comparable to the Easy Ensemble (EE) undersampling method. As a result, for solving the challenge of machine learning from class-imbalanced datasets, we advocate using either EE or our method.