Arbitrary Datasets Research Articles

A one-parameter family of clear-sky solar irradiance models adapted for different aerosol types

In this paper, a procedure for generating a family of parametrical broadband clear-sky models is described. The key element is the conversion of one or more model input variables into tunable mathematical parameters. The approach is tested on the PS model [Paulescu, M. and Schlett, Z., Theor. Appl. Climatol. 75, 203 (2003)], with the free parameter being the Ångström exponent (α). This allows us to fine-tune for conditions dominated by desert dust, urban-industrial, and mixed aerosols. We find that for an arbitrary set of data, the optimal value of the free parameter is not the same as its actual measured value (inserting the measured value in the model would result in a lower performance). We attribute this fact to the inaccurate nature of the base model. The optimal α value varies with the considered solar irradiance component, aerosol type and loading, and the error measure(s) used for assessing model accuracy. A set of recommended models for each aerosol type and loading class is given. The tabled values for the aerosol transmittance coefficients are also listed. A preliminary validation shows that the newly developed models are very reliable. The optimal version generally falls within a few percent of the results of REST2v5, a benchmark model in clear-sky solar irradiance estimation. While some established models outperform REST2v5 for certain aerosol types and for only one solar irradiance component, the new models prove competitive under most scenarios. Beyond showing the performance of the developed model family, these results hint at great potential of our approach.

Stronger convergence results for deep residual networks: network width scales linearly with training data size

Abstract Deep neural networks are highly expressive machine learning models with the ability to interpolate arbitrary datasets. Deep nets are typically optimized via first-order methods, and the optimization process crucially depends on the characteristics of the network as well as the dataset. This work sheds light on the relation between the network size and the properties of the dataset with an emphasis on deep residual networks (ResNets). Our contribution is that if the network Jacobian is full rank, gradient descent for the quadratic loss and smooth activation converges to the global minima even if the network width $m$ of the ResNet scales linearly with the sample size $n$ and logarithmically with the network depth $H$. Consequently, our work is able to provide a theoretical guarantee for the convergence of deep neural networks in the $m=\varOmega (n,\log H)$ regime.

Optimisation of polymer spur gear pairs with experimental validation

This study aims to present a procedure for optimisation of polymer gear pairs, along with the rough design guidelines. Multi-objective optimisation of polymer spur gear pairs was carried out. The gear pair module, face width, pinion number of teeth, and profile shift coefficients of both gears served as design variables. Two objective functions were used to rate the designs—volume and frictional power losses. Compared with well-researched steel gear pair optimisation, additional boundary conditions were necessary: tooth flank and root temperatures, abrasion wear, and tooth addendum displacement. Two arbitrary datasets were used as examples, each made of polyoxymethylene. For each of the sets, a Pareto optimal solution was manufactured to enable the experimental validation. For this purpose, an open-circuit experimental rig was designed. As no literature was found on the polymer gear pair optimisation, results were compared with ones for steel gear pairs. The results have shown that changes in macro-geometry affect polymer gears significantly different. The lower volume pairs made of polymer had greater face widths, while the opposite was observed in steel gears. The differences between the analytical and experimental results were up to 21%.

Exploiting relationships between outcomes in Bayesian multivariate network meta-analysis with an application to relapsing-remitting multiple sclerosis.

In multivariate network meta-analysis (NMA), the piecemeal nature of the evidence base means that there may be treatment-outcome combinations for which no data is available. Most existing multivariate evidence synthesis models are either unable to estimate the missing treatment-outcome combinations, or can only do so under particularly strong assumptions, such as perfect between-study correlations between outcomes or constant effect size across outcomes. Many existing implementations are also limited to two treatments or two outcomes, or rely on model specification that is heavily tailored to the dimensions of the dataset. We present a Bayesian multivariate NMA model that estimates the missing treatment-outcome combinations via mappings between the population mean effects, while allowing the study-specific effects to be imperfectly correlated. The method is designed for aggregate-level data (rather than individual patient data) and is likely to be useful when modeling multiple sparsely reported outcomes, or when varying definitions of the same underlying outcome are adopted by different studies. We implement the model via a novel decomposition of the treatment effect variance, which can be specified efficiently for an arbitrary dataset given some basic assumptions regarding the correlation structure. The method is illustrated using data concerning the efficacy and liver-related safety of eight active treatments for relapsing-remitting multiple sclerosis. The results indicate that fingolimod and interferon beta-1b are the most efficacious treatments but also have some of the worst effects on liver safety. Dimethyl fumarate and glatiramer acetate perform reasonably on all of the efficacy and safety outcomes in the model.

Weighted Mutual Information for Aggregated Kernel Clustering.

Background: A common task in machine learning is clustering data into different groups based on similarities. Clustering methods can be divided in two groups: linear and nonlinear. A commonly used linear clustering method is K-means. Its extension, kernel K-means, is a non-linear technique that utilizes a kernel function to project the data to a higher dimensional space. The projected data will then be clustered in different groups. Different kernels do not perform similarly when they are applied to different datasets. Methods: A kernel function might be relevant for one application but perform poorly to project data for another application. In turn choosing the right kernel for an arbitrary dataset is a challenging task. To address this challenge, a potential approach is aggregating the clustering results to obtain an impartial clustering result regardless of the selected kernel function. To this end, the main challenge is how to aggregate the clustering results. A potential solution is to combine the clustering results using a weight function. In this work, we introduce Weighted Mutual Information (WMI) for calculating the weights for different clustering methods based on their performance to combine the results. The performance of each method is evaluated using a training set with known labels. Results: We applied the proposed Weighted Mutual Information to four data sets that cannot be linearly separated. We also tested the method in different noise conditions. Conclusions: Our results show that the proposed Weighted Mutual Information method is impartial, does not rely on a single kernel, and performs better than each individual kernel specially in high noise.

On the insecurity of quantum Bitcoin mining

Grover's algorithm confers on quantum computers a quadratic advantage over classical computers for searching in an arbitrary data set, a scenario that describes Bitcoin mining. It has previously been argued that the only side-effect of quantum mining would be an increased difficulty. In this work, we argue that a crucial argument in the analysis of Bitcoin security breaks down when quantum mining is performed. Classically, a Bitcoin fork occurs rarely, i.e., when two miners find a block almost simultaneously, due to propagation time effects. The situation differs dramatically when quantum miners use Grover's algorithm, which repeatedly applies a procedure called a Grover iteration. The chances of finding a block grow quadratically with the number of Grover iterations applied. Crucially, a miner does not have to choose how many iterations to apply in advance. Suppose Alice receives Bob's new block. To maximize her revenue, she should stop and measure her state immediately in the hopes that her block (rather than Bob's) will become part of the longest chain. The strong correlation between the miners' actions and the fact that they all measure their states at the same time may lead to more forks -- which is known to be a security risk for Bitcoin. We propose a mechanism that, we conjecture, will prevent this form of quantum mining, thereby circumventing the high rate of forks.

Heterogeneous Network Architecture for Integration of AI and Quantum Optics by Means of Multiple-Valued Logic

Quantum optics is regarded as the acknowledged method to provide network quantum keys distribution and in the future secure distributed quantum computing, but it should also provide cryptography protection for mobile robots and the Internet of Things (IoT). This task requires the design of new secret coding schemes, which can be also based on multiple-valued logic (MVL). However, this very specific logic model reveals new possibilities for the hierarchical data clustering of arbitrary data sets. The minimization of multiple-valued logic functions is proposed for the analysis of aggregated objects, which is possible for an arbitrary number of variables. In order to use all the useful properties of the multiple-valued logic, the heterogeneous network architecture is proposed, which includes three allocated levels of artificial intelligence (AI) logic modeling for discrete multiple-valued logic, Boolean logic, and fuzzy logic. Multiple-valued logic is regarded as the possible platform for additional secret coding, data aggregation, and communications, which are provided by the united high dimensional space for network addressing and the targeted control of robotic devices. Models of Boolean and fuzzy logic are regarded as separate logic levels in order to simplify the integration of various algorithms and provide control of additional data protection means for robotic agents.

Global collaboration through local interaction in competitive learning

Feature maps, that preserve the global topology of arbitrary datasets, can be formed by self-organizing competing agents. So far, it has been presumed that global interaction of agents is necessary for this process. We establish that this is not the case, and that global topology can be uncovered through strictly local interactions. Enforcing uniformity of map quality across all agents results in an algorithm that is able to consistently uncover the global topology of diversely challenging datasets. The applicability and scalability of this approach is further tested on a large point cloud dataset, revealing a linear relation between map training time and size. The presented work not only reduces algorithmic complexity but also constitutes first step towards a distributed self organizing map.

Measuring the Significance of Writing Style for Recommending Where to Publish – A Case Study

Abstract Writing style plays a role in publication venue recommendation. However, such finding should be observed further; it is concluded from an arbitrary dataset which contains various topics and writing quality. This paper aims to observe that style's impact in a more controlled environment. A dataset with the same specific topic and writing quality was used and analysed. In our case, the dataset is comprised of papers published on reputable software engineering publication venues with natural language generation as the specific topic. According to our observation, writing style only affects significantly on paper title wherein the impact is proportional to n in n-gram. Moreover, the style’s impact becomes more salient when the venues are grouped per publisher or only a specific publication type is considered.

Heuristic filter feature selection methods for medical datasets

Gene selection is the process of selecting the optimal feature subset in an arbitrary dataset. The significance of gene selection is in high dimensional datasets in which the number of samples and features are low and high respectively. The major goals of gene selection are increasing the accuracy, finding the minimal effective feature subset, and increasing the performance of evaluations. This paper proposed two heuristic methods for gene selection, namely, Xvariance against Mutual Congestion. Xvariance tries to classify labels using internal attributes of features however Mutual Congestion is frequency based. The proposed methods have been conducted on eight binary medical datasets. Results reveal that Xvariance works well with standard datasets, however Mutual Congestion improves the accuracy of high dimensional datasets considerably.

Dictionary Indexing of Electron Back-Scatter Diffraction Patterns: a Hands-On Tutorial

Dictionary indexing of electron back-scatter patterns was recently proposed as an alternative to the commercially available indexing packages. In this tutorial paper, we describe in detail the various steps that need to be taken to successfully complete an indexing run on an arbitrary data set. We provide three toy data sets for the reader to experiment with: poly-crystalline nickel with several different acquisition conditions, orthorhombic forsterite, and a single slice from a large serial sectioning experiment on a Ni-based superalloy. The data files and all files produced by the indexing routine are made available as Supplementary Material (https://doi.org/10.1184/R1/7792505).

Image generation via latent space learning using improved combination

Many researches have brought progress in learning a good generative model by combing the advantages of GAN and VAE, where latent space learning is always important for generating high-quality images. But these existing works mainly seek to impose the latent space a given distribution in advance or to obey a Gaussian distribution with KL divergence penalty, which leads to the difficulties of deciding a suitable prior distribution corresponding to different datasets. Thus in this paper we develop a two-stage combining method of AE and GAN under unsupervised and supervised conditions respectively, each stage performed to improve the effect of modeling latent distribution. In the first stage, an adversarial procedure achieves to match the latent distribution with real data distribution determined by arbitrary dataset without having access to a pre-set prior distribution. In the second stage, besides one adversarial procedure trained for outputting images, the other adversarial procedure is designed to attain the goal of optimizing the latent distribution of the first stage via back-propagation. Therefore, loop optimization of the network parameters during training will at last allow the framework to map an input noise to a high quality image. Extensive experiments are conducted to verify the performance of representing latent space and generating images on different datasets including MNIST, fashion-MNIST, CIFAR-10, and CelebA. The code and tutorials already release at https://github.com/TwistedW/CAE-CGAN.

Simultaneous core multipartitions

We initiate the study of simultaneous core multipartitions, generalising simultaneous core partitions, which have been studied extensively in the recent literature. Given a multipartition datum(s∣c), which consists of a non-negative integer s and an l-tuple c of integers, we introduce the notion of an (s∣c)-core multipartition. Given an arbitrary set of multipartition data, we give necessary and sufficient conditions for the corresponding set of simultaneous core multipartitions to be finite. We then study the special case of simultaneous core bipartitions, giving exact enumerative results in some special subcases.

Load Profile Extraction by Mean-Shift Clustering with Sample Pearson Correlation Coefficient Distance

In this paper, a clustering method with proposed distance measurement to extract base load profiles from arbitrary data sets is studied. Recently, smart energy load metering devices are broadly deployed, and an immense volume of data is now collected. However, as this large amount of data has been explosively generated over such a short period of time, the collected data is hardly organized to be employed for study, applications, services, and systems. This paper provides a foundation method to extract base load profiles that can be utilized by power engineers, energy system operators, and researchers for deeper analysis and more advanced technologies. The base load profiles allow them to understand the patterns residing in the load data to discover the greater value. Up to this day, experts with domain knowledge often have done the base load profile realization manually. However, the volume of the data is growing too fast to handle it with the conventional approach. Accordingly, an automated yet precise method to recognize and extract the base power load profiles is studied in this paper. For base load profile extraction, this paper proposes Sample Pearson Correlation Coefficient (SPCC) distance measurement and applies it to Mean-Shift algorithm based nonparametric mode-seeking clustering. The superiority of SPCC distance over traditional Euclidean distance is validated by mathematical and numerical analysis.

Memberships of the Open Cluster NGC 6405 Based on a Combined Method: Gaussian Mixture Model and Random Forest

Abstract This paper presents a combined method of Gaussian mixture model and random forest to compute membership probabilities of stars by using large, high-dimensional data sets. A significant advantage of this method is that it allows us to easily identify likely cluster members in large data sets starting from small training samples. As a benchmark, we select 40318 stars in the field of the open cluster NGC 6405 from the Gaia Data Release 2 (Gaia-DR2) by means of all five astrometric (positions, proper motions, and parallax) and photometric parameters. We use this combined method to determine likely cluster members in an eleven-dimensional parameter space. A total number of 518 high-probability (≥0.6) memberships are obtained, and the mean parallax and proper motion of the cluster are determined to be 2.171 ± 0.005 mas (461 ± 1 pc) and ( ) = (−1.357 ± 0.023,−5.823 ± 0.020) mas yr−1, respectively. In addition, we quantitatively evaluate the relative importance of the parameters for membership determination and find that colors and magnitudes cannot be ignored in membership determination when using the RF method. Our results show that this combined method exhibits good performance in handling arbitrary high-dimensional and large data sets, such as Gaia-DR2, and it can also be used to investigate other open clusters.

Decomposition matrices for the special case of data on the triangular lattice of G(2)

A method for the decomposition of data functions sampled on a finite fragment of triangular lattice is described for the cases of lattices of any density corresponding to the simple Lie group G ( 2 ) . Its main advantage is the fact that the decomposition matrix needs to be calculated only once for arbitrary sets of data sampled on the same set of discrete points. The decomposition matrix applies to lattice of any density that carries data.

Simulation of a Multidimensional Input Quantum Perceptron

In this work, we demonstrate the improved data separation capabilities of the Multidimensional Input Quantum Perceptron (MDIQP), a fundamental cell for the construction of more complex Quantum Artificial Neural Networks (QANNs). This is done by using input controlled alterations of ancillary qubits in combination with phase estimation and learning algorithms. The MDIQP is capable of processing quantum information and classifying multidimensional data that may not be linearly separable, extending the capabilities of the classical perceptron. With this powerful component, we get much closer to the achievement of a feedforward multilayer QANN, which would be able to represent and classify arbitrary sets of data (both quantum and classical).

Why locality sensitive hashing works: A practical perspective

Locality Sensitive Hashing (LSH) is one of the most efficient approaches to the nearest neighbor search problem in high dimensional spaces. A family H of hash functions is called locality sensitive if the collision probability ph(r) of any two points 〈q,p〉 at distance r over a random hash function h decreases with r. The classic LSH algorithm employs a data structure consisting of k⁎ℓ randomly chosen hash functions to achieve more desirable collision curves and the collision probability Phkℓ(r) for 〈q,p〉 is equal to 1−(1−ph(r)k)ℓ. The great success of LSH is usually attributed to the solid theoretical guarantee for Phkℓ(r) and ph(r).In practice, however, users are more interested in recall rate, i.e., the probability that a random query collides with its r-near neighbor over a fixed LSH data structure hℓk. Implicitly or explicitly, Phkℓ(r) is often misinterpreted as recall rate and used to predict the performance of LSH. This is problematic because Phkℓ(r) is actually the expectation of recall rates. Interestingly, numerous empirical studies show that, for most (if not all) real datasets and a fixed sample of random LSH data structure, the recall rate is very close to Phkℓ(r). In this paper, we provide a theoretical justification for this phenomenon. We show that (1) for random datasets the recall rate is asymptotically equal to Phkℓ(r); (2) for arbitrary datasets the variance of the recall rate is very small as long as the parameter k and ℓ are properly chosen and the size of datasets is large enough. Our analysis (1) explains why the practical performance of LSH (the recall rate) matches so well with the theoretical expectation (Phkℓ(r)); and (2) indicates that, in addition to the nice theoretical guarantee, the mechanism by which LSH data structures are constructed and the huge amount of data are also the main causes for the success of LSH in practice.

Tackling business intelligence with bioinspired deep learning

To tackle the complex problem of providing business intelligence solutions based on business data, bioinspired deep learning has to be considered. This paper focuses on the application of artificial metaplasticity learning in business intelligence systems as an alternative paradigm of achieving a deeper information extraction and learning from arbitrary size data sets. As a case study, artificial metaplasticity multilayer perceptron applied to the automation of credit approval decision based on collected client data is analyzed, showing its potential and improvements over the state-of-the-art techniques. This paper successfully introduces the relevant novelty that the artificial neural network itself estimates the pdf of the input data to be used in the metaplasticity learning, so it is much closer to the biologic reality than previous implementations of artificial metaplasticity.

Estimator learning automata for feature subset selection in high‐dimensional spaces, case study: Email spam detection

SummaryOne of the difficult challenges facing data miners is that algorithm performance degrades if the feature space contains redundant or irrelevant features. Therefore, as a critical preprocess task, dimension reduction is used to build a smaller space containing valuable features. There are 2 different approaches for dimension reduction: feature extraction and feature selection, which itself is divided into wrapper and filter approaches. In high‐dimensional spaces, feature extraction and wrapper approaches are not applicable due to the time complexity. On the other hand, the filter approach suffers from inaccuracy. One main reason for this inaccuracy is that the subset's size is not determined considering specifications of the problem.In this paper, we propose ESS (estimator learning automaton‐based subset selection) as a new method for feature selection in high‐dimensional spaces. The innovation of ESS is that it combines wrapper and filter ideas and uses estimator learning automata to efficiently determine a feature subset that leads to a desirable tradeoff between the accuracy and efficiency of the learning algorithm. To find a qualified subset for a special processing algorithm that functions on an arbitrary dataset, ESS uses an automaton to score each candidate subset upon the scale of the subset and accuracy of the learning algorithm using it. In the end, the subset with the highest score is returned. We have used ESS for feature selection in the framework of spam detection, a text classification task for email as a pervasive communication medium. The results show achievement in reaching the goal stated above.