Kinds Of Datasets Research Articles

Monitoring olive mills waste disposal areas in Crete using very high resolution satellite data

This paper evaluates the efficiency of different image analysis techniques applied to high resolution multispectral satellite data so as to identify olive oil waste disposal areas in the island of Crete where huge quantities of wastes are produced. For this purpose very high spatial resolution images including Pleiades, SPOT 6, QuickBird, WorldView-2 and GeoEye 1 have been exploited. The research included the application of the Normalised Difference Vegetation Index, Olive Oil Mill Waste Index as well as Principal Component Analysis. Moreover Intensity–Hue–Saturation transformation was carried out. Furthermore, unsupervised classification was performed for a variety of classes (5; 10 and 15) over the same area for two different periods. In addition, supervised linear constrained spectral un-mixing technique has been applied for the WorldView-2 image, to evaluate the potential use of sub-pixel analysis. Indeed, as it is demonstrated NDVI and OOMW indices may be used to enhance the exposure of disposal areas in high resolution satellite datasets, while the application of the PCA and HIS transformations seems to be able to further improve the results. Unsupervised classification techniques, with no ground truth data, can sufficiently work; however temporal changes of the disposal areas can affect the performance of the classifier. The use of spectral library was able to detect OOMW areas with a relatively high rate of success improving the results from the unsupervised classification. Finally, a COSMO-SkyMed radar image has been examined and fused with a hyperspectral EO-ALI image, indicating that such kind of datasets might be also explored for this purpose.

Parameter Self-Optimizing Clustering for Autonomous Extraction of the Weld Seam Based on Orientation Saliency in Robotic MAG Welding

This paper presents an effective method which needs free parameters as little as possible to autonomously extract the weld seam profile and edges from the molten background in two kinds of weld images within robotic MAG welding. First, orientation saliency detection produced by Gabor filtering nicely highlights the weld seam profile and edges from the molten background. Then, an unsupervised clustering algorithm combing a cluster validity index via an optimization rule, referred to as parameter self-optimizing clustering, is applied to discern the weld seam profile and edges from interference data after the orientation saliency detection result is given threshold segmentation. The validity index is better than the classical ones in two kinds of data sets through considerable tests. Last, two common applications of weld seam identification demonstrate the effectiveness of the proposed method.

Improved Density Based Spatial Clustering of Applications of Noise Clustering Algorithm for Knowledge Discovery in Spatial Data

There are many techniques available in the field of data mining and its subfield spatial data mining is to understand relationships between data objects. Data objects related with spatial features are called spatial databases. These relationships can be used for prediction and trend detection between spatial and nonspatial objects for social and scientific reasons. A huge data set may be collected from different sources as satellite images, X-rays, medical images, traffic cameras, and GIS system. To handle this large amount of data and set relationship between them in a certain manner with certain results is our primary purpose of this paper. This paper gives a complete process to understand how spatial data is different from other kinds of data sets and how it is refined to apply to get useful results and set trends to predict geographic information system and spatial data mining process. In this paper a new improved algorithm for clustering is designed because role of clustering is very indispensable in spatial data mining process. Clustering methods are useful in various fields of human life such as GIS (Geographic Information System), GPS (Global Positioning System), weather forecasting, air traffic controller, water treatment, area selection, cost estimation, planning of rural and urban areas, remote sensing, and VLSI designing. This paper presents study of various clustering methods and algorithms and an improved algorithm of DBSCAN as IDBSCAN (Improved Density Based Spatial Clustering of Application of Noise). The algorithm is designed by addition of some important attributes which are responsible for generation of better clusters from existing data sets in comparison of other methods.

A Novel SMOTE-Based Classification Approach to Online Data Imbalance Problem

In many practical engineering applications, data are usually collected in online pattern. However, if the classes of these data are severely imbalanced, the classification performance will be restricted. In this paper, a novel classification approach is proposed to solve the online data imbalance problem by integrating a fast and efficient learning algorithm, that is, Extreme Learning Machine (ELM), and a typical sampling strategy, that is, the synthetic minority oversampling technique (SMOTE). To reduce the severe imbalance, the granulation division for major-class samples is made according to the samples’ distribution characteristic, and the original samples are replaced by the obtained granule core to prepare a balanced sample set. In online stage, we firstly make granulation division for minor-class and then conduct oversampling using SMOTE in the region around granule core and granule border. Therefore, the training sample set is gradually balanced and the online ELM model is dynamically updated. We also theoretically introduce fuzzy information entropy to prove that the proposed approach has the lower bound of model reliability after undersampling. Numerical experiments are conducted on two different kinds of datasets, and the results demonstrate that the proposed approach outperforms some state-of-the-art methods in terms of the generalization performance and numerical stability.

Ocean satellite data assimilation experiments in FIO-ESM using ensemble adjustment Kalman filter

Using Ensemble Adjustment Kalman Filter (EAKF), two types of ocean satellite datasets were assimilated into the First Institute of Oceanography Earth System Model (FIO-ESM), v1.0. One control experiment without data assimilation and four assimilation experiments were conducted. All the experiments were ensemble runs for 1-year period and each ensemble started from different initial conditions. One assimilation experiment was designed to assimilate sea level anomaly (SLA); another, to assimilate sea surface temperature (SST); and the other two assimilation experiments were designed to assimilate both SLA and SST but in different orders. To examine the effects of data assimilation, all the results were compared with an objective analysis dataset of EN3. Different from the ocean model without coupling, the momentum and heat fluxes were calculated via air-sea coupling in FIO-ESM, which makes the relations among variables closer to the reality. The outputs after the assimilation of satellite data were improved on the whole, especially at depth shallower than 1000 m. The effects due to the assimilation of different kinds of satellite datasets were somewhat different. The improvement due to SST assimilation was greater near the surface, while the improvement due to SLA assimilation was relatively great in the subsurface. The results after the assimilation of both SLA and SST were much better than those only assimilated one kind of dataset, but the difference due to the assimilation order of the two kinds of datasets was not significant.

Estimation method to identify scalar point source in turbulent flow based on Taylor’s diffusion theory

We introduce a new approach to diffusion-source estimation for quick identification of the unknown source, based on Taylor’s diffusion theory for turbulent transport of passive scalar from a fixed point source. In order to evaluate the method, we used planar laser-induced fluorescence to measure the concentration field of fluorescent dye in water flowing in a channel. We considered two kinds of datasets: basis data and observed data. The former is used to determine the basis functions characterizing the streamwise dependence of variances for three statistics: the mean concentration, root-mean-square (RMS) of fluctuations in the concentration, and RMS of the temporal gradient of the fluctuating concentration. Consistent with Taylor’s theory, we found that the lateral distribution of each statistic was basically Gaussian, and their standard deviations increased as a function of the square root of the distance from the emitted point. Based on these facts, a basis function can be formulated and expected to be valid for estimation of unknown sources. Source estimation was performed with the observed data, which corresponded to limited available information about the concentration from an unknown point source. We confirmed a good prediction accuracy of the proposed method with an averaged bias as small as the turbulent integral scale. Better precision was achieved by employing several statistics simultaneously. In this case, the standard deviation of the estimated source position was assessed at 14 % of the mean distance between the source and measurement points, after 100 source-estimate trials with different datasets. The methodology tested in this paper is expected to be applicable more general and complex environmental diffusion issues involving anisotropic turbulent dispersion, and space–time variable mainstream systems; but its versatility in such systems is currently under investigation.

Fast algorithms of attribute reduction for covering decision systems with minimal elements in discernibility matrix

Covering rough sets, which generalize traditional rough sets by considering coverings instead of partitions, are introduced to deal with set-valued, missing-valued or real-valued data sets. For decision systems with such kinds of data sets, attribute reduction with covering rough sets aims to delete superfluous attributes, and fast algorithms of finding reducts are clearly meaningful for practical problems. In the existing study of attribute reduction with covering rough sets, the approach of discernibility matrix is the theoretical foundation. However, it always shares heavy computation load and large store space because of finding and storing all elements in the discernibility matrix. In this paper, we find that only minimal elements in the discernibility matrix are sufficient to find reducts. This fact motivates us in this paper to develop algorithms to find reducts by only employing minimal elements without computing other elements in the discernibility matrix. We first define the relative discernible relation of covering to characterize the relationship between minimal elements in the discernibility matrix and particular sample pairs in the covering decision system. By employing this relative discernible relation, we then develop algorithms to search the minimal elements in the discernibility matrix and find reducts for the covering decision system. Finally, experimental comparisons with other existing algorithms of covering rough sets on several data sets demonstrate that the proposed algorithms in this paper can greatly reduce the running time of finding reducts.

Center symmetric local binary co-occurrence pattern for texture, face and bio-medical image retrieval

Content based image retrieval is a common problem for a large image database. Many methods have been proposed for image retrieval for some particular type of datasets. In the proposed work, a new image retrieval technique has been introduced. This technique is useful for different kind of dataset. In the proposed method, center symmetric local binary pattern has been extracted from the original image to obtain the local information. Co-occurrence of pixel pairs in local pattern map have been observed in different directions and distances using gray level co-occurrence matrix. Earlier methods have utilized histogram to extract the frequency information of local pattern map but co-occurrence of pixel pairs is more robust than frequency of patterns. The proposed method is tested on three different category of images, i.e., texture, face and medical image database and compared with typical state-of-the-art local patterns.

Histogram of Radon transform with angle correlation matrix for distortion invariant shape descriptor

A shape matching descriptor based on the histogram of Radon transform (HRT) with an angle correlation matrix is proposed. Our descriptor based on HRT is robust to shape rotation, scaling, and translation. Shape distortions provide sparse and dense distortions relative to the angle coordinate on our descriptor. Therefore, we compute an angle correlation matrix and apply the dynamic time warping for a non-linear angle matching to be robust to these transformations. Based on the beam search algorithm, we speed-up the time complexity of our method. Robustness to affine distortions for our approach is shown experimentally on different kinds of datasets and compared with the literature.

Yeast as a cell factory: current state and perspectives.

The yeast Saccharomyces cerevisiae is one of the oldest and most frequently used microorganisms in biotechnology with successful applications in the production of both bulk and fine chemicals. Yet, yeast researchers are faced with the challenge to further its transition from the old workhorse to a modern cell factory, fulfilling the requirements for next generation bioprocesses. Many of the principles and tools that are applied for this development originate from the field of synthetic biology and the engineered strains will indeed be synthetic organisms. We provide an overview of the most important aspects of this transition and highlight achievements in recent years as well as trends in which yeast currently lags behind. These aspects include: the enhancement of the substrate spectrum of yeast, with the focus on the efficient utilization of renewable feedstocks, the enhancement of the product spectrum through generation of independent circuits for the maintenance of redox balances and biosynthesis of common carbon building blocks, the requirement for accurate pathway control with improved genome editing and through orthogonal promoters, and improvement of the tolerance of yeast for specific stress conditions. The causative genetic elements for the required traits of the future yeast cell factories will be assembled into genetic modules for fast transfer between strains. These developments will benefit from progress in bio-computational methods, which allow for the integration of different kinds of data sets and algorithms, and from rapid advancement in genome editing, which will enable multiplexed targeted integration of whole heterologous pathways. The overall goal will be to provide a collection of modules and circuits that work independently and can be combined at will, depending on the individual conditions, and will result in an optimal synthetic host for a given production process.

Tractome: a visual data mining tool for brain connectivity analysis

Diffusion magnetic resonance imaging data allows reconstructing the neural pathways of the white matter of the brain as a set of 3D polylines. This kind of data sets provides a means of study of the anatomical structures within the white matter, in order to detect neurologic diseases and understand the anatomical connectivity of the brain. To the best of our knowledge, there is still not an effective or satisfactory method for automatic processing of these data. Therefore, a manually guided visual exploration of experts is crucial for the purpose. However, because of the large size of these data sets, visual exploration and analysis has also become intractable. In order to make use of the advantages of both manual and automatic analysis, we have developed a new visual data mining tool for the analysis of human brain anatomical connectivity. With such tool, humans and automatic algorithms capabilities are integrated in an interactive data exploration and analysis process. A very important aspect to take into account when designing this tool, was to provide the user with comfortable interaction. For this purpose, we tackle the scalability issue in the different stages of the system, including the automatic algorithm and the visualization and interaction techniques that are used.

INSILICO PROTEOME SCREENING TO IDENTIFY PROSPECTIVE DRUG TARGETS IN BACILLUS ANTHRACIS

Various Insilico based genome screening methods helped us in identifying the key drug targets for a pathogen. The accuracy of the predictions are systematically based on the benchmarks at different stages of methodology and the kind of dataset which is considered for the study. In the current study, we made an effort to screen the entire proteome of Bacillus anthracis for identification of putative drug targets. B. anthracis is the causautive agent for anthrax disease. Instead of genome sequence, the metabolically classified proteome of B. anthracis from JCVI-CMR database was considered for the present study. The entire proteome is been categorized into 25 different metabolisms and in each sub-categorised metabolisms respective protein sequences were retrieved and subjected to screening against Database of Essential Genes (DEG) and Human-Basic Local Alignment Search Tool (H-BLAST) databases. In total 136 essential genes/proteins were obtained from the DEGp (protein) screening whereas 145 Non-Human Homologs (NHHs) were predicted. The identified 145 NHHs are further subjected to criteria based selection to identify the most suitable, functional, putative drug targets. The 8 common hits of both DEG and H-BLAST were considered to be better potential targets as they justify the criteria of being an essential gene/protein, non-human homolog, availability of the 3D structure in PDB and having a significant functional role in the cellular biochemical processes.

Document Clustering Using Semantic Cliques Aggregation

The search engines are indispensable tools to find information amidst massive web pages and documents. A good search engine needs to retrieve information not only in a shorter time, but also relevant to the users’ queries. Most search engines provide short time retrieval to user queries; however, they provide a little guarantee of precision even to the highly detailed users’ queries. In such cases, documents clustering centered on the subject and contents might improve search results. This paper presents a novel method of document clustering, which uses semantic clique. First, we extracted the Features from the documents. Later, the associations between frequently co-occurring terms were defined, which were called as semantic cliques. Each connected component in the semantic clique represented a theme. The documents clustered based on the theme, for which we designed an aggregation algorithm. We evaluated the aggregation algorithm effectiveness using four kinds of datasets. The result showed that the semantic clique based document clustering algorithm performed significantly better than traditional clustering algorithms such as Principal Direction Divisive Partitioning (PDDP), k-means, Auto-Class, and Hierarchical Clustering (HAC). We found that the Semantic Clique Aggregation is a potential model to represent association rules in text and could be immensely useful for automatic document clustering.

Multiple nonlinear integral for classification

In this study, a new classification model-Multiple Nonlinear Integral with Multiple projections is proposed, which includes Double Nonlinear Integral extending to new variants for adapting for complicated data distribution and enhancing classification accuracy. When the performance is not satisfying by projecting with classical Nonlinear Integral, the second projection is need to stretch data in one dimension space to two dimension space. The value of two projection forms the 2-dimensional coordinates. All data in two-dimensional space can be classified by a straight line easily. The rest may be deduced by analogy, if the result is still not good for decision, the Multiple Nonlinear Integral can repeat n times double projections, in which n will be an optimized value to balance the performance and the complexity. The repeating can help adjust the data distribution in 2-dimensional space until being classified easily. The classification model based on Multiple Nonlinear Integral is applied to two kinds of datasets. One kind comes from the classical database; another kind is the real data about the HBV Hepatitis B Virus collected from hospital. The experimental results show that the new model has better performance compared with the classical algorithm and the classical Nonlinear Integral. Especially to the HBV data, Multiple Nonlinear Integral presents the superior on diagnosis to the others.

Review on Classification Based on Artificial Neural Networks

Artificial neural networks (ANN) consider classification as one of the most dynamic research and application areas. ANN is the branch of Artificial Intelligence (AI). The neural network was trained by back propagation algorithm. The different combinations of functions and its effect while using ANN as a classifier is studied and the correctness of these functions are analyzed for various kinds of datasets. The back propagation neural network (BPNN) can be used as a highly successful tool for dataset classification with suitable combination of training, learning and transfer functions. When the maximum likelihood method was compared with backpropagation neural network method, the BPNN was more accurate than maximum likelihood method. A high predictive ability with stable and well functioning BPNN is possible. Multilayer feed-forward neural network algorithm is also used for classification. However BPNN proves to be more effective than other classification algorithms.

Unsupervised feature selection using an improved version of Differential Evolution

In this article, an unsupervised feature selection algorithm is proposed using an improved version of a recently developed Differential Evolution technique called MoDE. The proposed algorithm produces an optimal feature subset while optimizing three criteria, namely, the average standard deviation of the selected feature subset, the average dissimilarity of the selected features, and the average similarity of non-selected features with respect to their first nearest neighbor selected features. Normalized mutual information score is employed for computing both the similarity as well as the dissimilarity measures. The experimental results confirm the superiority of the proposed algorithm over the other state-of-the-art unsupervised feature selection algorithms for eight different kinds of datasets with the number of points ranging from 80 to 6238 and the number of dimensions ranging from 30 to 649.

Training algorithms for Radial Basis Function Networks to tackle learning processes with imbalanced data-sets

Nowadays, many real applications comprise data-sets where the distribution of the classes is significantly different. These data-sets are commonly known as imbalanced data-sets. Traditional classifiers are not able to deal with these kinds of data-sets because they tend to classify only majority classes, obtaining poor results for minority classes. The approaches that have been proposed to address this problem can be categorized into three types: resampling methods, algorithmic adaptations and cost sensitive techniques.Radial Basis Function Networks (RBFNs), artificial neural networks composed of local models or RBFs, have demonstrated their efficiency in different machine learning areas. Centers, widths and output weights for the RBFs must be determined when designing RBFNs.Taking into account the locally tuned response of RBFs, the objective of this paper is to study the influence of global and local paradigms on the weights training phase, within the RBFNs design methodology, for imbalanced data-sets. Least Mean Square and the Singular Value Decomposition have been chosen as representatives of local and global weights training paradigms respectively. These learning algorithms are inserted into classical RBFN design methods that are run on imbalanced data-sets and also on these data-sets preprocessed with re-balance techniques. After applying statistical tests to the results obtained, some guidelines about the RBFN design methodology for imbalanced data-sets are provided.

COMPARISON OF VARIOUS CLASSIFICATION ALGORITHMS ON IRIS DATASETS USING WEKA

Classification is one of the most important task of data mining. Main task of data mining is data analysis. For study purpose various algorithm available for classification like decision tree, Navie Bayes, Back propagation, Neural Network, Artificial Neural, Multi-layer perception, Multi class classification, Support vector Machine, k-nearest neighbor etc. In this paper we introduce four algorithms from them. Study purpose we take iris.arff dataset. Implement this all algorithm in iris dataset and compare TP-rate, Fp-rate, Precision, Recall and ROC Curve parameter. Weka is inbuilt tools for data mining. So we used weka for implementation. Keyword: classification, K-nn, ROC, FP-rate, Decision tree, WEKA I. INTRODUCTION Generally, data mining (sometimes called data or knowledge discovery) is the process of analyzing data from different perspectives and summarizing it into useful information - information that can be used to increase revenue, cuts costs, or both. Data mining algorithms which carry out the assigning of objects into related classes are called classifiers. Classification algorithms include two main phases; in the first phase they try to find a model for the class attribute as a function of other variables of the datasets, and in the second phase, they apply previously designed model on the new and unseen datasets for determining the related class of each record (1)(3). There are different methods for data classification such as Decision Trees (DT), Rule Based Methods, Logistic Regression (LogR), Linear Regression (LR), Naive Bayes (NB), Support Vector Machine (SVM), k-Nearest Neighbor (k-NN), Artificial Neural Networks (ANN), Linear Classifier (LC) and so forth. The comparison of the classifiers and using the most predictive classifier is very important. Each of the classification methods shows different efficacy and accuracy based on the kind of dataset (2) . In addition, there are various evaluation metrics for comparing the classification methods that each of them could be useful depending on the kind of the problem. Among the other criteria for comparing the classification methods, one could mention; precision, recall, error rate, confusion matrix. In this article, using a new method, five usual data classification methods (Decision tree, Multi- layer perception, Naive Bayes, C4.5, SVM) have been compared based on the AUC criterion. These mentioned methods have been applied on the random generated datasets which are

Using the Agoraset dataset: Assessing for the quality of crowd video analysis methods

In this paper, we present a simulation-based crowd video dataset to be used for evaluation of low-level video crowd analysis methods, such as tracking or segmentation. Most of the time, an exact ground truth associated to real videos is difficult and time-consuming to produce, prone to errors, and these difficulties rise exponentially with the apparent density of the crowd in the image. We propose a synthetic crowd dataset to help researchers evaluate their methods against an objective and temporally dense synthetic ground truth. This dataset, named Agoraset, is presented in detail. The associated ground-truth and metrics are also described, together with a discussion on the use of this new kind of dataset in the field of pattern recognition. We believe this dataset to be the first bridge between simulation and pattern recognition in the field of dense crowd analysis. A discussion on the range of validity and limitations of the use of synthetic datasets in the contest of video processing is also proposed.

A Survey Paper on Trajectory Pattern Mining for Pattern Matching Query

amount information of moving objects on road network is being collected with the help of various recent technologies. The tracking of these moving objects on road networks is becoming important because of it's application in various areas. Classification has been used for classifying various kinds of data sets like graph, text documents. However, there is a lack of study on data like trajectories on road networks. Data mining techniques, especially, sequential pattern mining can be used to extract frequent spatio-temporal patterns. Again it needs to confine the length of sequential patterns to ensure high efficiency. After extracting frequent sequential patterns in trajectories, classification can be applied to classify patterns which provide useful information in applications such as city and transportation planning, road construction, design, and maintenance, marketing sector. In this paper, whole pattern matching query concept is adopted after the classification to find total traffic volume on given trajectory edge. At the same time, user can find number of vehicles moving in one as well as in both directions on that particular trajectory. Keywordssequential patterns, frequent pattern based classification, location based services, pattern matching.