Intermediate Dataset Research Articles

BwMMV-pred: a novel ensemble learning approach using blood smear images for malaria prediction

The use of machine learning in healthcare has become widespread, enhancing the capabilities of doctors and clinicians. This study introduces a novel ensemble learning approach named Blending with Meta Majority Voting (BwMMV) for malaria prediction using blood smear images. The BwMMV technique combines the strengths of eight base classifiers to form an intermediate dataset, which is subsequently used to train five distinct meta-models using different machine learning algorithms. A Local Binary Pattern Histogram (LBPH) method is employed to extract texture features from blood smear images, effectively capturing the underlying patterns necessary for classification. The final classification decision is determined through a majority voting mechanism, selecting the outcome with the most votes as the final prediction. Our results indicate that the BwMMV approach significantly outperforms traditional hard voting and blending techniques, achieving superior accuracy, robustness, and resilience in performance. This innovative method demonstrates promising potential as a powerful tool for automated diagnosis systems, with the ability to be expanded to analyze various datasets efficiently.

Unsupervised Pixel-Level Detection of Rail Surface Defects Using Multistep Domain Adaptation

Detection of rail surface defects using deep learning largely relies on supervision with pixel-level ground truths, where the detection accuracy suffers due to unseen image domains. In order to streamline the laborious labeling process and accurately identify defects on rail surfaces a U-Net-based network that integrates spatial and channel attention (SA) modules, multistep domain adaptation (MSDA), and progressive histogram matching (PHM) is developed in the context of pixel-level defects detection in this research. MSDA with an unlabeled intermediate dataset is used to reduce the domain gap underlying different domains. To further improve the adaptation of the transferred model, the PHM method and the self-training method are employed to enhance detection accuracy. Trained in the source domain with ground truths, the deep learning model is then tested in the target domain with no annotations. Experimental results on the RSDD dataset and the Rail-Joint dataset demonstrate the applicability and effectiveness of the proposed approach. By alleviating the domain discrepancy, the developed model drastically enhances the average precision (AP) score of unsupervised detection from 0.103 to 0.861, which shows competitive performance compared with the AP score of 0.895 by the state-of-the-art supervised models. The novelty of this research lies in the development of a hybrid deep learning approach that is able to achieve competitive detection accuracy with a low overlap ratio between domains and few data labeling.

Multi-label learning by extended multi-tier stacked ensemble method with label correlated feature subset augmentation

<p>Classification is one of the basic and most important operations that can be used in data science and machine learning applications. Multi-label classification is an extension of the multi-class problem where a set of class labels are associated with a particular instance at a time. In a multiclass problem, a single class label is associated with an instance at a time. However, there are many different stacked ensemble methods that have been proposed and because of the complexity associated with the multi-label problems, there is still a lot of scope for improving the prediction accuracy. In this paper, we are proposing the novel extended multi-tier stacked ensemble (EMSTE) method with label correlation<br />by feature subset selection technique and then augmenting those feature subsets while constructing the intermediate dataset for improving the prediction accuracy in the generalization phase of the stacking. The performance effect of the proposed method has been compared with existing methods and showed that our proposed method outperforms the other methods.</p>

Oil price forecasting: A hybrid GRU neural network based on decomposition–reconstruction methods

Significant fluctuations in the price of crude oil in recent years make accurate price estimations of critical importance. A reliable method for crude oil price forecasting is beneficial in guiding production and investment. This study presents two new hybrid predictors for forecasting oil prices based on recurrent neural networks using variational modal decomposition (VMD), sample entropy (SE), and gated recurrent units (GRUs). We consider the West Texas Intermediate daily closing oil prices from August 2, 2010, to December 31, 2019. The hybrid prediction methods first decompose the original time series and then reconstruct the decomposed data using SE. The reconstructed data are used to formulate independent price predictions, which are subsequently combined to produce an ensemble output. With the West Texas Intermediate dataset, the VMD-SE-GRU framework proposed in this paper gives a root mean square error of 0.6735, mean absolute error of 0.4585, mean absolute percentage error of 0.8059, and R2 value of 0.9272. Overall, the hybrid VMD-SE-GRU framework has several advantages over previous models and produces highly accurate forecasts with a shorter runtime. Thus, the new approach provides an effective tool for predicting oil prices.

Image Retrieval Scheme Using Quantized Bins of Color Image Components and Adaptive Tetrolet Transform

In this paper, a three stage hierarchical image retrieval scheme using a color, texture and shape visual contents (or descriptors) is proposed, since single visual content is not produce an adequate retrieval results effectively. This scheme has reduced the searching space during the image retrieval process at a certain extent due to the hierarchical mode. In initial stage, the shape feature descriptor has been computed by simple fusion of histograms of gradients and invariant moments of segmented image planes. The shape based retrieval process has reduced the search space by discarding the non-relevant images from the universal dataset (or original dataset) effectively and kept the retrieved images into the intermediate dataset. In the second stage, the texture feature descriptors have been computed from the intermediate sub-image dataset by applying the adaptive tetrolet transform on image plane of preprocessed HSV color image. This transform provides the multi-resolution images with finer details by employing the tetrominoes and the proper arrangement of tetrominoes contributes the effective local geometry of image plane. The gray level co-occurrence matrix based texture feature descriptor is obtained by computing second order statistical parameters from each decomposed sub-image. At this stage, the most of the irrelevant images are discarded by retrieving the images from intermediate dataset but still some undesired images are left, those will be handled at the last stage. At this stage, fused color information is captured by applying the color autocorrelogram on both the non-uniform quantized color components of the preprocessed HSV color image. Finally, the color feature descriptor produces the desired retrieval results by discarding the irrelevant images from the texture based sub-image dataset. The proposed scheme has also low computational overhead due to the use of three descriptors at different stages separately. The retrieved results show the better accuracy as compared to the other related visual contents based image retrieval schemes.

Cost-Effective Privacy Preserving of Intermediate Data using Group Search Optimization Algorithm

Cloud computing provides huge storage ability to clients, in order to convey their applications without any infrastructure investment. Alongside those applications, an extensive set of intermediate datasets will be created, and it is a challenging issue to protect the security of those intermediate datasets. Therefore, in this paper, we preserve only sensitive information which reduces the time and cost. For privacy preserving of the intermediate dataset, in this paper, we propose a combination of group search optimisation (GSO) and advanced encryption standard (AES). At first, the original dataset is split into an intermediate dataset and we select the corresponding node from the cloud for each intermediate dataset using GSO algorithm. After that, we separate the sensitive data using information gain. Finally, we apply the AES to encrypt the sensitive data. The performance of proposed methodology is in terms of encryption time and memory usage.

Tackling cycle skipping in full-waveform inversion with intermediate data

Full-waveform inversion (FWI) is a promising technique for recovering the earth models for exploration geophysics and global seismology. FWI is generally formulated as the minimization of an objective function, defined as the L2-norm of the data residuals. The nonconvex nature of this objective function is one of the main obstacles for the successful application of FWI. A key manifestation of this nonconvexity is cycle skipping, which happens if the predicted data are more than half a cycle away from the recorded data. We have developed the concept of intermediate data for tackling cycle skipping. This intermediate data set is created to sit between predicted and recorded data, and it is less than half a cycle away from the predicted data. Inverting the intermediate data rather than the cycle-skipped recorded data can then circumvent cycle skipping. We applied this concept to invert cycle-skipped first arrivals. First, we picked up the first breaks of the predicted data and the recorded data. Second, we linearly scaled down the time difference between the two first breaks of each shot into a series of time shifts, the maximum of which was less than half a cycle, for each trace in this shot. Third, we moved the predicted data with the corresponding time shifts to create the intermediate data. Finally, we inverted the intermediate data rather than the recorded data. Because the intermediate data are not cycle-skipped and contain the traveltime information of the recorded data, FWI with intermediate data updates the background velocity model in the correct direction. Thus, it produces a background velocity model accurate enough for carrying out conventional FWI to rebuild the intermediate- and short-wavelength components of the velocity model. Our numerical examples using synthetic data validate the intermediate-data concept for tackling cycle skipping and demonstrate its effectiveness for the application to first arrivals.

Privacy preserving of intermediate dataset using hybridisation of oppositional gravitational search algorithm and elliptic curve cryptography

Distributed computing gives the gigantic capacity ability to the clients to send their applications without any infrastructure investment. Based on the application lot of intermediate dataset will be created. To protecting these intermediate dataset is a challenging task. Moreover, encrypting all dataset is a time and cost consuming. To overcome the problem, in this paper we proposed a privacy preserving of intermediate dataset using a combination of oppositional gravitational search algorithm and elliptic curve cryptography (OGSA + ECC). Initially, we split the dataset into a number of the intermediate datasets, then, we choose the node corresponding intermediate dataset from the cloud using an oppositional gravitational search algorithm (OGSA). After that, we choose the sensitive data from the dataset using the information gain measure to minimise the processing time and cost. Then, using the ECC algorithm the sensitive data is encrypted and in the cloud the secure data are stored. The experimentation is carried out in terms of encryption time and memory use.

Privacy preserving of intermediate dataset using hybridisation of oppositional gravitational search algorithm and elliptic curve cryptography

Distributed computing gives the gigantic capacity ability to the clients to send their applications without any infrastructure investment. Based on the application lot of intermediate dataset will be created. To protecting these intermediate dataset is a challenging task. Moreover, encrypting all dataset is a time and cost consuming. To overcome the problem, in this paper we proposed a privacy preserving of intermediate dataset using a combination of oppositional gravitational search algorithm and elliptic curve cryptography (OGSA + ECC). Initially, we split the dataset into a number of the intermediate datasets, then, we choose the node corresponding intermediate dataset from the cloud using an oppositional gravitational search algorithm (OGSA). After that, we choose the sensitive data from the dataset using the information gain measure to minimise the processing time and cost. Then, using the ECC algorithm the sensitive data is encrypted and in the cloud the secure data are stored. The experimentation is carried out in terms of encryption time and memory use.

Oil price prediction using a supervised neural network

Many attempts have been made at designing efficient prediction systems for oil price due to the impact of oil on the world's economy. In this paper, we present an efficient oil price prediction system based on using a supervised neural network. The neural model receives at its input novel key economic and seasonal indicators, which we extract from the West Texas Intermediate (WTI) dataset of crude oil prices over 24 years. The model predicts at its output weekly oil prices within five dollars/barrel accuracy. The experimental results reveal that the obtained correct prediction rate of 88% is higher than rates reported in other related works, thus indicating that neural networks can be effectively used for predicting oil prices. [Received: August 3, 2016; Accepted: February 17, 2017]

A New Method of Oppositional Gravitational Search Algorithm Using in Cloud Computing

Cloud computing is that the extended unreal vision of hard as a utility, wherever cloud users will distantly store their info into the cloud thus on relish the on-demand nice quality implementations and facilities from a communal pool of configurable hard sources. Cloud Computing develops predominant, additional and additional sensitive information square measure being central into the cloud. These work emphases on giving a dependable and safe cloud information sharing service that allows customers dynamic access to their info. For attaining this, during this article, we have a tendency to progress the privacy protective intermediate dataset on the idea of the Oppositional gravitative Search algorithmic program and Elliptic Curve Cryptography encoding technique customary (OGSA+ECC). within the preliminary part, the intermediate dataset is made and OGSA algorithmic program is enforced for ruling the optimum node assortment from the cloud. later, we have a tendency to work out the sensitive info on the idea of the information gain. Lastly, we have a tendency to smear the code for sensitive info encoding and investigation is performed by the datasets. The investigation is achieved with the assistance of the knowledge sets to guage the performance within the name of encoding time and memory use. From the investigation, it are often complete that the encoding time and memory usage cut back with upsurge in threshold

Improved algorithms for intermediate dataset storage in a cloud-based dataflow

In order to run a dataflow with as low cost as possible, it is often faced with deciding which data-sets in a data-set sequence should be stored, with the rest regenerated. The Intermediate Data-set Storage problem arises from this situation. The current best algorithm for this problem takes O(n4) time. In this paper, we present two improved algorithms for this problem, the first of which can achieve a time complexity O(n2), the second of which O(rn), where n is the number of data-sets in a dataflow, r is a numerical number which indicates how large it is for the maximum storage cost to be divided by the minimum computation cost in the dataflow.

Risk-aware intermediate dataset backup strategy in cloud-based data intensive workflows

Data-intensive workflows are generally computing- and data-intensive with large volume of data generated during their execution. Therefore, some of the data should be saved to avoid the expensive re-execution of tasks in case of exceptions. However, cloud-based data storage services come at some expense. In this paper, we introduce the risk evaluation model tailored for workflow structure to measure and achieve the trade-off between the overhead of backup storage and the cost of data regeneration in failure, making the service selection and execution more efficient and robust. The proposed method computes and compares the potential loss with and without data backup to achieve the trade-off between overhead of intermediate dataset backup and task re-execution after exceptions. We also design the utility function with the model and apply a genetic algorithm to find the optimized schedule. The results show that the robustness of the schedule is increased while the possible risk of failure is minimized, especially when the volume of generated data is not large in comparison with the input.

Weighted registration of 123I‐FP‐CIT SPECT images improves accuracy of binding potential estimates in pathologically low striatal uptake

To test whether the use of a striatum weighted image may improve registration accuracy and diagnostic outcome in patients with parkinsonian syndromes (PS), weighted images were generated by increasing signal intensity of striatal voxels and used as intermediate dataset for co-registering the brain image onto template. Experimental validation was performed using an anthropomorphic striatal phantom. (123)I-FP-CIT SPECT binding ratios were manually determined in 67 PS subjects and compared to those obtained using unsupervised standard (UWR) and weighted registered (WR) approach. Normalized cost function was used to evaluate the accuracy of phantom and subjects registered images to the template. Reproducibility between unsupervised and manual ratios was assessed by using intra-class correlation coefficient (ICC) and Bland and Altman analysis. Correlation coefficient was used to assess the dependence of semi-quantitative ratios on clinical findings. Weighted method improves accuracy of brain registration onto template as determined by cost function in phantom (0.86 ± 0.06 vs. 0.98 ± 0.02; Student's t-test, P = 0.04) and in subject scans (0.69 ± 0.06 vs. 0.53 ± 0.06; Student's t-test, P < 0.0001). Agreement between manual and unsupervised derived binding ratios as measured by ICC was significantly higher on WR as compared to UWR images (0.91 vs. 0.76). Motor UPDRS score was significantly correlated with manual and unsupervised derived binding potential. In phantom, as well as in subjects studies, correlations were more significant using the WR method (BPm: R(2) = 0.36, P = 0.0001; BPwr: R(2) = 0.368, P = 0.0001; BPuwr: R(2) = 0.300, P = 0.0008). Weighted registration improves accuracy of binding potential estimates and may be a promising approach to enhance the diagnostic outcome of SPECT imaging, correlation with disease severity, and for monitoring disease progression in Parkinsonian syndromes.

A Novel RFID Data Mining System: Integration of Effective Sequential Pattern Mining and Fuzzy Rules Generation Techniques

Data warehousing and Data mining find enormous applications; RFID technology is one among them. A RFID data warehousing system with novel data cleaning, transformation and loading technique has been proposed in the previous work. The system has been dedicatedly implemented in one of the significant RFID applications tracking of goods in warehouses. The warehoused RFID data is in specific format and so an effective mining system is required to mine the needed information from the database. The existing mining algorithms are inefficient in extracting the information from the warehoused RFID data. In this paper, a novel data mining system is proposed, which effectively extracts the information regarding the nature of movement of the RFID tags. The proposed mining system generates an intermediate dataset (I-dataset) from the warehoused dataset. From the I-dataset, sequential patterns are mined with different pattern length combinations. From the mined sequential patterns, fuzzy rules are generated, which depicts the nature of movement of the RFID tags. The implementation results show that the proposed mining system performs well by extracting the significant RFID tags and its combinations and the nature of movement of the tags.

On-demand minimum cost benchmarking for intermediate dataset storage in scientific cloud workflow systems

Many scientific workflows are data intensive: large volumes of intermediate datasets are generated during their execution. Some valuable intermediate datasets need to be stored for sharing or reuse. Traditionally, they are selectively stored according to the system storage capacity, determined manually. As doing science on clouds has become popular nowadays, more intermediate datasets in scientific cloud workflows can be stored by different storage strategies based on a pay-as-you-go model. In this paper, we build an intermediate data dependency graph (IDG) from the data provenances in scientific workflows. With the IDG, deleted intermediate datasets can be regenerated, and as such we develop a novel algorithm that can find a minimum cost storage strategy for the intermediate datasets in scientific cloud workflow systems. The strategy achieves the best trade-off of computation cost and storage cost by automatically storing the most appropriate intermediate datasets in the cloud storage. This strategy can be utilised on demand as a minimum cost benchmark for all other intermediate dataset storage strategies in the cloud. We utilise Amazon clouds’ cost model and apply the algorithm to general random as well as specific astrophysics pulsar searching scientific workflows for evaluation. The results show that benchmarking effectively demonstrates the cost effectiveness over other representative storage strategies.

A haplotype inference algorithm for trios based on deterministic sampling

BackgroundIn genome-wide association studies, thousands of individuals are genotyped in hundreds of thousands of single nucleotide polymorphisms (SNPs). Statistical power can be increased when haplotypes, rather than three-valued genotypes, are used in analysis, so the problem of haplotype phase inference (phasing) is particularly relevant. Several phasing algorithms have been developed for data from unrelated individuals, based on different models, some of which have been extended to father-mother-child "trio" data.ResultsWe introduce a technique for phasing trio datasets using a tree-based deterministic sampling scheme. We have compared our method with publicly available algorithms PHASE v2.1, BEAGLE v3.0.2 and 2SNP v1.7 on datasets of varying number of markers and trios. We have found that the computational complexity of PHASE makes it prohibitive for routine use; on the other hand 2SNP, though the fastest method for small datasets, was significantly inaccurate. We have shown that our method outperforms BEAGLE in terms of speed and accuracy for small to intermediate dataset sizes in terms of number of trios for all marker sizes examined. Our method is implemented in the "Tree-Based Deterministic Sampling" (TDS) package, available for download at http://www.ee.columbia.edu/~anastas/tdsConclusionsUsing a Tree-Based Deterministic sampling technique, we present an intuitive and conceptually simple phasing algorithm for trio data. The trade off between speed and accuracy achieved by our algorithm makes it a strong candidate for routine use on trio datasets.

An assessment of surface matching for the automated co-registration of MOLA, HRSC and HiRISE DTMs

Martian topographic data has been collected by various exploration missions over the last decade. These products provide detailed topographic information and are invaluable for scientists to interpret and understand the geological and climate evolution which has occurred on Mars. In order to fully utilise these multi-sensor, multi-resolution and multi-scale Martian topographic products, a co-registration process has been developed which allows co-registration of Digital Terrain Models (DTMs) to be performed to co-align these multiple datasets. Surface matching is the core technique to implement this task and it is here assessed to determine the parameters of the most robust algorithm for DTM co-registration. Once this task was finished, the matching tool was developed accordingly with a decision algorithm. This algorithm was then employed to align DTMs derived from Mars Orbiter Laser Altimeter (MOLA), High Resolution Stereo Camera (HRSC) and High Resolution Imaging Science Experiment (HiRISE). For MOLA and HRSC DTMs, the co-registration was performed directly as the MOLA DTM acted as a reference surface within a bundle adjustment process. DTMs from different versions covering three HRSC orbital strips were used for the assessment process. As a result the mean bias of the height differences of a preliminary version HRSC DTM was significantly reduced from 38.596 m to 2.233 m, when compared against MOLA while the bias of a newer DTM was improved from 1.616 m to 0.161 m after matching. Regarding the co-registration of HiRISE and MOLA DTMs, a hierarchical approach employing a HRSC DTM as an intermediate dataset was assessed. The results demonstrated that the method is feasible and that the three DTMs were co-registered effectively. Due to the success highlighted in this paper, a surface matching tool is recommended to be applied to DTMs derived from multiple sources before these data are further used. Moreover, surface matching can be considered as an additional step of any workflow for Mars DTM creation.

Crustal Thickness Variations across the Blue Ridge Mountains, Southern Appalachians: An Alternative Procedure for Migrating Wide-Angle Reflection Data

Abstract Migration of wide-angle reflections generated by quarry blasts suggests that crustal thickness increases from 38 km beneath the Carolina Terrane to 47–51 km along the southeastern flank of the Blue Ridge. The migration algorithm, developed for generating single-fold images from explosions and earthquakes recorded with isolated, short-aperture arrays, uses the localized slant stack as an intermediate data set. In contrast with other methods, it includes an interpretive step that is based on the assumption that all coherent P -wave energy consists of reflections from planar interfaces. Each sample in the slant stack is mapped into a planar, dipping segment with a length that is determined by the recording aperture. Migrated sections from within the Blue Ridge show increases in reflectivity at depths of 20 and 40 km. High apparent reflectivity from 40 to 50–55 km suggests a layered zone in the lower crust that is similar to models proposed for the Cumberland Plateau in Tennessee and the Adirondacks. The migration results are consistent with regional gravity data and with the occurrence of crustal roots beneath the Urals, another Paleozoic orogen.

Seismic imaging of deep crust

We introduce here an integral two‐dimensional (2-D) scheme for the processing of deep crustal reflection profiles. This approach, in which migration occurs before stacking, is tailored to the unique character of the data in which nonvertically propagating energy is as important as vertically propagating energy. Since reflector depths range beyond 30 km, the horizontal displacement of reflections which occurs in migration can be as large as reflector depths; under these circumstances, the common‐midpoint (CMP) stack is inadequate. In our scheme, each common‐source trace gather is transformed into a set of traces (beams) corresponding to set of different incidence angles. A correction for wavefront curvature similar to the normal moveout (NMO) correction yields traces (focused beams) which are focused at image points along the direction of arrival. While the method is equivalent to the Kirchhoff integral migration method, and therefore to any complete continuation method, it gives rise to an intermediate data set which is characterized by the direction of arrival of the upward propagating energy. By a geometrical transformation of the beams and summation, we may synthesize images composed of a specified range of Fourier spatial components. Geologic examples suggest that complex structures in the basement may be most easily characterized by their local direction of layering, a quantity we may determine by this approach. Noise‐free synthetic data examples illustrate the limits of horizontal and vertical resolving power at mid‐crustal depths for any imaging method. Velocity determination is difficult at these depths due to the small NMO and may be possible only by evaluating the effects of velocity models on the imaged data. Examples of the imaged section from the COCORP test profile in Hardeman County, Texas, show a combination of horizontally continuous reflectors and an irregular pattern of scatterers with locally horizontal layering.