Deep Artificial Neural Networks Research Articles

Stable recovery of entangled weights: Towards robust identification of deep neural networks from minimal samples

In this paper we approach the problem of unique and stable identifiability from a finite number of input-output samples of generic feedforward deep artificial neural networks of prescribed architecture with pyramidal shape up to the penultimate layer and smooth activation functions. More specifically we introduce the so-called entangled weights, which compose weights of successive layers intertwined with suitable diagonal and invertible matrices depending on the activation functions and their shifts. We prove that instances of entangled weights are completely and stably approximated by an efficient and robust algorithm as soon as O(D2×m) nonadaptive input-output samples of the network are collected, where D is the input dimension and m is the number of neurons of the network. Moreover, we empirically observe that the approach applies to networks with up to O(D×mL) neurons, where mL is the number of output neurons at layer L. Provided knowledge of layer assignments of entangled weights and of remaining scaling and shift parameters, which may be further heuristically obtained by least squares fitting, the entangled weights identify the network completely and uniquely. To highlight the relevance of the theoretical result of stable recovery of entangled weights, we present numerical experiments, which demonstrate that multilayered networks with generic weights can be robustly identified and therefore uniformly approximated by the presented algorithmic pipeline.

Let's move forward: Image-computable models and a common model evaluation scheme are prerequisites for a scientific understanding of human vision.

In the target article, Bowers et al. dispute deep artificial neural network (ANN) models as the currently leading models of human vision without producing alternatives. They eschew the use of public benchmarking platforms to compare vision models with the brain and behavior, and they advocate for a fragmented, phenomenon-specific modeling approach. These are unconstructive to scientific progress. We outline how the Brain-Score community is moving forward to add new model-to-human comparisons to its community-transparent suite of benchmarks.

Artificial Intelligence, Machine Learning, and Intelligent Decision Support Systems: Iterative “Learning” SQGbased procedures for Distributed Models’ Linkage

In this paper we discuss the on-going joint work contributing to the IIASA (International Institute for Applied Systems Analysis, Laxenburg, Austria) and National Academy of Science of Ukraine projects on “Modeling and management of dynamic stochastic interdependent systems for food-water-energy-health security nexus” (see [1-2] and references therein). The project develops methodological and modeling tools aiming to create Intelligent multimodel Decision Support System (IDSS) and Platform (IDSP), which can integrate national Food, Water, Energy, Social models with the models operating at the global scale (e.g., IIASA GLOBIOM and MESSAGE), in some cases ‘downscaling’ the results of the latter to a national level. Data harmonization procedures rely on new type non-smooth stochastic optimization and stochastic quasigradient (SQG) [3-4] methods for robust of-line and on-line decisions involving large-scale machine learning and Artificial Intelligence (AI) problems in particular, Deep Learning (DL) including deep neural learning or deep artificial neural network (ANN). Among the methodological aims of the project is the development of “Models’ Linkage” algorithms which are in the core of the IDSS as they enable distributed models’ linkage and data integration into one system on a platform [5-8]. The linkage algorithms solve the problem of linking distributed models, e.g., sectorial and/or regional, into an inter-sectorial inter-regional integrated models. The linkage problem can be viewed as a general endogenous reinforced learning problem of how software agents (models) take decisions in order to maximize the “cumulative reward". Based on novel ideas of systems’ linkage under asymmetric information and other uncertainties, nested strategic-operational and local-global models are being developed and used in combination with, in general, non-Bayesian probabilistic downscaling procedures. In this paper we illustrate the importance of the iterative “learning” solution algorithms based on stochastic quasigradient (SQG) procedures for robust of-line and on-line decisions involving large-scale Machine Learning, Big Data analysis, Distributed Models Linkage, and robust decision-making problems. Advanced robust statistical analysis and machine learning models of, in general, nonstationary stochastic optimization allow to account for potential distributional shifts, heavy tails, and nonstationarities in data streams that can mislead traditional statistical and machine learning models, in particular, deep neural learning or deep artificial neural network (ANN). Proposed models and methods rely on probabilistic and non-probabilistic (explicitly given or simulated) distributions combining measures of chances, experts’ beliefs and similarity measures (for example, compressed form of the kernel estimators). For highly nonconvex models such as the deep ANN network, the SQGs allow to avoid local solutions. In cases of nonstationary data, the SQGs allow for sequential revisions and adaptation of parameters to the changing environment, possibly, based on of-line adaptive simulations. The non-smooth STO approaches and SQG-based iterative solution procedures are illustrated with examples of robust estimation, models’ linkage, machine learning, adaptive Monte Carlo optimization for cat risks (floods, earthquakes, etc.) modeling and management

Deep learning-based ensemble modeling of Vibrio parahaemolyticus concentration in marine environment

Vibrio parahaemolyticus(V.p) is a marine pathogenic bacterium that poses a high risk to human health and shellfish industry, yet an effective regional-scale nowcasting model for managing the risk remains lacking. This study presents the first regional-scale model for nowcasting the level of V.p in oysters in the marine environment by developing an ensemble modeling approach. The ensemble modeling approach involves the integration of genetic programming (GP) and deep artificial neural networks (DNN)-based modeling. The new approach was demonstrated by developing three GP-DNN ensemble models for predicting the V.p level in North Carolina, New Hampshire, and the combined region. Specifically, GP was employed to establish nonlinear functions between the V.p level and antecedent conditions of environmental variables. The nonlinear GP functions and current conditions of individual environmental variables were then utilized as inputs into a DNN model, forming a GP-DNN ensemble model. Modeling results indicated that the GP-DNN ensemble models were capable of predicting the V.p level with the correlation coefficient of 0.91, 0.90, and 0.80 for North Carolina, New Hampshire, and the combined region, respectively, demonstrating the impact of distinct environmental conditions in the local areas on accuracy of the combined regional-scale model. Sensitivity analysis results showed that sea surface temperature and sea surface salinity are the two most important environmental predictors for the abundance of V.p in oysters, followed by water level, pH, chlorophyll-a, and turbidity. The findings suggested that the GP-DNN ensemble models could be utilized as effective predictive tools for mitigating the V.p risk.

On the similarities of representations in artificial and brain neural networks for speech recognition.

In recent years, machines powered by deep learning have achieved near-human levels of performance in speech recognition. The fields of artificial intelligence and cognitive neuroscience have finally reached a similar level of performance, despite their huge differences in implementation, and so deep learning models can-in principle-serve as candidates for mechanistic models of the human auditory system. Utilizing high-performance automatic speech recognition systems, and advanced non-invasive human neuroimaging technology such as magnetoencephalography and multivariate pattern-information analysis, the current study aimed to relate machine-learned representations of speech to recorded human brain representations of the same speech. In one direction, we found a quasi-hierarchical functional organization in human auditory cortex qualitatively matched with the hidden layers of deep artificial neural networks trained as part of an automatic speech recognizer. In the reverse direction, we modified the hidden layer organization of the artificial neural network based on neural activation patterns in human brains. The result was a substantial improvement in word recognition accuracy and learned speech representations. We have demonstrated that artificial and brain neural networks can be mutually informative in the domain of speech recognition.

Exploring robust architectures for deep artificial neural networks

The architectures of deep artificial neural networks (DANNs) are routinely studied to improve their predictive performance. However, the relationship between the architecture of a DANN and its robustness to noise and adversarial attacks is less explored, especially in computer vision applications. Here we investigate the relationship between the robustness of DANNs in a vision task and their underlying graph architectures or structures. First we explored the design space of architectures of DANNs using graph-theoretic robustness measures and transformed the graphs to DANN architectures using various image classification tasks. Then we explored the relationship between the robustness of trained DANNs against noise and adversarial attacks and their underlying architectures. We show that robustness performance of DANNs can be quantified before training using graph structural properties such as topological entropy and Olivier-Ricci curvature, with the greatest reliability for complex tasks and large DANNs. Our results can also be applied for tasks other than computer vision such as natural language processing and recommender systems.

Deep Reinforcement Learning for Flow Control Exploits Different Physics for Increasing Reynolds Number Regimes

The increase in emissions associated with aviation requires deeper research into novel sensing and flow-control strategies to obtain improved aerodynamic performances. In this context, data-driven methods are suitable for exploring new approaches to control the flow and develop more efficient strategies. Deep artificial neural networks (ANNs) used together with reinforcement learning, i.e., deep reinforcement learning (DRL), are receiving more attention due to their capabilities of controlling complex problems in multiple areas. In particular, these techniques have been recently used to solve problems related to flow control. In this work, an ANN trained through a DRL agent, coupled with the numerical solver Alya, is used to perform active flow control. The Tensorforce library was used to apply DRL to the simulated flow. Two-dimensional simulations of the flow around a cylinder were conducted and an active control based on two jets located on the walls of the cylinder was considered. By gathering information from the flow surrounding the cylinder, the ANN agent is able to learn through proximal policy optimization (PPO) effective control strategies for the jets, leading to a significant drag reduction. Furthermore, the agent needs to account for the coupled effects of the friction- and pressure-drag components, as well as the interaction between the two boundary layers on both sides of the cylinder and the wake. In the present work, a Reynolds number range beyond those previously considered was studied and compared with results obtained using classical flow-control methods. Significantly different forms of nature in the control strategies were identified by the DRL as the Reynolds number Re increased. On the one hand, for Re≤1000, the classical control strategy based on an opposition control relative to the wake oscillation was obtained. On the other hand, for Re=2000, the new strategy consisted of energization of the boundary layers and the separation area, which modulated the flow separation and reduced the drag in a fashion similar to that of the drag crisis, through a high-frequency actuation. A cross-application of agents was performed for a flow at Re=2000, obtaining similar results in terms of the drag reduction with the agents trained at Re=1000 and 2000. The fact that two different strategies yielded the same performance made us question whether this Reynolds number regime (Re=2000) belongs to a transition towards a nature-different flow, which would only admits a high-frequency actuation strategy to obtain the drag reduction. At the same time, this finding allows for the application of ANNs trained at lower Reynolds numbers, but are comparable in nature, saving computational resources.

Respiratory motion prediction based on deep artificial neural networks in CyberKnife system: A comparative study.

In external beam radiotherapy, a prediction model is required to compensate for the temporal system latency that affects the accuracy of radiation dose delivery. This study focused on a thorough comparison of seven deep artificial neural networks to propose an accurate and reliable prediction model. Seven deep predictor models are trained and tested with 800 breathing signals. In this regard, a nonsequential-correlated hyperparameter optimization algorithm is developed to find the best configuration of parameters for all models. The root mean square error (RMSE), mean absolute error, normalized RMSE, and statistical F-test are also used to evaluate network performance. Overall, tuning the hyperparameters results in a 25%-30% improvement for all models compared to previous studies. The comparison between all models also shows that the gated recurrent unit (GRU) with RMSE=0.108±0.068mm predicts respiratory signals with higher accuracy and better performance. Overall, tuning the hyperparameters in the GRU model demonstrates a better result than the hybrid predictor model used in the CyberKnife VSI system to compensate for the 115ms system latency. Additionally, it is demonstrated that the tuned parameters have a significant impact on the prediction accuracy of each model.

Deep3DCANN: A Deep 3DCNN-ANN framework for spontaneous micro-expression recognition

Facial micro-expressions play a significant role in revealing concealed emotions. However, the recognition of micro-expressions is challenging due to their fleeting nature. Moreover, the visual features of the face and the visual relationships between the facial sub-regions have a strong influence on the presence of micro-expressions. In this work, a novel end-to-end facial micro-expression detection framework, called Deep3DCANN, is proposed to integrate these components for effective micro-expression detection. The first component of our framework is a deep 3D convolutional neural network that learns useful spatiotemporal features from a sequence of facial images. In the second component, a deep artificial neural network is utilized to trace the useful visual associations between different sub-regions of the face. Furthermore, a carefully crafted fusion mechanism is built to combine the learned facial features and the semantic relationships between the regions to predict the micro-expressions. We also construct a new loss function to jointly optimize both modules of our proposed architecture. Our proposed method performs favourably on five benchmark spontaneous micro-expression databases compared to existing micro-expression recognition baselines on videos. In addition, through an extended experiment, we show that our proposed approach can effectively recognize the frame-wise micro-expression changes in a sequence of video frames.

DEVELOPMENT OF SOFT COMPUTATIONAL SIMULATOR FOR OPTIMIZED DEEP ARTIFICIAL NEURAL NETWORKS FOR GEOMATICS APPLICATIONS: REMOTE SENSING CLASSIFICATION AS AN APPLICATION

Artificial neural networks (ANN) obtain more importance after the innovation of deep learning (DL) approach. This research is oriented towards development of soft computational simulator for geomatics research using ANN supporting the deep approach. ANN seems to be a black box due to its sensitivity towards initialization, architecture, and behavior. This research gives a spotlight on the dull areas of ANN algorithm by developing a soft computational simulator for it. The applied examples are chosen to cover geomatics data. DANNDO (Deep Artificial Neural Networks Designer and Optimizer) software is developed to achieve the research objective. Multi-layer perceptron (MLP) architecture is applied in this simulator. Geomatics (remote sensing multi- spectral data) is selected to be a testing paradigm to insure the reliability of the developed simulator. The developed simulator proved the high performance of applying both shallow and deep ANN (DANN).

Technical note: Phantom-based training framework for convolutional neural network CT noise reduction.

Deep artificial neural networks such as convolutional neural networks (CNNs) have been shown to be effective models for reducing noise in CT images while preserving anatomic details. A practical bottleneck for developing CNN-based denoising models is the procurement of training data consisting of paired examples of high-noise and low-noise CT images. Obtaining these paired data are not practical in a clinical setting where the raw projection data is not available. This work outlines a technique to optimize CNN denoising models using methods that are available in a routine clinical setting. To demonstrate a phantom-based training framework for CNN noise reduction that can be efficiently implemented on any CT scanner. The phantom-based training framework uses supervised learning in which training data are synthesized using an image-based noise insertion technique. Ten patient image series were used for training and validation (9:1) and noise-only images obtained from anthropomorphic phantom scans. Phantom noise-only images were superimposed on patient images to imitate low-dose CT images for use in training. A modified U-Net architecture was used with mean-squared-error and feature reconstruction loss. The training framework was tested for clinically indicated whole-body-low-dose CT images, as well as routine abdomen-pelvis exams for which projection data was unavailable. Performance was assessed based on root-mean-square error, structural similarity, line profiles, and visual assessment. When the CNN was tested on five reserved quarter-dose whole-body-low-dose CT images, noise was reduced by 75%, root-mean-square-error reduced by 34%, and structural similarity increased by 60%. Visual analysis and line profiles indicated that the method significantly reduced noise while maintaining spatial resolution of anatomic features. The proposed phantom-based training framework demonstrated strong noise reduction while preserving spatial detail. Because this method is based within the image domain, it can be easily implemented without access to projection data.

Predicting Effluent Quality in Full-Scale Wastewater Treatment Plants Using Shallow and Deep Artificial Neural Networks

This research focuses on applying artificial neural networks with nonlinear transformation (ANNs) models to predict the performance of wastewater treatment plant (WWTP) processes. The paper presents a novel machine learning (ML)-based approach for predicting effluent quality in WWTPs through explaining the relationships between the multiple influent and effluent pollution variables of an existing WWTP. We developed AI models such as feed-forward neural network (FFNN) and random forest (RF) as well as deep learning methods such as convolutional neural network (CNN), recurrent neural network (RNN), and pre-train stacked auto-encoder (SAE) in order to avoid various shortcomings of conventional mechanistic models. The developed models focus on providing an adaptive, functional, and alternative methodology for modeling the performance of the WWTP. They are based on pollution data collected over three years. It includes chemical oxygen demand (COD), biochemical oxygen demand (BOD5), phosphates (PO₄−3), and nitrates (NO₃−), as well as auxiliary indicators including the temperature (T), degree of acidity or alkalinity (pH), electric conductivity (EC), and the total dissolved solids (TDS). The paper presents the results of using SNN- and DNN-based models to predict the effluent concentrations. Our results show that SNN can predict plant performance with a correlation coefficient (R) up to 88%, 90%, 93%, and 96% for the single models COD, BOD5, NO₃−, and PO₄−3, respectively, and up to 88%, 96%, and 93% for the ensemble models (BOD5 and COD), (PO₄−3 and NO₃−), and (COD, BOD5, NO₃−, PO₄−3), respectively. The results also show that the two-hidden-layers model outperforms the one-hidden-layer model (SNN). Moreover, increasing the input parameters improves the performance of models with one and two hidden layers. We applied DNN (CNN, RNN, SAE) with three, four, and five hidden layers for WWTP modeling, but due to the small datasets, it gave a low performance and accuracy. In sum, this paper shows that SNN (one and two hidden layers) and the random forest (RF) machine learning technique provide effective modeling of the WWTP process and could be used in the WWTP management.

In-Depth Case Study on Artificial Neural Network Weights Optimization Using Meta-Heuristic and Heuristic Algorithmic Approach

The Meta-heuristic and Heuristic algorithms that have been introduced for deep neural network optimization is in this paper. Artificial Intelligence, and also the most used Deep Learning methods are all growing in popularity these days, thus we need faster optimization strategies for finding the results of future activities. Neural Network Optimization with Particle Swarm Optimization, Backpropagation (BP), Resilient Propagation (Rprop), and Genetic Algorithm (GA) is used for numerical analysis of different datasets and comparing each other to find out which algorithms work better for finding optimal solutions by reducing training loss. Genetic algorithm and also bio-inspired Particle Swarm Optimization is introduced in this paper. Besides, Resilient Propagation and Conventional Backpropagation algorithms which are application-specific algorithms have also been introduced. Meta-heuristic algorithms GA and PSO are a higher-level formula and problem-independent technique that may be used to a diverse number of challenges. The characteristic of Heuristic algorithms has extremely specific features that vary depending on the problem. The conventional Backpropagation (BP) based optimization, the Particle Swarm Optimization methodology, and Resilient Propagation (Rprop) are all fully presented, and how to apply these procedures in Artificial Deep Neural networks Optimization is also thoroughly described. Applied numerical simulation over several datasets proves that the Meta-heuristic algorithm Particle Swarm Optimization and also Genetic Algorithm performs better than the conventional heuristic algorithm like Backpropagation and Resilient Propagation.

A Pilot Machine Learning Study Using Trauma Admission Data to Identify Risk for High Length of Stay.

Trauma patients have diverse resource needs due to variable mechanisms and injury patterns. The aim of this study was to build a tool that uses only data available at time of admission to predict prolonged hospital length of stay (LOS). Data was collected from the trauma registry at an urban level one adult trauma center and included patients from 1/1/2014 to 3/31/2019. Trauma patients with one or fewer days LOS were excluded. Single layer and deep artificial neural networks were trained to identify patients in the top quartile of LOS and optimized on area under the receiver operator characteristic curve (AUROC). The predictive performance of the model was assessed on a separate test set using binary classification measures of accuracy, precision, and error. 2953 admitted trauma patients with more than one-day LOS were included in this study. They were 70% male, 60% white, and averaged 47 years-old (SD: 21). 28% were penetrating trauma. Median length of stay was 5days (IQR 3-9). For prediction of prolonged LOS, the deep neural network achieved an AUROC of 0.80 (95% CI: 0.786-0.814) specificity was 0.95, sensitivity was 0.32, with an overall accuracy of 0.79. Machine learning can predict, with excellent specificity, trauma patients who will have prolonged length of stay with only physiologic and demographic data available at the time of admission. These patients may benefit from additional resources with respect to disposition planning at the time of admission.

Reducing smearing effect in contribution plots and improving fault detection via polynomial approximated isomap embeddings

AbstractDimension reduction is an essential method used in multivariate statistical process monitoring for fault detection and diagnosis. Principal component analysis (PCA) and independent component analysis (ICA) are the most frequently used linear dimensional reduction tools, and the contribution plot is the most popular fault isolation method in the absence of any prior information on the faults. These methods, however, come with their shortcomings. The fault detection capability of linear methods may not be sufficient for non‐linear processes, and smearing effect is known to deteriorate the diagnostics obtained from contribution plots. While the fault detection rate may be increased by kernelized methods or deep artificial neural network models, tuning data‐dependent hyperparameter(s) and network structure with limited historical data is not an easy task. Furthermore, the resulting non‐linear models often do not directly possess fault isolation capability. In the current study, we aim to devise a novel method named ICApIso‐PCA, which offers non‐linear fault detection and isolation in a rather straightforward manner. The rationale of ICApIso‐PCA mainly involves building a non‐linear scores matrix, composed of principal component scores and high‐order polynomial approximated isomap embeddings, followed by implementation of the ICA‐PCA algorithm on this matrix. Applications on a toy dataset and the Tennessee Eastman plant show that the I2 index from ICApIso‐PCA yields a high fault detection rate and offers accurate contribution plots with diminished smearing effects compared to those from traditional monitoring methods. Easy implementation and the potential for future research are further advantages of the proposed method.

Post-hazard condition assessment of nuclear piping-equipment systems: Novel approach to feature extraction and deep learning

Over the past decade, the use of artificial intelligence techniques in the field of health-monitoring has gained significant interest, especially for structures such as building and bridges. However, applications to industrial systems such as equipment-piping systems in nuclear plants have not been explored. In this paper, it is shown that the existing techniques developed for buildings and bridges cannot be extended directly to equipment-piping systems as the response of such systems is governed by multiple localized modes unlike that in buildings and bridges. This paper proposes a new approach that consists of three key aspects: (i) a novel vector of degradation-sensitive features extracted from measured data, (ii) using a deep Artificial Neural Network (ANN) for diagnosis of degradation location and degradation severity, and (iii) consideration of uncertainty in degradation severity when training the ANN. Degradation in piping-equipment systems can occur due to flow-accelerated erosion and corrosion. These locations can potentially exhibit damage such as localized yielding or initiation of cracking due to an external event such as an earthquake. Moreover, such locations can at times go undetected by current inspection techniques. Therefore, a robust framework is needed for detection of degradation after a seismic event. This manuscript proposes a proof-of-concept framework, which utilizes data collected from sensors to generate a deep ANN database for predicting degraded locations and severity in a piping-equipment system. Degradation severity is classified as minor, moderate, and severe. In the suggested methodology, a novel vector of degradation-sensitive features is extracted from the sensor data to train the ANN. A simple piping-equipment system is selected to demonstrate feature extraction as a means to simplify pattern recognition, explore the design and parameters of an ANN, and develop a sensor placement strategy. The effectiveness of the proposed framework is demonstrated on a realistic primary safety system of a two-loop nuclear reactor. It is shown that the proposed post-hazard condition assessment framework is able to detect degraded locations along with the severity levels, including minor degradation, with considerably higher accuracy.

Convolutional neural network modeling and response surface analysis of compressible flow at sonic and supersonic Mach numbers

Ribs as passive control in suddenly expanded flow is exciting to enhance the base pressure around nozzle exit. Furthermore, predictions of compressible flow characteristics in duct expansion using machine learning are very rare. In this work, the evolution of compressible flow through a nozzle regulated by semi-circular rib passive control is analyzed experimentally using the data acquisition technique; to control the base pressure for sonic and four supersonic Mach numbers. Results shows nozzles flowing under favorable pressure becomes effective and significantly increases the base pressure. Artificial neural networks (ANN), deep ANN (DNN), convolutional NN (CNN), and deep CNN (DCNN) were used for the modeling of compressible flow data that are highly sensitive and non-linear. For the first time, CNN used to model high-speed data predicted accurately the flow characteristics. The base pressure continuously decreases, irrespective of Mach number and Nozzle Pressure Ratio, even when the flow is under-expanded due to a higher area ratio. DNN was found to be most effective, with R-squared above 0.9 for the base pressure and pressure loss, while for wall pressure, it is above 0.8. The accuracy of base pressure predictions is between 84% and 99% during the training and testing of all NN models.

Unsupervised learning of aging principles from longitudinal data

Age is the leading risk factor for prevalent diseases and death. However, the relation between age-related physiological changes and lifespan is poorly understood. We combined analytical and machine learning tools to describe the aging process in large sets of longitudinal measurements. Assuming that aging results from a dynamic instability of the organism state, we designed a deep artificial neural network, including auto-encoder and auto-regression (AR) components. The AR model tied the dynamics of physiological state with the stochastic evolution of a single variable, the “dynamic frailty indicator” (dFI). In a subset of blood tests from the Mouse Phenome Database, dFI increased exponentially and predicted the remaining lifespan. The observation of the limiting dFI was consistent with the late-life mortality deceleration. dFI changed along with hallmarks of aging, including frailty index, molecular markers of inflammation, senescent cell accumulation, and responded to life-shortening (high-fat diet) and life-extending (rapamycin) treatments.

Modeling Bitcoin Prices using Signal Processing Methods, Bayesian Optimization, and Deep Neural Networks.

Bitcoin is a volatile financial asset that runs on a decentralized peer-to-peer Blockchain network. Investors need accurate price forecasts to minimize losses and maximize profits. Extreme volatility, speculative nature, and dependence on intrinsic and external factors make Bitcoin price forecast challenging. This research proposes a reliable forecasting framework by reducing the inherent noise in Bitcoin time series and by examining the predictive power of three distinct types of predictors, namely fundamental indicators, technical indicators, and univariate lagged prices. We begin with a three-step hybrid feature selection procedure to identify the variables with the highest predictive ability, then use Hampel and Savitzky-Golay filters to impute outliers and remove signal noise from the Bitcoin time series. Next, we use several deep neural networks tuned by Bayesian Optimization to forecast short-term prices for the next day, three days, five days, and seven days ahead intervals. We found that the Deep Artificial Neural Network model created using technical indicators as input data outperformed other benchmark models like Long Short Term Memory, Bi-directional LSTM (BiLSTM), and Convolutional Neural Network (CNN)-BiLSTM. The presented results record a high accuracy and outperform all existing models available in the past literature with an absolute percentage error as low as 0.28% for the next day forecast and 2.25% for the seventh day for the latest out of sample period ranging from Jan 1, 2021, to Nov 1, 2021. With contributions in feature selection, data-preprocessing, and hybridizing deep learning models, this work contributes to researchers and traders in fundamental and technical domains.

Clinical and Radiomics-Based Deep Learning Predictive Model for Early Treatment Failure after Neoadjuvant Radiochemotherapy for Rectal Cancer

<h3>Purpose/Objective(s)</h3> As shown in RAPIDO and PRODIGE-23 trials, total neoadjuvant therapy for rectal cancer increases the complete response rate, however, the overall survival benefit remains uncertain. Although postponing surgery assures more time for tumor downstaging, it also increases surgical complications and can worsen prognosis in non-responding tumors. We aimed to develop a clinical and radiomics-based model for early treatment failure (ETF) prediction after neoadjuvant radiochemotherapy (RCHTx) for rectal cancer. <h3>Materials/Methods</h3> We conducted a prospective cohort analysis of 92 consecutive and eligible patients with rectal adenocarcinoma treated with neoadjuvant radiochemotherapy in the years 2018-2020, and followed until February 2022. We defined ETF as postoperative poor pathological response and no MRI-confirmed downstaging; or unresectability after neoadjuvant treatment; or cancer-related or treatment-related death within 18 months since treatment initiation. Collected clinical data included physical examination, grading, staging (before and after RCHTx), biomarker levels, RCHTx prescription details, and pre-operative MRI tumor and nodal characteristics. Radiomic features were extracted from GTV delineated on axial MRI T2 TSE sequences. To develop a predictive model, we followed OmicSelector-based (https://biostat.umed.pl/OmicSelector) feature selection, logistic regression (LR), and artificial deep neural network model development procedure. After splitting into training (60%), testing (20%), and validation (20%) datasets, the final feature set was selected based on the highest average logistic regression accuracy on all sets, while all patients with MRI performed outside our institution were included in the testing set (compensating possible batch effect). <h3>Results</h3> ETF was experienced by 24% of patients and was associated with significantly shorter overall survival (median 12.1 months [95%CI:8.5-16.5] vs. not reached, p<0.0001). None of the clinical factors was significantly associated with ETF (adjusted p value>0.05; AUC ROC range 0.39-0.64). LR model based on clinical factors presented overfitting (accuracy 82% on the training set, 56.2% on testing and 68.8% on validation set). The best feature set included 14 selected radiomic features, patient's weight, T-stage, grading, mesorectal fascia and nodal involvement. LR model based on those features achieved perfect accuracy on training and testing sets and 81.3% accuracy on the validation set. Deep neural network model utilizing selected clinical and radiomic features achieved 87.5% accuracy (80% sensitivity, 91% specificity) on testing and 93.8% accuracy (100% sensitivity, 91% specificity) on validation set. <h3>Conclusion</h3> Combined clinical and radiomics-based deep learning model could predict ETF after RCHTx for rectal cancer with at least 80% sensitivity and 90% specificity. Prediction of ETF after RCHTx could select high-risk patients who may require non-standard treatment.