Learning Training Process Research Articles

Multi‐Deep Learning Approach With Transfer Learning for 7‐Stages Diabetic Retinopathy Classification

ABSTRACTProposed novel investigation focused on leveraging an innovative diabetic retinopathy (DR) dataset comprising seven severity stages, an approach not previously examined. By capitalizing on this unique resource, this study′s findings set a new benchmark for DR classification, highlighting the transformative potential of incorporating advanced data into AI models. This study developed a Vgg16 transfer learning model and gauged its performance against established algorithms including Vgg‐19, AlexNet, and SqueezeNet. Remarkably, our results achieved accuracy rates of 96.95, 96.75, 96.09, and 92.96, respectively, emphasizing the contribution of our work. We strongly emphasized comprehensive severity rating, yielding perfect and impressive F1‐scores of 1.00 for “mild NPDR” and 97.00 for “no DR signs.” The Vgg16‐TL model consistently outperformed other models across all severity levels, reinforcing the value of our discoveries. Our deep learning training process, carefully selecting a learning rate of 1e‐05, allowed continuous refinements in training and validation accuracy. Beyond metrics, our investigation underscores the vital clinical importance of precise DR classification for preventing vision loss. This study conclusively establishes deep learning as a powerful transformative tool for developing effective DR algorithms with the potential to improve patient outcomes and advance ophthalmology standards.

Impulsive maneuver strategy for multi-agent orbital pursuit-evasion game under sparse rewards

To address the subjectivity of dense reward designs for the orbital pursuit-evasion game with multiple optimization objectives, this paper proposes the reinforcement learning method with a hierarchical network structure to guide game strategies under sparse rewards. Initially, to overcome the convergence challenges in the reinforcement learning training process under sparse rewards, a hierarchical network structure is proposed based on the hindsight experience replay. Subsequently, considering the strict constraints imposed by orbital dynamics on spacecraft state space, the reachable domain method is introduced to refine the subgoal space in the hierarchical network, further facilitating the achievement of subgoals. Finally, by adopting the centralized training-layered execution approach, a complete multi-agent reinforcement learning method with the hierarchical network structure is established, enabling networks at each level to learn effectively in parallel within sparse reward environments. Numerical simulations indicate that, under the single-agent reinforcement learning framework, the proposed method exhibits superior stability in the late training stage and enhances exploration efficiency in the early stage by 38.89% to 55.56% to the baseline method. Under the multi-agent reinforcement learning framework, as the relative distance decreases, the subgoals generated by the hierarchical network transition from long-term to short-term, aligning with human behavioral logic.

Path planning of 6-DOF free-floating space robotic manipulators using reinforcement learning

This paper presents a study on path planning for 6-DOF free-floating space robotic manipulators using Deep Deterministic Policy Gradient-based Reinforcement Learning. The focus is the development of a novel reward function tailored to address critical requirements for efficient and effective manipulation in space. These requirements include accurate pose alignment between the end-effector and the target, collision avoidance with both the target and other links of the manipulator, smoothing of joint velocities, adaptability to strong dynamic coupling between the manipulator and its base spacecraft due to high manipulator-spacecraft mass ratio, and resilience to noise in the state observations. Uniquely, the proposed reward function employs quaternions for orientation control to reduce pose misalignments and dynamic singularities, as opposed to traditional Euler angles. Our findings demonstrate that the Reinforcement Learning algorithm, when guided by this new reward function that integrates these enhancements and constraints, not only achieves the desired path planning objectives more efficiently but also exhibits faster convergence. Furthermore, the Reinforcement Learning successfully manages significant dynamic coupling effects caused by a high mass ratio between the robotic manipulator and the base spacecraft. Even under the challenge of noisy state observations, the trained agent successfully completes the path planning task, proving the Reinforcement Learning's applicability to real-space mission designs where the noise in observation is inevitable. The study highlights the critical role of reward function design in the Reinforcement Learning training process and its consequential impact on the solution quality, providing a solid foundation for future advancements in free-floating space robotic missions.

AdaGCR: An improved method for optimizing machine learning training process

In contemporary machine learning, training datasets are typically divided into batches, and models are updated incrementally through batch iterations to save memory and reduce overfitting. However, determining the optimal hyperparameters like learning rate, batch size and number of epochs remains a challenge which often relying on empirical insights. This paper explores a novel method called Adaptive Gradient Conflict Rate (AdaGCR) to optimize the training process. It leverages the idea of gradient conflict rate, which reflects the models position within a batch model set and accordingly adjusts the global learning rate. This proposed method is tested by training a Deep Neural Network (DNN) model with MNIST dataset which represents simple tasks and a ResNet-18 model with CIFAR-10 dataset which represents more complicated tasks for solving real world problems. Experiments conducted on DNN demonstrates the proposed methods effectiveness in reducing overfitting and enhancing convergence, particularly with a well-suited initial learning rate. However, its applicability to more complex models like ResNet-18 may require further refinements, such as layer-specific learning rate adjustments. Future research should focus on fine-tuning AdaGCR and extending its utility across diverse machine learning models and tasks.

On the efficacy of conditioned and progressive Latin hypercube sampling in supervised machine learning

In this paper, Latin Hypercube Sampling (LHS) method is compared as per its effectiveness in supervised machine learning procedures. Employing LHS saves computer's processing time and in conjunction with Latin hypercube design properties and space filling ability, is considered as one of the most advanced mechanisms in terms of sampling. Although more data usually deliver better results, when using LHS techniques, same quality outputs can be produced with less data and, as a result, storage cost and training time are reduced. Conditioned Latin Hypercube Sampling (cLHS) is one of those techniques, successfully performing in supervised machine learning tasks. Unfortunately, the minimum sufficient training dataset size cannot be known in advance. In this case, progressive sampling is recommended since it begins with a small sample and progressively increases its size until model accuracy no longer improves. Combining Latin hypercube sampling and the idea of sequentially incrementing sampling, we test Progressive Latin Hypercube Sampling (PLHS) while monitoring the performance of the sampling-based training as the sample size grows. PLHS and cLHS algorithms are applied in datasets with discrete variables securing that each sample is provided with the Latin hypercube design properties and preserves the principal ability of LHS for space filling, as illustrated in respective sample projecting diagrams. The performance of the above LHS methods in supervised machine learning is evaluated by the degree of training of the model, which is certified through the accuracy of the produced confusion matrices in test files. The results from the use of the above Latin Hypercube Sampling techniques compared against benchmark sampling method empirically prove that machine learning training process becomes less costfull, while remaining reliable.

Machine Learning Representations of the Three Lowest Adiabatic Electronic Potential Energy Surfaces for the ArH2+ Reactive System.

In this work, we present Gaussian process regression machine learning representations of the three lowest coupled 2A' adiabatic electronic potential energy surfaces of the ArH2+ reactive system in full dimensionality. Additionally, the nonadiabatic coupling matrix elements were calculated. These adiabatic potentials and their nonadiabatic couplings are necessary ingredients in the theoretical investigation of the nonadiabatic reaction dynamics of the Ar + H2+ → ArH+ + H and Ar+ + H2 → ArH+ + H reactions, as well as the competing charge transfer process, Ar + H2+↔ Ar+ + H2. Accurate ab initio electronic structure calculations (ic-MRCI+Q/aug-cc-pVQZ), whereby the effect of spin-orbit coupling in Ar+ has been accounted for through the state interaction method, serve as input for the machine learning training process. The potential energy surfaces are fitted with high accuracies, with root-mean-square errors on the order of 10-7 eV for the three surfaces, which meet the requirements for chemical dynamics at low temperature. It was found that quite a large number of training points (of the order of 5000 ab initio points) are needed in order to achieve these accuracies due to the complex topography of these electronic surfaces.

A Multi-objective Optimization Planning Framework for Active Distribution System Via Reinforcement Learning

The effective planning of active distribution networks is crucial for utility companies to make informed decisions regarding investments in distributed generation, reliability assessment, reactive power planning, substation revisions, and feeder repositioning. However, the dynamic nature of the solution space makes it challenging for model-based optimization methods to ensure computational performance in active distribution network planning. To address this issue, this study proposes a planning method that focuses on improving computational performance through the continuous updating of the planning model’s solution space during the reinforcement learning training process. Based on simulations conducted on the IEEE 33-bus test system, the proposed planning strategy successfully enhances computational performance while minimizing investment costs compared to other strategies. With the proposed method, the investment cost and the operation cost are reduced by 32.42% and 23.91%, respectively.

Physics constraint Deep Learning based radiative transfer model.

Deep Learning (DL) open sources libraries such as TensorFlow, Keras, and PyTorch have been widely and successfully applied for many applications in a forward model. We have developed the DL radiative transfer model over Oceans under a clear-sky condition. However, the derived physical model from the DL forward model has difficulties in predicting physical properties such as the Jacobian, because multiple solutions can fit the forward model results during the deep learning training process. The Jacobian model in a radiative transfer can calculate radiance sensitivities on geophysical parameters, which are required by satellite radiance assimilation in support of weather forecasts and for retrieving environmental data records. In this study, we introduce a physics constraint into our deep learning training for deriving the forward model that retains right physics. With this physics constraint, the radiance sensitivities are well captured by this new DL radiative transfer.

Simulation and Machine Learning Investigation on Thermoregulation Performance of Phase Change Walls

The outdoor thermal environment can be regarded as a significant factor influencing indoor thermal conditions. The application of phase change materials (PCMs) to the building envelope has the potential to improve the heat storage performance of building walls and, therefore, effectively regulate the temperature variations of the inner surfaces of walls. COMSOL Multiphysics software was adopted firstly to perform the simulations on the thermoregulation performance of phase change wall; the time duration of the temperature at the internal side maintained within the thermal comfort range was used as a quantitative evaluation index of the thermoregulation effects. It was revealed from the simulation results that the time durations of thermal comfort were extended to 5021 s and 4102 s, respectively, when the brick walls were filled with two types of composite PCMs, namely eutectic hydrate (EHS, Na2CO3·10H2O and Na2HPO4·12H2O with the ratio of 4∶6)/5 wt.% BN and EHS/5 wt.% BN/7.5 wt.% expanded graphite (EG), under the conditions of 18 °C ambient temperature and 60 °C heating temperature at the charging stage. Both of them were longer than 3011 s, which corresponds to a pure brick wall. EHS/5 wt.% BN/7.5 wt.% EG exhibited better leakage prevention performance and, therefore, was a candidate for actual application, in comparison with EHS/5 wt.% BN. Then, a machine learning training process focused on the temperature control effects of phase change wall was carried out using a BP neural network, where the heating surface and ambient temperature were used as input variables and the time duration of indoor thermal comfort was the output variable. Finally, the learning deviation between the raw data and the results obtained from machine learning was within 5%, indicating that machine learning can accurately predict the temperature control effects of the phase change wall. The results of the simulations and machine learning can provide information and guidance for the advantages and potentials of PCMs of hydrate salts when being applied to the building envelope. In addition, the accurate prediction of machine learning demonstrated its application prospects to the research of phase change walls.

Programmable In-Memory Computing Circuit for Solving Combinatorial Matrix Operation in One Step

Matrix operations are widely used in practical engineering, but the traditional processing methods rely on the loop iterations and neural network algorithm on the software, requiring a long time to calculate. To address such problem, this paper proposes full hardware in-memory computing circuits based on programmable memristor unit array that can solve combinatorial matrix operations of any order in just one step. First, two basic circuit modules are introduced, which can respectively solve matrix multiplication and matrix equation. Further, the basic modules can be linked to solve combinatorial matrix operations with different forms. It’s worth noting that every module can parallel program the value of each memristor in the memristor unit array and complete one-step computation by hardware. Then, some matrix operations are given in the paper as examples to prove the high accuracy of proposed method, where the average accuracy rate achieves <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$99\%$</tex-math> </inline-formula> . The PSPICE simulation results demonstrate that the processing speed is improved enormously according to the comparison of hardware and software. Moreover, the proposed method has broad application prospect in practical engineering, such as using designed combinational circuit to solve domain shift problem in zero-shot learning, which greatly accelerates the training process of zero-shot learning.

Design of Power Efficient Posit Multiplier using Compressor Based Adder

Abstract: Posit number system has been used in many applications, especially the deep learning. Because of how well its nonuniform number distribution aligns with deep learning's data distribution, deep learning's training process can be sped up. The hardware multiplier is typically built with the widest mantissa bit-width available due to the flexibility of posit numbers' bitwidth. Such multiplier designs consume a lot of power since the mantissa bit-width is not necessarily the maximum value. This is especially true when the mantissa bit-width is tiny. The mantissa multiplier is still built to have the widest bit-width feasible, but it is broken into numerous smaller multipliers. At run-time, just the necessary tiny multipliers are turned on. The regime bitwidth, which can be used to determine the mantissa bit-width, controls those smaller multipliers. This design technique is applied to 8-bit, 16-bit, and 32-bit posit formats.

Digitization of roll forming and its application

With the rapid development of metaverse and intelligent manufacturing, the requirement for the digitization of the traditional manufacturing process is a challenging task for industries. Roll forming is a highly efficient sheet metal forming process which can be widely been applied in a wide range of sectors, such as automotive, transportation, building other key sectors. However, this forming process is highly dependent on the experience of the on-site engineers. This paper presents a highly intelligent, flexible and self-adaption roll forming machine co-developed by the University of Electronic Science and Technology of China and data M. The machine consists of several flexible moving units controlled by codded stepper motors with a decent control system. Moreover, a high-performing artificial intelligence algorithm achieved from a systematic machine learning training process is integrated into this machine. The algorithm has already been use to predict and control the forming process to achieve a high-quality product. Data collection on a large scale and integrated sensors lead to an increased understanding of the process and provide the basis to develop self-optimizing roll forming machines, increasing the productivity, quality and predictability of the roll forming process. The application of digital twin and machine learning techniques is used to predict and reduce the amount of springback in the product. The first high-strength steel parts successfully manufactured with this new forming concept are presented.

The Use of an Errorless Learning Application to Support Re-Learning of (Instrumental) Activities for People Living with Korsakoff Syndrome.

Korsakoff syndrome (KS) is a severe neuropsychiatric syndrome derived from acute thiamine deficiency and concomitant alcohol use disorders. KS patients need lifelong assistance because of the severity of their cognitive problems. In clinical practice and research, errorless learning has proven to be an effective cognitive rehabilitation method for patients with KS. Our study focused on optimizing errorless learning by introducing new software technology to support the training process of errorless learning. Although the benefits of errorless learning for patients with Korsakoff's syndrome have been thoroughly investigated, it is currently unclear whether new technology could contribute to better learning and maintenance of everyday tasks. Therefore, an errorless learning application was built. This device is a web application and can be used on a tablet, laptop, or smartphone. The application allows clinicians and researchers to insert pictures, videoclips, timers, and audio fragments in the different steps of an errorless learning training plan. This way, the different steps are visible and easy to follow for patients. Moreover, it ensures as a learning method that the training is executed exactly the same way for each and every training. The aim of this study was twofold: to examine whether the use of the errorless learning application is effective, and whether it leads to better results than a regular errorless learning of everyday activities. In total, 13 patients with KS were trained in instrumental activities of daily living by means of the application, and 10 patients were trained with traditional instructions. Results showed an equal improvement for both training methods. Importantly, the technology group could better remember the training when probed at a later moment than the traditional errorless learning group. These results are promising for further development of novel technology to support errorless learning applications in clinical practice.

Energy management for hybrid electric vehicles based on imitation reinforcement learning

An effective energy management strategy (EMS) in hybrid electric vehicles (HEVs) is indispensable to promote consumption efficiency due to time-varying load conditions. Currently, learning based algorithms have been widely applied in energy controlling performance of HEVs. However, the enormous computation intensity, massive data training and rigid requirement of prediction of future operation state hinder their substantial exploitation. To mitigate these concerns, an imitation reinforcement learning-based algorithm with optimal guidance is proposed in this paper for energy control of hybrid vehicles to accelerate the solving process and meanwhile achieve preferable control performance. Firstly, offline global optimization is firstly conducted considering various driving conditions to search power allocation trajectories. Then, the battery depletion boundaries with respect to driving distance are introduced to generate a narrowed state space, in which the optimal trajectory is fused into the training process of reinforcement learning to guide the high-efficiency strategy production. The simulation validations reveal that the proposed method provides preferable energy reduction for HEVs in arbitrary driving scenarios, and suggests an efficient solution instruction for similar problems in mechanical and electrical systems with constraints and optimal information.

Experimental evaluation of the impact of physical beam misalignment on the performance of an underwater wireless optical communication network utilizing machine learning

Underwater wireless optical communication systems have garnered significant interest due to advantages in speed and security over radio-frequency and acoustic underwater transmission. One example utilizes Laguerre–Gaussian beams carrying orbital angular momentum to increase bit transfer rate by spatially multiplexing information into an alphabet of symbols, sized 2N, where N is the number of bits encoded in each symbol. This system uses a convolutional neural network (CNN) to demultiplex the information, which has been shown to be capable of high levels of accuracy even in turbid environments or in the presence of significant optical turbulence. However, it has been observed that CNNs struggle when shown images affected by physical environmental effects (different levels of turbulence, optical magnification, beam misalignment) that were not present in the training set. In this paper we further investigate physical beam misalignment under two experimental conditions, quiescent and strong optical turbulence. Physical beam misalignment is defined as a translation of the beam at the receiver. Our optically turbulent environment is characterized using the scintillation of a Gaussian beam, and is estimated to have a refractive index structure constant value of ∼10−11m−2/3. We use two alphabet sizes, of 16 and 256 symbols, and demonstrate classification of received images in a combination of scenarios with adequate success (>90% accuracy) when training and testing under the same conditions, and using a pre-trained CNN, AlexNet. However, when training and testing under physically misaligned conditions, we demonstrate an issue with impractical rates of classification, at 5%–50%. Our investigation provides CNN classification at significantly higher levels of complexity than previously seen, both in terms of experimental turbulence strength and number of classifiable symbols (alphabet size), and clearly demonstrates the importance of carefully designed experiments to aid the machine learning training process for communication systems.

Reinforcement Learning-Based Continuous Action Space Path Planning Method for Mobile Robots

A reinforcement learning-based continuous action space path planning method for mobile robots is proposed in this article. First, the kinematic model of the mobile robot is analyzed, and on this basis, the optimal state space is constructed according to the minimum depth of the field value in the depth image to characterize the distance between the robot and the obstacle. Then, by setting the reward function of the mobile robot based on the artificial potential field method, the information of the robot’s distance from obstacles is continuous, and a new reinforcement learning training process is proposed. Finally, by introducing a DDPG algorithm, the path planning of a mobile robot in an unknown environment is described as a Markov decision process, and the optimal planning of the mobile robot’s continuous action space path is realized with a high success rate. The results show that compared with other three comparison methods, the final success rates of the proposed method are the highest, which are 97.2%, 99.1%, 98.4%, and 98.6%, respectively.

The Health ChatBots in Telemedicine: Intelligent Dialog System for Remote Support

The objective of the Health Bot is to analyze and elaborate on the implementation of an intelligent system that can support telemedicine services using natural language processing (NLP) technology. Health Bot is a robust, modularized, and user-friendly platform that aims to improve the patient’s interactions with the health care system. Using NLP and speech recognition algorithms, the platform can analyze and classify free text and voice input data to symptoms. The classified data are used in the machine learning (ML) training process of the artificial intelligence (AI) models in order to predict the probability of the patient suffering from a specific disease and get alerted in case of disorders. The Health Bot is acting as a medical virtual agent in order to collect any information needed in the sense of a medical interview and provide medical assessments, close appointments with the doctors, and monitor/record the patient’s health condition. In this paper, two models have been trained for the Covid-19 and heart disease cases with 98.3% and 82% accuracy, respectively.

Optimizing Deep Learning Model for Software Cost Estimation Using Hybrid Meta-Heuristic Algorithmic Approach.

Effective software cost estimation significantly contributes to decision-making. The rising trend of using nature-inspired meta-heuristic algorithms has been seen in software cost estimation problems. The constructive cost model (COCOMO) method is a well-known regression-based algorithmic technique for estimating software costs. The limitation of the COCOMO models is that the values of these coefficients are constant for similar kinds of projects whereas, in reality, these parameters vary from one organization to another organization. Therefore, for accurate estimation, it is necessary to fine-tune the coefficients. The research community is now examining deep learning (DL) as a forward-looking solution to improve cost estimation. Although deep learning architectures provide some improvements over existing flat technologies, they also have some shortcomings, such as large training delays, over-fitting, and under-fitting. Deep learning models usually require fine-tuning to a large number of parameters. The meta-heuristic algorithm supports finding a good optimal solution at a reasonable computational cost. Additionally, heuristic approaches allow for the location of an optimum solution. So, it can be used with deep neural networks to minimize training delays. The hybrid of ant colony optimization with BAT (HACO-BA) algorithm is a hybrid optimization technique that combines the most common global optimum search technique for ant colonies (ACO) in association with one of the newest search techniques called the BAT algorithm (BA). This technology supports the solution of multivariable problems and has been applied to the optimization of a large number of engineering problems. This work will perform a two-fold assessment of algorithms: (i) comparing the efficacy of ACO, BA, and HACO-BA in optimizing COCOMO II coefficients; and (ii) using HACO-BA algorithms to optimize and improve the deep learning training process. The experimental results show that the hybrid HACO-BA performs better as compared to ACO and BA for tuning COCOMO II. HACO-BA also performs better in the optimization of DNN in terms of execution time and accuracy. The process is executed upto 100 epochs, and the accuracy achieved by the proposed DNN approach is almost 98% while NN achieved accuracy of up to 85% on the same datasets.

Autonomous localization and mapping based on biomimetic sonar in natural environments

Bats navigating in dense vegetation based on biosonar have to obtain the necessary sensory information from “clutter echoes,” i.e., echoes that are superpositions of contributions of many reflecting facets (e.g., leaves). Since the locations and reflective properties of the individual facets are unknown, clutter echoes have to be treated as random signals that can neither be predicted nor—under most practical circumstance—be replicated. Nevertheless, prior research has shown that deep neural networks are capable of extracting fairly precise location information from clutter echoes. This raises the question whether clutter echoes could be used to provide navigational guidance in a conventional simultaneous localization and mapping (SLAM) framework, which is to a large extent dependent on precise and deterministic measurement to generate a localized map. Our hypothesis is that biomimetic sonar echoes indeed contain the information that is necessary support to local path planning, which can be utilized by a suitable deep learning architecture and training process. To investigate this issue, we have collected data in natural, heavily vegetated environments using a biomimetic sonar head that mimics the periphery of the biosonar system in horseshoe bats. The annotation of the data and the network training process is currently undergoing.

FROM BIM TO POINTCLOUD: AUTOMATIC GENERATION OF LABELED INDOOR POINTCLOUD

Abstract. With the development of deep learning technology, a large number of indoor spatial applications, such as robotics and indoor navigation, have raised higher data requirements for indoor semantic model. However, creating deep learning classifiers requires a large number of labeled datasets, and the collection of such datasets requires a lot of manually labeling proces, which is labor-intensive and time-consuming. In this paper, we propose a method to automatically create 3D point clouds datasets with indoor semantic labels based on parametric BIM model. First, a automatic BIM generation method is proposed through simulating the structure of interior space Secondly, we use a viewpoint-guided labeled point cloud generation method to generate synthetic 3D point clouds with different labels, color information. Especially, noise are also simulated with a gaussian model. As shown in the experiments, the point cloud data with labels can be quickly obtained from existing BIM models, which will largely reduce the complexity of data labeling and improve efficiency. These simulated data can be used in the deep learning training process and improve the semantic segmentation accuracy.