Drastic Performance Improvement Research Articles

Abrupt and spontaneous strategy switches emerge in simple regularised neural networks.

Humans sometimes have an insight that leads to a sudden and drastic performance improvement on the task they are working on. Sudden strategy adaptations are often linked to insights, considered to be a unique aspect of human cognition tied to complex processes such as creativity or meta-cognitive reasoning. Here, we take a learning perspective and ask whether insight-like behaviour can occur in simple artificial neural networks, even when the models only learn to form input-output associations through gradual gradient descent. We compared learning dynamics in humans and regularised neural networks in a perceptual decision task that included a hidden regularity to solve the task more efficiently. Our results show that only some humans discover this regularity, whose behaviour was marked by a sudden and abrupt strategy switch that reflects an aha-moment. Notably, we find that simple neural networks with a gradual learning rule and a constant learning rate closely mimicked behavioural characteristics of human insight-like switches, exhibiting delay of insight, suddenness and selective occurrence in only some networks. Analyses of network architectures and learning dynamics revealed that insight-like behaviour crucially depended on a regularised gating mechanism and noise added to gradient updates, which allowed the networks to accumulate "silent knowledge" that is initially suppressed by regularised gating. This suggests that insight-like behaviour can arise from gradual learning in simple neural networks, where it reflects the combined influences of noise, gating and regularisation. These results have potential implications for more complex systems, such as the brain, and guide the way for future insight research.

Applying Large Graph Neural Networks to Predict Transition Metal Complex Energies Using the tmQM_wB97MV Data Set.

Machine learning (ML) methods have shown promise for discovering novel catalysts but are often restricted to specific chemical domains. Generalizable ML models require large and diverse training data sets, which exist for heterogeneous catalysis but not for homogeneous catalysis. The tmQM data set, which contains properties of 86,665 transition metal complexes calculated at the TPSSh/def2-SVP level of density functional theory (DFT), provided a promising training data set for homogeneous catalyst systems. However, we find that ML models trained on tmQM consistently underpredict the energies of a chemically distinct subset of the data. To address this, we present the tmQM_wB97MV data set, which filters out several structures in tmQM found to be missing hydrogens and recomputes the energies of all other structures at the ωB97M-V/def2-SVPD level of DFT. ML models trained on tmQM_wB97MV show no pattern of consistently incorrect predictions and much lower errors than those trained on tmQM. The ML models tested on tmQM_wB97MV were, from best to worst, GemNet-T > PaiNN ≈ SpinConv > SchNet. Performance consistently improves when using only neutral structures instead of the entire data set. However, while models saturate with only neutral structures, more data continue to improve the models when including charged species, indicating the importance of accurately capturing a range of oxidation states in future data generation and model development. Furthermore, a fine-tuning approach in which weights were initialized from models trained on OC20 led to drastic improvements in model performance, indicating transferability between ML strategies of heterogeneous and homogeneous systems.

Accounting for precipitation asymmetry in a multiplicative random cascade disaggregation model

Abstract. Analytical multiplicative random cascades (MRCs) are widely used for the temporal disaggregation of coarse-resolution precipitation time series. This class of models applies scaling models to represent the dependence of the cascade generator on the temporal scale and the precipitation intensity. Although determinant, the dependence on the external precipitation pattern is usually disregarded in the analytical scaling models. Our work presents a unified MRC modelling framework that allows the cascade generator to depend in a continuous way on the temporal scale, precipitation intensity and a so-called precipitation asymmetry index. Different MRC configurations are compared for 81 locations in Switzerland with contrasted climates. The added value of the dependence of the MRC on the temporal scale appears to be unclear, unlike what was suggested in previous works. Introducing the precipitation asymmetry dependence into the model leads to a drastic improvement in model performance for all statistics related to precipitation temporal persistence (wet–dry transition probabilities, lag-n autocorrelation coefficients, lengths of dry–wet spells). Accounting for precipitation asymmetry seems to solve this important limitation of previous MRCs. The model configuration that only accounts for the dependence on precipitation intensity and asymmetry is highly parsimonious, with only five parameters, and provides adequate performances for all locations, seasons and temporal resolutions. The spatial coherency of the parameter estimates indicates a real potential for regionalisation and for further application to any location in Switzerland.

Lot streaming in workforce scheduling problem for seru production system under Shojinka philosophy

Drawing upon efficiency, flexibility, and responsiveness, this study attempts to explore the impact of lot streaming and worker assignment strategies in the workforce scheduling problem for the seru production system (SPS), which has emerged as an alternative to traditional assembly lines. The addressed problem involves decisions related to employee timetabling, lot splitting, and sublot scheduling. While scheduling problems in SPS have been extensively researched in recent years, the workforce scheduling problem has not received much attention in the literature. Moreover, to our best knowledge, it has not been investigated with the sublot division methodologies within the context of SPS so far. To bridge this gap, a generic novel optimization model is developed with the objective of minimizing average flow time (AFT) by integrating lot streaming and the Shojinka philosophy through modules within the model. Additionally, the structural properties of the problem are examined, and lower and upper bound formulations are developed based on these properties. Given the NP-hard nature of the problem, customized approaches based on the genetic algorithm (GA) are proposed for solving large-sized problems, considering the Shojinka and division methodologies. The computational results clarify that achieving Shojinka with variable sublots division methodology significantly reduces the AFT objective. Furthermore, the findings confirm that adapting variable sublots in operational scenarios results in a drastic improvement in system performance, regardless of the implemented worker assignment strategy.

Cognitive Dynamic Systems: A Review of Theory, Applications, and Recent Advances

The field of cognitive dynamic systems (CDSs) is an emerging area of research, whereby engineering learns from neuroscience. Under this framework, engineering systems are configured in a manner that mimics the human brain and improves the utility and performance of traditional systems. In essence, a CDS builds on Fuster’s paradigm of cognition and is fulfilled with the presence of five cognitive processes: the perception-action cycle, memory, attention, intelligence, and language. When augmented with these processes, a system can be classified as a CDS and is afforded the capabilities of processing information and learning from experience through continued interactions with the environment. Tremendous benefit from adopting the CDS framework has been observed in the literature, especially in the fields of cognitive radio and cognitive radar. More recently, the framework has been extended to other areas, such as control theory, risk control, and the Internet of Things; where the potential for drastic performance improvements has been evident in the literature. This comprehensive article presents a thorough background and exposition of the CDS framework and each field where it has been applied. In addition, we provide a comprehensive review of the recent advancements and related works in each domain by summarizing the key facts relating to the methodologies, findings, and limitations of the surveyed papers. Our novel contributions involve being the first source of centralized information on this topic and forming the foundation for future research efforts by presenting suggestions regarding worthwhile avenues for further investigation.

Electrochemical Actuation of a DyP Peroxidase: A Facile Method for Drastic Improvement of the Catalytic Performance.

Dye decolorizing peroxidases (DyP) have attracted interest for applications such as dye-containing wastewater remediation and biomass processing. So far, efforts to improve operational pH ranges, activities, and stabilities have focused on site-directed mutagenesis and directed evolution strategies. Here, we show that the performance of the DyP from Bacillus subtilis can be drastically boosted without the need for complex molecular biology procedures by simply activating the enzyme electrochemically in the absence of externally added H2O2. Under these conditions, the enzyme shows specific activities toward a variety of chemically different substrates that are significantly higher than in its canonical operation. Moreover, it presents much broader pH activity profiles with the maxima shifted toward neutral to alkaline. We also show that the enzyme can be successfully immobilized on biocompatible electrodes. When actuated electrochemically, the enzymatic electrodes have two orders of magnitude higher turnover numbers than with the standard H2O2-dependent operation and preserve about 30% of the initial electrocatalytic activity after 5 days of operation-storage cycles.

An Economy-based Amendment to Learning Hidden Structure with Robust Interpretive Parsing

Robust Interpretive Parsing is a method of learning hidden structure with error-driven learning algorithms (Tesar and Smolensky 1998, 2000). When an algorithm makes an error while learning a word, it calculates what it considers to be the structural representation of the word (the “target parse”) and changes its grammar accordingly. Among potential directions of grammar change, it chooses the direction that best satisfies the current constraint ranking. However, this choice is problematic because the current constraint ranking is guaranteed to be erroneous: had it not been erroneous, an error would not have occurred in the first place. While this problem has been recognized conceptually by Jarosz (2013), there have not been demonstrations of the problem arising in actual learning simulations. I present evidence from new learning simulations that the choice of target parse based on constraint ranking can indeed lead to learning failure. I then suggest an alternative method of choosing the target parse. This method opts for the target which involves the least amount of rerankings to accommodate for. In other words, it rewards economical change. While this alternative method does not lead to drastic improvement of performance, it does result in more efficient convergence.

Using Novel Technologies to Implement Belize's First Formal Prehospital Emergency System

Introduction:Belize has no formal prehospital emergency medical system, leaving the majority of acutely sick and injured persons overwhelmingly dependent on private transport. To address this issue, a collaboration of public and non-profit partners worked with the Belize National Fire Service to implement the country's first formal prehospital emergency medical service using novel communications technologies. With new resources and vehicles already donated to the fire service, the collaboration focused specifically on the communications component of the response system, specifically to improve the handling of incoming requests for emergency assistance from the public, as well as to improve the process of dispatching prehospital personnel using readily-available mobile technologies.Method:Working with the Belize National Fire Service, program partners implemented the country's first emergency communications center, trained new dispatchers, field-tested the dispatch technology through intensive training sessions, and launched the system in the capital district of Cayo.Results:Launched in June 2022, the program has thus far achieved the following outcomes: Active Dispatchers: 26Active Responders: 104Emergencies Dispatched: 156Average Scene Response Time: 7m45sConclusion:Over the past year, partner NGOs Trek Medics and Empact Northwest have worked collaboratively to implement a first-ever centralized emergency dispatch system for the Belize National Fire Service, using a novel cloud-based dispatch software running on readily-available mobile phones and mobile. In addition to drastic improvements in response performance, satisfaction among system managers and response personnel is high, with plans currently underway to scale the program nationally.

CHICKEN BANDA PERFORMANCE IMPROVEMENT UTILIZING NEURO-FUZZY LOGIC TECHNIQUE

This study is on improvement of performance of the chicken Banda, using indoor change in environmental conditions for temperature control. The differential change in climatic conditions is technically used to put on the fan and the Banda so as to realize the right comfortable indoor conditions.
 The chicken chicks’ Banda Mathematical model is created, prototype designed, temperature controller to depict a two systems simulation of neuro fuzzy logic and fuzzy logic .The performance is analyzed by the use of Matlab Simulink latest edition. To monitor the temperature of the Chicken cage the neural fuzzy logic technique is utilized. As far as the prototype is concerned the chicks’ cage set temperature is fixed at 26.50C.
 The study will show that the reference input can be kept on track by the process controller hence proving the principle that the neural fuzzy control is much superior in optimizing performance compared to the fuzzy only controllers. The Back propagation (BP) and least square estimator (LSE) are the hybrid optimization methods which are used. For data training the gradient descent method (GDM) is used.
 The research reveal that there is drastic performance improvement in the behavior response where result show that there settling time is reduced from 0.75 to 0.48 seconds while the percentage overshoot is also reduced down from 29.9% to 0.9345%.

Adaptive boosting of random forest algorithm for automatic petrophysical interpretation of well logs

The power of Machine Learning is demonstrated for automatic interpretation of well logs and determining reservoir properties for volume of shale, porosity, and water saturation respectively for tight clastic sequences. Random Forest algorithms are reputed for their efficiency as they belong to a class of algorithms called ensemble methods, which are traditionally seen as weak learners, but can be transformed into strong performers and they promise to deliver highly accurate results. The study area is located offshore Australia in the Poseidon and Crown fields situated in the Browse Basin, which are gas fields in tight complex clastic reservoirs. There are 5 wells used in this study with one well manually interpreted which is subsequently used in developing a machine learning model which predicts the output for the other 4 wells. The basic open hole logs namely Natural gamma ray, Resistivity, Neutron Porosity, Bulk Density, P-wave and S-wave sonic travel-time, are used in interpretation. One of the wells has a missing S-wave travel-time log which was also predicted by developing a Random Forest Machine Learning model. The results indicate a very robust improvement in performance when Random Forest algorithm was combined with Adaptive Boosting when interpreting the well logs. The training accuracy using Random Forest alone was 98.21%, but testing was 77.62% which suggested over-fitting by the Random Forest model. The Adaptive Boosting of the Random Forest algorithm resulted in the overall training accuracy of 99.40% and an overall testing accuracy of 97.03%, indicating a drastic improvement in performance. S-wave travel-time log was predicted by preparing a training set consisting of Natural gamma ray, Resistivity, Neutron Porosity, Bulk Density, and P-wave travel-time logs for the 4 wells using Random Forest which gave a training accuracy of 99.79% and a testing accuracy of 98.54%. Machine learning algorithms can be successfully applied for interpreting well log data in complex sedimentary environment and their performance can be drastically improved using Adaptive Boosting.

EndoBeams.jl: A Julia finite element package for beam-to-surface contact problems in cardiovascular mechanics

The increasing use of mini-invasive and endovascular surgical techniques is at the origin of the pressing need for computational models to support planning and training. Several implantable devices have a wire-like structure, which can be modelled using beam elements. Our objective is to create an efficient Finite Element (FE) modelling framework for such devices. For that, we developed the EndoBeams.jl package, written exclusively in Julia, for the numerical simulation of contact interactions between wire-like structures and rigid surfaces. The package is based on a 3D FE corotational formulation for frictional contact dynamics of beams. The rigid target surface is described implicitly using a signed distance field, predefined in a volumetric grid. Since the main objective behind this package is to find the best compromise between computational speed and code readability, the algorithm, originally in Matlab, was translated and optimised in Julia, a programming language designed to combine the performance of low-level languages with the productivity of high-level ones. To evaluate the robustness, a set of tests were conducted to compare the simulation results and computational time of EndoBeams.jl against literature data, the original Matlab code and the commercial software Abaqus. The tests proved the accuracy of the underlying beam-to-surface formulation and showed the drastic performance improvement of the Julia code with respect to the original one. EndoBeams.jl is also slightly faster than Abaqus. Finally, as a proof of concept in cardiovascular medicine, a further example is shown where the deployment of a braided stent is simulated within an idealised artery.

(Invited, Digital Presentation) 50 Ns Ultrafast Memory Operation in 2D Heterostructured Non-Volatile Memory Device

2D heterostructures have been extensively investigated as next-generation non-volatile memory (NVM) devices. In the last decade, drastic performance improvements and further advanced functionalities have been demonstrated. However, this progress is not sufficiently supported by the understanding of their operations, obscuring the material and device structure design policy. Here, detailed operation mechanisms are elucidated by exploiting the floating gate voltage (V FG) trajectory measurements [1,2]. Systematic comparisons of MoTe2, WSe2, and MoS2 channel devices revealed that the tunneling behavior between the channel and FG is controlled by three kinds of current-limiting paths, i.e., tunneling barrier, 2D/metal contact, and pn junction in the channel. Based on the understanding of the memory operation mechanism through the V FG trajectory measurement, we propose the best all 2D NVM device structure with a direct tunneling path between source/drain electrodes and floating gate for ultrafast memory operation. Indeed, a 50 ns program/erase operation is successfully achieved [3]. Moreover, we examined the dielectric breakdown strength (E BD) of h-BN under short voltage pulses for the origin for this ultrafast operation, because a high dielectric breakdown strength allows a large tunneling current. Surprisingly, an E BD = 26.1 MV/cm for h-BN is realized under short voltage pulses, largely exceeding the E BD = ~12 MV/cm from the DC measurement. This suggests that the high E BD of h-BN can be the physical origin of the ultrafast operations. In this talk, I would like to discuss the future perspective of 2D NVM application.[1] T. Sasaki, KN, et al., "Understanding the Memory Window Overestimation of 2D Materials Based Floating Gate Type Memory Devices by Measuring Floating Gate Voltage", Small, 2020, 16, 2004907.[2] T. Sasaki, KN, et al., "Material and Device Structure Designs for 2D Memory Devices Based on the Floating Gate Voltage Trajectory", ACS nano, 2021, 15, 6658.[3] T. Sasaki, KN, et al., "Ultrafast Operation of All 2D-Heterostructured Nonvolatile Memory Devices Provided by the Strong Short-Time Dielectric Breakdown Strength of h-BN", Adv. Funct. Mater. (under revision).

Wake-Up Receiver-Based Routing for Clustered Multihop Wireless Sensor Networks.

The Wireless Sensor Network (WSN) is one of the most promising solutions for the supervision of multiple phenomena and for the digitisation of the Internet of Things (IoT). The Wake-up Receiver (WuRx) is one of the most trivial and effective solutions for energy-constrained networks. This technology allows energy-autonomous on-demand communication for continuous monitoring instead of the conventional radio. The routing process is one of the most energy and time-consuming processes in WSNs. It is, hence, crucial to conceive an energy-efficient routing process. In this paper, we propose a novel Wake-up Receiver-based routing protocol called Clustered WuRx based on Multicast wake-up (CWM), which ensures energy optimisation and time-efficiency at the same time for indoor scenarios. In our proposed approach, the network is divided into clusters. Each Fog Node maintains the routes from each node in its cluster to it. When a sink requires information from a given node, it’s corresponding Fog Node uses a multicast wake-up mechanism to wake up the intended node and all the intermediate nodes that will be used in the routing process simultaneously. Measurement results demonstrate that our proposed approach exhibits higher energy efficiency and has drastic performance improvements in the delivery delay compared with other routing protocols.

Rate-compatible multi-edge type low-density parity-check code ensembles for continuous-variable quantum key distribution systems

In this paper, we propose a design rule of rate-compatible punctured multi-edge type low-density parity-check (MET-LDPC) code ensembles with degree-one variable nodes for the information reconciliation (IR) of continuous-variable quantum key distribution (CV-QKD) systems. In addition to the rate compatibility, the design rule effectively resolves the high error-floor issue which has been known as a technical challenge of MET-LDPC codes at low rates. Thus, the proposed design rule allows one to implement rate-compatible MET-LDPC codes with good performances both in the threshold and low-error-rate regions. The rate compatibility and the improved error-rate performances significantly enhance the efficiency of IR for CV-QKD systems. The performance improvements are confirmed by comparing complexities and secret key rates of IR schemes with MET-LDPC codes whose ensembles are optimized with the proposed and existing design rules. In particular, the SNR range of positive secrecy rate increases by 1.44 times, and the maximum secret key rate improves by 2.10 times as compared to the existing design rules. The comparisons clearly show that an IR scheme can achieve drastic performance improvements in terms of both the complexity and secret key rate by employing rate-compatible MET-LDPC codes constructed with code ensembles optimized with the proposed design rule.

Pseudo Zero Pronoun Resolution Improves Zero Anaphora Resolution

Masked language models (MLMs) have contributed to drastic performance improvements with regard to zero anaphora resolution (ZAR). To further improve this approach, in this study, we made two proposals. The first is a new pretraining task that trains MLMs on anaphoric relations with explicit supervision, and the second proposal is a new finetuning method that remedies a notorious issue, the pretrain-finetune discrepancy. Our experiments on Japanese ZAR demonstrated that our two proposals boost the state-of-the-art performance, and our detailed analysis provides new insights on the remaining challenges.

Drastic performance improvement of indoor organic photovoltaics using a novel laminated homojunction hole-transport layer

A homojunction PTQ10 polymer offers suppressed charge recombination and maximized quasi-fermi level splitting thereby leading to an improvement of over 25% in the power conversion efficiency of PTQ10:Y6-based devices under halogen illumination.

Freedom of the mixer rotation axis improves performance in the quantum approximate optimization algorithm

Variational quantum algorithms such as the quantum approximate optimization algorithm (QAOA) are particularly attractive candidates for implementation on near-term quantum processors. As hardware realities such as error and qubit connectivity will constrain achievable circuit depth in the near future, new ways to achieve high-performance at low depth are of great interest. In this work, we present a modification to QAOA that adds additional variational parameters in the form of freedom of the rotation-axis in the $XY$-plane of the mixer Hamiltonian. Via numerical simulation, we show that this leads to a drastic performance improvement over standard QAOA at finding solutions to the MAXCUT problem on graphs of up to 7 qubits. Furthermore, we explore the Z-phase error mitigation properties of our modified ansatz, its performance under a realistic error model for a neutral atom quantum processor, and the class of problems it can solve in a single round.

Copper-bispidine-catalyzed aziridination – A new twist in small molecule activation

Described are copper(I) and copper(II) complexes of a series of tetradentate bispidine ligands together with their structural, spectroscopic and electrochemical properties. These complexes are efficient catalysts for the aziridination of styrene, and the efficiency is shown to be correlated to the CuII/I redox potential. While variation of the redox potential only leads to relatively small changes of the catalytic properties, it is shown that replacement of the in-plane tertiary amine of the bispidine scaffold by a secondary amine leads to an up to 1000-fold increase of the catalytic efficiency. In contrast, no significant increase of the catalytic performance is observed when the axial tertiary amine is replaced by a secondary amine. The effect of the in-plane substitution of the tertiary amine is shown to be related to a deprotonation at the secondary amine in presence of a base. This is demonstrated with a CuII acetonitrile complex for which a pKa value in MeCN of approx. 17.5 (corresponding to an approx. pKa value of 9.5 in H2O) was determined by spectrophotometric titration. It is proposed that the enhanced catalytic efficiency is due to deprotonation of the amine at the high-valent part of the catalytic cycle. This may lead to a stabilization of the nitrene form, but most important to a drastic improvement of the catalytic performance.

Hardware-Efficient Residual Neural Network Execution in Line-Buffer Depth-First Processing

Deep convolutional neural networks (CNNs) provide State-of-the-Art (SotA) results in a variety of machine learning (ML) applications, such as image classification and object detection. The trends towards higher-resolution input images and large-scale model structures have brought drastic performance improvements for these tasks. Yet, these trends also pose increased stress on hardware resources. Recent work proposed an advanced layer fusion method relying on the line-buffer depth-first (LBDF) processing, to reduce memory usage and off-chip memory accesses in high resolution CNN processing. However, when deploying such LBDF approach for residual networks, the required external memory access or on-chip memory capacity drastically increases again, hence canceling many benefits of the depth-first approach. This paper, therefore, outlines and analyzes the possible methods for handling residual connections (RC) in combination with LBDF. With the proposed ReLU-based compression and tiling techniques, we minimize the overhead of RC on on-chip memory requirements and external memory accesses. Moreover, a proposed hybrid strategy, optimally combining the different methods across different residual blocks of a residual network, yields for ResNet20 and SqueezeNet a reduction of 24.7% and 45.1% respectively in inference energy cost. Also the execution latency is reduced by 19.4% and 46.7% respectively in comparison to the best single method under certain on-chip memory budgets. In addition, compared to the SotAs, in the best case, our hybrid strategy only consumes 43% energy and 29.9% latency while use the same on-chip memories; only requires 27.4% on-chip memory to achieve the same performance.

Role of an oxide interface in a resistive switch

In the present era of data-driven architectures like 5G, Internet of things (IoT), Artificial Intelligence (AI), etc, the requirement of fast-switchable memory storage is more than ever. Oxide resistive switches are considered to be a primary choice in the non-volatile memory design. In this work, we have engineered the conventional metal-insulator-metal (MIM) structure of an oxide memristor (Ag/ZnO/ITO) by inducing an additional oxide layer La0.7Sr0.3MnO3 (LSMO) at the interface between the active layer (ZnO) and Ag electrode. The presence of LSMO acts as a reservoir for the oxygen vacancies, easing the conducting filament formation process in ZnO, thereby enabling drastic improvement of the switching performance and offering reliable endurance over multiple switching cycles. First-principles-based calculations suggested the role of oxygen vacancies in controlling the electronic state of ZnO and formation of vacancies in the resistive switching process, which is in agreement with the experimental observation. The current results pave ways for improving the switching performance of resistive memory circuits through simple structural engineering incorporation, which lies at the heart of oxide electronics.