Rate Schedule Research Articles

Device Variation Effects on Neural Network Inference Accuracy in Analog In‐Memory Computing Systems

In analog in‐memory computing systems based on nonvolatile memories such as resistive random‐access memory (RRAM), neural network models are often trained offline and then the weights are programmed onto memory devices as conductance values. The programmed weight values inevitably deviate from the target values during the programming process. This effect can be pronounced for emerging memories such as RRAM, PcRAM, and MRAM due to the stochastic nature during programming. Unlike noise, these weight deviations do not change during inference. The performance of neural network models is investigated against this programming variation under realistic system limitations, including limited device on/off ratios, memory array size, analog‐to‐digital converter (ADC) characteristics, and signed weight representations. Approaches to mitigate such device and circuit nonidealities through architecture‐aware training are also evaluated. The effectiveness of variation injection during training to improve the inference robustness, as well as the effects of different neural network training parameters such as learning rate schedule, will be discussed.

Multi-Objective Optimization of Low Reynolds Number Airfoil Using Convolutional Neural Network and Non-Dominated Sorting Genetic Algorithm

The airfoil is the prime component of flying vehicles. For low-speed flights, low Reynolds number airfoils are used. The characteristic of low Reynolds number airfoils is a laminar separation bubble and an associated drag rise. This paper presents a framework for the design of a low Reynolds number airfoil. The contributions of the proposed research are twofold. First, a convolutional neural network (CNN) is designed for the aerodynamic coefficient prediction of low Reynolds number airfoils. Data generation is discussed in detail and XFOIL is selected to obtain aerodynamic coefficients. The performance of the CNN is evaluated using different learning rate schedulers and adaptive learning rate optimizers. The trained model can predict the aerodynamic coefficients with high accuracy. Second, the trained model is used with a non-dominated sorting genetic algorithm (NSGA-II) for multi-objective optimization of the low Reynolds number airfoil at a specific angle of attack. A similar optimization is performed using NSGA-II directly calling XFOIL, to obtain the aerodynamic coefficients. The Pareto fronts of both optimizations are compared, and it is concluded that the proposed CNN can replicate the actual Pareto in considerably less time.

Effect of seed rate and irrigation scheduling on growth and yield of direct seeded rice (Oryza sativa)

Effect of seed rate and irrigation scheduling on growth and yield of direct seeded rice (Oryza sativa)

Resource Allocation and Task Off-Loading for 6G Enabled Smart Edge Environments

There has been an enormous increase in information flow and communication data due to the rapid rise in the number of Internet of Everything (IoE) devices and the development of cutting-edge technologies such as the rollout of the Sixth Generation (6G) network. The rising and inevitable off-loading requirements of IoE devices have resulted in an unprecedented increase in the reliance on edge and cloud paradigms. However, such a reliance on far-end technologies to access already scarce resources can often result in increased latency and unstable connection issues due to limited bandwidth. In this paper, we investigate the solution for such a stringent network design by presenting a conceptual cloud architecture based on key components such as resource allocation, scheduling and task off-loading for IoE devices. The IoE devices utilise a scheduler to access resources from nearby higher resourced IoE devices for their task computation, where the scheduler allocates incoming requests according to the availability of resources within a cluster of devices or to other devices in nearby clusters. Motivated by these design characteristics, we propose a design of a novel Main Task Off-loading Scheduling Algorithm (MTOSA) for efficient task allocation and dissemination. We present a theoretical analysis of five different scheduling policies namely Round Robin (RR), Strongest Channel (SC), Max Rate (MR), Proportional Fair (PF) and Priority Base (PB) scheduling to find an optimal technique for task off-loading in futuristic networks. Furthermore, we compare the performance of these five scheduling policies with the two existing scheduling policies from the literature. It is shown through various experiments that the proposed MTOSA algorithm performs better when compared with the existing schemes for different performance parameters.

Analysis and cost comparison of framed structure using different building materials under different load combinations

In the present study, the analysis and cost comparison of framed structures using different building materials under different load combinations were carried out. G + 7 storey framed structure of 24 m high was analysed for 8 load cases and 4 different load combinations. The bay width was maintained as 4 m in both directions (16 m × 16 m) and analysis was achieved through the licensed version of STAAD.Pro CONNECT V22. The material properties assigned were concrete, steel, and steel-concrete composite. The total cost of material utilised was calculated, with reference to the unit costs mentioned in the Standard schedule of rates. The cost comparison was carried out for each type of material under different load cases and different load combinations. It was observed that under extreme load combination, the concrete frame showed the increase in cost of 77%. Whereas the corresponding increase in steel-concrete composite frame and steel frame were 62% and 50% respectively. Even though the initial cost of steel structures is high, these structures were found to be optimal with less increment in cost under extreme load combinations.

DropCyclic: Snapshot Ensemble Convolutional Neural Network Based on a New Learning Rate Schedule for Land Use Classification

The ensemble learning method is a necessary process that provides robustness and is more accurate than the single model. The snapshot ensemble convolutional neural network (CNN) has been successful and widely used in many domains, such as image classification, fault diagnosis, and plant image classification. The advantage of the snapshot ensemble CNN is that it combines the cyclic learning rate schedule in the algorithm to snap the best model in each cycle. In this research, we proposed the dropCyclic learning rate schedule, which is a step decay to decrease the learning rate value in every learning epoch. The dropCyclic can reduce the learning rate and find the new local minimum in the subsequent cycle. We evaluated the snapshot ensemble CNN method based on three learning rate schedules: cyclic cosine annealing, max-min cyclic cosine learning rate scheduler, and dropCyclic then using three backbone CNN architectures: MobileNetV2, VGG16, and VGG19. The snapshot ensemble CNN methods were tested on three aerial image datasets: UCM, AID, and EcoCropsAID. The proposed dropCyclic learning rate schedule outperformed the other learning rate schedules on the UCM dataset and obtained high accuracy on the AID and EcoCropsAID datasets. We also compared the proposed dropCyclic learning rate schedule with other existing methods. The results show that the dropCyclic method achieved higher classification accuracy compared with other existing methods.

An Effective Learning Rate Scheduler for Stochastic Gradient Descent Based Deep Learning Model in Healthcare Diagnosis System

An Effective Learning Rate Scheduler for Stochastic Gradient Descent Based Deep Learning Model in Healthcare Diagnosis System

Policy Forum: Bunching in the Political Contribution Tax Credit Schedule—Theory, Evidence, and Policy

In Canada, the federal government provides non-refundable tax credits for political contributions. The tax credit rate schedule for political contributions has three thresholds, with the tax credit rate decreasing as contributions exceed each threshold. The authors of this article describe how these thresholds have changed over time, their rationale, and their incidence, and provide some evidence that taxfilers respond to the credit rate changes. The authors also describe, by way of comparison, the tax treatment of charitable donations, another voluntary contribution that may be eligible for a non-refundable tax credit.

Simply Fine-Tuned Deep Learning-Based Classification for Breast Cancer with Mammograms

A lump growing in the breast may be referred to as a breast mass related to the tumor. However, not all tumors are cancerous or malignant. Breast masses can cause discomfort and pain, depending on the size and texture of the breast. With an appropriate diagnosis, non-cancerous breast masses can be diagnosed earlier to prevent their cultivation from being malignant. With the development of the artificial neural network, the deep discriminative model, such as a convolutional neural network, may evaluate the breast lesion to distinguish benign and malignant cancers from mammogram breast masses images. This work accomplished breast masses classification relative to benign and malignant cancers using a digital database for screening mammography image datasets. A residual neural network 50 (ResNet50) model along with an adaptive gradient algorithm, adaptive moment estimation, and stochastic gradient descent optimizers, as well as data augmentations and fine-tuning methods, were implemented. In addition, a learning rate scheduler and -fold cross-validation were applied with training procedures to determine the best models. The results of training accuracy, -value, test accuracy, area under the curve, sensitivity, precision, F1-score, specificity, and kappa for adaptive gradient algorithm , , , and stochastic gradient descent fine-tunings indicate that the classifier is feasible for categorizing breast cancer into benign and malignant from the mammographic breast masses images.

An effective learning rate scheduler for stochastic gradient descent-based deep learning model in healthcare diagnosis system

An effective learning rate scheduler for stochastic gradient descent-based deep learning model in healthcare diagnosis system

Automated learning rate search using batch-level cross-validation

Deep learning researchers and practitioners have accumulated a significant amount of experience on training a wide variety of architectures on various datasets. However, given a network architecture and a dataset, obtaining the best model (i.e. the model giving the smallest test set error) while keeping the training time complexity low is still a challenging task. Hyper-parameters of deep neural networks, especially the learning rate and its (decay) schedule, highly affect the network's final performance. The general approach is to search the best learning rate and learning rate decay parameters within a cross-validation framework, a process that usually requires a significant amount of experimentation with extensive time cost. In classical cross-validation (CV), a random part of the dataset is reserved for the evaluation of model performance on unseen data. This technique is usually run multiple times to decide learning rate settings with random validation sets. In this paper, we explore batch-level cross-validation as an alternative to the classical dataset-level, hence macro, CV. The advantage of batch-level or micro CV methods is that the gradient computed during training is re-used to evaluate several different learning rates. We propose an algorithm based on micro CV and stochastic gradient descent with momentum, which produces a learning rate schedule during training by selecting a learning rate per epoch, automatically. In our algorithm, a random half of the current batch (of examples) is used for training and the other half is used for validating several different step sizes or learning rates. We conducted comprehensive experiments on three datasets (CIFAR10, SVHN and Adience) using three different network architectures (a custom CNN, ResNet and VGG) to compare the performances of our micro-CV algorithm and the widely used stochastic gradient descent with momentum in a early-stopping macro-CV setup. The results show that, our micro-CV algorithm achieves comparable test accuracy to macro-CV with a much lower computational cost.

Procedures for determining and then modifying the extinction component of multiple schedules for destructive behavior.

As a component of reinforcer schedule thinning following functional communication training, multiple schedules of reinforcement produce desirable rates and patterns of communication responses as an alternative response to destructive behavior. However, reinforcement schedule thinning is a gradual process that can take many sessions to obtain therapeutic goals. The desired outcome is that manding occurs only during signaled intervals of reinforcement with a sufficiently lean terminal schedule of reinforcement availability and low rates of destructive behavior. The purposes of this study were to (a) evaluate an assessment for informing the initial duration of extinction for alternative responding, (b) evaluate the utility of competing stimuli during extinction for alternative responding, and (c) assess a method for fading the availability of competing stimuli. With these procedures, all 4 participants experienced terminal schedules of reinforcement with rapid, robust reductions in destructive behavior soon after baseline. We discuss the implications and directions for future research.

Phenotypic correlation and regression of hydroponics maize fodder as influenced by seed rate, nutrient sources and spray schedule

The experiments were carried out at AICRP on Dry Land Agriculture, University of Agricultural Science, GKVK, Bengaluru to study the “Phenotypic correlation and regression of hydroponics maize fodder as influenced by seed rate, nutrient sources and spray schedule” during 2017-2018. The results of revealed significantly positive correlation between fodder yield per kg seed and yield parameter like total fresh weight of fodder per kg seed (r=0.95**), total dry weight per kg of seed (r=0.86*), growth parameters like shoot length (r= 0.90**), root length (r=0.85*) and seedling vigour (r=0.84*), physiological parameters like water use efficiency (r=0.86*), relative water content (r=0.91*) and SPAD (r=0.92**). With respect to nutrition study correlation coefficients of traits with fodder yield revealed positive correlation growth components like shoot length (r=0.90**), root length (r=0.74**), seedling vigour (r=0.86**) and dry matter accumulation (0.97**), yield parameters like Total fresh weight of fodder (r=0.96**), individual fresh weight of plant (r=0.95**) and individual dry weight of plant (r=0.97**), physiological parameters like SPAD (r=0.78**) and WUE (r=1.00**), quality parameters like nitrogen content, crude protein, crude fibre and total carbohydrates (r=0.927**, 0.927**, 0.770* and 0.905**, respectively). The study revealed significantly higher fodder yield with two combined sprays of urea and MOP each @ 1 per cent at 3rd and 10th Days after sowing (DAS) with 2.5 kg m-2 under hydroponic conditions.

Weight Update Skipping: Reducing Training Time for Artificial Neural Networks

Artificial Neural Networks (ANNs) are known as state-of-the-art techniques in Machine Learning (ML) and have achieved outstanding results in data-intensive applications, such as recognition, classification, and segmentation. These networks mostly use deep layers of convolution and/or fully connected layers with many filters in each layer, demanding a large amount of data and tunable hyperparameters to achieve competitive accuracy. As a result, storage, communication, and computational costs of training (in particular time spent for training) become limiting factors to scale them up. In this paper, we propose a new training methodology for ANNs that exploits the observation of improvement of accuracy shows temporal variations which allow us to skip updating weights when the variation is minuscule. During such time windows, we keep updating bias which ensures the network still trains and avoids overfitting; however, we selectively skip updating weights (and their time-consuming computations). This training approach virtually achieves the same accuracy with considerably less computational cost and reduces the time spent on training. We developed two variations of the proposed training method for selectively updating weights, and call them as i) Weight Update Skipping (WUS), and ii) Weight Update Skipping with Learning Rate Scheduler (WUS+LR). We evaluate these two approaches by analyzing state-of-the-art models, including AlexNet, VGG-11, VGG-16, ResNet-18 on CIFAR datasets. We also use ImageNet dataset for AlexNet, VGG-16, and Resnet-18. On average, WUS and WUS+LR reduced the training time (compared to the baseline) by 54%, and 50% on CPU and 22%, and 21% on GPU, respectively for CIFAR-10; and 43% and 35% on CPU and 22%, and 21% on GPU, respectively for CIFAR-100; and finally 30% and 27% for ImageNet, respectively.

A framework for occupancy prediction based on image information fusion and machine learning

It is of great energy-saving significance to provide accurate occupancy schedules for building energy simulation. Inaccurate occupancy information can lead to biased results. However, the reference occupancy schedules provided by the standard documents are fixed and differ significantly from the actual occupancy fluctuations. Moreover, the large amount of accurate occupancy information is difficult and expensive to obtain, limiting the development of occupancy fluctuation pattern exploration and occupancy prediction. In this paper, a framework is proposed to provide accurate and flexible occupancy schedules based on occupancy-related variables and occupancy information extracted from surveillance videos through Convolutional Neural Network-based density estimation methods. Machine learning algorithms are used in the framework to predict occupancy owing to their powerful predictive capabilities. A case study based on educational buildings is conducted to examine the performance of the framework and to compare its predicted occupancy schedules with those provided by national standard documents. The average recognition accuracy of the proposed framework for different crowd densities is as high as 95.67%, and the mean absolute percentage error for the prediction of the occupancy by the proposed framework is only 16.88%. The average deviation degree between the predicted occupancy rate schedules and the actual occupancy rate schedules is reduced from 676.08% to 23.19%. The average deviation degree of building energy simulation results caused by the occupancy rate schedule is reduced from 42.56% to 0.93%. These results provide conclusive evidence that the proposed framework can provide realistic and flexible occupancy schedules to improve the accuracy of building energy simulation.

Labor supply responses to income tax free and bracket expansions

PurposeAn important concern of economic policy analysis is how income taxes affect labor supply since this is crucial in assessing the efficiency costs of taxation and designing labor income taxation. The focus in the literature has been mostly to study the responses of high earners and women. The authors contribute to this literature by focusing more on how middle earners respond to financial incentives and whether the responses are different between men and women.Design/methodology/approachThe authors exploit substantial expansions in the level of individual income exempt from taxation and taxed at a lower marginal tax rate while the schedule of marginal tax rates remained the same. The authors adopt an empirical framework that is similar to Bosch and van der Klaauw (2012) and condition on the effects of other factors, such as inflows of foreign workers that may have affected the wages, participation and working hours of native males and females. The authors also conduct various sensitivity analyses to examine the robustness of the estimates.FindingsThe authors find robust evidence that the tax reforms increased the wages of medium and high educated married males and females significantly. They also had a positive impact on work participation that was more substantial for married women, especially the medium educated. The authors estimate significant positive own wage labor supply elasticities that are small and about the same for men and women when the authors condition on the labor outcome effects of inflows of EU and non-EU foreign workers, which changed the skill distribution of the economy and had a more significant impact on female labor outcomes. Smaller wage labor supply elasticities indicate lower disincentive effects and deadweight losses from the imposition of taxes and have implications on the design of optimal taxation of men and women.Originality/valuePrevious investigations of the labor supply responses of both men and women to a given policy change have been identified mostly by exploiting changes in joint income taxation and marginal tax rates. The authors exploit substantial expansions in the level of individual income exempt from taxation and taxed at a lower marginal tax rate while the schedule of marginal tax rates remained the same. The income effects of these reforms could be limited since the reduced marginal tax rates apply to only part of the income.

Application of System Reliability Indices in Electric Power System

System reliability indices deal with the adequacy of overall system supply and indicates the system behaviour and response. The index express interruption statistics in terms of the customers which can be an individual, firm, or organization that purchases electric services at one location under one rate classification, contract or schedule. The system reliability indices include: System Average Interruption Frequency Index (SAIFI), System Average Interruption Duration Index (SAIDI), Customer Average Interruption Duration Index (CAIDI), Average Service Availability Index (ASAI), Average Service Unavailability Index (ASUI), Expected Energy Not Supplied (EENS) and Average Energy Not Supplied (AENS). These indices are used as performance evaluation for power system reliability assessment. This assessment helps in system planning for long and short terms. This study therefore focuses reviewed past study on application of SAIFI, SAIDI and CAIDI for system reliability assessment on electric power system. These indices imply how often an average customer experiences sustained interruption over a predefined time of year and they are most used reliability indices for power system. Keywords: Reliability Indices, SAIDI, SAIFI, CAIDI, ASAI, ASUI, AENS, Reliability Assessment. DOI: 10.7176/JETP/11-6-03 Publication date: November 30 th 2021

Energy efficiency and economic analysis of variable frequency drive and variable pitch system: A case study of axial fan in hospital

Most hospitals use variable air volume systems to reduce their energy consumption while effectively responding to the frequently varying demands of cooling and heating. In this study, we applied the variable frequency drive (VFD) and variable pitch (VP) fan control methods to air handling units with axial fans in a tertiary hospital in Seoul, South Korea, and compared their power consumption and operating efficiencies according to the annual outdoor air intake rate schedule. The results of the comparison indicated that the VFD in the low-fan-output section and the VP method in the high-fan-output section consumed relatively less power. Concerning the economic feasibility analysis, we compared the annual operation and maintenance costs of the VFD and VP systems applied to the Air Handling Units (AHUs) in the hospital. In all cases, the annual energy consumption was lower in the VP than in the VFD; however, the VP was relatively high in maintenance and overhaul. It was shown that the VP is a more efficient control method due to the high-output operation of the AHUs for patient rooms. For other AHUs, the payback period was between 9 and 16 years when changing from VP to VFD. • Variable pitch control was energy conservative for the air handler operating at the high flow rate. • Variable frequency drive was energy efficient for the air handler at the low flow rate. • Operating cost of the variable pitch control was lower than that of the variable-frequency drive. • Maintenance cost of the variable pitch control was higher than that of the variable-frequency drive.

Marc Richelle (1930–2021) and the Study of Temporal Regulation of Behaviour in Animals

This article discusses the contribution of Marc Richelle to the study of temporal regulation of behaviour in animals. Richelle was a pioneer of behavioural pharmacology in Europe in the 1960s, and some of his early pharmacological experiments, particular those involving chlordiazepoxide, are discussed. Richelle frequently tested drug effects on performance on fixed-interval (FI) and differential reinforcement of low rate (DRL) schedules. Much of his later work, conducted with Helga Lejeune, involved cross-species comparisons of performance on FI and DRL, and often focused on potential differences between “timing competence” and “timing performance”. His work provided an unrivalled body of research on operant behaviour in different species, involving research on animals as different as cats and fish. Much of the work was reviewed in Richelle and Lejeune’s 1980 book Time in Animal Behaviour, which contained particularly influential accounts of collateral behaviour and inter-species comparisons.

AccuLearning: A User-Friendly Deep Learning Auto-Segmentation Platform for Radiotherapy

Despite the significant success of deep learning (DL) techniques in auto-segmentation of medical images, the application is often limited by various technical difficulties which are costly to implement for medical researchers without any programming experience. To address these issues, we developed a user-friendly deep learning platform, i.e., AccuLearning, to provide accessible DL resources for clinical application to researchers in medical field.The goal of AccuLearning is to simplify the DL model building workflow, hide the complexities of model training, and provide highly integrated, efficient and flexible platform for the implementation of deep learning auto-segmentation technique. To do this, all the major components were integrated into an automatic pipeline. Gradient-based adaptive network structure building and targeted loss function make the training process of the model easier and more stable. To make the platform generalizable, adaptive image pre-processing strategies were employed, such as automatic dataset characteristics extraction and standardization. With an easy-to-use graphical user interface and optimized default parameters, minimal user intervention is needed. To demonstrate the effectiveness, we applied AccuLearning on a total of 9 auto-segmentation tasks using open-access medical image datasets, covering 64 ROIs (including 4 GTVs and 60 OARs) on various anatomy sites (abdomen, head and neck, and pelvis) on CTs or MRIs. The model performance was evaluated using Dice and HD95 compared with ground truth. In addition, the impact of number of training cases were also evaluated.AccuLearning can achieve competitive model accuracy using default settings, with less than 3 hours of training time and 45 seconds of average inference time per patient. The average Dice and HD were 0.81 ± 0.11 and 7.7 ± 7.4mm for all the OARs. 18.3% (11/60) of the OARs have Dice larger than 0.9, while 66.7% (40/60) larger than 0.8. For HD95, 26.7% (16/60) of the OARs have HD95 smaller than 3mm and 73.3% (44/60) smaller than 10mm. The DL model trained with only 20 cases showed no significant difference with the model trained with 160 cases in OARs segmentation. For nasopharyngeal carcinoma GTV auto-segmentation, the average Dice was 0.62. Through adjustments to model dimensions, loss function and learning rate schedule, the average Dice for GTV was increased to 0.69, which is reasonably comparable to the inter-observer variations Dice 0.71.AccuLearning can build robust and accurate DL models for different auto-segmentation tasks on both CT and MRI with minimal user interventions, short model training time and low demand of training sample size. The platform can be implemented into current workflow, enabling fast and accurate target and OARs segmentation for clinical practice in radiotherapy field.