Full Batch Research Articles

Efficient dynamic sampling of batch processes through operation recipes

The complexity of dynamic phenomena present in chemical processes often results in high evaluation costs of accurate first-principles models. This limits their real-time applicability, e.g. for advanced process control. A common solution is the derivation of simpler but faster data-driven surrogate models trained on simulated time series generated from dynamic samplings of a mechanistic model. For batch processes, known non-adaptive dynamic sampling methods lead to unrealistic or even infeasible operation cycles, raising the cost of generating simulated datasets with sufficient information content to train accurate surrogate models. An alternative sampling strategy is developed and analyzed, where sampled input trajectories are constrained to process knowledge in the form of parametrized operation recipes. The proposed methodology is tested for the case studies of full batch cycles of a crystallizer and a batch distillation column, showing that it is more efficient in terms of convergent simulations compared to an established dynamic sampling strategy.

An Online Service Performance Prediction Learning Method

In order to improve the quality of service operations, it is necessary to take the initiative to prevent service failures and service performance fluctuations, instead of triggering handlers when service errors occur. Effective prediction and analysis of the large-scale services performance is an effective and feasible proactive prevention tool. However, the traditional service performance prediction model mostly adopts the full batch training mode, it is difficult to meet the real-time requirements of large-scale service calculation. Based on the comprehensive trade-off between the method of full batch learning and the stochastic gradient descent method, a large-scale service performance prediction model is established based on online learning, and a service performance prediction method is proposed based on small batch online learning. Through properly setting the batch parameters, the proposed approach only need to train the sample data with small batches in one iteration, the time efficiency is improved for large-scale service performance prediction.

Towards Discriminant Analysis Classifiers Using Online Active Learning via Myoelectric Interfaces

We propose a discriminant analysis (DA) classifier that uses online active learning to address the need for the frequent training of myoelectric interfaces due to covariate shift. This online classifier is initially trained using a small set of examples, and then updated over time using streaming data that are interactively labeled by a user or pseudo-labeled by a soft-labeling technique. We prove, theoretically, that this yields the same model as training a DA classifier via full batch learning. We then provide experimental evidence that our approach improves the performance of DA classifiers and is robust to mislabeled data, and that our soft-labeling technique has better performance than existing state-of-the-art methods. We argue that our proposal is suitable for real-time applications, as its time complexity w.r.t. the streaming data remains constant.

Convergence analysis of the discrete consensus-based optimization algorithm with random batch interactions and heterogeneous noises

We present stochastic consensus and convergence of the discrete consensus-based optimization (CBO) algorithm with random batch interactions and heterogeneous external noises. Despite the wide applications and successful performance in many practical simulations, the convergence of the discrete CBO algorithm was not rigorously investigated in such a generality. In this work, we introduce a generalized discrete CBO algorithm with a weighted representative point and random batch interactions, and show that the proposed discrete CBO algorithm exhibits stochastic consensus and convergence toward the common equilibrium state exponentially fast under suitable assumptions on system parameters. For this, we recast the given CBO algorithm with random batch interactions as a discrete consensus model with a random switching network topology, and then we use the mixing property of interactions over sufficiently long time interval to derive stochastic consensus and convergence estimates in mean square and almost sure senses. Our proposed analysis significantly improves earlier works on the convergence analysis of CBO models with full batch interactions and homogeneous external noises.

Usefulness of Automatic Hyperparameter Optimization in Developing Radiation Emulator in a Numerical Weather Prediction Model

To improve the forecasting accuracy of a radiation emulator in a weather prediction model over the Korean peninsula, the learning rate used in neural network training was automatically optimized using the Sherpa. The Sherpa experiment results were compared with two control simulation results using learning rates of 0.0001 and 1 for different batch sizes (full to 500). In the offline evaluation, the Sherpa results showed significant improvements in predicting longwave/shortwave heating rates and fluxes compared to the lowest learning rate results, whereas the improvements compared to the highest learning rate were relatively small because the optimized values by the Sherpa were 0.4756–0.6656. The online evaluation results over one month, which were linked with the weather prediction model, demonstrated the usefulness of Sherpa on a universal performance for the radiation emulator. In particular, at the full batch size, Sherpa contributed to reducing the one-week forecast errors for longwave/shortwave fluxes, skin temperature, and precipitation by 39–125%, 137–159%, and 24–26%, respectively, compared with the two control simulations. Considering the widespread use of parallel learning based on full batch, Sherpa can contribute to producing robust results regardless of batch sizes used in neural network training for developing radiation emulators.

A qualitative investigation exploring why dance festivals are risky environments for drug use and potential adverse outcomes

BackgroundDance festivals have been shown to be high-risk events for use of drugs such as ecstasy/MDMA and possible adverse effects associated with use. However, few studies have examined what makes festivals such risky environments. We aimed to determine festival-specific risk factors for adverse outcomes related to drug use.MethodsIn-depth interviews were conducted with 35 key informants in North America who deemed themselves experts in new psychoactive substances, and identified as drug checkers, sellers, or experienced users. Interviews were coded in an inductive manner, and we conducted thematic analysis to identify relevant themes.ResultsWe identified four main themes focusing on festival attendance as a risk factor for risky drug use and related outcomes: attendees inexperienced with electronic dance music parties and party drugs, risky drug purchasing, risky drug use practices, and festival-specific environmental risk factors. Festivals attract a wide array of people not experienced with party drugs, yet drugs like ecstasy are commonly sought by such individuals inside festivals. Relying on strangers inside to purchase drugs is a risk factor for purchasing adulterated product. Fear of security/police at festivals leads to risky drug-taking such as ingesting one’s full batch of drugs at the entrance. These risks are compounded by environmental factors including crowding, hot temperature, and lack of water (which lead to dehydration), long/consecutive event days (which can lead to exhaustion), and inadequate medical emergency response.ConclusionsWe determined modifiable risk factors which can both inform future research and future prevention and harm reduction efforts in this scene.

Minimal Model Structure Analysis for Input Reconstruction in Federated Learning

\ac{fl} proposed a distributed \ac{ml} framework where every distributed worker owns a complete copy of global model and their own data. The training is occurred locally, which assures no direct transmission of training data. However, the recent work \citep{zhu2019deep} demonstrated that input data from a neural network may be reconstructed only using knowledge of gradients of that network, which completely breached the promise of \ac{fl} and sabotaged the user privacy. In this work, we aim to further explore the theoretical limits of reconstruction, speedup and stabilize the reconstruction procedure. We show that a single input may be reconstructed with the analytical form, regardless of network depth using a fully-connected neural network with one hidden node. Then we generalize this result to a gradient averaged over batches of size $B$. In this case, the full batch can be reconstructed if the number of hidden units exceeds $B$. For a \ac{cnn}, the number of required kernels in convolutional layers is decided by multiple factors, e.g., padding, kernel and stride size, etc. We require the number of kernels $h\geq (\frac{d}{d^{\prime}})^2C$, where we define $d$ as input width, $d^{\prime}$ as output width after convolutional layer, and $C$ as channel number of input. We validate our observation and demonstrate the improvements using bio-medical (fMRI, \ac{wbc}) and benchmark data (MNIST, Kuzushiji-MNIST, CIFAR100, ImageNet and face images).

Effect of electrode rinse solutions on the electrodialysis of concentrated salts

The data presented indicate that modest attention to the chemistry of the electrode rinse solution can yield as much as 56% improvement in process efficiency. When operating electrodialysis (ED) with salt concentrations greater than 10,000 mg/l (as NaCl) the amperage can be relatively high even at low voltages. To maximize the overall rate of ion transport from diluate to concentrate, there is a need to minimize resistances in the electrode cells since these can represent as much as 30% of the total resistances in the entire process and in this particular ED unit, represented resistance roughly equivalent to that within the membrane stack. This work describes the performance of a 10-cell pair, 200 cm2 (0.02 m2) per membrane, pilot ED unit operated in batch mode at 5 V potential. The feedstock to the stack was varied from 0.5% to 6% NaCl. Results from Volt-amp profiles (2.0–15 V) were used as the rationale for choosing the preferred electrode rinse solution, 90 g/l (kg/m3) disodium sulfate at pH 12.5. The recommended solution of 30 g/l (kg/m3) disodium sulfate at neutral pH was tested against stronger solutions (60, 90, and 120 g/l (kg/m3) disodium sulfate) at neutral pH and with 1 g/l (kg/m3) sodium hydroxide added to yield solutions at pH 12.5. There were stark differences in the performance of the ED unit between the solutions at pH 7 and those at pH 12.5. The slopes of the Volt-amp profiles improved indicating less resistance to ion flow at pH 12.5. Most of the difference can be attributed to the greater conductivity of sodium hydroxide compared to disodium sulfate, where even a modest addition of 1 g/l (kg/m3) sodium hydroxide increased the conductivity of the solution between 8 and 20% depending on the concentration of disodium sulfate. Most notably, the initiation voltage was 2.39 ± 0.08 V with solutions at pH 7 and 1.99 V ± 0.04 V when the rinse solutions were around pH 12.5, a difference of around 0.42 V. Since the full batch runs were performed at 5.0 V, this 0.42 V differential represented a loss of 8% in process efficiency. A series of tests showed a shift in initiation voltage occurred between pH 11.5 and 12.5. Nernst equations for water electrolysis are coupled with a simple flux model that estimates the concentration of hydroxide and hydronium ions at the electrode surfaces. This model indicates that the potential demanded by the electrodes is directly linked to the current. Furthermore, there is a steep drop in the potential demanded by the anode when there is sufficient hydroxide available to generate oxygen from hydroxide as opposed to generation from water. The voltage demand is minimal at pH 12.5 and reflective of the reference standard voltage (just over 1.23 V predicted at 0.1 amps). However, the flux model predicts that the voltage demanded by the electrodes at pH 7 is 1.61 V at 0.1 amps (the lowest amperage measurable with the power pack). Comparing this result to the reference standard (E0 = −1.23 V) yields a satisfactory explanation for the observed difference below reference standard of 0.42 V.

Convergence toward equilibrium of the first-order consensus model with random batch interactions

We study convergence analysis of the first-order stochastic consensus model with random batch interactions. The proposed model can be obtained via random batch method (RBM) from the first-order nonlinear consensus model. This model has two competing mechanisms, namely intrinsic free flow and nonlinear consensus interaction terms. From the competition between the two mechanisms, the original (full batch) model can admit relative equilibria and relaxation of the dynamics to the relative equilibrium in a large coupling regime. In authors' earlier work, we have studied the RBM approximation and its uniform error analysis. In this paper, we present two convergence analysis of RBM solutions toward the relative equilibrium. More precisely, we show that the variances of displacement processes between the full batch and random batch solutions tend to zero exponentially fast, as time goes to infinity. Second, we also show that, almost surely, the diameter process of displacement tends to zero exponentially.

In-line cosmetic end-point detection of batch coating processes for colored tablets using multivariate image analysis

In this study, an in-line Process Analytical Technology (PAT) for cosmetic (non-functional) coating unit operations is developed using images of the tablet bed acquired in real-time by an inexpensive industrial camera and lighting system. The cosmetic end-point of multiple batches, run under different operating conditions, is automatically computed from these images using a Multivariate Image Analysis (MIA) methodology in conjunction with a stability determination strategy. The end-points detected by the algorithm differed, on average, by 3% in terms of total batch time from those identified visually by a trained operator. Since traditional practice typically relies on a coating overage to ensure full batch aspect homogeneity in the face of disturbances, the current in-line method can be used to reduce coating material and processing time (over 40% for the operating policy adopted in this work). Additionally, monitoring of the color features calculated by the algorithm allowed the identification of abnormal process conditions affecting visible coating uniformity. This work also addresses practical challenges related to image acquisition in the harsh environment of a pan coater, bringing this tool closer to a state of maturity for implementation in production units and opening the path for their optimization, monitoring, and automatic control.

FedPD: A Federated Learning Framework With Adaptivity to Non-IID Data

Federated Learning (FL) has become a popular paradigm for learning from distributed data. To effectively utilize data at different devices without moving them to the cloud, algorithms such as the Federated Averaging (FedAvg) have adopted a "computation then aggregation" (CTA) model, in which multiple local updates are performed using local data, before sending the local models to the cloud for aggregation. However, these schemes typically require strong assumptions, such as the local data are identically independent distributed (i.i.d), or the size of the local gradients are bounded. In this paper, we first explicitly characterize the behavior of the FedAvg algorithm, and show that without strong and unrealistic assumptions on the problem structure, the algorithm can behave erratically for non-convex problems (e.g., diverge to infinity). Aiming at designing FL algorithms that are provably fast and require as few assumptions as possible, we propose a new algorithm design strategy from the primal-dual optimization perspective. Our strategy yields a family of algorithms that take the same CTA model as existing algorithms, but they can deal with the non-convex objective, achieve the best possible optimization and communication complexity while being able to deal with both the full batch and mini-batch local computation models. Most importantly, the proposed algorithms are {\it communication efficient}, in the sense that the communication pattern can be adaptive to the level of heterogeneity among the local data. To the best of our knowledge, this is the first algorithmic framework for FL that achieves all the above properties.

Microwave cell lysis for ADDA‐ELISA analysis of total microcystins and nodularins

AbstractA fast and effective microwave treatment procedure was developed for the ADDA–enzyme‐linked immunosorbent assay analysis of total extracellular and intracellular microcystins and nodularins in drinking water and ambient water. This microwave treatment procedure for a full batch of samples required 10–20 min versus the 3–4 hr typically needed for three freeze/thaw cycles. The obtained lowest‐concentration minimum reporting level and method detection limit were 0.21 and 0.09 μg/L microcystin‐LR (MC‐LR) equivalents, respectively. Mean recoveries of 80%–92% with relative standard deviations of 4.2%–9.0% were obtained for all studied water matrices fortified at concentrations of 0.3–2.0 μg/L MC‐LR. Compared with US Environmental Protection Agency Method 546, this alternate procedure provided more effective cell lysis and subsequently detected significantly higher concentrations of total microcystins and nodularins in a few water samples collected during harmful algal blooms, such as 61.0 versus 34.4 μg/L MC‐LR equivalents in Lake Erie and 11.5 versus 9.1 μg/L MC‐LR equivalents in an Indiana reservoir.

Two-agent scheduling on mixed batch machines to minimise the total weighted makespan

This paper studies a two-agent scheduling problem on mixed batch machines in parallel. A mixed batch machine can process several jobs simultaneously as a batch, as long as the number of jobs in the batch does not exceed the machine capacity. The processing time of a mixed batch is the weighted sum of the maximum processing time and the total processing time of jobs in the batch. The objective is to minimise the weighted sum of two agents' makespans. We present four approximation algorithms based on two strategies: the machine-centric strategy and the agent-centric strategy. For each strategy, a full batch longest processing time (FBLPT) rule and a longest processing time greedy (LPTG) rule are used. We conduct theoretical analyses based on the worst-case performance ratio to provide the provable guarantees on the performances of the algorithms, and simulation analyses based on randomly generated instances to evaluate the average performances of the algorithms. Furthermore, we verify the consistency between the theoretical and simulation results. The algorithms using agent-centric strategy perform better than ones using machine-centric strategy. Finally, we provide managerial insights for the problem by analysing the technological parameters of batches, importance of agents, and demand seasonality.

IN2LAAMA: Inertial Lidar Localization Autocalibration and Mapping

In this article, we present inertial lidar localization autocalibration and mapping: an offline probabilistic framework for localization, mapping, and extrinsic calibration based on a 3-D lidar and a six-degree-of-freedom inertial measurement unit. Most of today's lidars collect geometric information about the surrounding environment by sweeping lasers across their field of view. Consequently, 3-D points in one lidar scan are acquired at different timestamps. If the sensor trajectory is not accurately known, the scans are affected by the phenomenon known as motion distortion. The proposed method leverages preintegration with a continuous representation of the inertial measurements to characterize the system's motion at any point in time. It enables precise correction of the motion distortion without relying on any explicit motion model. The system's pose, velocity, biases, and time shift are estimated via a full batch optimization that includes automatically generated loop closure constraints. The autocalibration and the registration of lidar data rely on planar and edge features matched across pairs of scans. The performance of the framework is validated through simulated and real-data experiments.

Development of a fully automated low-pressure [11 C]CO carbonylation apparatus.

[11C]carbon monoxide ([11C]CO) is a versatile synthon for radiolabeling of drug‐like molecules for imaging studies with positron emission tomography (PET). We here report the development of a novel, user‐friendly, fully automated, and good manufacturing practice (GMP) compliant low‐pressure synthesis module for 11C‐carbonylation reactions using [11C]CO. In this synthesis module, [11C]CO was reliably prepared from cyclotron‐produced [11C]carbon dioxide ([11C]CO2) by reduction over heated molybdenum and delivered to the reaction vessel within 7 min after end of bombardment, with an overall radiochemical yield (RCY) of 71%. [11C]AZ13198083, a histamine type‐3 receptor ligand, was used as a model compound to assess the functionality of the radiochemistry module. At full batch production conditions (55 μA, 30 min), our newly developed low‐pressure 11C‐carbonylation apparatus enabled us to prepare [11C]AZ13198083 in an isolated radioactivity of 8540 ± 1400 MBq (n = 3). The radiochemical purity of each of the final formulated batches exceeded 99%, and all other quality control tests results conformed with specifications typically set for carbon‐11 labeled radiopharmaceuticals. In conclusion, this novel radiochemistry system offers a convenient GMP compliant production drugs and radioligands for imaging studies in human subjects.

Improved Approaches to Minimize the Makespan on Single-Machine Scheduling with Periodic Preventive Maintenance Activities

This paper addresses a single-machine scheduling problem with periodic preventive maintenance activities that are predeterministic so that the machine is not available all the time, and jobs have to be processed between two consecutive maintenance periods. We propose a mixed integer programming (MIP) model and two heuristics to minimize the makespan. With more constraints in our model, the model is more efficient than the recent model of Perez-Gonzalez and Framinan , and our model could solve problems with up to fifty jobs. Two heuristic algorithms, namely, H (MW) and H (LB∗), are also proposed, in which two bin-packing policies of the minimum waste and minimum lower bound are used, respectively. Furthermore, we also proposed an improvement procedure. The results showed that the heuristic H (MW) outperformed other heuristics of the paper, indicating that the bin-packing policy of the minimum waste is more effective than well-known ones such as full batch and best fit. Additionally, all the heuristic algorithms addressed in this paper combined with the improvement procedure could achieve a similar and high quality of solutions with a very tiny increase in computational expense.

Stochastic calibration of radio interferometers

ABSTRACTWith ever-increasing data rates produced by modern radio telescopes like LOFAR and future telescopes like the SKA, many data-processing steps are overwhelmed by the amount of data that needs to be handled using limited compute resources. Calibration is one such operation that dominates the overall data processing computational cost; none the less, it is an essential operation to reach many science goals. Calibration algorithms do exist that scale well with the number of stations of an array and the number of directions being calibrated. However, the remaining bottleneck is the raw data volume, which scales with the number of baselines, and which is proportional to the square of the number of stations. We propose a ‘stochastic’ calibration strategy where we read only in a mini-batch of data for obtaining calibration solutions, as opposed to reading the full batch of data being calibrated. None the less, we obtain solutions that are valid for the full batch of data. Normally, data need to be averaged before calibration is performed to accommodate the data in size-limited compute memory. Stochastic calibration overcomes the need for data averaging before any calibration can be performed, and offers many advantages, including: enabling the mitigation of faint radio frequency interference; better removal of strong celestial sources from the data; and better detection and spatial localization of fast radio transients.

Numerical Study on Transient Behaviour of Finite Bulk Arrival or Service Queues with Multiple Working Vacations

The transient behaviour of single server bulk queues with finite-buffer capacity is discussed numerically under the assumption that the server may take multiple working vacations after completing a busy period. It is considered that the inter-arrival and the service times are exponentially distributed and independent of each other. The buffer space is limited, therefore partial and full batch rejection policies are studied for the bulk arrival queueing model. However, for the batch service, the general bulk service rule is considered. Using probabilistic arguments and relating the state of the systems at two consecutive time epochs, differential equations are obtained to model such phenomena. Further, these equations are solved numerically by Runge-Kutta method and the time dependent numerical solutions are compared with the exact stationary solutions. The blocking probability and the mean waiting time of the first, last and an arbitrary customer are also reviewed mathematically and computed numerically.

On the information bottleneck theory of deep learning**This article is an updated version of: Saxe A M, Bansal Y, Dapello J, Advani M, Kolchinsky A, Tracey B D and Cox D D 2018 On the information bottleneck theory of deep learning International Conference on Learning Representations (ICLR 2018).

The practical successes of deep neural networks have not been matched by theoretical progress that satisfyingly explains their behavior. In this work, we study the information bottleneck (IB) theory of deep learning, which makes three specific claims: first, that deep networks undergo two distinct phases consisting of an initial fitting phase and a subsequent compression phase; second, that the compression phase is causally related to the excellent generalization performance of deep networks; and third, that the compression phase occurs due to the diffusion-like behavior of stochastic gradient descent. Here we show that none of these claims hold true in the general case, and instead reflect assumptions made to compute a finite mutual information metric in deterministic networks. When computed using simple binning, we demonstrate through a combination of analytical results and simulation that the information plane trajectory observed in prior work is predominantly a function of the neural nonlinearity employed: double-sided saturating nonlinearities like yield a compression phase as neural activations enter the saturation regime, but linear activation functions and single-sided saturating nonlinearities like the widely used ReLU in fact do not. Moreover, we find that there is no evident causal connection between compression and generalization: networks that do not compress are still capable of generalization, and vice versa. Next, we show that the compression phase, when it exists, does not arise from stochasticity in training by demonstrating that we can replicate the IB findings using full batch gradient descent rather than stochastic gradient descent. Finally, we show that when an input domain consists of a subset of task-relevant and task-irrelevant information, hidden representations do compress the task-irrelevant information, although the overall information about the input may monotonically increase with training time, and that this compression happens concurrently with the fitting process rather than during a subsequent compression period.

Mixed batch scheduling on identical machines

This paper studies a new mixed batch scheduling problem that arises in vacuum heat treatment. A mixed batch machine can process at most a given number of jobs simultaneously. The processing time of a batch is the weighted sum of the maximum processing time and the total processing time of jobs in the batch. The objective is to minimize the makespan. We first prove that the problem on a single machine can be solved in polynomial time, while the problem on multiple identical machines is NP-hard. Then, we develop a pseudopolynomial time exact algorithm when the number of machines is fixed. Further, we analyze the worst-case performance ratio of a full batch longest processing time algorithm and design Algorithm LPT-Greedy with improved worst-case performance.