Random Initialization Research Articles

On Minimizers and Convolutional Filters: Theoretical Connections and Applications to Genome Analysis.

Minimizers and convolutional neural networks (CNNs) are two quite distinct popular techniques that have both been employed to analyze categorical biological sequences. At face value, the methods seem entirely dissimilar. Minimizers use min-wise hashing on a rolling window to extract a single important k-mer feature per window. CNNs start with a wide array of randomly initialized convolutional filters, paired with a pooling operation, and then multiple additional neural layers to learn both the filters themselves and how they can be used to classify the sequence. In this study, our main result is a careful mathematical analysis of hash function properties showing that for sequences over a categorical alphabet, random Gaussian initialization of convolutional filters with max-pooling is equivalent to choosing a minimizer ordering such that selected k-mers are (in Hamming distance) far from the k-mers within the sequence but close to other minimizers. In empirical experiments, we find that this property manifests as decreased density in repetitive regions, both in simulation and on real human telomeres. We additionally train from scratch a CNN embedding of synthetic short-reads from the SARS-CoV-2 genome into 3D Euclidean space that locally recapitulates the linear sequence distance of the read origins, a modest step toward building a deep learning assembler, although it is at present too slow to be practical. In total, this article provides a partial explanation for the effectiveness of CNNs in categorical sequence analysis.

Novel Initialization Functions for Metaheuristic-Based Online Virtual Network Embedding

Virtual network embedding (VNE) is the process of allocating resources in a substrate (i.e. physical) network to support virtual networks optimally. The VNE problem is an NP-hard problem that has been studied for more than a decade in the continuous seek to maximize the revenue of physical infrastructures with more efficient VNE solutions. Metaheuristics have been widely used in online VNE as they incorporate mechanisms to avoid local optimum solutions, explore larger search spaces, and keep acceptable execution times. All metaheuristic optimization algorithms require initialization for which the vast majority of online VNE solutions implement random initialization. This paper proposes three novel initialization functions namely, Initialization Based on Node Selection (IFNS), Initialization Function Based on Community Detection (IFCD), and Initialization Function Based on Previous Solutions (IFPS), intending to enhance the performance of the online VNE process. Through simulation, our initialization functions have been proven to enhance the acceptance rate, revenue, and revenue-to-cost metrics of the VNE process. The enhancements achieved by our initialization functions are statistically significant and their implementation does not add computational overhead to the classic VNE approaches.

A Sinh-Cosh-Enhanced DBO Algorithm Applied to Global Optimization Problems.

The Dung beetle optimization (DBO) algorithm, devised by Jiankai Xue in 2022, is known for its strong optimization capabilities and fast convergence. However, it does have certain limitations, including insufficiently random population initialization, slow search speed, and inadequate global search capabilities. Drawing inspiration from the mathematical properties of the Sinh and Cosh functions, we proposed a new metaheuristic algorithm, Sinh-Cosh Dung Beetle Optimization (SCDBO). By leveraging the Sinh and Cosh functions to disrupt the initial distribution of DBO and balance the development of rollerball dung beetles, SCDBO enhances the search efficiency and global exploration capabilities of DBO through nonlinear enhancements. These improvements collectively enhance the performance of the dung beetle optimization algorithm, making it more adept at solving complex real-world problems. To evaluate the performance of the SCDBO algorithm, we compared it with seven typical algorithms using the CEC2017 test functions. Additionally, by successfully applying it to three engineering problems, robot arm design, pressure vessel problem, and unmanned aerial vehicle (UAV) path planning, we further demonstrate the superiority of the SCDBO algorithm.

Twin support vector machines based on chaotic mapping dung beetle optimization algorithm

Abstract Twin Support Vector Machine (TSVM) is a powerful machine learning method that is usually used to solve binary classification problems. But although the classification speed and performance of TSVM is better than that of primitive support vector machine, TSVM still faces the problem of difficult parameter selection; therefore, to overcome the problem of parameter selection of TSVM, this paper proposes a Chaotic Mapping Dung Beetle Optimization Algorithm-based Twin Support Vector Machine (CMDBO-TSVM) for automatic parameter selection. Due to the uncertainty of the random initialization population of the original Dung Beetle Optimization Algorithm, this paper additionally adds chaotic mapping initialization to improve the Dung Beetle Optimization Algorithm. Experiments on the dataset through this paper show that the classification accuracy of the CMDBO-TSVM has a better performance.

On the Initialization of Swarm Intelligence Algorithms for Vector Quantization Codebook Design.

Vector Quantization (VQ) is a technique with a wide range of applications. For example, it can be used for image compression. The codebook design for VQ has great significance in the quality of the quantized signals and can benefit from the use of swarm intelligence. Initialization of the Linde-Buzo-Gray (LBG) algorithm, which is the most popular VQ codebook design algorithm, is a step that directly influences VQ performance, as the convergence speed and codebook quality depend on the initial codebook. A widely used initialization alternative is random initialization, in which the initial set of codevectors is drawn randomly from the training set. Other initialization methods can lead to a better quality of the designed codebooks. The present work evaluates the impacts of initialization strategies on swarm intelligence algorithms for codebook design in terms of the quality of the designed codebooks, assessed by the quality of the reconstructed images, and in terms of the convergence speed, evaluated by the number of iterations. Initialization strategies consist of a combination of codebooks obtained by initialization algorithms from the literature with codebooks composed of vectors randomly selected from the training set. The possibility of combining different initialization techniques provides new perspectives in the search for the quality of the VQ codebooks. Nine initialization strategies are presented, which are compared with random initialization. Initialization strategies are evaluated on the following algorithms for codebook design based on swarm clustering: modified firefly algorithm-Linde-Buzo-Gray (M-FA-LBG), modified particle swarm optimization-Linde-Buzo-Gray (M-PSO-LBG), modified fish school search-Linde-Buzo-Gray (M-FSS-LBG) and their accelerated versions (M-FA-LBGa, M-PSO-LBGa and M-FSS-LBGa) which are obtained by replacing the LBG with the accelerated LBG algorithm. The simulation results point out to the benefits of the proposed initialization strategies. The results show gains up to 4.43 dB in terms of PSNR for image Clock with M-PSO-LBG codebooks of size 512 and codebook design time savings up to 67.05% for image Clock, with M-FF-LBGa codebooks with size N=512, by using initialization strategies in substitution to Random initialization.

Computation cost reduction in 3D shape optimization of nanophotonic components

Inverse design methodologies effectively optimize many design parameters of a photonic device with respect to a primary objective, uncovering locally optimal designs in a typically non-convex parameter space. Often, a variety of secondary objectives (performance metrics) also need to be considered before fabrication takes place. Hence, a large collection of optimized designs is useful, as their performance on secondary objectives often varies. For certain classes of components such as shape-optimized devices, the most efficient optimization approach is to begin with 2D optimization from random parameter initialization and then follow up with 3D re-optimization. Nevertheless, the latter stage is substantially time- and resource-intensive. Thus, obtaining a desired collection of optimized designs through repeated 3D optimizations is a computational challenge. To address this issue, a machine learning-based regression model is proposed to reduce the computation cost involved in the 3D optimization stage. The regression model correlates the 2D and 3D optimized structural parameters based on a small dataset. Using the predicted design parameters from this model as the initial condition for 3D optimization, the same optima are reached faster. The effectiveness of this approach is demonstrated in the shape optimization-based inverse design of TE0-TE1 mode converters, an important component in mode-division multiplexing applications. The final optimized designs are identical in both approaches, but leveraging a machine learning-based regression model offers a 35% reduction in computation load for the 3D optimization step. The approach provides a more effective means for sampling larger numbers of 3D optimized designs.

Velocity Estimations in Blood Microflows via Machine Learning Symmetries

Improving velocity forecasts of blood microflows could be useful in biomedical applications. We focus on estimating the velocity of the blood in capillaries. Modeling blood microflow in capillaries is a complex process. In this paper, we use artificial intelligence techniques for this modeling: more precisely, artificial neural networks (ANNs). The selected model is able to accurately forecast the velocity, with an R2 of 0.8992 comparing the forecast with the actual velocity. A key part of ANN model creation is selecting the appropriate parameters for the ANN, such as the number of neurons, the number of layers and the type of training algorithm used. A grid approach with 327,600 simulations was used. It is shown that there are substantial, statistically significant differences when different types of ANN structures are used. It is also shown that the proposed model is robust regarding the initial random initialization of weights in the ANN. Additionally, the sensitivity of the selected models to additional noise was also tested.

Advancing fluid identification via well-logging data: Leveraging persistent initialization and transformer modeling

In the domain of energy exploration, the forecasting of fluid via well logging is pivotal in directing exploration endeavors. Understanding the composition of fluid underground is key for exploration teams to accurately determine the size, potential reserves, and quality of oil and gas resources. This knowledge is critical in refining exploration tactics and employing resources wisely. We present a novel machine learning architecture termed “PIFormer” for predicting fluid. This design merges Persistence Initialization with a Transformer module. The combination of persistent initialization and Transformer modules is achieved by using the persistent initialization feature representation as input to the Transformer model. Persistent initialization provides a stable starting point, enabling the Transformer model to converge to effective feature representations more rapidly during the learning process. This combination helps address issues in existing methods such as training instability, slow convergence, and local optima problems caused by random initialization. By integrating persistent initialization and the Transformer model, prior knowledge and global information can be more effectively utilized, enhancing the accuracy and robustness of fluid identification. Compared to existing models, the combination of persistent initialization and the Transformer model demonstrates higher accuracy and robustness in fluid identification tasks. Specifically, our approach achieves significant improvements in fluid identification accuracy and outperforms existing models across various types of fluid identification problems. Additionally, our method significantly reduces model training time and improves convergence speed. These results clearly indicate that the combination of persistent initialization and the Transformer model effectively addresses limitations in existing models for fluid identification tasks, providing new avenues and methods for further research and application in this field.

CGH-GTO method for model parameter identification based on improved grey wolf optimizer, honey badger algorithm, and gorilla troops optimizer

The current–voltage (I–V) characteristics are significant for applied research in photovoltaics (PV). The I–V characteristics calculated by double diode model (DDM) are accurate. DDM contains seven parameters, which are hard to solve. The combination method (CGH-GTO) based on improved grey wolf optimizer (IGWO), honey badger algorithm (HBA), and gorilla troops optimizer (GTO) for model parameter identification is proposed. First, the random initialization of GWO is replaced by Circle chaotic mapping (CCM). IGWO is employed to provide good initial values of parameters. Then, the bird-guided method of HBA is applied to optimize GTO for precise identification. Finally, the effectiveness, stability, and practicality of CGH-GTO are verified through experiments. The optimal RMSE (Root Mean Square Error) and R2 (Coefficient of determination) of CGH-GTO are 0.00241A and 0.99999 for TSM-240. Compared to the reference method GTO, the improvement ratio of RMSE for CGH-GTO can reach 7.3%. For Photowatt-PWP 201, the average RMSE of CGH-GTO is 0.00184A in the stability experiment. The RMSE of CGH-GTO is stabilized at [0.00182A, 0.00350A]. For three PV cell models, the results of evaluation metrics for DDM present the optimum. CGH-GTO achieves the highest accuracy in model parameter identification for DDM with RMSE and R2 of 0.00182A and 0.99999.

Multilevel Leapfrogging Initialization Strategy for Quantum Approximate Optimization Algorithm

AbstractRecently, Zhou et al. have proposed an Interpolation‐based (INTERP) strategy to generate the initial parameters for Quantum Approximate Optimization Algorithm (QAOA). INTERP guesses the initial parameters at level by applying interpolation to the optimized parameters at level , achieving better performance than random initialization (RI). Nevertheless, INTERP consumes extensive costs for deep QAOA because it necessitates optimization at each level depth. To address it, a Multilevel Leapfrogging Interpolation (MLI) strategy is proposed. MLI produces initial parameters from level to () at level , omitting the optimization rounds from level to . MLI executes optimization at few levels rather than each level, and this operation is called Multilevel Leapfrogging optimization (M‐Leap). The performance of MLI is investigated on the Maxcut problem. The simulation results demonstrate MLI achieves the same quasi‐optima as INTERP while consuming 1/2 of costs required by INTERP. Besides, for MLI, where there is no RI except for level 1, the greedy‐MLI strategy is presented. The simulation results suggest greedy‐MLI has better stability than INTERP and MLI beyond obtaining the quasi‐optima. According to the efficiency of finding the quasi‐optima, the idea of M‐Leap might be extended to other training tasks, especially those requiring numerous optimizations.

Deep reinforcement learning based active surge control for aeroengine compressors

This study proposes an active surge control method based on deep reinforcement learning to ensure the stability of compressors when adhering to the pressure rise command across the wide operating range of an aeroengine. Initially, the study establishes the compressor dynamic model with uncertainties, disturbances, and Close-Coupled Valve (CCV) actuator delay. Building upon this foundation, a Partially Observable Markov Decision Process (POMDP) is defined to facilitate active surge control. To address the issue of unobservability, a nonlinear state observer is designed using a finite-time high-order sliding mode. Furthermore, an Improved Soft Actor-Critic (ISAC) algorithm is developed, incorporating prioritized experience replay and adaptive temperature parameter techniques, to strike a balance between exploration and convergence during training. In addition, reasonable observation variables, error-segmented reward functions, and random initialization of model parameters are employed to enhance the robustness and generalization capability. Finally, to assess the effectiveness of the proposed method, numerical simulations are conducted, and it is compared with the fuzzy adaptive backstepping method and Second-Order Sliding Mode Control (SOSMC) method. The simulation results demonstrate that the deep reinforcement learning based controller outperforms other methods in both tracking accuracy and robustness. Consequently, the proposed active surge controller can effectively ensure stable operation of compressors in the high-pressure-ratio and high-efficiency region.

Stronger and Transferable Node Injection Attacks

Despite the increasing popularity of graph neural networks (GNNs), the security risks associated with their deployment have not been well explored. Existing works follow the standard adversarial attacks to maximize cross-entropy loss within an L-infinity norm bound. We analyze the robustness of GNNs against node injection attacks (NIAs) in black-box settings by allowing new nodes to be injected and attacked. In this work, we propose to design stronger and transferable NIAs. First, we propose margin aware attack (MAA) that uses a maximum margin loss to generate NIAs. We then propose a novel margin and direction aware attack (MDA) that diversifies the initial directions of MAA attack by minimizing the cosine similarity of the injected nodes with respect to their respective random initialization in addition to the maximization of max-margin loss. This makes the NIAs stronger. We further observe that using L2 norm of gradients in the attack step leads to an enhanced diversity amongst the node features, thereby further enhancing the strength of the attack. We incorporate transferability in NIAs by perturbing the surrogate model before generating the attack. An analysis of eigen spectrum density of the hessian of the loss emphasizes that perturbing the weights of the surrogate model improves the transferability. Our experimental results demonstrate that the proposed resilient node injection attack (R-NIA) consistently outperform PGD by margins about 7-15% on both large and small graph datasets. R-NIA is significantly stronger and transferable than existing NIAs on graph robustness benchmarks.

An Analysis of Loss Functions for Heavily Imbalanced Lesion Segmentation.

Heavily imbalanced datasets are common in lesion segmentation. Specifically, the lesions usually comprise less than 5% of the whole image volume when dealing with brain MRI. A common solution when training with a limited dataset is the use of specific loss functions that rebalance the effect of background and foreground voxels. These approaches are usually evaluated running a single cross-validation split without taking into account other possible random aspects that might affect the true improvement of the final metric (i.e., random weight initialisation or random shuffling). Furthermore, the evolution of the effect of the loss on the heavily imbalanced class is usually not analysed during the training phase. In this work, we present an analysis of different common loss metrics during training on public datasets dealing with brain lesion segmentation in heavy imbalanced datasets. In order to limit the effect of hyperparameter tuning and architecture, we chose a 3D Unet architecture due to its ability to provide good performance on different segmentation applications. We evaluated this framework on two public datasets and we observed that weighted losses have a similar performance on average, even though heavily weighting the gradient of the foreground class gives better performance in terms of true positive segmentation.

Developing a multivariate time series forecasting framework based on stacked autoencoders and multi-phase feature

Time series forecasting across different domains has received massive attention as it eases intelligent decision-making activities. Recurrent neural networks and various deep learning algorithms have been applied to modeling and forecasting multivariate time series data. Due to intricate non-linear patterns and significant variations in the randomness of characteristics across various categories of real-world time series data, achieving effectiveness and robustness simultaneously poses a considerable challenge for specific deep-learning models. We have proposed a novel prediction framework with a multi-phase feature selection technique, a long short-term memory-based autoencoder, and a temporal convolution-based autoencoder to fill this gap. The multi-phase feature selection is applied to retrieve the optimal feature selection and optimal lag window length for different features. Moreover, the customized stacked autoencoder strategy is employed in the model. The first autoencoder is used to resolve the random weight initialization problem. Additionally, the second autoencoder models the temporal relation between non-linear correlated features with convolution networks and recurrent neural networks.Finally, the model's ability to generalize, predict accurately, and perform effectively is validated through experimentation with three distinct real-world time series datasets. In this study, we conducted experiments on three real-world datasets: Energy Appliances, Beijing PM2.5 Concentration, and Solar Radiation. The Energy Appliances dataset consists of 29 attributes with a training size of 15,464 instances and a testing size of 4239 instances. For the Beijing PM2.5 Concentration dataset, there are 18 attributes, with 34,952 instances in the training set and 8760 instances in the testing set. The Solar Radiation dataset comprises 11 attributes, with 22,857 instances in the training set and 9797 instances in the testing set. The experimental setup involved evaluating the performance of forecasting models using two distinct error measures: root mean square error and mean absolute error. To ensure robust evaluation, the errors were calculated at the identical scale of the data. The results of the experiments demonstrate the superiority of the proposed model compared to existing models, as evidenced by significant advantages in various metrics such as mean squared error and mean absolute error. For PM2.5 air quality data, the proposed model's mean absolute error is 7.51 over 12.45, about ∼40% improvement. Similarly, the mean square error for the dataset is improved from 23.75 to 11.62, which is ∼51%of improvement. For the solar radiation dataset, the proposed model resulted in ∼34.7% improvement in means squared error and ∼75% in mean absolute error. The recommended framework demonstrates outstanding capabilities in generalization and outperforms datasets spanning multiple indigenous domains.

A Population Initialization Method Based on Similarity and Mutual Information in Evolutionary Algorithm for Bi-objective Feature Selection

Feature selection (FS) is an important data pre-processing technique in classification. It aims to remove redundant and irrelevant features from the data, which reduces the dimensionality of data and improves the performance of the classifier. Thus, FS is a bi-objective optimization problem, and evolutionary algorithms (EAs) have been proven to be effective in solving bi-objective FS problems. EA is a population-based metaheuristic algorithm, and the quality of the initial population is an important factor affecting the performance of EA. An improper initial population may negatively affect the convergence speed of the EA and even make the algorithm fall into the local optimum. In this paper, we propose a similarity and mutual information-based initialization method, named SMII, to improve the quality of the initial population. This method determines the distribution of initial solutions based on similarity, and shields features with high correlation to the selected features according to mutual information. In the experiment, we embed SMII, the latest four initialization methods, and a traditional random initialization method into NSGA-II, and compared their performance on 15 public datasets. The experimental results show that SMII performs best on most datasets, and can effectively improve the performance of the algorithm. Moreover, we compare the performance of two other EA before and after embedding SMII on 15 datasets, the results further prove that the proposed method can effectively improve the search capability of the EA for FS.

An improved particle swarm optimization algorithm for scheduling tasks in cloud environment

AbstractCloud computing provide services dynamically according to the contract between service providers and users. However, Inappropriateness of scheduling task on VMs can lead huge resource waste and load unbalance, which becomes a seriously challenging problem. Current Swarm intelligence algorithms like genetic algorithm (GA), particle swarm optimization (PSO) are combination of random initialization and local search algorithm. It avoids inconsistent results for different problem instances. However, existing Swarm intelligence works sometimes search the optima without analysing task scheduling situations comprehensively, global search efficiency is low and convergence is too early. In this paper, we propose SNSK‐IPSO algorithm, which develops as a two‐phases algorithm: enumerating all distributed solutions between VMs and tasks, finding the optimal solution through IPSO. It not only minimizes the execution time, but also improves resource utilization and load balance. Several experiments demonstrate that our novel algorithm outperforms others in terms of achieving load balance, higher resource utilization and lower execution times.

MVP-Stereo: A Parallel Multi-View Patchmatch Stereo Method with Dilation Matching for Photogrammetric Application

Multi-view stereo plays an important role in 3D reconstruction but suffers from low reconstruction efficiency and has difficulties reconstructing areas with low or repeated textures. To address this, we propose MVP-Stereo, a novel multi-view parallel patchmatch stereo method. MVP-Stereo employs two key techniques. First, MVP-Stereo utilizes multi-view dilated ZNCC to handle low texture and repeated texture by dynamically adjusting the matching window size based on image variance and using a portion of pixels to calculate matching costs without increasing computational complexity. Second, MVP-Stereo leverages multi-scale parallel patchmatch to reconstruct the depth map for each image in a highly efficient manner, which is implemented by CUDA with random initialization, multi-scale parallel spatial propagation, random refinement, and the coarse-to-fine strategy. Experiments on the Strecha dataset, the ETH3D benchmark, and the UAV dataset demonstrate that MVP-Stereo can achieve competitive reconstruction quality compared to state-of-the-art methods with the highest reconstruction efficiency. For example, MVP-Stereo outperforms COLMAP in reconstruction quality by around 30% of reconstruction time, and achieves around 90% of the quality of ACMMP and SD-MVS in only around 20% of the time. In summary, MVP-Stereo can efficiently reconstruct high-quality point clouds and meet the requirements of several photogrammetric applications, such as emergency relief, infrastructure inspection, and environmental monitoring.

Load Forecasting and Operation Optimization of Ice-Storage Air Conditioners Based on Improved Deep-Belief Network

The prediction of cold load in ice-storage air conditioning systems plays a pivotal role in optimizing air conditioning operations, significantly contributing to the equilibrium of regional electricity supply and demand, mitigating power grid stress, and curtailing energy consumption in power grids. Addressing the issues of minimal correlation between input and output data and the suboptimal prediction accuracy inherent in traditional deep-belief neural-network models, this study introduces an enhanced deep-belief neural-network combination prediction model. This model is refined through an advanced genetic algorithm in conjunction with the “Statistical Products and Services Solution” version 25.0 software, aiming to augment the precision of ice-storage air conditioning load predictions. Initially, the input data undergo processing via the “Statistical Products and Services Solution” software, which facilitates the exclusion of samples exhibiting low coupling. Subsequently, the improved genetic algorithm implements adaptive adjustments to surmount the challenge of random weight parameter initialization prevalent in traditional deep-belief networks. Consequently, an optimized deep-belief neural-network load prediction model, predicated on the enhanced genetic algorithm, is established and subjected to training. Ultimately, the model undergoes simulation validation across three critical dimensions: operational performance, prediction evaluation indices, and operating costs of ice-storage air conditioners. The results indicate that, compared to existing methods for predicting the cooling load of ice-storage air conditioning, the proposed model achieves a prediction accuracy of 96.52%. It also shows an average improvement of 14.12% in computational performance and a 14.32% reduction in model energy consumption. The prediction outcomes align with the actual cooling-load variation patterns. Furthermore, the daily operational cost of ice-storage air conditioning, derived from the predicted cooling-load data, has an error margin of only 2.36%. This contributes to the optimization of ice-storage air conditioning operations.

Smoothness Enforcing Projected Gradient Descent Adversarial Training

Abstract The paper proposes a novel adversarial training method based on a smoothness enforcing framework called Smoothness Enforcing Adversarial Training (SEAT). It is known that the traditional adversarial training methods suffer from prohibitive computational overhead and mismatched class distributions. To this end, our method is combined with smoothness enforcing methods in adversarial training iterations to achieve better robustness and generalization. SEAT reuses gradient information computed when updating model parameters. Meanwhile, random initialization and output smearing are integrated into the process to optimize the mismatched problems with better generalization performance. Furthermore, the temporal ensembling serves as an implicit self-ensemble of the adversarial information which benefits from a longer memory. In different iterations, the smoothness constraints are imposed to enforce smoothness. Unlike existing adversarial training methods, our method is free from arbitrarily complicated distributions and expensive generation of adversarial examples. Extensive experiments validate the effectiveness of SEAT in comparison with state-of-the-art adversarial training methods.

Learning to Learn Task-Adaptive Hyperparameters for Few-Shot Learning.

The objective of few-shot learning is to design a system that can adapt to a given task with only few examples while achieving generalization. Model-agnostic meta-learning (MAML), which has recently gained the popularity for its simplicity and flexibility, learns a good initialization for fast adaptation to a task under few-data regime. However, its performance has been relatively limited especially when novel tasks are different from tasks previously seen during training. In this work, instead of searching for a better initialization, we focus on designing a better fast adaptation process. Consequently, we propose a new task-adaptive weight update rule that greatly enhances the fast adaptation process. Specifically, we introduce a small meta-network that can generate per-step hyperparameters for each given task: learning rate and weight decay coefficients. The experimental results validate that learning a good weight update rule for fast adaptation is the equally important component that has drawn relatively less attention in the recent few-shot learning approaches. Surprisingly, fast adaptation from random initialization with ALFA can already outperform MAML. Furthermore, the proposed weight-update rule is shown to consistently improve the task-adaptation capability of MAML across diverse problem domains: few-shot classification, cross-domain few-shot classification, regression, visual tracking, and video frame interpolation.