Gradient Descent Search Research Articles

Threshold-based successive elimination algorithm for block motion estimation

The successive elimination algorithm (SEA) has already proved successful in block motion estimation. This paper presents a simple and efficient algorithm called the threshold-based SEA as a search engine to take advantage of the simplicity of the SEA while avoiding its redundant computation. In addition, the threshold-based SEA utilizes a threshold adaptation mechanism to efficiently distribute the available computational power of the employed codec to blocks of video sequences. Since the threshold-based approach is performed at each search candidate block, it can be combined not only with the SEA but also with other fast algorithms, including the three-step search, the block-based gradient descent search, and the diamond search. Depending on the motion activity in the sequences, experimental results indicate that the threshold-based approach can uniformly achieve a quality improvement over the original SEA for the same computation.

A Paper Currency Recognition System based on Neural Networks with Gaussian Functions and an Optimizing Method for Its Parameters on Way to Learning

In this paper, in order to improve rejection capabilities of the paper currency recognition system for unknown currency patterns on promise of ensuring recognition capabilities for known currency patterns, a feed-forward neural network (FNN) with Gaussian activation function is proposed. The proposed activation function is a ridge-like function. Moreover, a hybrid-learning algorithm for optimizing the width parameters of the Gaussian function is proposed. In the network the Gaussian activation function instead of the sigmoid function is employed in all units of hidden and output layers. The algorithm consists of two steps, one is exploring local minima by employing the gradient descent search, and the other is extricating the search from local minima, in which a random search with the downhill simplex method is employed. The results of simulation reveal the potential effectiveness of the proposed activation function and the algorithm. The system with the proposed activation function and the proposed algorithm can recognize known currency patterns and reject the unknown currency patterns effectively.

Adaptive fast block-matching algorithm by switching search patterns for sequences with wide-range motion content

Content with rapid, moderate, and slow motion is frequently mixed together in real video sequences. Until now, no fast block-matching algorithm (FBMA), including the well-known three-step search (TSS), the block-based gradient descent search (BBGDS), and the diamond search (DS), can efficiently remove the temporal redundancy of sequences with wide range motion content. This paper proposes an adaptive FBMA, called A-TDB, to solve this problem. Based on the characteristics of a proposed predicted profit list, the A-TDB can adaptively switch search patterns among the TSS, DS, and BBGDS, according to the motion content. Experimental results reveal that the A-TDB successfully adopts the search patterns to remove the temporal redundancy of sequences with slow, moderate and rapid motion content.

PAPR Reduction of OFDM Signals Using a Reduced Complexity PTS Technique

A major drawback of orthogonal frequency division multiplexing (OFDM) is the high peak-to-average power ratio (PAPR) of the transmitted signal. Partial transmit sequence (PTS) technique can improve the PAPR statistics of OFDM signals. In the PTS technique, the data block to be transmitted is partitioned into disjoint subblocks and the subblocks are combined using phase factors to minimize PAPR. As ordinary PTS technique requires an exhaustive search over all combinations of allowed phase factors, the search complexity increases exponentially with the number of subblocks. In the proposed technique, a gradient descent search is performed to find the phase factors. It is shown that the proposed technique achieves significant reduction in search complexity with little performance degradation.

A sub-mW MPEG-4 motion estimation processor core for mobile video application

This paper describes a sub-mW motion estimation processor core for MPEG-4 video encoding. It features a gradient descent search (GDS) algorithm that reduces required computational complexity to 15 MOPS. The GDS algorithm combined with a sub-block search method upgrades picture quality. The quality is almost equal to that of a full search method. An SIMD datapath architecture optimized for the algorithm decreases a clock frequency and supply voltage. A dedicated three-port SRAM macro for image data caches of the processor is newly designed to reduce power consumption. It has been fabricated with 0.18-/spl mu/m five-layer metal CMOS technology. The VLSI processing QCIF 15-f/s video consumes 0.4-mW power at 0.85-MHz clock frequency with 1.0-V supply voltage. It is applicable to mobile video applications.

A simple and efficient block motion estimation algorithm based on full-search array architecture

This paper first presents an array structure using ± 1 full-search (FS) architecture as the search engine of block motion estimation which takes advantage of the design regularity of FS. An efficient algorithm named modified gradient-descent search (MGDS) is then introduced based on the proposed architecture. MGDS utilizes an adaptive computation distribution mechanism to efficiently allocate computation of the employed ± 1 FS array to blocks or frames of video sequences. Experimental results indicate that MGDS uniformly achieves a higher quality than FS by an amount that is dependent on motion activities of sequences.

A new predictive search area approach for fast block motion estimation

According to the observation on the distribution of motion differentials among the motion vector of any block and those of its four neighboring blocks from six real video sequences, this paper presents a new predictive search area approach for fast block motion estimation. Employing our proposed simple predictive search area approach into the full search (FS) algorithm, our improved FS algorithm leads to 93.83% average execution-time improvement ratio, but only has a small estimation accuracy degradation. We also investigate the advantages of computation and estimation accuracy of our improved FS algorithm when compared to the edge-based search algorithm of Chan and Siu; experimental results reveal that our improved FS algorithm has 74.33% average execution-time improvement ratio and has a higher estimation accuracy. Finally, we further compare the performance among our improved FS algorithm, the three-step search algorithm, and the block-based gradient descent search algorithm.

An Approach to Off-Line Handwritten Chinese Character Recognition Based on Hierarchical Radical Decomposition

Off-line recognition of handwritten Chinese characters is of considerable practical importance as well as representing a very hard pattern recognition problem. A popular approach is to decompose characters into their component or ‘primitive’ parts – most usually strokes. Here, however, we take the less usual approach of decomposing into radicals. Active shape modelling is applied and developed into active radical modelling. In training, 60 examples of each radical are represented by ‘landmark’ points, labelled semi-automatically, with radicals in different characteristic positions treated as distinctly different radicals. Principal component analysis then captures the main variation around the mean radical. In recognition, the dynamic tunnelling algorithm is incorporated with gradient descent to search for optimal shape parameters in terms of chamfer distance minimisation. Although prior landmark labelling is time-consuming and gradient descent search during recognition is computationally expensive, the method is theoretically well motivated, incorporates prior knowledge about the structure of Chinese characters in an appropriate way, and avoids problems implicit in stroke extraction. Experiments are conducted on 280,000 loosely constrained characters from 200 writers. There are 98 different categories of radical included in 1400 character categories, and approximately 590,000 radicals in total. The matching rate on this large test set is 94.2% radicals correct (writer-independent), greatly superior to existing radical approaches. Assuming character composition to be a Markov process in which up to four radicals are combined in some assumed sequential order, we can recognise complete, hierarchically composed characters using the Viterbi algorithm. This results in a character recognition rate of 92.6%.

A MULTI-SCALE TWO-STEP FAST SEARCH ALGORITHM FOR BLOCK MOTION ESTIMATION

Block motion estimation using full search is computationally intensive. Previously proposed fast algorithms reduce computation by limiting the number of search locations and search directions in a predefined search region. This is accomplished at the expense of accuracy of motion estimation and a large mean squared error for motion-compensated images, especially for image sequences with large displacement and rotation. In this paper, a novel efficient search algorithm for block motion estimation is presented to produce better performance than some fast search algorithms that have been developed, such as three step search, orthogonal search, 2D-logarithmic search, four step search, and block-based gradient descent search, in large displacement and rotation image cases. The proposed algorithm is based on the notion of locally multi-scale operation, search of global minimum, and two layer search strategy. Experimental results show that the proposed algorithm produces anticipative performance while costing much less computation power than the full search algorithm.

Material parameter estimation with terahertz time-domain spectroscopy.

Imaging systems based on terahertz (THz) time-domain spectroscopy offer a range of unique modalities owing to the broad bandwidth, subpicosecond duration, and phase-sensitive detection of the THz pulses. Furthermore, the possibility exists for combining spectroscopic characterization or identification with imaging because the radiation is broadband in nature. To achieve this, we require novel methods for real-time analysis of THz waveforms. This paper describes a robust algorithm for extracting material parameters from measured THz waveforms. Our algorithm simultaneously obtains both the thickness and the complex refractive index of an unknown sample under certain conditions. In contrast, most spectroscopic transmission measurements require knowledge of the sample's thickness for an accurate determination of its optical parameters. Our approach relies on a model-based estimation, a gradient descent search, and the total variation measure. We explore the limits of this technique and compare the results with literature data for optical parameters of several different materials.

An empirical study of design and testing of hybrid evolutionary–neural approach for classification

We propose a hybrid evolutionary–neural approach for binary classification that incorporates a special training data over-fitting minimizing selection procedure for improving the prediction accuracy on holdout sample. Our approach integrates parallel global search capability of genetic algorithms (GAs) and local gradient-descent search of the back-propagation algorithm. Using a set of simulated and real life data sets, we illustrate that the proposed hybrid approach fares well, both in training and holdout samples, when compared to the traditional back-propagation artificial neural network (ANN) and a genetic algorithm-based artificial neural network (GA-ANN).

An efficient gray search algorithm for the estimation of motion vectors

Motion vector estimation plays an important role in motion-compensated video coding. An efficient and fast search algorithm is proposed for the estimation of motion vectors. With the help of gray prediction, the algorithm can determine the motion vectors of image blocks quickly and correctly. Since the proposed algorithm performs better than other search algorithms [e.g. the three-step search (TSS), cross-search (CS), new three-step search (NTSS), four-step search (FSS), block-based gradient descent search (BBGDS), simple-and-efficient search (SES), prediction search (PS) and gray prediction search (GPS)], it is very beneficial in applications where the video coding speed is important.

Parallel computation of a maximum-likelihood estimator of a physical map.

Reconstructing a physical map of a chromosome from a genomic library presents a central computational problem in genetics. Physical map reconstruction in the presence of errors is a problem of high computational complexity that provides the motivation for parallel computing. Parallelization strategies for a maximum-likelihood estimation-based approach to physical map reconstruction are presented. The estimation procedure entails a gradient descent search for determining the optimal spacings between probes for a given probe ordering. The optimal probe ordering is determined using a stochastic optimization algorithm such as simulated annealing or microcanonical annealing. A two-level parallelization strategy is proposed wherein the gradient descent search is parallelized at the lower level and the stochastic optimization algorithm is simultaneously parallelized at the higher level. Implementation and experimental results on a distributed-memory multiprocessor cluster running the parallel virtual machine (PVM) environment are presented using simulated and real hybridization data.

An efficient search strategy for block motion estimation using image features

Block motion estimation using the exhaustive full search is computationally intensive. Fast search algorithms offered in the past tend to reduce the amount of computation by limiting the number of locations to be searched. Nearly all of these algorithms rely on this assumption: the mean absolute difference (MAD) distortion function increases monotonically as the search location moves away from the global minimum. Essentially, this assumption requires that the MAD error surface be unimodal over the search window. Unfortunately, this is usually not true in real-world video signals. However, we can reasonably assume that it is monotonic in a small neighborhood around the global minimum. Consequently, one simple strategy, but perhaps the most efficient and reliable, is to place the checking point as close as possible to the global minimum. In this paper, some image features are suggested to locate the initial search points. Such a guided scheme is based on the location of certain feature points. After applying a feature detecting process to each frame to extract a set of feature points as matching primitives, we have extensively studied the statistical behavior of these matching primitives, and found that they are highly correlated with the MAD error surface of real-world motion vectors. These correlation characteristics are extremely useful for fast search algorithms. The results are robust and the implementation could be very efficient. A beautiful point of our approach is that the proposed search algorithm can work together with other block motion estimation algorithms. Results of our experiment on applying the present approach to the block-based gradient descent search algorithm (BBGDS), the diamond search algorithm (DS) and our previously proposed edge-oriented block motion estimation show that the proposed search strategy is able to strengthen these searching algorithms. As compared to the conventional approach, the new algorithm, through the extraction of image features, is more robust, produces smaller motion compensation errors, and has a simple computational complexity.

Enhanced FFT-based parametric algorithm for simultaneous multiple harmonics analysis

An enhanced FFT-based parametric (E-FFT) algorithm suitable for on-line harmonic analysis of electrical power systems is presented. This E-FFT algorithm exploits its iteration loops in combination with the characteristic of steep-descent gradient search strategy, to limit the sensitiveness of the total harmonic distortion caused by changes in the number of parameters involved in distorted signal models. The E-FFT algorithm performs reasonably well with short data record length. Unlike most gradient-descent search algorithms for a global minimum point, the proposed E-FFT algorithm averts the risk of being trapped at any local minimum point in the search path. The E-FFT algorithm differs from other FFT and Kalman filter based tracking algorithms, in that it is able to provide simultaneous tracking of co-variations between integer and non-integer (sub) harmonics in a small number of iteration steps. Numerical illustrative examples demonstrating the operation of this E-FFT algorithm and its simulated performance results are also presented.

Determination of weights for relaxation recurrent neural networks

A theorem which establishes the solutions of a given optimization problem as stable points in the state space of single-layer relaxation-type recurrent neural networks is proposed. This theorem establishes the necessary conditions for the neural network to converge to a solution by proposing certain values for the constraint weight parameters of the network. Convergence performance of the discrete Hopfield network with the proposed bounds on constraint weight parameters is tested on a set of constraint satisfaction and optimization problems including the traveling salesman problem, the assignment problem, the weighted matching problem, the N-queens problem and the graph path search problem. Simulation and stability analysis results indicate that the set of solutions becomes a subset of the set of stable points in the state space as a result of the suggested bounds. For the cases of the traveling salesman, assignment and weighted matching problems, two sets are equal leading to convergence to a solution after each relaxation. Convergence to a solution after each relaxation is not guaranteed for the N-queens and the graph path search problems since the solution set is a proper subset of the stable point set. Furthermore the simulation results indicate that the discrete Hopfield network converged to mostly average quality solutions as expected from a gradient-descent search algorithm. In conclusion, the suggested bounds on weight parameters guarantee that the discrete Hopfield network will locate a solution after each relaxation for a class of optimization problems of any size, although the solutions will be average quality rather than optimum.

Motion estimation using a one-dimensional gradient descent search

This work presents a low-complexity high-performance motion estimation method using a one-dimensional (1-D) gradient descent search. The proposed method consists of initial-point determination, initial-direction determination, a gradient descent search using variable step sizes and conjugate directions, and convergence checking. For each block, one of the original and predictive points with the best block-matching performance is the initial point where the predictive ones are motion vectors of its neighboring four searched blocks. In addition, the searching path is optimized by determining the initial direction based on the vector from the original point to the initial point. After determining the initial point and direction, the gradient descent search using variable step sizes and conjugate directions is performed until the searched point attains a better performance than its neighboring points. Moreover, variable step sizes for moving and slowly moving, or stationary blocks, are well addressed to improve the convergence performance of a gradient descent search. Simulation results demonstrate that our method yields a superior performance in terms of computational complexity and picture quality as compared to the three-step search, 1-D full search, block-based gradient descent search, and one-at-a-time search methods.

Iterative inversion of fuzzified neural networks

The inversion of a neural network is a process of computing inputs that produce a given target when fed into the neural network. The inversion algorithm of crisp neural networks is based on the gradient descent search in which a candidate inverse is iteratively refined to decrease the error between its output and the target. In this paper. we derive an inversion algorithm of fuzzified neural networks from that of crisp neural networks. First, we present a framework of learning algorithms of fuzzified neural networks and introduce the idea of adjusting schemes for fuzzy variables. Next, we derive the inversion algorithm of fuzzified neural networks by applying the adjusting scheme for fuzzy variables to total inputs in the input layer. Finally, we make three experiments on the parity-three problem, examine the effect of the size of training sets on the inversion, and investigate how the fuzziness of inputs and targets of training sets affects the inversion.

Optimised feature map finite-state vector quantisation for image coding

An optimised feature map finite-state vector quantisation (referred to as optimised FMFSVQ) is presented for image coding. Based on the block-based gradient descent search algorithm used for motion estimation in video coding, the optimised FMFSVQ system finds a neighbourhood-based optimal codevector for each input vector by extending the associated state codebook stage by stage, thus rendering each state quantiser a variable rate vector quantisation. The optimised FMFSVQ system can be interpreted as a cascade of a finite-state vector quantiser and classified vector quantisers. Furthermore, an adaptive optimised FMFSVQ is obtained. Experiments demonstrate the superior rate-distortion performance of the adaptive optimised FMFSVQ compared with the original adaptive FMFSVQ and the memoryless vector quantisation.

A new diamond search algorithm for fast block-matching motion estimation

Based on the study of motion vector distribution from several commonly used test image sequences, a new diamond search (DS) algorithm for fast block-matching motion estimation (BMME) is proposed in this paper. Simulation results demonstrate that the proposed DS algorithm greatly outperforms the well-known three-step search (TSS) algorithm. Compared with the new three-step search (NTSS) algorithm, the DS algorithm achieves close performance but requires less computation by up to 22% on average. Experimental results also show that the DS algorithm is better than the four-step search (4SS) and block-based gradient descent search (BBGDS), in terms of mean-square error performance and required number of search points.