First-order Information Research Articles

Integrated feature set using aggregate channel features and histogram of sparse codes for human detection

The scientific community witnessed revolutionary changes with algorithms and data sets aiming for precise human detection from images and videos which are largely driven by the quality of features extracted. Regardless of the labyrinth of the existing detectors, featured human detection with near accuracy from complicated real-time data sets remains a major challenge. Here we propose an improved feature set by merging the fast and accurate aggregate channel features (ACF) and the data specific dictionary learned histogram of sparse codes (HSC) for human detection. This integrated feature set efficiently fuses the first-order information from the histogram of oriented gradient channels embedded in the ACF detector and the data specific intelligence contained in the HSC channels. The proposed detector outperforms the state-of-the-art ACF detector in terms of miss rate and average precision on challenging datasets. It is worth to be noted that there is a decrease with the miss rate by a factor of 13% and 5% for INRIA and Caltech pedestrian datasets respectively in comparison with baseline detector. Along with the detection of more instances, our detector reduced the number of false positives compared to other existing detectors. Although further modifications are warranted, our proposed detector could produce a tangible and palpable response with human detection in the vast arena of computer vision.

A general-purpose signal processing algorithm for biological profiles using only first-order derivative information

BackgroundAutomatic signal-feature extraction algorithms are crucial for profile processing in bioinformatics. Both baseline drift and noise seriously affect the position and peak area of signals. An efficient algorithm named the derivative passing accumulation (DPA) method for simultaneous baseline correction and signal extraction is presented in this article. It is an efficient method using only the first-order derivatives which are obtained through taking the simple differences.ResultsWe developed a new signal feature extracting procedure. The vector representing the discrete first-order derivative was divided into negative and positive parts and then accumulated to build a signal descriptor. The signals and background fluctuations are easily separated according to this descriptor via thresholding. In addition, the signal peaks are simultaneously located by checking the corresponding intervals in the descriptor. Therefore, the eternal issues of parsing the 1-dimensional output of detectors in biological instruments are solved together. Thereby, the baseline is corrected, and the signal peaks are extracted.ConclusionsWe have introduced a new method for signal peak picking, where baseline computation and peak identification are performed jointly. The testing results of both authentic and artificially synthesized data illustrate that the new method is powerful, and it could be a better choice for practical processing.

Cooperation through indirect reciprocity: The impact of higher-order history

Amid rapidly growing globalization and digitalization, trades often occur in one-time encounters, where cooperation depends on indirect reciprocity. This study investigates how people use higher-order information to aid in cooperation decisions. A random matching prisoners' dilemma experiment with optional history information up to the second order is conducted. With a novel continuous extension, we show that reputation scores standing and judging significantly affect cooperation decisions. Majority of the subjects respond to higher-order scores individually, and incorporating individual heterogeneity significantly improves the overall fit in aggregate regressions. A modified pooled mixture model classifies 31.7% of the subjects' behaviors into higher-order types. Subjects learn to use the judging score through their experiences. The reputation effect is stronger when subjects have first played the no-information baseline game, where reputation scores are more effective in a complementary way. We also find that cooperation improvement is significantly lower in the treatment with first-order information only.

Lower complexity bounds of first-order methods for convex-concave bilinear saddle-point problems

On solving a convex-concave bilinear saddle-point problem (SPP), there have been many works studying the complexity results of first-order methods. These results are all about upper complexity bounds, which can determine at most how many efforts would guarantee a solution of desired accuracy. In this paper, we pursue the opposite direction by deriving lower complexity bounds of first-order methods on large-scale SPPs. Our results apply to the methods whose iterates are in the linear span of past first-order information, as well as more general methods that produce their iterates in an arbitrary manner based on first-order information. We first work on the affinely constrained smooth convex optimization that is a special case of SPP. Different from gradient method on unconstrained problems, we show that first-order methods on affinely constrained problems generally cannot be accelerated from the known convergence rate $O(1/t)$ to $O(1/t^2)$, and in addition, $O(1/t)$ is optimal for convex problems. Moreover, we prove that for strongly convex problems, $O(1/t^2)$ is the best possible convergence rate, while it is known that gradient methods can have linear convergence on unconstrained problems. Then we extend these results to general SPPs. It turns out that our lower complexity bounds match with several established upper complexity bounds in the literature, and thus they are tight and indicate the optimality of several existing first-order methods.

Confidence Weighted Multitask Learning

Traditional online multitask learning only utilizes the firstorder information of the datastream. To remedy this issue, we propose a confidence weighted multitask learning algorithm, which maintains a Gaussian distribution over each task model to guide online learning process. The mean (covariance) of the Gaussian Distribution is a sum of a local component and a global component that is shared among all the tasks. In addition, this paper also addresses the challenge of active learning on the online multitask setting. Instead of requiring labels of all the instances, the proposed algorithm determines whether the learner should acquire a label by considering the confidence from its related tasks over label prediction. Theoretical results show the regret bounds can be significantly reduced. Empirical results demonstrate that the proposed algorithm is able to achieve promising learning efficacy, while simultaneously minimizing the labeling cost.

Adaptive Sparse Confidence-Weighted Learning for Online Feature Selection

In this paper, we propose a new online feature selection algorithm for streaming data. We aim to focus on the following two problems which remain unaddressed in literature. First, most existing online feature selection algorithms merely utilize the first-order information of the data streams, regardless of the fact that second-order information explores the correlations between features and significantly improves the performance. Second, most online feature selection algorithms are based on the balanced data presumption, which is not true in many real-world applications. For example, in fraud detection, the number of positive examples are much less than negative examples because most cases are not fraud. The balanced assumption will make the selected features biased towards the majority class and fail to detect the fraud cases. We propose an Adaptive Sparse Confidence-Weighted (ASCW) algorithm to solve the aforementioned two problems. We first introduce an `0-norm constraint into the second-order confidence-weighted (CW) learning for feature selection. Then the original loss is substituted with a cost-sensitive loss function to address the imbalanced data issue. Furthermore, our algorithm maintains multiple sparse CW learner with the corresponding cost vector to dynamically select an optimal cost. We theoretically enhance the theory of sparse CW learning and analyze the performance behavior in F-measure. Empirical studies show the superior performance over the stateof-the-art online learning methods in the online-batch setting.

Pedestrian detection using first- and second-order aggregate channel features

The content-based analysis of visual multimedia like images and videos are urgently needed to empower human society for the automation of difficult tasks. Pedestrian detection serves as a backbone for a multitude of image processing and machine learning algorithms and secures quite a lot of real-world applications. Keeping this fact in mind, here, we deal with the fabrication of suitable features to identify human/pedestrian instances from images with near accuracy. Accordingly, we introduce second-order aggregate channel features (SOACF) to enhance the performance of much-celebrated pedestrian detection algorithm which was mainly based on the first-order information in an image—aggregate channel features detector (ACF detector). We experimentally proved the complementary nature of ACF and SOACF. Designed to garner both these features together, instead of simple concatenation, or direct merging of the two detectors, we employed a weighted non-maximum suppression merging algorithm. The prospective detector not only performed well on INRIA, Caltech and KITTI pedestrian data set but also, mitigate the miss rate by $$\sim 4\%$$ in Caltech data set and $$\sim 2\%$$ in KITTI data set in comparison with ACF detector. Despite the fact that our in-house generated detector uses only a few channels, it surpasses many state-of-the-art methods based on baseline ACF detector. Moreover, the detection speed is 100 times faster than the topmost pedestrian detector based on ACF.

Second-Order Information Recovery in Oriented-Plane Curvature Reproduction

Oriented-plane curvature reproduction, the control of fingertip deformation by orientating a flat plate on contact points to generate a sensation of curvature, is one of the curvature reproduction methods with favorable effects. However, among the three geometric cues of curvature, only the zeroth- and first-order information is recovered with this method, whereas the second-order information (i.e., local curvature) is neglected due to the oriented plate's inherent characteristic of having a planar surface. The second-order information will change the contact length between the finger and the surface and thus affects the feeling. As a solution for this problem, this paper proposes to control the indentation depth of the finger pulp on the oriented plate such that the correct contact length is reproduced during the rendering. A finger wearable device is developed to control that indentation depth. Three psychophysical experiments are carried out and the results indicate that the second-order information plays a positive role in curvature rendering.

An almost cyclic 2-coordinate descent method for singly linearly constrained problems

A block decomposition method is proposed for minimizing a (possibly non-convex) continuously differentiable function subject to one linear equality constraint and simple bounds on the variables. The proposed method iteratively selects a pair of coordinates according to an almost cyclic strategy that does not use first-order information, allowing us not to compute the whole gradient of the objective function during the algorithm. Using first-order search directions to update each pair of coordinates, global convergence to stationary points is established for different choices of the stepsize under an appropriate assumption on the level set. In particular, both inexact and exact line search strategies are analyzed. Further, linear convergence rate is proved under standard additional assumptions. Numerical results are finally provided to show the effectiveness of the proposed method.

Sub-sampled Newton methods

For large-scale finite-sum minimization problems, we study non-asymptotic and high-probability global as well as local convergence properties of variants of Newton’s method where the Hessian and/or gradients are randomly sub-sampled. For Hessian sub-sampling, using random matrix concentration inequalities, one can sub-sample in a way that second-order information, i.e., curvature, is suitably preserved. For gradient sub-sampling, approximate matrix multiplication results from randomized numerical linear algebra provide a way to construct the sub-sampled gradient which contains as much of the first-order information as possible. While sample sizes all depend on problem specific constants, e.g., condition number, we demonstrate that local convergence rates are problem-independent.

Inference on Causal and Structural Parameters using Many Moment Inequalities

Abstract This article considers the problem of testing many moment inequalities where the number of moment inequalities, denoted by $p$, is possibly much larger than the sample size $n$. There is a variety of economic applications where solving this problem allows to carry out inference on causal and structural parameters; a notable example is the market structure model of Ciliberto and Tamer (2009) where $p=2^{m+1}$ with $m$ being the number of firms that could possibly enter the market. We consider the test statistic given by the maximum of $p$ Studentized (or $t$-type) inequality-specific statistics, and analyse various ways to compute critical values for the test statistic. Specifically, we consider critical values based upon (1) the union bound combined with a moderate deviation inequality for self-normalized sums, (2) the multiplier and empirical bootstraps, and (3) two-step and three-step variants of (1) and (2) by incorporating the selection of uninformative inequalities that are far from being binding and a novel selection of weakly informative inequalities that are potentially binding but do not provide first-order information. We prove validity of these methods, showing that under mild conditions, they lead to tests with the error in size decreasing polynomially in $n$ while allowing for $p$ being much larger than $n$; indeed $p$ can be of order $\exp (n^{c})$ for some $c > 0$. Importantly, all these results hold without any restriction on the correlation structure between $p$ Studentized statistics, and also hold uniformly with respect to suitably large classes of underlying distributions. Moreover, in the online supplement, we show validity of a test based on the block multiplier bootstrap in the case of dependent data under some general mixing conditions.

On the Local and Superlinear Convergence of a Secant Modified Linear-Programming-Newton Method

Fil: Martinez, Maria de Los Angeles. Universidad Nacional de Cordoba; Argentina. Consejo Nacional de Investigaciones Cientificas y Tecnicas. Centro Cientifico Tecnologico Conicet - Cordoba. Centro de Investigacion y Estudios de Matematica. Universidad Nacional de Cordoba. Centro de Investigacion y Estudios de Matematica; Argentina

The Moho structure beneath the Yarlung Zangbo Suture and its implications: Evidence from large dynamite shots

The Yarlung Zangbo Suture (YZS) is the collisional front between the Indian and Eurasian plates. The deep structure beneath the YZS has long been a focus of research. Its particular tectonic location and formation time provide a natural laboratory for the study of ongoing continental collisional processes. Lateral variations in the Moho discontinuity provide evidence of basic processes governing plate tectonic evolution. The depth and geometry of the Moho provides first-order information for restoration of complex geodynamic systems. This paper presents a studying result of the Moho, both adjacent to and beneath the YZS, obtained using five large dynamite shots along two profiles. These five large shots were combined to construct two single-fold profiles (YZS-A and YZS-B) and obtain high-resolution images of the Moho beneath the YZS. Both profiles show that continental crust thickness beneath the YZS is twice the global average of continental crust, and the depth of the Moho along profile YZS-B is deeper than YZS-A. YZS-A (81°E) indicates a relatively flat Moho without offset related to continent-continent convergence beneath the collision zone. Conversely, a Moho structure with offset and overlapping reflections was imaged beneath profile YZS-B (88°E). Significant lateral variations in the geometry of the Moho discontinuity along the YZS may indicate that the Indian lithosphere is thrusting beneath Tibet at a different angle from west to east.

Deep deformation of the Longmenshan fault zone related to the 2008 Wenchuan earthquake

This paper reviews studies of the past ten year related to the crustal deformation of the Longmenshan fault zone (LMSFZ) which was struck by the M 8.0 Wenchuan earthquake on May 12, 2008. The tectonics of the LMSFZ located at the eastern margin of the Tibetan Plateau is complex, with complicated fault geometry and a heterogeneous velocity structure. The Wenchuan earthquake rupture extended for about 300 km along the middle and northern segments of the LMSFZ, with a very complicated rupture process involving multiple sub-events. The Wenchuan slip distribution is characterized by two major slip patches near Hongkou/Yingxiu and Beichuan with peak slip of 5.0–15.5 m and extended from near the surface to below 20 km depth. Prior to the Wenchuan earthquake, the LMSFZ had been seismically quiet for several centuries and there were no hints that suggested that such an M 8.0 earthquake might strike the area. The long-term geological investigations and short-term geodetic measurements before the Wenchuan earthquake generally agree that the horizontal slip rate along the LMSFZ is no more than 3 mm per year. The low slip rate observed at the surface around the LMSFZ may not reveal the real state of accumulated strain at depth where the devastating Wenchuan earthquake nucleated. Rates of aseismic slip at depth derived from seismological investigation of repeating microearthquakes were found to be approximately twice as large as the interseismic rates inferred from surface GPS and geological data. Most of the clusters of repeating microearthquakes are located at the edge of locked areas where large coseismic slips were observed during the Wenchuan earthquake, suggesting a close relationship between microearthquakes and impending large earthquakes. A two-dimensional viscoelastic finite-element model produces a depth-related slip rate pattern around the LMSFZ that is consistent with that revealed by the seismological observation of repeating earthquakes. The measured in situ deep slip rates increase with depth and vary from 3.5 to 9.6 mm/a over a depth range of 4–18 km.The seismological observations of deep slip rates and microseismicity in the three decades before the Wenchuan earthquake reveal that the LMSFZ is indeed not as “quiet” as traditionally assumed in comparison with its neighboring fault systems. Considering the deep slip rates (3.5−9.6 mm/a) from the repeating microearthquakes and coseismic peak offsets of 5.0–15.5 m, the recurrence interval of Wenchuan-like events is estimated to be about 500−4500 years. The estimated recurrence interval based on the deep slip rates is much smaller than those estimates using the same coseismic displacements divided by GPS-derived or geological slip rates. Slip rate increases with depth were also recognized in the Parkfield section of the San Andreas fault zone and in the northeastern Japan subduction zone before the ruptures of the 2004 M w6.0 Parkfield earthquake and the 2011 M w9.0 Tohoku-oki earthquake, respectively. Accelerated slip is thought to have preceded a number of recent large subduction zone earthquakes and the 2008 Wenchuan earthquake, and repeating earthquakes may document short-term precursory slip at depth. Alternatively, the rapid slip rates indicated by the repeating microearthquakes may represent transiently accelerated slip preceding the Wenchuan mainshock. We suggest that slip rates at seismogenic depths are of critical importance in seismic hazard analysis. Repeating earthquakes can be regarded as “deep creepmeters” that measure the in - situ deep slip rate on otherwise aseismically slipping faults. For less well defined and widespread faults within the continents, it is essential to reveal a fault’s or region’s seismic history over different time scale. Combining a better understanding of earthquake diversity with modern technology is the key to effective and comprehensive hazard mitigation practices. The potential earthquake hazard of locked faults with unusually high inferred deep slow slip rates should be paid more attention. The 2008 Wenchuan earthquake is the best-studied continental earthquake to date with a large number of scientific publications enabled by the vast collected data sets. The research community efforts have provided first-order information about this unexpected event regarding its coseismic slip distribution and fault geometry. However, there are many remaining questions about the basic nature of this earthquake to be further constrained with multiple data sets, including the relationship of its rupture with prior coupling and interseismic creep, the varying dynamical rupture processes with depth, the frequency dependence of seismic radiation across the fault zone, and the role of multiple postseismic deformation processes.

Adaptive Cost-Sensitive Online Classification

Cost-Sensitive Online Classification has drawn extensive attention in recent years, where the main approach is to directly online optimize two well-known cost-sensitive metrics: (i) weighted sum of sensitivity and specificity; (ii) weighted misclassification cost. However, previous existing methods only considered first-order information of data stream. It is insufficient in practice, since many recent studies have proved that incorporating second-order information enhances the prediction performance of classification models. Thus, we propose a family of cost-sensitive online classification algorithms with adaptive regularization in this paper. We theoretically analyze the proposed algorithms and empirically validate their effectiveness and properties in extensive experiments. Then, for better trade off between the performance and efficiency, we further introduce the sketching technique into our algorithms, which significantly accelerates the computational speed with quite slight performance loss. Finally, we apply our algorithms to tackle several online anomaly detection tasks from real world. Promising results prove that the proposed algorithms are effective and efficient in solving cost-sensitive online classification problems in various real-world domains.

Intuitive optics: what great apes infer from mirrors and shadows

There is ongoing debate about the extent to which nonhuman animals, like humans, can go beyond first-order perceptual information to abstract structural information from their environment. To provide more empirical evidence regarding this question, we examined what type of information great apes (chimpanzees, bonobos, and orangutans) gain from optical effects such as shadows and mirror images. In an initial experiment, we investigated whether apes would use mirror images and shadows to locate hidden food. We found that all examined ape species used these cues to find the food. Follow-up experiments showed that apes neither confused these optical effects with the food rewards nor did they merely associate cues with food. First, naïve chimpanzees used the shadow of the hidden food to locate it but they did not learn within the same number of trials to use a perceptually similar rubber patch as indicator of the hidden food reward. Second, apes made use of the mirror images to estimate the distance of the hidden food from their own body. Depending on the distance, apes either pointed into the direction of the food or tried to access the hidden food directly. Third, apes showed some sensitivity to the geometrical relation between mirror orientation and mirrored objects when searching hidden food. Fourth, apes tended to interpret mirror images and pictures of these mirror images differently depending on their prior knowledge. Together, these findings suggest that apes are sensitive to the optical relation between mirror images and shadows and their physical referents.

Stable IL-1-Activation in an Inflammasome Signalling Model Depends on Positive and Negative Feedbacks and Tight Regulation of Protein Production

NLRP3-dependent inflammasome signalling is a key pathway during inflammatory processes and its deregulation is implicated in several diseases. NLRP3-inflammasome pathway activation leads to the rapid, phosphorylation-driven NF$\kappa$κB-pathway signalling, subsequently proceeds via slower transcription/translation process for producing pro-enzymes, and finally leads to the medium-speed enzymatic activation of the central inflammatory mediator IL-$1\beta$1β[1] . We here were interested in how the timing of the rate-limiting step of transcription/translation and the presence of a positive and negative auto-regulation would pose conditions for meaningful and stable IL-$1\beta$1β-activation. We extracted the essential topology of the inflammasome pathway network using a linear chain of first-order reaction and a second-order reaction for inhibitory feedback. We then performed an analytical treatment of the resulting ODE set to obtain closed-form formulae. We therefore looked for the steady states and characterized their stability by using a Jacobian-based, local analysis. We employed the Small Gain Theorem from Control Theory as recently applied by us [2] and the Gershgorin Circle Theorem to obtain mathematically exact conditions for a positive ON state and stabilities for ON and OFF steady states. We identified an ON- and one OFF- steady state whose properties we characterized in terms of the kinetic parameters by closed-form formulae. We found that under the assumption of a first-order information flow through the network, the existence of a biologically reasonable ON steady state required the simultaneous presence of the positive and the negative feedback. Assuming non-competitivity between IL-$1\beta$1β entities binding to different receptors, we found that a minimum kinetics for protein production is required to sustain a steady state with IL-$1\beta$1β activation. Assuming competitivity between IL-$1\beta$1β entities introduced additional restrictions on the maximum protein production speed to guarantee a biologically reasonable ON steady state. Finally, for both models, we ruled out bistability, suggesting that IL-$1\beta$1β activation would undergo a smooth change upon alterations of its parameters. Exemplified by the core pathway of NLRP3-inflammasome signalling, we here demonstrate that a mostly linear activation cascade containing an intermediate rate limiting step poses kinetic restrictions on this step and requires positive and negative autoregulation for obtaining a meaningful ON steady state. Due to the generality of our framework, our results are important for a wide class of receptor mediated-pathways, where a fast initial phosphorylation cascade is followed by a (slower) transcriptional response and subsequent autoregulation. Our results may further provide important design principles for synthetic biological networks involving biochemical activation and transcription/translation, by relating timing considerations and autoregulation to stable pathway activation.

Stochastic Training of Neural Networks via Successive Convex Approximations.

This paper proposes a new family of algorithms for training neural networks (NNs). These are based on recent developments in the field of nonconvex optimization, going under the general name of successive convex approximation techniques. The basic idea is to iteratively replace the original (nonconvex, highly dimensional) learning problem with a sequence of (strongly convex) approximations, which are both accurate and simple to optimize. Different from similar ideas (e.g., quasi-Newton algorithms), the approximations can be constructed using only first-order information of the NN function, in a stochastic fashion, while exploiting the overall structure of the learning problem for a faster convergence. We discuss several use cases, based on different choices for the loss function (e.g., squared loss and cross-entropy loss), and for the regularization of the NN's weights. We experiment on several medium-sized benchmark problems and on a large-scale data set involving simulated physical data. The results show how the algorithm outperforms the state-of-the-art techniques, providing faster convergence to a better minimum. Additionally, we show how the algorithm can be easily parallelized over multiple computational units without hindering its performance. In particular, each computational unit can optimize a tailored surrogate function defined on a randomly assigned subset of the input variables, whose dimension can be selected depending entirely on the available computational power.

A quasi-Newton modified LP-Newton method

We consider a method to solve constrained system of nonlinear equations based on a modification of the Linear-Programming-Newton method and replacing the first-order information with a quasi-Newton secant update, providing a computationally simple method. The proposed strategy combines good properties of two methods: the least change secant update for unconstrained system of nonlinear equations with isolated solutions and the Linear-Programming-Newton for constrained nonlinear system of equations with possible nonisolated solutions. We analyse the local convergence of the proposed method under a standard error bound condition proving its linear convergence for nonisolated solutions. Numerical experiments were done in order to show the claimed convergence rate.

Data driven discrete-time parsimonious identification of a nonlinear state-space model for a weakly nonlinear system with short data record

Many real world systems exhibit a quasi linear or weakly nonlinear behavior during normal operation, and a hard saturation effect for high peaks of the input signal. In this paper, a methodology to identify a parsimonious discrete-time nonlinear state space model (NLSS) for the nonlinear dynamical system with relatively short data record is proposed. The capability of the NLSS model structure is demonstrated by introducing two different initialisation schemes, one of them using multivariate polynomials. In addition, a method using first-order information of the multivariate polynomials and tensor decomposition is employed to obtain the parsimonious decoupled representation of the set of multivariate real polynomials estimated during the identification of NLSS model. Finally, the experimental verification of the model structure is done on the cascaded water-benchmark identification problem.