Replica Analysis Research Articles

Replica analysis of overfitting in regression models for time to event data: the impact of censoring

We use statistical mechanics techniques, viz. the replica method, to model the effect of censoring on overfitting in Cox’s proportional hazards model, the dominant regression method for time-to-event data. In the overfitting regime, Maximum Likelihood (ML) parameter estimators are known to be biased already for small values of the ratio of the number of covariates over the number of samples. The inclusion of censoring was avoided in previous overfitting analyses for mathematical convenience, but is vital to make any theory applicable to real-world medical data, where censoring is ubiquitous. Upon constructing efficient algorithms for solving the new (and more complex) Replica Symmetric (RS) equations and comparing the solutions with numerical simulation data, we find excellent agreement, even for large censoring rates. We then address the practical problem of using the theory to correct the biased ML estimators without knowledge of the data-generating distribution. This is achieved via a novel numerical algorithm that self-consistently approximates all relevant parameters of the data generating distribution while simultaneously solving the RS equations. We investigate numerically the statistics of the corrected estimators, and show that the proposed new algorithm indeed succeeds in removing the bias of the ML estimators, for both the association parameters and for the cumulative hazard.

Elaeagnus umbellata Fruit Extract Protects Skin from Ultraviolet-Mediated Photoaging in Hairless Mice.

Photoaging refers to the accumulation of skin damage which includes wrinkle formation, loss of elasticity, and epidermal thickening due to repeated ultraviolet (UV) irradiation. The present study investigated the protective effects of Elaeagnus umbellata fruit extract (Elaea) on UV-mediated photoaged skin of SKH1 hairless mice and compared the effects of Elaea with ascorbic acid. Although there was no difference in body weight between groups during experimental period, oral administration of 50-200 mg/kg Elaea once daily for 15 weeks significantly prevented an increase in skin weight, epithelial thickening of epidermis, and apoptosis caused by UV irradiation. Skin replica and histopathological analyses revealed that Elaea dose-dependently decreased wrinkle and microfold formation. In addition, Elaea administration restored UV-mediated reduction in type I collagen and hyaluronan through the inhibition of matrix metalloproteinases and p38 mitogen-activated protein kinase expression. Moreover, Elaea suppressed UV-dependent increases in superoxide anion production, fatty acid oxidation, and protein nitration by up-regulating antioxidant system. Furthermore, Elaea alleviated infiltration of inflammatory cells in UV-irradiated skin. The preventive effects of 100 mg/kg Elaea administration against UV-induced photoaging were similar to those by 100 mg/kg ascorbic acid. Collectively, the present study suggests that the E. umbellata fruit is a promising edible candidate to prevent skin photoaging.

Replica Analysis of the Linear Model With Markov or Hidden Markov Signal Priors

Replica Analysis of the Linear Model With Markov or Hidden Markov Signal Priors

About the de Almeida–Thouless line in neural networks

In this work we present a rigorous and straightforward method to detect the onset of the instability of replica-symmetric theories in information processing systems, which does not require a full replica analysis as in the method originally proposed by de Almeida and Thouless for spin glasses. The method is based on an expansion of the free-energy obtained within one-step of replica symmetry breaking (RSB) around the RS value. As such, it requires solely continuity and differentiability of the free-energy and it is robust to be applied broadly to systems with quenched disorder. We apply the method to the Hopfield model and to neural networks with multi-node Hebbian interactions, as case studies. In the appendices we test the method on the Sherrington–Kirkpatrick and the Ising P-spin models, recovering the AT lines known in the literature for these models, as a special limit, which corresponds to assuming that the transition from the RS to the RSB phase can be obtained by varying continuously the order parameters. Our method provides a generalization of the AT approach, which does not rely on this limit and can be applied to systems with discontinuous phase transitions, as we show explicitly for the spherical P-spin model, recovering the known RS instability line.

Evaluating the Impact of a Cream Containing Horse Placental Extract on Eye Corner Wrinkles in Healthy Women: Single-Blind Comparative Study.

Placental extract has been mostly used in skin care for cosmetic purposes. However, the use of various placental extracts has been limited due to the lack of established and effective application methods. In this study, we investigated the antiwrinkle effect of a cream formulation-LNC wrinkle eye cream (LNC-EC)-containing horse placental extract as the main ingredient. A total of 24 healthy women, aged 37-54 years, with wrinkle grades 1-3, were treated with LNC-EC for 2 weeks. The cream was applied on one-half of the participants' faces, and the results were compared with the untreated half of the face. Visual inspection, using the wrinkle grade standard, showed that the area treated with LNC-EC had a significantly lower wrinkle grade than the untreated area when comparing before and after the application of LNC-EC. In addition, replica analysis showed a significant reduction in both the maximum wrinkle width and the number of wrinkles in the LNC-EC-treated area in comparison to the untreated area before and after the application. These results suggest that LNC-EC has an antiwrinkle effect on the corners of the eyes based on parameters like the maximum wrinkle width and the number of wrinkles. LNC-EC, with horse placental extract as its main ingredient, was shown to be effective in improving wrinkles at the eye corners, presumably due to a reduction in the maximum wrinkle width and the number of wrinkles. Interpretation of the results is limited because this study was conducted only in the intervention group. A randomized controlled trial with a placebo control group is necessary to verify the antiwrinkle effects of horse placental extract.

The mRNA and protein levels of the glycolytic enzymes lactate dehydrogenase A (LDHA) and phosphofructokinase platelet (PFKP) are good predictors of survival time, recurrence, and risk of death in cervical cancer patients.

Patients with cervical cancer (CC) may experience local recurrence very often after treatment; when only clinical parameters are used, most cases are diagnosed in late stages, which decreases the chance of recovery. Molecular markers can improve the prediction of clinical outcome. Glycolysis is altered in 70% of CCs, so molecular markers of this pathway associated with the aggressiveness of CC can be identified. The expression of 14 glycolytic genes was analyzed in 97 CC and 29 healthy cervical tissue (HCT) with microarray; only LDHA and PFKP were validated at the mRNA and protein levels in 36 of those CC samples and in 109 new CC samples, and 31 HCT samples by qRT-PCR, Western blotting, or immunohistochemistry. A replica analysis was performed on 295 CC from The Cancer Genome Atlas (TCGA) database. The protein expression of LDHA and PFKP was associated with poor overall survival [OS: LDHA HR = 4.0 (95% CI = 1.4-11.1); p = 8.0 × 10-3 ; PFKP HR = 3.3 (95% CI = 1.1-10.5); p = 4.0 × 10-2 ] and disease-free survival [DFS: LDHA HR = 4.5 (95% CI = 1.9-10.8); p = 1.0 × 10-3 ; PFKP HR = 3.2 (95% CI = 1.2-8.2); p = 1.8 × 10-2 ] independent of FIGO clinical stage, and the results for mRNA expression were similar. The risk of death was greater in patients with overexpression of both biomarkers than in patients with advanced FIGO stage [HR = 8.1 (95% CI = 2.6-26.1; p = 4.3 × 10-4 ) versus HR = 7 (95% CI 1.6-31.1, p = 1.0 × 10-2 )] and increased exponentially as the expression of LDHA and PFKP increased. LDHA and PFKP overexpression at the mRNA and protein levels was associated with poor OS and DFS and increased risk of death in CC patients regardless of FIGO stage. The measurement of these two markers could be very useful for evaluating clinical evolution and the risk of death from CC and could facilitate better treatment decision making.

Replica analysis of the lattice-gas restricted Boltzmann machine partition function

We study the expectation value of the logarithm of the partition function of large binary-to-binary lattice-gas restricted Boltzmann machines (RBMs) within a replica-symmetric ansatz, averaging over the disorder represented by the parameters of the RBM Hamiltonian. Averaging over the Hamiltonian parameters is done with a diagonal covariance matrix. Due to the diagonal form of the parameter covariance matrix not being preserved under the isomorphism between the Ising and lattice-gas forms of the RBM, we find differences in the behaviour of the quenched log partition function of the lattice-gas RBM compared to that of the Ising RBM form usually studied. We obtain explicit expressions for the expectation and variance of the lattice-gas RBM log partition function per node in the thermodynamic limit. We also obtain explicit expressions for the leading order finite size correction to the expected log partition function per node, and the threshold for the stability of the replica-symmetric approximation. We show that the stability threshold of the replica-symmetric approximation is equivalent, in the thermodynamic limit, to the stability threshold of a recent message-passing algorithm used to construct a mean-field Bethe approximation to the RBM free energy. Given the replica-symmetry assumption breaks down as the level of disorder in the spin-spin couplings increases, we obtain asymptotic expansions, in terms of the variance controlling this disorder, for the replica-symmetric log partition function and the replica-symmetric stability threshold. We confirm the various results derived using simulation.

Macroscopic Analysis of Vector Approximate Message Passing in a Model-Mismatched Setting

In this study, macroscopic properties of the vector approximate message passing (VAMP) algorithm for inference of generalized linear models are investigated using a non-rigorous heuristic method of statistical mechanics when the true posterior cannot be used and the measurement matrix is a sample from rotation-invariant random matrix ensembles. The focus is on the correspondence between the non-rigorous replica analysis of statistical mechanics and the performance assessment of VAMP in the model-mismatched setting. The correspondence of this kind is well-known when the measurement matrix has independent and identically distributed entries. However, when the measurement matrix follows a general rotation-invariant matrix ensemble, the correspondence has been validated only under limited cases, such as the Bayes optimal inference or the convex empirical risk minimization. The result presented in this paper is to extend the scope of such correspondence. Herein, we heuristically derive the explicit formula of state-evolution equations, which macroscopically describe VAMP dynamics for the current model-mismatched case, and show that their fixed point is generally consistent with the replica symmetric solution obtained by the replica method of statistical mechanics. We also show that the fixed point of VAMP can exhibit a microscopic instability, which indicates that message variables continue to move by VAMP while their macroscopically summarized quantities converge to fixed values. The critical condition the for microscopic instability agrees with that for breaking the replica symmetry that is derived within the non-rigorous replica analysis. The results of the numerical experiments cross-check our findings.

Quantitative matching of forensic evidence fragments utilizing 3D microscopy analysis of fracture surface replicas.

Silicone casts are widely used by practitioners in the comparative analysis of forensic items. Fractured surfaces carry unique details that can provide accurate quantitative comparisons of forensic fragments. In this study, a statistical analysis comparison protocol was applied to a set of 3D topological images of fractured surface pairs and their replicas to provide confidence in the quantitative statistical comparison between fractured items and their silicone cast replicas. A set of 10 fractured stainless steel samples were fractured from the same metal rod under controlled conditions and were replicated using a standard forensic casting technique. Six 3D topological maps with 50% overlap were acquired for each fractured pair. Spectral analyses were utilized to identify the correlation between topological surface features at different length scales of the surface topology. We selected two frequency bands over the critical wavelength (greater than two‐grain diameters) for statistical comparison. Our statistical model utilized a matrix‐variate t‐distribution that accounts for overlap between images to model match and non‐match population densities. A decision rule identified the probability of matched and unmatched pairs of surfaces. The proposed methodology correctly classified the fractured steel surfaces and their replicas with a posterior probability of match exceeding 99.96%. Moreover, the replication technique shows potential in accurately replicating fracture surface topological details with a wavelength greater than 20 μm, which far exceeds the feature comparison range on most metallic alloy surfaces. Our framework establishes the basis and limits for forensic comparison of fractured articles and their replicas while providing a reliable fracture mechanics‐based quantitative statistical forensic comparison.

Identifying Wheel-Thrown Vases in Middle Minoan Crete? Preliminary Analysis of Experimental Replicas of Plain Handleless Conical Cups from Protopalatial Phaistos

Recent work in Middle Bronze Age Crete has revealed that most Protopalatial or First Palace period pottery is produced through the use of a combination of coil-building and the wheel, i.e., wheelcoiling. Experimental work conducted on pottery from Minoan sites of Northern and Eastern Crete (e.g., Knossos, Myrtos Pyrgos, Palaikastro) has indeed determined that Minoan potters did not develop the skills required to adopt the wheel-throwing technique. However, my recent technological study of Protopalatial ceramic material from Middle Minoan IIA (19th century BC) deposits from the First Palace at Phaistos, in Southern Crete, has revealed that though pottery was produced by the wheelcoiling techniques, yet other forming techniques were practised too. In this paper I present a preliminary analysis of experimental replicas of MM IIA Phaistian plain handleless conical cups, manufactured on the potter’s wheel using three different forming techniques: wheel-pinching, wheel-coiling, and throwing-off-the-hump. This analysis will proffer answers to several questions on the use of the potter’s wheel in Middle Bronze Age Crete and opens the possibility that at MM IIA Phaistos there co-existed potters who had developed skills to employ different forming techniques on the wheel, including possibly that of throwing-off-the-hump.

TEM replica analysis of particle phases in a tempered martensite ferritic Cr steel after long term creep

Tempered martensite ferritic steels (TMFSs) have been and are being used for critical components in high temperature plant operating in the 600 °C range. They are exposed to creep conditions for long time periods, exceeding 100,000 h. In the present study we investigate a 12% Cr TMFS, after creep-testing at 550 °C at 120 MPa for 139,000 h. We had previously investigated this material in the TEM using thin foils. We now use an extraction replica technique to analyze four particle families: M23C6, MX, Laves-phase and Z-phase, considering statistically relevant numbers of particles (between 120 and 720). We show how EELS mapping can help in identifying Z-phase particles and use Cr-V-maps to differentiate between the four particle families. The chemical evolution of particles is investigated. The experimental results are discussed in the light of previous thin foil data and with respect to predictions from computational thermodynamics. The strength and weakness of thin foil and replica procedures are compared. Improvements for thermodynamic databases are suggested.

Comparative Study on Remaining Life Assessment of a Pressure Vessel, Working in Severe Conditions, Based on Results of Metallographic Replicas and Creep Tests

The paper presents and compares the results on the reliability and remaining life assessment of a reactor (coxing box) from a petrochemical plant. The reactor shell is made of 16Mo5 (W1.5423) steel, with a thickness of 25 mm, plated with 3 mm thick X6CrAl13 (W1.4002) stainless steel. The assessment was made in two steps. For preliminary remnant life assessment, specifications of section VII of the ASME code was used followed by iRiS‑Thermo expert system. Further, experimental creep and metallographic replica analysis were performed. Results comparison of the two methods applied revealed a reduction of the preliminary estimated remaining live obtained using metallographic replica analysis. Based on the results obtained, the possibility to extend the service duration of the coxing box in the safety condition, using current process parameters, with of 20.000 hours was highlighted.

Replica analysis of overfitting in generalized linear regression models

Nearly all statistical inference methods were developed for the regime where the number N of data samples is much larger than the data dimension p. Inference protocols such as maximum likelihood (ML) or maximum a posteriori probability (MAP) are unreliable if , due to overfitting. This limitation has for many disciplines with increasingly high-dimensional data become a serious bottleneck. We recently showed that in Cox regression for time-to-event data the overfitting errors are not just noise but take mostly the form of a bias, and how with the replica method from statistical physics one can model and predict this bias and the noise statistics. Here we extend our approach to arbitrary generalized linear regression models (GLM), with possibly correlated covariates. We analyse overfitting in ML/MAP inference without having to specify data types or regression models, relying only on the GLM form, and derive generic order parameter equations for the case of L2 priors. Second, we derive the probabilistic relationship between true and inferred regression coefficients in GLMs, and show that, for the relevant hyperparameter scaling and correlated covariates, the L2 regularization causes a predictable direction change of the coefficient vector. Our results, illustrated by application to linear, logistic, and Cox regression, enable one to correct ML and MAP inferences in GLMs systematically for overfitting bias, and thus extend their applicability into the hitherto forbidden regime .

Непрофессиональный политический дискурс: интенциональный аспект (на материале интернет-комментариев к политическим новостям)

This study examines the issues of generating non-professional political discourse in an intentional aspect. Non-professional political discourse is born on special platforms designed to discuss events of a political nature by each user, regardless of their profession, social status, etc. Among these venues is the space of Internet comments. The multiple nature of their generation is determined by a number of discourse-forming factors (linguistic in combination with extra-linguistic ones), including the intentions of Internet users: to learn the latest political events and give their own assessment of what is happening, discuss current news and argue their political position. Despite the dependence of these texts on the primary text of political news, as well as on the preconditioned set form of a commentary, the analysis of user replicas explicates a rich repertoir of intentions of the participants. The author suggests considering the category of intentionality in the aspect of its implementation in non-professional political discourse in three guises: receptive-cognitive, communicative-interpretative and communicative-interactive.

Compressed Sensing Performance Analysis Via Replica Method Using Bayesian Framework

Compressive sensing (CS) is a new methodology to capture signals at lower rate than the Nyquist sampling rate when the signals are sparse or sparse in some domain. The performance of CS estimators is analyzed in this paper using tools from statistical mechanics, especially called replica method. This method has been used to analyze communication systems like Code Division Multiple Access (CDMA) and multiple input multi- ple output (MIMO) systems with large size. Replica analysis, now days rigorously proved, is an efficient tool to analyze large systems in general. Specifically, we analyze the performance of some of the estimators used in CS like LASSO (the Least Absolute Shrinkage and Selection Operator) estimator and Zero-Norm regularizing estimator as a special case of maximum a posteriori (MAP) estimator by using Bayesian framework to connect the CS estimators and replica method. We use both replica symmetric (RS) ansatz and one-step replica symmetry breaking (1RSB) ansatz, clamming the latter is efficient when the problem is not convex. This work is more analytical in its form. It is deferred for next step to focus on the numerical results.

Presynaptic GABAB receptors functionally uncouple somatostatin interneurons from the active hippocampal network.

Information processing in cortical neuronal networks relies on properly balanced excitatory and inhibitory neurotransmission. A ubiquitous motif for maintaining this balance is the somatostatin interneuron (SOM-IN) feedback microcircuit. Here, we investigated the modulation of this microcircuit by presynaptic GABAB receptors (GABABRs) in the rodent hippocampus. Whole-cell recordings from SOM-INs revealed that both excitatory and inhibitory synaptic inputs are strongly inhibited by GABABRs, while optogenetic activation of the interneurons shows that their inhibitory output is also strongly suppressed. Electron microscopic analysis of immunogold-labelled freeze-fracture replicas confirms that GABABRs are highly expressed presynaptically at both input and output synapses of SOM-INs. Activation of GABABRs selectively suppresses the recruitment of SOM-INs during gamma oscillations induced in vitro. Thus, axonal GABABRs are positioned to efficiently control the input and output synapses of SOM-INs and can functionally uncouple them from local network with implications for rhythmogenesis and the balance of entorhinal versus intrahippocampal afferents.

Imaginary replica analysis of loopy regular random graphs

We present an analytical approach for describing spectrally constrained maximum entropy ensembles of finitely connected regular loopy graphs, valid in the regime of weak loop-loop interactions. We derive an expression for the leading two orders of the expected eigenvalue spectrum, through the use of infinitely many replica indices taking imaginary values. We apply the method to models in which the spectral constraint reduces to a soft constraint on the number of triangles, which exhibit ‘shattering’ transitions to phases with extensively many disconnected cliques, to models with controlled numbers of triangles and squares, and to models where the spectral constraint reduces to a count of the number of adjacency matrix eigenvalues in a given interval. Our predictions are supported by MCMC simulations based on edge swaps with nontrivial acceptance probabilities.

Replica analysis of Bayesian data clustering

We use statistical mechanics to study model-based Bayesian data clustering. In this approach, each partition of the data into clusters is regarded as a microscopic system state, the negative data log-likelihood gives the energy of each state, and the data set realisation acts as disorder. Optimal clustering corresponds to the ground state of the system, and is hence obtained from the free energy via a low ‘temperature’ limit. We assume that for large sample sizes the free energy density is self-averaging, and we use the replica method to compute the asymptotic free energy density. The main order parameter in the resulting (replica symmetric) theory, the distribution of the data over the clusters, satisfies a self-consistent equation which can be solved by a population dynamics algorithm. From this order parameter one computes the average free energy, and all relevant macroscopic characteristics of the problem. The theory describes numerical experiments perfectly, and gives a significant improvement over the mean-field theory that was used to study this model in past.

Noise analysis of replica driving technique and its verification to 12‐bit 200 MS/s pipelined ADC

This study demonstrates the noise analysis of a replica driving MDAC architecture, which is verified by implementing a 12-bit 200 MS/s replica driving pipelined analogue-to-digital converter (ADC). Based on the noise design strategy with the target effective number of bits = 10.5-bit, the overall dynamic performance degradation by KT/C noise and thermal noise by an amplifier is alleviated by removing the front-end sample-and-hold (S/H) circuit, and the transconductance (g m) of the inner source follower is maximised by increasing the current and threshold voltage (V T) reduction. Replica input sampling networks are designed for the first-stage sub-ADC and the first-stage MDAC with different aspect ratios to minimise the sampling skew for the S/H-less architecture. A prototype 12-bit 200 MS/s ADC is fabricated in a 65 nm complementary metal oxide semiconductor. The measured spurious-free dynamic range (SFDR) and signal-to-noise distortion ratio (SNDR) at a 1.0 MHz input signal is 82.6 and 65.6 dB, respectively, and SFDR and SNDR at the Nyquist (=99.0 MHz) input are 77.3 and 58.6 dB, respectively. The ADC core and the reference driver consume 53.9 and 13.2 mW, respectively, at a 1.2 V supply voltage.

Minimal Investment Risk with Cost and Return Constraints: A Replica Analysis

Previous studies into the budget constraint of portfolio optimization problems based on statistical mechanical informatics have not considered that the purchase cost per unit of each asset is distinct. Moreover, the fact that the optimal investment allocation differs depending on the size of investable funds has also been neglected. In this paper, we approach the problem of investment risk minimization using replica analysis. This problem imposes cost and return constraints. We also derive the macroscopic theory indicated by the optimal solution and confirm the validity of our proposed method through numerical experiments.