Uncertainty In Model Estimates Research Articles

Advanced AI, Machine Learning and Deep Learning Techniques for Climate Change Studies: A Review

Climate models have become an important tool in understanding as well as projecting future climate conditions. They thus become very critical to decision-making processes, especially in mitigation and adaptation strategies. This paper provides a general overview of methodologies that are applied in climate modeling, its validation, and uncertainty mapping, notably their strengths, limitations, and challenges. The paper takes the reader through all the intricacies associated with the modeling of climate systems: global circulation models, regional climate models, and statistical downscaling. Much emphasis is given to the issue of model choice depending on the research objectives and spatial scales. Thorough model validation is considered, emphasizing that observational data forms the core and good quality and availability are issues. Climate modeling should incorporate uncertainty quantification as a core activity. This paper debates estimation and representation techniques of model uncertainties applied, such as probabilistic approaches, sensitivity analysis, and ensemble modeling. Introduce an idea called here uncertainty mapping, as a means of communicating the climate-related risks. This paper sets this tone by asserting that uncertainties in climate modeling have to be communicated as clearly as possible to decision-makers. No matter how intrinsically complex and limited they are, climate models represent, by themselves, very valuable means for the task of estimating the probable futures of climates. This quest ranges from enhancing model accuracy to ever-lowering estimates of uncertainties, all impelled by new advances made in computational powers and deeper insights into the climate systems.

NetGSR: Towards Efficient and Reliable Network Monitoring with Generative Super Resolution

Network monitoring systems are a key building block in today's networks. They all follow a common framework where measurement data from network elements is aggregated at a central collector for network-wide visibility. When designing network monitoring systems, two key properties have to be taken into account: (1) efficiency, to minimize the communication overhead from network elements to the collector; (2) high-fidelity, to faithfully represent the network status. However, in presence of network dynamics, tracking the right operating point to ensure both high fidelity and efficiency is hard and we observe that prior monitoring approaches trade off one for the other. In this paper, we show that it is possible to satisfy both these properties with NetGSR, a new deep learning based solution we introduce that reconstructs the fine-grained behavior of network status at the collector while requiring low resolution measurement data from network elements. This is achieved through a combination of a new custom-tailored conditional deep generative model (DistilGAN), and a new feedback mechanism (Xaminer) based on model uncertainty estimation and denoising that allows the collector to adjust the sampling rate for measurement data from network elements, at run-time. We extensively evaluate NetGSR using three different network scenarios with corresponding real-world network monitoring datasets as well as two downstream use cases. We show that NetGSR can faithfully reconstruct fine-grained network status with 25x greater measurement efficiency than prior approaches while requiring only few ms of inference time at the collector.

Expected improvements in precision when integrating opportunistic close-kin mark-recapture data into fisheries stock assessments

Close-Kin Mark-Recapture (CKMR) sampling, by providing information on abundance and survival rates (and potentially other quantities), offers a promising new data source for fisheries stock assessments. Sample design in order to achieve a desired precision is somewhat straightforward in simple CKMR models; however when integrated within a full stock assessment model with many other data sources, the value of the data (in terms of a reduction in uncertainty of model estimates) is less clear. Herein I demonstrate, using self-test simulations, the expected improvements in precision and accuracy of derived quantities and estimated parameters within statistical catch-at-age models when opportunistic CKMR sampling is conducted and the data integrated within the assessment. By opportunistic CKMR sampling I mean to describe the genetic sampling of individuals that comprise the age composition data, such that increases in CKMR sampling would also increase the age composition samples (and vice versa). I examine the expected improvements across three life history types (cod-like, flatfish-like, and sardine-like) and different amounts of data available to the assessment, including the uncertainty and inclusion of an abundance index and the sample size and time series length of CKMR and age composition samples. Results suggest CKMR data can provide considerable improvements in accuracy and precision of spawning stock biomass at the end of the time series and parameters defining natural mortality and scale of the population, provided an adequate annual sample size is collected relative to the spawning abundance of the stock during the period of CKMR inference. The time-series length of CKMR data and uncertainty or inclusion of an abundance index played a much more moderate role in how much improvement CKMR data provided over models fit without CKMR. This result was likely a function of the model being privy to an effectively known catch time series and known steepness, allowing it to estimate stock scale and trend reasonably well without CKMR data given informative composition data. I recommend simulation analyses including stock assessments as estimation models be carried out for those considering routinely collecting and integrating CKMR data into fisheries stock assessments.

Still No Evidence for an Effect of the Proportion of Non-Native Speakers on Natural Language Complexity.

In a recent study, I demonstrated that large numbers of L2 (second language) speakers do not appear to influence the morphological or information-theoretic complexity of natural languages. This paper has three primary aims: First, I address recent criticisms of my analyses, showing that the points raised by my critics were already explicitly considered and analysed in my original work. Furthermore, I show that the proposed alternative analyses fail to withstand detailed examination. Second, I introduce new data on the information-theoretic complexity of natural languages, with the estimates derived from various language models-ranging from simple statistical models to advanced neural networks-based on a database of 40 multilingual text collections that represent a wide range of text types. Third, I re-analyse the information-theoretic and morphological complexity data using novel methods that better account for model uncertainty in parameter estimation, as well as the genealogical relatedness and geographic proximity of languages. In line with my earlier findings, the results show no evidence that large numbers of L2 speakers have an effect on natural language complexity.

Uncertainty-Aware Health Diagnostics via Class-Balanced Evidential Deep Learning.

Uncertainty quantification is critical for ensuring the safety of deep learning-enabled health diagnostics, as it helps the model account for unknown factors and reduces the risk of misdiagnosis. However, existing uncertainty quantification studies often overlook the significant issue of class imbalance, which is common in medical data. In this paper, we propose a class-balanced evidential deep learning framework to achieve fair and reliable uncertainty estimates for health diagnostic models. This framework advances the state-of-the-art uncertainty quantification method of evidential deep learning with two novel mechanisms to address the challenges posed by class imbalance. Specifically, we introduce a pooling loss that enables the model to learn less biased evidence among classes and a learnable prior to regularize the posterior distribution that accounts for the quality of uncertainty estimates. Extensive experiments using benchmark data with varying degrees of imbalance and various naturally imbalanced health data demonstrate the effectiveness and superiority of our method. Our work pushes the envelope of uncertainty quantification from theoretical studies to realistic healthcare application scenarios. By enhancing uncertainty estimation for class-imbalanced data, we contribute to the development of more reliable and practical deep learning-enabled health diagnostic systems.

Automated Speech Analysis for Risk Detection of Depression, Anxiety, Insomnia, and Fatigue: Algorithm Development and Validation Study.

While speech analysis holds promise for mental health assessment, research often focuses on single symptoms, despite symptom co-occurrences and interactions. In addition, predictive models in mental health do not properly assess the limitations of speech-based systems, such as uncertainty, or fairness for a safe clinical deployment. We investigated the predictive potential of mobile-collected speech data for detecting and estimating depression, anxiety, fatigue, and insomnia, focusing on other factors than mere accuracy, in the general population. We included 865 healthy adults and recorded their answers regarding their perceived mental and sleep states. We asked how they felt and if they had slept well lately. Clinically validated questionnaires measuring depression, anxiety, insomnia, and fatigue severity were also used. We developed a novel speech and machine learning pipeline involving voice activity detection, feature extraction, and model training. We automatically modeled speech with pretrained deep learning models that were pretrained on a large, open, and free database, and we selected the best one on the validation set. Based on the best speech modeling approach, clinical threshold detection, individual score prediction, model uncertainty estimation, and performance fairness across demographics (age, sex, and education) were evaluated. We used a train-validation-test split for all evaluations: to develop our models, select the best ones, and assess the generalizability of held-out data. The best model was Whisper M with a max pooling and oversampling method. Our methods achieved good detection performance for all symptoms, depression (Patient Health Questionnaire-9: area under the curve [AUC]=0.76; F1-score=0.49 and Beck Depression Inventory: AUC=0.78; F1-score=0.65), anxiety (Generalized Anxiety Disorder 7-item scale: AUC=0.77; F1-score=0.50), insomnia (Athens Insomnia Scale: AUC=0.73; F1-score=0.62), and fatigue (Multidimensional Fatigue Inventory total score: AUC=0.68; F1-score=0.88). The system performed well when it needed to abstain from making predictions, as demonstrated by low abstention rates in depression detection with the Beck Depression Inventory and fatigue, with risk-coverage AUCs below 0.4. Individual symptom scores were accurately predicted (correlations were all significant with Pearson strengths between 0.31 and 0.49). Fairness analysis revealed that models were consistent for sex (average disparity ratio [DR] 0.86, SD 0.13), to a lesser extent for education level (average DR 0.47, SD 0.30), and worse for age groups (average DR 0.33, SD 0.30). This study demonstrates the potential of speech-based systems for multifaceted mental health assessment in the general population, not only for detecting clinical thresholds but also for estimating their severity. Addressing fairness and incorporating uncertainty estimation with selective classification are key contributions that can enhance the clinical utility and responsible implementation of such systems.

Uncertainty in model estimates of global groundwater depth

Abstract Knowing the depth at which groundwater can be found below the land surface is critical for understanding its potential accessibility by ecosystems and society. Uncertainty in global scale water table depth (WTD) limits our ability to assess groundwater’s role in a water cycle altered by changing climate, land cover, and human water use. Global groundwater models offer a top–down pathway to gain this knowledge, but their uncertainty is currently poorly quantified. Here, we investigate four global groundwater models and reveal steady-state WTD disagreements of more than 100 m for one-third of the global land area. We find that model estimates of land areas with shallow groundwater at <10 m depth vary from 10% to 71% (mean of 23%). This uncertainty directly translates into subsequent assessments, as land areas with potential groundwater accessibility for forests, population, and areas equipped for irrigation, differ substantially depending on the chosen model. We explore reasons for these differences and find that contrary to observations, 3 out of 4 models show deeper water tables in humid than in arid climates and greatly overestimate how strongly topographic slope controls WTD. These results highlight substantial uncertainty associated with any global-scale groundwater analysis, which should be considered and ultimately reduced.

Improving Re-Identification by Estimating and Utilizing Diverse Uncertainty Types for Embeddings

In most re-identification approaches, embedding vectors are compared to identify the best match for a given query. However, this comparison does not take into account whether the encoded information in the embedding vectors was extracted reliably from the input images. We propose the first attempt that illustrates how all three types of uncertainty, namely model uncertainty (also known as epistemic uncertainty), data uncertainty (also known as aleatoric uncertainty), and distributional uncertainty, can be estimated for embedding vectors. We provide evidence that we do indeed estimate these types of uncertainty, and that each type has its own value for improving re-identification performance. In particular, while the few state-of-the-art approaches that employ uncertainty for re-identification during inference utilize only data uncertainty to improve single-shot re-identification performance, we demonstrate that the estimated model uncertainty vector can be utilized to modify the feature vector. We explore the best method for utilizing the estimated model uncertainty based on the Market-1501 dataset and demonstrate that we are able to further enhance the performance above the already strong baseline UAL. Additionally, we show that the estimated distributional uncertainty resembles the degree to which the current sample is out-of-distribution. To illustrate this, we divide the distractor set of the Market-1501 dataset into four classes, each representing a different degree of out-of-distribution. By computing a score based on the estimated distributional uncertainty vector, we are able to correctly order the four distractor classes and to differentiate them from an in-distribution set to a significant extent.

Leveraging multiple data types for improved compound-kinase bioactivity prediction

Machine learning provides efficient ways to map compound-kinase interactions. However, diverse bioactivity data types, including single-dose and multi-dose-response assay results, present challenges. Traditional models utilize only multi-dose data, overlooking information contained in single-dose measurements. Here, we propose a machine learning methodology for compound-kinase activity prediction that leverages both single-dose and dose-response data. We demonstrate that our two-stage approach yields accurate activity predictions and significantly improves model performance compared to training solely on dose-response labels. This superior performance is consistent across five diverse machine learning methods. Using the best performing model, we carried out extensive experimental profiling on a total of 347 selected compound-kinase pairs, achieving a high hit rate of 40% and a negative predictive value of 78%. We show that these rates can be improved further by incorporating model uncertainty estimates into the compound selection process. By integrating multiple activity data types, we demonstrate that our approach holds promise for facilitating the development of training activity datasets in a more efficient and cost-effective way.

Can deep learning classify cerebral ultrasound images for the detection of brain injury in very preterm infants?

Cerebral ultrasound (CUS) is the main imaging screening tool in preterm infants. The aim of this work is to develop deep learning (DL) models that classify normal vs abnormal CUS to serve as a computer-aided detection tool providing timely interpretation of the scans. A population-based cohort of very preterm infants (220-306 weeks) born between 2004 and 2016 in Nova Scotia, Canada. A set of nine sequential CUS images per infant was retrieved at three specific coronal landmarks at three pre-identified times (first, sixth weeks, and term age). A radiologist manually labeled each image as normal or abnormal. The dataset was split into training/development/test subsets (80:10:10). Different convolutional neural networks were tested, with filtering of the most uncertain prediction. The model's performance was assessed using precision/recall and the receiver operating area under the curve. Sequential CUS retrieved for 538/665 babies (81% of the cohort). Four thousand one hundred eighty images were used to develop and test the model. The model performance was only discrete at the beginning but, through different machine learning strategies was boosted to good levels averaging 0.86 ROC AUC (95% CI: 0.82, 0.90) and 0.87 PR AUC (95% CI: 0.84, 0.90) (model uncertainty estimation filters using normalized entropy threshold = 0.5). This study offers proof of the feasibility of applying DL to CUS. This basic diagnostic model showed good discriminative ability to classify normal versus abnormal CUS. This serves as a CAD and a framework for constructing a prognostic model. This DL model can serve as a computer-aided detection tool to classify CUS of very preterm babies as either normal or abnormal. This model will also be used as a framework to develop a prognostic model. Binary computer-aided detection models of CUS are applicable for classifying ultrasound images in very preterm babies. This model acts as a step towards developing a model for predicting neurodevelopmental outcomes in very preterm babies. This model serves as a tool for interpretation of CUS in this patient population with a heightened risk of brain injury.

Laser-excited surface acoustic wave method for detecting subsurface damage of processed silicon nitride ceramics

The non-destructive testing for subsurface damage of processed silicon nitride ceramics is significant to the improvement of processing technology and evaluation of product performance. A novel method for the measurement of subsurface damage based on dispersion of laser-excited surface acoustics wave is proposed in this paper. The subsurface damage distribution is quantified as the damage density and depth of the damage layer. The forward model of surface acoustics wave propagation in damage layer is derived by the stiffness matrix method. Bayesian inversion is applied to estimate model parameters and uncertainties of subsurface damage from the experimental dispersion data. Laser ultrasonic experiments are carried out on four samples with different substrate material parameters and surface qualities to demonstrate the feasibility of the method for detecting varying degrees of subsurface damage. The results prove that with reasonable use of prior information about damage depth predicted by surface roughness, subsurface damage with a depth ∼10 times smaller than the minimum wavelength of surface acoustics wave is accurately characterized. The reliability of the results is further verified by Vickers microhardness tester.

Estimating Uncertainty of Geographic Atrophy Segmentations with Bayesian Deep Learning

To apply methods for quantifying uncertainty of deep learning segmentation of geographic atrophy (GA). Retrospective analysis of OCT images and model comparison. One hundred twenty-six eyes from 87 participants with GA in the SWAGGER cohort of the Nonexudative Age-Related Macular Degeneration Imaged with Swept-Source OCT (SS-OCT) study. The manual segmentations of GA lesions were conducted on structural subretinal pigment epithelium en face images from the SS-OCT images. Models were developed for 2 approximate Bayesian deep learning techniques, Monte Carlo dropout and ensemble, to assess the uncertainty of GA semantic segmentation and compared to a traditional deep learning model. Model performance (Dice score) was compared. Uncertainty was calculated using the formula for Shannon Entropy. The output of both Bayesian technique models showed a greater number of pixels with high entropy than the standard model. Dice scores for the Monte Carlo dropout method (0.90, 95% confidence interval 0.87-0.93) and the ensemble method (0.88, 95% confidence interval 0.85-0.91) were significantly higher (P<0.001) than for the traditional model (0.82, 95% confidence interval 0.78-0.86). Quantifying the uncertainty in a prediction of GA may improve trustworthiness of the models and aid clinicians in decision-making. The Bayesian deep learning techniques generated pixel-wise estimates of model uncertainty for segmentation, while also improving model performance compared with traditionally trained deep learning models. Proprietary or commercial disclosure may be found in the Footnotes and Disclosures at the end of this article.

Testing the ionospheric model delay and uncertainty estimates with an uncombined navigation filter

In the last decade, new algorithmic positioning techniques have been developed for Global Navigation Satellite Systems (GNSS). These have brought a new focus on high accuracy applications which do not combine multiple frequencies to remove ionospheric errors (i.e. PPP-RTK, Fast-PPP). Not only do these algorithms focus on improvements in the position domain but also in acquiring the positioning solution as fast as possible. In this work, capabilities of different global ionospheric models are assessed, analyzing both the Ionospheric delay accuracy and the associated model uncertainty. Accurate model uncertainties are crucial for reducing the convergence time in uncombined filters, and to guarantee unbiased convergence in the first place. The assessment is done using an uncombined navigation filter with different ionospheric models: GPS ICA, IGS vTEC (vertical Total Electron Content) maps (IGSG, CODG and UQRG), two realizations of the ESA-UGI (Voxel and Multi-Layer), the Madrigal TEC, and the Spire Global vTEC maps. To quantify the model uncertainties without the use of a reference ionospheric model, global maps of an uncertainty inflation factor are computed to show the inflation required to produce optimal filter convergence. These maps demonstrate that some models are too optimistic in the reporting of their own uncertainty estimates, requiring an uncertainty factor up to 10 times the quoted value.

Parameters estimation and sensitivity analysis of lithium-ion battery model uncertainty based on osprey optimization algorithm

To advance the field of lithium-ion battery (LIB) research, this paper unveils an accurate modelling of LIB that primarily relies on the equivalent circuit model, backed by the Osprey Optimization Algorithm (OOA). In the modelling stage, both single and double resistance-capacitance models are evaluated to depict the charge dynamics, incorporating the effects of fading, load, and temperature variations. The OOA approach is utilized to minimize integral squared errors between the actual measured and model-predicted battery voltages under constraints imposed by the model design variables. This approach is applied to a commercial 2.6 Ahr Panasonic LIB, with the performance of the OOA-based model being benchmarked against models developed by means of other optimization algorithms for further validation. Moreover, the robustness of the OOA method is assessed under battery uncertainty conditions or model parameter variation. A sensitivity analysis is performed on the battery model by employing a proposed approach that evaluates the impact of varying each parameter of the battery model by ±5 %, in a sequence that ascends and descends from 0 to 5 %. The single resistance-capacitance model is selected for in-depth validations. Notably, the OOA approach excels in estimating parameters for LIB modeling under both normal and abnormal operating conditions.

Designing Cell-Type-Specific Promoter Sequences Using Conservative Model-Based Optimization.

Gene therapies have the potential to treat disease by delivering therapeutic genetic cargo to disease-associated cells. One limitation to their widespread use is the lack of short regulatory sequences, or promoters, that differentially induce the expression of delivered genetic cargo in target cells, minimizing side effects in other cell types. Such cell-type-specific promoters are difficult to discover using existing methods, requiring either manual curation or access to large datasets of promoter-driven expression from both targeted and untargeted cells. Model-based optimization (MBO) has emerged as an effective method to design biological sequences in an automated manner, and has recently been used in promoter design methods. However, these methods have only been tested using large training datasets that are expensive to collect, and focus on designing promoters for markedly different cell types, overlooking the complexities associated with designing promoters for closely related cell types that share similar regulatory features. Therefore, we introduce a comprehensive framework for utilizing MBO to design promoters in a data-efficient manner, with an emphasis on discovering promoters for similar cell types. We use conservative objective models (COMs) for MBO and highlight practical considerations such as best practices for improving sequence diversity, getting estimates of model uncertainty, and choosing the optimal set of sequences for experimental validation. Using three relatively similar blood cancer cell lines (Jurkat, K562, and THP1), we show that our approach discovers many novel cell-type-specific promoters after experimentally validating the designed sequences. For K562 cells, in particular, we discover a promoter that has 75.85% higher cell-type-specificity than the best promoter from the initial dataset used to train our models.

Adaptive Distributed Heterogeneous Formation Control for UAV-USVs with Input Quantization

This paper investigates the cooperative formation trajectory tracking problem for heterogeneous unmanned aerial vehicle (UAV) and multiple unmanned surface vessel (USV) systems with input quantization performance. Firstly, at the kinematic level, a distributed guidance law based on an extended state observer (ESO) is designed to compensate for the unknown speed of neighbor agents for expected trajectory tracking, and subsequently at the dynamic level, an ESO is utilized to estimate model uncertainties and environmental disturbances. Following that, a linear analytic model is employed to depict the input quantization process, and the corresponding adaptive quantization controller is designed without necessitating prior information on quantization parameters. Based on the input-to-state stability, the stability of the proposed control structure is proved, and all the signals in the closed-loop system are ultimately bounded. Finally, a simulation study is provided to show the efficacy of the proposed strategy.

A concept for the global assessment of tomographic resolution and uncertainty

SUMMARY A major challenge in seismic tomography consists in quantifying and representing model resolution and uncertainty, particularly at global scales. This information is crucial for interpretations of tomographic images and their technical application in geodynamics. However, due to large computational costs, there have been only few attempts so far to coherently analyse the spatially varying resolving power for a complete set of model parameters. Here, we present a concept for an effective evaluation and global representation of the 3-D resolution information contained in a full set of averaging kernels. In our case, these kernels are constructed using the ‘Subtractive Optimally Localized Averages’ (SOLA) method, a variant of classic Backus-Gilbert inversion suitable for global tomography. Our assessment strategy incorporates the following steps: (1) a 3-D Gaussian function is fitted to each averaging kernel to measure resolution lengths in different directions and (2) we define a classification scheme for the quality of the averaging kernels based on their focus with respect to the estimated 3-D Gaussian, allowing us to reliably identify whether the inferred resolution lengths are robust. This strategy is not restricted to SOLA inversions, but can, for example, be applied in all cases where point-spread functions are computed in other tomographic frameworks. Together with model uncertainty estimates that are derived from error propagation in the SOLA method, our concept reveals at which locations, resolution lengths and interpretations of model values are actually meaningful. We finally illustrate how the complete information from our analysis can be used to calibrate the SOLA inversion parameters—locally tunable target resolution kernels and trade-off parameters—without the need for visual inspection of the individual resulting averaging kernels. Instead, our global representations provide a tool for designing tomographic models with specific resolution-uncertainty properties that are useful in geodynamic applications, especially for linking seismic inversions to models of mantle flow.

Steady but model dependent Arctic amplification of the forced temperature response in 21st century CMIP6 projections

We examine sources of uncertainty in projections of Arctic amplification (AA) using the CMIP6 multi-model (MM) ensemble and single model initial-condition large ensembles of historical and future scenario simulations. In the CMIP6 MM mean, the annual mean AA ratio is steady at approximately 2.5, both in time and across scenarios, resulting in negligibly small scenario uncertainty in the magnitude of AA. Deviations from the steady value can be found at the low and high emission scenarios due to different root causes, with the latter being mostly evident in the summer and autumn seasons. Best estimates of model uncertainty are at least an order of magnitude larger than scenario uncertainty in CMIP6. The large ensembles reveal that irreducible internal variability has a similar magnitude to model uncertainty for most of the 21st century, except in the lowest emission scenario at the end of the 21st century when it could be twice as large.

F2M: Ensemble-based uncertainty estimation model for fire detection in indoor environments

Early fire detection and timely notification are paramount for preventing human and material casualties caused by fire. As a result, scientists have developed various fire monitoring systems based on sensors and images. Image-based systems have proven more advantageous than sensor-based ones as they provide additional details about the fire, such as its location, intensity, and progression. However, accurately determining the shape and boundaries of flames in image-based systems is difficult due to various factors, including backgrounds, different fire sizes, and interference from objects that resemble flames. In this study, we first evaluate convolutional neural networks used in related research and benchmark them on our challenging indoor fire dataset. Secondly, we propose a novel Feature Merging Model (F2M), which combines the output fire segmentation masks of the top five segmentation models obtained through the performed evaluation. The F2M relies on a novel mechanism that introduces bias into the mask estimation and achieves better results than all the tested methods, along with Monte-Carlo dropout for uncertainty estimation. The F2M ensemble-based uncertainty estimation model achieved improvements in comparison to the best performance convolutional neural network U-Net++, on three different image resolutions: 256 × 256, 640 × 640 and 800 × 800, according to the following metrics: Total error, Dice coefficient, and IoU score.

Watch Your Head: Assembling Projection Heads to Save the Reliability of Federated Models

Federated learning encounters substantial challenges with heterogeneous data, leading to performance degradation and convergence issues. While considerable progress has been achieved in mitigating such an impact, the reliability aspect of federated models has been largely disregarded. In this study, we conduct extensive experiments to investigate the reliability of both generic and personalized federated models. Our exploration uncovers a significant finding: federated models exhibit unreliability when faced with heterogeneous data, demonstrating poor calibration on in-distribution test data and low uncertainty levels on out-of-distribution data. This unreliability is primarily attributed to the presence of biased projection heads, which introduce miscalibration into the federated models. Inspired by this observation, we propose the "Assembled Projection Heads" (APH) method for enhancing the reliability of federated models. By treating the existing projection head parameters as priors, APH randomly samples multiple initialized parameters of projection heads from the prior and further performs targeted fine-tuning on locally available data under varying learning rates. Such a head ensemble introduces parameter diversity into the deterministic model, eliminating the bias and producing reliable predictions via head averaging. We evaluate the effectiveness of the proposed APH method across three prominent federated benchmarks. Experimental results validate the efficacy of APH in model calibration and uncertainty estimation. Notably, APH can be seamlessly integrated into various federated approaches but only requires less than 30% additional computation cost for 100x inferences within large models.