Standard Validation Methods Research Articles

Construction and validation of an overconfidence scale in investment decisions

Existing studies that directly measure the three types of overconfidence—overprecision, overplacement, and overestimation—are largely exploratory, highlighting the need for further confirmatory research to establish robust overconfidence measures. Moreover, there is no comprehensive instrument for analyzing these types of overconfidence in the context of investment decisions. To address these gaps, the present study aims to develop and validate a scale for assessing overconfidence in investment decisions, considering its three forms. The research involved 380 Brazilian participants and employed confirmatory factor analysis and standard multivariate validation methods. The scale, based on a theoretical framework of the three types of overconfidence, consisted of 30 questions (10 for each type) designed to measure the degree of overconfidence in investment decision-making. The scale demonstrated acceptable factor loadings and internal and external validity, indicating its suitability for evaluating overconfidence in investment decisions. The standardized factor loadings showed p values < 0.01 for all factors. The average variance extracted was above 0.5, and the composite reliability exceeded 0.7 for all three constructs analyzed. These results support the convergent validity of the proposed measurement model. In this way, the developed scale fills a gap in the literature by providing a valid tool to assess overconfidence in investment decision-making, which can be practically applied by financial market participants, such as investors, fund managers, and financial professionals. This instrument enables more informed and personalized investment decisions, mitigates the risks of bias in decision-making, and contributes to a more efficient and resilient financial market.

Catalytic performances of CeO2@SBA-15 as nanostructured material for biodiesel production from Podocarpus falcatus oil

Biodiesel was synthesized by transesterification of Podocapus falcatus oil (PFO) using functionalized mesoporous silica (CeO2@SBA-15) as a heterogeneous nanocatalyst. SBA-15 and CeO2@SBA-15 catalysts were synthesized using one-step solvothermal method. Physicochemical properties of both catalysts (SBA-15 and CeO2@SBA-15) were analyzed using X-ray diffraction (XRD), thermogravimetric analysis (TGA), Fourier transform infrared (FT-IR) spectroscopy, scanning electron microscopy (SEM) and N2 adsorption-desorption. In addition, PFO and resulting biodiesel were characterized by FT-IR, followed by the physicochemical properties’ determination. To optimize the production of biodiesel, some parameters such as reaction temperature, reaction time, catalyst concentration and methanol/oil molar ratio were varied. As a result, the optimum amount of 80.2 % in biodiesel was obtained in a molar ratio of 40:1 (alcohol: oil) at 6 % weight of catalyst, at 100 °C for 5 h. Obtained results from physicochemical properties of the synthesized biodiesel were in good agreement with the standard validation method of American Society for Testing and Materials (ASTM6751).

Functional Nanomaterials Enhancing Electrochemical Biosensors as Smart Tools for Detecting Infectious Viral Diseases.

Electrochemical biosensors are known as analytical tools, guaranteeing rapid and on-site results in medical diagnostics, food safety, environmental protection, and life sciences research. Current research focuses on developing sensors for specific targets and addresses challenges to be solved before their commercialization. These challenges typically include the lowering of the limit of detection, the widening of the linear concentration range, the analysis of real samples in a real environment and the comparison with a standard validation method. Nowadays, functional nanomaterials are designed and applied in electrochemical biosensing to support all these challenges. This review will address the integration of functional nanomaterials in the development of electrochemical biosensors for the rapid diagnosis of viral infections, such as COVID-19, middle east respiratory syndrome (MERS), influenza, hepatitis, human immunodeficiency virus (HIV), and dengue, among others. The role and relevance of the nanomaterial, the type of biosensor, and the electrochemical technique adopted will be discussed. Finally, the critical issues in applying laboratory research to the analysis of real samples, future perspectives, and commercialization aspects of electrochemical biosensors for virus detection will be analyzed.

A Method for Increasing the Robustness of Stable Feature Selection for Biomarker Discovery in Molecular Medicine Developed Using Serum Small Extracellular Vesicle Associated miRNAs and the Barrett's Oesophagus Disease Spectrum.

The biomarker development field within molecular medicine remains limited by the methods that are available for building predictive models. We developed an efficient method for conservatively estimating confidence intervals for the cross validation-derived prediction errors of biomarker models. This new method was investigated for its ability to improve the capacity of our previously developed method, StaVarSel, for selecting stable biomarkers. Compared with the standard cross validation method, StaVarSel markedly improved the estimated generalisable predictive capacity of serum miRNA biomarkers for the detection of disease states that are at increased risk of progressing to oesophageal adenocarcinoma. The incorporation of our new method for conservatively estimating confidence intervals into StaVarSel resulted in the selection of less complex models with increased stability and improved or similar predictive capacities. The methods developed in this study have the potential to improve progress from biomarker discovery to biomarker driven translational research.

Proposal and validation of a financial protection perception scale

AbstractThis article proposes and validates a financial protection perception scale (FPPS) from the perspective of the financial services consumer. The scale was applied to a sample of 1137 individuals residing in Brazil and validated using exploratory and confirmatory factor analysis and standard multivariate validation methods. The FPPS, composed of 14 items, allows the development of an indicator that defines the level of protection perceived by the consumer of financial services, evaluated along the dimensions of transparency, suitability, security, and complaints. The tests and fit indices indicated the reliability and validity of the FPPS. With this scale, researchers, managers, and policymakers can assess the extent to which consumers feel protected when using financial services.

Identification of the types of disease for tomato plants using a modified gray wolf optimization optimized MobileNetV2 convolutional neural network architecture driven computer vision framework

SUMMARYTomato is a widely consumed fruit across the world due to its high nutritional values. Leaf diseases in tomato are very common which incurs huge damages but early detection of leaf diseases can help in avoiding that. The existing practices for detecting different diseases by the human experts are costly, time consuming and subjective in nature. Computer vision plays important role toward early detection of tomato leaf detection. However, implementation of computationally less expensive model and improvement of detection performance is still open. This article reports a computer vision based system to classify seven different categories of diseases, namely, bacterial spot, early blight, late blight, leaf mold, septoria leaf spot, spider mites, and target spots using optimized MobileNetV2 architecture. A modified gray wolf optimization approach has been adopted for optimization of MobileNetV2 hyperparameters for improved performance. The model has been validated using standard internal and external validation methods and found to provide the classification accuracy in the tune of 98%. The results reflect the promising potential of the presented framework for early detection of tomato leaf diseases which can help to avoid substantial agricultural loss.

A validation standard for area of habitat maps for terrestrial birds and mammals

Abstract. Area of habitat (AOH) is a deductive model which maps the distribution of suitable habitats at suitable altitudes for a species inside its broad geographical range. The AOH maps have been validated using presence-only data for small subsets of species for different taxonomic groups, but no standard validation method exists when absence data are not available. We develop a novel two-step validation protocol for AOH which includes first a model-based evaluation of model prevalence (i.e, the proportion of suitable habitat within a species' range), and second a validation using species point localities (presence-only) data. We applied the protocol to AOH maps of terrestrial birds and mammals. In the first step we built logistic regression models to predict expected model prevalence (the proportion of the range retained as AOH) as a function of each species' elevation range, mid-point of elevation range, number of habitats, realm and, for birds, seasonality. AOH maps with large differences between observed and predicted model prevalence were identified as outliers and used to identify a number of sources of systematic error which were then corrected when possible. For the corrected AOH, only 1.7 % of AOH maps for birds and 2.3 % of AOH maps for mammals were flagged as outliers in terms of the difference between their observed and predicted model prevalence. In the second step we calculated point prevalence, the proportion of point localities of a species falling in pixels coded as suitable in the AOH map. We used 48 336 141​​​​​​​ point localities for 4889 bird species and 107 061 point localities for 420 mammals. Where point prevalence exceeded model prevalence, the AOH was a better reflection of species' distribution than random selection. We also found that 4689 out of 4889 (95.9 %) AOH maps for birds, and 399 out of 420 (95.0 %) AOH maps for mammals were better than random. Possible reasons for the poor performance of a small proportion of AOH maps are discussed.

A color channel based on multiple Random Forest coupled with a computer vision technique for the detection and prediction of Sudan dye‐I adulteration in turmeric powder

AbstractAdulterants can cause different health hazards upon prolonged consumption, but it is difficult to detect with human eyes. Non‐destructive turmeric adulteration detection is a challenging research area. The existing adulteration detection processes are largely instrumental and analytical with high accuracy but include limitations like long testing time, expensiveness, and lack of mobility. This work reports a new computer vision framework, which can simultaneously detect the presence of adulteration and predict the possible percentage of adulteration addressing the stated limitations. The scope has been remained to screening of Sudan dye‐I adulteration in turmeric powder. An in‐house database prepared with images of pure and adulterated turmeric powder samples has been used for experimentation. Random Forest algorithm has been employed for both classification and prediction. The model has been validated with standard internal and external validation methods to assess the stability and generalization potential of the model to avoid over‐ and under‐fitting problems. The results of classification show that the presented framework can provide more than 99% accuracy in detection while high correlation coefficient (R2) value in the tune of .99 for prediction. The novelty of the work is its simple histogram‐based color feature extraction, development of ensemble Random Forest prediction model that resulted in high accuracy and development of a faster, non‐invasive, less‐expensive, and validated screening method for adulterated turmeric powder that can be considered as a potential immediate screening method in the supply chain of powdered spices prior to confirmatory testing methods following two‐tiered food fraud testing approach.

The reliability and validity of the Turkish version of the postdialysis fatigue scale.

Haemodialysis (HD) commonly leads to postdialysis fatigue (PDF) and, thus, worsens the quality of life and the clinical outcomes. The aim of this study was to translate, linguistically validate and determine the psychometric properties of the "Postdialysis Fatigue" Scale for Turkish patients with HD. A methodological study. The standard lingual translation and validation methods for the initial translation of the PDF Scale and then assessed the psychometric properties of reliability and validity of the translated scale. Visual analogue scale (VAS) and Piper Fatigue Scale were used to assess concurrent and construct validity. In addition, postdialysis recovery time was used it as an indirect measurement method for PDF. A total of 93 patients participated in the study. The Cronbach's alpha coefficient of PDF was 0.773. There was a statistically significant relationship between the test and retest scores (ICC: 0.973; P<.001). PDF Scale was acceptable with 11 items. There was a positive correlation with a statistically significant relationship was present between the PDF scores and Piper Fatigue Scale (r=.488; P<.001) and VAS score (r=.656; P<.001). There was a statistically significant positive correlation between the PDF Scale and the postdialysis recovery time (r=.270; P=.009), CONCLUSION: The study results have shown that the Turkish form of the PDF Scale is a valid and reliable tool in patients undergoing HD treatment. Postdialysis recovery time may be used as an indirect measurement method to evaluate PDF.

Spatial validation reveals poor predictive performance of large-scale ecological mapping models

Mapping aboveground forest biomass is central for assessing the global carbon balance. However, current large-scale maps show strong disparities, despite good validation statistics of their underlying models. Here, we attribute this contradiction to a flaw in the validation methods, which ignore spatial autocorrelation (SAC) in data, leading to overoptimistic assessment of model predictive power. To illustrate this issue, we reproduce the approach of large-scale mapping studies using a massive forest inventory dataset of 11.8 million trees in central Africa to train and validate a random forest model based on multispectral and environmental variables. A standard nonspatial validation method suggests that the model predicts more than half of the forest biomass variation, while spatial validation methods accounting for SAC reveal quasi-null predictive power. This study underscores how a common practice in big data mapping studies shows an apparent high predictive power, even when predictors have poor relationships with the ecological variable of interest, thus possibly leading to erroneous maps and interpretations.

Comparison of Statistical Analysis Models for Susceptibility Assessment of Earthquake-Triggered Landslides: A Case Study from 2015 Earthquake in Lefkada Island

The main purpose of this study is to comparatively assess the susceptibility of earthquake-triggered landslides in the island of Lefkada (Ionian Islands, Greece) using two different statistical analysis models, a bivariate model represented by frequency ratio (FR), and a multivariate model represented by logistic regression (LR). For the implementation of the models, the relationship between geo-environmental factors contributing to landslides and documented events related to the 17th November 2015 earthquake was investigated by geographic information systems (GIS)-based analysis. A landslide inventory with events attributed to the specific earthquake was prepared using satellite imagery interpretation and field surveys. Eight factors: Elevation, slope angle, slope aspect, distance to main road network, distance to faults, land cover, geology, and peak ground acceleration (PGA), were considered and used as thematic data layers. The prediction capability of the models and the accuracy of the resulting susceptibility maps were tested by a standard validation method, the receiver operator characteristic (ROC) analysis. Based on the validation results, the output map with the highest reliability could potentially constitute an ideal basis for use within regional spatial planning as well as for the organization of emergency actions by local authorities.

Reliability and Validity of the Turkish Version of the FACIT-PAL Quality of Life Instrument

ContextThe accurate measurement of quality of life (QoL) among people with chronic and incurable illnesses is essential for evaluating service delivery, understanding the impact of illness and treatment effects, and testing intervention effectiveness. Palliative care interventions are relatively new in Turkey; therefore, it is important that reliable and valid QoL instruments are available to evaluate palliative care effectiveness in Turkish speakers. ObjectivesThe purpose of this study was to translate, linguistically validate, and determine the psychometric properties of the newly translated Functional Assessment of Chronic Illness Therapy–Palliative Care (FACIT-Pal) scale (FACIT-Pal-TR) for Turkish-speaking cancer patients. MethodsWe used standard multilingual translation and validation methods for the initial translation of the FACIT-Pal-TR and then assessed the psychometric properties of reliability and validity of the translated scale. We used the Karnofsky Performance Scale and Edmonton Symptom Assessment Scale to assess concurrent and construct validity. ResultsA convenience sample of 232 cancer patients participated in this study. The Cronbach's alpha coefficient of FACIT-Pal-TR was 0.932 (between 0.732 and 0.860 for subscales). There was a statistically significant relationship between test and retest scores (r = 0.877, P < 0.001). The factor loadings of FACIT-Pal-TR were between 0.205 and 0.815. FACIT-Pal-TR construct validity was acceptable with 45 items and five subscales. There were statistically significant relationships between FACIT-Pal-TR scores and Edmonton Symptom Assessment Scale (P < 0.001) and Karnofsky Performance Scale (r = 0.656; P < 0.001) scores. ConclusionsThe FACIT-Pal-TR demonstrates strong reliability and validity for evaluating palliative care–specific QoL in Turkish cancer patients.

A novel approach for segmentation and classification of brain MR images using cluster deformable based fusion approach.

Segmentation of tumor form brain MR images is the most important and tedious task in the medical field. In this paper, A Cluster deformable based fusion approach which uses both deformable and K-Means clustering scheme for Segmentation is discussed. The features of tumor and non tumor cases are extracted with the use of the Power Local Binary Pattern (LBP) Operator after completion of the segmentation process. The extracted features are fed to Naive Bayes classifier to perform the process of classification. Here, the validation of the proposed system is done using standard validation methods such as accuracy, specificity, sensitivity and RoC metrics. The developed method is applied for MR images collected from standard SimBRATS database. Experimentation results shows that the proposed method performs better when compared to the traditional clustering and deformable methods and this scheme got accuracy of 84.8%.

Testing MapReduce programs: A systematic mapping study

SummaryContextMapReduce is a processing model used in Big Data to facilitate the analysis of large data under a distributed architecture.ObjectiveThe aim of this study is to identify and categorize the state of the art of software testing in MapReduce applications, determining trends and gaps.MethodSystematic mapping study to discuss and classify according to international standards 54 relevant studies in relation to reasons for testing, types of testing, quality characteristics, test activities, tools, roles, processes, test levels, and research validations.ResultsThe principal reasons for testing MapReduce applications are performance issues, potential failures, issues related to the data, or to satisfy the agreements with efficient resources. The efforts are focused on performance and, to a lesser degree, on functionality. Performance testing is carried out through simulation and evaluation, whereas functional testing considers some program characteristics (such as specification and structure). Despite the type of testing, the majority of efforts are focused at the unit and integration test levels of the specific MapReduce functions without considering other parts of the technology stack.ConclusionsResearchers have both opportunities and challenges in performance and functional testing, and there is room to improve their research though the use of mature and standard validation methods.

Methods for Activity Monitor Validation Studies: An Example With the Fitbit Charge

Purpose: To advance research practices with consumer monitors, standard validation methods are needed. This study provides an example of best practices through systematically evaluating the validity of the Fitbit Charge (FBC) under free-living conditions using a strong reference measure and robust measurement agreement methods. Methods: 94 healthy participants (Mage 41.8 ±9.3 yrs) wore a FBC and two research grade accelerometers (Actigraph GT3X and activPAL) as they went about normal activities for a week. Estimated daily minutes of moderate to vigorous physical activity (MVPA) from the FBC were compared against reference estimates obtained from the Sojourns Including Posture (SIP) methodology, while daily step counts were compared against the activPAL. Results: Correlations with reference indicators were high for average daily MVPA (r = 0.8; p &lt; .0001) and steps (r = 0.76; p &lt; .0001), but the FBC overestimated time spent in MVPA by 56% and steps by 15%. The mean absolute percent errors of MVPA and steps estimated by FBC were 71.5% and 30.0%, respectively. Neither of the MVPA and step estimates from the FBC fell into the ±10% equivalence zone set by the criterion. The Kappa statistics of the classification agreement between the two MVPA assessment methods was 0.32 with a low sensitivity of 30.1% but a high specificity of 96.7%. Conclusion: The FBC overestimated minutes of MVPA and steps when compared to both reference assessments in free-living conditions. Standardized reporting in future studies will facilitate comparisons with other monitors and with future versions of the FBC.

Exploring the Impact of Analysis Scale on Landslide Susceptibility Modeling: Empirical Assessment in Northern Peloponnese, Greece

The main purpose of this study is to explore the impact of analysis scale on the performance of a quantitative model for landslide susceptibility assessment through empirical analyses in the northern Peloponnese, Greece. A multivariate statistical model like logistic regression (LR) was applied at two different scales (a regional and a more detailed scale). Due to this scale difference, the implementation of the model was based on two landslide inventories representing in a different way the landslide occurrence (as point and polygon features), and two datasets of similar geo-environmental factors characterized by a different size of grid cells (90 m and 20 m). Model performance was tested by a standard validation method like receiver operating characteristics (ROC) analysis. The validation results in terms of accuracy (about 76%) and prediction ability (Area under the Curve (AUC) = 0.84) of the model revealed that the more detailed scale analysis is more appropriate for landslide susceptibility assessment and mapping in the catchment under investigation than the regional scale analysis.

TEMPO-based immuno-lateral flow quantitative detection of dengue NS1 protein

The development of a rapid, affordable, and sensitive diagnostic kit for point-of-care is important in most healthcare settings. In this follow-up paper to our previous work on quantification of dengue NS1 protein via impedimetric measurement, our present technology aims to provide quantification by utilizing proprietary stabilized and improved electroactive immunonanoparticles that bind to the target biomarker and subsequently move along toward the biofunctionalized screen-printed gold electrodes (SPGE) to generate an amperometric signal. The SPGE functions simultaneously as a signal transducer and a solid-state support for a sandwich ELISA-like immunoassay. The successful immunocomplex formation is then recorded electrochemically using a potentiostat, whereby the signal was contributed by the presence of a more hydrophilic redox label than ferrocene, namely radical TEMPO (TEMPO), on the formulated nanoparticles. In this paper, a bifunctional ligand, thiolated polyethylene glycol (PEG-thiol) polymer, was used to stabilize 20 nm gold colloidal nanoparticles (AuNPs) in the formulation. PEG was incorporated to not only prevent the salt-mediated AuNPs aggregations, but also provide an anchor for antibody and redox species conjugation. To-date, we have successfully miniaturized a 3D-printed prototype device able to sensitively detect and quantify dengue NS1 protein with only 0.6 μL human clinical serum samples diluted in a volume ratio of 1:100 (PBS diluent) in less than 30 min with a simple cyclic voltammetry analysis. The positive and negative samples were also tested with ELISA, a gold standard validation method, which means that the development of the prototype is a promising start as a point-of-care diagnostic technology.

Estimating the prediction performance of spatial models via spatial k-fold cross validation

ABSTRACTIn machine learning, one often assumes the data are independent when evaluating model performance. However, this rarely holds in practice. Geographic information datasets are an example where the data points have stronger dependencies among each other the closer they are geographically. This phenomenon known as spatial autocorrelation (SAC) causes the standard cross validation (CV) methods to produce optimistically biased prediction performance estimates for spatial models, which can result in increased costs and accidents in practical applications. To overcome this problem, we propose a modified version of the CV method called spatial k-fold cross validation (SKCV), which provides a useful estimate for model prediction performance without optimistic bias due to SAC. We test SKCV with three real-world cases involving open natural data showing that the estimates produced by the ordinary CV are up to 40% more optimistic than those of SKCV. Both regression and classification cases are considered in our experiments. In addition, we will show how the SKCV method can be applied as a criterion for selecting data sampling density for new research area.

Automatic adventitious respiratory sound analysis: A systematic review.

BackgroundAutomatic detection or classification of adventitious sounds is useful to assist physicians in diagnosing or monitoring diseases such as asthma, Chronic Obstructive Pulmonary Disease (COPD), and pneumonia. While computerised respiratory sound analysis, specifically for the detection or classification of adventitious sounds, has recently been the focus of an increasing number of studies, a standardised approach and comparison has not been well established.ObjectiveTo provide a review of existing algorithms for the detection or classification of adventitious respiratory sounds. This systematic review provides a complete summary of methods used in the literature to give a baseline for future works.Data sourcesA systematic review of English articles published between 1938 and 2016, searched using the Scopus (1938-2016) and IEEExplore (1984-2016) databases. Additional articles were further obtained by references listed in the articles found. Search terms included adventitious sound detection, adventitious sound classification, abnormal respiratory sound detection, abnormal respiratory sound classification, wheeze detection, wheeze classification, crackle detection, crackle classification, rhonchi detection, rhonchi classification, stridor detection, stridor classification, pleural rub detection, pleural rub classification, squawk detection, and squawk classification.Study selectionOnly articles were included that focused on adventitious sound detection or classification, based on respiratory sounds, with performance reported and sufficient information provided to be approximately repeated.Data extractionInvestigators extracted data about the adventitious sound type analysed, approach and level of analysis, instrumentation or data source, location of sensor, amount of data obtained, data management, features, methods, and performance achieved.Data synthesisA total of 77 reports from the literature were included in this review. 55 (71.43%) of the studies focused on wheeze, 40 (51.95%) on crackle, 9 (11.69%) on stridor, 9 (11.69%) on rhonchi, and 18 (23.38%) on other sounds such as pleural rub, squawk, as well as the pathology. Instrumentation used to collect data included microphones, stethoscopes, and accelerometers. Several references obtained data from online repositories or book audio CD companions. Detection or classification methods used varied from empirically determined thresholds to more complex machine learning techniques. Performance reported in the surveyed works were converted to accuracy measures for data synthesis.LimitationsDirect comparison of the performance of surveyed works cannot be performed as the input data used by each was different. A standard validation method has not been established, resulting in different works using different methods and performance measure definitions.ConclusionA review of the literature was performed to summarise different analysis approaches, features, and methods used for the analysis. The performance of recent studies showed a high agreement with conventional non-automatic identification. This suggests that automated adventitious sound detection or classification is a promising solution to overcome the limitations of conventional auscultation and to assist in the monitoring of relevant diseases.

Validation of Serious Games

The application of games for behavioral change has seen a surge in popularity but evidence on the efficacy of these games is contradictory. Anecdotal findings seem to confirm their motivational value whereas most quantitative findings from randomized controlled trials (RCT) are negative or difficult to interpret. One cause for the contradictory evidence could be that the standard RCT validation methods are not sensitive to serious games’ effects. To be able to adapt validation methods to the properties of serious games we need a framework that can connect properties of serious game design to the factors that influence the quality of quantitative research outcomes. The Persuasive Game Design model [1] is particularly suitable for this aim as it encompasses the full circle from game design to behavioral change effects on the user. We therefore use this model to connect game design features, such as the gamification method and the intended transfer effect, to factors that determine the conclusion validity of an RCT. In this paper we will apply this model to develop guidelines for setting up validation methods for serious games. This way, we offer game designers and researchers handles on how to develop tailor-made validation methods.