Current Methods For Evaluation Research Articles

VR rehabilitation system evaluator: A fNIRS-based and LLM-enabled evaluation paradigm for Mild Cognitive Impairment

Virtual Reality (VR) has significant potential for Mild Cognitive Impairment (MCI) rehabilitation by offering dynamic and engaging tasks designed to stimulate various cognitive domains. However, how efficiently evaluating the VR tasks is always challenging. The current effectiveness evaluation methods for task evaluation and optimisation typically rely on post-training metrics like game performance or subjective questionnaires, which cannot directly reflect the comprehensiveness and intensity of cognitive stimulation. To bridge these gaps, we propose a novel approach integrating functional near-infrared spectroscopy (fNIRS) data and a Large Language Model (LLM) for VR rehabilitation task evaluation and optimisation. Firstly, we leverage fundamental cognitive theories to propose a systematic evaluation paradigm for assessing the scope of the cognitive domains stimulated by VR tasks. Secondly, we propose fNIRS-derived graph parameters to objectively evaluate the cognitive stimulation with high-time resolution. Finally, we develop a prompt-generating strategy to facilitate Large Language Model (LLM)-based analysis, which transforms fNIRS-derived metrics combined with the scope of stimulated cognitive domains into easy-to-understand evaluation reports and actionable recommendations for task optimisation. We demonstrate this approach with two VR rehabilitation tasks. Several experts evaluated the LLM-based task evaluation results and task optimisation recommendation and agreed that the LLM-generated results are meaningful and applicable. This approach offers a rapid, objective method to evaluate VR task effectiveness and improves their design, marking a significant step in developing more effective MCI interventions.

Examining the Difficulties in Identifying and Handling Cardiac Amyloidosis; Acquiring Important Knowledge and Robust Treatment Methods.

Systemic amyloidosis is a rare protein misfolding and deposition condition that causes slow organ failure. Each of the more than 15 exclusive sorts of systemic amyloidosis, which encourage amyloid production and tissue deposition, is introduced by a unique precursor protein. Amyloidosis can affect various organs, including the heart, kidneys, liver, nerves, gastrointestinal tract, lungs, muscles, skin, and soft tissues. It can either be acquired or hereditary. Insidious and doubtful signs often cause a put-off in diagnosis. In the closing decade, noteworthy progressions have been made in the identity, prediction, and handling of amyloidosis. Shotgun proteomics based on mass spectrometry has revolutionized amyloid typing and enabled the identification of novel amyloid forms. It is critical to correctly identify the precursor protein implicated in amyloidosis because the kind of protein influences the proper treatment strategy. Cardiac amyloidosis is a disorder characterized by the systemic accumulation of amyloid protein in the myocardium's extracellular space, which causes a variety of symptoms. The buildup of amyloid aggregates precipitates myocardial thickening and stiffening, culminating in diastolic dysfunction and, in due course, heart failure. We examine every kind of systemic amyloidosis in this text to offer practitioners beneficial equipment for diagnosing and treating those unusual diseases. This review presents a comprehensive analysis of cardiac amyloidosis and consolidates current methods for screening, diagnosis, evaluation, and treatment alternatives.

Label-free model evaluation and weighted uncertainty sample selection for domain adaptive instance segmentation

This paper addresses the challenges of model evaluation and optimization that arise from domain differences between the target and source domains during model deployment. Current methods for model accuracy evaluation require a fully annotated test set. However, obtaining additional human labels for every unique application scenario can be costly and time-intensive. To tackle this problem, this paper proposes an instance segmentation model evaluation method based on domain differences, which can give the prediction accuracy of the model on unlabeled test sets. Moreover, to enhance deployment accuracy cost-effectively, this paper proposes an “effective operation”-based labeling cost computation method and a weighted uncertainty sample selection method. The former accurately computes labeling costs for instance segmentation, while the latter selects the most valuable samples from the target domain for labeling and training. Model evaluation experiments demonstrate that this method’s root mean square error (RMSE) on Cityscapes is approximately 4% less than that of other existing model evaluation methods. Model optimization experiments demonstrate that the proposed method achieves greater model accuracy than comparative methods under four distinct data partitioning protocols. The code is available at https://github.com/licongguan/Lamer.

High-density Surface Electromyography as Biomarker of Muscle Aging.

Sarcopenia is a muscle disease with adverse changes that increase throughout the lifetime but with different chronological scales between individuals. Addressing "early muscle aging" is becoming a critical issue for prevention. Through the CHRONOS study, we demonstrated the ability of the high-density surface electromyography (HD-sEMG), a noninvasive, wireless, portable technology, to detect both healthy muscle aging and accelerated muscle aging related to a sedentary lifestyle, one of the risk factors of sarcopenia. The HD-sEMG signals were analyzed in 91 healthy young, middle-aged, and old subjects (25-75 years) distributed according to their physical activity status (82 active and 9 sedentary; International Physical Activity Questionnaire) and compared with current methods for muscle evaluation, including muscle mass (dual-energy X-ray absorptiometry [DXA], ultrasonography), handgrip strength, and physical performance. The HD-sEMG signals were recorded from the rectus femoris during sit-to-stand trials, and 2 indexes were analyzed: muscular contraction intensity and muscle contraction dynamics. The clinical parameters did not differ significantly across the aging and physical activity levels. Inversely, the HD-sEMG indexes were correlated to age and were different significantly through the age categories of the 82 active subjects. They were significantly different between sedentary subjects aged 45-54 years and active ones at the same age. The HD-sEMG indexes of sedentary subjects were not significantly different from those of older active subjects (≥55 years). The muscle thicknesses evaluated using ultrasonography were significantly different between the 5 age decades but did not show a significant difference with physical activity. The HD-sEMG technique can assess muscle aging and physical inactivity-related "early aging," outperforming clinical and DXA parameters.

Three-Dimensional (3D) Imaging Technology to Monitor Growth and Development of Holstein Heifers and Estimate Body Weight, a Preliminary Study

The choice of rearing strategy for dairy cows can have an effect on production yield, at least during the first lactation. For this reason, it is important to closely monitor the growth and development of young heifers. Unfortunately, current methods for evaluation can be costly, time-consuming, and dangerous because of the need to physically manipulate animals, and as a result, this type of monitoring is seldom performed on farms. One potential solution may be the use of tools based on three-dimensional (3D) imaging, which has been studied in adult cows but not yet in growing individuals. In this study, an imaging approach that was previously validated for adult cows was tested on a pilot population of five randomly selected growing Holstein heifers, from 5 weeks of age to the end of the first gestation. Once a month, all heifers were weighed and an individual 3D image was recorded. From these images, we estimated growth trends in morphological traits such as heart girth or withers height (188.1 ± 3.7 cm and 133.5 ± 6.0 cm on average at one year of age, respectively). From other traits, such as body surface area and volume (5.21 ± 0.32 m2 and 0.43 ± 0.05 m3 on average at one year of age, respectively), we estimated body weight based on volume (402.4 ± 37.5 kg at one year of age). Body weight estimates from images were on average 9.7% higher than values recorded by the weighing scale (366.8 ± 47.2 kg), but this difference varied with age (19.1% and 1.8% at 6 and 20 months of age, respectively). To increase accuracy, the predictive model developed for adult cows was adapted and completed with complementary data on young heifers. Using imaging data, it was also possible to analyze changes in the surface-to-volume ratio that occurred as body weight and age increased. In sum, 3D imaging technology is an easy-to-use tool for following the growth and management of heifers and should become increasingly accurate as more data are collected on this population.

Contemporary challenges in the diagnosis and management of chronic inflammatory demyelinating polyneuropathy

ABSTRACT Introduction Despite extensive research, multiple inter-related diagnostic and management challenges remain for chronic inflammatory demyelinating polyneuropathy (CIDP). Areas Covered A literature review was performed on diagnosis and treatment in CIDP. The clinical features and disease course were evaluated. Investigative techniques, including electrophysiology, cerebrospinal fluid examination, neuropathology, imaging, and neuroimmunology, were considered in relation to technical aspects, sensitivity, specificity, availability, and cost. Available evidenced-based treatments and those with possible efficacy despite lack of evidence, were considered, as well as current methods for evaluation of treatment effects. Expert Opinion CIDP remains a clinical diagnosis, supported first and foremost by electrophysiology. Other investigative techniques have limited impact. Most patients with CIDP respond to available first-line treatments and immunosuppression may be efficacious in those who do not. Consideration of the natural history and of the high reported remission rate, of under-recognized associated disabling features, of treatment administration modalities and assessment methods, require enhanced attention.

Genomic Prediction of Complex Traits in Animal Breeding with Long Breeding History, the Dairy Cattle Case.

In accordance with the infinitesimal model for quantitative traits, a very large number of genes affect nearly all economic traits. In only two cases has the causative polymorphism been determined for genes affecting economic traits in dairy cattle. Most current methods for genomic evaluation are based on the "two-step" method. Genetic evaluations are computed by the individual animal model, and functions of the evaluations of progeny-tested sires are the dependent variable for estimation of marker effects. With the adoption of genomic evaluation in 2008, annual rates of genetic gain in the US increased from ∼50-100% for yield traits and from threefold to fourfold for lowly heritable traits, including female fertility, herd-life and somatic cell concentration. Gradual elimination of the progeny test scheme has led to a reduction in the number of sires with daughter records and less genetic ties between years. As genotyping costs decrease, the number of cows genotyped will continue to increase, and these records will become the basic data used to compute genomic evaluations, most likely via application of "single-step" methodologies. Less emphasis in selection goals will be placed on milk production traits, and more on health, reproduction, and efficiency traits and "environmentally friendly" production. Genetic variance for economic traits is maintained by increase in frequency of rare alleles, new mutations, and changes in selection goals and management.

Study on Relative Accuracy and Verification Method of High-Definition Maps for Autonomous Driving

High-definition maps (HDM) for autonomous driving (AD) are an important component of AD systems. HDMs accurately provide a priori information, including lane lines, and road signs, for AD systems. It is an important task to make a reasonable accuracy assessment of the HDM. The current methods for relative accuracy evaluation of general maps in the field of mapping are not fully applicable to HDMs. In this study, a method based on point set alignment and resampling is used to evaluate the relative accuracy of lane lines, and experiments are conducted based on relevant real HDM data. The results show that the relative accuracy of the lane lines is more detailed and relevant than the traditional method. This has implications for the quality control of HDM production.

Calibration of the pixel crosstalk in spatial light modulators for 4f pulse shaping.

The targeted shaping of femtosecond pulses in 4f pulse shapers is complicated by, among other factors, the crosstalk between adjacent pixels of a spatial light modulator (SLM). Current methods for the crosstalk evaluation require setting up a different experiment, which is highly inconvenient. Here, we propose a simple procedure to extract the pixel crosstalk within the standard SLM calibration used in pulse shaping. The calibration is based on an analysis of the contrast of a periodic modulation in the spectra induced via SLM. We demonstrate the calibration procedure on a liquid-crystal-based SLM and show that we attain a constant crosstalk effect represented by a Gaussian function with σ=1.0 pix over a broad operational range of the SLM.

Spatiotemporal Heterogeneity across Metastases and Organ-Specific Response Informs Drug Efficacy and Patient Survival in Colorectal Cancer.

The sum of target lesions is routinely used to evaluate patient objective responses to treatment in the RECIST criteria, but it fails to address response heterogeneity across metastases. This study argues that spatiotemporal heterogeneity across metastases and organ-specific response is informative for drug efficacy and patient survival. We analyzed the longitudinal data of 11,404 metastatic lesions in 2,802 colorectal cancer patients from five phase III clinical trials. Initially, a metric Gower distance was applied to quantify response heterogeneity across metastases. Next, the spatiotemporal response heterogeneity across anatomic sites, therapies, and KRAS mutation status was assessed and examined for its association with drug efficacy and long-term patient survival. The response of metastatic lesions broadly differed across anatomic sites and therapies. About 60% of patients had at least one lesion respond contrarily from total tumor size. High interlesion heterogeneity was associated with shorter progression-free survival and overall survival. Targeted therapies (bevacizumab or panitumumab) combined with standard chemotherapy reduced interlesion heterogeneity and elicited more favorable effects from liver lesions (P < 0.001) than chemotherapy alone. Moreover, the favorable responses in liver metastases (> 30% shrinkage) were associated with extended patient overall survival (P < 0.001), in contrast to lesions in the lungs and lymph nodes. Altogether, the spatiotemporal response heterogeneity across metastases informed drug efficacy and patient survival, which could improve the current methods for treatment evaluation and patient prognosis. SIGNIFICANCE: These findings support the modification of RECIST criteria to include individual lesion response to improve assessments of drug efficacy.

Bacterial Cell Cultures in a Lab-on-a-Disc: A Simple and Versatile Tool for Quantification of Antibiotic Treatment Efficacy.

Pathogenic bacterial biofilms can be life-threatening, greatly decrease patient's quality of life, and are a substantial burden on the healthcare system. Current methods for evaluation of antibacterial treatments in clinics and in vitro systems used in drug development and screening either do not facilitate biofilm formation or are cumbersome to operate, need large reagent volumes, and are costly, limiting their usability. To address these issues, this work presents the development of a robust in vitro cell culture platform compatible with confocal microscopy. The platform shaped as a compact disc facilitates long-term bacterial culture without external pumps and tubing and can be operated for several days without additional liquid handling. As an example, Pseudomonas aeruginosa biofilm is grown from single cells, and it is shown that (1) the platform delivers reproducible and reliable results; (2) growth is dependent on flow rate and growth medium composition; and (3) efficacy of antibiotic treatment depends on the formed biofilm. This platform enables biofilm growth, quantification, and treatment as in a conventional flow setup while decreasing the application barrier of lab-on-chip systems. It provides an easy-to-use, affordable option for end users working with cell culturing in relation to, e.g., diagnostics and drug screening.

The Identification of Novel Biomarkers Is Required to Improve Adult SMA Patient Stratification, Diagnosis and Treatment.

Spinal muscular atrophy (SMA) is currently classified into five different subtypes, from the most severe (type 0) to the mildest (type 4) depending on age at onset, best motor function achieved, and copy number of the SMN2 gene. The two recent approved treatments for SMA patients revolutionized their life quality and perspectives. However, upon treatment with Nusinersen, the most widely administered therapy up to date, a high degree of variability in therapeutic response was observed in adult SMA patients. These data, together with the lack of natural history information and the wide spectrum of disease phenotypes, suggest that further efforts are needed to develop precision medicine approaches for all SMA patients. Here, we compile the current methods for functional evaluation of adult SMA patients treated with Nusinersen. We also present an overview of the known molecular changes underpinning disease heterogeneity. We finally highlight the need for novel techniques, i.e., -omics approaches, to capture phenotypic differences and to understand the biological signature in order to revise the disease classification and device personalized treatments.

Development of Railway Ride Comfort Prediction Model: Incorporating Track Geometry and Rolling Stock Conditions

AbstractPassenger ride comfort (PRC) is one of the most important performance indicators of railway transportation. The current methods for computation and evaluation of railway ride comfort requir...

Performance of the MasSpec Pen for Rapid Diagnosis of Ovarian Cancer.

Accurate tissue diagnosis during ovarian cancer surgery is critical to maximize cancer excision and define treatment options. Yet, current methods for intraoperative tissue evaluation can be time intensive and subjective. We have developed a handheld and biocompatible device coupled to a mass spectrometer, the MasSpec Pen, which uses a discrete water droplet for molecular extraction and rapid tissue diagnosis. Here we evaluated the performance of this technology for ovarian cancer diagnosis across different sample sets, tissue types, and mass spectrometry systems. MasSpec Pen analyses were performed on 192 ovarian, fallopian tube, and peritoneum tissue samples. Samples were evaluated by expert pathologists to confirm diagnosis. Performance using an Orbitrap and a linear ion trap mass spectrometer was tested. Statistical models were generated using machine learning and evaluated using validation and test sets. High performance for high-grade serous carcinoma (n = 131; clinical sensitivity, 96.7%; specificity, 95.7%) and overall cancer (n = 138; clinical sensitivity, 94.0%; specificity, 94.4%) diagnoses was achieved using Orbitrap data. Variations in the mass spectra from normal tissue, low-grade, and high-grade serous ovarian cancers were observed. Discrimination between cancer and fallopian tube or peritoneum tissues was also achieved with accuracies of 92.6% and 87.9%, respectively, and 100% clinical specificity for both. Using ion trap data, excellent results for high-grade serous cancer vs normal ovarian differentiation (n = 40; clinical sensitivity, 100%; specificity, 100%) were obtained. The MasSpec Pen, together with machine learning, provides robust molecular models for ovarian serous cancer prediction and thus has potential for clinical use for rapid and accurate ovarian cancer diagnosis.

Remote Monitoring of Reliability for Water Conveyance Hydraulic Structures

Analysis of current methods for control and evaluation of technical state for different water conveyance constructions in the field of hydraulic engineering and reclamation was carried out and it showed the necessity of their improvement and development. Significant indices of technical state and operational capability resource of the constructions were defined. Promising methods for evaluation of structures technical state based on the methods of non-destroying control are substantiated. Fundamental principles to carry out technical state monitoring and evaluation of remaining live of projects operational capability were formulated, with the results of methodological approaches analysis in various fields of knowledge were taken into account. Automated technology for technical diagnostics of constructions by the methods of non-destroying instrumental control was developed. This control secures determination of remaining live operational capability of structural elements and structure safety deficit for information support of decision making on operation measures. Ecologo-economic efficiency from adaptation of software and hardware complex for reliability diagnostics of hydraulic structures is stipulated by time expenditure shortening on survey carrying out, by higher quality of decision making and by automation of routine operations for information, collecting, processing and interpretating.

Stochastic response analysis for nonlinear vibration systems with adjustable stiffness property under random excitation.

Nonlinear vibration systems with adjustable stiffness property have attracted considerable attentions for their prominent broadband performances. In the present manuscript, we consider the stochastic dynamical systems with adjustable stiffness and proposed a numerical method for the random responses analysis of the Gaussian white noise excited systems. A multi-dimensional Fokker-Plank-Kolmogorov equation governing the joint probability density of the mechanical states is derived according to the theory of diffusion processes. We solve the multi-dimensional equation using a splitting method and obtained the stationary probability densities and the mean-square responses directly. Two classical nonlinear vibration systems with adjustable stiffness, including the energy harvesting system and the Duffing system with Dahl friction, are presented as examples. Their comparisons with the results from Monte-Carlo simulations illustrate the effectiveness of the proposed procedure for both monostable and bistable cases, even for cases with strong excitation. In addition, the splitting method is efficient for higher-dimensional problem and has advantages of simple implementation, less storage of intermediate values and so on. Hence, in terms of the application scope, the proposed procedure is superior to the current mainstream methods for the random response evaluation of nonlinear vibration systems with adjustable stiffness.

Methods for the Real-World Evaluation of Fall Detection Technology: A Scoping Review

Falls in older adults present a major growing healthcare challenge and reliable detection of falls is crucial to minimise their consequences. The majority of development and testing has used laboratory simulations. As simulations do not cover the wide range of real-world scenarios performance is poor when retested using real-world data. There has been a move from the use of simulated falls towards the use of real-world data. This review aims to assess the current methods for real-world evaluation of fall detection systems, identify their limitations and propose improved robust methods of evaluation. Twenty-two articles met the inclusion criteria and were assessed with regard to the composition of the datasets, data processing methods and the measures of performance. Real-world tests of fall detection technology are inherently challenging and it is clear the field is in its infancy. Most studies used small datasets and studies differed on how to quantify the ability to avoid false alarms and how to identify non-falls, a concept which is virtually impossible to define and standardise. To increase robustness and make results comparable, larger standardised datasets are needed containing data from a range of participant groups. Measures that depend on the definition and identification of non-falls should be avoided. Sensitivity, precision and F-measure emerged as the most suitable robust measures for evaluating the real-world performance of fall detection systems.

Cross-polarized VNIR hyperspectral reflectance imaging for non-destructive quality evaluation of dried banana slices, drying process monitoring and control

Banana and its dried products are one of the most common foods consumed all over the world. Quality attributes of dried banana slices are typically evaluated using time-consuming and labor intensive methods which do not allow to evaluate each individual slice. Replacement of the current destructive methods for quality evaluation of dried banana slices by fast, non-destructive methods would provide large added value to the food industry. Therefore, the aim of this study was to monitor the moisture content, texture and color of banana slices during the drying process in a fast and non-destructive way. VNIR hyperspectral reflectance imaging in the 400–1000 nm range was selected for this purpose. Thanks to a cross-polarized configuration the effects of glare or specular reflection on the banana slice surfaces in the hyperspectral diffuse reflectance images was largely reduced. Reference quality attributes for all the slices (moisture content, texture and color (L*, a*, b* values) obtained using conventional destructive methods and colorimeter were predicted from their corresponding average reflectance spectra extracted from the hyperspectral images by means of partial least squares regression (PLSR). The PLSR calibration models were validated on samples which had not been used for model calibration. The results were very good for water content (R2P = 0.97, RMSEP = 0.05 kg water/kg DM), quite good for and b* value (R2P = 0.83, RMSEP = 1.95), and reasonable for texture (R2P = 0.66, RMSEP = 11.8 N), a* value (R2P = 0.53, RMSEP = 1.32) and L* value (R2P = 0.61, RMSEP = 5.92). Subsequently, these calibration models were used to predict those quality attributes at pixel level for the validation slices to visualize the spatial distribution of these quality parameters at different stages during the drying process. The obtained results clearly indicate the potential of cross-polarized hyperspectral reflectance imaging for non-destructive monitoring of the quality attributes of banana slices during drying.

Ecological niche modeling re-examined: A case study with the Darwin's fox.

Many previous studies have attempted to assess ecological niche modeling performance using receiver operating characteristic (ROC) approaches, even though diverse problems with this metric have been pointed out in the literature. We explored different evaluation metrics based on independent testing data using the Darwin's Fox (Lycalopex fulvipes) as a detailed case in point. Six ecological niche models (ENMs; generalized linear models, boosted regression trees, Maxent, GARP, multivariable kernel density estimation, and NicheA) were explored and tested using six evaluation metrics (partial ROC, Akaike information criterion, omission rate, cumulative binomial probability), including two novel metrics to quantify model extrapolation versus interpolation (E‐space index I) and extent of extrapolation versus Jaccard similarity (E‐space index II). Different ENMs showed diverse and mixed performance, depending on the evaluation metric used. Because ENMs performed differently according to the evaluation metric employed, model selection should be based on the data available, assumptions necessary, and the particular research question. The typical ROC AUC evaluation approach should be discontinued when only presence data are available, and evaluations in environmental dimensions should be adopted as part of the toolkit of ENM researchers. Our results suggest that selecting Maxent ENM based solely on previous reports of its performance is a questionable practice. Instead, model comparisons, including diverse algorithms and parameterizations, should be the sine qua non for every study using ecological niche modeling. ENM evaluations should be developed using metrics that assess desired model characteristics instead of single measurement of fit between model and data. The metrics proposed herein that assess model performance in environmental space (i.e., E‐space indices I and II) may complement current methods for ENM evaluation.

Forensic proteomics for the evaluation of the post-mortem decay in bones

Current methods for evaluation the of post-mortem interval (PMI) of skeletal remains suffer from poor accuracy due to the great number of variables that affect the diagenetic process and to the lack of specific guidelines to address this issue. During decomposition, proteins can undergo cumulative decay over the time, resulting in a decrease in the range and abundance of proteins present (i.e., the proteome) in different tissues as well as in an increase of post-translational modifications occurring in these proteins. In this study, we investigate the applicability of bone proteomic analyses to simulated forensic contexts, looking for specific biomarkers that may help the estimation of PMI, as well as evaluate a previously discovered marker for the estimation of biological age. We noticed a reduction of particular plasma and muscle proteins with increasing PMIs, as well as an increased deamidation of biglycan, a protein with a role in modulating bone growth and mineralization. We also corroborated our previous results regarding the use of fetuin-A as a potential biomarker for the estimation of age-at-death, demonstrating the applicability and the great potential that proteomics may have towards forensic sciences. SignificanceThe estimation of the post-mortem interval has a key role in forensic investigations, however nowadays it still suffers from poor reliability, especially when body tissues are heavily decomposed. Here we propose for the first time the application of bone proteomics to the estimation of the time elapsed since death and found several new potential biomarkers to address this, demonstrating the applicability of proteomic analyses to forensic sciences.