Missing Rate Research Articles

ContrAttNet: Contribution and attention approach to multivariate time-series data imputation

ABSTRACT The imputation of missing values in multivariate time-series data is a basic and popular data processing technology. Recently, some studies have exploited Recurrent Neural Networks (RNNs) and Generative Adversarial Networks (GANs) to impute/fill the missing values in multivariate time-series data. However, when faced with datasets with high missing rates, the imputation error of these methods increases dramatically. To this end, we propose a neural network model based on dynamic contribution and attention, denoted as ContrAttNet. ContrAttNet consists of three novel modules: feature attention module, iLSTM (imputation Long Short-Term Memory) module, and 1D-CNN (1-Dimensional Convolutional Neural Network) module. ContrAttNet exploits temporal information and spatial feature information to predict missing values, where iLSTM attenuates the memory of LSTM according to the characteristics of the missing values, to learn the contributions of different features. Moreover, the feature attention module introduces an attention mechanism based on contributions, to calculate supervised weights. Furthermore, under the influence of these supervised weights, 1D-CNN processes the time-series data by treating them as spatial features. Experimental results show that ContrAttNet outperforms other state-of-the-art models in the missing value imputation of multivariate time-series data, with average 6% MAPE and 9% MAE on the benchmark datasets.

Fast Incomplete Multi-View Clustering With View-Independent Anchors.

Multi-view clustering (MVC) methods aim to exploit consistent and complementary information among each view and achieve encouraging performance improvement than single-view counterparts. In practical applications, it is common to obtain instances with partially available information, raising researches of incomplete multi-view clustering (IMC) issues. Recently, several fast IMC methods have been proposed to process the large-scale partial data. Though with considerable acceleration, these methods seek view-shared anchors and ignore specific information among single views. To tackle the above issue, we propose a fast IMC with view-independent anchors (FIMVC-VIA) method in this article. Specifically, we learn individual anchors based on the diversity of distribution among each incomplete view and construct a unified anchor graph following the principle of consistent clustering structure. By constructing an anchor graph instead of pairwise full graph, the time and space complexities of our proposed FIMVC-VIA are proven to be linearly related to the number of samples, which can efficiently solve the large-scale task. The experiment performed on benchmarks with different missing rate illustrates the improvement in complexity and effectiveness of our method compared with other IMC methods. Our code is publicly available at https://github.com/Tracesource/ FIMVC-VIA.

Comparison of the application of new and old specification for occupational health surveillance of radiation workers and analysis of influencing factors

Objective: To investigate the impact of the implementation of GBZ 98-2020 "Health Requirements and Surveillance Specifications for Radiation Worker" on the results of occupational health examination for radiation workers. Methods: In April 2022, the subjects of the study were the radiation workers who underwent occupational health examination in Occupational Disease Prevention and Treatment Institute of Hefei. The radiation workers whose registration period was from May 1, 2021 to April 30, 2022 were the new standard group, and the occupational health surveillance standard was GBZ 98-2020 "Health Requirements and Surveillance Specifications for Radiation Worker". The radiationl workers registered from May 1, 2020 to April 30, 2021 were the old standard group, whose occupational health surveillance standards were GBZ 98- 2017 "Health Requirements for Radiation Workers" and GBZ 235-2011 "Specifications for Occupational Health Surveillance for Radiation Workers". To analyze whether there were differences between the two groups in the detection rate of missing items in the examination, re-examination, and the detection rate of occupational contraindications. The radiation workers whose occupational health examination results showed re-examination and/or occupational contraindications were judged to be in the unqualified group. Univariate and multivariate logistic regression analysis was used to find the factors affecting the determination of unqualified group. Results: The missing item detection rate of radiation workers in the new standard group was 3.04% (63/2074) , significantly higher than that in the old standard group (0.68%, 14/2054) (P<0.05) . The re-examination and occupational contraindications detection rates in the new standard group were 5.93% (123/2074) and 0.58% (12/2074) , respectively, which were significantly lower than those in the old standard group (13.83%, 284/2054) and 2.34% (48/2054) (P<0.05) . The missing item detection rate of males in the new standard group was 2.78% (40/1440) , which was significantly higher than that in the old standard group (0.72%, 11/1536) (P<0.05) . The re-examination and occupational contraindications detection rates of males in the new standard group were 3.61% (52/1440) and 0.21% (3/1440) , respectively, which were significantly lower than those in the old standard group (12.17%, 187/1536) and 2.08% (32/1536) (P<0.05) . The missing item detection rate of females in the new standard group was 3.63% (23/634) , which was significantly higher than that in the old standard group (0.58%, 3/518) (P<0.05) . The re-examination detection rate of females in the new standard group was 11.20% (71/634) , which was significantly lower than that of females in the old standard group (18.73%, 97/518) (P<0.05) . Univariate logistic regression analysis showed that gender, radiation classification, determination basis, occupational health examination category, and registration category were all influencing factors for the unqualified occupational health examination results of radiation workers (P<0.05) . Multivariate logistic regression analysis showed that the risk of being judged as unqualified based on the old standard was 2.466 times that of the new standard (95%CI: 1.975-3.080, P<0.05) , and the risk of being judged as unqualified for females was 1.869 times that of males (95%CI: 1.498-2.333, P<0.05) , the risk of being judged as unqualified for radiation workers during and after employment was 0.802 times that of pre-employment individuals (95%CI: 0.650-0.989, P<0.05) , and the risk of being judged as unqualified for re-examined individuals was 4.056 times that of initial examinees (95%CI: 3.161-5.203, P<0.05) . Conclusion: The results of occupational health examination of radiation workers are related to the determination basis, gender, occupational health examination category, and registration category. The implementation of GBZ 98-2020 "Health Requirements and Surveillance Specifications for Radiation Worker" may reduce the detection rate of unqualified personnel.

Enhancing the National Cancer Database content using natural language processing and electronic health record data

BackgroundThe prevalence of missing data in the National Cancer Database (NCDB) has marked implications on clinical care and research. The objective of this study was to enhance the NCDB by decreasing rates of missingness and adding new variables using automated statistical methodology. MethodsOne health system’s NCDB data from 2011–2021 was linked to electronic health record (EHR). Variables with frequent missingness and new clinically significant variables not yet included in the NCDB including patient Eastern Cooperative Oncology Group (ECOG) score, specific chemotherapy regimen, American Society of Anesthesiologists Physical Status Classification (ASA class), and discrete surgical procedure were identified in structured and unstructured EHR data. After automated incorporation of structured data from EHR, a natural language processing tool incorporating rule-based algorithms was designed to further extract variables from unstructured notes. Rates of missingness were compared between the original NCDB and the enhanced dataset, and example multivariable models were run to assess for altered model performance with reduced missingness and the addition of new clinically significant variables (chemotherapy regimen). ResultsA total of 6050 patients with NCDB records were linked to their EHR data. Prior to enhancement, rates of missingness for key variables ranged from 2.0% to 5.3%. Following dataset enhancement, missingness was significantly reduced, with relative missingness being reduced between 31.9% to 68.0%. Of the new variables added, 1367 (22.6%) of 6050 patients gained ECOG score, and 1099 (57.8%) of 1901 who received chemotherapy gained their chemotherapy regimen. Of 2989 who underwent surgery, 979 (32.8%) gained their procedure name and 621 (20.8%) gained ASA class. Comparison of the multivariable models demonstrated significant differences between the original NCDB and the enhanced dataset. Specifically, when replacing the binary predictor for chemotherapy in the original NCDB data with discrete regimens, the effect of ethnicity diminished, and the effect of radiation became significant. DiscussionWe applied statistical methodology to reduce rates of missingness in existing variables and add new variables to enrich the NCDB. While further refinement is needed to decrease missingness in new variables, this automated methodology can replace or augment manual chart review and improve the ability of to use the NCDB to study unanswered questions leading to clinical advancements in oncology.

A new approach to predict pump transient phenomena in Molten salt reactor Experiment (MSRE) by missing data identification and regeneration

Molten Salt Reactor Experiment (MSRE) is being extensively used for validating the computational tools for Molten Salt Reactors (MSRs). The pump transient tests were performed during the operation of MSRE to obtain the reactivity response to flow perturbation. The transient flow rate during this set of tests is required as input to predict the reactivity response. Since the primary loop of MSRE was not equipped with a flow rate meter, the transient flow rate, which is the crucial factor for the reactivity change in MSRE, is missing. Achieving an accurate simulation of the reactivity response to the MSRE pump transient tests necessitates a precise estimation of the transient flow rate.This paper endeavors to reconstruct the missing flow rate with the aim of offering a valuable input for simulating reactivity responses in the pump transient test series. In order to obtain an accurate transient flow rate, we employed a centrifugal pump transient model based on the affinity laws and solved the model for the MSRE secondary pump to validate the underlying assumptions of the model. In addition, we constructed the homologous pump head curves based on the water test data for the same purpose. The affinity law approximation is proved to be adequate in predicting the MSRE pump startup transient with the root-mean-square error (RMSE) in the normalized flow rate to be 0.03. On the other hand, for the MSRE pump coastdown transient, the preliminary results indicate that neither the affinity law approximation nor the homologous head curve is sufficient to provide acceptable predictions. To overcome this noticeable modeling deficit, we propose an innovative approach based on a straightforward data mining technique to regenerate the MSRE pump coastdown homologous relations using the measured data of the secondary pump. These relations are transferred to the primary pump and used to simultaneously solve for the impeller speed and the flow rate of the pump in the primary pump coastdown. With the new approach, the estimated RMSE in the normalized pump speed for the primary pump coastdown is reduced 0.0052. This excellent agreement validates the accurate calculations for the flow rate during the pump coastdown transient. We also performed an uncertainty analysis to quantify the confidence interval of the predicted quantities, which further justifies the robustness of the proposed approach.

Detection and recognition of Chinese porcelain inlay images of traditional Lingnan architectural decoration based on YOLOv4 technology

With the rapid development of machine learning technology, it has become possible to automatically identify cultural heritage elements in traditional buildings. This research aimed to develop a machine learning model based on the YOLOv4 architecture to identify the traditional Chinese porcelain inlay pattern in the Lingnan region. The researchers collected and annotated a large quantity of Lingnan Chinese porcelain inlay image data and then used these data to train the studied model. The research results show that (1) the model in this study was specifically adjusted to effectively identify a variety of Chinese porcelain inlay pattern types, including traditional patterns such as plum blossoms and camellias. (2) In the 116th epoch, the model showed excellent generalization ability, and the verification loss reached the lowest value of 0.88. The lowest training loss in the 195th epoch was 0.99, indicating that the model reached an optimal balance point for both recognition accuracy and processing speed. (3) By comparing different models for detecting Chinese porcelain inlay images across 581 pictures, our YOLOv4 model demonstrated greater accuracy in most classification tasks than did the YOLOv8 model, especially in the classification of chrysanthemums, where it achieved an accuracy rate of 87.5%, significantly outperforming YOLOv8 by 58.82%. However, the study also revealed that under certain conditions, such as detecting apples and pears in low-light environments, YOLOv8 showed a lower missing data rate, highlighting the limitations of our model in dealing with complex detection conditions.

The degree of importance for partially labeled incomplete real-valued data with application to semi-supervised attribute reduction

Incomplete real-valued data often misses some labels due to the high cost of labeling data. This paper investigates for partially labeled incomplete real-valued data and considers its application in semi-supervised attribute reduction. There are two decision information systems (DISs) in a partially labeled incomplete real-valued data DIS (p-IRVDIS): a labeled incomplete real-valued data DIS (l-IRVDIS) and a unlabeled incomplete real-valued data DIS (u-IRVDIS). The degree of importance on an attribute subset in a p-IRVDIS are defined using an indistinguishable relation and conditional information entropy. It is the weighted sum of l-IRVDIS and u-IRVDIS using the missing rate of label to measure p-IRVDIS uncertainty. Based on the degree of importance, an adaptive semi-supervised attribute reduction algorithm in a p-IRVDIS is proposed. This algorithm can automatically adapt to various missing rates of label. The experimental results on 8 datasets reveal that the proposed algorithm performs statistically better than some state-of-the-art algorithms.

Attribute selection for partially labeled data based on uncertainty measure

Due to the relatively high cost of labeling data, only a fraction of the available data is typically labeled in machine learning. Some existing research handled attribute selection for partially labeled data by using the importance of an attribute subset or uncertainty measure (UM). Nevertheless, it overlooked the missing rate of labels or the choice of the UM with optimal performance. This study uses discernibility relation and the missing rate of labels to UM for partially labeled data and applies it to attribute selection. To begin with, a decision information system for partially labeled data (pl-DIS) can be used to induce two equivalent decision information systems (DISs): a DIS is constructed for labeled data (l-DIS), and separately, another DIS is constructed for unlabeled data (ul-DIS). Subsequently, a discernibility relation and the percentage of missing labels are introduced. Afterwards, four importance of attribute subset are identified by taking into account the discernibility relation and the missing rate of labels. The sum of their importance, which is determined by the label missing rates of two DISs, is calculated by weighting each of them and adding them together. These four importance may be seen as four UMs. In addition, numerical simulations and statistical analyses are carried out to showcase the effectiveness of four UMs. In the end, as its application for UM, the UM with optimal performance is used to attribute selection for partially labeled data and the corresponding algorithm is proposed. The experimental outcomes demonstrate the excellence of the proposed algorithm.

Mitigating Missing Rate and Early Cyberattack Discrimination Using Optimal Statistical Approach with Machine Learning Techniques in a Smart Grid

In the Industry 4.0 era of smart grids, the real-world problem of blackouts and cascading failures due to cyberattacks is a significant concern and highly challenging because the existing Intrusion Detection System (IDS) falls behind in handling missing rates, response times, and detection accuracy. Addressing this problem with an early attack detection mechanism with a reduced missing rate and decreased response time is critical. The development of an Intelligent IDS is vital to the mission-critical infrastructure of a smart grid to prevent physical sabotage and processing downtime. This paper aims to develop a robust Anomaly-based IDS using a statistical approach with a machine learning classifier to discriminate cyberattacks from natural faults and man-made events to avoid blackouts and cascading failures. The novel mechanism of a statistical approach with a machine learning (SAML) classifier based on Neighborhood Component Analysis, ExtraTrees, and AdaBoost for feature extraction, bagging, and boosting, respectively, is proposed with optimal hyperparameter tuning for the early discrimination of cyberattacks from natural faults and man-made events. The proposed model is tested using the publicly available Industrial Control Systems Cyber Attack Power System (Triple Class) dataset with a three-bus/two-line transmission system from Mississippi State University and Oak Ridge National Laboratory. Furthermore, the proposed model is evaluated for scalability and generalization using the publicly accessible IEEE 14-bus and 57-bus system datasets of False Data Injection (FDI) attacks. The test results achieved higher detection accuracy, lower missing rates, decreased false alarm rates, and reduced response time compared to the existing approaches.

Graph-based phishing detection: URLGBM model driven by machine learning

Phishing attacks are a form of social engineering that involves the transmission of deceptive communications imitating a trustworthy source to deceive users into revealing confidential information. Antiphishing systems have made significant strides in recent years, allowing internet users to secure confidential and private information against such attacks. In this study, we propose a URL graph representation based on a random walk algorithm, specifically PageRank, for weighting URL tokens. To create the graph, an imaginary walker visits the URL tokens one at a time and assigns a value to each token based on the probability of encountering the target URL during the walk. We studied different random walk (rw) variations and their effects on the URL string. The BM25 algorithm was employed to produce a sparse matrix for the classification task from the token scores obtained. Experiments conducted with logistic regression revealed that the proposed model achieved an accuracy of 98.98%, a false alarm rate of 1.72%, and a missing alarm rate of 0.302%. The model also attained a 97.17% accuracy on a benchmark dataset.

Comparing FY-2F/CTA products to ground-based manual total cloud cover observations in Xinjiang under complex underlying surfaces and different weather conditions

Abstract. Clouds are an important parameter of artificial water augmentation, which is of substantial significance to judge the precipitation capacity. Xinjiang is an arid region in northwestern China, where weather stations are sparsely distributed, the types of underlying surface are complex, and the climate between the southern and northern region varies greatly. However, the retrieval of the total cloud cover (TCC) from satellite in arid areas is a challenging task. Based on the TCC data observed by ground observation stations (GOSs) from June 2015 to May 2016 considering the complex underlying surfaces and different weather conditions, the precision, consistency, and error between the cloud total amount from the FengYun-2F stationary satellite (FY-2F/CTA) and manually observed TCC are compared and evaluated in the Xinjiang region. The findings of this study are as follows: (1) the precision rate (PR) of FY-2F/CTA in the Xinjiang region is 75.6 %, which gradually decreases from north to south, demonstrating a high false rate (FR) and a low missing rate (MR). The consistency rate (CR) is 51.5 %, with little difference among three subregions of Xinjiang, all showing a high weak rate (WR) and low strong rate (SR), which means that the TCC values inverted from FY-2 satellite data are generally lower than those observed by GOSs, especially in southern Xinjiang. The bias is −20 %, and all the error indexes (EIs) including bias, MAE, and RMSE increase from central to the north and south of Xinjiang; that means the EIs are the lowest in Tianshan and the highest in southern Xinjiang. FY-2F/CTA exhibits higher PR and CR in the underlying surface of vegetation compared to non-vegetation; that is to say that FY-2F/CTA performs best in the underlying surfaces of forest and plowland, while their performance is relatively poorer in the underlying surface of snow and ice. (2) With rising temperature the PR and CR of FY-2F/CTA increase, while the EIs decrease. Under various temperature conditions, FY-2F/CTA has always exhibited high MR, low FR (on the contrary in January), high WR, and low SR. From low elevation to high elevation, the PR and CR of FY-2F/CTA decrease, but the PR increases significantly when the altitude is higher than 2000 m. (3) Dust reduces the CR of FY-2F/CTA and increases their WR and MR but has a relatively minor impact on the identification of cloud and non-cloud. (4) Under different cloud cover levels, the PR and EIs of FY-2F/CTA are directly proportional to the amount of TCC, while the CR is inversely proportional to it: that is, the CR is higher and the PR and EIs are lower under clear-sky and partly cloudy conditions, and the CR is lower and the PR and EIs are higher under cloudy and overcast conditions. This study assessed the FY-2F/CTA under various conditions in arid areas of Xinjiang, including complex underlying surface, various temperature and altitude, dust effects, and different cloud cover levels. Thus, the research finding could serve as a valuable reference for satellite-based retrieval and applications related to TCC in arid regions.

A survey of practice in the anesthetic management of adolescent idiopathic scoliosis spine fusion by the North American Pediatric Spine Anesthesiologists Collaborative.

Adolescent Idiopathic Scoliosis (AIS) affects 2%-4% of the general pediatric population. While surgical correction remains one of the most common orthopedic procedures performed in pediatrics, limited consensus exists on the perioperative anesthetic management. To examine the current state of anesthetic management of typical AIS spine fusions at institutions which have a dedicated pediatric orthopedic spine surgeon. A web-based survey was sent to all members of the North American Pediatric Spine Anesthesiologists (NAPSA) Collaborative. This group included 34 anesthesiologists at 19 different institutions, each of whom has a Harms Study Group surgeon performing spine fusions at their hospital. Thirty-one of 34 (91.2%) anesthesiologists completed the survey, with a missing response rate from 0% to 16.1% depending on the question. Most anesthesia practices (77.4%; 95% confidence interval [CI], 67.7-93.4) do not have patients come for a preoperative visit prior to the day of surgery. Intravenous induction was the preferred method (74.2%; 95% CI 61.3-89.9), with the majority utilizing two peripheral IVs (93.5%; 95% CI 90.3-100) and an arterial line (100%; 95% CI 88.8-100). Paralytic administration for intubation and/or exposure was divided (51.6% rocuronium/vecuronium, 45.2% no paralytic, and 3.2% succinylcholine) amongst respondents. While tranexamic acid was consistently utilized for reducing blood loss, dosing regimens varied. When faced with neuromonitoring signal issues, 67.7% employ a formal protocol. Most anesthesiologists (93.5%; 95% CI 78.6-99.2) extubate immediately postoperatively with patients admitted to an inpatient floor bed (77.4%; 95% CI 67.7-93.3). Most anesthesiologists (87.1%; 95% CI 80.6-99.9) report the use of some form of an anesthesia-based protocol for AIS fusions, but our survey results show there is considerable variation in all aspects of perioperative care. Areas of agreement on management comprise the typical vascular access required, utilization of tranexamic acid, immediate extubation, and disposition to a floor bed. By recognizing the diversity of anesthetic care, we can develop areas of research and improve the perioperative management of AIS.

Context-aware deep learning enables high-efficacy localization of high concentration microbubbles for super-resolution ultrasound localization microscopy

Ultrasound localization microscopy (ULM) enables deep tissue microvascular imaging by localizing and tracking intravenously injected microbubbles circulating in the bloodstream. However, conventional localization techniques require spatially isolated microbubbles, resulting in prolonged imaging time to obtain detailed microvascular maps. Here, we introduce LOcalization with Context Awareness (LOCA)-ULM, a deep learning-based microbubble simulation and localization pipeline designed to enhance localization performance in high microbubble concentrations. In silico, LOCA-ULM enhanced microbubble detection accuracy to 97.8% and reduced the missing rate to 23.8%, outperforming conventional and deep learning-based localization methods up to 17.4% in accuracy and 37.6% in missing rate reduction. In in vivo rat brain imaging, LOCA-ULM revealed dense cerebrovascular networks and spatially adjacent microvessels undetected by conventional ULM. We further demonstrate the superior localization performance of LOCA-ULM in functional ULM (fULM) where LOCA-ULM significantly increased the functional imaging sensitivity of fULM to hemodynamic responses invoked by whisker stimulations in the rat brain.

EDA: Evolving and Distinct Anchors for Multimodal Motion Prediction

Motion prediction is a crucial task in autonomous driving, and one of its major challenges lands in the multimodality of future behaviors. Many successful works have utilized mixture models which require identification of positive mixture components, and correspondingly fall into two main lines: prediction-based and anchor-based matching. The prediction clustering phenomenon in prediction-based matching makes it difficult to pick representative trajectories for downstream tasks, while the anchor-based matching suffers from a limited regression capability. In this paper, we introduce a novel paradigm, named Evolving and Distinct Anchors (EDA), to define the positive and negative components for multimodal motion prediction based on mixture models. We enable anchors to evolve and redistribute themselves under specific scenes for an enlarged regression capacity. Furthermore, we select distinct anchors before matching them with the ground truth, which results in impressive scoring performance. Our approach enhances all metrics compared to the baseline MTR, particularly with a notable relative reduction of 13.5% in Miss Rate, resulting in state-of-the-art performance on the Waymo Open Motion Dataset. Appendix and code are available at https://github.com/Longzhong-Lin/EDA.

RedCore: Relative Advantage Aware Cross-Modal Representation Learning for Missing Modalities with Imbalanced Missing Rates

Multimodal learning is susceptible to modality missing, which poses a major obstacle for its practical applications and, thus, invigorates increasing research interest. In this paper, we investigate two challenging problems: 1) when modality missing exists in the training data, how to exploit the incomplete samples while guaranteeing that they are properly supervised? 2) when the missing rates of different modalities vary, causing or exacerbating the imbalance among modalities, how to address the imbalance and ensure all modalities are well-trained. To tackle these two challenges, we first introduce the variational information bottleneck (VIB) method for the cross-modal representation learning of missing modalities, which capitalizes on the available modalities and the labels as supervision. Then, accounting for the imbalanced missing rates, we define relative advantage to quantify the advantage of each modality over others. Accordingly, a bi-level optimization problem is formulated to adaptively regulate the supervision of all modalities during training. As a whole, the proposed approach features Relative advantage aware Cross-modal representation learning (abbreviated as RedCore) for missing modalities with imbalanced missing rates. Extensive empirical results demonstrate that RedCore outperforms competing models in that it exhibits superior robustness against either large or imbalanced missing rates. The code is available at: https://github.com/sunjunaimer/RedCore.

Imputation of data Missing Not at Random: Artificial generation and benchmark analysis

Experimental assessment of different missing data imputation methods often compute error rates between the original values and the estimated ones. This experimental setup relies on complete datasets that are injected with missing values. The injection process is straightforward for the Missing Completely At Random and Missing At Random mechanisms; however, the Missing Not At Random mechanism poses a major challenge, since the available artificial generation strategies are limited. Furthermore, the studies focused on this latter mechanism tend to disregard a comprehensive baseline of state-of-the-art imputation methods. In this work, both challenges are addressed: four new Missing Not At Random generation strategies are introduced and a benchmark study is conducted to compare six imputation methods in an experimental setup that covers 10 datasets and five missingness levels (10% to 80%). The overall findings are that, for most missing rates and datasets, the best imputation method to deal with Missing Not At Random values is the Multiple Imputation by Chained Equations, whereas for higher missingness rates autoencoders show promising results.

A transferred spatio-temporal deep model based on multi-LSTM auto-encoder for air pollution time series missing value imputation

Air pollution is one of the most severe problems facing the world. Research on air quality prediction and analysis of influencing factors also continues to grow. When conducting this research, valid, authentic, and high-quality air pollution data are necessary to obtain reasonable results. However, Missing values are unavoidable in multivariate time series due to multiple causes, such as sensor and communication failure. Most previous algorithms on missing data cannot effectively pay attention to air pollution’s temporal and spatial mechanism, handle multiple missing patterns, or deal with high missing rates sequences. This paper proposes a new deep spatiotemporal imputation methodology to address this problem effectively, namely transferred Multiple LSTM based deep auto-encoder (TMLSTM-AE). Our idea is intuitive: train an auto-encoder to estimate the missing values. It uses spatial and time series information to fill in single missing, multiple missing, block missing, and long-interval consecutive missing in air quality data. To verify the effectiveness and priority of the proposed model, we conducted a case study in a city in Shaanxi, China. Long-interval consecutive missing and different missing patterns PM2.5 data are filled. The results indicate that the model proposed in this paper performs well and outperforms existing models for different missing patterns and long-interval consecutive missing.

EMDet: An entropy blending and multi-link parallel feature enhancement detection model for gas pipeline weak leakage detection

To improve the detection accuracy of gas pipeline leakage detection (PLD) especially when the leakage is weak, we propose a hybrid deep learning model, named EMDet in this paper, which incorporates two proposed algorithms called entropy blending (EB) and multi-link parallel feature enhancement (MPFE). Firstly, the EB algorithm is used to determine the optimal number of decomposition layer Lo and the optimal intrinsic mode function Io for variational mode decomposition (VMD), elevating the leakage-related information in Io. Then, three coarse-grained scales have been applied to extract effective and robust features, where the coarse-grained scales are calculated based on the sizes required by the MPFE algorithm. Moreover, the MPFE algorithm is proposed to enhance the representation of features, thereby improving the detection accuracy of PLD. After that, the performance of the proposed EMDet has been evaluated on the negative pressure wave (NPW) data collected from realistic urban environments. Compared with the state-of-the-art PLD models, the performance of EMDet not only reaches the total accuracy of 98.67%, the F1-score of 98.65%, and the fault detection rate of 98.68%, but also reduces the false alarm rate to 0.66% and the missing alarm rate to 1.32%. Finally, the robustness results demonstrate the superior performance and broad application prospects of EMDet for PLD.

What importance do donors and recipients attribute to the nuclear DNA-related genetic heritage of oocyte donation?

How do oocyte donors and recipients perceive the genetic link related to the transfer of nuclear DNA between donors and offspring? Whether they are donors or recipients, individuals attach great importance to the transmission of their genetic heritage, since 94.5% would opt for the pronuclear transfer method to preserve this genetic link in the context of oocyte donation. Since 1983, the use of oocyte donation has increased worldwide. Performed in France since the late 1980s and initially offered to women with premature ovarian insufficiency, its indications have progressively expanded and now it is proposed in many indications to prevent the transmission of genetically inherited diseases. This has resulted in an increase in the waiting time for access to oocyte donation due to the difficulty in recruiting oocyte donors in French ART centres. Several articles have discussed how to fairly distribute donor oocytes to couples, but few have interviewed women in the general population to record their feelings about oocyte donation, as either the donor or recipient and the importance given to the genetic link between the oocyte donors and the children born. Mitochondrial replacement therapy (MRT) is a technique originally developed for women at risk of transmitting a mitochondrial DNA mutation. Recently, MRT has been considered for embryo arrest and oocyte rejuvenation as it could help females to reproduce with their own genetic material through the transfer of their oocyte nucleus into a healthy donor oocyte cytoplasm. We conducted an opinion survey from January 2021 to December 2021, during which 1956 women completed the questionnaire. Thirteen participants were excluded from the analysis due to incomplete responses to all the questions. Consequently, 1943 women were included in the study. We specifically developed a questionnaire for this study, which was created and distributed using the Drag'n Survey® software. The questionnaire consisted of 21 items presented alongside a video created with whiteboard animation software. The aim was to analyse whether certain factors, such as age, education level, marital status, number of children, use of ART for pregnancy, video viewing, and knowledge about oocyte donation, were associated with feelings towards oocyte donation, by using a univariate conditional logistic regression model. This statistical method was also used to assess whether women would be more inclined to consider oocyte donation with the pronuclear transfer technique rather than the whole oocyte donation. All parameters found to be statistically significant in the univariate analysis were subsequently tested in a multivariate model using logistic regression. Most women were concerned about the biological genetic contribution of the donated oocyte (94.8%). The most common reason for a women's reluctance to donate their oocytes was their unwillingness to pass on their genetic material (33.3%). Nearly 70% of women who were initially hesitant to donate their oocytes indicated that they would reconsider their decision if the oocyte donation was conducted using donated cytoplasm and the pronuclear transfer technique. Concomitantly, >75% of the respondents mentioned that it would be easier to receive a cytoplasm donation. The largest proportion of the population surveyed (94.5%) expressed their support for its legalization. In this study, a substantial portion of the responses came from individuals with medical or paramedical backgrounds, potentially introducing a recruitment bias among potential donors. The rate of missing responses to the question regarding the desire to become an oocyte donor was 13.6%, while the question about becoming an oocyte cytoplasm donor had a missing response rate of 23%. These missing responses may introduce a bias in the interpretation of the data. This study was the first to demonstrate that, for the French population studied, the combination of oocyte cytoplasm donation with pronuclear transfer could offer a promising approach to enhance the acceptance of oocyte donation for both the donor and the recipient. No external funding was used for this study. The authors have no conflicts of interest. N/A.

Multiple Imputation When Variables Exceed Observations: An Overview of Challenges and Solutions

Missing data are a prevalent problem in psychological research that can reduce statistical power and bias parameter estimates. These problems can be mostly resolved with multiple imputation, a modern missing data treatment that is increasingly used. Imputation, however, requires the number of variables to be smaller than the number of observations (i.e., non-missing values), and this number is often exceeded due to, e.g., large assessments, high missing data rates, the inclusion of variables predictive of missing values, and the inclusion of non-linear transformations. Even when the ratio of variables to observations meets the minimum requirement, convergence failure can occur in large, complex models. Specialized techniques have been developed to overcome the challenges related to having too many variables in an imputation model, but they are still relatively unknown by researchers in psychology. Accordingly, this paper presents an overview of four imputation techniques that can be used to reduce the number of predictors in an imputation model: item aggregation with scales and parcels, passive imputation, principal component analysis (PcAux) and two-fold fully conditional specification. The purpose, advantages, limitations, and applications of each method are discussed, along with recommendations and illustrative examples, with the aims of (1) understanding different imputation methods and (2) identifying methods that could be useful for one’s imputation problem.