High-quality Models Research Articles

Learn by Oneself: Exploiting Weight-Sharing Potential in Knowledge Distillation Guided Ensemble Network

Recent CNNs (convolutional neural networks) have become more and more compact. The elegant structure design highly improves the performance of CNNs. With the development of knowledge distillation technique, the performance of CNNs gets further improved. However, existing knowledge distillation guided methods either rely on offline pretrained high-quality large teacher models or online heavy training burden. To solve the above problems, we propose a feature-sharing and weight-sharing based ensemble network (training framework) guided by knowledge distillation (EKD-FWSNet) to make baseline models stronger in terms of representation ability with less training computation and memory cost involved. Specifically, motivated by getting rid of the dependence of offline pretrained teacher model, we design an end-to-end online training scheme to optimize EKD-FWSNet. Motivated by decreasing the online training burden, we only introduce one auxiliary classmate branch to construct multiple forward branches, which will then be integrated as ensemble teacher to guide baseline model. Compared to previous online ensemble training frameworks, EKD-FWSNet can provide diverse output predictions without relying on increasing auxiliary classmate branches. Motivated by maximizing the optimization power of EKD-FWSNet, we exploit the representation potential of weight-sharing blocks and design efficient knowledge distillation mechanism in EKD-FWSNet. Extensive comparison experiments and visualization analysis on benchmark datasets (CIFAR-10/100, tiny-ImageNet, CUB-200 and ImageNet) show that self-learned EKD-FWSNet can boost the performance of baseline models by large margin, which has obvious superiority compared to previous related methods. Extensive analysis also proves the interpretability of EKD-FWSNet. Our code is available at https://github.com/cv516Buaa/EKD-FWSNet.

Parameter Calibration of Wind Farm With Error Tracing Technique and Correlated Parameter Identification

With the increasing penetrations of wind power in power systems, periodical calibration of the large-scale Wind Farm (WF) model is crucial to maintaining high-quality model performance for stability investigations. However, the existing calibration methods mainly focus on correcting synchronous generators, not the more dynamical WF model. Challenges in tracing errors of inaccurate models and identifying the correlated parameters would further restrain the calibrating performance. To address such limitations, a novel parameter calibration approach for the WF model is developed explicitly in this article. With the development of a trajectory sensitivity based dominant parameters selection strategy, the parameters which significantly impact the model responses can be precisely located. Then, for discriminating the ill-conditioned parameters (deviating from their actual values), an improved error tracing method incorporating Hybrid Dynamic Simulation (HDS) and Hilbert-Huang Transform (HHT) is proposed. Lastly, to enhance the model performance, the ill-conditioned parameters are iteratively tuned by a well-designed multistage identification approach, with the consideration of parameter correlations. Additionally, theoretical proof of the error tracing technique is provided to guarantee the proposed method's validity mathematically. With realistic WF and a modified IEEE 39-bus system, the proposed method demonstrated its excellent performance in tracing and tuning inaccurate parameters.

Cost-Effective Full-Color 3D Dental Imaging Based on Close-Range Photogrammetry.

Dental imaging plays a crucial role in clinical dental practice. Conventional 2D dental imaging serves general-purpose tasks, such as patient documentation, while high-precision 3D dental scanning is tailored for specialized procedures, such as orthodontics and implant surgeries. In this study, we aimed to develop a cost-effective 3D imaging technique that could bridge the gap between conventional dental photography and high-precision 3D dental scanning, with the goal of improving patient dental care. We developed a 3D imaging technique based on close-range photogrammetry and termed it close-range photogrammetry-based dental imaging (CPDI). We evaluated this technique on both in vitro dental models and in vivo teeth. For dental models, we conducted a parametric study to examine the effects of the depth of field and specular reflection on reconstruction quality. We showed that the optimal results were achieved with an f/5.6 lens and without a circular polarizer for reflection suppression. This configuration generated 3D scans with 57.7 ± 3.2% and 82.4 ± 2.7% of reconstructed points falling within ±0.1 mm and ±0.2 mm error margins, respectively. With such accuracy, these 3D dental models can faithfully represent dental morphology and features. During in vivo imaging, we were able to reconstruct high-quality 3D models of the anterior arch, further demonstrating its clinical relevance. The reconstructed models carry both 3D shapes and detail full-color surface textures, which positions CPDI as a versatile imaging tool in different areas of clinical dental care.

Genome-scale metabolic modeling of the human gut bacterium Bacteroides fragilis strain 638R.

Bacteroides fragilis is a universal member of the dominant commensal gut phylum Bacteroidetes. Its fermentation products and abundance have been linked to obesity, inflammatory bowel disease, and other disorders through its effects on host metabolic regulation and the immune system. As of yet, there has been no curated systems-level characterization of B. fragilis' metabolism that provides a comprehensive analysis of the link between human diet and B. fragilis' metabolic products. To address this, we developed a genome-scale metabolic model of B. fragilis strain 638R. The model iMN674 contains 1,634 reactions, 1,362 metabolites, three compartments, and reflects the strain's ability to utilize 142 metabolites. Predictions made with this model include its growth rate and efficiency on these substrates, the amounts of each fermentation product it produces under different conditions, and gene essentiality for each biomass component. The model highlights and resolves gaps in knowledge of B. fragilis' carbohydrate metabolism and its corresponding transport proteins. This high quality model provides the basis for rational prediction of B. fragilis' metabolic interactions with its environment and its host.

TT3D: Leveraging precomputed protein 3D sequence models to predict protein-protein interactions.

High-quality computational structural models are now precomputed and available for nearly every protein in UniProt. However, the best way to leverage these models to predict which pairs of proteins interact in a high-throughput manner is not immediately clear. The recent Foldseek method of van Kempen et al. encodes the structural information of distances and angles along the protein backbone into a linear string of the same length as the protein string, using tokens from a 21-letter discretized structural alphabet (3Di). We show that using both the amino acid sequence and the 3Di sequence generated by Foldseek as inputs to our recent deep-learning method, Topsy-Turvy, substantially improves the performance of predicting protein-protein interactions cross-species. Thus TT3D (Topsy-Turvy 3D) presents a way to reuse all the computational effort going into producing high-quality structural models from sequence, while being sufficiently lightweight so that high-quality binary protein-protein interaction predictions across all protein pairs can be made genome-wide. TT3D is available at https://github.com/samsledje/D-SCRIPT. An archived version of the code at time of submission can be found at https://zenodo.org/records/10037674.

Redefining hybrid flow shop group scheduling: Unveiling a novel hybrid modeling paradigm and assessing 48 MILP and CP models

A hybrid flow shop group scheduling problem (HFGSP) involves two distinct sub-problems, the arrangement of groups and the configuration of jobs within each group. Constructing its mathematical model based on the classification of these sub-problems can help better understand the problem's characteristics. However, the existing literature fails to establish MILP models of HFGSP based on the interrelation between each subproblem and the respective process constraints satisfied by each subproblem. To address this gap, our study first categorizes all the constraints in the HFGSP according to several factors: group arrangement, job arrangement within the group, process requirements between groups, process requirements between jobs from the same group, process requirements between adjacent stages, and the relationship between the decision variables of completion time and the objective constraint. Following this classification, we construct 48 available MILP models of HFGSP. Additionally, we also propose an efficient constraint programming (CP) model of HFGSP. Numerous experiments are conducted to verify the correctness and performance of all 48 MILP models across different test instances, analyze the complexities of all models, and excavate their intrinsic characteristics. Through our evaluation for the 48 models, we observe that models 24 and 48 exhibit superior performance. This highlights the effectiveness of the hybrid modeling approach, which synergistically combines sequence-based modeling for groups and position-based adjacent modeling for jobs within each group. The utilization of this hybrid modeling approach enables the construction of high-quality MILP models for addressing the HFGSP problem. Our intention is to bridge the gap between MILP modeling of HFGSP and problem-specific algorithms, thereby providing a comprehensive understanding of the problem and offering potential solutions.

Federated Latent Dirichlet Allocation for User Preference Mining

In the field of Web services computing, a recent demand trend is to mine user preferences based on user requirements when creating Web service compositions, in order to meet comprehensive and ever evolving user needs. Machine learning methods such as the latent Dirichlet allocation (LDA) have been applied for user preference mining. However, training a high-quality LDA model typically requires large amounts of data. With the prevalence of government regulations and laws and the enhancement of people’s awareness of privacy protection, the traditional way of collecting user data on a central server is no longer applicable. Therefore, it is necessary to design a privacy-preserving method to train an LDA model without massive collecting or leaking data. In this paper, we present novel federated LDA techniques to learn user preferences in the Web service ecosystem. On the basis of a user-level distributed LDA algorithm, we establish two federated LDA models in charge of two-layer training scenarios: a centralized synchronous federated LDA (CSFed-LDA) for synchronous scenarios and a decentralized asynchronous federated LDA (DAFed-LDA) for asynchronous ones. In the former CSFed-LDA model, an importance-based partially homomorphic encryption (IPHE) technique is developed to protect privacy in an efficient manner. In the latter DAFed-LDA model, blockchain technology is incorporated and a multi-channel-based authority control scheme (MCACS) is designed to enhance data security. Extensive experiments over a real-world dataset ProgrammableWeb.com have demonstrated the model performance, security assurance and training speed of our approach.

Privacy and evolutionary cooperation in neural-network-based game theory

How cooperation evolves is one of the fundamental research problems of multi-agent systems. With a deeper understanding of the forces that promote cooperation, we could increase the proportion of cooperative agents. However, existing methods for cultivating cooperation have two common limitations. Firstly, most do not take the privacy for the agents into account. Privacy preservation is an essential to the systems with multiple agents, because, without privacy preservation, an adversarial agent can use the private information of other agents to maximize its own payoff. Beyond the obvious black hat implication, this is also detrimental because maximizing one’s own payoff typically implies minimizing the payoffs of others, which tends to reduce the number of agents willing to cooperate. The second limitation is that most existing methods have a limited scope for generalization. Their performance is usually highly dependent on specific circumstances, e.g., the system topology, the initial proportion of cooperative agents, etc. To overcome these two drawbacks, we propose a novel method that combines differential privacy to protect an agent’s private information from adversaries with a neural network architecture to optimize the decision making power of agents in a wider range of situations. Through this joint implementation, each agent’s privacy is guaranteed, yet agents are still encouraged to cooperate even when adversaries are present. One notable application example is federated learning (FL) systems. In FL, clients can be incentivized by our method to actively cooperate with the central server by contributing high-quality model updates, while necessitating the utmost assurance of their data privacy protection.

Genome-scale metabolic models consistently predict in vitro characteristics of Corynebacterium striatum.

Introduction: Genome-scale metabolic models (GEMs) are organism-specific knowledge bases which can be used to unravel pathogenicity or improve production of specific metabolites in biotechnology applications. However, the validity of predictions for bacterial proliferation in in vitro settings is hardly investigated. Methods: The present work combines in silico and in vitro approaches to create and curate strain-specific genome-scale metabolic models of Corynebacterium striatum. Results: We introduce five newly created strain-specific genome-scale metabolic models (GEMs) of high quality, satisfying all contemporary standards and requirements. All these models have been benchmarked using the community standard test suite Metabolic Model Testing (MEMOTE) and were validated by laboratory experiments. For the curation of those models, the software infrastructure refineGEMs was developed to work on these models in parallel and to comply with the quality standards for GEMs. The model predictions were confirmed by experimental data and a new comparison metric based on the doubling time was developed to quantify bacterial growth. Discussion: Future modeling projects can rely on the proposed software, which is independent of specific environmental conditions. The validation approach based on the growth rate calculation is now accessible and closely aligned with biological questions. The curated models are freely available via BioModels and a GitHub repository and can be used. The open-source software refineGEMs is available from https://github.com/draeger-lab/refinegems.

Preprocessing models for speech technologies

This paper describes the linguistic preprocessing methods on hybrid systems provided by an Artificial Intelligence (AI) international company, Defined.ai. The startup focuses on providing high-quality data, models, and AI tools. The main goal of this work is to enhance and advance the quality of preprocessing models by applying linguistic knowledge. Thus, we focus on two introductory linguistic models in a speech pipeline: Normalizer and Grapheme-to-Phone (G2P). To do so, two initiatives were conducted in collaboration with the Defined.ai Machine Learning team. The first project focuses on expanding and improving a European Portuguese Normalizer model. The second project covers creating G2P models for two different languages – Swedish and Russian. Results show that having a rule-based approach to the Normalizer and G2P increases its accuracy and performance, representing a significant advantage in improving Defined.ai tools and speech pipelines. Also, with the results obtained on the first project, we improved the normalizer in ease of use by increasing each rule with linguistic knowledge. Accordingly, our research demonstrates the added value of linguistic knowledge in preprocessing models.

Reconstruction and metabolic profiling of the genome-scale metabolic network model of Pseudomonas stutzeri A1501

Pseudomonas stutzeri A1501 is a non-fluorescent denitrifying bacteria that belongs to the gram-negative bacterial group. As a prominent strain in the fields of agriculture and bioengineering, there is still a lack of comprehensive understanding regarding its metabolic capabilities, specifically in terms of central metabolism and substrate utilization. Therefore, further exploration and extensive studies are required to gain a detailed insight into these aspects. This study reconstructed a genome-scale metabolic network model for P. stutzeri A1501 and conducted extensive curations, including correcting energy generation cycles, respiratory chains, and biomass composition. The final model, iQY1018, was successfully developed, covering more genes and reactions and having higher prediction accuracy compared with the previously published model iPB890. The substrate utilization ability of 71 carbon sources was investigated by BIOLOG experiment and was utilized to validate the model quality. The model prediction accuracy of substrate utilization for P. stutzeri A1501 reached 90 %. The model analysis revealed its new ability in central metabolism and predicted that the strain is a suitable chassis for the production of Acetyl CoA-derived products. This work provides an updated, high-quality model of P. stutzeri A1501for further research and will further enhance our understanding of the metabolic capabilities.

QES-Plume v1.0: a Lagrangian dispersion model

Abstract. Low-cost simulations providing accurate predictions of transport of airborne material in urban areas, vegetative canopies, and complex terrain are demanding because of the small-scale heterogeneity of the features influencing the mean flow and turbulence fields. Common models used to predict turbulent transport of passive scalars are based on the Lagrangian stochastic dispersion model. The Quick Environmental Simulation (QES) tool is a low-computational-cost framework developed to provide high-resolution wind and concentration fields in a variety of complex atmospheric-boundary-layer environments. Part of the framework, QES-Plume, is a Lagrangian dispersion code that uses a time-implicit integration scheme to solve the generalized Langevin equations which require mean flow and turbulence fields. Here, QES-Plume is driven by QES-Winds, a 3D fast-response model that computes mass-consistent wind fields around buildings, vegetation, and hills using empirical parameterizations, and QES-Turb, a local-mixing-length turbulence model. In this paper, the particle dispersion model is presented and validated against analytical solutions to examine QES-Plume’s performance under idealized conditions. In particular, QES-Plume is evaluated against a classical Gaussian plume model for an elevated continuous point-source release in uniform flow, the Lagrangian scaling of dispersion in isotropic turbulence, and a non-Gaussian plume model for an elevated continuous point-source release in a power-law boundary-layer flow. In these cases, QES-Plume yields a maximum relative error below 6 % when compared with analytical solutions. In addition, the model is tested against wind-tunnel data for a uniform array of cubical buildings. QES-Plume exhibits good agreement with the experiment with 99 % of matched zeros and 59 % of the predicted concentrations falling within a factor of 2 of the experimental concentrations. Furthermore, results also emphasize the importance of using high-quality turbulence models for particle dispersion in complex environments. Finally, QES-Plume demonstrates excellent computational performance.

The reservoir characteristics and their controlling factors of the sublacustrine fan in the Paleogene Dongying Formation, Bohai Sea, China

Sublacustrine fan is an important element in continental lacustrine basins and is significant for reservoir exploration. Oil and gas resources have been found in the sublacustrine fan sandstone reservoirs of the Paleogene Dongying Formation in the Bohai Sea. In this study, the characteristics of the sublacustrine fan reservoirs and the controlling factors are studied using information from logging, cores, physical properties, casting thin sections, X-ray diffraction of clay minerals, vitrinite reflectance, rock pyrolysis, manometry data, and 3D seismic data. The sublacustrine fans of the Dongying Formation in the LD10, QHD34, and BZ21 structures show high-quality reservoirs with porosity >15 % and permeability >5 mD. The main controlling factors of the high-quality reservoirs are attributed to the favorable sedimentary facies type and negligible compaction, and cementation, substantial dissolution of K-feldspar, overpressure, and the development of faults and fractures. A high-quality sublacustrine fan reservoir model has been established to explain how these factors affected the physical properties. The favorable targets for oil and gas exploration in the Dongying Formation of Bohai Sea include undercompacted sandy debris flow reservoirs showing the dissolution of K-feldspar, and the reservoirs which are connected to the source rocks by faults. This study provides insights to establish the relationship between sedimentology, diagenesis and reservoir quality. The results of this study are significant for the exploration and development of the sublacustrine fan sandstone reservoirs in the Bohai Sea and analogous sandstone reservoirs elsewhere.

High-precision composite 3D shape measurement of aeroengine blade based on parallel single-pixel imaging and high-dynamic range N-step fringe projection profilometry

Aeroengine blade is a core component of aircrafts. Accurate measurement of its profile has a great significance for flight safety. However, obtaining a high-quality 3D model of blade is difficult because traditional methods are limited in simultaneously meeting the requirements of large-scale measurement of smooth areas and high-precision measurement of fine structures. We develop a high-precision composite 3D shape measurement method of aeroengine blade. The proposed system consists of two sensors: a global sensor for large-range measurement and a local sensor for high-precision measurement. The global sensor based on parallel single-pixel imaging is used for smooth areas such as blade body and back, which undertakes most of the measurement tasks. The local sensor based on high-dynamic range N-step fringe projection profilometry has a high resolution to measure fine structures such as blade edge and top. We also use the local sensor to complete the missing point cloud of strong reflective areas. A phase-based data alignment method is developed to obtain coordinate transformation between the global and local sensors. Experiment results demonstrate that the proposed method can achieve accurate measurement and finally reconstruct the high-quality 3D model of aeroengine blade.

Modelling labs for a course on differential equations

Differential equations are widely used tools for modelling the world around us, making a course in differential equations a natural place for students to connect concrete mathematical applications to abstract concepts. Since students grasp the concepts better by applying them, introducing differential equations through modelling becomes essential. However, students often struggle to create and understand these models in the course. The purpose of this article is to share and examine a scaffolded sequence of labs implemented at regular intervals throughout a semester intended to train students in construction and analysis of mathematical models using differential equations. In these labs students learn basic principles of modelling that better prepare them for many of the high-quality modelling projects already available in differential equations. The course in which these activities were developed is taught using the SIMIODE text, Differential Equations: A Toolbox for Modelling the World, by Kurt Bryan. The labs are split into two components, with the first completed during class time and the second as out of class assignments. Furthermore, the course was re-organised to allow for the time needed to introduce the labs in class. We include classroom materials for labs, analysis, results on student perceptions and pedagogical recommendations.

A comparative evaluation of streamflow prediction using the SWAT and NNAR models in the Meenachil River Basin of Central Kerala, India.

Reliable and accurate modelling of streamflow is still a challenging task due to their complex behaviour, need for extensive parameter for development as well as lack of complete or accurate data. In this study, the applicability of an emerging data-driven model, specifically a neural network autoregression (NNAR) model, was evaluated for the first time as a substitute to the physically based hydrological model Soil and Water Assessment Tool (SWAT) for predicting streamflow under data-scarce conditions and for immediate high-quality modelling results. The inputs to the NNAR model were the lagged values of the daily streamflow time series data, and the output was the predicted value for the next day. Using streamflow data that was windowed by 20 days, the NNAR model produced the best prediction. The results of the statistical metrics used to evaluate the performance of the NNAR model were satisfactory (R = 0.90, RMSE = 28.27, MAE = 11.92, R2 = 0.83), indicating a high degree of agreement between the predicted and observed streamflow. The NNAR model outputs demonstrated its ability to accurately predict streamflow in the river basin, even without an explicit understanding of the physical processes that govern the system.

Diffusion-Denoising Process with Gated U-Net for High-Quality Document Binarization

The binarization of degraded documents represents a crucial preprocessing task for various document analyses, including optical character recognition and historical document analysis. Various convolutional neural network models and generative models have been used for document binarization. However, these models often struggle to deliver generalized performance on noise types the model has not encountered during training and may have difficulty extracting intricate text strokes. We herein propose a novel approach to address these challenges by introducing the use of the latent diffusion model, a well-known high-quality image-generation model, into the realm of document binarization for the first time. By leveraging an iterative diffusion-denoising process within the latent space, our approach excels at producing high-quality, clean, binarized images and demonstrates excellent generalization using both data distribution and time steps during training. Furthermore, we enhance our model’s ability to preserve text strokes by incorporating a gated U-Net into the backbone network. The gated convolution mechanism allows the model to focus on the text region by combining gating values and features, facilitating the extraction of intricate text strokes. To maximize the effectiveness of our proposed model, we use a combination of the latent diffusion model loss and pixel-level loss, which aligns with the model’s structure. The experimental results on the Handwritten Document Image Binarization Contest and Document Image Binarization Contest benchmark datasets showcase the superior performance of our proposed model compared to existing methods.

ОБОСНОВАНИЕ МЕТОДИКИ ПРОВЕДЕНИЯ ИССЛЕДОВАНИЯ ИНТЕНСИВНОСТИ ТРАНСПОРТНЫХ ПОТОКОВ

This paper presents a methodology for measuring the intensity of traffic flows, which provides the necessary initial data for the construction of a high-quality transport model. For further calculations, a full-scale survey of the accounting points necessary for the analysis was carried out. The video material obtained as a result of the surveys was processed in accordance with the current regulatory documentation for all types of vehicles and presented in the form of intensity pass-ports for each accounting point.

A Laboratory for the Integration of Geomatic and Geomechanical Data: The Rock Pinnacle “Campanile di Val Montanaia”

This work describes a procedure for building a high-quality 3D model of a rocky pinnacle in the Dolomites, Italy, using Structure from Motion (SfM) techniques. The pinnacle, known as “Campanile di Val Montanaia”, is challenging to survey due to its high elevation and sub-vertical cliffs. The construction of the 3D model is the first step in a multi-disciplinary approach to characterize the rock mass and understand its behavior and evolution. This paper discusses the surveying operations, which involved climbing the pinnacle to collect Ground Control Points (GCPs) and using a UAV to capture aerial imagery. The photographs were processed using SfM software to generate point clouds, mesh, and texture, which were then used for rock mass discontinuity mapping. The study compares models of different qualities and point densities to determine the optimal trade-off between processing time and accuracy in terms of discontinuity mapping. The results show that higher quality models allow for more detailed mapping of discontinuities, with some drawbacks due to noise in the case of the densest solution (e.g., increase in frequency of outliers across the point cloud). These pros and cons are also discussed in relation to the computational cost necessary to build the models. The study also examines the limitations and challenges of performing discontinuity mapping in the different models, including subjectivity in interpretation. A further element of interest is the publication of a high-quality 3D georeferenced model of the “Campanile di Val Montanaia” to be used for several potential further applications, such as stability analyses and numerical modeling.

Synthesis and Binding Mode Predictions of Novel Siglec-7 Ligands.

Siglec-7 regulates immune cell activity and is a promising target for immunomodulation. Here, we report the discovery of novel sialic acid derivatives binding to Siglec-7. Synthesis and affinity measurements are complemented by high-quality models of sialoside-Siglec-7 complexes based on molecular dynamics (MD) simulations on the microsecond time scale. We provide details for the predicted binding modes for the new ligands, e.g., that an extension of the carbon backbone leads to a different molecular interaction pattern with the receptor and the nearby water structure than found for known Siglec-7 ligands. Further on, we uncover some shortcomings of the GLYCAM06 and GAFF2 force fields when used for the simulation of sialoside-based glycomimetics. Our results open new opportunities for the rational design of Siglec-7 inhibitors. In addition, we provide strategies on how to use and visualize MD simulations to describe and investigate sialoside-Siglec complexes in general.