Merger Model Research Articles

Evolutionary optimization of model merging recipes

Abstract Large language models (LLMs) have become increasingly capable, but their development often requires substantial computational resources. Although model merging has emerged as a cost-effective promising approach for creating new models by combining existing ones, it currently relies on human intuition and domain knowledge, limiting its potential. Here we propose an evolutionary approach that overcomes this limitation by automatically discovering effective combinations of diverse open-source models, harnessing their collective intelligence without requiring extensive additional training data or compute. Our approach operates in both parameter space and data flow space, allowing optimization beyond just the weights of the individual models. This approach even facilitates cross-domain merging, generating models such as a Japanese LLM with math reasoning capabilities. Surprisingly, our Japanese math LLM achieved state-of-the-art performance on a variety of established Japanese LLM benchmarks, even surpassing models with substantially more parameters, despite not being explicitly trained for such tasks. Furthermore, a culturally aware Japanese vision–language model generated through our approach demonstrates its effectiveness in describing Japanese culture-specific content, outperforming previous Japanese vision–language models. This work not only contributes new state-of-the-art models back to the open-source community but also introduces a new paradigm for automated model composition, paving the way for exploring alternative, efficient approaches to foundation model development.

Lime and marble deposits mapping and estimation through deep neural layers-random forest merger and remote sensing imagery

Abstract Lime and marble are sedimentary rocks (also called carbonated rocks), mainly composed of calcite and dolomite as major ingredients. Calcite (CaCo3) and dolomite (MgCo3) offer excellent reflection in visible and short-wave infrared bands (0.4 to 2.5 µm), which distinguish them from other rock types and makes them a solid application area for multispectral sentinel-2 sensors. In this study, monumental deposits of carbonated rocks of the intended study region are mapped and estimated using a novel Deep Neural Layers-Random Forest merger model and Sentinel-2 imagery. We used ArcGIS 10.5 to prepare a high-quality classified map of the study regions, which can be used further to effectively and efficiently reach out and extract those deposits. The proposed model, trained on enough purified data containing 131,750 pixels of size 10 m, mapped the deposits with 0.95 accuracy (kappa coefficient 0.94), which is 70 % of the total mineral deposits of the study regions. The proposed model is the improved version of a prior experimented model consisting of convolution layers and dense layers only, whose accuracy was 0.93.

The impact of mass uncertainties on r-process nucleosynthesis in neutron star mergers

Theoretically predicted yields of elements created by the rapid neutron capture (r-) process carry potentially large uncertainties associated with incomplete knowledge of nuclear properties and approximative hydrodynamical modeling of the matter ejection processes. One of the dominant uncertainties in determining the ejecta composition and radioactive decay heat stems from the still unknown nuclear masses of exotic neutron-rich nuclei produced during the neutron irradiation. We investigate both the model (systematic) and parameter (statistical) uncertainties affecting nuclear mass predictions and explore their impact on r-process production, and subsequently on the composition of neutron star merger ejecta. The impact of correlated model uncertainties on masses is estimated by considering five different nuclear mass models that are known to provide an accurate description of known masses. In addition, the uncorrelated uncertainties associated with local variation of model parameters are estimated using a variant of the backward-forward Monte Carlo method, to constrain the parameter changes to experimentally known masses before propagating them consistently to the unknown masses of neutron-rich nuclei. The impact of nuclear mass uncertainties is propagated to r-process nucleosynthesis in a 1.38-1.38 M_⊙ neutron star merger model considering a large and representative number of trajectories. We find that the uncorrelated parameter uncertainties lead to ejected abundance uncertainties of 20% up to A ≃ 130 and 40% between A=150 and 200, with peaks around A ≃ 140 and A ≃ 203 giving rise to deviations around 100 to 300%. The correlated model uncertainties remain larger than the parameter ones for most nuclei. However, both model and parameter uncertainties have an important impact on heavy nuclei production. Improvements to nuclear models are still crucial in reducing uncertainties in predictions related to r-process nucleosynthesis. Both correlated model uncertainties and coherently determining parameter uncertainties are key in the sensitivity analysis.

Invariant Representation Learning in Multimedia Recommendation with Modality Alignment and Model Fusion.

Multimedia recommendation systems aim to accurately predict user preferences from multimodal data. However, existing methods may learn a recommendation model from spurious features, i.e., appearing to be related to an outcome but actually having no causal relationship with the outcome, leading to poor generalization ability. While previous approaches have adopted invariant learning to address this issue, they simply concatenate multimodal data without proper alignment, resulting in information loss or redundancy. To overcome these challenges, we propose a framework called M3-InvRL, designed to enhance recommendation system performance through common and modality-specific representation learning, invariant learning, and model merging. Specifically, our approach begins by learning modality-specific representations along with a common representation for each modality. To achieve this, we introduce a novel contrastive loss that aligns representations and imposes mutual information constraints to extract modality-specific features, thereby preventing generalization issues within the same representation space. Next, we generate invariant masks based on the identification of heterogeneous environments to learn invariant representations. Finally, we integrate both invariant-specific and shared invariant representations for each modality to train models and fuse them in the output space, reducing uncertainty and enhancing generalization performance. Experiments on real-world datasets demonstrate the effectiveness of our approach.

Merging white dwarf binaries produce Type Ia supernovae in elliptical galaxies

ABSTRACT I find that Type Ia supernovae (SNe Ia) with bimodal nebular emission profiles occur almost exclusively in massive (${\rm M_\star } \gtrsim 10^{11}~{\rm M_\odot }$) galaxies with low star formation rates (SFR $\lesssim 0.5~{\rm M_\odot }$ yr−1). The bimodal profiles are likely produced by two white dwarfs (WDs) that exploded during a merger or collision, supported by a correlation between the peak-to-peak velocity separation ($v_{\rm sep}$) and the SN Ia peak luminosity ($M_V$) which arises naturally from more massive WD binaries synthesizing more $^{56}$Ni during the explosion. The distributions of SNe Ia with and without bimodal nebular lines differ in host mass, SFR, and specific SFR with Kolmogorov–Smirnov test probabilities of $3.1{{\ \rm per\ cent}}$, $0.03{{\ \rm per\ cent}}$, and $0.02{{\ \rm per\ cent}}$, respectively. Viewing angle effects can fully explain the SNe Ia in quiescent hosts without bimodal emission profiles and the dearth of merger/collision driven SNe Ia in star-forming hosts requires at least two distinct progenitor channels for normal SNe Ia. $30\!-\!40{{\ \rm per\ cent}}$ of all SNe Ia originate from mergers or collisions depending on how cleanly host environment distinguishes progenitor scenarios. Existing models for WD mergers and collisions broadly reproduce the $v_{\rm sep}$–$M_V$ correlation and future analyses may be able to infer the masses/mass-ratios of merging WDs in external galaxies.

A Comparative Analysis of Instruction Fine-Tuning Large Language Models for Financial Text Classification

Large Language Models (LLMs) have demonstrated impressive capabilities across diverse Natural Language Processing (NLP) tasks, including language understanding, reasoning, and generation. However, general-domain LLMs often struggle with financial tasks due to the technical and specialized nature of financial texts. This study investigates the efficacy of instruction fine-tuning smaller-scale LLMs, including Mistral-7B, Llama3-8B, and Phi3-mini, to enhance their performance in financial text classification tasks. We fine-tuned both instruction-tuned and base models across four financial classification tasks, achieving significant improvements in task-specific performance. Furthermore, we evaluated the zero-shot capabilities of these fine-tuned models on three unseen complex financial tasks, including argument classification, deal completeness classification, and causal classification. Our results indicate while base model fine-tuning led to greater degradation, instruction-tuned models maintained more robust performance. To address this degradation, we employed model merging techniques, integrating single-task domain-specific fine-tuned models with the base model. Using this merging method resulted in significant enhancements in zero-shot performance, even exceeding the original model’s accuracy on certain datasets. Our findings underscore the effectiveness of instruction fine-tuning and model merging for adapting LLMs to specialized financial text classification tasks.

Analytical study of the merging rate of low-mass intermediate-mass black holes in preparation for the future Einstein Telescope

Context. The detection of gravitational wave (GW) signals by Advanced LIGO, Virgo, and KAGRA interferometers opened a new chapter in our understanding of the formation of compact objects. In particular, the detection of GW190521 is observational confirmation of the existence of intermediate-mass black holes (IMBHs); yet more direct observations are needed to better understand the mechanisms behind their formation. Aims. In this study, we explore the potential of the next-generation ground-based detector, the Einstein Telescope (ET), to advance our understanding of astrophysics through the detection of GWs emitted by IMBHs. To achieve this, the ET is designed to have improved sensitivity in the low-frequency range of approximately 2–10 Hz, enabling the detection of GWs originating from binary systems containing IMBHs with masses in the range of approximately 102–105 M⊙. Methods. We consider black holes (BHs) in the pair-instability form via the hierarchical merger model in galaxies, and approximate the number of events that could be observed by the ET. Results. Our findings indicate that ET could detect a binary black hole (BBH) merger rate of around 2 × 105 Gpc−3 yr−1 for BH masses ranging from 10 to 200 M⊙, with around 100 Gpc−3 yr−1 of this rate specifically attributed to BHs in the 100–200 M⊙ mass range, which we classify as low-mass IMBHs in this study. This suggests that ET could detect several dozen events similar to GW190521. The exact locations of these BBH mergers are not specified and we count our BH mergers across the entire universe up to a redshift of z ≈ 2. Conclusions. Observations made with the ET are expected to significantly enhance our comprehension of galactic BH growth, and the existence and characteristics of low-mass IMBHs.

A Hybrid GaN HEMT Model Merging Artificial Neural Networks and ASM-HEMT for Parameter Precision and Scalability

A Hybrid GaN HEMT Model Merging Artificial Neural Networks and ASM-HEMT for Parameter Precision and Scalability

Using Four-Component Instructional Design to Create an Interactive Point-of-Care Ultrasound Curriculum for Physician Associate Students.

Many physician assistant/associate (PA) programs lack point-of-care ultrasound (POCUS) education in the PA curricula and a standardized approach to ultrasound training. The four-component instructional design (4C/ID) model merges 4 concepts for developing effective instructional design content for complex content delivery, such as ultrasound. This research study created an interactive, 2-part ultrasound curriculum with an instructional designer for PA students using the 4C/ID model. Two interactive ultrasound curriculum models were created from the 4C/ID model using a knowledge-based precurriculum quiz, learning tasks with online modules and recordings, part-task practice with ultrasound experts, skills checkoffs, and a postcurriculum quiz including questions on students' perspectives of ultrasound. Two-tailed paired t-test analysis assessed the effectiveness of training. Evaluation of prelearning and postlearning quizzes demonstrated multiple statistically significant assessments supporting the use of instructional design technology for a PA program's ultrasound curriculum. This small study demonstrated that the 4C/ID model may be beneficial for PA POCUS training. The 2-part interactive curriculum improved PA student competency of ultrasound principles and applications and increased confidence for future clinical use of ultrasound.

Predicting pathological complete response following neoadjuvant chemoradiotherapy (nCRT) in patients with locally advanced rectal cancer using merged model integrating MRI-based radiomics and deep learning data

BackgroundTo construct and compare merged models integrating clinical factors, MRI-based radiomics features and deep learning (DL) models for predicting pathological complete response (pCR) to neoadjuvant chemoradiotherapy (nCRT) in patients with locally advanced rectal cancer (LARC).MethodsTotally 197 patients with LARC administered surgical resection after nCRT were assigned to cohort 1 (training and test sets); meanwhile, 52 cases were assigned to cohort 2 as a validation set. Radscore and DL models were established for predicting pCR applying pre- and post-nCRT MRI data, respectively. Different merged models integrating clinical factors, Radscore and DL model were constituted. Their predictive performances were validated and compared by receiver operating characteristic (ROC) and decision curve analyses (DCA).ResultsMerged models were established integrating selected clinical factors, Radscore and DL model for pCR prediction. The areas under the ROC curves (AUCs) of the pre-nCRT merged model were 0.834 (95% CI: 0.737–0.931) and 0.742 (95% CI: 0.650–0.834) in test and validation sets, respectively. The AUCs of the post-nCRT merged model were 0.746 (95% CI: 0.636–0.856) and 0.737 (95% CI: 0.646–0.828) in test and validation sets, respectively. DCA showed that the pretreatment algorithm could yield enhanced clinically benefit than the post-nCRT approach.ConclusionsThe pre-nCRT merged model including clinical factors, Radscore and DL model constitutes an effective non-invasive tool for pCR prediction in LARC.

Development of a Non-Linear Battery Management System Model for Lithium-Ion Batteries Using MATLAB Simulation

Abstract: This research focuses on developing a comprehensive battery model tailored for Battery Management Systems (BMS) using lithium-ion batteries. Given the increasing reliance on lithium-ion batteries in electric vehicles and portable electronic devices, this study aims to construct a non-linear battery model that accurately reflects the charging and discharging characteristics under various load conditions. The research identifies suitable characteristics of lithium-ion batteries essential for effective battery model construction. A MATLAB-Simulink approach is employed to build the model, which integrates both charging and discharging processes into a single simulation framework. This merged model enhances the existing non-linear models by offering improved accuracy in simulating battery behavior under fixed parameters. The outcome is expected to provide a reference model for future studies and applications in power management systems. Additionally, this research underscores the critical role of BMS in extending battery life and ensuring safety, making it a significant contribution to the field of energy management. The study's findings highlight the potential of the proposed model to serve as a foundation for further research and development in battery technologies.

A 3D finite deformation constitutive model for anisotropic shape memory polymer composites integrating viscoelasticity and phase transition concept

The phase transition theory of shape memory polymers (SMPs) often involves a phenomenological assumption that the reference configuration of the newly transformed phase deviates from that of the initial phase. This distinction serves as a crucial mechanism in the manifestation of the shape memory effect. However, elucidating the precise definition of the reference configuration of the transformed phase poses a significant challenge in the formulation of the constitutive model. To tackle this challenge, a three-dimensional (3D) finite deformation constitutive model incorporating effective phase evolution for SMPs has been developed. This model merges insights from the classical viscoelastic framework with the phase transition theory. The anisotropic thermo-viscoelastic constitutive model is further developed by introducing hyperelastic fibers, which integrate the anisotropy of the fibers into a continuous thermodynamic framework through structure tensors. Implemented within the ABAQUS software via a user material (UMAT) subroutine, the proposed model has been meticulously validated against experimental data, showcasing its prowess in simulating stress-strain responses and shape memory characteristics of SMPs and their composites (SMPCs). This innovative model stands as an invaluable instrument for the design and of sophisticated SMP and SMPC structures.

CoMPers: A configurable conflict management framework for personalized collaborative modeling

Context:Modeling is an activity in the software development life cycle where experts and stakeholders collaborate as a team. In collaborative modeling, adhering to the optimistic versioning paradigm allows users to make concurrent changes to the same model, but conflicts may arise. To achieve an integrated and consistent merged model, conflicts must be resolved. Objective:The primary objective of this study was to provide a customizable and extensible framework for conflict management in personalized change propagation during collaborative modeling. Methods:We propose CoMPers, a customizable and extensible conflict management framework designed to address various conflicts encountered in collaborative modeling. We present the duel algorithm for automatically detecting and resolving conflicts according to user preferences. The framework utilizes personalized change propagation to customize collaboration and supports the conflict management process by executing the duel algorithm based on user preferences. As a proof-of-concept, we have implemented the CoMPers framework and extended the EMF.cloud modeling framework to demonstrate its applicability. Results:We have constructed a proof-of-concept implementation and conducted a real-world case study, a benchmark experiment, and a user experience evaluation. Our findings demonstrate that: (1) CoMPers enables collaborators to configure propagation strategies according to their habits; (2) CoMPers successfully identifies all anticipated conflicts and achieves a 100% accuracy in conflict handling; (3) The majority of participants agreed that CoMPers is user-friendly for collaborative modeling. Conclusion:This paper presents the CoMPers framework, which is based on personalized change propagation, and helps collaborators customize conflict management activities. The results confirm the feasibility and advantages of consistent and concurrent modeling within the collaborative CoMPers platform, with an acceptable functionality for approximately ten collaborators.

Mergers, firm size, and volatility in a granular economy

We study the firm dynamics associated with mergers and acquisitions (M&A) and their implications at the micro and macro levels. Our paper presents three main findings: (i) mergers generate a more fat-tailed firm-size distribution, thereby amplifying granular fluctuations and increasing aggregate volatility; (ii) the impact of mergers depends on strategic market power and endogenous markups; and (iii) under endogenous markups, we provide a novel characterization of the firm size-volatility relationship in which volatility declines disproportionately with size. We build a quantitative model of domestic horizontal mergers and find a sizeable impact of mergers on aggregate volatility using counterfactual analysis.

Hybrid Teaching Design for the "Software Project Management" Course

This paper investigates a hybrid instructional design to improve the educational outcomes of the "Software Project Management" course. Confronted with the challenges of globalization and rapid technological progress, traditional lecture-based teaching has proven inadequate to fulfill the goals of contemporary education. The hybrid teaching model merges online self-directed learning with in-person interactive sessions, facilitating the acquisition and application of knowledge and the enhancement of practical skills through pre-class video instruction, classroom discussions, and case analyses. The educational objectives encompass the transmission and internalization of knowledge, the cultivation of professional ethics, and the improvement of students' comprehensive qualities. The instructional process is meticulously designed to include pre-class self-study, in-class knowledge assessment, case study discussions, practical coursework, and final evaluations. Moreover, the study places a special focus on integrating ideological and political elements within the curriculum to elevate students' ideological and moral qualities.

Waveform systematics in gravitational-wave inference of signals from binary neutron star merger models incorporating higher-order modes information

Waveform systematics in gravitational-wave inference of signals from binary neutron star merger models incorporating higher-order modes information

Quasi-two-dimensionality of three-dimensional, magnetically dominated, decaying turbulence

Decaying magnetohydrodynamic (MHD) turbulence is important in various astrophysical contexts, including early universe magnetic fields, star formation, turbulence in galaxy clusters, magnetospheres and solar corona. Previously known in the nonhelical case of magnetically dominated decaying turbulence, we show that magnetic reconnection is important also in the fully helical case and is likely the agent responsible for the inverse transfer of energy. Again, in the fully helical case, we find that there is a similarity in power law decay exponents in both 2.5D and 3D simulations. To understand this intriguing similarity, we investigate the possible quasi-two-dimensionalization of the 3D system. We perform Minkowski functional analysis and find that the characteristic length scales of a typical magnetic structure in the system are widely different, suggesting the existence of local anisotropies. Finally, we provide a quasi-two-dimensional hierarchical merger model which recovers the relevant power law scalings. In the nonhelical case, we show that a helicity-based invariant cannot constrain the system, and the best candidate is still anastrophy or vector potential squared, which is consistent with the quasi-two-dimensionalization of the system.

Cephalo: Multi‐Modal Vision‐Language Models for Bio‐Inspired Materials Analysis and Design

AbstractCephalo is presented as a series of multimodal vision large language models (V‐LLMs) designed for materials science applications, integrating visual and linguistic data for enhanced understanding. A key innovation of Cephalo is its advanced dataset generation method. Cephalo is trained on integrated image and text data from thousands of scientific papers and science‐focused Wikipedia data demonstrates it can interpret complex visual scenes, generate precise language descriptions, and answer queries about images effectively. The combination of a vision encoder with an autoregressive transformer supports multimodal natural language understanding, which can be coupled with other generative methods to create an image‐to‐text‐to‐3D pipeline. To develop more capable models from smaller ones, both mixture‐of‐expert methods and model merging are reported. The models are examined in diverse use cases that incorporate biological materials, fracture and engineering analysis, protein biophysics, and bio‐inspired design based on insect behavior. Generative applications include bio‐inspired designs, including pollen‐inspired architected materials, as well as the synthesis of bio‐inspired material microstructures from a photograph of a solar eclipse. Additional model fine‐tuning with a series of molecular dynamics results demonstrate Cephalo's enhanced capabilities to accurately predict statistical features of stress and atomic energy distributions, as well as crack dynamics and damage in materials.

Constraining Near-simultaneous Radio Emission from Short Gamma-Ray Bursts Using CHIME/FRB

We use the Canadian Hydrogen Intensity Mapping Experiment (CHIME) Fast Radio Burst (FRB) Project to search for FRBs that are temporally and spatially coincident with gamma-ray bursts (GRBs) occurring between 2018 July 7 and 2023 August 3. We do not find any temporal (within 1 week) and spatial (within overlapping 3σ localization regions) coincidences between any CHIME/FRB candidates and all GRBs with 1σ localization uncertainties <1°. As such, we use CHIME/FRB to constrain the possible FRB-like radio emission for 27 short gamma-ray bursts (SGRBs) that were within 17° of CHIME/FRB’s meridian at a point either 6 hr prior up to 12 hr after the high-energy emission. Two SGRBs, GRB 210909A and GRB 230208A, were above the horizon at CHIME at the time of their high-energy emission and we place some of the first constraints on simultaneous FRB-like radio emission from SGRBs. While neither of these two SGRBs have known redshifts, we construct a redshift range for each GRB based on their high-energy fluence and a derived SGRB energy distribution. For GRB 210909A, this redshift range corresponds to z = [0.009, 1.64] with a mean of z = 0.13. Thus, for GRB 210909A, we constrain the radio luminosity at the time of the high-energy emission to L < 2 × 1046 erg s−1, L < 5 × 1044 erg s−1, and L < 3 × 1042 erg s−1 assuming redshifts of z = 0.85, z = 0.16, and z = 0.013, respectively. We compare these constraints with the predicted simultaneous radio luminosities from different compact object merger models.

High-redshift Merger Model for Low-frequency Gravitational Wave Background

In 2023, the Pulsar Timing Array Collaborations announced the discovery of a gravitational wave background (GWB), predominantly attributed to supermassive black hole binary (SMBHB) mergers. However, the detected GWB is several times stronger than the default value expected from galactic observations at low and moderate redshifts. Recent findings by the James Webb Space Telescope have unveiled a substantial number of massive, high-redshift galaxies, suggesting more massive SMBHB mergers at these early epochs. Motivated by these findings, we propose an “early merger” model that complements the standard merger statistics by incorporating these early, massive galaxies. We compare the early and standard “late merger” models, which assume peak merger rates in the local Universe, and match both merger models to the currently detected GWB. Our analysis shows that the early merger model has a significantly lower detection probability for single binaries and predicts a ∼30% likelihood that the first detectable single source will be highly redshifted and remarkably massive with rapid frequency evolution. In contrast, the late merger model predicts a nearly monochromatic first source at low redshift. The future confirmation of an enhanced population of massive high-redshift galaxies and the detection of fast-evolving binaries would strongly support the early merger model, offering significant insights into the evolution of galaxies and SMBHs.