Distillation Method Research Articles

RMKD: Relaxed matching knowledge distillation for short-length SSVEP-based brain-computer interfaces.

Accurate decoding of electroencephalogram (EEG) signals in the shortest possible time is essential for the realization of a high-performance brain-computer interface (BCI) system based on the steady-state visual evoked potential (SSVEP). However, the degradation of decoding performance of short-length EEG signals is often unavoidable due to the reduced information, which hinders the development of BCI systems in real-world applications. In this paper, we propose a relaxed matching knowledge distillation (RMKD) method to transfer both feature-level and logit-level knowledge in a relaxed manner to improve the decoding performance of short-length EEG signals. Specifically, the long-length EEG signals and short-length EEG signals are decoded into the frequency representation by the teacher and student models, respectively. At the feature-level, the frequency-masked generation distillation is designed to improve the representation ability of student features by forcing the randomly masked student features to generate full teacher features. At the logit-level, the non-target class knowledge distillation and the inter-class relation distillation are combined to mitigate loss conflicts by imitating the distribution of non-target classes and preserve the inter-class relation in the prediction vectors of the teacher and student models. We conduct comprehensive experiments on two public SSVEP datasets in the subject-independent scenario with six different signal lengths. The extensive experimental results demonstrate that the proposed RMKD method has significantly improved the decoding performance of short-length EEG signals in SSVEP-based BCI systems.

Inspect quantitative signals in placental histopathology: Computer-assisted multiple functional tissues identification through multi-model fusion and distillation framework.

Pathological analysis of placenta is currently a valuable tool for gaining insights into pregnancy outcomes. In placental histopathology, multiple functional tissues can be inspected as potential signals reflecting the transfer functionality between fetal and maternal circulations. However, the identification of multiple functional tissues is challenging due to (1) severe heterogeneity in texture, size and shape, (2) distribution across different scales and (3) the need for comprehensive assessment at the whole slide image (WSI) level. To solve aforementioned problems, we establish a brand new dataset and propose a computer-aided segmentation framework through multi-model fusion and distillation to identify multiple functional tissues in placental histopathologic images, including villi, capillaries, fibrin deposits and trophoblast aggregations. Specifically, we propose a two-stage Multi-model Fusion and Distillation (MMFD) framework. Considering the multi-scale distribution and heterogeneity of multiple functional tissues, we enhance the visual representation in the first stage by fusing feature from multiple models to boost the effectiveness of the network. However, the multi-model fusion stage contributes to extra parameters and a significant computational burden, which is impractical for recognizing gigapixels of WSIs within clinical practice. In the second stage, we propose straightforward plug-in feature distillation method that transfers knowledge from the large fused model to a compact student model. In self-collected placental dataset, our proposed MMFD framework demonstrates an improvement of 4.3% in mean Intersection over Union (mIoU) while achieving an approximate 50% increase in inference speed and utilizing only 10% of parameters and computational resources, compared to the parameter-efficient fine-tuned Segment Anything Model (SAM) baseline. Visualization of segmentation results across entire WSIs on unseen cases demonstrates the generalizability of our proposed MMFD framework. Besides, experimental results on a public dataset further prove the effectiveness of MMFD framework on other tasks. Our work can present a fundamental method to expedite quantitative analysis of placental histopathology.

Mathematical and experimental study of the direct contact membrane distillation method for desalination using a hydrophobic electrospun nanofibers membrane

This paper examines the performance of a Direct Contact Membrane Distillation (DCMD) system experimentally and theoretically. The system uses a super hydrophobic electrospun nanofiber membrane to desalinate water. Investigations were carried out into how the feed concentration, feed flow rate, and feed temperature affected permeate flux. as system operating parameters to aid in comprehending the factors impacting the DCMD process. The application of DOE and Taguchi methods achieved statistical optimization of the DCMD process's performance. In addition, the study of mass and heat transport in DCMD was described by a theoretical model. While the feed concentration (0- 210 g/L) significantly affected flux, the feed's temperature (35-55 °C) and flow rate (0.2-0.6 L/min) mostly dominated the impact on system performance. The created model numerically solved the DCMD process using MATLAB software, describing it as a system of nonlinear equations. Various operating conditions were used to investigate the efficiency of the superhydrophobic electrospun nanofiber membrane in treating 210 g/L NaCl salt water. Changing the feed temperature and concentration affected the hypothetically suggested path across the membrane, according to the simulation results presented in this paper. Excellent agreement was observed between the experiment results and the constructed model's predicted results. Every instance maintained a high salt rejection rate (over 99.9%). The DCMD produced a gain output ratio (GOR) of 0.87 and a temperature polarization coefficient of 0.78 to 0.91. The system achieved a maximum thermal efficiency of 73.5%. The optimal parameters, which are 70 g/L, 0.6 L/min, and 55°C.

Distilling the knowledge from large-language model for health event prediction

Health event prediction is empowered by the rapid and wide application of electronic health records (EHR). In the Intensive Care Unit (ICU), precisely predicting the health related events in advance is essential for providing treatment and intervention to improve the patients outcomes. EHR is a kind of multi-modal data containing clinical text, time series, structured data, etc. Most health event prediction works focus on a single modality, e.g., text or tabular EHR. How to effectively learn from the multi-modal EHR for health event prediction remains a challenge. Inspired by the strong capability in text processing of large language model (LLM), we propose the framework CKLE for health event prediction by distilling the knowledge from LLM and learning from multi-modal EHR. There are two challenges of applying LLM in the health event prediction, the first one is most LLM can only handle text data rather than other modalities, e.g., structured data. The second challenge is the privacy issue of health applications requires the LLM to be locally deployed, which may be limited by the computational resource. CKLE solves the challenges of LLM scalability and portability in the healthcare domain by distilling the cross-modality knowledge from LLM into the health event predictive model. To fully take advantage of the strong power of LLM, the raw clinical text is refined and augmented with prompt learning. The embedding of clinical text are generated by LLM. To effectively distill the knowledge of LLM into the predictive model, we design a cross-modality knowledge distillation (KD) method. A specially designed training objective will be used for the KD process with the consideration of multiple modality and patient similarity. The KD loss function consists of two parts. The first one is cross-modality contrastive loss function, which models the correlation of different modalities from the same patient. The second one is patient similarity learning loss function to model the correlations between similar patients. The cross-modality knowledge distillation can distill the rich information in clinical text and the knowledge of LLM into the predictive model on structured EHR data. To demonstrate the effectiveness of CKLE, we evaluate CKLE on two health event prediction tasks in the field of cardiology, heart failure prediction and hypertension prediction. We select the 7125 patients from MIMIC-III dataset and split them into train/validation/test sets. We can achieve a maximum 4.48% improvement in accuracy compared to state-of-the-art predictive model designed for health event prediction. The results demonstrate CKLE can surpass the baseline prediction models significantly on both normal and limited label settings. We also conduct the case study on cardiology disease analysis in the heart failure and hypertension prediction. Through the feature importance calculation, we analyse the salient features related to the cardiology disease which corresponds to the medical domain knowledge. The superior performance and interpretability of CKLE pave a promising way to leverage the power and knowledge of LLM in the health event prediction in real-world clinical settings.

DualCFGL: dual-channel fusion global and local features for sequential recommendation

Sequential recommendation systems capture the dynamic interests of users and predict their future preferences. A noteworthy problem in sequential recommendation is coping with the intrinsic changes of user interests. The sequence of user interactions is generated by more than a single and stable global preference, users may have interest drift that occur in a short period of time. We call this short-term interest drift as the local preference of users, which is often a key factor affecting the final choice of users. However, existing methods have limitations in observing local preferences, which leads to an incomplete consideration of the local preferences. Moreover, using a single model to represent global–local preferences obscure the distinct features of each, limiting the potential synergistic benefits. To alleviate the above limitations, we propose a novel model with a dual-channel structure to monitor both global and local preferences and ensure they complement each other. The model extracts the global preferences of users with a bidirectional Transformer using random masking and a sliding window, and extracts the local preferences with a patch-based stacked bottleneck residual convolution. To enable the model to consider both the global and local preferences of users, we design an adaptive orthogonal fusion module, which effectively fuses the two preferences and enables the two feature types to complement and enhance each other. We integrate the fused user preferences with a knowledge distillation method that further improves the model’s expressive ability. We conduct extensive experiments on three widely used datasets, and the results show that our model outperforms current state-of-the-art models.

Phytochemical Screening, Chemical Constitutions, and Antibacterial Activity for three Medicinal Plants Against Multidrug-Resistant Bacteria

Due to the importance of therapeutic plants in the production and preparation of drugs, The purpose of this study is to extract three medicinal plants, including seeds of Syzygium aromaticum “S. aromaticum”, leaves of Thymus vulgaris (T. vulgaris) and Myrtus communis (M. communis) using the steam distillation method. It evaluated their ability to inhibit pathogenic and antibiotic-resistant bacteria, including Staphylococcus aureus (S.aureus), Pseudomonas aeruginosa (P. aeruginosa), and Escherichia coli (E.coli) isolated from skin ulcers. Moreover, these extracts were cured to evaluate their phytochemicals and chemical constitutions using classical methods and Gc-mass spectroscopy. The phytochemicals results showed that the Flavonoids and terpenoids were observed in all samples, while the Alkaloids didn't appear. The extracts of S. aromaticum, T. vulgaris, and M. communis showed interesting compounds during the GC-mass spectroscopy test. It showed the presence of chemical compounds with active groups that could have a significant role in bacterial treatment, such as 9-octadecenamide, Caryophyllene, and Limonene. Moreover, The results of the bacterial test showed a high effectiveness of these oils on the types of bacteria used, especially S. aureus, and E. coli, where the highest results were for Myrtus on E. coli bacteria with an inhibition diam of 56 mm, while clove oil did not show any effectiveness against S. aureus, the inhibition zone diameter is zero. In general, the effectiveness of these essential oils against Gram N. P.aeruginosa was very weak compared to the effectiveness of these essential oils on other bacterial species used in the study, where the highest diameter of inhibition of these bacteria was by clove oil and reached 15 mm.

Distillation method influences flavor characteristics of Soju

Distillation method influences flavor characteristics of Soju

Hybrid Process Flow Diagram for Separation of Fusel Oil into Valuable Components

Ethanol production by fermentation results in obtaining, in addition to the main product, ethyl alcohol, by-products and secondary products, which include carbon dioxide, fusel oil, and ester–aldehyde cut. Fusel oil, despite its low yield and the large volume of ethanol production, accumulates at distilleries, which ultimately raises the question of its disposal or the rational use of this by-product. Fusel oil, being a complex mixture, can serve as a source of technical alcohols used in various sectors of the economy, including the food industry, pharmaceuticals, organic synthesis, perfume, and cosmetics industries, as well as the production of paints and varnishes. However, the complexity of using fusel oil lies in its difficult separation. The reason for this is the presence of water, which forms low-boiling azeotropes with aliphatic alcohols. Our study aimed to develop a process flow diagram (PFD) that allows individual components from fusel oil to be obtained without extraneous separating agents (not inherent in fusel oil). This condition is necessary to obtain products labeled as natural for further use in the food, perfume, cosmetic, and pharmaceutical industries. The distinctive feature of this work is that the target product is not only isoamyl alcohol but also all other alcohols present in the composition of fusel oil. To achieve this goal and create a mathematical model, the Aspen Plus V14 application, the Non-Random Two Liquid (NRTL) thermodynamic model, and the Vap-Liq/Liq-Liq phase equilibrium were used. Fusel oil separation was modeled using a continuous separation PFD to obtain ethanol, water, isoamyl alcohol, and raw propanol and butanol cuts. The Sorel and Barbet distillation technique was used to isolate ethanol. The isolation of isopropanol and 1-propanol, as well as isobutanol and 1-butanol, was modeled using the batch distillation method. The isolation of fusel oil components was based on their thermodynamic properties and the selection of appropriate techniques for their separation, such as extraction, distillation, pressure swing distillation, and decantation. The simulation of fusel oil separation PFD showed the possibility of obtaining the components of a complex mixture without separating agents, as discussed earlier. Ethanol corresponds to the quality of rectified ethyl alcohol, and 1-butanol and isoamyl alcohols to anhydrous alcohols, whereas isopropanol (which contains an admixture of ethanol), 1-propanol, and isobutanol are obtained as aqueous solutions of different concentrations of alcohols. However, due to a distillation boundary in the raw propanol and butanol cuts, these mixtures cannot be separated completely, which leads to the production of intermediate fractions. To eliminate intermediate fractions and obtain anhydrous isopropanol, 1-propanol, and isobutanol in the future, it is necessary to solve the dehydration problem of either fusel oil or the propanol–butanol mixture.

Metal-Organic Framework-Based NF3 Nanotrap for the Separation of NF3 and CF4.

As an electronic specialty gas, nitrogen trifluoride (NF3) has been widely used in the semiconductor, photovoltaic, and display industries. However, NF3 in industrial settings is typically mixed with CF4, and the distillation method commonly used for their separation consumes significant energy. Herein, we propose a novel NF3 nanoadsorber featuring relatively proximal unsaturated metal sites. The potential field overlap between the two unsaturated metal centers creates an NF3-affinitive environment. The adsorption mechanism of NF3 molecules in the metal-organic frameworks (MOFs) was investigated through density functional theory (DFT) optimization. The results from ideal adsorbed solution theory (IAST) calculations and breakthrough experiments reveal that the MOF-based NF3 nanotrap possesses the highest adsorption capacity and excellent NF3/CF4 selectivity among the physically adsorptive materials reported to date. This work presents a novel solution for the separation and purification of NF3, contributing to the field of gas separation technology.

A unified multi‐class anomaly detection model based on reverse distillation

AbstractThe multi‐class single‐model detection paradigm is a prevalent design for industrial anomaly detection, exhibiting suitability for varying industrial classes and dynamic, flexible production scenarios. This approach not only enhances model adaptability but also minimizes maintenance costs. However, the current popular methods are susceptible to the ‘copying shortcut’ phenomenon, which constrains their performance on benchmark datasets. To overcome this limitation, this article proposes a multi‐class anomaly detection model: UniRD, based on the reverse distillation method. This model creates an image corruptor that expands the dataset and generates ‘normal‐corrupt’ image pairs. During the training process, their correspondence is used to optimize the reverse distillation. This process greatly exploits and releases the potential of the student decoder. Furthermore, a teacher feature adaptation module is devised to enhance the compatibility between the pre‐trained model and the anomaly detection task. This has the effect of reducing the discrepancy between teacher and student features while ensuring the consistency of normal sample features. The comprehensive evaluation results in two mainstream datasets, MVTec and VisA, and demonstrates that the proposed method exhibits improvement in all indicators compared to benchmark methods. The proposed method attains the state‐of‐the‐art, substantiating its effectiveness and advancement.

Evaluation of Biological Activity of Essential Oil from Teucrium polium L.

Background: Teucrium polium L. has an essential place in traditional medicine throughout the world. Their use, based on empirical knowledge passed down from generation to generation, has helped to treat a multitude of illnesses and relieve a variety of symptoms. These traditional practices have laid the foundations for modern research, which seeks to understand the significant antibacterial properties. The bioactive compounds present in the plant, such as essential oils and polyphenols, have demonstrated antimicrobial activity against a wide range of pathogenic bacteria. This includes bacteria responsible for gastrointestinal, skin and respiratory infections. Methods: The focus of this study was the extraction of essential oils from the aerial portions of Teucrium polium L. via the hydrodistillation method, their anti-oxidant effect using the 2,2-diphenyl-1-picrylhydrazyl (DPPH) free radical scavenging assay method and iron reduction and antimicrobial activity of these essential oils was evaluated against seven ATCC bacterial strains: Staphylococcus aureus ATCC25923, Escherichia coli ATCC25922, Salmonella typhimurium ATCC10428, Pseudomonas aeruginosa ATCC27853, Enterococcus faecalis ATCC29212, Klebsiella pneumoniae ATCC700603 and Bacillus subtilis ATCC7033. In addition, the minimum inhibitory concentrations (MICs) were determined by incorporation method. These tests were carried out at the Laboratoires de the analytical chemistry laboratory at the Department of Pharmacy, Badji Mokhtar University, Annaba, Algeria. Result: The hydro distillation method yielded an essential oil of 0.45%. Antimicrobial activity was tested by the sensitivity test, which revealed inhibition zone diameters ranging from 11.5 mm to £20 mm. In addition, minimum inhibitory concentrations (MICs) were determined, showing variations between bacterial strains, suggesting potential therapeutic importance for some strains. Additionally, the antioxidant activity of the essential oils was assessed using the iron-reducing capacity assay and the DPPH free radical scavenging assay. Although the results indicated notable antioxidant potential, the IC50 values obtained were relatively high when compared to ascorbic acid, registering 107.38 mg/ml for iron reduction and 91.62 mg/ml for DPPH reduction. These findings contribute to the exploration of the therapeutic potential of Teucrium polium L. essential oils for their antimicrobial and antioxidant properties.

A novel knowledge distillation framework for enhancing small object detection in blurry environments with unmanned aerial vehicle-assisted images

Deep learning-based object detectors excel on mobile devices but often struggle with blurry images that are common in real-world scenarios, like unmanned aerial vehicle (UAV)-assisted images. Current models are designed for sharp images, leading to potential detection failures in blurry images. Using image deblurring before object detection is an option, but it demands significant computing power and relies heavily on the accuracy of the deblurring algorithms. Another common issue is the suitable dataset for the specific problem. To address the aforementioned issues, we develop a UAV-assisted small object detection dataset and propose a novel knowledge distillation method for object detection in blurry images in complex environments. Following this, we employ a technique known as self-supervised knowledge distillation, where we introduce a deblurring subnet module with the help of two attention modules, where both networks are trained in a fully-supervised manner. Based on the experiment results, our proposed model achieves an improvement of 4.3% accuracy in the VisDrone synthetic motion blur dataset and 4.6% in detecting objects within synthetic blurry images in our developed small object detection dataset (SOD-Dataset), as well as competitive results compared with other state-of-the-art methods. Meanwhile, ablation experiments and a visualization analysis validate the contributions of each component of the model.

Pixel Distillation: Cost-Flexible Distillation Across Image Sizes and Heterogeneous Networks.

Previous knowledge distillation (KD) methods mostly focus on compressing network architectures, which is not thorough enough in deployment as some costs like transmission bandwidth and imaging equipment are related to the image size. Therefore, we propose Pixel Distillation that extends knowledge distillation into the input level while simultaneously breaking architecture constraints. Such a scheme can achieve flexible cost control for deployment, as it allows the system to adjust both network architecture and image quality according to the overall requirement of resources. Specifically, we first propose an input spatial representation distillation (ISRD) mechanism to transfer spatial knowledge from large images to student's input module, which can facilitate stable knowledge transfer between CNN and ViT. Then, a Teacher-Assistant-Student (TAS) framework is further established to disentangle pixel distillation into the model compression stage and input compression stage, which significantly reduces the overall complexity of pixel distillation and the difficulty of distilling intermediate knowledge. Finally, we adapt pixel distillation to object detection via an aligned feature for preservation (AFP) strategy for TAS, which aligns output dimensions of detectors at each stage by manipulating features and anchors of the assistant. Comprehensive experiments on image classification and object detection demonstrate the effectiveness of our method.

Mitigating Accuracy-Robustness Trade-Off via Balanced Multi-Teacher Adversarial Distillation.

Adversarial Training is a practical approach for improving the robustness of deep neural networks against adversarial attacks. Although bringing reliable robustness, the performance towards clean examples is negatively affected after Adversarial Training, which means a trade-off exists between accuracy and robustness. Recently, some studies have tried to use knowledge distillation methods in Adversarial Training, achieving competitive performance in improving the robustness but the accuracy for clean samples is still limited. In this paper, to mitigate the accuracy-robustness trade-off, we introduce the Balanced Multi-Teacher Adversarial Robustness Distillation (B-MTARD) to guide the model's Adversarial Training process by applying a strong clean teacher and a strong robust teacher to handle the clean examples and adversarial examples, respectively. During the optimization process, to ensure that different teachers show similar knowledge scales, we design the Entropy-Based Balance algorithm to adjust the teacher's temperature and keep the teachers' information entropy consistent. Besides, to ensure that the student has a relatively consistent learning speed from multiple teachers, we propose the Normalization Loss Balance algorithm to adjust the learning weights of different types of knowledge. A series of experiments conducted on three public datasets demonstrate that B-MTARD outperforms the state-of-the-art methods against various adversarial attacks.

3D Snapshot: Invertible Embedding of 3D Neural Representations in a Single Image.

3D neural rendering enables photo-realistic reconstruction of a specific scene by encoding discontinuous inputs into a neural representation. Despite the remarkable rendering results, the storage of network parameters is not transmission-friendly and not extendable to metaverse applications. In this paper, we propose an invertible neural rendering approach that enables generating an interactive 3D model from a single image (i.e., 3D Snapshot). Our idea is to distill a pre-trained neural rendering model (e.g., NeRF) into a visualizable image form that can then be easily inverted back to a neural network. To this end, we first present a neural image distillation method to optimize three neural planes for representing the original neural rendering model. However, this representation is noisy and visually meaningless. We thus propose a dynamic invertible neural network to embed this noisy representation into a plausible image representation of the scene. We demonstrate promising reconstruction quality quantitatively and qualitatively, by comparing to the original neural rendering model, as well as video-based invertible methods. On the other hand, our method can store dozens of NeRFs with a compact restoration network (5 MB), and embedding each 3D scene takes up only 160 KB of storage. More importantly, our approach is the first solution that allows embedding a neural rendering model into image representations, which enables applications like creating an interactive 3D model from a printed image in the metaverse.

Design of a Groutless Clear Coffee Machine Using Distillation Method

Coffee is a plantation commodity that is made into a drink as one of the products that is popular with the public. Many coffee drink innovations have been made, one of which is a clear coffee drink without dregs. The process of making clear coffee can be done by the distillation method, namely the process of condensing the steam produced by a mixture of coffee powder and water. This study aims to obtain the design results of a clear coffee machine without dregs using the distillation method. The design is carried out based on the results of a patented study on making clear coffee. The coffee machine is made with a heating tube for a mixture of ground coffee and water, a steam collector tube and a distillation tube. The design results obtained a heating tube with a diameter of 25 cm and a height of 27 cm for a process capacity of 13.25 liters. Heating is performed using an LPG-fueled heater. The distillation process is carried out by cooling the water using a 15 watt aerator pump. The trial was carried out by processing 500 grams of fine coffee powder mixed with 2000 ml of water. The trial results obtained 500 ml of clear coffee with an average process time of ± 25 minutes.

Formation of Hybrid Membranes for Water Desalination by the Method of Membrane Distillation

A method has been developed for the formation of hybrid membranes consisting of a hydrophilic microporous substrate and a hydrophobic nanofiber polymer layer deposited by electroforming. A track-etched membrane made of polyethylene terephthalate was used as a hydrophilic microporous substrate, on the surface of which a thin layer of titanium was applied by magnetron sputtering to ensure adhesion of the nanofiber layer. Simultaneously, the titanium coating was used to make a conductive track-etched membrane that served as a collector electrode. It is been shown that the application of this method for the formation of polymer coatings when used as a starting material for the formation of polyvinylidene fluoride nanofibers makes it possible to obtain a layer with highly hydrophobic properties, the water contact angle of the surface of which, depending on the deposition density, averages 143.3 ± 1.3°. A study of the morphology of the nanofiber coating shows that it has a microstructure typical of non-woven materials. The nanofibers forming the porous system of this layer have a wide range in size. The study of the molecular structure of the nanofiber layer by IR-Fourier spectroscopy and X-ray diffraction analysis showed that its structure is dominated by the β-phase, which is characterized by a maximum dipole moment. It is been shown that the hybrid membranes of the developed sample provide high separation selectivity when desalting an aqueous solution of sodium chloride with a concentration of 26.5 g/l by membrane distillation. The salt rejection coefficient for membranes with a nanofiber layer density from 20.7 ± 0.2 to 27.6 ± 0.2 g/m2 in the studied mode of the membrane distillation process is 99.97−99.98%. It has been established that the use of a highly hydrophobic nanofiber layer with a developed pore structure in combination with a hydrophilic microporous base makes it possible to increase the productivity of the membrane distillation process. The value of the maximum condensate flow through the membranes is on average 7.0 kg m2/h and its depends on the density of the deposited nanofiber layer.

Overcoming the challenges of multi-modal medical image sharing: A novel data distillation strategy via contrastive learning

Multi-modal medical images serve as a critical basis for disease diagnosis. However, sharing multi-modal medical data between hospitals is a significant challenge due to privacy protection concerns as well as the substantial costs associated with data transfer and storage. These hurdles are further compounded by two primary issues: (1) the complexity involved in sharing data features, which arises from the intricate process of multi-modal image registration; (2) the difficulty and high cost of acquiring accurate data labels. To address these issues, we propose a novel multi-modal medical dataset distillation method based on contrastive learning. We consider each modality as a unique approach to data augmentation for the corresponding imaging region, offering a unique perspective on the same anatomical region, which is then fused to collectively contribute to a comprehensive global representation of the medical image. Validation results on two datasets indicate our method is capable of reducing the original dataset to just 5% of its original size while retaining classification performance comparable to that of the full dataset, and notably, this efficiency is achieved using only 30% of the total labels, which significantly lowers the cost and effort associated with dataset labeling.

SIGKD: A Structured Instance Graph Distillation Method for Efficient Object Detection in Remote Sensing Images

In the realm of remote sensing image object detection compression distillation, establishing an efficient method for instance feature knowledge transfer between teacher and student models holds paramount importance. To this end, this paper introduces an innovative deep structured instance graph distillation method that endeavors to delve into the underlying information between instance features, thereby optimizing detection performance. Specifically, our proposed method incorporates feature instances and their relations into a graph-based structure (SIG). In this graph, feature instances serve as the nodes, while the relations between them serve as the edges. This structure enables us to capture both the individual significance of each feature instance and their collective influence within the context. Furthermore, in the experiment, we found that the index of some dense and small-target objects did not improve much because the edge assembly generated by a large number of background feature nodes in the SIG module inhibited the loss. To address the perennial imbalance between foreground and background features, we introduce an adaptive background feature mining strategy. Through carefully calibrated weights, this strategy effectively extracts and integrates background information, thereby minimizing noise interference in detection results and augmenting the expressive capacity of foreground features. We achieved state-of-the-art results on both the challenging DIOR and DOTA datasets, with the two-stage Oriented RCNN-based student Resnet18 model achieving a 73.23 mAP on the DOTA benchmark, close to the teacher Resnet101’s 76.16. In addition, on the DIOR dataset, the student Resnet18 based on the two-stage Faster RCNN achieved 70.13 mAP, higher than the baseline 66.31, and the student Resnet50 achieved 72.28, higher than the teacher’s 72.25.

Investigation of the behavior of manganese, iron, cobalt, and nickel at precipitate formation due to oxide addition in molten chlorides

Spent nuclear fuels are generated with the operation of nuclear power generation. The process of recovering nuclear fuel materials from spent nuclear fuels is called reprocessing, and through this reprocessing, nuclear fuel materials are recycled as nuclear fuel. The main reprocessing methods are wet reprocessing and pyro reprocessing. Molten salt electrolysis is used for the pyro reprocessing methods. In pyro reprocessing methods, salt bath is used repeatedly, as a result, ultimately spent salt containing a small amount of nuclear fuel materials is generated. In terms of nuclear material management, the nuclear fuel materials must be isolated and recovered. Therefore, we propose a nuclear fuel material recovery process that combines a precipitation method and a distillation method. However, it has been suggested that spent salt is contaminated by other radioactive materials derived from nuclear reactors, fuel structural materials, and molten salt electrolyzers used in pyro reprocessing. In this study, the behavior of these radioactive substances during precipitate formation was estimated.LiCl–KCl eutectic salt or NaCl–2CsCl salt was placed in a quartz tube in an Ar circulation glove box (GB). 10 wt% of MnCl2 or CoCl2 was added to the salt bath. The precipitant Li2O was added at stoichiometric amounts of 50 %, 100 %, 150 %, and 200 % relative to the amount of Mn(II) or Co(II), respectively. The quartz tube was placed in an electric furnace inside the GB and heated at 700 ℃ in the LiCl–KCl bath and 800 ℃ in the NaCl–2CsCl bath to melt the sample. After the sample was naturally cooled and solidified in the quartz tube, the precipitate and supernatant salt were collected. The precipitate was crushed, washed with pure water, and mixed with BN after drying. The mixture was formed into pellets and subjected to XAFS measurement. XAFS measurements were performed with KEK PF BL-27B, AichiSR BL5S1, and SPring-8 BL22XU. A portion of the supernatant salt was collected, dissolved in pure water, and analyzed using ICP-OES and AAS. The amount of Mn(II) or Co(II) contained and the constituent elements of the salt bath was determined from the analysis results, and the precipitation ratio was calculated. In addition, the reactions assumed in the experiment were evaluated by simulating the precipitation ratio using the thermodynamic database MALT and the multi-element chemical equilibrium calculation software gem.The precipitation ratio is shown in Figure 1 (left). The precipitation ratio of Mn(II) and Co(II) tended to increase as the amount of precipitant added increased. Both Mn(II) and Co(II) tended to increase up to 150 %, but there was no significant difference between 150 and 200 %. It is thought that the chloride produced an oxide precipitate by adding the precipitant. Theoretically, the precipitation ratio would increase to around 100 % if the amount of precipitant added was 100 %, but it was less than 80 % for both Mn(II) and Co(II). There are two possible reasons. One is that O ions derived from Li2O have the property of dissolving in salt bath, so there is a possibility that some of the precipitate will be dissolved in the salt bath. The other reason is that the total amount of Li2O added did not react with Mn(II) and Co(II), and a portion of it was dissolved in the salt bath. Furthermore, between Mn(II) and Co(II), the precipitation ratio of Co(II) tended to be higher. This is thought to be due to the fact that Co(II) reacts more easily with the precipitant than Mn(II). Therefore, in a mixed system, it is assumed that Co(II) precipitates first. Figure 1 (right) is the EXAFS radial structure functions of the Co(II) precipitate and the control sample. In the EXAFS radial structure function, not only the first neighborhood structural peak of the Co-O bond at approximately 1.8 Å coincided with the precipitate, but also the correlations at longer distances showed almost the same behavior. From this, it is assumed that the precipitate is CoO. In the presentation, we also plan to explain Fe(III) and Ni(II).As a result of the experiment, it was found that Mn(II) and Co(II) were recovered as oxides. Since the goal is to recover nuclear fuel materials as oxides, Mn(II) and Co(II) are likely to be entrained in the nuclear fuel materials. Therefore, we believe that it is necessary to add a process to separate Mn(II) and Co(II) from nuclear fuel materials. Figure 1