Generative Tasks Research Articles

Aligning Large Language Models for Enhancing Psychiatric Interviews Through Symptom Delineation and Summarization: Pilot Study.

Recent advancements in large language models (LLMs) have accelerated their use across various domains. Psychiatric interviews, which are goal-oriented and structured, represent a significantly underexplored area where LLMs can provide substantial value. In this study, we explore the application of LLMs to enhance psychiatric interviews by analyzing counseling data from North Korean defectors who have experienced traumatic events and mental health issues. This study aims to investigate whether LLMs can (1) delineate parts of the conversation that suggest psychiatric symptoms and identify those symptoms, and (2) summarize stressors and symptoms based on the interview dialogue transcript. Given the interview transcripts, we align the LLMs to perform 3 tasks: (1) extracting stressors from the transcripts, (2) delineating symptoms and their indicative sections, and (3) summarizing the patients based on the extracted stressors and symptoms. These 3 tasks address the 2 objectives, where delineating symptoms is based on the output from the second task, and generating the summary of the interview incorporates the outputs from all 3 tasks. In this context, the transcript data were labeled by mental health experts for the training and evaluation of the LLMs. First, we present the performance of LLMs in estimating (1) the transcript sections related to psychiatric symptoms and (2) the names of the corresponding symptoms. In the zero-shot inference setting using the GPT-4 Turbo model, 73 out of 102 transcript segments demonstrated a recall mid-token distance d<20 for estimating the sections associated with the symptoms. For evaluating the names of the corresponding symptoms, the fine-tuning method demonstrates a performance advantage over the zero-shot inference setting of the GPT-4 Turbo model. On average, the fine-tuning method achieves an accuracy of 0.82, a precision of 0.83, a recall of 0.82, and an F1-score of 0.82. Second, the transcripts are used to generate summaries for each interviewee using LLMs. This generative task was evaluated using metrics such as Generative Evaluation (G-Eval) and Bidirectional Encoder Representations from Transformers Score (BERTScore). The summaries generated by the GPT-4 Turbo model, utilizing both symptom and stressor information, achieve high average G-Eval scores: coherence of 4.66, consistency of 4.73, fluency of 2.16, and relevance of 4.67. Furthermore, it is noted that the use of retrieval-augmented generation did not lead to a significant improvement in performance. LLMs, using either (1) appropriate prompting techniques or (2) fine-tuning methods with data labeled by mental health experts, achieved an accuracy of over 0.8 for the symptom delineation task when measured across all segments in the transcript. Additionally, they attained a G-Eval score of over 4.6 for coherence in the summarization task. This research contributes to the emerging field of applying LLMs in psychiatric interviews and demonstrates their potential effectiveness in assisting mental health practitioners.

Feature Selection as Deep Sequential Generative Learning

Feature selection aims to identify the most pattern-discriminative feature subset. In prior literature, filter (e.g., backward elimination) and embedded (e.g., LASSO) methods have hyperparameters (e.g., top- k , score thresholding) and tie to specific models, thus, hard to generalize; wrapper methods search a feature subset in a huge discrete space and is computationally costly. To transform the way of feature selection, we regard a selected feature subset as a selection decision token sequence and reformulate feature selection as a deep sequential generative learning task that distills feature knowledge and generates decision sequences. Our method includes three steps: (1) We develop a deep variational transformer model over a joint of sequential reconstruction, variational, and performance evaluator losses. Our model can distill feature selection knowledge and learn a continuous embedding space to map feature selection decision sequences into embedding vectors associated with utility scores. (2) We leverage the trained feature subset utility evaluator as a gradient provider to guide the identification of the optimal feature subset embedding; (3) We decode the optimal feature subset embedding to autoregressively generate the best feature selection decision sequence with autostop. Extensive experimental results show this generative perspective is effective and generic, without large discrete search space and expert-specific hyperparameters. The code is available at http://tinyurl.com/FSDSGL .

Graph-based diffusion model for fast shower generation in calorimeters with irregular geometry

Denoising diffusion models have gained prominence in various generative tasks, prompting their exploration for the generation of calorimeter responses. Given the computational challenges posed by detector simulations in high-energy physics experiments, the necessity to explore new machine-learning-based approaches is evident. This study introduces a novel graph-based diffusion model designed specifically for rapid calorimeter simulations. The methodology is particularly well-suited for low-granularity detectors featuring irregular geometries. We apply this model to the ATLAS dataset published in the context of the Fast Calorimeter Simulation Challenge 2022, marking the first application of a graph diffusion model in the field of particle physics. Published by the American Physical Society 2024

Reinforcement Learning for Improving Chemical Reaction Performance.

Deep learning (DL) methods have gained notable prominence in predictive and generative tasks in molecular space. However, their application in chemical reactions remains grossly underutilized. Chemical reactions are intrinsically complex: typically involving multiple molecules besides bond-breaking/forming events. In reaction discovery, one aims to maximize yield and/or selectivity that depends on a number of factors, mostly centered on reacting partners and reaction conditions. Herein, we introduce RE-EXPLORE, a novel approach that integrates deep reinforcement learning (RL) with an RNN-based deep generative model to identify prospective new reactants/catalysts, whose yield/selectivity is estimated using a pretrained regressor. Three chemical databases (ChEMBL, ZINC, and COCONUT containing half a million to one million unlabeled molecules) are independently used for pretraining the generators to enrich them with valuable information from diverse chemical space. Standard RL methods are found to be insufficient, as learners tend to prioritize exploitation for immediate gains, resulting in repetitive generation of same/similar molecules. Our engineered reward function includes a Tanimoto-based uniqueness factor within the RL loop that improved the exploration of the environment and has helped accrue larger returns. Integration of a user-defined core fragment into the generated molecules facilitated learning of specific reaction types. Together, RE-EXPLORE can navigate the reaction space toward practically meaningful regions and offers notable improvements across the three distinct reaction types considered in this study. It identifies high-yielding substrates and highly enantioselective chiral catalysts. This RL-based approach has the potential to expedite reaction discovery and aid in the synthesis planning of important compounds, including drugs and pharmaceuticals.

Generative learning-based lightweight MRI brain tumor segmentation with missing modalities

Accurate segmentation of brain tumors across multiple MRI modalities is crucial for clinical diagnosis and prognosis. However, encountering missing modalities due to practical factors is common. Recent years have witnessed the emergence of various approaches, including 3D convolutional and transformer-based models, showcasing impressive performance in addressing this challenge. Nonetheless, these methods often suffer from high GPU memory requirements during inference, limiting their applicability. To explore alternative solutions, we turned to 2D-convolution-based techniques. Generative learning-based methods have produced visually appealing reconstructed MRI images, yet few consider the gap between visualization and segmentation. Drawing inspiration from clinicians who leverage cross-modality information for tumor estimation, we adopt a generative modality reconstruction task for pre-training, followed by fine-tuning for segmentation on MRI data with missing modalities. Furthermore, our investigation revealed common feature representations shared between reconstruction and segmentation tasks in latent space, as demonstrated through feature distributions. By utilizing a multi-encoder Unet architecture with fine-tuned adjustments, our method not only attains competitive performance in segmentation but also markedly decreases GPU memory usage during inference.

Denoising diffusion probabilistic models for generative alloy design

Inverse material design is an extremely challenging optimization task made difficult by, in part, the highly nonlinear relationship linking performance with composition. Quantitative approaches have improved significantly owing to advances in high throughput experimentation and computational thermodynamics. However, existing physics-based tools are mostly forward models; input a chemistry and obtain a prediction. More recently the materials community has leveraged advances in the machine learning community to establish novel inverse design frameworks. Very recently denoising diffusion probabilistic models have been shown to be extremely powerful generators producing synthetic data of various modalities e.g. images, text, audio, tables, etc.. In this work a novel framework for alloy design and optimization is proposed leveraging these class of models. Five key generative tasks are demonstrated (1) unconditional generation (2) composition conditioned generation (3) property conditioned generation (4) multi-feedstock conditioned generation and (5) generative optimization. These methods were tested on three case studies: high entropy alloy design, superalloy binder jet additive manufacturing, and in-situ dual-feedstock wire-arc additive manufacturing. Results indicate that the established models are extremely flexible, expressive, and robust. The architecture’s flexibility and training procedure empower the model to learn complex intra-compositional and composition-property relationships. Furthermore, the probabilistic nature of these models makes them well suited for addressing solution non-uniqueness and tackling uncertainty quantification tasks. While the fidelity and quantity of the underlying training data is paramount, we envision that future alloy design frameworks will make extensive use of these kinds of machine learning models as “search” tools bolstering the utility of experimental and computational approaches.

Prediction of Turbulent Boundary Layer Flow Dynamics with Transformers

Time-marching of turbulent flow fields is computationally expensive using traditional Computational Fluid Dynamics (CFD) solvers. Machine Learning (ML) techniques can be used as an acceleration strategy to offload a few time-marching steps of a CFD solver. In this study, the Transformer (TR) architecture, which has been widely used in the Natural Language Processing (NLP) community for prediction and generative tasks, is utilized to predict future velocity flow fields in an actuated Turbulent Boundary Layer (TBL) flow. A unique data pre-processing step is proposed to reduce the dimensionality of the velocity fields, allowing the processing of full velocity fields of the actuated TBL flow while taking advantage of distributed training in a High Performance Computing (HPC) environment. The trained model is tested at various prediction times using the Dynamic Mode Decomposition (DMD) method. It is found that under five future prediction time steps with the TR, the model is able to achieve a relative Frobenius norm error of less than 5%, compared to fields predicted with a Large Eddy Simulation (LES). Finally, a computational study shows that the TR achieves a significant speed-up, offering computational savings approximately 53 times greater than those of the baseline LES solver. This study demonstrates one of the first applications of TRs on actuated TBL flow intended towards reducing the computational effort of time-marching. The application of this model is envisioned in a coupled manner with the LES solver to provide few time-marching steps, which will accelerate the overall computational process.

Dynamic MRI interpolation in temporal direction using an unsupervised generative model

PurposeCardiac cine magnetic resonance imaging (MRI) is an important tool in assessing dynamic heart function. However, this technique requires long acquisition time and long breath holds, which presents difficulties. The aim of this study is to propose an unsupervised neural network framework that can perform cardiac cine interpolation in time, so that we can increase the temporal resolution of cardiac cine without increasing acquisition time. MethodsIn this study, a subject-specific unsupervised generative neural network is designed to perform temporal interpolation for cardiac cine MRI. The network takes in a 2D latent vector in which each element corresponds to one cardiac phase in the cardiac cycle and then the network outputs the cardiac cine images which are acquired on the scanner. After the training of the generative network, we can interpolate the 2D latent vector and input the interpolated latent vector into the network and the network will output the frame-interpolated cine images. The results of the proposed cine interpolation neural network (CINN) framework are compared quantitatively and qualitatively with other state-of-the-art methods, the ground truth training cine frames, and the ground truth frames removed from the original acquisition. Signal-to-noise ratio (SNR), structural similarity index measures (SSIM), peak signal-to-noise ratio (PSNR), strain analysis, as well as the sharpness calculated using the Tenengrad algorithm were used for image quality assessment. ResultsAs shown quantitatively and qualitatively, the proposed framework learns the generative task well and hence performs the temporal interpolation task well. Furthermore, both quantitative and qualitative comparison studies show the effectiveness of the proposed framework in cardiac cine interpolation in time. ConclusionThe proposed generative model can effectively learn the generative task and perform high quality cardiac cine interpolation in time.

What can attribution methods show us about chemical language models?

Language models trained on molecular string representations have shown strong performance in predictive and generative tasks. However, practical applications require not only making accurate predictions, but also explainability - the ability to explain the reasons and rationale behind the predictions. In this work, we explore explainability for a chemical language model by adapting a transformer-specific and a model-agnostic input attribution technique. We fine-tune a pretrained model to predict aqueous solubility, compare training and architecture variants, and evaluate visualizations of attributed relevance. The model-agnostic SHAP technique provides sensible attributions, highlighting the positive influence of individual electronegative atoms, but does not explain the model in terms of functional groups or explain how the model represents molecular strings internally to make predictions. In contrast, the adapted transformer-specific explainability technique produces sparse attributions, which cannot be directly attributed to functional groups relevant to solubility. Instead, the attributions are more characteristic of how the model maps molecular strings to its latent space, which seems to represent features relevant to molecular similarity rather than functional groups. These findings provide insight into the representations underpinning chemical language models, which we propose may be leveraged for the design of informative chemical spaces for training more accurate, advanced and explainable models.

A novel prompting method for few-shot NER via LLMs

In various natural language processing tasks, significant strides have been made by Large Language Models (LLMs). Researchers leverage prompt method to conduct LLMs in accomplishing specific tasks under few-shot conditions. However, the prevalent use of LLMs’ prompt methods mainly focuses on guiding generative tasks, and employing existing prompts may result in poor performance in Named Entity Recognition (NER) tasks. To tackle this challenge, we propose a novel prompting method for few-shot NER. By enhancing existing prompt methods, we devise a standardized prompts tailored for the utilization of LLMs in NER tasks. Specifically, we structure the prompts into three components: task definition, few-shot demonstration, and output format. The task definition conducts LLMs in performing NER tasks, few-shot demonstration assists LLMs in understanding NER task objectives through specific output demonstration, and output format restricts LLMs’ output to prevent the generation of unnecessary results. The content of these components has been specifically tailored for NER tasks. Moreover, for the few-shot demonstration within the prompts, we propose a selection strategy that utilizes feedback from LLMs’ outputs to identify more suitable few-shot demonstration as prompts. Additionally, to enhance entity recognition performance, we enrich the prompts by summarizing error examples from the output process of LLMs and integrating them as additional prompts.

A general supply-inspect cost framework to regulate the reliability-usability trade-offs for few-shot inference

Language models and other recent machine learning paradigms blur the distinction between generative and discriminative tasks, in a continuum that is regulated by the degree of pre- and post-supervision that is required from users, as well as the tolerated level of error. In few-shot inference, we need to find a trade-off between the number and cost of the solved examples that have to be supplied, those that have to be inspected (some of them accurate but others needing correction) and those that are wrong but pass undetected. In this paper, we define a new Supply-Inspect Cost Framework, associated graphical representations and comprehensive metrics that consider all these elements. To optimise few-shot inference under specific operating conditions, we introduce novel algorithms that go beyond the concept of rejection rules in both static and dynamic contexts. We illustrate the effectiveness of all these elements for a transformative domain, data wrangling, for which language models can have a huge impact if we are able to properly regulate the reliability-usability trade-off, as we do in this paper.

How does virtual peer presence relate to learning from video lectures and subsequent explanation generation? The moderated mediating roles of motivation and explanation characteristics

Finding effective ways to improve students’ learning from video lectures will not only improve online education efficacy, it will also play an important role in the digital transformation of education. Generating written explanations has shown some benefits for learning, and peer presence may be an implementation-related boundary condition. This study tested the impacts of virtual peer presence (presence vs. absence) and generative tasks (generating explanations vs. no generation) on students’ learning in a video-based learning context. Students’ attention, meta-comprehension accuracy, motivation, cognitive load, learning achievements, and explanation characteristics were measured. Results showed some benefits of having a virtual peer, and of generating written explanations on students’ learning. Moreover, mediating analyses revealed the mediating role of motivation between peer presence and learning achievement, and moderated mediating analyses revealed the moderating role of explanation characteristics between peer presence and motivation. Our findings not only extend the understanding of the positive effects of generating written explanations in video-based learning contexts, they also provide implications for improving students’ learning from video lectures as well as optimizing online video lectures.

Generative innovations for paleography: enhancing character image synthesis through unconditional single image models

Data scarcity in paleographic image datasets poses a significant challenge to researchers and scholars in the field. Unlike modern printed texts, historical manuscripts and documents are often scarce and fragile, making them difficult to digitize and create comprehensive datasets. Recently many innovations have been arrived on single image generative models for natural images but none of them are focused on paleographic character images and other handwritten datasets. In paleographic images like stone inscription characters, maintaining exact shape and structure of character is important unlike natural images. In this paper we propose an unconditional single image generative model, CharGAN for isolated paleographic character images. In the proposed system, augmented images are generated from a single image using generative adversarial networks, while maintaining their structure. Specifically, an external augmentation inducer is used to create higher-level augmentations in the generated images. In addition, the input to the generator is replaced with dynamic sampling from a Gaussian mixture model to make changes to the low-level features. From our experimental results, we infer that these two enhancements make single-image generative models suitable not only for natural images, but also for paleographic character images and other handwritten character datasets, the AHCD dataset, and EMNIST, where the global structure is important. Both the qualitative and quantitative results show that our approach is effective and superior in single-image generative tasks, particularly in isolated character image generation.

Applied artificial intelligence in dentistry: emerging data modalities and modeling approaches.

Artificial intelligence (AI) is increasingly applied across all disciplines of medicine, including dentistry. Oral health research is experiencing a rapidly increasing use of machine learning (ML), the branch of AI that identifies inherent patterns in data similarly to how humans learn. In contemporary clinical dentistry, ML supports computer-aided diagnostics, risk stratification, individual risk prediction, and decision support to ultimately improve clinical oral health care efficiency, outcomes, and reduce disparities. Further, ML is progressively used in dental and oral health research, from basic and translational science to clinical investigations. With an ML perspective, this review provides a comprehensive overview of how dental medicine leverages AI for diagnostic, prognostic, and generative tasks. The spectrum of available data modalities in dentistry and their compatibility with various methods of applied AI are presented. Finally, current challenges and limitations as well as future possibilities and considerations for AI application in dental medicine are summarized.

Efficient distributed continual learning for steering experiments in real-time

Deep learning has emerged as a powerful method for extracting valuable information from large volumes of data. However, when new training data arrives continuously (i.e., is not fully available from the beginning), incremental training suffers from catastrophic forgetting (i.e., new patterns are reinforced at the expense of previously acquired knowledge). Training from scratch each time new training data becomes available would result in extremely long training times and massive data accumulation. Rehearsal-based continual learning has shown promise for addressing the catastrophic forgetting challenge, but research to date has not addressed performance and scalability. To fill this gap, we propose an approach based on a distributed rehearsal buffer that efficiently complements data-parallel training on multiple GPUs to achieve high accuracy, short runtime, and scalability. It leverages a set of buffers (local to each GPU) and uses several asynchronous techniques for updating these local buffers in an embarrassingly parallel fashion, all while handling the communication overheads necessary to augment input minibatches using unbiased, global sampling. We further propose a generalization of rehearsal buffers to support both classification and generative learning tasks, as well as more advanced rehearsal strategies (notably Dark Experience Replay, leveraging knowledge distillation). We illustrate this approach with a real-life HPC streaming application from the domain of ptychographic image reconstruction. We run extensive experiments on up to 128 GPUs of the ThetaGPU supercomputer to compare our approach with baselines representative of training-from-scratch (the upper bound in terms of accuracy) and incremental training (the lower bound). Results show that rehearsal-based continual learning achieves a top-5 validation accuracy close to the upper bound, while simultaneously exhibiting a runtime close to the lower bound.

SCGAN: Semi-Centralized Generative Adversarial Network for image generation in distributed scenes

Distributed Generative Adversarial Networks (GANs) are pivotal for generative tasks in distributed data environments. Conventional architectures, whether centralized or decentralized, inadequately balance global and local data distribution integration. We propose the Semi-Centralized Distributed GAN (SCGAN), a hybrid framework combining the merits of both approaches. SCGAN employs dual generators — one decentralized, one centralized — and a feature extractor that optimizes their outputs by dynamically calculating feature distances, thereby achieving a balanced semi-centralized generator. Extensive experiments on five benchmark image datasets reveal that SCGAN surpasses existing models, delivering superior Fréchet Inception Distance (FID) scores and generating highly realistic images. Moreover, SCGAN’s robustness and versatility are demonstrated through its efficacy in downstream applications, including image augmentation and distributed privacy protection. Notably, SCGAN’s generated images were evaluated using a distributed classifier, a quality metric (IS), and human observation, consistently outperforming existing methods. This study highlights SCGAN’s significant advancements in distributed generative modeling and its nuanced approach to managing complex data distributions, particularly excelling in privacy-preserving scenarios.

Target-Guided Diffusion Models for Unpaired Cross-Modality Medical Image Translation.

In a clinical setting, the acquisition of certain medical image modality is often unavailable due to various considerations such as cost, radiation, etc. Therefore, unpaired cross-modality translation techniques, which involve training on the unpaired data and synthesizing the target modality with the guidance of the acquired source modality, are of great interest. Previous methods for synthesizing target medical images are to establish one-shot mapping through generative adversarial networks (GANs). As promising alternatives to GANs, diffusion models have recently received wide interests in generative tasks. In this paper, we propose a target-guided diffusion model (TGDM) for unpaired cross-modality medical image translation. For training, to encourage our diffusion model to learn more visual concepts, we adopted a perception prioritized weight scheme (P2W) to the training objectives. For sampling, a pre-trained classifier is adopted in the reverse process to relieve modality-specific remnants from source data. Experiments on both brain MRI-CT and prostate MRI-US datasets demonstrate that the proposed method achieves a visually realistic result that mimics a vivid anatomical section of the target organ. In addition, we have also conducted a subjective assessment based on the synthesized samples to further validate the clinical value of TGDM.

Generative preparation tasks in digital collaborative learning: actor and partner effects of constructive preparation activities on deep comprehension.

Deep learning from collaboration occurs if the learner enacts interactive activities in the sense of leveraging the knowledge externalized by co-learners as resource for own inferencing processes and if these interactive activities in turn promote the learner's deep comprehension outcomes. This experimental study investigates whether inducing dyad members to enact constructive preparation activities can promote deep learning from subsequent collaboration while examining prior knowledge as moderator. In a digital collaborative learning environment, 122 non-expert university students assigned to 61 dyads studied a text about the human circulatory system and then prepared individually for collaboration according to their experimental conditions: the preparation tasks varied across dyads with respect to their generativity, that is, the degree to which they required the learners to enact constructive activities (note-taking, compare-contrast, or explanation). After externalizing their answer to the task, learners in all conditions inspected their partner's externalization and then jointly discussed their text understanding via chat. Results showed that more rather than less generative tasks fostered constructive preparation but not interactive collaboration activities or deep comprehension outcomes. Moderated mediation analyses considering actor and partner effects indicated the indirect effects of constructive preparation activities on deep comprehension outcomes via interactive activities to depend on prior knowledge: when own prior knowledge was relatively low, self-performed but not partner-performed constructive preparation activities were beneficial. When own prior knowledge was relatively high, partner-performed constructive preparation activities were conducive while one's own were ineffective or even detrimental. Given these differential effects, suggestions are made for optimizing the instructional design around generative preparation tasks to streamline the effectiveness of constructive preparation activities for deep learning from digital collaboration.

DBEF-Net: Diffusion-Based Boundary-Enhanced Fusion Network for medical image segmentation

Medical image segmentation aims to locate lesions within a given image to assist doctors in diagnosis and treatment, playing a crucial role in improving patient outcomes. Recently, the Denoising Diffusion Probabilistic Model (DDPM) has demonstrated its advantages in various generative tasks. Building upon these advantages, we propose DBEF-Net: Diffusion-Based Boundary-Enhanced Fusion Network for Medical Image Segmentation. When applying diffusion models to image segmentation, the challenge of inconsistent semantic features and noise embedding needs to be addressed. To overcome this issue, we introduce the Group Attention Fusion Module (GAFM), which merges image features and noise features in groups to better utilize semantic information and noise characteristics in the diffusion model. Additionally, we design the Boundary-Attentive Fusion Module (BAFM) to incorporate boundary priors into the diffusion model to enhance sensitivity to fuzzy boundaries. We also introduce the Spatial Context Fusion Module (SCFM) to fully exploit multi-scale information in the encoder and decoder. We conduct experiments on four datasets with DBEF-Net, comparing it with other methods. Encouragingly, our approach shows significant performance improvement while maintaining novelty.

Exploring Alternative Approaches to Language Modeling for Learning from Data and Knowledge

Despite their extensive application in language understanding tasks, large language models (LLMs) still encounter challenges including hallucinations - occasional fabrication of information - and alignment issues - lack of associations with human-curated world models (e.g., intuitive physics or common-sense knowledge). Moreover, the black-box nature of LLMs presents significant obstacles in training them effectively to achieve desired behaviors. In particular, modifying the concept embedding spaces of LLMs can be highly intractable. This process involves analyzing the implicit impact of such adjustments on the myriad parameters within LLMs and the resulting inductive biases. We propose a novel architecture that wraps powerful function approximation architectures within an outer, interpretable read-out layer. This read-out layer can be scrutinized to explicitly observe the effects of concept modeling during the training of the LLM. Our method stands in contrast with gradient-based implicit mechanisms, which depend solely on adjustments to the LLM parameters and thus evade scrutiny. By conducting extensive experiments across both generative and discriminative language modeling tasks, we evaluate the capabilities of our proposed architecture relative to state-of-the-art LLMs of similar sizes. Additionally, we offer a qualitative examination of the interpretable read-out layer and visualize the concepts it captures. The results demonstrate the potential of our approach for effectively controlling LLM hallucinations and enhancing the alignment with human expectations.