Deep learning for psychiatric genomics: from tools to applications.

Artificial intelligence (AI), particularly deep learning, has demonstrated remarkable performance in medical imaging across a variety of modalities, including X-ray, computed tomography (CT), magnetic resonance imaging (MRI), ultrasound, positron emission tomography (PET), and pathological imaging. However, most existing state-of-the-art AI techniques are task-specific and focus on a limited range of imaging modalities. Compared to these task-specific models, emerging foundation models represent a significant milestone in AI development. These models can learn generalized representations of medical images and apply them to downstream tasks through zero-shot or few-shot fine-tuning. Foundation models have the potential to address the comprehensive and multifactorial challenges encountered in clinical practice. This article reviews the clinical applications of both task-specific and foundation models, highlighting their differences, complementarities, and clinical relevance. We also examine their future research directions and potential challenges. Unlike the replacement relationship seen between deep learning and traditional machine learning, task-specific and foundation models are complementary, despite inherent differences. While foundation models primarily focus on segmentation and classification, task-specific models are integrated into nearly all medical image analyses. However, with further advancements, foundation models could be applied to other clinical scenarios. In conclusion, all indications suggest that task-specific and foundation models, especially the latter, have the potential to drive breakthroughs in medical imaging, from image processing to clinical workflows.

Harmonizing Return of Results Policies In International Psychiatric Genomics Research Collaborations

Recent advancements in deep learning, particularly large language models (LLMs), made a significant impact on how researchers study microbiome and metagenomics data. Microbial protein and genomic sequences, like natural languages, form a language of life, enabling the adoption of LLMs to extract useful insights from complex microbial ecologies. In this paper, we review applications of deep learning and language models in analyzing microbiome and metagenomics data. We focus on problem formulations, necessary datasets, and the integration of language modeling techniques. We provide an extensive overview of protein/genomic language modeling and their contributions to microbiome studies. We also discuss applications such as novel viromics language modeling, biosynthetic gene cluster prediction, and knowledge integration for metagenomics studies.

Pre-trained language models have transformed the field of natural language processing (NLP), and their success has inspired efforts in genomics to develop domain-specific foundation models (FMs). However, creating high-quality genomic FMs from scratch is resource-intensive, requiring significant computational power and high-quality pre-training data. The success of large language models (LLMs) in NLP has largely been driven by industrial-scale efforts leveraging vast, diverse corpora and massive computing infrastructure. In this work, we aim to bypass the data and computational bottlenecks of creating genomic FMs from scratch and instead propose repurposing existing LLMs for genomics tasks. Inspired by the recently observed ‘cross-modal transfer’ phenomenon – where transformers pre-trained on natural language can generalize to other modalities – we introduce L2G, which adapts a pre-trained LLM architecture for genomics using neural architecture search (NAS) and a novel three-stage training procedure. Remarkably, without requiring extensive pre-training on DNA sequence data, L2G achieves superior performance to fine-tuned genomic FMs and task-specific models on more than half of tasks across multiple genomics benchmarks. In an enhancer activity prediction task, L2G further demonstrates its capacity to identify significant transcription factor motifs. Our work not only highlights the generalizability and efficacy of language models in out-of-domain tasks such as genomics, but also opens new avenues for more efficient and less resource-intensive methodologies in genomic research.

Extracting structured and semantically annotated materials information from unstructured scientific literature is a crucial step toward constructing machine-interpretable knowledge graphs and accelerating data-driven materials research. This is especially important in materials science, which is adversely affected by data scarcity. Data scarcity further motivates employing solutions such as foundation language models for extracting information which can in principle address several subtasks of the information extraction problem in a range of domains without the need of generating costly large-scale annotated datasets for each downstream task. However, foundation language models struggle with tasks like Named Entity Recognition (NER) due to domain-specific terminologies, fine-grained entities, and semantic ambiguity. The issue is even more pronounced when entities must map directly to pre-existing domain ontologies. This work aims to assess whether foundation large language models (LLMs) can successfully perform ontology-conformal NER in the materials mechanics and fatigue domain. Specifically, we present a comparative evaluation of in-context learning (ICL) with foundation models such as GPT-4 against fine-tuned task-specific language models, including MatSciBERT and DeBERTa. The study is performed on two materials fatigue datasets, which contain annotations at a comparatively fine-grained level adhering to the class definitions of a formal ontology to ensure semantic alignment and cross-dataset interoperability. Both datasets cover adjacent domains to assess how well both NER methodologies generalize when presented with typical domain shifts. Task-specific models are shown to significantly outperform general foundation models on an ontology-constrained NER. Our findings reveal a strong dependence on the quality of few-shot demonstrations in ICL to handle domain-shift. The study also highlights the significance of domain-specific pre-training by comparing task-specific models that differ primarily in their pre-training corpus.

Even though active learning forms an important pillar of machine learning, deep learning tools are not prevalent within it. Deep learning poses several difficulties when used in an active learning setting. First, active learning (AL) methods generally rely on being able to learn and update models from small amounts of data. Recent advances in deep learning, on the other hand, are notorious for their dependence on large amounts of data. Second, many AL acquisition functions rely on model uncertainty, yet deep learning methods rarely represent such model uncertainty. In this paper we combine recent advances in Bayesian deep learning into the active learning framework in a practical way. We develop an active learning framework for high dimensional data, a task which has been extremely challenging so far, with very sparse existing literature. Taking advantage of specialised models such as Bayesian convolutional neural networks, we demonstrate our active learning techniques with image data, obtaining a significant improvement on existing active learning approaches. We demonstrate this on both the MNIST dataset, as well as for skin cancer diagnosis from lesion images (ISIC2016 task).

The research groups in computer vision, graphics, and machine learning have dedicated a substantial amount of attention to the areas of 3D object reconstruction, augmentation, and registration. Deep learning is the predominant method used in artificial intelligence for addressing computer vision challenges. However, deep learning on three-dimensional data presents distinct obstacles and is now in its nascent phase. There have been significant advancements in deep learning specifically for three-dimensional data, offering a range of ways to address these issues. This study offers a comprehensive examination of the latest advancements in deep learning methodologies. We examine many benchmark models for the tasks of 3D object registration, augmentation, and reconstruction. We thoroughly analyse their architectures, advantages, and constraints. In summary, this report provides a comprehensive overview of recent advancements in three-dimensional deep learning and highlights unresolved research areas that will need to be addressed in the future.

With growing size of resting state fMRI datasets and advances in deep learning methods, there are ever increasing opportunities to leverage progress in deep learning to solve challenging tasks in neuroimaging. In this work, we build upon recent advances in deep metric learning, to learn embeddings of rs-fMRI data, which can then be potentially used for several downstream tasks. We propose an efficient training method for our model and compare our method with other widely used models. Our experimental results indicate that deep metric learning can be used as an additional refinement step to learn representations of fMRI data, that significantly improves performance on downstream modeling tasks.

A survey of multimodal hybrid deep learning for computer vision: Architectures, applications, trends, and challenges

In the last few year advances in deep learning and artificial intelligence have made it possible to generate high-quality text, audio, and visual content automatically for a wide range of application areas including research and education. However, designing and customizing an effective R&D tool capable of providing necessary tool-specific output, and breaking down complex research tasks requires a great deal of expertise and effort, and is often a time-consuming and expensive process. Using existing Generative Pre-trained Transformers (GPT) and foundational models, it is now possible to leverage the Large Language Model GPTs already trained on specific datasets to be effective in common research and development workflow. In this paper, we develop and test a customized version of autonomous pretrained generative transformer which is an experimental open-source project built on top of GPT-4 language model that chains together LLM "thoughts", to autonomously achieve and regress towards specifics goals. Our implementation, referred to as TAUCHI-GPT, which uses an automated approach to text generation that leverages deep learning and output reflection to create high-quality text, visual and auditory output, achieve common research and development tasks. TAUCHI-GPT is based on the GPT-4 architecture and connects to Stable Diffusion and ElevenLabs to input and output complex multimodal streams through chain prompting. Moreover, using the Google Search API, TAUCHI-GPT can also scrap online repositories to understand, learn and deconstruct complex research tasks, identify relevant information, and plan appropriate courses of action by implementing a chain of thought (CoT).

Recent advancements in deep learning have introduced new opportunities for quality management in manufacturing, particularly through transformer-based architectures capable of learning from limited datasets and handling complex, multimodal inputs. Among these, Large Language Models (LLMs) have emerged as a significant innovation, demonstrating strong capabilities in forecasting and representing the cutting edge of artificial intelligence (AI). Through transfer learning, LLMs effectively process and generate extended text sequences, and recent developments show their potential for multimodal integration, including text, images, audio, and video data. Quality management is a critical area for industrial innovation, rapidly evolving as manufacturers seek to close the quality-manufacturing loop and achieve zero-defect production goals. While computer vision techniques based on deep learning have been widely implemented for visual inspection tasks, integrating multiple heterogeneous data sources offers the possibility for even greater improvements. Despite the success of LLMs in language tasks, their application to time series data remains relatively unexplored. Alternative statistical approaches and deep learning models have proven effective for time series forecasting. Nevertheless, LLMs could provide additional advantages in industrial contexts, offering opportunities to enhance in-line quality control, defect prevention, and predictive discarding strategies across various sectors. This paper investigates the potential of applying LLMs to time series analysis by comparing the performance of an LLM (GPT-2), originally trained on textual data, with a model specifically designed for time series data (TimeGPT), and a more conventional transformer-based architecture. Our study includes a dedicated time series GPT model and a general-purpose LLM in a comparative evaluation. Through this analysis, we aim to better understand how language models can be effectively adapted to time series forecasting tasks and explore their transfer learning potential for enhancing quality management in manufacturing.

Psychiatric disorders have complex genetic architectures with substantial genetic overlap across conditions, which may partially explain their high levels of comorbidity. This presents significant challenges to research. Genome-wide association studies (GWASs) have uncovered hundreds of loci associated with single disorders, but the genetic landscape of psychiatric disorders has remained largely obscure. Moving beyond the conventional infinitesimal model, uni-, bi-, and trivariate MiXeR tools, applied to GWAS summary statistics, has enabled us to more comprehensively describe the genetic architecture of complex disorders and traits and their overlap. Furthermore, the GSA-MiXeR tool improves biological interpretation of GWAS findings to better elucidate causal mechanisms. Here, we outline the methodology that underlies the MiXeR tools together with instructions for their optimal use. We review results from studies that have investigated the genetic architecture of psychiatric disorders and their overlap using the MiXeR toolset. These studies have revealed generally high polygenicity and low discoverability among psychiatric disorders, particularly in contrast to somatic disorders. There is also pervasive genetic overlap across psychiatric disorders and behavioral traits, while their overlap with somatic traits is smaller, consistent with differences in polygenicity. Finally, GSA-MiXeR has quantified the contribution of gene sets to the heritability of psychiatric disorders, prioritizing small, biologically coherent gene sets. Together, these findings have implications for our understanding of the complex relationships between psychiatric disorders and related traits. MiXeR tools have provided new insights into the genetic architecture of psychiatric disorders, generating a better understanding of their underlying biological mechanisms and potential for clinical utility.

Defining the Genetic, Genomic, Cellular, and Diagnostic Architectures of Psychiatric Disorders

In the recent past, more than 5 years or so, Deep Learning (DL) especially the large language models (LLMs) has generated extensive studies out of a distinctly average downturn field of knowledge made up of a traditional society of researchers. As a result, DL is now so pervasive that its use is widespread across the body of research related to machine learning computing. The rapid emergence and apparent dominance of DL architectures over traditional machine learning techniques on a variety of tasks have been truly astonishing to witness. DL models outperformed in a variety of areas, including natural language processing (NLP), image analysis, language understanding, machine translation, computer vision, speech processing, audio recognition, style imitation, and computational biology. In this study, the aim is to explain the rudiments of DL, such as neural networks, convolutional neural networks, deep belief networks, and various variants of DL. The study will explore how these models have been applied to NLP and delve into the underlying mathematics behind them. Additionally, the study will investigate the latest advancements in DL and NLP, while acknowledging the key challenges and emerging trends in the field. Furthermore, it will discuss the core component of DL, namely embeddings, from a taxonomic perspective. Moreover, a literature review will be provided focusing on the application of DL models for six popular pattern recognition tasks: speech recognition, question answering, part of speech tagging, named entity recognition, text classification, and machine translation. Finally, the study will demystify state-of-the-art DL libraries/frameworks and available resources. The outcome and implication of this study reveal that LLMs face challenges in dealing with pragmatic aspects of language due to their reliance on statistical learning techniques and lack of genuine understanding of context, presupposition, implicature, and social norms. Furthermore, this study provides a comprehensive analysis of the current state-of-the-art advancements and highlights significant obstacles and emerging developments. The article has the potential to enhance readers’ understanding of the subject matter.

The genetic nature of psychiatric disorders was observed by clinicians long before DNA had been identified as the molecule of inheritance. The greatest identified risk factor of many psychiatric disorders still is a positive family history. Until recently this knowledge has not contributed substantially to treatment efforts or to a better understanding of the disease processes because we lacked the necessary genetic data. Advances in genotyping technologies have brought an end to this data shortage which is leading to a better understanding of the genetic architecture of psychiatric disorders. Two patterns started to emerge which were uncommon in earlier studied Mendelian disorders. (i) most of the genetic part of disease risk is conferred by a large number of genetic loci of small effect, and (ii) genetic loci often influence a large number of traits at the same time. While this is true of many traits (complex traits), these two phenomena (polygenicity and pleiotropy) are particularly pronounced in psychiatric disorders. This has wide-reaching consequences for the analysis and interpretation of genetic data and provides challenges as well as opportunities. This thesis focuses on two areas in particular: genetic heterogeneity and genetic risk prediction. Genetic heterogeneity in a phenotypically homogenous group describes a situation where different and distinct genetic risk profiles are causing similar symptoms in different people. This can be easily identified under a Mendelian inheritance pattern, but proves to be challenging under polygenicity. The presence of genetic heterogeneity can limit the accuracy of genetic risk prediction. The aim of genetic risk prediction is to use the information that has been gathered on the effects of genetic loci to estimate the genetic liability of an individual to develop a disease. Here, pleiotropy offers an opportunity to increase the accuracy of genetic prediction by leveraging information from multiple diseases at the same time. The aim in this thesis is to describe several projects which center around the two concepts of genetic risk prediction based on multiple traits and of genetic heterogeneity. Chapter 1 sets the scene with an overview of recently developed polygenic methods. Chapter 2 deals with the effects of genetic heterogeneity on heritability estimates and demonstrates how genetic heterogeneity might contribute to the phenomenon of missing heritability, which is 2 the discrepancy between twin study heritability estimates and the variance explained by the sum of individual genetic loci. Chapter 3 addresses the question of whether genotype clustering can detect groups with different genetic risk profiles. Chapter 4 describes the implementation of a multivariate extension to the univariate Best Linear Unbiased Prediction (BLUP) method and its application to five psychiatric traits. Chapters 4 and 5 investigate whether this multivariate BLUP model can be approximated when only summary statistics, not individual level genotype data, are available for the predicted traits. Theory is derived for such an approximation, which is then tested in a simulation setup and applied to two psychiatric disorders, as well as to a range of other traits. Finally, the discussion places the work into wider context and discusses the findings and limitations of each project, and highlights similarities and differences between the two prediction projects.