In-house Dataset Research Articles

Automatic Brain Midline Surface Delineation on 3D CT Images With Intracranial Hemorrhage.

Brain midline delineation plays an important role in guiding intracranial hemorrhage surgery, which still remains a challenging task since hemorrhage shifts the normal brain configuration. Most previous studies detected brain midline on 2D plane and did not handle hemorrhage cases well. We propose a novel and efficient hemisphere-segmentation framework (HSF) for 3D brain midline surface delineation. Specifically, we formulate the brain midline delineation as a 3D hemisphere segmentation task, and employ an edge detector and a smooth regularization loss to generate the midline surface. We also introduce a distance-weighted map to keep the attention on the midline. Furthermore, we adopt rectification learning to handle various head poses. Finally, considering the complex situation of ventricle break-in for hemorrhages in bilateral intraventricular (B-IVH) cases, we identify those cases via a classification model and design a midline correction strategy to locally adjust the midline. To our best knowledge, it is the first study focusing on delineating the brain midline surface on 3D CT images of hemorrhage patients and handling the situation of ventricle break-in. Extensive validation on our large in-house datasets (519 patients) and the public CQ500 dataset (491 patients), demonstrates that our method outperforms state-of-the-art methods on brain midline delineation.

Variants in HNRNPH1 are associated with high myopia in humans and ocular coloboma in zebrafish.

High myopia is one of the most common causes for blindness due to its associated complications. Genetic factor has been considered as the major cause for early-onset high myopia (eoHM), but exact genetic defects for most eoHM are yet to be identified. Through multi-step bioinformatics analysis of our in-house whole exome sequencing dataset from 6397 individuals, variants from 928 probands with eoHM were further compared with those from in-house controls as well as gnomAD database. The results showed that loss-of-function (LoF) variants in a novel gene HNRNPH1 were identified in two of 928 probands with eoHM but in none of 5469probands with other eye conditions (p=0.02). LoF variants in HNRNPH1 were extremely rare and intolerant, while two LoF variants in 928 eoHM were statistically higher than their frequency in gnomAD (p=5.98 × 10-4 ). These two LoF variants, c.2dup/p.(M1?) and c.121dup/p.(Q41Pfs*20), were absent from existing database. Variants in HNRNPH1 have not been associated with any inherited eye disease before. Expression of HNRNPH1 was enriched in ganglion cell layer and inner nuclear layer in humans. Knockdown of hnrnph1 in zebrafish resulted in ocular coloboma. All these suggest that HNRNPH1 is potential contribution to eoHM when mutated.

Robust Table Detection and Structure Recognition from Heterogeneous Document Images

We introduce a new table detection and structure recognition approach named RobusTabNet to detect the boundaries of tables and reconstruct the cellular structure of each table from heterogeneous document images. For table detection, we propose to use CornerNet as a new region proposal network to generate higher quality table proposals for Faster R-CNN, which has significantly improved the localization accuracy of Faster R-CNN for table detection. Consequently, our table detection approach achieves state-of-the-art performance on three public table detection benchmarks, namely cTDaR TrackA, PubLayNet and IIIT-AR-13K, by only using a lightweight ResNet-18 backbone network. Furthermore, we propose a new split-and-merge based table structure recognition approach, in which a novel spatial CNN based separation line prediction module is proposed to split each detected table into a grid of cells, and a Grid CNN based cell merging module is applied to recover the spanning cells. As the spatial CNN module can effectively propagate contextual information across the whole table image, our table structure recognizer can robustly recognize tables with large blank spaces and geometrically distorted (even curved) tables. Thanks to these two techniques, our table structure recognition approach achieves state-of-the-art performance on three public benchmarks, including SciTSR, PubTabNet and cTDaR TrackB2-Modern. Moreover, we have further demonstrated the advantages of our approach in recognizing tables with complex structures, large blank spaces, as well as geometrically distorted or even curved shapes on a more challenging in-house dataset.

An interactive dual-branch network for hard palate segmentation of the oral cavity from CBCT images

Automatic and accurate segmentation of the hard palate from Cone Beam and Computed Tomography (CBCT) images is a fundamental task for the insertion of orthodontic mini-implants. U-Net and its variants fail to handle it well when facing the hard palate with diversity of shape. Further, hard palate has low-density soft tissues and high-density hard bones so as to exhibit different intensity of regions in CBCT images, which may result in mis-segmentation or missing-segmentation. Motivated by these challenges, a novel Interactive Dual-Branch Network (IDBNet) is presented to achieve automatic and accurate segmentation of the hard palate in the oral cavity from CBCT images. Specifically, we introduce the designs for hard palate segmentation as follows: (1) Dual-branch encoder–decoder is developed for hard palate with the ability of data expansion and extracting multi-scale interactive features. (2) Channel map interaction module is proposed to reinforce the cross-channel information communication between different level features, thereby enhancing feature representations for variable hard palate. (3) Guide map interaction module is designed to model the similarities and differences of dual-branch hard palate boundaries. Owing to the remarkable designs, our IDBNet has the ability to gradually mine ambiguous hard palate affected by different intensity of regions. Moreover, we collected an in-house dataset with 30 challenging cases, which contained 2260 CBCT slices. Among them, 70% for training and 30% for testing are considered. Five-fold cross-validation experiments on this dataset demonstrate that our method outperforms the Res-UNet method by 8.05% of mean Dice score and 7.97% of structure measure. The utilization of our proposed segmentation approach for hard palate of the oral cavity warrants further investigation on implantable area, which could aid in formulating clinical plans on the insertion regions of palatal mini-implants.

SECANT: a biology-guided semi-supervised method for clustering, classification, and annotation of single-cell multi-omics.

The recent advance of single cell sequencing (scRNA-seq) technology such as Cellular Indexing of Transcriptomes and Epitopes by Sequencing (CITE-seq) allows researchers to quantify cell surface protein abundance and RNA expression simultaneously at single cell resolution. Although CITE-seq and other similar technologies have gained enormous popularity, novel methods for analyzing this type of single cell multi-omics data are in urgent need. A limited number of available tools utilize data-driven approach, which may undermine the biological importance of surface protein data. In this study, we developed SECANT, a biology-guided SEmi-supervised method for Clustering, classification, and ANnoTation of single-cell multi-omics. SECANT is used to analyze CITE-seq data, or jointly analyze CITE-seq and scRNA-seq data. The novelties of SECANT include (1) using confident cell type label identified from surface protein data as guidance for cell clustering, (2) providing general annotation of confident cell types for each cell cluster, (3) utilizing cells with uncertain or missing cell type label to increase performance, and (4) accurate prediction of confident cell types for scRNA-seq data. Besides, as a model-based approach, SECANT can quantify the uncertainty of the results through easily interpretable posterior probability, and our framework can be potentially extended to handle other types of multi-omics data. We successfully demonstrated the validity and advantages of SECANT via simulation studies and analysis of public and in-house datasets from multiple tissues. We believe this new method will be complementary to existing tools for characterizing novel cell types and make new biological discoveries using single-cell multi-omics data.

Towards Automated Brain Aneurysm Detection in TOF-MRA: Open Data, Weak Labels, and Anatomical Knowledge

Brain aneurysm detection in Time-Of-Flight Magnetic Resonance Angiography (TOF-MRA) has undergone drastic improvements with the advent of Deep Learning (DL). However, performances of supervised DL models heavily rely on the quantity of labeled samples, which are extremely costly to obtain. Here, we present a DL model for aneurysm detection that overcomes the issue with “weak” labels: oversized annotations which are considerably faster to create. Our weak labels resulted to be four times faster to generate than their voxel-wise counterparts. In addition, our model leverages prior anatomical knowledge by focusing only on plausible locations for aneurysm occurrence. We first train and evaluate our model through cross-validation on an in-house TOF-MRA dataset comprising 284 subjects (170 females / 127 healthy controls / 157 patients with 198 aneurysms). On this dataset, our best model achieved a sensitivity of 83%, with False Positive (FP) rate of 0.8 per patient. To assess model generalizability, we then participated in a challenge for aneurysm detection with TOF-MRA data (93 patients, 20 controls, 125 aneurysms). On the public challenge, sensitivity was 68% (FP rate = 2.5), ranking 4th/18 on the open leaderboard. We found no significant difference in sensitivity between aneurysm risk-of-rupture groups (p = 0.75), locations (p = 0.72), or sizes (p = 0.15). Data, code and model weights are released under permissive licenses. We demonstrate that weak labels and anatomical knowledge can alleviate the necessity for prohibitively expensive voxel-wise annotations.

ICOSeg: Real-Time ICOS Protein Expression Segmentation from Immunohistochemistry Slides Using a Lightweight Conv-Transformer Network.

Simple SummaryInducible T-cell COStimulator (ICOS) is a biomarker of interest in checkpoint inhibitor therapy, and as a means of assessing T-cell regulation as part of a complex process of adaptive immunity. The aim of our study is to segment the ICOS positive cells using a lightweight deep-learning segmentation network. We aim to assess the potential of a convolutional neural network and transformer together that permits the capture of relevant features from immunohistochemistry images. The proposed study achieved remarkable results compared to the existing biomedical segmentation methods on our in-house dataset and surpassed our previous analysis by only utilizing the Efficient-UNet network.In this article, we propose ICOSeg, a lightweight deep learning model that accurately segments the immune-checkpoint biomarker, Inducible T-cell COStimulator (ICOS) protein in colon cancer from immunohistochemistry (IHC) slide patches. The proposed model relies on the MobileViT network that includes two main components: convolutional neural network (CNN) layers for extracting spatial features; and a transformer block for capturing a global feature representation from IHC patch images. The ICOSeg uses an encoder and decoder sub-network. The encoder extracts the positive cell’s salient features (i.e., shape, texture, intensity, and margin), and the decoder reconstructs important features into segmentation maps. To improve the model generalization capabilities, we adopted a channel attention mechanism that added to the bottleneck of the encoder layer. This approach highlighted the most relevant cell structures by discriminating between the targeted cell and background tissues. We performed extensive experiments on our in-house dataset. The experimental results confirm that the proposed model achieves more significant results against state-of-the-art methods, together with an 8× reduction in parameters.

Enhancement of lower limb motor imagery ability via dual-level multimodal stimulation and sparse spatial pattern decoding method.

Recently, motor imagery brain-computer interfaces (MI-BCIs) with stimulation systems have been developed in the field of motor function assistance and rehabilitation engineering. An efficient stimulation paradigm and Electroencephalogram (EEG) decoding method have been designed to enhance the performance of MI-BCI systems. Therefore, in this study, a multimodal dual-level stimulation paradigm is designed for lower-limb rehabilitation training, whereby visual and auditory stimulations act on the sensory organ while proprioceptive and functional electrical stimulations are provided to the lower limb. In addition, upper triangle filter bank sparse spatial pattern (UTFB-SSP) is proposed to automatically select the optimal frequency sub-bands related to desynchronization rhythm during enhanced imaginary movement to improve the decoding performance. The effectiveness of the proposed MI-BCI system is demonstrated on an the in-house experimental dataset and the BCI competition IV IIa dataset. The experimental results show that the proposed system can effectively enhance the MI performance by inducing the α, β and γ rhythms in lower-limb movement imagery tasks.

Genetic variants in autophagy-related gene ATG2B predict the prognosis of colorectal cancer patients receiving chemotherapy.

Autophagy-related genes have a vital effect on colorectal cancer (CRC) by affecting genomic stability and regulating immune responses. However, the associations between genetic variants in autophagy-related genes and CRC outcomes for chemotherapy therapy remain unclear. The Cox regression model was used to evaluate the associations between single-nucleotide polymorphisms (SNPs) in autophagy-related genes and overall survival (OS) and progression-free survival (PFS) of CRC patients. The results were corrected by the false discovery rate (FDR) correction. We used the logistic regression model to investigate the associations of SNPs with the disease control rate (DCR) of patients. Gene expression analysis was explored based on an in-house dataset and other databases. The associations between gene expression and infiltrating immune cells were evaluated using the Tumor Immune Estimation Resource (TIMER) database. We observed that ATG2B rs17094017 A > T was significantly associated with increased OS (HR = 0.65, 95% CI = 0.50-0.86, P = 2.54×10-3), PFS (HR = 0.76, 95% CI = 0.62-0.93, P = 7.34×10-3), and DCR (OR = 0.60, 95% CI = 0.37-0.96, P = 3.31×10-2) of CRC patients after chemotherapy. The expression of ATG2B was down-expressed in CRC tissues than in adjacent normal tissues. Moreover, ATG2B expression influenced the infiltration of CD8+ T cells, CD4+ T cells, B cells, and T cell receptor signaling pathways, which may inhibit the occurrence of CRC by affecting the immune system. This study suggests that genetic variants in the autophagy-related gene ATG2B play a critical role in predicting the prognosis of CRC prognosis undergoing chemotherapy.

Multi-PPTP: Multiple Probabilistic Pedestrian Trajectory Prediction in the Complex Junction Scene

Pedestrian trajectory prediction in dynamic, multi-agent traffic junction scene is an important problem in the context of self-driving cars. Accurately predicting the trajectory of the agent, especially the pedestrian with high randomness, is of great significance to autonomous driving technology. In this paper, we propose Multi-PPTP, a novel trajectory prediction model that first utilizes a composite rasterized map to model the complex representations and interactions of road components, including dynamic obstacles (e.g., pedestrians, vehicles, and cyclists) and static road information (e.g., lanes, traffic lights, and sidewalks). Then it uses MapNet and AgentNet to extract spatio-temporal features by deep convolutional networks and LSTM to automatically derive relevant features, and next exploits Interaction-AttNet aggregate features to learn the high interactions among all components by affine transformation and multi-head attention mechanism. Additionally, we also propose a series of unique loss functions to predict multiple possible trajectories of each pedestrian while estimating their probabilities. Following extensive offline evaluation and comparison to the state-of-the-art models, our approach outperforms significantly the state-of-the-art models on our large scale in-house and public nuScenes dataset.

Semi-supervised medical image segmentation via uncertainty rectified pyramid consistency.

Despite that Convolutional Neural Networks (CNNs) have achieved promising performance in many medical image segmentation tasks, they rely on a large set of labeled images for training, which is expensive and time-consuming to acquire. Semi-supervised learning has shown the potential to alleviate this challenge by learning from a large set of unlabeled images and limited labeled samples. In this work, we present a simple yet efficient consistency regularization approach for semi-supervised medical image segmentation, called Uncertainty Rectified Pyramid Consistency (URPC). Inspired by the pyramid feature network, we chose a pyramid-prediction network that obtains a set of segmentation predictions at different scales. For semi-supervised learning, URPC learns from unlabeled data by minimizing the discrepancy between each of the pyramid predictions and their average. We further present multi-scale uncertainty rectification to boost the pyramid consistency regularization, where the rectification seeks to temper the consistency loss at outlier pixels that may have substantially different predictions than the average, potentially due to upsampling errors or lack of enough labeled data. Experiments on two public datasets and an in-house clinical dataset showed that: 1) URPC can achieve large performance improvement by utilizing unlabeled data and 2) Compared with five existing semi-supervised methods, URPC achieved better or comparable results with a simpler pipeline. Furthermore, we build a semi-supervised medical image segmentation codebase to boost research on this topic: https://github.com/HiLab-git/SSL4MIS.

A Multi-Task Convolutional Neural Network for Lesion Region Segmentation and Classification of Non-Small Cell Lung Carcinoma.

Targeted therapy is an effective treatment for non-small cell lung cancer. Before treatment, pathologists need to confirm tumor morphology and type, which is time-consuming and highly repetitive. In this study, we propose a multi-task deep learning model based on a convolutional neural network for joint cancer lesion region segmentation and histological subtype classification, using magnified pathological tissue images. Firstly, we constructed a shared feature extraction channel to extract abstract information of visual space for joint segmentation and classification learning. Then, the weighted losses of segmentation and classification tasks were tuned to balance the computing bias of the multi-task model. We evaluated our model on a private in-house dataset of pathological tissue images collected from Qilu Hospital of Shandong University. The proposed approach achieved Dice similarity coefficients of 93.5% and 89.0% for segmenting squamous cell carcinoma (SCC) and adenocarcinoma (AD) specimens, respectively. In addition, the proposed method achieved an accuracy of 97.8% in classifying SCC vs. normal tissue and an accuracy of 100% in classifying AD vs. normal tissue. The experimental results demonstrated that our method outperforms other state-of-the-art methods and shows promising performance for both lesion region segmentation and subtype classification.

Acoustic-Based Automatic Addressee Detection for Technical Systems: A Review

ObjectiveAcoustic addressee detection is a challenge that arises in human group interactions, as well as in interactions with technical systems. The research domain is relatively new, and no structured review is available. Especially due to the recent growth of usage of voice assistants, this topic received increased attention. To allow a natural interaction on the same level as human interactions, many studies focused on the acoustic analyses of speech. The aim of this survey is to give an overview on the different studies and compare them in terms of utilized features, datasets, as well as classification architectures, which has so far been not conducted.MethodsThe survey followed the Preferred Reporting Items for Systematic reviews and Meta-Analysis (PRISMA) guidelines. We included all studies which were analyzing acoustic and/or acoustic characteristics of speech utterances to automatically detect the addressee. For each study, we describe the used dataset, feature set, classification architecture, performance, and other relevant findings.Results1,581 studies were screened, of which 23 studies met the inclusion criteria. The majority of studies utilized German or English speech corpora. Twenty-six percent of the studies were tested on in-house datasets, where only limited information is available. Nearly 40% of the studies employed hand-crafted feature sets, the other studies mostly rely on Interspeech ComParE 2013 feature set or Log-FilterBank Energy and Log Energy of Short-Time Fourier Transform features. 12 out of 23 studies used deep-learning approaches, the other 11 studies used classical machine learning methods. Nine out of 23 studies furthermore employed a classifier fusion.ConclusionSpeech-based automatic addressee detection is a relatively new research domain. Especially by using vast amounts of material or sophisticated models, device-directed speech is distinguished from non-device-directed speech. Furthermore, a clear distinction between in-house datasets and pre-existing ones can be drawn and a clear trend toward pre-defined larger feature sets (with partly used feature selection methods) is apparent.

PKS: A photogrammetric key-frame selection method for visual-inertial systems built on ORB-SLAM3

Key-frame selection methods were developed in the past years to reduce the complexity of frame processing in visual odometry (VO) and visual simultaneous localization and mapping (VSLAM) algorithms. Key-frames help increasing algorithm's performances by sparsifying frames while maintaining its accuracy and robustness. Unlike current selection methods that rely on many heuristic thresholds to decide which key-frame should be selected, this paper proposes a photogrammetric-based key-frame selection method built upon ORB-SLAM3. The proposed algorithm, named Photogrammetric Key-frame Selection (PKS), replaces static heuristic thresholds with photogrammetric principles, ensuring algorithm’s robustness and better point cloud quality. A key-frame is chosen based on adaptive thresholds and the Equilibrium Of Center Of Gravity (ECOG) criteria as well as Inertial Measurement Unit (IMU) observations. To evaluate the proposed PKS method, the European Robotics Challenge (EuRoC) and an in-house datasets are used. Quantitative and qualitative evaluations are made by comparing trajectories, point clouds quality and completeness and Absolute Trajectory Error (ATE) in mono-inertial and stereo-inertial modes. Moreover, for the generated dense point clouds, extensive evaluations, including plane-fitting error, model deformation, model alignment error, and model density and quality, are performed. The results show that the proposed algorithm improves ORB-SLAM3 positioning accuracy by 18% in stereo-inertial mode and 20% in mono-inertial mode without the use of heuristic thresholds, as well as producing a more complete and accurate point cloud up to 50%. The open-source code of the presented method is available at https://github.com/arashazimi0032/PKS.

A User-Centric approach for Personalization based on Human Activity Recognition.

The objective of this work focuses on multiple independent user profiles that capture behavioral, emotional, medical, and physical patterns in the working and living environment resulting in one general user profile. Depending on the user's current activity (e.g. walking, eating, etc.), medical history, and other influential factors, the developed framework acts as a supplemental assistant to the user by providing not only the ability to enable supportive functionalities (e.g. image filtering, magnification, etc.) but also informative recommendations (e.g. diet, alcohol, etc.). The personalization of such a profile lies within the user's past preferences using human activity recognition as a base, and it is achieved through a statistical model, the Bayesian belief network. Training and real-time methodological pipelines are introduced and validated. The employed deep learning techniques for identifying human activities are presented and validated in publicly available and in-house datasets. The overall accuracy of human activity recognition reaches up to 86.96 %.

Cross-Modal Distillation to Improve MRI-Based Brain Tumor Segmentation With Missing MRI Sequences.

Convolutional neural networks (CNNs) for brain tumor segmentation are generally developed using complete sets of magnetic resonance imaging (MRI) sequences for both training and inference. As such, these algorithms are not trained for realistic, clinical scenarios where parts of the MRI sequences which were used for training, are missing during inference. To increase clinical applicability, we proposed a cross-modal distillation approach to leverage the availability of multi-sequence MRI data for training and generate an enriched CNN model which uses only single-sequence MRI data for inference but outperforms a single-sequence CNN model. We assessed the performance of the proposed method for whole tumor and tumor core segmentation with multi-sequence MRI data available for training but only T1-weighted ([Formula: see text]) sequence data available for inference, using BraTS 2018, and in-house datasets. Results showed that cross-modal distillation significantly improved the Dice score for both whole tumor and tumor core segmentation when only [Formula: see text] sequence data were available for inference. For the evaluation using the in-house dataset, cross-modal distillation achieved an average Dice score of 79.04% and 69.39% for whole tumor and tumor core segmentation, respectively, while a single-sequence U-Net model using [Formula: see text] sequence data for both training and inference achieved an average Dice score of 73.60% and 62.62%, respectively. These findings confirmed cross-modal distillation as an effective method to increase the potential of single-sequence CNN models such that segmentation performance is less compromised by missing MRI sequences or having only one MRI sequence available for segmentation.

Structure-Aware Long Short-Term Memory Network for 3D Cephalometric Landmark Detection.

Detecting 3D landmarks on cone-beam computed tomography (CBCT) is crucial to assessing and quantifying the anatomical abnormalities in 3D cephalometric analysis. However, the current methods are time-consuming and suffer from large biases in landmark localization, leading to unreliable diagnosis results. In this work, we propose a novel Structure-Aware Long Short-Term Memory framework (SA-LSTM) for efficient and accurate 3D landmark detection. To reduce the computational burden, SA-LSTM is designed in two stages. It first locates the coarse landmarks via heatmap regression on a down-sampled CBCT volume and then progressively refines landmarks by attentive offset regression using multi-resolution cropped patches. To boost accuracy, SA-LSTM captures global-local dependence among the cropping patches via self-attention. Specifically, a novel graph attention module implicitly encodes the landmark's global structure to rationalize the predicted position. Moreover, a novel attention-gated module recursively filters irrelevant local features and maintains high-confident local predictions for aggregating the final result. Experiments conducted on an in-house dataset and a public dataset show that our method outperforms state-of-the-art methods, achieving 1.64 mm and 2.37 mm average errors, respectively. Furthermore, our method is very efficient, taking only 0.5 seconds for inferring the whole CBCT volume of resolution 768×768×576 .

Path Signature Neural Network of Cortical Features for Prediction of Infant Cognitive Scores.

Studies have shown that there is a tight connection between cognition skills and brain morphology during infancy. Nonetheless, it is still a great challenge to predict individual cognitive scores using their brain morphological features, considering issues like the excessive feature dimension, small sample size and missing data. Due to the limited data, a compact but expressive feature set is desirable as it can reduce the dimension and avoid the potential overfitting issue. Therefore, we pioneer the path signature method to further explore the essential hidden dynamic patterns of longitudinal cortical features. To form a hierarchical and more informative temporal representation, in this work, a novel cortical feature based path signature neural network (CF-PSNet) is proposed with stacked differentiable temporal path signature layers for prediction of individual cognitive scores. By introducing the existence embedding in path generation, we can improve the robustness against the missing data. Benefiting from the global temporal receptive field of CF-PSNet, characteristics consisted in the existing data can be fully leveraged. Further, as there is no need for the whole brain to work for a certain cognitive ability, a top K selection module is used to select the most influential brain regions, decreasing the model size and the risk of overfitting. Extensive experiments are conducted on an in-house longitudinal infant dataset within 9 time points. By comparing with several recent algorithms, we illustrate the state-of-the-art performance of our CF-PSNet (i.e., root mean square error of 0.027 with the time latency of 518 milliseconds for each sample).

Unsupervised domain selective graph convolutional network for preoperative prediction of lymph node metastasis in gastric cancer.

Preoperative prediction of lymph node (LN) metastasis based on computed tomography (CT) scans is an important task in gastric cancer, but few machine learning-based techniques have been proposed. While multi-center datasets increase sample size and representation ability, they suffer from inter-center heterogeneity. To tackle the above issue, we propose a novel multi-source domain adaptation framework for this diagnosis task, which not only considers domain-invariant and domain-specific features, but also achieves the imbalanced knowledge transfer and class-aware feature alignment across domains. First, we develop a 3D improved feature pyramidal network (i.e., 3D IFPN) to extract common multi-level features from the high-resolution 3D CT images, where a feature dynamic transfer (FDT) module can promote the network's ability to recognize the small target (i.e., LN). Then, we design an unsupervised domain selective graph convolutional network (i.e., UDS-GCN), which mainly includes three types of components: domain-specific feature extractor, domain selector and class-aware GCN classifier. Specifically, multiple domain-specific feature extractors are employed for learning domain-specific features from the common multi-level features generated by the 3D IFPN. A domain selector via the optimal transport (OT) theory is designed for controlling the amount of knowledge transferred from source domains to the target domain. A class-aware GCN classifier is developed to explicitly enhance/weaken the intra-class/inter-class similarity of all sample pairs across domains. To optimize UDS-GCN, the domain selector and the class-aware GCN classifier provide reliable target pseudo-labels to each other in the iterative process by collaborative learning. The extensive experiments are conducted on an in-house CT image dataset collected from four medical centers to demonstrate the efficacy of our proposed method. Experimental results verify that the proposed method boosts LN metastasis diagnosis performance and outperforms state-of-the-art methods. Our code is publically available at https://github.com/infinite-tao/LN_MSDA.

Multilingual Speech to Text using Deep Learning based on MFCC Features

The proposed methodology presented in the paper deals with solving the problem of multilingual speech recognition. Current text and speech recognition and translation methods have a very low accuracy in translating sentences which contain a mixture of two or more different languages. The paper proposes a novel approach to tackling this problem and highlights some of the drawbacks of current recognition and translation methods. The proposed approach deals with recognition of audio queries which contain a mixture of words in two different languages - Kannada and English. The novelty in the approach presented, is the use of a next Word Prediction model in combination with a Deep Learning speech recognition model to accurately recognise and convert the input audio query to text. Another method proposed to solve the problem of multilingual speech recognition and translation is the use of cosine similarity between the audio features of words for fast and accurate recognition. The dataset used for training and testing the models was generated manually by the authors as there was no pre-existing audio and text dataset which contained sentences in a mixture of both Kannada and English. The DL speech recognition model in combination with the Word Prediction model gives an accuracy of 71% when tested on the in-house multilingual dataset. This method outperforms other existing translation and recognition solutions for the same test set. Multilingual translation and recognition is an important problem to tackle as there is a tendency for people to speak in a mixture of languages. By solving this problem, the barrier of language and communication can be lifted and thus can help people connect better and more comfortably with each other.