Fully-connected Network Research Articles

Asynchronous quantum repeater using multiple quantum memory.

A full-fledged quantum network relies on the formation of entangled links between remote location with the help of quantum repeaters. The famous Duan-Lukin-Cirac-Zoller quantum repeater protocol is based on long distance single-photon interference, which not only requires high phase stability but also cannot generate maximally entangled state. Here, we propose a quantum repeater protocol using the idea of post-matching, which retains the same efficiency as the single-photon interference protocol, reduces the phase-stability requirement and can generate maximally entangled state in principle. We also outline an implementation of our scheme based on the Kerr nonlinear resonator. Numerical simulations show that our protocol has its superiority by comparing with existing protocols under a generic noise model and show the feasibility of building a large-scale quantum communication network with our scheme. We believe our work represents a crucial step towards the construction of a fully-connected quantum network.

A Novel Fully-Connected Beamforming Network Architecture With the QR Factorization Synthesis Method

A Novel Fully-Connected Beamforming Network Architecture With the QR Factorization Synthesis Method

SMDDH: Singleton Mention Detection using Deep Learning in Hindi Text

Mention detection is an important component of the Coreference Resolution (CR) system, where mentions such as name, nominal, and pronominals are identified. These mentions can be purely coreferential mentions or singleton mentions (non-coreferential mentions). Coreferential mentions are those mentions in a text that refer to the same entities in the real world. Whereas, singleton mentions are mentioned only once in the text and do not participate in the coreference as they are not mentioned again in the following text. Filtering of these singleton mentions can substantially improve the performance of a CR process. This paper proposes a singleton mention detection module based on a Fully Connected Network (FCN) and a Long Short-Term Memory for Hindi text and model identifying singleton mentions so that these mentions can be filtered out to reduce the search space for CR. A CR system can look for the previous reference of that mention in the text and if these mentions are removed from the list of mentions, then it reduces the searching time and also space time. This model utilizes a few hand-crafted features, context information, and embedding for words from word2vec and a multilingual Bidirectional Encoder Representations from Transformers (mBERT) language model. The coreference annotated Hindi dataset comprising 3.6K sentences, and 78K tokens are used for the task. The singleton mention detection model is analyzed extensively by experimenting with various lengths of context windows for each mention. The performance of the model is significant with two window sizes of context as compared to other various window sizes of contexts such as 2,3,4,5, etc., and all previous and all next words of each mention. The Precision, Recall, and F-measure of the LSTM-FCN model with mBERT (Word + Context + Syntactic) with two window sizes of context for identifying the singleton mentions are 63%, 71%, and 67% respectively.

Comparative Analysis of Neural Network Architectures for Automated Fracture Detection in Hand X-ray Images

The application of several neural network architectures—including Fully Connected Networks (FCN), Convolutional Neural Networks (CNN), pretrained ResNet, Vision Transformer (ViT-B-16)—for the classification of hand X-ray images into "Fractured" and "Not Fractured"—categories is investigated in this work. The main goals are to evaluate these models' fracture detection ability and determine which architectural design fits this work. Because transfer learning let the model use past information from big-scale picture datasets, the pretrained ResNet model emerged as the most effective with high accuracy, stability, and resilience. The bespoke CNN also performed well, displaying excellent feature extraction powers especially for medical imaging. But the non-pretrained ResNet model overfitted, meaning deeper networks find it difficult to generalize without pretraining. Though innovative, the Vision Transformer performed poorly since it depends on a lot of training data and finds difficult learning of intricate spatial properties from little datasets. Although acting as a baseline, the FCN's simple architecture and incapacity to detect spatial hierarchies in images meant it could not match the efficacy of CNN models. Emphasizing the important function of transfer learning in clinical applications, the results show that pretrained CNN architectures, especially ResNet, offer the most consistent and accurate method for automatic fracture diagnosis in medical pictures.

Extraction of Winter Wheat Planting Plots with Complex Structures from Multispectral Remote Sensing Images Based on the Modified Segformer Model

As one of the major global food crops, the monitoring and management of the winter wheat planting area is of great significance for agricultural production and food security worldwide. Today, the development of high-resolution remote sensing imaging technology has provided rich sources of data for extracting the visual planting information of winter wheat. However, the existing research mostly focuses on extracting the planting plots that have a simple terrain structure. In the face of diverse terrain features combining mountainous areas, plains, and saline alkali land, as well as small-scale but complex planting structures, the extraction of planting plots through remote sensing imaging is subjected to great challenges in terms of recognition accuracy and model complexity. In this paper, we propose a modified Segformer model for extracting winter wheat planting plots with complex structures in rural areas based on the 0.8 m high-resolution multispectral data obtained from the Gaofen-2 satellite, which significantly improves the extraction accuracy and efficiency under complex conditions. In the encoder and decoder of this method, new modules were developed for the purpose of optimizing the feature extraction and fusion process. Specifically, the improvement measures of the proposed method include: (1) The MixFFN module in the original Segformer model is replaced with the Multi-Scale Feature Fusion Fully-connected Network (MSF-FFN) module, which enhances the model’s representation ability in handling complex terrain features through multi-scale feature extraction and position embedding convolution; furthermore, the DropPath mechanism is introduced to reduce the possibility of overfitting while improving the model’s generalization ability. (2) In the decoder part, after fusing features at four different scales, a CoordAttention module is added, which can precisely locate important regions with enhanced features in the images by utilizing the coordinate attention mechanism, therefore further improving the model’s extraction accuracy. (3) The model’s input data are strengthened by incorporating multispectral indices, which are also conducive to the improvement of the overall extraction accuracy. The experimental results show that the accuracy rate of the modified Segformer model in extracting winter wheat planting plots is significantly increased compared to traditional segmentation models, with the mean Intersection over Union (mIOU) and mean Pixel Accuracy (mPA) reaching 89.88% and 94.67%, respectively (an increase of 1.93 and 1.23 percentage points, respectively, compared to the baseline model). Meanwhile, the parameter count and computational complexity are significantly reduced compared to other similar models. Furthermore, when multispectral indices are input into the model, the mIOU and mPA reach 90.97% and 95.16%, respectively (an increase of 3.02 and 1.72 percentage points, respectively, compared to the baseline model).

Enhancing the analysis of murine neonatal ultrasonic vocalizations: Development, evaluation, and application of different mathematical models.

Rodents employ a broad spectrum of ultrasonic vocalizations (USVs) for social communication. As these vocalizations offer valuable insights into affective states, social interactions, and developmental stages of animals, various deep learning approaches have aimed at automating both the quantitative (detection) and qualitative (classification) analysis of USVs. So far, no notable efforts have been made to determine the most suitable architecture. We present the first systematic evaluation of different types of neural networks for USV classification. We assessed various feedforward networks, including a custom-built, fully-connected network, a custom-built convolutional neural network, several residual neural networks, an EfficientNet, and a Vision Transformer. Our analysis concluded that convolutional networks with residual connections specifically adapted to USV data, are the most suitable architecture for analyzing USVs. Paired with a refined, entropy-based detection algorithm (achieving recall of 94.9 % and precision of 99.3 %), the best architecture (achieving 86.79 % accuracy) was integrated into a fully automated pipeline capable of analyzing extensive USV datasets with high reliability. In ongoing projects, our pipeline has proven to be a valuable tool in studying neonatal USVs. By comparing these distinct deep learning architectures side by side, we have established a solid foundation for future research.

PAOFCDN: A novel method for predictive analysis of solar irradiance

Solar photovoltaic power is the most feasible Renewable Energy Source (RES) for Pakistan, due to ample sunlight availability throughout the year. Since solar photovoltaic power is primarily dependent on solar irradiance, forecasting of solar irradiance is essential for reliable, secure and effective incorporation of solar photovoltaic power in power systems. Considering the importance of solar irradiance forecasting, in this study, predictive analysis of Islamabad’s solar irradiance is performed by using a novel proposed model named as Pelican Algorithm-based Optimized Fully-Connected Deep Network (PAOFCDN). The initial weights of Fully-Connected Deep Network (FCDN) are optimized through an effective optimization technique known as Pelican optimization. The accuracy of the optimized network PAOFCDN is enhanced many fold as compared to the FCDN network trained with randomly initialized weights. The inherent issue of poor generalization in FCDN is also resolved by optimization. The superior performance of PAOFCDN is evident from its comparative evaluation with existing benchmark methods, i.e., Long Short-Term Memory (LSTM), Support Vector Regression (SVR), Least Square Boosting (LSBoost) and standard FCDN. PAOFCDN achieves the least Normalized Root Mean Square Error (NRMSE) of 0.0503 as compared to 0.1179 of LSTM, 0.1256 of FCDN, and 0.2992 of SVR and LSBoost. The proposed model is applied to three real-world solar irradiance datasets having different resolutions of 10-minutes, hourly and daily. This study took the initiative of performing predictions on three datsets having multiple resolutions in perspective of south asia.

Sublinear message bounds of authenticated implicit Byzantine agreement

This paper studies the message complexity of authenticated Byzantine agreement (BA) in synchronous, fully-connected distributed networks under an honest majority. We focus on the so-called implicit Byzantine agreement problem where each node starts with an input value and at the end a non-empty subset of the honest nodes should agree on a common input value by satisfying the BA properties (i.e., there can be undecided nodes)3. We show that a sublinear (in n, number of nodes) message complexity BA protocol under honest majority is possible in the standard PKI model when the nodes have access to an unbiased global coin and hash function. In particular, we present a randomized Byzantine agreement algorithm which, with high probability achieves implicit agreement, uses O˜(n) messages, and runs in O˜(1) rounds while tolerating (1/2−ϵ)n Byzantine nodes for any fixed ϵ>0, the notation O˜ hides a O(polylogn) factor4. The algorithm requires standard cryptographic setup PKI and hash function with a static Byzantine adversary. The algorithm works in the CONGEST model and each node does not need to know the identity of its neighbors, i.e., works in the KT0 model. The message complexity (and also the time complexity) of our algorithm is optimal up to a polylog n factor, as we show a Ω(n) lower bound on the message complexity. We further extend the result to Byzantine subset agreement, where a non-empty subset of nodes should agree on a common value. Lastly, we analyze several relevant results that follow from the construction of the main result.To the best of our knowledge, this is the first sublinear message complexity result of Byzantine agreement. A quadratic message lower bound is known for any deterministic BA protocol (due to Dolev-Reischuk [JACM 1985]). The existing randomized BA protocols have at least quadratic message complexity in the honest majority setting. Our result shows the power of a global coin in achieving significant improvement over the existing results. It can be viewed as a step towards understanding the message complexity of randomized Byzantine agreement in distributed networks with PKI.

Low-rank sparse fully-connected tensor network for tensor completion

Fully-connected tensor network (FCTN) has recently drawn lots of attention in tensor completion due to its full description of all correlations between any two modes. However, the FCTN model has multiple ranks, and existing methods often ignore the difficulty brought about by rank selection, especially when the model is rank-sensitive. To overcome this drawback, this paper proposed a low-rank sparse FCTN for tensor completion. Specifically, we theoretically show that, for a tensor with FCTN structure, its subtensor can be represented as a coefficient sum of basic tensors, where the coefficients are remained in the FCTN’s factors. This means that factors’ sparsity can improve the robustness of FCTN to larger rank selection. Moreover, according to the relation between the target tensor and its FCTN’s factors, low-rank constrain is used to enhance rank robustness. Lastly, we optimize the proposed model by the alternating direction method of multipliers (ADMM) algorithm. Experimental results show that the low-rank sparse constraint effectively improves the rank robustness of the FCTN model, and achieves excellent results compared with other state-of-the-art completion methods.

Deep learning-based GoogLeNet-embedded no-pooling dimension fully-connected network for short-term wind power prediction

The dependence of wind power on the natural environment leads to volatility, which can cause hidden dangers to the safe and stable operation of the power grid. In this work, a deep learning-based GoogLeNet-embedded no-pooling dimension fully-connected prediction network is proposed for the short-term prediction issue of wind power generation, and the deep learning-based GoogLeNet-embedded no-pooling dimension fully-connected network is compared with five algorithms including long short-term memory network and NasNet. The dataset was collected in Natal. The six algorithms employed predicted the value of wind power for the coming day. Among all, the deep learning-based GoogLeNet embedded no-pooling dimension fully-connected network achieved the optimal prediction results and evaluation metrics. The percentage reduction of each metric value from the second smallest long short-term memory network for the deep learning-based GoogLeNet-embedded no-pooling dimension fully-connected network is 27.0% for mean absolute error, 27.2% for mean absolute percentage error, 34.8% for mean squared error, 19.9% for root mean square error and 21.6% for symmetric mean absolute percentage error.

On the benefit of attention in inverse design of thin films filters

Abstract Attention layers are a crucial component in many modern deep learning models, particularly those used in natural language processing and computer vision. Attention layers have been shown to improve the accuracy and effectiveness of various tasks, such as machine translation, image captioning, etc. Here, the benefit of attention layers in designing optical filters based on a stack of thin film materials is investigated. The superiority of Attention layers over fully-connected Deep Neural Networks is demonstrated for this task.

Multi-Dimensional Data Recovery via Feature-Based Fully-Connected Tensor Network Decomposition

Multi-Dimensional Data Recovery via Feature-Based Fully-Connected Tensor Network Decomposition

A Block-Wised and Sparsely-Connected ANN for the Prediction of Human Joint Moment Based on the Improved Hill Musculoskeletal Model

Human joint moment plays an important role in rehabilitation assessment and human-robot interaction, which cannot be measured directly but can only be predicted via indirect measurement by an artificial neural network (ANN). However, most existing ANN models for human joint moment prediction use fully-connected network which has complex structure and no inclusion of domain knowledge. Thus, this study introduced a novel block-wised and sparsely-connected ANN model (BSANN) for human joint moment prediction, which significantly reduced the computational and storage costs. In this BSANN model, by using an improved Hill musculoskeletal (HMS) model, a single-output fully-connected network was established as a block to take each electromyograph (EMG) signal for the prediction of the muscle moment, and all muscle moments were connected together as inputs to obtain the joint moment. Compared to the ANN, our BSANN model decreased 80.7% connections and keeps good prediction accuracy. It provides embedded portable systems a powerful tool to predict joint moment.

Entropy-based guidance of deep neural networks for accelerated convergence and improved performance

Neural networks have dramatically increased our capacity to learn from large, high-dimensional datasets across innumerable disciplines. However, their decisions are not easily interpretable, their computational costs are high, and building and training them are not straightforward processes. To add structure to these efforts, we derive new mathematical results to efficiently measure the changes in entropy as fully-connected and convolutional neural networks process data. By measuring the change in entropy as networks process data effectively, patterns critical to a well-performing network can be visualized and identified. Entropy-based loss terms are developed to improve dense and convolutional model accuracy and efficiency by promoting the ideal entropy patterns. Experiments in image compression, image classification, and image segmentation on benchmark datasets demonstrate these losses guide neural networks to learn rich latent data representations in fewer dimensions, converge in fewer training epochs, and achieve higher accuracy.

FlexLogNet: A flexible deep learning-based well-log completion method of adaptively using what you have to predict what you are missing

Well logs are essential tools for understanding the characteristics of subsurface formations and exploring petroleum resources. However, well logs are often missing randomly due to cost constraints, instrument failures, or other factors. Many methods have been developed for completing missing well logs, but these methods are all based on fixed types of known well-log inputs to predict specific types of missing logs. This fixed input–output mode severely limits the application of these methods in actual data, where the known and missing well-log types are often varying. To address this problem, we propose a hybrid deep learning method with two heads of heterogeneous graph neural network (HGNN) and fully connected network (FCN) to achieve mutual prediction among multiple types of well logs. It can adaptively use all known well logs to predict any missing well logs, achieving a very flexible and practical well log completion function of using what you have to complete what you are missing. Specifically, the HGNN head infers the inter-relationships among multiple well logs to predict normalized logs that contain detailed information, which achieved by using multiple independent kernels to extracting and aggregating the features of the multiple logs. The FCN head estimates the global statistics of the predicted logs, including means and standard deviations, for de-normalizing the well logs estimated by the HGNN head. Both the HGNN and FCN heads are trained simultaneously by a hybrid loss function to ensure the consistency of their predictions. Furthermore, we present an adaptive training strategy that leverages all well logs, including those with missing segments. We demonstrate the capability of our model using four well logs: gamma ray (GR), bulk density (RHOB), neutron porosity (NPHI), and compressional waves sonic (DTC). Theoretically, the model trained on other logs can also predict each other. Our approach yields high Pearson correlation coefficients and small root mean square error on a dataset obtained from an offshore North Sea field near Norway, demonstrating the efficacy of our proposed technique.

Artificial intelligence detects awareness of functional relation with the environment in 3 month old babies

A recent experiment probed how purposeful action emerges in early life by manipulating infants’ functional connection to an object in the environment (i.e., tethering an infant’s foot to a colorful mobile). Vicon motion capture data from multiple infant joints were used here to create Histograms of Joint Displacements (HJDs) to generate pose-based descriptors for 3D infant spatial trajectories. Using HJDs as inputs, machine and deep learning systems were tasked with classifying the experimental state from which snippets of movement data were sampled. The architectures tested included k-Nearest Neighbour (kNN), Linear Discriminant Analysis (LDA), Fully connected network (FCNet), 1D-Convolutional Neural Network (1D-Conv), 1D-Capsule Network (1D-CapsNet), 2D-Conv and 2D-CapsNet. Sliding window scenarios were used for temporal analysis to search for topological changes in infant movement related to functional context. kNN and LDA achieved higher classification accuracy with single joint features, while deep learning approaches, particularly 2D-CapsNet, achieved higher accuracy on full-body features. For each AI architecture tested, measures of foot activity displayed the most distinct and coherent pattern alterations across different experimental stages (reflected in the highest classification accuracy rate), indicating that interaction with the world impacts the infant behaviour most at the site of organism~world connection.

A data-driven intravoxel mean diffusivities distribution approach for molecular classifications and MIB-1 prediction of gliomas.

Measuring non-parametric intravoxel mean diffusivity distributions (MDDs) using magnetic resonance imaging (MRI) is a sensitive method for detecting intracellular diffusivity changes during physiological alterations. Histological and molecular glioma classifications are essential for prognosis and treatment, with distinct water diffusion dynamics among subtypes. We developed a data-driven approach using a fully connected network (FCN) to enhance the speed and stability of calculating MDDs across varying SNRs, enable tumor microstructural mapping, and test its reliability in identifying MIB-1 labeling index (LI) levels and molecular status of gliomas. An FCN was trained to learn the mapping between the simulated diffusion decay curves and the ground truth MDDs. We performed 5 000 000 simulation curves with various diffusivity components and random SNR . Eighty percent of simulation curves were used for the FCN training, 10% for validation, and the others were external tests for the FCN performance evaluation. In vivo data were collected to evaluate its clinical reliability. One hundred one patients (44 years 14, 67 men) with gliomas and six healthy controls underwent a 3.0 T MRI examination with a spin echo-echo planar imaging (SE-EPI) diffusion-weighted imaging (DWI) sequence. The trained FCN was employed to calculate MDDs of each brain voxel by voxel. We used the Fuzzy C-means algorithm to cluster the MDDs of tumor voxels, facilitating the characterization of distinct glioma tissues. Quantitative assessments were conducted through sectional integrals of the MDDs, demarcated by six bands to derive signal fractions ( ) and diffusivities of the maximum peaks ( ). Cosine similarity scores (CSS) were used for MDD similarity. ANOVA and Mann-Whitney U test were used for difference analysis. Logistic regression and area under the receiver operator characteristic curve (AUC) were used for classification evaluation. The simulation results showed that the FCN-based MDD approach (FCN-MDD) achieved higher CSS than non-negative least squares-based MDD (NNLS-MDD). For in vivo data, the spectra of ET and NET obtained by FCN-MDD are more distinguishable than NNLS-MDD. Fraction maps delineate the characteristics of different tumor tissues (enhancing and non-enhancing tumor, edema, and necrosis). showed a positive and negative correlation with MIB-1 respectively ( , all ). The AUC of for predicting MIB-1 LI levels was 0.900 (95% CI, 0.826-0.974), versus 0.781 (0.677-0.886) of ADC. The highest AUC of isocitrate dehydrogenase (IDH) mutation status, assessed by a logistic regression model ( ) was 0.873 (95% CI, 0.802-0.944). The proposed FCN-MDD method was more robust to variations in SNR and less reliant on empirically set regularization values than the NNLS-MDD method. FCN-MDD also enabled qualitative and quantitative evaluation of the composition of gliomas.

Fluid viscosity prediction leveraging computer vision and robot interaction

Accurately determining fluid viscosity is crucial for various industrial and scientific applications. Traditional methods of viscosity measurement, though reliable, often require manual intervention and cannot easily adapt to real-time monitoring. With advancements in machine learning and computer vision, this work explores the feasibility of predicting fluid viscosity by analyzing fluid oscillations captured in video data. The pipeline employs a 3D convolutional autoencoder pretrained in a self-supervised manner to extract and learn features from semantic segmentation masks of oscillating fluids. Then, the latent representations of the input data, produced from the pretrained autoencoder, are processed with a distinct inference head to infer either the fluid category (classification) or the fluid viscosity (regression) in a time-resolved manner. When the latent representations generated by the pre-trained autoencoder are used for classification, the system achieves a 97.1% accuracy across a total of 4140 test datapoints. Similarly, for regression tasks, employing an additional fully-connected network as a regression head allows the pipeline to achieve a mean absolute error of 0.258 cP over 4416 test datapoints. This study represents an innovative contribution to both fluid characterization and the evolving landscape of Artificial Intelligence, demonstrating the potential of deep learning in achieving near real-time viscosity estimation and addressing practical challenges in fluid dynamics through the analysis of video data capturing oscillating fluid dynamics.

SVD-based algorithms for fully-connected tensor network decomposition

SVD-based algorithms for fully-connected tensor network decomposition

Nonequilibrium phase transitions and absorbing states in a model for the dynamics of religious affiliation

We propose a simple model to describe the dynamics of religious affiliation. For such purpose, we built a compartmental model with three distinct subpopulations, namely religious committed individuals, religious noncommitted individuals and not religious affiliated individuals. The transitions among the compartments are governed by probabilities, modeling social interactions among the groups and also spontaneous transitions among the compartments. First of all, we consider the model on a fully-connected network. Thus, we write a set of ordinary differential equations to study the evolution of the subpopulations. Our analytical and numerical results show that there is an absorbing state in the model where only one of the subpopulations survive in the long-time limit. There are also regions of parameters where some of the subpopulations coexist (two or three). We also verified the occurrence of two distinct critical points. In addition, we also present Monte Carlo simulations of the model on two-dimensional square lattices, in order to analyze the impact of the presence of a lattice structure on the critical behavior of the model. Comparison of the models’ results with data for religious affiliation in Northern Ireland shows a good qualitative agreement. Finally, we considered the presence of inflexible individuals in the population, i.e., individuals that never change their states. The impact of such special agents on the critical behavior of the model is also discussed.