Proposed Model Research Articles

A Lightweight Recurrent Learning Network for Sustainable Compressed Sensing

Recently, deep learning-based compressed sensing (CS) has achieved great success in reducing the sampling and computational cost of sensing systems and improving the reconstruction quality. These approaches, however, largely overlook the issue of the computational cost; they rely on complex structures and task-specific operator designs, resulting in extensive storage and high energy consumption in CS imaging systems. In this article, we propose a lightweight but effective deep neural network based on recurrent learning to achieve a sustainable CS system; it requires a smaller number of parameters but obtains high-quality reconstructions. Specifically, our proposed network consists of an initial reconstruction sub-network and a residual reconstruction sub-network. While the initial reconstruction sub-network has a hierarchical structure to progressively recover the image, reducing the number of parameters, the residual reconstruction sub-network facilitates recurrent residual feature extraction via recurrent learning to perform both feature fusion and deep reconstructions across different scales. In addition, we also demonstrate that, after the initial reconstruction, feature maps with reduced sizes are sufficient to recover the residual information, and thus we achieved a significant reduction in the amount of memory required. Extensive experiments illustrate that our proposed model can achieve a better reconstruction quality than existing state-of-the-art CS algorithms, and it also has a smaller number of network parameters than these algorithms. Our source codes are available at: <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/C66YU/CSRN</uri> .

Diversifying Collaborative Filtering via Graph Spreading Network and Selective Sampling.

Graph neural network (GNN) is a robust model for processing non-Euclidean data, such as graphs, by extracting structural information and learning high-level representations. GNN has achieved state-of-the-art recommendation performance on collaborative filtering (CF) for accuracy. Nevertheless, the diversity of the recommendations has not received good attention. Existing work using GNN for recommendation suffers from the accuracy-diversity dilemma, where slightly increases diversity while accuracy drops significantly. Furthermore, GNN-based recommendation models lack the flexibility to adapt to different scenarios' demands concerning the accuracy-diversity ratio of their recommendation lists. In this work, we endeavor to address the above problems from the perspective of aggregate diversity, which modifies the propagation rule and develops a new sampling strategy. We propose graph spreading network (GSN), a novel model that leverages only neighborhood aggregation for CF. Specifically, GSN learns user and item embeddings by propagating them over the graph structure, utilizing both diversity-oriented and accuracy-oriented aggregations. The final representations are obtained by taking the weighted sum of the embeddings learned at all layers. We also present a new sampling strategy that selects potentially accurate and diverse items as negative samples to assist model training. GSN effectively addresses the accuracy-diversity dilemma and achieves improved diversity while maintaining accuracy with the help of a selective sampler. Moreover, a hyper-parameter in GSN allows for adjustment of the accuracy-diversity ratio of recommendation lists to satisfy the diverse demands. Compared to the state-of-the-art model, GSN improved R @20 by 1.62%, N @20 by 0.67%, G @20 by 3.59%, and E @20 by 4.15% on average over three real-world datasets, verifying the effectiveness of our proposed model in diversifying overall collaborative recommendations.

EgPDE-Net: Building Continuous Neural Networks for Time Series Prediction With Exogenous Variables.

While exogenous variables have a major impact on performance improvement in time series analysis, interseries correlation and time dependence among them are rarely considered in the present continuous methods. The dynamical systems of multivariate time series could be modeled with complex unknown partial differential equations (PDEs) which play a prominent role in many disciplines of science and engineering. In this article, we propose a continuous-time model for arbitrary-step prediction to learn an unknown PDE system in multivariate time series whose governing equations are parameterized by self-attention and gated recurrent neural networks. The proposed model, exogenous-guided PDE network (EgPDE-Net), takes account of the relationships among the exogenous variables and their effects on the target series. Importantly, the model can be reduced into a regularized ordinary differential equation (ODE) problem with specially designed regularization guidance, which makes the PDE problem tractable to obtain numerical solutions and feasible to predict multiple future values of the target series at arbitrary time points. Extensive experiments demonstrate that our proposed model could achieve competitive accuracy over strong baselines: on average, it outperforms the best baseline by reducing 9.85% on RMSE and 13.98% on MAE for arbitrary-step prediction.

Knowledge Graph Alignment Under Scarce Supervision: A General Framework With Active Cross-View Contrastive Learning.

Over recent years, a number of knowledge graphs (KGs) have emerged. Nevertheless, a KG can never reach full completeness. A viable approach to increase the coverage of a KG is KG alignment (KGA). The majority of previous efforts merely focus on the matching between entities, while largely neglect relations. Besides, they heavily rely on labeled data, which are difficult to obtain in practice. To address these issues, in this work, we put forward a general framework to simultaneously align entities and relations under scarce supervision. Our proposal consists of two main components, relation-enhanced active instance selection (RAS), and cross-view contrastive learning (CCL). RAS aims to select the most valuable instances to be labeled with the guidance of relations, while CCL contrasts cross-view representations to augment scarce supervision signals. Our proposal is agnostic to the underlying entity and relation alignment models, and can be used to improve their performance under limited supervision. We conduct experiments on a wide range of popular KG pairs, and the results demonstrate that our proposed model and its components can consistently boost the alignment performance under scarce supervision.

Optimized Hybrid Deep Learning for Real-Time Pandemic Data Forecasting: Long and Short-Term Perspectives

Background: With new variants of COVID-19 causing challenges, we need to focus on integrating multiple deep-learning frameworks to develop intelligent healthcare systems for early detection and diagnosis. Objective: This article suggests three hybrid deep learning models, namely CNN-LSTM, CNN-Bi- LSTM, and CNN-GRU, to address the pressing need for an intelligent healthcare system. These models are designed to capture spatial and temporal patterns in COVID-19 data, thereby improving the accuracy and timeliness of predictions. An output forecasting framework integrates these models, and an optimization algorithm automatically selects the hyperparameters for the 13 baselines and the three proposed hybrid models. Methods: Real-time time series data from the five most affected countries were used to test the effectiveness of the proposed models. Baseline models were compared, and optimization algorithms were employed to improve forecasting capabilities. Results: CNN-GRU and CNN-LSTM are the top short- and long-term forecasting models. CNNGRU had the best performance with the lowest SMAPE and MAPE values for long-term forecasting in India at 3.07% and 3.17%, respectively, and impressive results for short-term forecasting with SMAPE and MAPE values of 1.46% and 1.47%. Conclusion: Hybrid deep learning models, like CNN-GRU, can aid in early COVID-19 assessment and diagnosis. They detect patterns in data for effective governmental strategies and forecasting. This helps manage and mitigate the pandemic faster and more accurately.

Modified RNN for Solving Comprehensive Sylvester Equation With TDOA Application.

The augmented Sylvester equation, as a comprehensive equation, is of great significance and its special cases (e.g., Lyapunov equation, Sylvester equation, Stein equation) are frequently encountered in various fields. It is worth pointing out that the current research on simultaneously eliminating the lagging error and handling noises in the nonstationary complex-valued field is rather rare. Therefore, this article focuses on solving a nonstationary complex-valued augmented Sylvester equation (NCASE) in real time and proposes two modified recurrent neural network (RNN) models. The first proposed modified RNN model possesses gradient search and velocity compensation, termed as RNN-GV model. The superiority of the proposed RNN-GV model to traditional algorithms including the complex-valued gradient-based RNN (GRNN) model lies in completely eliminating the lagging error when employed in the nonstationary problem. The second model named complex-valued integration enhanced RNN-GV with the nonlinear acceleration (IERNN-GVN) model is proposed to adapt to a noisy environment and accelerate the convergence process. Besides, the convergence and robustness of these two proposed models are proved via theoretical analysis. Simulative results on an illustrative example and an application to the moving source localization coincide with the theoretical analysis and illustrate the excellent performance of the proposed models.

Joint Entity and Relation Extraction With Set Prediction Networks.

Joint entity and relation extraction is an important task in natural language processing, which aims to extract all relational triples mentioned in a given sentence. In essence, the relational triples mentioned in a sentence are in the form of a set, which has no intrinsic order between elements and exhibits the permutation invariant feature. However, previous seq2seq-based models require sorting the set of relational triples into a sequence beforehand with some heuristic global rules, which destroys the natural set structure. In order to break this bottleneck, we treat joint entity and relation extraction as a direct set prediction problem, so that the extraction model is not burdened with predicting the order of multiple triples. To solve this set prediction problem, we propose networks featured by transformers with non-autoregressive parallel decoding. In contrast to autoregressive approaches that generate triples one by one in a specific order, the proposed networks are able to directly output the final set of relational triples in one shot. Furthermore, we also design a set-based loss that forces unique predictions through bipartite matching. Compared with cross-entropy loss that highly penalizes small shifts in triple order, the proposed bipartite matching loss is invariant to any permutation of predictions; thus, it can provide the proposed networks with a more accurate training signal by ignoring triple order and focusing on relation types and entities. Various experiments on two benchmark datasets demonstrate that our proposed model significantly outperforms the current state-of-the-art (SoTA) models. Training code and trained models are now publicly available at https://github.com/DianboWork/SPN4RE.

Explainable Detection of Fake News on Social Media Using Pyramidal Co-Attention Network

In today’s world, fake news on social media is a universal trend and has severe consequences. There has been a wide variety of countermeasures developed to offset the effect and propagation of Fake News. The most common are linguistic-based techniques, which mostly use deep learning (DL) and natural language processing (NLP). Even government-sponsored organizations spread fake news as a cyberwar strategy. In literature, computational-based detection of fake news has been investigated to minimize it. The initial results of these studies are good but not significant. However, we argue that the explainability of such detection, particularly why a certain news item is detected as fake, is a vital missing element of the studies. In real-world settings, the explainability of the system’s decisions is just as important as its accuracy. This article explores explainable fake news detection and proposes a sentence-comment-based co-attention sub-network model. The proposed model uses user comments and news contents to mutually apprehend top- <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$k$</tex-math> </inline-formula> explainable check-worthy user comments and sentences for detecting fake news. The experimental result on real-world datasets shows that our proposed model outperforms state-of-the-art techniques by 5.56% in the <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$F$</tex-math> </inline-formula> 1 score. In addition, our model outperforms other baselines by 16.4% in normalized cumulative gain (NDCG) and 22.1% in Precision in identifying top- <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$k$</tex-math> </inline-formula> comments from users, which indicates why a news article can be fake.

Multi-Task Credible Pseudo-Label Learning for Semi-Supervised Crowd Counting.

As a widely used semi-supervised learning strategy, self-training generates pseudo-labels to alleviate the labor-intensive and time-consuming annotation problems in crowd counting while boosting the model performance with limited labeled data and massive unlabeled data. However, the noise in the pseudo-labels of the density maps greatly hinders the performance of semi-supervised crowd counting. Although auxiliary tasks, e.g., binary segmentation, are utilized to help improve the feature representation learning ability, they are isolated from the main task, i.e., density map regression and the multi-task relationships are totally ignored. To address the above issues, we develop a multi-task credible pseudo-label learning (MTCP) framework for crowd counting, consisting of three multi-task branches, i.e., density regression as the main task, and binary segmentation and confidence prediction as the auxiliary tasks. Multi-task learning is conducted on the labeled data by sharing the same feature extractor for all three tasks and taking multi-task relations into account. To reduce epistemic uncertainty, the labeled data are further expanded, by trimming the labeled data according to the predicted confidence map for low-confidence regions, which can be regarded as an effective data augmentation strategy. For unlabeled data, compared with the existing works that only use the pseudo-labels of binary segmentation, we generate credible pseudo-labels of density maps directly, which can reduce the noise in pseudo-labels and therefore decrease aleatoric uncertainty. Extensive comparisons on four crowd-counting datasets demonstrate the superiority of our proposed model over the competing methods. The code is available at: https://github.com/ljq2000/MTCP.

Toxicity Prediction for Immune Thrombocytopenia Caused by Drugs Based on Logistic Regression with Feature Importance

Background: One of the problems in drug discovery that can be solved by artificial intelligence is toxicity prediction. In drug-induced immune thrombocytopenia, toxicity can arise in patients after five to ten days by significant bleeding caused by drugdependent antibodies. In clinical trials, when this condition occurs, all the drugs consumed by patients should be stopped, although sometimes this is not possible, especially for older patients who are dependent on their medication. Therefore, being able to predict toxicity in drug-induced immune thrombocytopenia is very important. Computational technologies, such as machine learning, can help predict toxicity better than empirical techniques owing to the lower cost and faster processing. Objective: Previous studies used the KNN method. However, the performance of these approaches needs to be enhanced. This study proposes a Logistic Regression to improve accuracy scores. Methods: In this study, we present a new model for drug-induced immune thrombocytopenia using a machine learning method. Our model extracts several features from the Simplified Molecular Input Line Entry System (SMILES). These features were fused and cleaned, and the important features were selected using the SelectKBest method. The model uses a Logistic Regression that is optimized and tuned by the Grid Search Cross Validation. Results: The highest accuracy occurred when using features from PADEL, CDK, RDKIT, MORDRED, BLUEDESC combinations, resulting in an accuracy of 80%. Conclusion: Our proposed model outperforms previous studies in accuracy categories. The information and source code is accessible online at Github: https://github.com/Osphanie/Thrombocytopenia

Mitigating Modality Discrepancies for RGB-T Semantic Segmentation.

Semantic segmentation models gain robustness against adverse illumination conditions by taking advantage of complementary information from visible and thermal infrared (RGB-T) images. Despite its importance, most existing RGB-T semantic segmentation models directly adopt primitive fusion strategies, such as elementwise summation, to integrate multimodal features. Such strategies, unfortunately, overlook the modality discrepancies caused by inconsistent unimodal features obtained by two independent feature extractors, thus hindering the exploitation of cross-modal complementary information within the multimodal data. For that, we propose a novel network for RGB-T semantic segmentation, i.e. MDRNet+, which is an improved version of our previous work ABMDRNet. The core of MDRNet+ is a brand new idea, termed the strategy of bridging-then-fusing, which mitigates modality discrepancies before cross-modal feature fusion. Concretely, an improved Modality Discrepancy Reduction (MDR+) subnetwork is designed, which first extracts unimodal features and reduces their modality discrepancies. Afterward, discriminative multimodal features for RGB-T semantic segmentation are adaptively selected and integrated via several channel-weighted fusion (CWF) modules. Furthermore, a multiscale spatial context (MSC) module and a multiscale channel context (MCC) module are presented to effectively capture the contextual information. Finally, we elaborately assemble a challenging RGB-T semantic segmentation dataset, i.e., RTSS, for urban scene understanding to mitigate the lack of well-annotated training data. Comprehensive experiments demonstrate that our proposed model surpasses other state-of-the-art models on the MFNet, PST900, and RTSS datasets remarkably.

Webly Supervised Knowledge-Embedded Model for Visual Reasoning.

Visual reasoning between visual images and natural language remains a long-standing challenge in computer vision. Conventional deep supervision methods target at finding answers to the questions relying on the datasets containing only a limited amount of images with textual ground-truth descriptions. Facing learning with limited labels, it is natural to expect to constitute a larger scale dataset consisting of several million visual data annotated with texts, but this approach is extremely time-intensive and laborious. Knowledge-based works usually treat knowledge graphs (KGs) as static flattened tables for searching the answer, but fail to take advantage of the dynamic update of KGs. To overcome these deficiencies, we propose a Webly supervised knowledge-embedded model for the task of visual reasoning. On the one hand, vitalized by the overwhelming successful Webly supervised learning, we make much use readily available images from the Web with their weakly annotated texts for an effective representation. On the other hand, we design a knowledge-embedded model, including the dynamically updated interaction mechanism between semantic representation models and KGs. Experimental results on two benchmark datasets demonstrate that our proposed model significantly achieves the most outstanding performance compared with other state-of-the-art approaches for the task of visual reasoning.

Toward Graph Self-Supervised Learning With Contrastive Adjusted Zooming.

Graph representation learning (GRL) is critical for graph-structured data analysis. However, most of the existing graph neural networks (GNNs) heavily rely on labeling information, which is normally expensive to obtain in the real world. Although some existing works aim to effectively learn graph representations in an unsupervised manner, they suffer from certain limitations, such as the heavy reliance on monotone contrastiveness and limited scalability. To overcome the aforementioned problems, in light of the recent advancements in graph contrastive learning, we introduce a novel self-supervised GRL algorithm via graph contrastive adjusted zooming, namely, G-Zoom, to learn node representations by leveraging the proposed adjusted zooming scheme. Specifically, this mechanism enables G-Zoom to explore and extract self-supervision signals from a graph from multiple scales: micro (i.e., node level), meso (i.e., neighborhood level), and macro (i.e., subgraph level). First, we generate two augmented views of the input graph via two different graph augmentations. Then, we establish three different contrastiveness on the above three scales progressively, from node, neighboring, to subgraph level, where we maximize the agreement between graph representations across scales. While we can extract valuable clues from a given graph on the micro and macro perspectives, the neighboring-level contrastiveness offers G-Zoom the capability of a customizable option based on our adjusted zooming scheme to manually choose an optimal viewpoint that lies between the micro and macro perspectives to better understand the graph data. In addition, to make our model scalable to large graphs, we use a parallel graph diffusion approach to decouple model training from the graph size. We have conducted extensive experiments on real-world datasets, and the results demonstrate that our proposed model outperforms the state-of-the-art methods consistently.

Comprehensive Multiview Representation Learning via Deep Autoencoder-Like Nonnegative Matrix Factorization.

Learning a comprehensive representation from multiview data is crucial in many real-world applications. Multiview representation learning (MRL) based on nonnegative matrix factorization (NMF) has been widely adopted by projecting high-dimensional space into a lower order dimensional space with great interpretability. However, most prior NMF-based MRL techniques are shallow models that ignore hierarchical information. Although deep matrix factorization (DMF)-based methods have been proposed recently, most of them only focus on the consistency of multiple views and have cumbersome clustering steps. To address the above issues, in this article, we propose a novel model termed deep autoencoder-like NMF for MRL (DANMF-MRL), which obtains the representation matrix through the deep encoding stage and decodes it back to the original data. In this way, through a DANMF-based framework, we can simultaneously consider the multiview consistency and complementarity, allowing for a more comprehensive representation. We further propose a one-step DANMF-MRL, which learns the latent representation and final clustering labels matrix in a unified framework. In this approach, the two steps can negotiate with each other to fully exploit the latent clustering structure, avoid previous tedious clustering steps, and achieve optimal clustering performance. Furthermore, two efficient iterative optimization algorithms are developed to solve the proposed models both with theoretical convergence analysis. Extensive experiments on five benchmark datasets demonstrate the superiority of our approaches against other state-of-the-art MRL methods.

A Novel Industrial Software Model for Social Manufacturing

Abstract Distinct from the conventional manufacturing paradigm, the functioning of the Social Manufacturing (SocialM) model relies on a vast and dispersed array of social manufacturing resources. SocialM communities/groups emerge autonomously through business interactions, resource self-organization, and production information sharing, all driven by product orders within the SocialM system. This necessitates a more open, distributed, and autonomous software model to facilitate the allocation and operation of social manufacturing resources in support of the SocialM model. Consequently, this paper introduces a novel software development model designed specifically for SocialM. This proposed model employs business interaction software as the system's primary control portal, enabling efficient management of resources and users based on the social manufacturing communities/groups. Additionally, it facilitates the administration and oversight of numerous applications by utilizing an industrial Appstore. Moreover, the paper delves into the technological roadmap for realizing the essential technologies of the software model by integrating multi-agent and Message Queuing Telemetry Transport (MQTT) technologies. A 3D printing task simulation case was employed to demonstrate the model's working principle, which served to confirm the reliability and scalability of the SocialM software, as well as the feasibility of the SocialM theory.

New improved model for joint segmentation and registration of multi-modality images: application to medical images

Joint segmentation and registration of images is a focused area of research nowadays. Jointly segmenting and registering noisy images and images having weak boundaries/intensity inhomogeneity is a challenging task. In medical image processing, joint segmentation and registration are essential methods that aid in distinguishing structures and aligning images for precise diagnosis and therapy. However, these methods encounter challenges, such as computational complexity and sensitivity to variations in image quality, which may reduce their effectiveness in real-world applications. Another major issue is still attaining effective joint segmentation and registration in the presence of artifacts or anatomical deformations. In this paper, a new nonparametric joint model is proposed for the segmentation and registration of multi-modality images having weak boundaries/noise. For segmentation purposes, the model will be utilizing local binary fitting data term and for registration, it is utilizing conditional mutual information. For regularization of the model, we are using linear curvature. The new proposed model is more efficient to segmenting and registering multi-modality images having intensity inhomogeneity, noise and/or weak boundaries. The proposed model is also tested on the images obtained from the freely available CHOAS dataset and compare the results of the proposed model with the other existing models using statistical measures such as the Jaccard similarity index, relative reduction, Dice similarity coefficient and Hausdorff distance. It can be seen that the proposed model outperforms the other existing models in terms of quantitatively and qualitatively.

Combining permuted language model and adversarial training for Chinese machine reading comprehension

Due to the polysemy and complexity of the Chinese language, Chinese machine reading comprehension has always been a challenging task. To improve the semantic understanding and robustness of Chinese machine reading comprehension models, we propose a model that utilizes adversarial training algorithms and Permuted Language Model (PERT). Firstly, we employ the PERT pre-training model to embed paragraphs and questions into vector space to obtain corresponding sequential representations. Secondly, we use a multi-head self-attention mechanism to extract key textual information from the sequence and employ a Bi-GRU network to semantically fuse the output feature vectors, aiming to learn deep semantic representations in the text. Finally, we introduce perturbations into the model training process. We achieve this by utilizing adversarial training algorithms such as Fast Gradient Method (FGM) and Projected Gradient Descent (PGD). These algorithms generate adversarial samples to enhance the model’s robustness and stability when facing diverse inputs. We conducted comparative experiments on the publicly available Chinese reading comprehension datasets CMRC2018 and DRCD. The experimental results show that our proposed model has achieved significant improvements in both EM and F1-Score compared to the baseline model. To validate the model’s generalization and robustness, we utilized ChatGPT to construct a scientific dataset that includes a large number of domain-specific terms, sentences with mixed Chinese and English, and complex comprehension tasks. Our model also performed remarkably well on the self-built dataset. In conclusion, the proposed model not only effectively enhances the understanding of semantic information in Chinese text but also demonstrates a certain level of generalization capability.

SADCNN-ORBM: a hybrid deep learning model based citrus disease detection and classification

Citrus disease has a significant influence on agricultural productivity these days, so technology based on artificial intelligence has been developed for creating computer vision (CV) models. By spotting disease in its early stages and enabling necessary productivity actions, CV in agriculture improves the production of agricultural goods. In this paper, we developed a CV-based citrus disease detection model called the self-attention dilated convolutional neural network optimized restricted Boltzmann machine (SADCNN-ORBM) model, which consists of two crucial parts: a SADCNN for disease segmentation and an ORBM optimized by the self-adaptive coati optimization (SACO) algorithm to improve the classification performance of diseases, which successfully divides the disease type into three groups: anthracnose, melanose, and brown spot. Numerous feature sets, such as texture features, three-channel red, green, blue (RGB) features, local binary pattern (LBP) features, and speeded-up robust features (SURF) features, are combined and given as input into the classification layer in the proposed model. We compare our proposed model's performance with existing methods by using several evaluation metrics. The findings demonstrate the SADCNN-ORBM model's superiority in precisely recognizing and classifying citrus illnesses, outperforming all available techniques.

A novel dynamic enterprise architecture model: leveraging MAPE-K loop and case-based reasoning for context awareness

Nowadays, enterprises are required to take the uncertainty of the environment as decisive factor of success. For this reason, Enterprises should be prepared up-stream to react dynamically to the turbulent context. Considering that enterprise architecture is a tool drawing a blueprint that gives a holistic view of the enterprise, this blueprint should be able to represent this awareness to context and implements the techniques and mechanisms to react in a dynamic manner depending on the triggers of change. In this paper, the proposed model stipulates a “context-awareness” that monitors the internal and external context, and then adapt its reaction in alignment with the prefixed goals. The operationalization of our conception is realized through the monitor-analyze-plan-execute-knowledge (MAPE-K) loop, the case-based reasoning and machine learning techniques organized and orchestrated through a global algorithm of 6 main functions to monitor, compare, analyze, plan, execute and enrich the knowledge base. The results are verified in the light of a case study that demonstrates the applicability of our proposed model.

Learning to Synthesize Compatible Fashion Items Using Semantic Alignment and Collocation Classification: An Outfit Generation Framework.

The field of fashion compatibility learning has attracted great attention from both the academic and industrial communities in recent years. Many studies have been carried out for fashion compatibility prediction, collocated outfit recommendation, artificial intelligence (AI)-enabled compatible fashion design, and related topics. In particular, AI-enabled compatible fashion design can be used to synthesize compatible fashion items or outfits to improve the design experience for designers or the efficacy of recommendations for customers. However, previous generative models for collocated fashion synthesis have generally focused on the image-to-image translation between fashion items of upper and lower clothing. In this article, we propose a novel outfit generation framework, i.e., OutfitGAN, with the aim of synthesizing a set of complementary items to compose an entire outfit, given one extant fashion item and reference masks of target synthesized items. OutfitGAN includes a semantic alignment module (SAM), which is responsible for characterizing the mapping correspondence between the existing fashion items and the synthesized ones, to improve the quality of the synthesized images, and a collocation classification module (CCM), which is used to improve the compatibility of a synthesized outfit. To evaluate the performance of our proposed models, we built a large-scale dataset consisting of 20 000 fashion outfits. Extensive experimental results on this dataset show that our OutfitGAN can synthesize photo-realistic outfits and outperform the state-of-the-art methods in terms of similarity, authenticity, and compatibility measurements.