Real-world Datasets Research Articles

LID-Net: A lightweight image dehazing network for automatic driving vision systems

Visual system provides comprehensive road information for autonomous driving vehicles. Haze adversely affects the quality of driving images captured by onboard cameras, which poses a significant challenge to the safe operation of vehicles relying on pure vision-based autonomous driving solutions. To address the above issues, a lightweight image dehazing algorithm using a multi-scale architecture called LID-Net is proposed. LID-Net consists of Haze Extraction (HE) blocks and Haze Removal (HR) blocks. The HE block captures more haze features by employing a larger receptive field. The HR block introduces different attention weights to different regions to better remove haze of different concentrations. Furthermore, a Brightness Compensation unit is designed to address the issue of reduced brightness in images following dehazing. This unit compensates for the image's brightness without changing the color of the dehazed image. LID-Net is compared with other state-of-the-art methods on two real-world foggy weather datasets. The results indicate that LID-Net outperforms other methods in terms of dehazing effectiveness. LID-Net can efficiently process an image with a resolution of 1280 × 720 at 125 frames per second, which can fully meet the requirements of real-time processing of automatic driving systems.

Mitigating implicit and explicit bias in structured data without sacrificing accuracy in pattern classification

AbstractUsing biased data to train Artificial Intelligence (AI) algorithms will lead to biased decisions, discriminating against certain groups or individuals. Bias can be explicit (one or several protected features directly influence the decisions) or implicit (one or several protected features indirectly influence the decisions). Unsurprisingly, biased patterns are difficult to detect and mitigate. This paper investigates the extent to which explicit and implicit against one or more protected features in structured classification data sets can be mitigated simultaneously while retaining the data’s discriminatory power. The main contribution of this paper concerns an optimization-based bias mitigation method that reweights the training instances. The algorithm operates with numerical and nominal data and can mitigate implicit and explicit bias against several protected features simultaneously. The trade-off between bias mitigation and accuracy loss can be controlled using parameters in the objective function. The numerical simulations using real-world data sets show a reduction of up to 77% of implicit bias and a complete removal of explicit bias against protected features at no cost of accuracy of a wrapper classifier trained on the data. Overall, the proposed method outperforms the state-of-the-art bias mitigation methods for the selected data sets.

CUSTOMER SEGMENTATION USING RFM ANALYSIS

This research presents a comprehensive approach to customer segmentation using Recency, Frequency, and Monetary (RFM) analysis, combining statistical insights, data visualization, and machine learning techniques. The study utilizes a real-world dataset obtained from a retail environment, aiming to categorize customers based on their recent purchasing behavior, visit frequency, and monetary contributions to the store. The code begins with data preparation and exploration, ensuring data integrity by addressing issues such as negative quantities and missing customer identifiers. Following this, the Recency, Frequency, and Monetary metrics are computed, providing a holistic view of customer engagement and spending patterns. Visualizations, including violin plots, histograms, and box plots, are employed to intuitively convey the distribution of these metrics. The research then delves into the quantile-based segmentation of customers, allowing for a more granular classification. Quantiles are calculated to divide customers into four segments for each RFM metric. The resulting quantile labels are applied to the dataset, enabling the creation of a compound RFM quantile that combines recency, frequency, and monetary information. This combined quantile facilitates the definition of distinct customer segments. To further enhance the interpretability of customer segments, the study introduces a set of rules for labeling customers based on their RFM quantiles. These rules yield segments such as "Best Customer," "Loyal Customer," "Big Spender," "Dead Beats," and "Lost Customer." The resulting customer segmentation is presented visually through histograms and a pie chart, providing a clear and concise representation of the distribution of customers across different segments. Moreover, the research integrates machine learning models, including XGBClassifier, and CatBoostClassifier, to explore the potential of automating the segmentation process and predicting customer segments based on historical data. However, the machine learning aspect is introduced with commented-out sections, leaving room for further exploration and experimentation. In conclusion, this research contributes a comprehensive and detailed code implementation for RFM-based customer segmentation. The integration of visualization techniques aids in the interpretation of customer behavior, while the inclusion of machine learning models opens avenues for predictive analytics in customer segmentation. The presented approach provides valuable insights for businesses seeking to tailor marketing and customer relationship strategies based on individualized customer segments. KEYWORDS— Customer Segmentation, Frequency, Monetary Value, Recency, RFM Analysis.

Click-through rate prediction model based on graph networks and feature squeeze-and-excitation mechanism

Purpose Predicting a user’s click-through rate on an advertisement or item often uses deep learning methods to mine hidden information in data features, which can provide users with more accurate personalized recommendations. However, existing works usually ignore the problem that the drift of user interests may lead to the generation of new features when they compute feature interactions. Based on this, this paper aims to design a model to address this issue. Design/methodology/approach First, the authors use graph neural networks to model users’ interest relationships, using the existing user features as the node features of the graph neural networks. Second, through the squeeze-and-excitation network mechanism, the user features and item features are subjected to squeeze operation and excitation operation, respectively, and the importance of the features is adaptively adjusted by learning the channel weights of the features. Finally, the feature space is divided into multiple subspaces to allocate features to different models, which can improve the performance of the model. Findings The authors conduct experiments on two real-world data sets, and the results show that the model can effectively improve the prediction accuracy of advertisement or item click events. Originality/value In the study, the authors propose graph network and feature squeeze-and-excitation model for click-through rate prediction, which is used to dynamically learn the importance of features. The results indicate the effectiveness of the model.

Multi-Encoder Spatio-Temporal Feature Fusion Network for Electric Vehicle Charging Load Prediction

Electric vehicles (EVs) have been initiated as a preference for decarbonizing road transport. Accurate charging load prediction is essential for the construction of EV charging facilities systematically and for the coordination of EV energy demand with the requisite peak power supply. It is noted that the charging load of EVs exhibits high complexity and randomness due to temporal and spatial uncertainties. Therefore, this paper proposes a SEDformer-based charging road prediction method to capture the spatio-temporal characteristics of charging load data. As a deep learning model, SEDformer comprises multiple encoders and a single decoder. In particular, the proposed model includes a Temporal Encoder Block based on the self-attention mechanism and a Spatial Encoder Block based on the channel attention mechanism with sequence decomposition, followed by an aggregated decoder for information fusion. It is shown that the proposed method outperforms various baseline models on a real-world dataset from Palo Alto, U.S., demonstrating its superiority in addressing spatio-temporal data-driven load forecasting problems.

A Genetic Algorithm and Fuzzy Neural Network-Based Intelligent Temperature Control Decision Model for Coagulation Cooling Systems

The coagulation cooling system is a common key component in many industrial processes, and reasonable temperature control is crucial. However, due to the complexity of the coagulation cooling system, traditional temperature control methods often cannot achieve optimal performance. To solve this problem, we design an intelligent temperature control decision model by a combination of genetic algorithm and fuzzy neural network. The study firstly utilizes genetic algorithm to optimize the objective function and constraint conditions of the coagulation cooling PID system. At the same time, fuzzy neural network is fused with genetic algorithm to establish a dedicated T-S/2 neural network structure, completing the complete model design of this study. Finally, cooling efficiency, task completion rate and model stability analysis are evaluated on real-world datasets. To validate the proposed model, an example of a coagulation cooling system was constructed in the laboratory and compared with traditional temperature control methods. The experimental results show that the proposal can significantly improve the performance of temperature control and reduce energy consumption under different conditions. In addition, the proposal has the characteristics of adaptability and optimization performance, and can effectively achieve optimal temperature control in uncertain and complex environments.

Autocleandeepfood: auto-cleaning and data balancing transfer learning for regional gastronomy food computing

AbstractFood computing has emerged as a promising research field, employing artificial intelligence, deep learning, and data science methodologies to enhance various stages of food production pipelines. To this end, the food computing community has compiled a variety of data sets and developed various deep-learning architectures to perform automatic classification. However, automated food classification presents a significant challenge, particularly when it comes to local and regional cuisines, which are often underrepresented in available public-domain data sets. Nevertheless, obtaining high-quality, well-labeled, and well-balanced real-world labeled images is challenging since manual data curation requires significant human effort and is time-consuming. In contrast, the web has a potentially unlimited source of food data but tapping into this resource has a good chance of corrupted and wrongly labeled images. In addition, the uneven distribution among food categories may lead to data imbalance problems. All these issues make it challenging to create clean data sets for food from web data. To address this issue, we present AutoCleanDeepFood, a novel end-to-end food computing framework for regional gastronomy that contains the following components: (i) a fully automated pre-processing pipeline for custom data sets creation related to specific regional gastronomy, (ii) a transfer learning-based training paradigm to filter out noisy labels through loss ranking, incorporating a Russian Roulette probabilistic approach to mitigate data imbalance problems, and (iii) a method for deploying the resulting model on smartphones for real-time inferences. We assess the performance of our framework on a real-world noisy public domain data set, ETH Food-101, and two novel web-collected datasets, MENA-150 and Pizza-Styles. We demonstrate the filtering capabilities of our proposed method through embedding visualization of the feature space using the t-SNE dimension reduction scheme. Our filtering scheme is efficient and effectively improves accuracy in all cases, boosting performance by 0.96, 0.71, and 1.29% on MENA-150, ETH Food-101, and Pizza-Styles, respectively.

Autocleandeepfood: auto-cleaning and data balancing transfer learning for regional gastronomy food computing

AbstractFood computing has emerged as a promising research field, employing artificial intelligence, deep learning, and data science methodologies to enhance various stages of food production pipelines. To this end, the food computing community has compiled a variety of data sets and developed various deep-learning architectures to perform automatic classification. However, automated food classification presents a significant challenge, particularly when it comes to local and regional cuisines, which are often underrepresented in available public-domain data sets. Nevertheless, obtaining high-quality, well-labeled, and well-balanced real-world labeled images is challenging since manual data curation requires significant human effort and is time-consuming. In contrast, the web has a potentially unlimited source of food data but tapping into this resource has a good chance of corrupted and wrongly labeled images. In addition, the uneven distribution among food categories may lead to data imbalance problems. All these issues make it challenging to create clean data sets for food from web data. To address this issue, we present AutoCleanDeepFood, a novel end-to-end food computing framework for regional gastronomy that contains the following components: (i) a fully automated pre-processing pipeline for custom data sets creation related to specific regional gastronomy, (ii) a transfer learning-based training paradigm to filter out noisy labels through loss ranking, incorporating a Russian Roulette probabilistic approach to mitigate data imbalance problems, and (iii) a method for deploying the resulting model on smartphones for real-time inferences. We assess the performance of our framework on a real-world noisy public domain data set, ETH Food-101, and two novel web-collected datasets, MENA-150 and Pizza-Styles. We demonstrate the filtering capabilities of our proposed method through embedding visualization of the feature space using the t-SNE dimension reduction scheme. Our filtering scheme is efficient and effectively improves accuracy in all cases, boosting performance by 0.96, 0.71, and 1.29% on MENA-150, ETH Food-101, and Pizza-Styles, respectively.

Spatiotemporal Dynamic Multi-Hop Network for Traffic Flow Forecasting

Accurate traffic flow forecasting is vital for intelligent transportation systems, especially with urbanization worsening traffic congestion, which affects daily life, economic growth, and the environment. Precise forecasts aid in managing and optimizing transportation systems, reducing congestion, and improving air quality by cutting emissions. However, predicting outcomes is difficult due to intricate spatial relationships, nonlinear temporal patterns, and the challenges associated with long-term forecasting. Current research often uses static graph structures, overlooking dynamic and long-range dependencies. To tackle these issues, we introduce the spatiotemporal dynamic multi-hop network (ST-DMN), a Seq2Seq framework. This model incorporates spatiotemporal convolutional blocks (ST-Blocks) with residual connections in the encoder to condense historical traffic data into a fixed-dimensional vector. A dynamic graph represents time-varying inter-segment relationships, and multi-hop operation in the encoder’s spatial convolutional layer and the decoder’s diffusion multi-hop graph convolutional gated recurrent units (DMGCGRUs) capture long-range dependencies. Experiments on two real-world datasets METR-LA and PEMS-BAY show that ST-DMN surpasses existing models in three metrics.

Dynamic Causal Explanation Based Diffusion-Variational Graph Neural Network for Spatiotemporal Forecasting.

Graph neural networks (GNNs), especially dynamic GNNs, have become a research hotspot in spatiotemporal forecasting problems. While many dynamic graph construction methods have been developed, relatively few of them explore the causal relationship between neighbor nodes. Thus, the resulting models lack strong explainability for the causal relationship between the neighbor nodes of the dynamically generated graphs, which can easily lead to a risk in subsequent decisions. Moreover, few of them consider the uncertainty and noise of dynamic graphs based on the time series datasets, which are ubiquitous in real-world graph structure networks. In this article, we propose a novel dynamic diffusion-variational GNN (DVGNN) for spatiotemporal forecasting. For dynamic graph construction, an unsupervised generative model is devised. Two layers of graph convolutional network (GCN) are applied to calculate the posterior distribution of the latent node embeddings in the encoder stage. Then, a diffusion model is used to infer the dynamic link probability and reconstruct causal graphs (CGs) in the decoder stage adaptively. The new loss function is derived theoretically, and the reparameterization trick is adopted in estimating the probability distribution of the dynamic graphs by evidence lower bound (ELBO) during the backpropagation period. After obtaining the generated graphs, dynamic GCN and temporal attention are applied to predict future states. Experiments are conducted on four real-world datasets of different graph structures in different domains. The results demonstrate that the proposed DVGNN model outperforms state-of-the-art approaches and achieves outstanding root mean square error (RMSE) results while exhibiting higher robustness. Also, by F1-score and probability distribution analysis, we demonstrate that DVGNN better reflects the causal relationship and uncertainty of dynamic graphs. The website of the code is https://github.com/gorgen2020/DVGNN.

Research to classrooms: a co-designed curriculum brings All of Us data to secondary schools.

We describe new curriculum materials for engaging secondary school students in exploring the "big data" in the NIH All of Us Research Program's Public Data Browser and the co-design processes used to collaboratively develop the materials. We also describe the methods used to develop and validate assessment items for studying the efficacy of the materials for student learning as well as preliminary findings from these studies. Secondary-level biology teachers from across the United States participated in a 2.5-day Co-design Summer Institute. After learning about the All of Us Research Program and its Data Browser, they collaboratively developed learning objectives and initial ideas for learning experiences related to exploring the Data Browser and big data. The Genetic Science Learning Center team at the University of Utah further developed the educators' ideas. Additional teachers and their students participated in classroom pilot studies to validate a 22-item instrument that assesses students' knowledge. Educators completed surveys about the materials and their experiences. The "Exploring Big Data with the All of Us Data Browser" curriculum module includes 3 data exploration guides that engage students in using the Data Browser, 3 related multimedia pieces, and teacher support materials. Pilot testing showed substantial growth in students' understanding of key big data concepts and research applications. Our co-design process provides a model for educator engagement. The new curriculum module serves as a model for introducing secondary students to big data and precision medicine research by exploring diverse real-world datasets.

Research to classrooms: a co-designed curriculum brings All of Us data to secondary schools.

We describe new curriculum materials for engaging secondary school students in exploring the "big data" in the NIH All of Us Research Program's Public Data Browser and the co-design processes used to collaboratively develop the materials. We also describe the methods used to develop and validate assessment items for studying the efficacy of the materials for student learning as well as preliminary findings from these studies. Secondary-level biology teachers from across the United States participated in a 2.5-day Co-design Summer Institute. After learning about the All of Us Research Program and its Data Browser, they collaboratively developed learning objectives and initial ideas for learning experiences related to exploring the Data Browser and big data. The Genetic Science Learning Center team at the University of Utah further developed the educators' ideas. Additional teachers and their students participated in classroom pilot studies to validate a 22-item instrument that assesses students' knowledge. Educators completed surveys about the materials and their experiences. The "Exploring Big Data with the All of Us Data Browser" curriculum module includes 3 data exploration guides that engage students in using the Data Browser, 3 related multimedia pieces, and teacher support materials. Pilot testing showed substantial growth in students' understanding of key big data concepts and research applications. Our co-design process provides a model for educator engagement. The new curriculum module serves as a model for introducing secondary students to big data and precision medicine research by exploring diverse real-world datasets.

A Study on Graph Optimization Method for GNSS/IMU Integrated Navigation System Based on Virtual Constraints.

In GNSS/IMU integrated navigation systems, factors like satellite occlusion and non-line-of-sight can degrade satellite positioning accuracy, thereby impacting overall navigation system results. To tackle this challenge and leverage historical pseudorange information effectively, this paper proposes a graph optimization-based GNSS/IMU model with virtual constraints. These virtual constraints in the graph model are derived from the satellite's position from the previous time step, the rate of change of pseudoranges, and ephemeris data. This virtual constraint serves as an alternative solution for individual satellites in cases of signal anomalies, thereby ensuring the integrity and continuity of the graph optimization model. Additionally, this paper conducts an analysis of the graph optimization model based on these virtual constraints, comparing it with traditional graph models of GNSS/IMU and SLAM. The marginalization of the graph model involving virtual constraints is analyzed next. The experiment was conducted on a set of real-world data, and the results of the proposed method were compared with tightly coupled Kalman filtering and the original graph optimization method. In instantaneous performance testing, the method maintains an RMSE error within 5% compared with real pseudorange measurement, while in a continuous performance testing scenario with no available GNSS signal, the method shows approximately a 30% improvement in horizontal RMSE accuracy over the traditional graph optimization method during a 10-second period. This demonstrates the method's potential for practical applications.

A dyeing clustering algorithm based on ant colony path-finding mechanism

Clustering has a wide range of applications in various industries. In different research ideas, bionic algorithms have heuristic help for clustering research. However, most of the clustering algorithms based on bionics often have problems of insufficient information extraction, too many required parameters, and limited application scope. Therefore, these algorithms have lower accuracy than traditional clustering algorithms. To solve these problems, this paper proposes a new ant colony clustering algorithm, which is called Dyeing Clustering Algorithm based on Ant Colony Path-finding. This algorithm combines the idea of dyeing with pheromones for the first time and realizes clustering by using ants to crawl and dye each point, bringing new inspiration to the artificial intelligence research. This can better capture data features, improve fault tolerance and achieve a better clustering effect. In addition, the algorithm is constructed as a dynamic system to deeply mine data information with the strategy of swarm intelligence, including the ant colony generation model, path-finding model, path-finding termination model, and dyeing clustering model. Besides, to evaluate the algorithm, we selected 2 simulation data sets, 5 artificial data sets, 5 real-world data sets, 13 comparison algorithms, and 8 evaluation indexes. From the comprehensive multi-round experiments, seven of the eight evaluation indexes of the Dyeing Clustering Algorithm based on Ant Colony Path-finding algorithm are more advantageous than other comparison algorithms, which demonstrates its enormous potential in industrial applications.

A Dyeing Clustering Algorithm based on Ant Colony Path-finding Mechanism

Clustering has a wide range of applications in various industries. In different research ideas, bionic algorithms have heuristic help for clustering research. However, most of the clustering algorithms based on bionics often have problems of insufficient information extraction, too many required parameters, and limited application scope. Therefore, these algorithms have lower accuracy than traditional clustering algorithms. To solve these problems, this paper proposes a new ant colony clustering algorithm, which is called Dyeing Clustering Algorithm based on Ant Colony Path-finding. This algorithm combines the idea of dyeing with pheromones for the first time and realizes clustering by using ants to crawl and dye each point, bringing new inspiration to the artificial intelligence research. This can better capture data features, improve fault tolerance and achieve a better clustering effect. In addition, the algorithm is constructed as a dynamic system to deeply mine data information with the strategy of swarm intelligence, including the ant colony generation model, path-finding model, path-finding termination model, and dyeing clustering model. Besides, to evaluate the algorithm, we selected 2 simulation data sets, 5 artificial data sets, 5 real-world data sets, 13 comparison algorithms, and 8 evaluation indexes. From the comprehensive multi-round experiments, seven of the eight evaluation indexes of the Dyeing Clustering Algorithm based on Ant Colony Path-finding algorithm are more advantageous than other comparison algorithms, which demonstrates its enormous potential in industrial applications.

Machine learning-based causal inference for evaluating intervention in travel behaviour research: A difference-in-differences framework

Causal inference with the difference-in-differences (DID) framework is popular in identifying causal effects with observational data and has started to be applied in recent travel behaviour studies. Most relevant transportation research adopts the conventional linear parametric DID model, which is known to be inflexible and restrictive. This study applies non-parametric DID estimators facilitated by machine learning (ML) models for causal inference in a variety of data scenarios. Semi-parametric and doubly robust estimators are established and integrated with the ML-based cross-fitting pipeline. Simulation studies and empirical case studies are conducted to showcase the ability of ML-based DID to detect causal effects from both simulated and real-world datasets. Results suggest that the proposed methods outperform conventional DID models in all data scenarios. Light working models are generally preferred over hyperparameter-dependent ones for their comparable performance, lower computational burden, and higher levels of compatibility to real-world empirical analysis. Empirical case studies also demonstrate how the proposed DID method could be applied to evaluate the impacts of various interventions on travel behaviour in different contexts. The present study adds to the existing travel behaviour literature by leveraging machine learning algorithms and non-parametric estimators to the impact evaluation of external interventions on travel characteristics and expanding the application of causal inference approaches in transportation research.

GenTrajRec: A Graph-Enhanced Trajectory Recovery Model Based on Signaling Data

Signaling data are records of the interactions of users’ mobile phones with their nearest cellular stations, which could provide long-term and continuous-time location data of large-scale citizens, and therefore have great potential in intelligent transportation, smart cities, and urban sensing. However, utilizing the raw signaling data often suffers from two problems: (1) Low positioning accuracy. Since the signaling data only describes the interaction between the user and the mobile base station, they can only restore users’ approximate geographical location. (2) Poor data quality. Due to the limitations of mobile signals, user signaling may be missing and drifting. To address the above issues, we propose a graph-enhanced trajectory recovery network, GenTrajRec, to recover precise trajectories from signaling data. GenTrajRec encodes signaling data through spatiotemporal encoders and enhances the traveling semantics by constructing a signaling transition graph. In fusing the spatiotemporal information as well as the deep traveling semantics, GenTrajRec can well tackle the challenge of poor data quality, and recover precise trajectory from raw signaling data. Extensive experiments have been conducted on two real-world datasets from Mobile Signaling and Geolife, and the results confirm the effectiveness of our approach, and the positioning accuracy can be improved from 315 m per point to 82 m per point for signaling data using our network.

Open-world structured sequence learning via dense target encoding

Structured sequences are popularly used to describe graph data with time-evolving node features and edges. A typical real-world scenario of structured sequences is that unknown class labels continuously arrive and thus the training and testing often across different class spaces. This scenario is also referred to as the open-world learning problem on structured sequences. In this paper, we present a new Dense Open-world Structured Sequence Learning model (DOSSL for short) to learn graph streams in the open-world learning setting. To capture both structural and temporal information, DOSSL uses a GNN-based stochastic recurrent neural network for learning node representation in graph streams, then a truncated Laplacian distribution to describe the latent distribution of graph nodes, and a sampling function is used to generate node representations. Further, DOSSL learns dense target embeddings for the known classes to improve the compactness of known class distribution and reserve enough space for open-world unknown classes. The ultimate open-world classifier is optimized to detect the samples from unknown classes under the constraints of DVAE loss, label loss, class uncertainty loss, and dense target loss. Through empirical analysis conducted on real-world datasets, it has been demonstrated that the advanced technique known as DOSSL exhibits the ability to acquire precise node classifiers by harnessing the power of graph streams.

GenTrajRec: A Graph-Enhanced Trajectory Recovery Model Based on Signaling Data

Signaling data are records of the interactions of users’ mobile phones with their nearest cellular stations, which could provide long-term and continuous-time location data of large-scale citizens, and therefore have great potential in intelligent transportation, smart cities, and urban sensing. However, utilizing the raw signaling data often suffers from two problems: (1) Low positioning accuracy. Since the signaling data only describes the interaction between the user and the mobile base station, they can only restore users’ approximate geographical location. (2) Poor data quality. Due to the limitations of mobile signals, user signaling may be missing and drifting. To address the above issues, we propose a graph-enhanced trajectory recovery network, GenTrajRec, to recover precise trajectories from signaling data. GenTrajRec encodes signaling data through spatiotemporal encoders and enhances the traveling semantics by constructing a signaling transition graph. In fusing the spatiotemporal information as well as the deep traveling semantics, GenTrajRec can well tackle the challenge of poor data quality, and recover precise trajectory from raw signaling data. Extensive experiments have been conducted on two real-world datasets from Mobile Signaling and Geolife, and the results confirm the effectiveness of our approach, and the positioning accuracy can be improved from 315 m per point to 82 m per point for signaling data using our network.

AGENDA: Predicting Trip Purposes with A New Graph Embedding Network and Active Domain Adaptation

Trip purpose is a meaningful aspect of travel behaviour for the understanding of urban mobility. However, it is non-trivial to automatically obtain trip purposes. On one hand, trip purposes are naturally diverse and complicated, but the available predictive data sources are limited in real-world scenarios. On the other hand, since trip purpose labelling is costly and the development levels of cities are unbalanced, it is infeasible to access large-scale labeled data in less developed cities to train advanced prediction models. To narrow the gaps, this paper presents A new G raph E mbedding N etwork and active D omain A daptation based framework (AGENDA) that only requires open data sources and is capable of predicting in both label-rich cities and label-scarce cities. Specifically, in label-rich source cities, we first use the vehicle’s GPS trajectory and open POI check-ins to augment trip contexts. Then we establish a supervised graph embedding network with two attention mechanisms to extract the passenger’s latent activity semantics and a classifier to predict trip purpose. To enable the prediction in label-scarce target cities, we further devise an active domain adaptation framework, in which adversarial domain adaptation is used to transfer the source-learned knowledge, and active learning is used to integrate human intelligence in the model training. A group of experiments are conducted with real-world datasets in Beijing and Shanghai. Evaluation results demonstrate that the proposed framework significantly outperforms existing trip purpose prediction algorithms, and could make accurate trip purpose prediction in label-scarce cities with much fewer labelling efforts.