Flickr Dataset Research Articles

Crowdsourced mapping of tourism distribution and dynamics in Iceland using Flickr

ABSTRACT Social media has become a catalyst for tourism in nature-based destinations, raising management challenges to cope with abrupt visitation growth in sensitive settings. Access to location-based social media data provides valuable material to identify and monitor spatial distribution patterns in tourism visitation and longitudinal trends, especially in vast, remote settings. This paper aims to evaluate the use of Flickr geodata for mapping spatiotemporal dynamics of tourist behaviors in Iceland. The consistency of the Flickr dataset with other proxies supports its validity and relevance. Tourism visitation appears closely related with accessibility factors, which are characterized by a strong seasonal variation. On longer timescales, the data reflect the increase of off-season visitation and the emergence of new tourist sites. This study demonstrates the value and potential of such a dataset to get an overview of the spatial patterns and dynamics of tourism at a national-level and outlines the limits inherent to its use.

DEEPFAKE FACE RECOGNITION THROUGH MODIFIED AND IMPORVED DEEP TRANSFER LEARNING

The manipulation of photo, audio, and video content has been a topic of interest for many years, as people uses fake faces to indulge in various immoral act like pornography, fraud, and defamation. In the early days, classification of real and fake faces was done using traditional methods such as editing frames by frame or using chroma keying, this traditional approach is time consuming and lacks enough editors that have the technical skills to do the frame-by-frame edition or use the chroma keying. With technological advancement, new techniques have been developed that allow for much more sophisticated and realistic manipulations, one such technique is deepfakes. Deepfakes are created using deep learning algorithms to swap or replace faces in videos or images. This can be done with a high degree of realism, making it difficult to distinguish between real and fake content. This research aims to develop a deep fake detection system using deep transfer learning (modified VGG19 and ResNet50 models), these two models were chosen over other CNN architectures due to their proven better performance, faster recognition time and lesser memory usage. The research modified the original VGG19 and ResNet architectures by replacing the last five layer with a customized dense layers that will help with faster and accurate recognition of faces. A balanced dataset comprising 70,000 real faces from the Flickr dataset and 70,000 fake faces generated by StyleGAN was utilized. This research employed hold-out evaluation method. VGG19 gave an accuracy, f1score of 91.59% and 91.47% respectively while RestNet50 gave an average accuracy and F1score of 96.61% and 96.59% respectively on the testing dataset. This shows that ResNet 50 gave the best performance both on the training, validation and testing dataset. The developed system was also compared with other state-of-art methods and they were all outperformed.

TOWARDS LARGE-SCALE BUILDING ATTRIBUTE MAPPING USING CROWDSOURCED IMAGES: SCENE TEXT RECOGNITION ON FLICKR AND PROBLEMS TO BE SOLVED

Abstract. Crowdsourced platforms provide huge amounts of street-view images that contain valuable building information. This work addresses the challenges in applying Scene Text Recognition (STR) in crowdsourced street-view images for building attribute mapping. We use Flickr images, particularly examining texts on building facades. A Berlin Flickr dataset is created, and pre-trained STR models are used for text detection and recognition. Manual checking on a subset of STR-recognized images demonstrates high accuracy. We examined the correlation between STR results and building functions, and analysed instances where texts were recognized on residential buildings but not on commercial ones. Further investigation revealed significant challenges associated with this task, including small text regions in street-view images, the absence of ground truth labels, and mismatches in buildings in Flickr images and building footprints in OpenStreetMap (OSM). To develop city-wide mapping beyond urban hotspot locations, we suggest differentiating the scenarios where STR proves effective while developing appropriate algorithms or bringing in additional data for handling other cases. Furthermore, interdisciplinary collaboration should be undertaken to understand the motivation behind building photography and labeling. The STR-on-Flickr results are publicly available at https://github.com/ya0-sun/STR-Berlin.

Recommendation rules to personalize itineraries for tourists in an unfamiliar city

Currently, tourism has become increasingly related to new technologies. In this context, itinerary planning and trip recommendation are challenging tasks for tourists with different profiles. Tourists are generally not familiar with different Points-of-Interest (POIs) in a new city. To this end, they need to select and organize POIs that align with their interest preferences and trip constraints such as departure location, available trip duration, etc. as an itinerary. In this paper, we propose to extract recommendation rules using Multi-Objective Evolutionary Algorithms (MOEAs), to find a trade-off between two objectives: (1) maximizing the POIs popularity, and (2) maximizing the time user interest. We conducted a comparative study of several Multi-Objective Evolutionary Algorithms (MOEAs), such as NSGAII, SPEA2, and IBEA, based on the Flickr dataset of different cities. Our findings confirm the efficiency of NSGA-II in generating recommendation rules to personalize itineraries for tourists in unfamiliar cities.

Low Dimensional Relevance Coding for Personalized Tag Recommendation in Image Tagging Applications

An approach of image coding for tag recommendation based on feature clustering and weighted coding is presented in this paper. The existing tag recommendation approach develops a decision based on correlation of image features and their tag annotated. The descriptive feature of the image sample defines the content of an image and is correlated with database features for tag recommendation. The feature dimension and its representation have a greater impact on the recommendation performance. The recent method tag recommendation developed CNN based visual features and proposed a tag recommendation based on weight factor. The dimensional feature and the isolated weight allocation limit the performance of presented tag recommendation system. This paper presents a new weight allocation and feature clustering method for tag recommendation. An approach of integral coding for weighted image-tag is presented to improve recommendation accuracy. The proposed recommendation system performance is tested on Flickr dataset for retrieval and recommendation accuracy.

Crowdsourcing social values data: Flickr and public participation GIS provide different perspectives of ecosystem services in a remote coastal region

Spatial planning and environmental management are expected to adopt participatory processes. However, the needed spatial data on social values of ecosystem services are limited, especially for marine spatial planning for large, remote coastal areas, and the collection of such information can be time and resource intensive. Crowdsourcing techniques are cost efficient sources of social values data, but must meet the information needs of planners and managers with sufficient confidence. We evaluated the reliability of crowdsourced social values data by assessing the agreement between geotagged photos posted to the social media platform Flickr and data from an online public participation GIS (PPGIS) survey conducted to support marine spatial planning in the Kimberley region of Western Australia.Flickr could not represent all of the social values mapped in the PPGIS dataset, was notably dominated by scenic values, and under-represented the importance of recreational fishing and Aboriginal cultural values. The Flickr dataset was more affected by accessibility from settlements and transportation infrastructure (roads, boat ramps) and more restricted spatially. Maximum entropy (maxent) distribution models developed to explore the socioecological drivers determining where social values occur also highlighted that Flickr and PPGIS samples provide different perspectives on social values, even after accounting for differences in sampling intensity. The Flickr dataset suggested that the social values occurred under narrower ranges of socioecological conditions, and the PPGIS dataset was more likely to find protected areas to be valued, likely because participants believe protected areas to support those social values, but may not themselves pursue them there.Flickr and PPGIS crowdsourced datasets provide different perspectives on the spatial distribution of social values for several reasons: They differ in their spatial and demographic biases, as well as their measurement of revealed vs. stated preferences. Flickr is most effective at representing social values that are easily photographed and interpretable in photos, and may not capture some information needs of managers. PPGIS is under greater investigator control but, because stated preferences can be disconnected from actual use of the sites mapped, may be best suited for assessing management preferences and social acceptability. With understanding of the nuances of these datasets, crowdsourced social values data can be applied most appropriately to support successful planning outcomes.

A heterogeneous graph neural recommendation model with hierarchical social trust

In traditional social recommendation algorithms, methods based on Graph Neural Networks (GNNs) representation of social networks are widely used, but the increasing sparsity of data and the tendency of users’ social influence to be complex make it difficult for the algorithms to improve the recommendation quality efficiently. To address the issues, a heterogeneous graph neural recommendation model with hierarchical social trust (GraphNON) is proposed. The algorithm is improved by fully exploiting user-friend relationships and modeling different propagation channels of social influence in multiple ways respectively. Firstly, different types of potential friends are mined based on user-user social graph and user-item rating graph respectively. Different dynamic influences of direct friends and indirectly friends among users are modeled through hierarchical influence propagation. Secondly, associations between items are mined from user-item rating graph, and influence transmission among items is modeled through hierarchical influence propagation. Finally, updated user characteristics and item characteristics are used to predict whether a user is interested in an item or not. It is shown on both datasets that the model has significant improvement in both hit rate (HR) and normalized discounted cumulative gain (NDCG) metrics compared to the optimal benchmark model. For a potential feature dimension of 64, the HR of the Yelp dataset improved by 18.3% and the NDCG by 29.3%, while the HR of the Flickr dataset improved by 31.4% and the NDCG by 37.1%.

Adaptive optimized residual convolutional image annotation model with bionic feature selection model

The advancements in Internet and Web 2.0 technologies resulted in the easier generation and distribution of digital images. Many images are stored without any form of significant labelling. This makes searching for desired images tedious and time-consuming, resulting in an increased demand for an effective image retrieval technique. Image annotation provides one or more labels to an image that describes its content. These annotations greatly help to retrieve images from large databases using different related queries. Automatic Image Annotation (AIA) applications in computer vision, data mining and web search are increasing progressively. This work proposes a hybrid Deep Learning (DL) based optimized image annotation model for AIA. Initially, the pre-processing of an input image is carried out to remove the image noise using the triple Gaussian guided filter. In the second stage, the image's features like shape, colour and texture are extracted using Slantlet transform, YCrCb Colour space and Local Binary Pattern (LBP). The feature dimensionality is reduced by selecting features based on the adaptive red colobuses monkey bionic model. Finally, the adaptive image annotation is obtained using the deep-optimized convolutional residual image annotation model. The loss function reduction and updating of weight parameters are facilitated using the hunger game bionic model. The experiments are performed in Python using Corel-5 K, ESP-Game and MIR FLICKR datasets. The proposed model attains an overall precision of 56, 99.2 and 90%, respectively. Thus, the experimental evaluation results demonstrate the excellent performance of a proposed model over existing approaches.

An Image Recommendation Algorithm Based on Target Alternating Attention and User Affiliation Network

Currently, how to exploit the deep features of images in image recommender systems to achieve image enhancement still needs further research. In addition, little research has explored the implicit and increasing preferences of users by using the affiliation generated by indirect users and virtual users of the main users, which leads to the phenomenon of information cocoon. An Image Recommendation Algorithm Based on Target Alternating Attention and User Affiliation Network (TAUA) is proposed in this paper that addresses the problems of inadequate extraction of semantic features in an image and information cocoon in image recommender systems. First, to complete the multi-dimensional description of the image, we extract the category, color, and style features of the image through a multi-channel convolutional neural network (MCNN), and we then perform migration and integration on these features. Then, to enhance the pixel-level representation ability of the image and achieve image feature enhancement, we propose target alternating attention to capture the information of surrounding pixels alternately from inside to outside. Finally, a user affiliation network, including indirect users and virtual users, is established according to the user behavior and transaction record, and the users’ increasing preferences and affiliated users are mined through the implicit interaction relationship of users. Experimental results show that compared with baselines on the Amazon dataset, the results of F@10, NDCG@10, and AUC of the proposed algorithm are 4.02%, 5.00%, and 2.14% higher than those of ACF, and 5.76%, 0.86% and 1.16% higher than those of VPOI. On the Flickr dataset, our algorithm outperforms ACF by 5.74%, 5.12%, and 3.68% in F@10, NDCG@10, and AUC, respectively, and outperforms VPOI by 0.45%, 0.47%, and 0.49%. TAUA has better recommendation performance and can significantly improve the recommendation effect.

Deep learning approaches for robust deep fake detection

Detecting deepfake images using a deep learning approach, particularly using model Densenet121, involves training a neural network to differentiate between authentic and manipulated images. Deepfakes have gained prominence due to advances in deep learning, especially generative adversarial networks (GANs). They pose significant challenges to the veracity of digital content, as they can be used to create realistic and deceptive media. Deepfakes are realistic looking fake media generated by many artificial intelligence tools like face2face and deepfake, which pose a severe threat to public. As more deepfakes are spreading, we really need better ways to find and prevent them. Deepfake involves creation of highly realistic images and videos and misuse them for spreading fake news, defaming individuals, and possess a significant threat to the integrity of digital content. Our project “Deep Learning Approaches for Robust Deep Fake Detection” aims to address this critical issue by developing a robust system for identification and localization of deep fake content by using ‘Densenet121’ model. This proposed framework seamlessly integrates forgery detection and localization. The dataset used in this project is “140k Real and Fake Faces”, and it consists of 70k real faces from Flickr dataset collected by Nvidia and 70k fake faces sampled from the 1 million Fake faces generated by StyleGAN. For localization purpose, we use GRAD-CAM method to accurately identify the morphed regions. Overall, our goal is to make deepfake detection more effective and reliable in today’s digital landscape.

Transformer-based local-global guidance for image captioning

Image captioning is a difficult problem for machine learning algorithms to compress huge amounts of images into descriptive languages. The recurrent models are popularly used as the decoder to extract the caption with significant performance, while these models have complicated and inherently sequential overtime issues. Recently, transformers provide modeling long dependencies and support parallel processing of sequences compared to recurrent models. However, recent transformer-based models assign attention weights to all candidate vectors based on the assumption that all vectors are relevant and ignore the intra-object relationships. Besides, the complex relationships between key and query vectors cannot be provided using a single attention mechanism. In this paper, a new transformer-based image captioning structure without recurrence and convolution is proposed to address these issues. To this end, a generator network and a selector network to generate textual descriptions collaboratively are designed. Our work contains three main steps: (1) Design a transformer-based generator network as word-level guidance to generate next words based on the current state. (2) Train a latent space to learn the mapping of captions and images into the same embedding space to learn the text-image relation. (3) Design a selector network as sentence-level guidance to evaluate next words by assigning fitness scores to the partial captions through the embedding space. Compared with the architecture of existing methods, the proposed approach contains an attention mechanism without the dependencies of time. It executes each state to select the next best word using local–global guidance. In addition, the proposed model maintains dependencies between the sequences, and can be trained in parallel. Several experiments on the COCO and Flickr datasets demonstrate that the proposed approach can outperform various state-of-the-art models over well-known evaluation measures.

Detecting Deepfake Images Using Deep Learning Techniques and Explainable AI Methods

Nowadays, deepfake is wreaking havoc on society. Deepfake content is created with the help of artificial intelligence and machine learning to replace one person’s likeness with another person in pictures or recorded videos. Although visual media manipulations are not new, the introduction of deepfakes has marked a breakthrough in creating fake media and information. These manipulated pictures and videos will undoubtedly have an enormous societal impact. Deepfake uses the latest technology like Artificial Intelligence (AI), Machine Learning (ML), and Deep Learning (DL) to construct automated methods for creating fake content that is becoming increasingly difficult to detect with the human eye. Therefore, automated solutions employed by DL can be an efficient approach for detecting deepfake. Though the “black-box” nature of the DL system allows for robust predictions, they cannot be completely trustworthy. Explainability is the first step toward achieving transparency, but the existing incapacity of DL to explain its own decisions to human users limits the efficacy of these systems. Though Explainable Artificial Intelligence (XAI) can solve this problem by interpreting the predictions of these systems. This work proposes to provide a comprehensive study of deepfake detection using the DL method and analyze the result of the most effective algorithm with Local Interpretable Model-Agnostic Explanations (LIME) to assure its validity and reliability. This study identifies real and deepfake images using different Convolutional Neural Network (CNN) models to get the best accuracy. It also explains which part of the image caused the model to make a specific classification using the LIME algorithm. To apply the CNN model, the dataset is taken from Kaggle, which includes 70 k real images from the Flickr dataset collected by Nvidia and 70 k fake faces generated by StyleGAN of 256 px in size. For experimental results, Jupyter notebook, TensorFlow, NumPy, and Pandas were used as software, InceptionResnetV2, DenseNet201, InceptionV3, and ResNet152V2 were used as CNN models. All these models’ performances were good enough, such as InceptionV3 gained 99.68% accuracy, ResNet152V2 got an accuracy of 99.19%, and DenseNet201 performed with 99.81% accuracy. However, InceptionResNetV2 achieved the highest accuracy of 99.87%, which was verified later with the LIME algorithm for XAI, where the proposed method performed the best. The obtained results and dependability demonstrate its preference for detecting deepfake images effectively.

AN EXPLORATION OF INTERACTIONS BETWEEN URBAN HERITAGES AND TOURIST’S DIGITAL FOOTPRINT: NETWORK AND TEXTUAL ANALYSIS VIA GEOTAGGED FLICKR DATA IN AMSTERDAM

Abstract. Spatial information retrieved by geotagged social media data allows the analysis of when and where questions and supports a better understanding of people’s spatial interaction. The purpose of this paper is to reveal alternative heritage spots for reducing tourism pressure in certain areas within Amsterdam’s historical core by presenting tourists’ points of interests (POIs), their patterns between POIs, and the sentiments attached to these POIs. To fulfil the research aim, network analysis, textual analysis, and heat maps were conducted with geotagged Flickr data representing tourists’ uploaded images. The results show that the Flickr data set was useful to reveal the existing POIs and their relations. Moreover, the further analysis that is based on the intersection between the revealed spatial patterns and Amsterdam heritage sites showed alternative POIs nearby current POIs. Based on the results, we demonstrated how a dataset constructed from geotagged Flickr data can provide useful practical information for sustainable tourism development. Our research has the potential to support urban heritage and tourism researchers and policy makers with a better understanding of POIs, their relations and possible other POIs. The outcomes of this research can advise heritage tourists on alternative POIs to visit, and policy makers to plan alternative POIs.

Image caption generator using cnn and lstm


 
 
 For this paper, we use CNN and LSTM to become aware of the caption of the image. Image caption generation is a system that comprehends natural language processing & computer vision standards to recognize the connection of the image in English. In this research paper, we cautiously pursue a number of important concepts of photograph captioning and its familiar processes. We talk about Kera’s library, NumPy and Jupiter notebooks for the making of this paper. We also talk about Flickr dataset and CNN used for photo classification.
 
 

Deep objectness hashing using large weakly tagged photos

CNN-based hashing methods have greatly boosted the performance of image retrieval, under the strong supervision of large amounts of manually annotated labels. In recent years, a large number of social media images with user tags have been generated on the Internet. These images can be regarded as weakly labeled training data, which can provide rich samples for training hash network, and greatly reduce the cost of obtaining training data. However, there are noise and visual irrelevant tags in user tags, and different tags may describe different objects in the image. In the previous CNN-based hashing method, a training image usually corresponds to a manual label and generates a hash code. These methods are difficult to use the image described by user tags with noise. For solving the above problem, we propose a CNN-based objectness hash learning method using user tags as a guide for training. First of all, the user tags are roughly filtered to remove noise tags that are not related to the visual content of images. Secondly, we quantify user tags into a unified semantic space and extract the highest-frequency words of the semantic space from similar objectness areas as their labels. Then, these objectness areas with their labels are grouped into a series of triple units as training data. So that the generated hash code can inherit the semantic similarity of the objectness areas well, that is, the Hamming distance between hash codes generated by similar objectness areas is closer, the reverse is the farther. Experimental results on NUS-WIDE and Flickr datasets show that our method can effectively extract object-level semantic information from weak user tags, and improve the accuracy of image retrieval.

Sentiment Interaction Distillation Network for Image Sentiment Analysis

Sentiment is a high-level abstraction, and it is a challenging task to accurately extract sentimental features from visual contents due to the “affective gap”. Previous works focus on extracting more concrete sentimental features of individual objects by introducing saliency detection or instance segmentation into their models, neglecting the interaction among objects. Inspired by the observation that interaction among objects can impact the sentiment of images, we propose the Sentiment Interaction Distillation (SID) Network, which utilizes object sentimental interaction to guide feature learning. Specifically, we first utilize a panoptic segmentation method to obtain objects in images; then, we propose a sentiment-related edge generation method and employ Graph Convolution Network to aggregate and propagate object relation representation. In addition, we propose a knowledge distillation framework to utilize interaction information guiding global context feature learning, which can avoid noisy features introduced by error propagation and a varying number of objects. Experimental results show that our method outperforms the state-of-the-art algorithm, e.g., about 1.2% improvement on the Flickr dataset and 1.7% on the most challenging subset of Twitter I. It is demonstrated that the reasonable use of interaction features can improve the performance of sentiment analysis.

IPARS: An Image-based Personalized Advertisement Recommendation System on Social Networks

Social media has become a primary source of information for decision-makers, organizations, and scientists in today’s fast-paced world. Indeed, because of the large volume of user-generated data available on social media, these online platforms are viewed as computable data sources that potentially mirror reality. It could be an authentic environment for the task of target customers identification for marketing. This paper presents a novel image-based personalized advertisement recommendation system named IPARS to identify target customers in social media using image processing and machine learning techniques for an online advertisement. Assume having a set of advertising images; The problem is identifying a group of social media users who are likely to be the potential target of those images. In IPARS, a given social network is first converted into a weighted bipartite graph where the nodes are the users and keywords. Then, another bipartite graph is formed by decomposing the advertising images into their objects, labels, concepts, and sentiments. We propose a couple of formulas to calculate the weight of edges for both graphs. An algorithm is proposed to search the social graph and identify and rank the best group of users. We have evaluated our proposed model on a set of images and users from the Flickr dataset and Twitter. The results showed that IPARS has a better performance compared with other algorithms.

GTour: Multiple itinerary recommendation engine for group of tourists

Tourism is currently extremely significant in the world because it is one of a country’s primary sources of revenue and employment. Tourists experience a number of challenges in selecting suitable trips, consisting of a range of itineraries in relation to their interest preferences and distinct limitations. Tourists may also want to visit in a group, e.g. members of the families or friends. In such scenario it is very difficult to please each and every member of the group individually. In this article, we propose an algorithm named as gTour to solve these problems. The gTour algorithm is designed specifically to suggest multiple itineraries for a group. The gTour algorithm also works if a tourist decides to explore unknown places where the tourists have no past travel history. The performance of the proposed technique has been analyzed using Flickr dataset. The performance comparison shows that the proposed gTour algorithm performs better than other baseline algorithms.

Passive citizen science: The role of social media in wildlife observations.

Citizen science plays an important role in observing the natural environment. While conventional citizen science consists of organized campaigns to observe a particular phenomenon or species there are also many ad hoc observations of the environment in social media. These data constitute a valuable resource for 'passive citizen science'-the use of social media that are unconnected to any particular citizen science program, but represent an untapped dataset of ecological value. We explore the value of passive citizen science, by evaluating species distributions using the photo sharing site Flickr. The data are evaluated relative to those submitted to the National Biodiversity Network (NBN) Atlas, the largest collection of species distribution data in the UK. Our study focuses on the 1500 best represented species on NBN, and common invasive species within UK, and compares the spatial and temporal distribution with NBN data. We also introduce an innovative image verification technique that uses the Google Cloud Vision API in combination with species taxonomic data to determine the likelihood that a mention of a species on Flickr represents a given species. The spatial and temporal analyses for our case studies suggest that the Flickr dataset best reflects the NBN dataset when considering a purely spatial distribution with no time constraints. The best represented species on Flickr in comparison to NBN are diurnal garden birds as around 70% of the Flickr posts for them are valid observations relative to the NBN. Passive citizen science could offer a rich source of observation data for certain taxonomic groups, and/or as a repository for dedicated projects. Our novel method of validating Flickr records is suited to verifying more extensive collections, including less well-known species, and when used in combination with citizen science projects could offer a platform for accurate identification of species and their location.

A new point-of-interest approach based on multi-itinerary recommendation engine

The significance of tourism in the globe today is enormous since it is a major source of income and jobs for a nation. Tourists are facing a range of difficulties as they select suitable tours, consisting of several itineraries in terms of their interests and distinct constraints. An itinerary consists of many Points of Interest (POIs) and a POI can further be splitted into several attractions which are named as POI within POI. For selecting the itinerary, the existing techniques use the characteristics of POIs. However, a POI consists of many attractions. Out of these, one dominating attraction’s type is considered as POI type. This ignores the other type of attraction’s present in that POI. It may cause improper selection of itineraries. Therefore, selection of itineraries by considering POI within POI is of great benefit. But, it is very challenging. For this task, we suggest an algorithm called PWP. It recommends multiple itineraries that are based on the interest of visitors, popularity of itineraries and the cost of itineraries. If a tourist wants to visit unknown areas, the PWP algorithm can be expanded further. We have taken the similar user’s features to advise multiple itineraries using the Flickr dataset. The findings show that the proposed PWP algorithm out-performs the baseline algorithms in terms of real-life matrices and heuristic based metrics.