Chart Images Research Articles

End-to-end on-line rescheduling from Gantt chart images using deep reinforcement learning

With the advent of the socio-technical manufacturing paradigm, the way in which rescheduling decisions are taken at the shop floor has radically changed in order to guarantee highly efficient production under increasingly dynamic conditions. To cope with uncertain production environments, a drastic increase in the type and degree of automation used at the shop floor for handling unforeseen events and unplanned disturbances is required. In this work, the on-line rescheduling task is modelled as a closed-loop control problem in which an artificial autonomous agent implements a control policy generated off-line using a schedule simulator to learn schedule repair policies directly from high-dimensional sensory inputs. The rescheduling control policy is stored in a deep neural network, which is used to select repair actions in order to achieve a small set of repaired goal states. The rescheduling agent is trained using Proximal Policy Optimisation based on a wide variety of simulated transitions between schedule states using colour-rich Gantt chart images and negligible prior knowledge as inputs. An industrial example is discussed to highlight that the proposed approach enables end-to-end deep learning of successful rescheduling policies to encode task-specific control knowledge that can be understood by human experts.

Dissimilarity-Based Regularized Learning of Charts

Chart images exhibit significant variabilities that make each image different from others even though they belong to the same class or categories. Classification of charts is a major challenge because each chart class has variations in features, structure, and noises. However, due to the lack of affiliation between the dissimilar features and the structure of the chart, it is a challenging task to model these variations for automatic chart recognition. In this article, we present a novel dissimilarity-based learning model for similar structured but diverse chart classification. Our approach jointly learns the features of both dissimilar and similar regions. The model is trained by an improved loss function, which is fused by a structural variation-aware dissimilarity index and incorporated with regularization parameters, making the model more prone toward dissimilar regions. The dissimilarity index enhances the discriminative power of the learned features not only from dissimilar regions but also from similar regions. Extensive comparative evaluations demonstrate that our approach significantly outperforms other benchmark methods, including both traditional and deep learning models, over publicly available datasets.

Perceptual Fusion of Electronic Chart and Marine Radar Image

Electronic charts and marine radars are indispensable equipment in ship navigation systems, and the fusion display of these two parts ensures that the vessel can display dangerous moving targets and various obstacles on the sea. To reduce the noise interference caused by external factors and hardware, a novel radar image denoising algorithm using the concept of Generative Adversarial Network (GAN) using Wasserstein distance is proposed. GAN focuses on transferring the image noise distribution between strong and weak noise, while the perceptual loss approach is to suppress the noise by comparing the perceptual characteristics of the output after denoising. Afterwards, an image registration method based on image transformation is proposed to eliminate the imaging difference between the radar image and chart image, in which the visual attribute transfer approach is used to transform images. Finally, the sparse theory is used to process the high frequency and low frequency subband coefficients of the detection image obtained by the fast Fourier transform in parallel to realizing the image fusion. The results show that the fused contour has a high consistency, fast training speed and short registration time.

Calculation of contrast for computer simulated resolution chart image

The work is devoted to the analysis of the resolution chart image created with the Image Simulation mode of ZEMAX software. The changes in image quality are simulated using the model of Helios-44M-4 photographic objective lens. The research is conducted for monochromatic light conditions and computer simulated resolution test chart as an object for the lens with different chart positions. An axial beam and an off-axis beam that corresponds to the maximum object space field angle are analyzed. The calculation of contrast of the simulated resolution chart images is made modeling different spatial frequencies of the chart lines. The contrast is calculated in three ways. The first way uses the maximum and minimum values of the image illumination, and the second and third ways are based on the integral values of the illumination of black and white lines, calculated with the method of rectangles and the method of trapeziums, respectively. Having been calculated the values of contrast are compared with the theoretical values of the diffraction modulation transfer function obtained with ZEMAX.

Chart Classification Using Siamese CNN.

In recovering information from the chart image, the first step should be chart type classification. Throughout history, many approaches have been used, and some of them achieve results better than others. The latest articles are using a Support Vector Machine (SVM) in combination with a Convolutional Neural Network (CNN), which achieve almost perfect results with the datasets of few thousand images per class. The datasets containing chart images are primarily synthetic and lack real-world examples. To overcome the problem of small datasets, to our knowledge, this is the first report of using Siamese CNN architecture for chart type classification. Multiple network architectures are tested, and the results of different dataset sizes are compared. The network verification is conducted using Few-shot learning (FSL). Many of described advantages of Siamese CNNs are shown in examples. In the end, we show that the Siamese CNN can work with one image per class, and a 100% average classification accuracy is achieved with 50 images per class, where the CNN achieves only average classification accuracy of 43% for the same dataset.

Evaginating scientific charts: Recovering direct and derived information encodings from chart images

Evaginating scientific charts: Recovering direct and derived information encodings from chart images

Impact of chart image characteristics on stock price prediction with a convolutional neural network.

Stock price prediction has long been the subject of research because of the importance of accuracy of prediction and the difficulty in forecasting. Traditionally, forecasting has involved linear models such as AR and MR or nonlinear models such as ANNs using standardized numerical data such as corporate financial data and stock price data. Due to the difficulty of securing a sufficient variety of data, researchers have recently begun using convolutional neural networks (CNNs) with stock price graph images only. However, we know little about which characteristics of stock charts affect the accuracy of predictions and to what extent. The purpose of this study is to analyze the effects of stock chart characteristics on stock price prediction via CNNs. To this end, we define the image characteristics of stock charts and identify significant differences in prediction performance for each characteristic. The results reveal that the accuracy of prediction is improved by utilizing solid lines, color, and a single image without axis marks. Based on these findings, we describe the implications of making predictions only with images, which are unstructured data, without using large amounts of standardized data. Finally, we identify issues for future research.

Application of Diagram Image Classification Based on PCBMER

Nowadays, in order to express data more clearly and accurately, data visualization has become a wave of the times. In this context, due to the different types and quantities of data stored in computer systems, different forms of data visualization can be seen in web pages, demonstration slides and papers. However, in many cases, we can’t directly obtain the underlying data, so the classification of chart images becomes very important. Only with the classification technology of chart images can we understand it more intuitively and deal with it more concretely, such as generating new charts according to the extracted data. Using depth learning technology to classify chart images is now a very popular way. In the field of image classification, convolution neural network (CNN) is the most widely used technology in depth learning, not only because of its high accuracy, but also because it encapsulates the process of feature extraction. At the same time, using the software architecture of PCBMER framework to design application programs can maintain high stability of the system. From the experimental results, this new technology is feasible, and the architecture can be further optimized to make the results more accurate.

Chartem: Reviving Chart Images with Data Embedding.

In practice, charts are widely stored as bitmap images. Although easily consumed by humans, they are not convenient for other uses. For example, changing the chart style or type or a data value in a chart image practically requires creating a completely new chart, which is often a time-consuming and error-prone process. To assist these tasks, many approaches have been proposed to automatically extract information from chart images with computer vision and machine learning techniques. Although they have achieved promising preliminary results, there are still a lot of challenges to overcome in terms of robustness and accuracy. In this paper, we propose a novel alternative approach called Chartem to address this issue directly from the root. Specifically, we design a data-embedding schema to encode a significant amount of information into the background of a chart image without interfering human perception of the chart. The embedded information, when extracted from the image, can enable a variety of visualization applications to reuse or repurpose chart images. To evaluate the effectiveness of Chartem, we conduct a user study and performance experiments on Chartem embedding and extraction algorithms. We further present several prototype applications to demonstrate the utility of Chartem.

Towards Assisting the Visually Impaired: A Review on Techniques for Decoding the Visual Data From Chart Images

The textual data of a document is supplemented by the graphical information in it. To make communication easier, they contain tables, charts and images. However, it excludes a section of our population - the visually impaired. With technological advancements, the blind can access the documents through text to speech software solutions. In this method, even images can be conveyed by reading out the figure captions. However, charts and other statistical comparisons which involve critical information are difficult to be “read” out this way. Aim of this paper is to analyse various methods available to solve this vexatious issue. We survey the state-of-the-art works that do the exact opposite of graphing tools. In this paper, we explore the existing literature in understanding the graphs and extracting the visual encoding from them. We classify these approaches into modality-based approaches, conventional and deep-learning based methods. The survey also contains comparisons and analyses relevant study datasets. As an outcome of this survey, we observe that: (i) All existing works under each category need decoding in a variety of graphs. (ii) Among the approaches, deep learning performs remarkably well in localisation and classification. However, it needs further improvements in reasoning from chart images. (iii) Research works are still in progress to access data from vector images. Recreating data from the raster images has unresolved issues. Based on this study, the various applications of decoding the graphs, challenges and future possibilities are also discussed. This paper explores current works in the extraction of chart data, which seek to enable researchers in Human Computer Interaction to achieve human-level perception of visual data by machines. In this era of visual summarisation of data, the AI approaches can automate the underlying data extraction and hence provide the natural language descriptions to support visually disabled users.

The 3D Nuclide Chart

We present an overview of The 3D Nuclide Chart, a new online nuclide chart designed for producing high quality 3D nuclide chart images for use in publications, presentations, education and outreach. An historical overview of the design of nuclide charts is given, followed by a description of The 3D Nuclide Chart itself.

ChartFuse: a novel fusion method for chart classification using heterogeneous microstructures

Chart images exhibit significant structural variabilities, which is called as micro-structural variabilities, which makes each image type different from others even though chart image belongs to the same class or categories. The lack of affiliation between the heterogeneous features and the structure of the chart images, make it challenging to learn these micro-variabilities features by any learning model for automatic chart recognition and interpretation. However, extracting low-level heterogeneous features from chart images remains challenging. This paper presents a novel chart image classification method by using local feature descriptor. We proposed a new heterogeneous feature extractor, namely the heterogeneity index (HI) fused with local penta pattern. Here, the microstructural features are defined on the similarity of the chroma effects, and HI is computed depending upon the basic colors and its intensity in the microstructures with similar chroma effects. HI integrates colors, textures, structural layout, and illumination details with the local features altogether for the image classification. The proposed method is tested on chart image datasets, namely FigureQA and our handcrafted chart dataset. Experimental results depict that our method classify images with an accuracy rate of 95%–97% which is an increase of 5%–10% as compared with the customary methods.

A Real-World Approach on the Problem of Chart Recognition Using Classification, Detection and Perspective Correction.

Data charts are widely used in our daily lives, being present in regular media, such as newspapers, magazines, web pages, books, and many others. In general, a well-constructed data chart leads to an intuitive understanding of its underlying data. In the same way, when data charts have wrong design choices, a redesign of these representations might be needed. However, in most cases, these charts are shown as a static image, which means that the original data are not usually available. Therefore, automatic methods could be applied to extract the underlying data from the chart images to allow these changes. The task of recognizing charts and extracting data from them is complex, largely due to the variety of chart types and their visual characteristics. Other features in real-world images that can make this task difficult are photo distortions, noise, alignment, etc. Two computer vision techniques that can assist this task and have been little explored in this context are perspective detection and correction. These methods transform a distorted and noisy chart in a clear chart, with its type ready for data extraction or other uses. This paper proposes a classification, detection, and perspective correction process that is suitable for real-world usage, when considering the data used for training a state-of-the-art model for the extraction of a chart in real-world photography. The results showed that, with slight changes, chart recognition methods are now ready for real-world charts, when taking time and accuracy into consideration.

Multi-label classification of line chart images using convolutional neural networks

In this paper, we propose a new convolutional neural network (CNN) architecture to build a multi-label classifier that categorizes line chart images according to their characteristics. The class labels are organized in the form of trend property (increasing or decreasing) and functional property (linear or exponential). In the proposed method, the Canny edge detection technique is applied as a data preprocessing step to increase both the classification accuracy and training speed. In addition, two different multi-label solution approaches are compared: label powerset (LP) and binary relevance (BR) methods. The experimental studies show that the proposed LP-CNN model achieves 93.75% accuracy, while the BR-CNN model reaches 92.97% accuracy on the test set, which contains real-world line chart images. The aim of this study is to build an efficient classifier that can be used for many purposes, such as automatically captioning the chart images, providing recommendations, redesigning charts, organizing a collection of chart images and developing better search engines.

Pie Chart or Pizza: Identifying Chart Types and Their Virality on Twitter

We aim to understand how data, rendered visually as charts or infographics, “travels” on social media. To do so we propose a neural network architecture that is trained to distinguish among different types of charts, for instance line graphs or scatter plots, and predict how much they will be shared. This poses significant challenges because of the varying format and quality of the charts that are posted, and the limitations in existing training data. To start with, our proposed system outperforms related work in chart type classification on the ReVision corpus. Furthermore, we use crowdsourcing to build a new corpus, more suitable to our aims, consisting of chart images shared by data journalists on Twitter. We evaluate our system on the second corpus with respect to both chart identification and virality prediction, with promising results.

Chart Mining: A Survey of Methods for Automated Chart Analysis.

Charts are useful communication tools for the presentation of data in a visually appealing format that facilitates comprehension. There have been many studies dedicated to chart mining, which refers to the process of automatic detection, extraction and analysis of charts to reproduce the tabular data that was originally used to create them. By allowing access to data which might not be available in other formats, chart mining facilitates the creation of many downstream applications. This paper presents a comprehensive survey of approaches across all components of the automated chart mining pipeline, such as (i) automated extraction of charts from documents; (ii) processing of multi-panel charts; (iii) automatic image classifiers to collect chart images at scale; (iv) automated extraction of data from each chart image, for popular chart types as well as selected specialized classes; (v) applications of chart mining; and (vi) datasets for training and evaluation, and the methods that were used to build them. Finally, we summarize the main trends found in the literature and provide pointers to areas for further research in chart mining.

Convolutional Neural Network for Stock Price Prediction Using Transfer Learning

The goal of this paper is to build a trading algorithm by applying image recognition neural network - Convolutional Neural Network(CNN) - to the 2D technical candle stick charts. First, this paper shows a research survey of the previous paper. Second, this paper explains the basic theory of CNN model and how it can works on chart images. Next, this project performs an experimental study of CNN on S&P 500 index from January 1, 1985 to June 30, 2020. The CNN model structure used in this paper is transferred from inception v3 with three additional layers, and the technical indicators used in the input chart image are simple moving average (25 days). The label data used in the model are categorical - either up, flat, or down. The model has 50% accuracy on the test set when conducting three-days ahead forecast, which is higher than the simple momentum strategy and contrarian strategy, indicating its high alpha generating potential. One-day ahead forecast and five-days ahead forecast have lower accuracy than the three-days forecast. This means you might have the best performance when you close your position at T + 3.

Forecasting stock prices with a feature fusion LSTM-CNN model using different representations of the same data.

Forecasting stock prices plays an important role in setting a trading strategy or determining the appropriate timing for buying or selling a stock. We propose a model, called the feature fusion long short-term memory-convolutional neural network (LSTM-CNN) model, that combines features learned from different representations of the same data, namely, stock time series and stock chart images, to predict stock prices. The proposed model is composed of LSTM and a CNN, which are utilized for extracting temporal features and image features. We measure the performance of the proposed model relative to those of single models (CNN and LSTM) using SPDR S&P 500 ETF data. Our feature fusion LSTM-CNN model outperforms the single models in predicting stock prices. In addition, we discover that a candlestick chart is the most appropriate stock chart image to use to forecast stock prices. Thus, this study shows that prediction error can be efficiently reduced by using a combination of temporal and image features from the same data rather than using these features separately.

Compensating MTF Measurements for Chart Quality Limitations

Objective measurements of imaging system sharpness (Modulation Transfer Function; MTF) are typically derived from test chart images. It is generally assumed that if testing recommendations are followed, test chart sharpness (which we also call “chart quality”) will have little impact on overall measurements. Standards such as ISO 12233 [1] ignore test chart sharpness. Situations where this assumption is not valid are becoming increasingly frequent, in part because extremely high-resolution cameras (over 30 megapixels) are becoming more common and in part because manufacturing test stations, which have limited space, often use charts that are smaller than optimum. Inconsistent MTF measurements caused by limited chart sharpness can be problematic in manufacturing supply chains that require consistency in measurements taken at different locations. We describe how to measure test chart sharpness, fit the measurement to a model, quantify the effects of chart sharpness on camera system MTF measurements, then compensate for these effects using deconvolution–by dividing measured system MTF by a model of the chart MTF projected on the image sensor. We use results of measurements with and without MTF compensation to develop a set of empirical guidelines to determine when chart quality is • good enough so that no compensation is needed, and • too low to be reliably compensated.

Closed-loop Rescheduling using Deep Reinforcement Learning

Fil: Palombarini, Jorge Andres. Consejo Nacional de Investigaciones Cientificas y Tecnicas. Centro Cientifico Tecnologico Conicet - Cordoba. Centro de Investigaciones y Transferencia de Villa Maria. Universidad Nacional de Villa Maria. Centro de Investigaciones y Transferencia de Villa Maria; Argentina