Skew Detection Algorithm Research Articles

Load Balancing Mechanism Based on Linear Regression Partition Prediction in Spark

With the rise of cloud computing, the internet of things, social networks and other technologies, information data is expanding rapidly, and traditional processing and storage systems have been difficult to deal with massive data. Spark is a fast and efficient MapReduce implementation developed after Hadoop. However, the shuffle operation in spark will cause the data set on some working nodes to be too large, while other nodes may be idle, which will affect the performance of spark jobs. This phenomenon is called data skew. Aiming at the problem of data skew in spark platform, this paper proposes a load balancing mechanism based on linear regression partition prediction: SP-LRP (Spark load balancing mechanism based on Linear Regression Partition), SP-LRP predicts the partition size of the reduce tasks at run-time, leverages the skew detection algorithm to identify the skew partition and adjusts the task resource allocation according to the fine-grained resource allocation algorithm. We use the benchmark dataset to evaluate SP-LRP, and compare the average execution time of two algorithms and the load balancing degree of reducers in different situations. The experimental confirmed the efficiency of SP-LRP in their respective usage scenarios.

An Efficient Document Skew Detection Method Using Probability Model and Q Test

Document skew detection is one of the key technologies in most of the document analysis systems. However, existing skew detection methods either have low accuracy or require a large amount of computation. To achieve a good tradeoff between efficiency and performance, we propose a novel skew detection approach based on bounding boxes, probability model, and Dixon’s Q test. Firstly, bounding boxes are used to pick out the eligible connected components (ECC). Then, we calculate the slopes of the skew document with the probability model. Finally, we find the optimal result with Dixon’s Q test and projection profile method. Moreover, the proposed method can detect the skew angle in a wider range. The experimental results show that our skew detection algorithm can achieve high speed and accuracy simultaneously compared with existing algorithms.

An Application of an EM Algorithm for Skew Detection of Signatures in Text Images

For security purposes of important documents and transactions in real world applications, we generally use biometric techniques for the authentication and validation of a person. If one has to achieve accurate results in the identification and verification process using a signature in text images as a biometric trait, we need to remove the skew of the signature in text images. In the preprocessing stage many phases are being carried out, among these phases, the signature in the text image, skew detection is the most significant phase, because these deskewed results will be used as one of the features in the feature extraction phase to identify and verify the signature. In this article we are proposing a novel method for skew detection of the signatures in text images using an estimation and maximization (EM) algorithm which is efficient and fast. The EM algorithm sequentially works in two stages, the combination of estimation (E-step) and the maximization (M-step) which helps in detection of the skew in skewed signatures in text image accurately.

A New Algorithm for Skew Detection of Telugu Language Document based on Principle-axis Farthest Pairs Quadrilateral (PFPQ)

A New Algorithm for Skew Detection of Telugu Language Document based on Principle-axis Farthest Pairs Quadrilateral (PFPQ)

A Novel Fast Hierarchical Projection Algorithm for Skew Detection in Multimedia Big Data Environment

Optical character recognition is an effective way for information input of paper media and skew detection of document images is a key stage of it. An algorithm for skew detection employing hierarchical projection is proposed in this paper. Projection histograms at various directions in a given range are acquired according to an initial angle step length. Then variances of it and absolute difference of the variances are calculated respectively and the angle corresponding to the maximum difference is served as rough skew estimate. The similar work above is implemented in which the projection angle range is two times the initial step length and symmetric about the estimate. Finally, the maximum value of the variances is found and the angle corresponding to it is served as skew angle. Experimental results show the algorithm has such advantages as fast processing speed, high detection accuracy, insensitivity to noise and suitable for complex layout.

Skew detection and correction based on an axes-parallel bounding box

Skew detection and correction of a scanned document is a preprocessing step for optical character recognition systems. We present a novel approach in skew detection and correction of a typed document by minimizing the area of the axis-parallel bounding box. Advantage of our approach over existing methods is that our algorithm is script and content independent. Moreover, our algorithm is not subject to skew angle limitations. The performance of our algorithm was evaluated using images of different scripts with varying skew angles with and without graphical images. Our experiments show that our algorithm outperforms existing state-of-the-art skew detection algorithms.

Efficient skew detection of printed document images based on novel combination of enhanced profiles

Document skew is often introduced during the capturing process of the document image processing pipeline and may seriously affect the performance of subsequent stages of segmentation and recognition. Skew detection is often accomplished with the use of horizontal projections, while recently, a new approach that is based on vertical projections has been introduced. In this paper, we use the technique of minimum bounding box area in order to combine a horizontal with a new reinforced vertical projection profile method. We are motivated by the fact that the horizontal and the novel vertical projection profiles are found to be complementary to each other. We claim that the proposed approach has more accurate performance compared with other state-of-the-art skew detection algorithms; it deals with all the drawbacks of the projection profile methods; it is more noise and warp resistant and gives accurate results for any kind of printed document image. For these reasons, it can be efficiently applied to historical machine printed or multicolumn documents, documents with figures and tables, while it is robust for any kind of script. Extended experimental results on two databases in different skew angle range, with representative printed documents of all kinds, as well as printed documents of two historical books, prove the efficiency of the proposed approach. There is also a comparison with commercial products in several cases where the contribution of the proposed algorithm is demonstrated at optical character recognition level. Moreover, an analysis of the accuracy performance of the main elements of the proposed technique is also performed.

A Skew Detection Algorithm for PDF417 Based on Feature Point

The automatic identification of 2D (two dimensional) bar code PDF417 is very sensitive to skew angle, but, the common skew angle detection methods have shortcomings such as weak performance in time complexity. In this paper, based on the properties of PDF417 character code and the extraction of feature points, we get skew angle of PDF417 bar code image using the least square method. Experiments show that this algorithm has virtue of less computation and high accuracy.

Skew detection in document images based on rectangular active contour

The digitalization processes of documents produce frequently images with small rotation angles. The skew angles in document images degrade the performance of optical character recognition (OCR) tools. Therefore, skew detection of document images plays an important role in automatic document analysis systems. In this paper, we propose a Rectangular Active Contour Model (RAC Model) for content region detection and skew angle calculation by imposing a rectangular shape constraint on the zero-level set in Chan–Vese Model (C-V Model) according to the rectangular feature of content regions in document images. Our algorithm differs from other skew detection methods in that it does not rely on local image features. Instead, it uses global image features and shape constraint to obtain a strong robustness in detecting skew angles of document images. We experimented on different types of document images. Comparing the results with other skew detection algorithms, our algorithm is more accurate in detecting the skews of the complex document images with different fonts, tables, illustrations, and layouts. We do not need to pre-process the original image, even if it is noisy, and at the same time the rectangular content region of a document image is also detected.

Skew detection using wavelet decomposition and projection profile analysis

In this paper, a novel document skew detection algorithm based on wavelet decompositions and projection profile analysis is proposed. First, the skewed document images are decomposed by the wavelet transform. The matrix containing the absolute values of the horizontal sub-band coefficients, which preserves the text’s horizontal structure, is then rotated through a range of angles. A projection profile is computed at each angle, and the angle that maximizes a criterion function is regarded as the skew angle. Experimental results show that this algorithm performs well on document images of various layouts and is also robust to different languages. The effects of various wavelet basis, number of decomposition levels, and parameters of the criterion function are investigated too.

A ROBUST SYSTEM FOR THRESHOLDING AND SKEW DETECTION IN MIXED TEXT/GRAPHICS DOCUMENTS

This article presents an automatic system that takes in grayscale scanned images, which could be mixed text/graphic documents, and performs thresholding and skew detection on the document images. The system consists of two major components; multistage thresholding and skew detection. The proposed skew detection algorithm has no restriction on detectable angle range and does not rely on large blocks of text. It works well on textual document images, graphical images and mixed text and graphic images. The performance of the systems was evaluated using over 60 images that consist of real life documents like envelopes and artificial mixed text/graphic icons. The superior performance of thresholding is clear compared to other techniques from the evaluation. The skew detection algorithm is robust when compared with other methods when very few text lines are present in the document image.

A Document Skew Detection Method Using the Hough Transform

Document image processing has become an increasingly important technology in the automation of office documentation tasks. Automatic document scanners such as text readers and OCR (Optical Character Recognition) systems are an essential component of systems capable of those tasks. One of the problems in this field is that the document to be read is not always placed correctly on a flatbed scanner. This means that the document may be skewed on the scanner bed, resulting in a skewed image. This skew has a detrimental effect on document on document analysis, document understanding, and character segmentation and recognition. Consequently, detecting the skew of a document image and correcting it are important issues in realising a practical document reader. In this paper we describe a new algorithm for skew detection. We then compare the performance and results of this skew detection algorithm to other publidhed methods form O'Gorman, Hinds, Le, Baird, Posel and Akuyama. Finally, we discuss the theory of skew detection and the different apporaches taken to solve the problem of skew in documents. The skew correction algorithm we propose has been shown to be extremenly fast, with run times averaging under 0.25 CPU seconds to calculate the angle on the DEC 5000/20 workstation.

A robust and fast skew detection algorithm for generic documents

A robust and fast skew detection algorithm based on hierarchical Hough transform is proposed. It is capable of detecting the skew angle for various document images, including technical articles, postal labels, handwritten text, forms, drawings and bar codes. The algorithm is robust even when black margins introduced by photocopying are present in the image and when the document is scanned at a low resolution of 50 dpi. The algorithm consists of two steps. In the first step we quickly extract the centroids of connected components using a graph data structure. Then, a hierarchical Hough transform (at two different angular resolutions) is applied to the selected centroids. The skew angle corresponds to the location of the highest peak in the Hough space. The performance of the algorithm is shown on a number of document images collected from various application domains. The algorithm is not very sensitive to algorithmic parameters. For an A4 size document image scanned at 50 dpi (typically 413 × 575 pixels), our algorithm is able to detect the skew angle with an accuracy of 0.1° in 0.4s of CPU time on a SunSparc 20 workstation.

Comparative study of skew detection algorithms

Document image processing has become an increasingly important technology in the automation of office documentation tasks. Automatic document scanners such as text readers and optical character recognition (OCR) systems are an essential component of systems capable of those tasks. One of the problems in this field is that the document to be read is not always placed correctly on a flat-bed scanner. This means that the document may be skewed on the scanner bed, resulting in a skewed image. This skew has a detrimental effect on document analysis, document understanding, and character segmentation and recognition. Consequently, detecting the skew of a document image and correcting it are important issues in realizing a practical document reader. We describe a new algorithm for skew detection. We then compare the performance and results of this skew detection algorithm to other published methods from O'Gorman, Hinds, Le, Baird, Postl, and Akiyama. Finally, we discuss the theory of skew detection and the different approaches taken to solve the problem of skew in documents. The skew correction algorithm we propose has been shown to be extremely fast, with run times averaging under 0.25 CPU seconds to calculate the angle on a DEC 5000/20 workstation.