Optical Character Recognition Tools Research Articles

INAPPROPRIATE CONTENT DETECTION IN IMAGE

The proliferation of social media platforms has led to a significant increase in the sharing of memes and other image-based content. However, some of these images may contain inappropriate or offensive text, which can be challenging to detect using traditional methods. Because the open-source existing solutions does not classify the text content in images. This project aims to develop a robust solution for detecting Not Safe for Work (NSFW) text content in images, with a particular focus on social media memes. The objectives of this project are to accurately extract text from images using optical character recognition (OCR) techniques, and to classify the extracted text as either NSFW or Safe for Work (SFW) using a fine-tuned natural language processing (NLP) model. The methodology employed in this project involves several key steps. First, the Keras OCR library is utilized to extract text from input images, as it has demonstrated superior performance compared to other OCR tools for social media images. Next, a preprocessing step is performed to align the extracted text in the correct sequence using coordinates, distance from the image origin, and the Pythagorean theorem. The pre-processed text is then passed to a fine-tuned BERT-BASE-UNCASED language model, which has been trained on a dataset of Reddit posts (NSFW and SFW) to classify the text as either NSFW or SFW. Key Words: OCR, Inappropriate, BERT, NSFW, SFW, Image, Text

Improving accessibility of digitization outputs: EODOPEN project research findings

PurposeThe current predominant delivery format resulting from digitization is PDF, which is not appropriate for the blind, partially sighted and people who read on mobile devices. To meet the needs of both communities, as well as broader ones, alternative file formats are required. With the findings of the eBooks-On-Demand-Network Opening Publications for European Netizens project research, this study aims to improve access to digitized content for these communities.Design/methodology/approachIn 2022, the authors conducted research on the digitization experiences of 13 EODOPEN partners at their organizations. The authors distributed the same sample of scans in English with different characteristics, and in accordance with Web content accessibility guidelines, the authors created 24 criteria to analyze their digitization workflows, output formats and optical character recognition (OCR) quality.FindingsIn this contribution, the authors present the results of a trial implementation among EODOPEN partners regarding their digitization workflows, used delivery file formats and the resulting quality of OCR results, depending on the type of digitization output file format. It was shown that partners using the OCR tool ABBYY FineReader Professional and producing scanning outputs in tagged PDF and PDF/UA formats achieved better results according to set criteria.Research limitations/implicationsThe trial implementations were limited to 13 project partners’ organizations only.Originality/valueThis research paper can be a valuable contribution to the field of massive digitization practices, particularly in terms of improving the accessibility of the output delivery file formats.

초등 학생들의 필기 능력 개선을 위한 딥러닝 기반 광학 문자인식 도구 개발

Objectives The purpose of this study is to investigate how a deep learning-based optical character recognition (OCR) tool can improve students' handwriting. We aim to develop and validate the effectiveness of a handwriting checking tool to counteract the increase in bad handwriting due to the acceleration of digitalization.
 Methods This study began with the development of a notification letter checker tool using Jetson Nano and NAVER CLOVA OCR. The tool was then applied to 20 fifth-grade students for about two months, and pre- and post-training visuoperceptual development tests were conducted to measure changes before and after training.
 Results The results of the study demonstrated that the use of a deep learning-based optical character recognition tool had a positive impact on students' handwriting skills. Improvements in performance were observed in some of the visuoperceptual developmental test items, and substantial changes in handwriting confidence were also observed.
 Conclusions This study confirms the potential of deep learning-based OCR tools as an important tool for improv-ing students' handwriting skills. However, since the study was conducted on a limited sample group, further re-search is needed, and the accumulated Korean handwriting data from this study can be developed into a more ef-fective tool. The tools developed in this study are available on GitHub at https://github.com/jkf87/autostampper.

Delving Deep into NutriScan: Automated Nutrition Table Extraction and Ingredient Recognition

Abstract: Our paper introduces an innovative solution for automating the detection of nutrition tables and recognition of ingredients on packaged food products, addressing the growing demand for tools that facilitate the extraction of essential dietary information from food packaging. The increasing interest in healthy eating and dietary awareness underscores the need for precise and efficient tools to assist consumers and health-conscious individuals in making well-informed food choices. Harnessing state-of-the-art technology, we enhance the accuracy of nutrition table and ingredient detection through fine-tuning the EfficientDet model. We further extract textual content from these tables using PaddleOCR a sophisticated optical character recognition tool. Additionally, we employ regular expressions to meticulously capture crucial nutritional details, including salt or sodium content, carbohydrates, total fat, saturated fat, trans fat, protein values, additives, and allergens found in the product. To empower users in their dietary decision-making, we introduce a dynamic user preference system. Users can tailor their dietary requirements, specifying allergies and preferences for sugar, saturated fat, total fat, or salt content. Our system then evaluates food products and offers personalized recommendations including the nutri-score, indicating their alignment with the user's health goals. Extensive experimentation and evaluation confirm the high accuracy and efficiency of our method in both nutrition table and ingredient recognition. These paper underscores the transformative potential of our approach, which can revolutionize the accessibility of dietary information, encourage healthier eating habits, and promote dietary transparency

Deep learning-based text detection and recognition on architectural floor plans

An important aspect of automatic floor plan analysis is the extraction of textual information, as it is essential for a thorough understanding of the drawing. This paper presents a text extraction approach utilizing a deep learning-based object detection model and state-of-the-art Optical Character Recognition (OCR) methods. The paper contributes to the research community in three ways: First, it introduces additional annotations to existing data sets to encompass text elements. Second, it proposes a specialized data synthesis pipeline, allowing for generating training images that mimic important characteristics of real data. Finally, it documents a comparative study of deep learning-based object detection architectures (Tesseract, EAST, CRAFT, Faster R-CNN, YOLOv5, YOLOR, YOLOv7, and YOLOv8) and OCR tools (PARSEq, MATRN, EasyOCR, and Tesseract) for the task. Results indicate that YOLOv7 yields the best text detection performance (up to 97.5% wmAP) and PARSEq excels in character recognition (85.2% CER). The data sets are made available.

PACMAN: A Framework for Pulse Oximeter Digit Detection and Reading in a Low-Resource Setting

In light of the COVID-19 pandemic, patients were required to manually input their daily oxygen saturation (SpO2) and pulse rate (PR) values into a health monitoring system—unfortunately, such a process trend to be an error in typing. Several studies attempted to detect the physiological value from the captured image using optical character recognition (OCR). However, the technology has limited availability with high cost. Thus, this study aimed to propose a novel framework called PACMAN (Pandemic Accelerated Human-Machine Collaboration) with a low-resource deep learning-based computer vision. We compared state-of-the-art object detection algorithms (scaled YOLOv4, YOLOv5, and YOLOR), including the commercial OCR tools for digit recognition on the captured images from the pulse oximeter display. All images were derived from crowdsourced data collection with varying quality and alignment. YOLOv5 was the best-performing model against the given model comparison across all datasets, notably the correctly orientated image dataset. We further improved the model performance with the digits auto-orientation algorithm and applied a clustering algorithm to extract SpO2 and PR values. The accuracy performance of YOLOv5 with the implementations was approximately 81.0-89.5%, which was enhanced compared to without any additional implementation. Accordingly, this study highlighted the completion of the PACMAN framework to detect and read digits in real-world datasets. The proposed framework has been currently integrated into the patient monitoring system utilized by hospitals nationwide.

IUKL Smart Car Parking App Using Automatic License Plate Recognition and Raspberry Pi

This Paper outlines the development of the IUKL Smart Parking mobile app developed for the Infrastructure University of Kuala Lumpur, Malaysia. The Android mobile application was developed using android studio and allows users to make reservations for the IUKL campus parking facilities. Furthermore, an automatic license plate recognition algorithm was developed using a Raspberry Pi computer and tested using sample license plate images displayed on a smartphone. The license plate detection algorithm was implemented using Python as the programming language and the Python library Python-tesseract, which is an optical character recognition tool for Python. The automatic license plate recognition technique was created using this Python module. Additionally, the algorithm was tested using sample license plate images and this was accomplished using images displayed from a mobile device and yielded mainly accurate results, though not 100%. The results were displayed by emitting different LED colored lights, “Red” was emitted when a plate number was not identified, and “Green” was emitted when the plate number was identified successfully. Moreover, a database was developed using PhpMyAdmin which consists of data regarding users and parking lots. This was done to solve the issue whereby no information was recorded regarding the availability of parking spaces at the IUKL campus. This paper can also serve as a template and guide for the complete implementation of a smart parking system in a university-based environment.

ALICE Software: Machine learning & computer vision for automatic label extraction

Insects make up over 70% of the world's known species (Resh and Carde 2009). This is well represented in collections across the world, with the Natural History Museum's pinned insect collection alone making up nearly 37% of the museum's remarkable 80 million specimen collection. Thus, this extraordinary dataset is a major focus of digitisation efforts here at the Museum. While hardware developments have seen digitisation processes significantly improve and speed up (Blagoderov et al. 2017), we now concentrate on the latest software and explore whether machine learning can lend a bigger hand in accelerating our digitisation of pinned insects. Traditionally, the digitisation of pinned specimens involves the removal of labels (as well as any supplementary specimen miscellanies) prior to photographing the specimen. In order to document labels, this process is typically followed by additional photographs of labels as the label documentation is often obstructed by their stacking on a pin, the specimen and additional specimen material, or the pin itself. However, these steps not only slow down the process of digitisation but also increase the risk of specimen damage. This encouraged the team at the Natural History Museum to develop a novel setup that would bypass the need for removing labels during digitisation. This led to the development of ALICE (Angled Label Image Capture and Extraction) (Dupont and Price 2019). ALICE is a multi-camera setup designed to capture images of angled specimens, which allows users to get a full picture of a specimen in a collection, including that of the label and the text within. Specifically, ALICE involves four cameras angled at different viewpoints in order to capture label information, as well as two additional cameras providing a lateral and dorsal view of the specimen. By viewing all the images taken from one specimen simultaneously, we can obtain a full account of the labels and of the specimen, despite any obstructions. This setup notably accelerates parts of the digitisation process, sometimes by up to 7 times (Price et al. 2019). Furthermore, ALICE presents the opportunity to incorporate machine learning and computer vision techniques to create a software that automates the process of transcribing the information contained on labels. Automatically transcribing text (whether typed or handwritten) from label images, leads to the topic of Optical Character Recognition (OCR). Regardless of any obstructions to the labels, standard OCR methods will often fail at detecting text from these angled specimens if no preprocessing is done. This was emphasised in Bieniecki et al. (2007), which showed that some standard OCR algorithms were highly sensitive to geometric distortions such as bad angles. Therefore, in our latest ALICE software, we take on a 5-step approach that segments labels and merges them together using machine learning and computer vision, before turning to standard OCR tools to transcribe texts. Our 5-step framework is described as follows: Label segmentation with Convolutional Neural Networks (CNNs), Label corner approximation, Perspective transformation of label, followed by image registration with a given template, Label-image merging using an averaging technique, OCR on merged label. Label segmentation with Convolutional Neural Networks (CNNs), Label corner approximation, Perspective transformation of label, followed by image registration with a given template, Label-image merging using an averaging technique, OCR on merged label. While ALICE aims to reveal specimen labels using a multi-camera setup, we ask ourselves whether an alternative approach can also be taken. This takes us to the next phase of our digitisation acceleration research on smarter cameras with cobots (collaborative robots) and the associated software. We explore the potential of a single camera setup that is capable of zooming into labels. Where intelligence was incorporated post-processing with ALICE, using cobots, we can incorporate machine learning and computer vision techniques in-situ, in order to extract label information. This all forms the focus of our presentation.

Simple is not Easy: A Simple Strong Baseline for TextVQA and TextCaps

Texts appearing in daily scenes that can be recognized by OCR (Optical Character Recognition) tools contain significant information, such as street name, product brand and prices. Two tasks -- text-based visual question answering and text-based image captioning, with a text extension from existing vision-language applications, are catching on rapidly. To address these problems, many sophisticated multi-modality encoding frameworks (such as heterogeneous graph structure) are being used. In this paper, we argue that a simple attention mechanism can do the same or even better job without any bells and whistles. Under this mechanism, we simply split OCR token features into separate visual- and linguistic-attention branches, and send them to a popular Transformer decoder to generate answers or captions. Surprisingly, we find this simple baseline model is rather strong -- it consistently outperforms state-of-the-art (SOTA) models on two popular benchmarks, TextVQA and all three tasks of ST-VQA, although these SOTA models use far more complex encoding mechanisms. Transferring it to text-based image captioning, we also surpass the TextCaps Challenge 2020 winner. We wish this work to set the new baseline for these two OCR text related applications and to inspire new thinking of multi-modality encoder design. Code is available at https://github.com/ZephyrZhuQi/ssbaseline

Quote examiner: verifying quoted images using web-based text similarity

Over the last few years, there has been a rapid growth in digital data. Images with quotes are spreading virally through online social media platforms. Misquotes found online often spread like a forest fire through social media, which highlights the lack of responsibility of the web users when circulating poorly cited quotes. Thus, it is important to authenticate the content contained in the images being circulated online. So, there is a need to retrieve the information within such textual images to verify quotes before its usage in order to differentiate a fake or misquote from an authentic one. Optical Character Recognition (OCR) is used in this paper, for converting textual images into readable text format, but none of the OCR tools are perfect in extracting information from the images accurately. In this paper, a method of post-processing on the retrieved text to improve the accuracy of the detected text from images has been proposed. Google Cloud Vision has been used for recognizing text from images. It has also been observed that using post-processing on the extracted text improved the accuracy of text recognition by 3.5% approximately. A web-based text similarity approach (URLs and domain name) has been used to examine the authenticity of the content of the quoted images. Approximately, 96.26% accuracy has been achieved in classifying quoted images as verified or misquoted. Also, a ground truth dataset of authentic site names has been created. In this research, images with quotes by famous celebrities and global leaders have been used. A comparative analysis has been performed to show the effectiveness of our proposed algorithm.

Polarity Identification for Handwritten Text in Multilingual Documents Using Open Source Optical Character Recognition Tools

Sentimental analysis is a prerequisite for many applications. We propose a model which scans handwritten text in English and Kannada languages by a CamScanner and then translated into editable text by using various Open Source Optical Character Recognition tools. The performances of different OCRs are analyzed and tabulated. Sentimental analysis is performed on the statements written in both English and Kannada languages using Wordnet, Algorithmia Rest API and local dictionaries and we have obtained the satisfied results. The same sentimental analysis module is also applied on customer reviews for the mobile product and reviews are taken from Amazon Web Services. The opinion of the customer about the product can be identified correctly.

Psc2code

Programming screencasts have become a pervasive resource on the Internet, which help developers learn new programming technologies or skills. The source code in programming screencasts is an important and valuable information for developers. But the streaming nature of programming screencasts (i.e., a sequence of screen-captured images) limits the ways that developers can interact with the source code in the screencasts. Many studies use the Optical Character Recognition (OCR) technique to convert screen images (also referred to as video frames) into textual content, which can then be indexed and searched easily. However, noisy screen images significantly affect the quality of source code extracted by OCR, for example, no-code frames (e.g., PowerPoint slides, web pages of API specification), non-code regions (e.g., Package Explorer view, Console view), and noisy code regions with code in completion suggestion popups. Furthermore, due to the code characteristics (e.g., long compound identifiers like ItemListener), even professional OCR tools cannot extract source code without errors from screen images. The noisy OCRed source code will negatively affect the downstream applications, such as the effective search and navigation of the source code content in programming screencasts. In this article, we propose an approach named psc2code to denoise the process of extracting source code from programming screencasts. First, psc2code leverages the Convolutional Neural Network (CNN) based image classification to remove non-code and noisy-code frames. Then, psc2code performs edge detection and clustering-based image segmentation to detect sub-windows in a code frame, and based on the detected sub-windows, it identifies and crops the screen region that is most likely to be a code editor. Finally, psc2code calls the API of a professional OCR tool to extract source code from the cropped code regions and leverages the OCRed cross-frame information in the programming screencast and the statistical language model of a large corpus of source code to correct errors in the OCRed source code. We conduct an experiment on 1,142 programming screencasts from YouTube. We find that our CNN-based image classification technique can effectively remove the non-code and noisy-code frames, which achieves an F1-score of 0.95 on the valid code frames. We also find that psc2code can significantly improve the quality of the OCRed source code by truly correcting about half of incorrectly OCRed words. Based on the source code denoised by psc2code , we implement two applications: (1) a programming screencast search engine; (2) an interaction-enhanced programming screencast watching tool. Based on the source code extracted from the 1,142 collected programming screencasts, our experiments show that our programming screencast search engine achieves the precision@5, 10, and 20 of 0.93, 0.81, and 0.63, respectively. We also conduct a user study of our interaction-enhanced programming screencast watching tool with 10 participants. This user study shows that our interaction-enhanced watching tool can help participants learn the knowledge in the programming video more efficiently and effectively.

HDPA: historical document processing and analysis framework

Nowadays, the accessibility of digitized historical documents is extremely important to facilitate fast and efficient retrieval of historical information and knowledge extraction from such data. To provide such functionality, it is necessary to convert document images into plain text using optical character recognition (OCR). Many OCR related methods and tools have been proposed, however, they are often too complicated for a standard user, some important parts are missing or they are not available in free versions. Therefore, this paper describes a complex and flexible web framework for historical document manipulation and analysis with the main focus on OCR. The framework contains eight modules to facilitate three main tasks: image pre-processing and segmentation, creation of data for OCR model training and the OCR itself. This framework is freely available for non commercial purposes. We have experimentally evaluated this framework on real data and we have shown that this system is efficient and can save human labour in the process of annotated data preparation. Moreover, we have reached state-of-the-art OCR results.

Rendszámtábla azonosító rendszer tervezése Python és OpenCV használatával

This article describes the structure of an implementation of a license plate identification system capable of reading the characters of a license plate from a digital image. It also collects the license plate detecting and character recognizing methods mentioned in the article, and we can read about rules regarding hungary's registered license plates as well. We can gain insight into the world of computer vision, as well as understand what image manipulation procedures a digital image must go through in order to extract characters from it. The system was written in Python and also used methods provided by OpenCV, as well as the Python-Tesseract optical character recognition tool to identify license plate characters.

THE COMPARISONS OF OCR TOOLS: A CONVERSION CASE IN THE MALAYSIAN HANSARD CORPUS DEVELOPMENT

Optical Character Recognition (OCR) is a tool in computational technology that allows a recognition of printed characters by manipulating photoelectric devices and computer software. It runs by converting images or texts that are scanned beforehand into machine-readable and editable texts. There are a various numbers of OCR tools in the market for commercial and research use, which are obtainable for free or restrained with purchases. An OCR tool is able to enhance the accuracy of the results which as well relies on pre-processing and subdivision of algorithms. This study intends to investigate the performances of OCR tools in converting the Parliamentary Reports of Hansard Malaysia for developing the Malaysian Hansard Corpus (MHC). By comparing four OCR tools, the study has converted ten reports of Parliamentary Reports which contains a number of 62 pages to see the conversion accuracy and error rate of each conversion tool. In this study, all of the tools are manipulated to convert Adobe Portable Document Format (PDF) files into Plain Text File (txt). The objective of this study is to give an overview based on accuracy and error rate of how each OCR tools essentially works and how it can be utilized to provide assistance towards corpus building. The study indicates that each tool possesses a variety of accuracy and error rates to convert the whole documents from PDF into txt or plain text files. The study proposes that a step of corpus building can be made easier and manageable when a researcher understands the way an OCR tool works in order to choose the best OCR tool prior to the outset of the corpus development.

An Accuracy Examination of OCR Tools

In this research paper, the authors have aimed to do a comparative study of optical character recognition using different open source OCR tools. Optical character recognition (OCR) method has been used in extracting the text from images. OCR has various applications which include extracting text from any document or image or involves just for reading and processing the text available in digital form. The accuracy of OCR can be dependent on text segmentation and pre-processing algorithms. Sometimes it is difficult to retrieve text from the image because of different size, style, orientation, a complex background of image etc. From vehicle number plate the authors tried to extract vehicle number by using various OCR tools like Tesseract, GOCR, Ocrad and Tensor flow. The authors in this research paper have tried to diagnose the best possible method for optical character recognition and have provided with a comparative analysis of their accuracy

An Efficient Algorithm to Find the Height of a Text Line and Overcome Overlapped and Broken Line Problem during Segmentation

Line segmentation is a critical phase of the Optical Character Recognition (OCR) which separates the individual lines from the image documents. The accuracy rate of the OCR tool is directly proportional to the line segmentation accuracy followed by the word/character segmentation. In this context, an algorithm, named height_based_segmentation is proposed for the text line segmentation of printed Odia documents. The proposed algorithm finds the average height of a text line and it helps to minimize the overlapped text line cases. The algorithm also includes post-processing steps to combine the modifier zone with the base zone. The performance of the algorithm is evaluated through the ground truth and also by comparing it with the existing segmentation approaches.

Smart pens to assist fibre optic sensors research: Evaluating OCR tools

Reproducibility is one of the big challenges in research. Lab notebooks have been used to record data, observations and relevant remarks of the research processes. Smart pens are devices that record audio, handwriting notes thanks to micro patterned paper, and generate pdf files and audio enriched notes (pencasts). The handwriting notes can then be processed using optical character recognition (OCR) software to generate digital documents allowing the user to archive and access these notes in an easier way. However, OCR for handwriting is still a challenge in the computer vision research area. In this paper, we report the evaluation results of different OCR tools when processing handwriting notes written by 7 participants focusing on the main elements and technical vocabulary identified in fibre optic sensors research.

Development and customization of in-house developed OCR and its evaluation

PurposeThe purpose of this paper is to develop a text extraction tool for scanned documents that would extract text and build the keywords corpus and key phrases corpus for the document without manual intervention.Design/methodology/approachFor text extraction from scanned documents, a Web-based optical character recognition (OCR) tool was developed. OCR is a well-established technology, so to develop the OCR, Microsoft Office document imaging tools were used. To account for the commonly encountered problem of skew being introduced, a method to detect and correct the skew introduced in the scanned documents was developed and integrated with the tool. The OCR tool was customized to build keywords and key phrases corpus for every document.FindingsThe developed tool was evaluated using a 100 document corpus to test the various properties of OCR. The tool had above 99 per cent word read accuracy for text only image documents. The customization of the OCR was tested with samples of Microfiches, sample of Journal pages from back volumes and samples from newspaper clips and the results are discussed in the summary. The tool was found to be useful for text extraction and processing.Social implicationsThe scanned documents are converted to keywords and key phrases corpus. The tool could be used to build metadata for scanned documents without manual intervention.Originality/valueThe tool is used to convert unstructured data (in the form of image documents) to structured data (the document is converted into keywords, and key phrases database). In addition, the image document is converted to editable and searchable document.

Getting fast, free, and anonymous answers to questions asked by people with visual impairments

Much of the information we encounter in daily life is visual. For people with visual impairments, this information is not inherently accessible. Technology can be used to provide access to this visual information, through automated tools like currency identifiers, object recognizers, and optical character recognition, which allow for immediate access to visual information in a users' environment. However, these tools are often are limited to a specific domain, and can fail in non-ideal conditions (e.g., an optical character recognition tool trying to identify handwritten text). By incorporating people into the information access pipeline, human-powered tools can allow for more complex information to be extracted from the user's surroundings. Human-powered access tools connect people who have disabilities to other people who can access information on their behalf. While people may not be as quick as automated tools, the person answering can make inferences based on prior knowledge and experiences, ask for clarification, and reason over the information provided. In this article, we explore the long-term public deployment and lessons learned from VizWiz Social, a human-powered access tool that connects people with visual impairments to sighted workers or friends and family members who can answer their visual questions. By analyzing the use of this service, and running controlled experiments to determine how users value different answer sources, we have been able to build upon the original VizWiz design to create social microvolunteering, a method for getting fast, free, and anonymous answers to visual questions.