Automatic Image Understanding Research Articles

SARCASTIC v2.0—High-Performance SAR Simulation for Next-Generation ATR Systems

Synthetic aperture radar has been a mainstay of the remote sensing field for many years, with a wide range of applications across both civilian and military contexts. However, the lack of openly available datasets of comparable size and quality to those available for optical imagery has severely hampered work on open problems such as automatic target recognition, image understanding and inverse modelling. This paper presents a simulation and analysis framework based on the upgraded SARCASTIC v2.0 engine, along with a selection of case studies demonstrating its application to well-known and novel problems. In particular, we demonstrate that SARCASTIC v2.0 is capable of supporting complex phase-dependent processing such as interferometric height extraction whilst maintaining near-realtime performance on complex scenes.

Object Detection Method for High Resolution Remote Sensing Imagery Based on Convolutional Neural Networks with Optimal Object Anchor Scales

ABSTRACT Object detection is a key technique for automatic information extraction, analysis and understanding of high spatial resolution remote-sensing images (HSRIs). Object anchor scales are a critical factor for object detection of HSRIs based on convolutional neural networks (CNNs). With respect to adaptively obtaining optimal object anchor scales for object detection of HSRIs, this paper proposes a novel object detection method for HSRIs based on CNNs with optimal object anchor scales. First, optimal object anchor scales for object detection of HSRIs are obtained using an adaptive object-scale learning operator. Then, a CNN object detection framework for HSRIs is designed based on optimal object anchor scales. Using multiple-object detection datasets, the proposed method is compared with some state-of-the-art object detection algorithms. Experimental results show that the proposed method can achieve 90.42%, 91.98% and 85.07% mAP on WHU-RSONE, UCAS-AOD and HSRC2016, respectively, and outperforms state-of-the-art object detection algorithms.

Data-driven floor plan understanding in rural residential buildings via deep recognition

Automatic understanding of floor plan images is a key component of various applications. Due to the style diversity of rural housing design, the latest learning-based approaches cannot achieve satisfactory recognition results. In this paper, we present a new framework for parsing floor plans of rural residence that combines semantic neural networks with a post-processed room segmentation. First, we take case studies from typical residential buildings in China’s rural areas and provide a novel image dataset, called RuralHomeData, containing 800 rural residence floor plans with accurate man-made annotations. Based on the dataset, we propose a new deep learning-based recognition framework using a joint neural network to predict the geometric elements and text information on the floor plan simultaneously. Our insight is that walls and openings (doors and windows) are the basic elements corresponding to the room boundary that a closed 1D loop must form a certain room. Then the semantic information (e.g., the room function) of room regions can be obtained through text detection and identification. Furthermore, we use the MIQP algorithm to divide the area containing multiple room type texts into multiple room areas. Finally, the input floor plan can be transformed into a room layout graph with room attributes and adjacent relationships. The proposed algorithm has been tested on both urban and rural datasets, and the experimental results demonstrate our efficiency and robustness compared with the state-of-the-art methods.

Combination of Color and Focus Segmentation for Medical Images with Low Depth-of-Field

Abstract Image segmentation plays an increasingly important role in image processing. It allows for various applications including the analysis of an image for automatic image understanding and the integration of complementary data. During vascular surgeries, the blood flow in the vessels has to be checked constantly, which could be facilitated by a segmentation of the affected vessels. The segmentation of medical images is still done manually, which depends on the surgeon’s experience and is time-consuming. As a result, there is a growing need for automatic image segmentation methods. We propose an unsupervised method to detect the regions of no interest (RONI) in intraoperative images with low depth-of-field (DOF). The proposed method is divided into three steps. First, a color segmentation using a clustering algorithm is performed. In a second step, we assume that the regions of interest (ROI) are in focus whereas the RONI are unfocused. This allows us to segment the image using an edge-based focus measure. Finally, we combine the focused edges with the color RONI to determine the final segmentation result. When tested on different intraoperative images of aneurysm clipping surgeries, the algorithm is able to segment most of the RONI not belonging to the pulsating vessel of interest. Surgical instruments like the metallic clips can also be excluded. Although the image data for the validation of the proposed method is limited to one intraoperative video, a proof of concept is demonstrated.

A review of automated image understanding within 3D baggage computed tomography security screening.

Baggage inspection is the principal safeguard against the transportation of prohibited and potentially dangerous materials at airport security checkpoints. Although traditionally performed by 2D X-ray based scanning, increasingly stringent security regulations have led to a growing demand for more advanced imaging technologies. The role of X-ray Computed Tomography is thus rapidly expanding beyond the traditional materials-based detection of explosives. The development of computer vision and image processing techniques for the automated understanding of 3D baggage-CT imagery is however, complicated by poor image resolutions, image clutter and high levels of noise and artefacts. We discuss the recent and most pertinent advancements and identify topics for future research within the challenging domain of automated image understanding for baggage security screening CT.

Reliable and reproducible classification system for scoliotic radiograph using image processing techniques.

Scoliosis classification is useful for guiding the treatment and testing the clinical outcome. State-of-the-art classification procedures are inherently unreliable and non-reproducible due to technical and human judgmental error. In the current diagnostic system each examiner will have diagrammatic summary of classification procedure, number of scoliosis curves, apex level, etc. It is very difficult to define the required anatomical parameters in the noisy radiographs. The classification system demands automatic image understanding system. The proposed automated classification procedures extracts the anatomical features using image processing and applies classification procedures based on computer assisted algorithms. The reliability and reproducibility of the proposed computerized image understanding system are compared with manual and computer assisted system using Kappa values.

Surgical Vision

The emergence of Minimal Access Surgery (MAS) as a paradigm in modern healthcare treatment has created new challenges and opportunities for automated image understanding and computer vision. In MAS, images recovered from inside the body using specialized devices are used to visualize and operate on the surgical site but they can also be used to computationally infer in vivo 3D tissue surface shape, soft-tissue morphology, and surgical instrument motion. This information is important for facilitating in vivo biophotonic imaging modalities where the interaction between light and tissue is used to infer the structural and functional properties of the tissue. This article provides a review of the literature for computer vision and image understanding techniques applied to MAS images. The focus of this article is to elucidate a perspective on how computer vision techniques can be used to support and enhance the capabilities of biophotonic imaging modalities during surgery. Note that while MAS encompasses a variety of surgical specializations this review does not involve procedures performed in the interventional suite. The review has been carried out based on searches in the PubMed and IEEE databases using the article's keywords.

IBM smart surveillance system (S3): event based video surveillance system with an open and extensible framework

The increasing need for sophisticated surveillance systems and the move to a digital infrastructure has transformed surveillance into a large scale data analysis and management challenge. Smart surveillance systems use automatic image understanding techniques to extract information from the surveillance data. While the majority of the research and commercial systems have focused on the information extraction aspect of the challenge, very few systems have explored the use of extracted information in the search, retrieval, data management and investigation context. The IBM smart surveillance system (S3) is one of the few advanced surveillance systems which provides not only the capability to automatically monitor a scene but also the capability to manage the surveillance data, perform event based retrieval, receive real time event alerts thru standard web infrastructure and extract long term statistical patterns of activity. The IBM S3 is easily customized to fit the requirements of different applications by using an open-standards based architecture for surveillance.

Image languages in intelligent radiological palm diagnostics

This paper presents a new technique of computer-aided analysis and recognition of pathological wrist bone lesions. This method uses artificial intelligence (AI) techniques and mathematical linguistics allowing to evaluate automatically and analyse the structure of the said bones, based on palm radiological images. Possibilities of computer interpretation of selected images, based on the methodology of automatic medical image understanding, as introduced by the authors, were created owing to the introduction of an original relational description of individual palm (wrist) bones. This description has been built with the use of graph linguistic formalisms already applied in artificial intelligence. These were, however, developed and adjusted to the needs of automatic medical image understanding in earlier works of the authors, as specified in the bibliography section of this paper. The research described in this paper has demonstrated that the for needs of palm (wrist) bone diagnostics, specialist linguistic tools such as expansive graph grammars and EDT-label graphs are particularly well-suited. Defining a graph image language adjusted to the specific features of the scientific problem here-described allowed for a semantic description of correct palm bone structures (with consideration to idiosyncratic features). It also enabled interpretation of images showing some in-born lesions, such as additional bones; or acquired lesions such as their incorrect junctions resulting from injuries and synostoses.

Automated interpretation of sub-pixel vegetation from IRS LISS-II images

Satellite sensor data are important for monitoring and assessment of natural resources. As vegetation is one of the most valuable natural resources, automated interpretation of vegetative cover from satellite images is prerequisite for various applications and decision processes. This paper defines a system that classifies as well as interprets vegetation from satellite images automatically. The system applies a knowledge-based approach wherein features are represented by linguistic variables in terms of their fuzzy labels. The accuracy of the system has been found to be more than 95% for hard class and more than 85% in the case of sub-pixel classification. Thus, it can be concluded that the approach adopted can be utilized in developing any automated image understanding system.

Automated interpretation of sub-pixel water cover from satellite images

Accurate information and estimation of area under water cover is important for different engineering applications and decision processes. This paper defines a method for direct interpretation and estimation of the sub-pixel water cover from satellite images. It is a fuzzy knowledge based approach wherein features are represented by Linguistic Variables. In this study, more than 95% accuracy has been found in case of hard classification and 85% in case of sub-pixel estimation. The proposed approach can be utilised in an automated image understanding system.

A word extraction algorithm for machine-printed documents using a 3D neighborhood graph model

Automatic character recognition and image understanding of a given paper document are the main objectives of the computer vision field. For these problems, a basic step is to isolate characters and group words from these isolated characters. In this paper, we propose a new method for extracting characters from a mixed text/graphic machine-printed document and an algorithm for distinguishing words from the isolated characters. For extracting characters, we exploit several features (size, elongation, and density) of characters and propose a characteristic value for classification using the run-length frequency of the image component. In the context of word grouping, previous works have largely been concerned with words which are placed on a horizontal or vertical line. Our word grouping algorithm can group words which are on inclined lines, intersecting lines, and even curved lines. To do this, we introduce the 3D neighborhood graph model which is very useful and efficient for character classification and word grouping. In the 3D neighborhood graph model, each connected component of a text image segment is mapped onto 3D space according to the area of the bounding box and positional information from the document. We conducted tests with more than 20 English documents and more than ten oriental documents scanned from books, brochures, and magazines. Experimental results show that more than 95% of words are successfully extracted from general documents, even in very complicated oriental documents.

HOUSE EXTRACTION WITH MULTIRESOLUTION ANALYSIS AND INFORMATION FUSION

ABSTRACT Houses are key elements both in aerial photographs and maps. Their extraction has significance not only to military, but also to automatical relative orientation, automatical absolute orientation, feature-based matching, systematic description, automatic data aquisition, structure analysis, image understanding, digital mapping, and construction of urban geographic information system (UGIS). For this purpose, this paper mainly addresses: (1) a brief introduction to the multiresolution analysis of wavelet theory and its difference form Laplacian pyramid, (2) how to generate the multiresolution images and edge features, and (3) how to employ these multiresolution images and edge features to extract the houses from aerial photographs by multiresolution structure and information fusion.

Polarization vision: a new sensory approach to image understanding

Overviewed are recent results of a new general approach to image understanding and computer vision utilizing the sensing of polarization of light. Whereas human vision is oblivious to components of light polarization, polarization parameters of light are shown to provide an important visual extension to intensity and color significantly expanding the application potential for image understanding. A physical state of polarization can be visualized directly in human terms as a particular hue and saturation, and this paper utilizes such a scheme presenting images of ordinary scenes as never seen before by humans in the domain of polarization vision. Metaphorically, humans are ‘color blind’ with respect to the perception of polarization, and even though this does not appear to inhibit human visual performance, we show how polarization vision is a sensory augmentation that can significantly enhance both automated image understanding and even possibly improve human visual performance itself under certain conditions. Sensors, called polarization cameras, have been developed that automatically sense components of partial linear polarization and computationally process these components to produce polarization images. Prototypes of different polarization camera sensors have been presented in earlier literature. A recent advancement in the design of polarization cameras has made it possible to interface low-cost modular components with almost any existing imaging device converting it into an automatic polarization camera. This compatibility with small portable imaging devices is making polarization imaging for the first time accessible to a number of application areas outside the laboratory, both outdoors and underwater, revealing polarization vision as a vast new visually augmented domain with unique capabilities. This paper presents various results from three on-going field applications: natural object recognition, inspection of ship hulls for damage, and marine biology.

Towards automated image understanding

In this paper, we discuss an approach to the pattern synthesis and analysis of biological shapes which show a lot of variability, but at the same time also show a characteristic structure. This structure is captured by way of ‘shape classes' which are constructed via a deformable template. Examples of pattern analysis in both two and three dimensions are presented. We define what we call intrinsic and extrinsic understanding of images and apply this to the detection of abnormalities.

Towards automated image understanding

Towards automated image understanding

A knowledge acquisition environment for scene analysis

This paper describes a knowledge acquisition environment under development to help capture expertise from domain experts involved in analysing scenes from aerial imagery. The research is important because automated image understanding systems are increasingly relying on expert knowledge to help analyse objects and control the analysis process. It is desirable to enable the domain experts to enter and manipulate the domain knowledge directly. The research described is based on the concept of an integrated knowledge acquisition environment (KAE). The goal is to integrate the domain inputs, the translation into internal representations and the actual execution and feedback. The KAE contains a collection of computer-based tools facilitating: viewing and editing domain knowledge in both textual and graphic format (analysts tend to be visually oriented), knowledge base execution and testing, and expert system performance analysis.