Large Image Repositories Research Articles

Ranking loss and sequestering learning for reducing image search bias in histopathology

Recently, deep learning has started to play an essential role in healthcare applications, including image search in digital pathology. Despite the recent progress in computer vision, significant issues remain for image searching in histopathology archives. A well-known problem is AI bias and lack of generalization. A more particular shortcoming of deep models is the ignorance toward search functionality. The former affects every model, the latter only search and matching. Due to the lack of ranking-based learning, researchers must train models based on the classification error and then use the resultant embedding for image search purposes. Moreover, deep models appear to be prone to internal bias even if using a large image repository of various hospitals. This paper proposes two novel ideas to improve image search performance. First, we use a ranking loss function to guide feature extraction toward the matching-oriented nature of the search. By forcing the model to learn the ranking of matched outputs, the representation learning is customized toward image search instead of learning a class label. Second, we introduce the concept of sequestering learning to enhance the generalization of feature extraction. By excluding the images of the input hospital from the matched outputs, i.e., sequestering the input domain, the institutional bias is reduced. The proposed ideas are implemented and validated through the largest public dataset of whole slide images. The experiments demonstrate superior results compare to the-state-of-art.

Central Cohesion Gradual Hashing for Remote Sensing Image Retrieval

With the recent development of remote sensing technology, large image repositories have been collected. In order to retrieve the desired images of massive remote sensing data sets effectively and efficiently, we propose a novel central cohesion gradual hashing (CCGH) mechanism for remote sensing image retrieval. Firstly, we design a deep hashing model based on ResNet-18 which has a shallow architecture and extracts features of remote sensing imagery effectively and efficiently. Then, we propose a new training model by minimizing a central cohesion loss which guarantees that remote sensing hash codes are as close to their hash code centers as possible. We also adopt a quantization loss which promotes that outputs are binary values. The combination of both loss functions produces highly discriminative hash codes. Finally, a gradual sign-like function is used to reduce quantization errors. By means of the aforementioned developments, our CCGH achieves state-of-the-art accuracy in the task of remote sensing image retrieval. Extensive experiments are conducted on two public remote sensing image data sets. The obtained results support the fact that our newly developed CCGH is competitive with other existing deep hashing methods.

An expandable informatics framework for enhancing central cancer registries with digital pathology specimens, computational imaging tools, and advanced mining capabilities

BackgroundPopulation-based state cancer registries are an authoritative source for cancer statistics in the United States. They routinely collect a variety of data, including patient demographics, primary tumor site, stage at diagnosis, first course of treatment, and survival, on every cancer case that is reported across all U.S. states and territories. The goal of our project is to enrich NCI’s Surveillance, Epidemiology, and End Results (SEER) registry data with high-quality population-based biospecimen data in the form of digital pathology, machine-learning-based classifications, and quantitative histopathology imaging feature sets (referred to here as Pathomics features). Materials and methodsAs part of the project, the underlying informatics infrastructure was designed, tested, and implemented through close collaboration with several participating SEER registries to ensure consistency with registry processes, computational scalability, and ability to support creation of population cohorts that span multiple sites. Utilizing computational imaging algorithms and methods to both generate indices and search for matches makes it possible to reduce inter- and intra-observer inconsistencies and to improve the objectivity with which large image repositories are interrogated. ResultsOur team has created and continues to expand a well-curated repository of high-quality digitized pathology images corresponding to subjects whose data are routinely collected by the collaborating registries. Our team has systematically deployed and tested key, visual analytic methods to facilitate automated creation of population cohorts for epidemiological studies and tools to support visualization of feature clusters and evaluation of whole-slide images. As part of these efforts, we are developing and optimizing advanced search and matching algorithms to facilitate automated, content-based retrieval of digitized specimens based on their underlying image features and staining characteristics. ConclusionTo meet the challenges of this project, we established the analytic pipelines, methods, and workflows to support the expansion and management of a growing repository of high-quality digitized pathology and information-rich, population cohorts containing objective imaging and clinical attributes to facilitate studies that seek to discriminate among different subtypes of disease, stratify patient populations, and perform comparisons of tumor characteristics within and across patient cohorts. We have also successfully developed a suite of tools based on a deep-learning method to perform quantitative characterizations of tumor regions, assess infiltrating lymphocyte distributions, and generate objective nuclear feature measurements. As part of these efforts, our team has implemented reliable methods that enable investigators to systematically search through large repositories to automatically retrieve digitized pathology specimens and correlated clinical data based on their computational signatures.

Deep learning model with low-dimensional random projection for large-scale image search

Developing deep learning models that can scale to large image repositories is increasingly gaining significant efforts in the domain of image search. The current deep neural networks rely on the computational power of accelerators (e.g. GPUs) to tackle the processing limitations associated with features extraction and model training. This paper introduces and investigates a deep model of Convolutional Neural Networks (CNNs) to efficiently extract, index, and retrieve images in the context of large-scale Content-Based Image Retrieval (CBIR). Random Maclaurin projection is used to generate low-dimensional image descriptors and their discriminating efficiency is evaluated on standard image datasets. The scalability of deep architectures is also evaluated on one million image dataset over a High-Performance Computing (HPC) platform, which is assessed in terms of the retrieval accuracy, speed of features extraction and memory costs. Additionally, the controlling GPU kernels of the proposed model are examined under several optimization factors to evaluate their impact on the processing and retrieval performance. The experimental results show the effectiveness of the proposed model in the retrieval accuracy, GPU utilisation, speed of features extraction, and storage of image indexing.

A 4D information system for the exploration of multitemporal images and maps using photogrammetry, web technologies and VR/AR

<p class="VARAbstract">This contribution shows the comparison, investigation, and implementation of different access strategies on multimodal data. The first part of the research is structured as a theoretical part opposing and explaining the terms of conventional access, virtual archival access, and virtual museums while additionally referencing related work. Especially, issues that still persist in repositories like the ambiguity or missing of metadata is pointed out. The second part explains the practical implementation of a workflow from a large image repository to various four-dimensional applications. Mainly, the filtering of images and in the following, the orientation of images is explained. Selection of the relevant images is partly done manually but also with the use of deep convolutional neural networks for image classification. In the following, photogrammetric methods are used for finding the relative orientation between image pairs in a projective frame. For this purpose, an adapted Structure from Motion (SfM) workflow is presented, in which the step of feature detection and matching is replaced by the Radiant-Invariant Feature Transform (RIFT) and Matching On Demand with View Synthesis (MODS). Both methods have been evaluated on a benchmark dataset and performed superior than other approaches. Subsequently, the oriented images are placed interactively and in the future automatically in a 4D browser application showing images, maps, and building models Further usage scenarios are presented in several Virtual Reality (VR) and Augmented Reality (AR) applications. The new representation of the archival data enables spatial and temporal browsing of repositories allowing the research of innovative perspectives and the uncovering of historical details.</p><p>Highlights:</p><ul><li>Strategies for a completely automated workflow from image repositories to four-dimensional (4D) access approaches.</li><li>The orientation of historical images using adapted and evaluated feature matching methods.</li><li>4D access methods for historical images and 3D models using web technologies and Virtual Reality (VR)/Augmented Reality (AR).</li></ul><p> </p>

GEO-INFORMATION TECHNOLOGIES FOR A MULTIMODAL ACCESS ON HISTORICAL PHOTOGRAPHS AND MAPS FOR RESEARCH AND COMMUNICATION IN URBAN HISTORY

Abstract. This contribution shows ongoing interdisciplinary research of the project HistStadt4D, concerning the investigation and development of different multimodal access strategies on large image repositories. The first part of the presented research introduces different methods of access, where classical analogue access stands in contrast to digital access strategies such as online collections, Web3D, Augmented Reality (AR) and Virtual Reality (VR). We discuss the main persisting issues of libraries, advantages of digital methods, and different access tools. The second part shows technologies and workflows used to create various access possibilities. The photogrammetric and geo-informational work serves as a technical basis for a 3D WebGIS as well as multiple AR/VR applications, which require spatial oriented images, object coordinates, and further spatial data. We introduce a research environment that allows art historians spatial access to historical photography, integrating 3D/4D models with photographic documents of the respective architecture. For dissemination of research results in installations and museums, we present fully immersive VR as well as handheld AR applications allowing users a free exploration of historical photography in a spatial setting.

Open access image repositories: high-quality data to enable machine learning research

Originally motivated by the need for research reproducibility and data reuse, large-scale, open access information repositories have become key resources for training and testing of advanced machine learning applications in biomedical and clinical research. To be of value, such repositories must provide large, high-quality data sets, where quality is defined as minimising variance due to data collection protocols and data misrepresentations. Curation is the key to quality. We have constructed a large public access image repository, The Cancer Imaging Archive, dedicated to the promotion of open science to advance the global effort to diagnose and treat cancer. Drawing on this experience and our experience in applying machine learning techniques to the analysis of radiology and pathology image data, we will review the requirements placed on such information repositories by state-of-the-art machine learning applications and how these requirements can be met.

Semantic image retrieval using random forest-based AdaBoost learning

Efficient image retrieval from a large image repository is still a challenging task because of the semantic gap. In this paper, a stride is made towards reducing the semantic gap by proposing an efficient approach using relevance feedback and random forest-based AdaBoost learning. Initially, user feedback is used to move the query point more towards the relevant images. The user feedback is also used to train the random forest classifier, for learning the user retrieval intention. Then AdaBoost learning is used to identify the weak classifiers and to assign more weights to weak classifiers in the weighted majority voting scheme. AdaBoost learning is adopted to overcome the prediction variance of the random forest classifier. The experimental evaluation is performed on two different real world image databases and shows that the proposed approach is more efficient, as an average precision of 95% is achieved in six iterations of relevance feedback.

Interactive Image Contents Search Based on High Dimensional Information Theory

Content-Based Image Retrieval aims at developing techniques that support effective searching and browsing of large image repositories based on image features. The current Content-Based Image Retrieval suffer from the semantic gap. it often leads to distraction in the search. To overcome these disadvantages, Combine the theory of high dimensional information property, we propose different dimensional space's vector angel cosine algorithm based on high dimensional information theory. The experimental results demonstrate the performance of the proposed algorithm.

Automation, machine learning, and artificial intelligence in echocardiography: A brave new world

Automation, machine learning, and artificial intelligence (AI) are changing the landscape of echocardiography providing complimentary tools to physicians to enhance patient care. Multiple vendor software programs have incorporated automation to improve accuracy and efficiency of manual tracings. Automation with longitudinal strain and 3D echocardiography has shown great accuracy and reproducibility allowing the incorporation of these techniques into daily workflow. This will give further experience to nonexpert readers and allow the integration of these essential tools into more echocardiography laboratories. The potential for machine learning in cardiovascular imaging is still being discovered as algorithms are being created, with training on large data sets beyond what traditional statistical reasoning can handle. Deep learning when applied to large image repositories will recognize complex relationships and patterns integrating all properties of the image, which will unlock further connections about the natural history and prognosis of cardiac disease states. The purpose of this review article was to describe the role and current use of automation, machine learning, and AI in echocardiography and discuss potential limitations and challenges of in the future.

Diverse fuzzy c-means for image clustering

Image clustering is a key technique for better accomplishing image annotation and searching in large image repositories. Fuzzy c-means and its variations have achieved excellent performance on image clustering because they allow each image to belong to more than one cluster. However, these methods neglect the relations between different image clusters, and hence often suffer from the “cluster one-sidedness” problem that redundant centers are learned to characterize the same or similar image clusters. To this issue, we propose a diverse fuzzy c-means for image clustering via introducing a novel diversity regularization into the traditional fuzzy c-means objective. This diversity regularization guarantees the learned image cluster centers to be different from each other and to fill the image data space as much as possible. An efficient optimization algorithm is exploited to address the diverse fuzzy c-means objective, which is proved to converge to local optimal solutions and has a satisfactory time complexity. Experiments on synthetic and six image datasets demonstrate the effectiveness of the proposed method as well as the necessity of the diversity regularization.

An efficient image retrieval system with structured query based feature selection and filtering initial level relevant images using range query

Content Based Image Retrieval is a proficient way of storing, managing, indexing, searching, browsing, mining or retrieving images from a large image repository. Most of the researchers are intensely competing for developing an efficient and precise image retrieval system with less time and space constraint. The proposed method creates two different techniques to reduce the space and time constraints. The first method develops an efficient CBIR system by reducing a number of features to obtain an optimal feature subset using SQL query based feature selection for the normalized feature set. The second method uses SQL range query to filter out initial level relevant images and further the Euclidean distance is applied to refine the filtered subspace inorder to obtain the most relevant images. Gray-level co-occurrence matrix, Region based image descriptors and dominant color descriptor are used to extract the features. Elapsed time, retrieval precision and recall are the evaluation metrics used to analyze the performance with other image retrieval systems. The experiment was performed on Corel dataset and it shows superior performance over the previous systems.

Content based Image Retrieval using Query based Feature Reduction with K-means Cluster Index

Content Based Image Retrieval is an active way of storing, searching, browsing, or retrieving images from a large image repository. Researchers are intensely competing for developing efficient and precise image retrieval tool which can be applicable to various image searching devices that runs on different platform. The proposed method develops an efficient Content Based Image Retrieval system by reducing number of features to obtain an optimal feature subset for the normalized feature set using query based feature selection. It also provides indexing for each cluster using k-means clustering. During the query phase, images are searched from the immediate neighboring cluster and also the next closest cluster in order to obtain the true positive images present in other nearby clusters. Gray Level Co-occurrence Matrix as texture features, Region based shape features and dominant color in RGB space are used for extracting the feature vector and Euclidean distance is used as dissimilarity measure. Performance can be compared among existing system by evaluating the retrieval precision and recall. The experiment is performed on Corel dataset and it shows competing performance with previous system.

Data Mining and Knowledge Discovery tools for exploiting big Earth-Observation data

Abstract. The continuous increase in the size of the archives and in the variety and complexity of Earth-Observation (EO) sensors require new methodologies and tools that allow the end-user to access a large image repository, to extract and to infer knowledge about the patterns hidden in the images, to retrieve dynamically a collection of relevant images, and to support the creation of emerging applications (e.g.: change detection, global monitoring, disaster and risk management, image time series, etc.). In this context, we are concerned with providing a platform for data mining and knowledge discovery content from EO archives. The platform’s goal is to implement a communication channel between Payload Ground Segments and the end-user who receives the content of the data coded in an understandable format associated with semantics that is ready for immediate exploitation. It will provide the user with automated tools to explore and understand the content of highly complex images archives. The challenge lies in the extraction of meaningful information and understanding observations of large extended areas, over long periods of time, with a broad variety of EO imaging sensors in synergy with other related measurements and data. The platform is composed of several components such as 1.) ingestion of EO images and related data providing basic features for image analysis, 2.) query engine based on metadata, semantics and image content, 3.) data mining and knowledge discovery tools for supporting the interpretation and understanding of image content, 4.) semantic definition of the image content via machine learning methods. All these components are integrated and supported by a relational database management system, ensuring the integrity and consistency of Terabytes of Earth Observation data.

Interactive browsing of image collections on mobile devices

Image collections are growing at a rapid rate and hence visual information is becoming more and more important. Clearly, these image repositories need to be managed, and tools for effectively and efficiently searching image databases are highly sought after, especially on mobile devices where more and more images are being stored. In this paper, we present an image browsing system for interactive exploration of image collections on mobile devices. Images are arranged so that visually similar images are grouped together while large image repositories become accessible through a hierarchical, browsable tree structure, arranged on a hexagonal lattice. The developed system provides an intuitive and fast interface for navigating through image databases using a variety of touch gestures.

Categorical image retrieval through genetically optimized support vector machines (GOSVM) and hybrid texture features

Content-based image retrieval (CBIR) systems provide a potential solutions of retrieving semantically similar images from large image repositories against any query image. The research community is competing for more efficient and effective methods of content-based image retrieval, so they can be employed in serving time critical applications in scientific and industrial domains. In this paper, we have combined genetic algorithm and support vector machines to reduce the existing gap between high-level semantic content of the images and the information provided by their low-level descriptors. To maximize the performance of proposed technique, an efficient feature extraction method is introduced, which is based on the concept of in-depth texture analysis. To further enhance the capabilities of proposed method, we employed a way through which the risk of mis-associations can be avoided. To justify the effectiveness of the proposed method, we compared it against several popular CBIR techniques and show a significant improvement in terms of accuracy and stability based on Corel image gallery.

Embedding neural networks for semantic association in content based image retrieval

Content based image retrieval (CBIR) systems provide potential solution of retrieving semantically similar images from large image repositories against any query image. The research community are competing for more effective ways of content based image retrieval, so they can be used in serving time critical applications in scientific and industrial domains. In this paper a Neural Network based architecture for content based image retrieval is presented. To enhance the capabilities of proposed work, an efficient feature extraction method is presented which is based on the concept of in-depth texture analysis. For this wavelet packets and Eigen values of Gabor filters are used for image representation purposes. To ensure semantically correct image retrieval, a partial supervised learning scheme is introduced which is based on K-nearest neighbors of a query image, and ensures the retrieval of images in a robust way. To elaborate the effectiveness of the presented work, the proposed method is compared with several existing CBIR systems, and it is proved that the proposed method has performed better then all of the comparative systems.

Large Visual Repository Search with Hash Collision Design Optimization

Visual search over large image repositories in real time is one of the key challenges for applications such as mobile visual query-by-capture, augmented reality, and biometrics-based identification. Search accuracy and response speed are two important performance factors. This article focuses on one of the important elements of this technology that enables large-scale visual search: indexing (or hashing). Indexing is the process of organizing a database of searchable elements into an efficiently searchable configuration. The searchable elements in our case are compact features extracted from images. This article explores a new indexing scheme. The authors optimize the design of a hash-code collision and counting scheme to enable fast search of visual features of MPEG CDVS.

Robust adaptive flow line detection in sewer pipes

As part of a novel approach to automatic sewer inspection, this paper presents a robust algorithm for automatic flow line detection. A large image repository is obtained from about 50,000 m sewers to represent the high variability of real world sewer systems. Automatic image processing combines Canny edge detection, Hough transform for straight lines and cost minimization using Dijkstra's shortest path algorithm. Assuming that flow lines are mostly smoothly connected horizontal structures, piecewise flow line delineation is reduced to a process of selecting adjacent line candidates. Costs are derived from the gap between adjacent candidates and their reliability. A single parameter α enables simple control of the algorithm. The detected flow line may precisely follow the segmented edges (α = 0.0) or minimize gaps at joints (α = 1.0). Both, manual and ground truth-based analysis indicate that α = 0.8 is optimal and independent of the sewer's material. The algorithm forms an essential step to further automation of sewer inspection.

A content based image retrieval system for a biological specimen collection

Digital photography and decreasing cost of storing data in digital form has led to an explosion of large digital image repositories. Since the number of images in image databases can be large (millions in some cases) it is important to develop automated tools to search them. In this paper, we present a content based image retrieval system for a database of parasite specimen images. Unlike most content based image retrieval systems, where the database consists of objects that vary widely in shape and size, the objects in our database are fairly uniform. These objects are characterized by flexible body shapes, but with fairly rigid ends. We define such shapes to be FleBoRE (Flexible Body Rigid Extremities) objects, and present a shape model for this class of objects. We have defined similarity functions to compute the degree of likeness between two FleBoRE objects and developed automated methods to extract them from specimen images. The system has been tested with a collection of parasite images from the Harold W. Manter Laboratory for Parasitology. Empirical and expert-based evaluations show that query by shape approach is effective in retrieving specimens of the same class.