Multidimensional Image Processing Research Articles

Image scanning microscopy reconstruction by autocorrelation inversion

Abstract Confocal laser scanning microscopy (CLSM) stands out as one of the most widely used microscopy techniques thanks to its three-dimensional imaging capability and its sub-diffraction spatial resolution, achieved through the closure of a pinhole in front of a single-element detector. However, the pinhole also rejects useful photons, and beating the diffraction limit comes at the price of irremediably compromising the signal-to-noise ratio (SNR) of the data. Image scanning microscopy (ISM) emerged as the rational evolution of CLSM, exploiting a small array detector in place of the pinhole and the single-element detector. Each sensitive element is small enough to achieve sub-diffraction resolution through the confocal effect, but the size of the whole detector is large enough to guarantee excellent collection efficiency and SNR. However, the raw data produced by an ISM setup consists of a 4D dataset, which can be seen as a set of confocal-like images. Thus, fusing the dataset into a single super-resolved image requires a dedicated reconstruction algorithm. Conventional methods are multi-image deconvolution, which requires prior knowledge of the system point spread functions (PSFs), or adaptive pixel reassignment (APR), which is effective only on a limited range of experimental conditions. In this work, we describe and validate a novel concept for ISM image reconstruction based on autocorrelation inversion. We leverage unique properties of the autocorrelation to discard low-frequency components and maximize the resolution of the reconstructed image without any assumption on the image or any knowledge of the PSF. Our results push the quality of the ISM reconstruction beyond the level provided by APR and open new perspectives for multi-dimensional image processing.

ImageM: a user-friendly interface for the processing of multi-dimensional images with Matlab

Modern imaging devices provide a wealth of data often organized as images with many dimensions, such as 2D/3D, time and channel. Matlab is an efficient software solution for image processing, but it lacks many features facilitating the interactive interpretation of image data, such as a user-friendly image visualization, or the management of image meta-data (e.g. spatial calibration), thus limiting its application to bio-image analysis. The ImageM application proposes an integrated user interface that facilitates the processing and the analysis of multi-dimensional images within the Matlab environment. It provides a user-friendly visualization of multi-dimensional images, a collection of image processing algorithms and methods for analysis of images, the management of spatial calibration, and facilities for the analysis of multi-variate images. ImageM can also be run on the open source alternative software to Matlab, Octave. ImageM is freely distributed on GitHub: https://github.com/mattools/ImageM.

MS-Net: A lightweight separable ConvNet for multi-dimensional image processing

As the core technology of deep learning, convolutional neural networks have been widely applied in a variety of computer vision tasks and have achieved state-of-the-art performance. However, it’s difficult and inefficient for them to deal with high dimensional image signals due to the dramatic increase of training parameters. In this paper, we present a lightweight and efficient MS-Net for the multi-dimensional(MD) image processing, which provides a promising way to handle MD images, especially for devices with limited computational capacity. It takes advantage of a series of one dimensional convolution kernels and introduces a separable structure in the ConvNet throughout the learning process to handle MD image signals. Meanwhile, multiple group convolutions with kernel size 1 × 1 are used to extract channel information. Then the information of each dimension and channel is fused by a fusion module to extract the complete image features. Thus the proposed MS-Net significantly reduces the training complexity, parameters and memory cost. The proposed MS-Net is evaluated on both 2D and 3D benchmarks CIFAR-10, CIFAR-100 and KTH. Extensive experimental results show that the MS-Net achieves competitive performance with greatly reduced computational and memory cost compared with the state-of-the-art ConvNet models.

The ImageJ ecosystem: Open-source software for image visualization, processing, and analysis.

For decades, biologists have relied on software to visualize and interpret imaging data. As techniques for acquiring images increase in complexity, resulting in larger multidimensional datasets, imaging software must adapt. ImageJ is an open-source image analysis software platform that has aided researchers with a variety of image analysis applications, driven mainly by engaged and collaborative user and developer communities. The close collaboration between programmers and users has resulted in adaptations to accommodate new challenges in image analysis that address the needs of ImageJ's diverse user base. ImageJ consists of many components, some relevant primarily for developers and a vast collection of user-centric plugins. It is available in many forms, including the widely used Fiji distribution. We refer to this entire ImageJ codebase and community as the ImageJ ecosystem. Here we review the core features of this ecosystem and highlight how ImageJ has responded to imaging technology advancements with new plugins and tools in recent years. These plugins and tools have been developed to address user needs in several areas such as visualization, segmentation, and tracking of biological entities in large, complex datasets. Moreover, new capabilities for deep learning are being added to ImageJ, reflecting a shift in the bioimage analysis community towards exploiting artificial intelligence. These new tools have been facilitated by profound architectural changes to the ImageJ core brought about by the ImageJ2 project. Therefore, we also discuss the contributions of ImageJ2 to enhancing multidimensional image processing and interoperability in the ImageJ ecosystem.

Compressive Spectral Light Field Image Reconstruction via Online Tensor Representation.

In recent years there has been an increasing interest in sensing devices that capture multidimensional information such as the spectral light field (SLF) images, which are 5-dimensional (5D) representations of a scene including 2D spatial, 2D angular and 1D spectral information. Spatio-spectral and angular information plays an important role in modern applications spanning from microscopy to computer vision. However, SLF sensors use expensive beam-splitters or cameras arrays placed in tandem, which split the sensing problem in two time consuming and independent tasks: spectral and light field imaging tasks. This work proposes a compressive spectral light field imaging architecture that builds on the principles of the compressive imaging framework, to capture multiplexed representations of the multidimensional information, so that, less measurements are required to capture the SLF data cube. Alongside, we propose a computational algorithm to recover the 5D information from the compressed measurements, exploiting the inherent high correlations within the SLF by treating them as 3D tensors. Furthermore, exploiting the geometry properties of the proposed optical architecture, the Tucker decomposition is applied to the set of compressed measurements, so that, an ad-hoc dictionary-like image representation basis is calculated online. This in turn, entails a more accurate reconstruction of the SLF since the dictionary fits the specific characteristics of the image itself. We demonstrate through simulations over three SLF datasets captured in our laboratory, and an experimental proof-of-concept implementation, that the proposed compressive imaging device together with the proposed computational algorithm represent an efficient alternative to capture SLF, compared to conventional methods that employ either side-information or multiple sensors. Also, we show that the tensor-based proposed algorithm exhibits a lower computational complexity than the matrix-based state of the art counterparts, thus enabling fast processing of multidimensional images.

Journal of Real-Time Image Processing: second issue of volume 9

The editorial board of JRTIP met in Brussels in April 2014 where they were attending the SPIE Conference on RealTime Image and Video Processing. The editorial board will have its next meeting in San Francisco in February 2015 where the same conference will take place as part of the SPIE Electronic Imaging Symposium. In the recently held editorial meeting (see Fig. 1), various items related to the status of the journal were discussed; a summary of these items is provided below. To address the backlog of printed versions of the accepted manuscripts that appear on Online First, we are pleased to state that the number of printed pages per issue will be increased to 200 pages per issue starting in 2015 for a total page count of 800 pages annually. It is hoped that the backlog situation is eased as a result of this page count increase. We will revisit this issue in 2015 to see whether further actions would be needed to resolve this issue. The journal performance numbers that have become available for the year 2013 clearly reflect the elevated recognition level of JRTIP. In 2013, similar to 2012, the acceptance rate of manuscripts submitted to JRTIP has remained at about 25 %. This is an indication that our review process is conducted in such a way that only quality papers are published in JRTIP. It is worth noting that the usage factor (UF) of JRTIP, which is the ratio of the number of paper downloads over the number of papers for a 2-year period, has increased by about 75 % from 2011 to 2012. With the substantial increase in the number of downloads to 34,000? in 2013 from 25,500? in 2012, it is expected that the 2013 UF would increase significantly. We would like to particularly thank the Associate Editors and the Springer Editorial Office for the major roles they have played in increasing these numerical indicators while maintaining the quality of published papers. Here, it is worth pointing out that the third issue in 2014 will be allocated to two special issues that are scheduled to appear together. These special issues are named ‘‘Algorithms and Architectures for Real-time Multi-Dimensional Image Processing’’ and ‘‘Convergence of Real-Time 3D Imaging and Optical Communications’’. This regular issue of JRTIP includes six papers. The first paper by Juliet et al. presents a compound image segmentation algorithm based on the mixed raster content (MRC) model (foreground/mask/background). After segmenting a compound image into different classes, each class is transformed to the MRC model according to its properties. The foreground and the background layers are compressed using JPEG 2000 while the mask layer is compressed using JBIG2. It is shown that this MRC compression achieves a higher subjective quality and coding efficiency than the state-of-the-art compression algorithms, such as JPEG, JPEG 2000 and H.264/AVC-I. The second paper by Bartovsky et al. introduces an additional, intra-operator level of parallelism for dilation/ erosion operations. Realized in a dedicated hardware, for rectangular structuring elements with programmable size, the implementation allows obtaining real-time performances for these operators that were previously unachievable. Low latency and memory requirements are the other main attributes of this parallelism. The third paper by Gaceb et al. describes a document classification approach based on physical layout features N. Kehtarnavaz (&) University of Texas at Dallas, Richardson, USA e-mail: kehtar@utdallas.edu

Special issue on algorithms and architectures for real-time multi-dimensional image processing

This special issue collects eight contributions, selected among many submitted papers after a multi-cycle review process, which are representative of the recent advances on real-time multi-dimensional image processing algorithms and architectures. The published works come from both academia and industry R&D sites in Europe, North America, Asia and Africa. These works show that more and more applications involve multi-dimensional image processing (multi- and hyperspectral imaging, multi-camera vision, multi-frame imaging, etc.), such as remote sensing, surveillance, computer vision and image/video enhancement for avionic and automotive.

Multi-dimensional color image storage and retrieval for a normal arbitrary quantum superposition state

Multi-dimensional color image processing has two difficulties: One is that a large number of bits are needed to store multi-dimensional color images, such as, a three-dimensional color image of $$1024 \times 1024 \times 1024$$1024?1024?1024 needs $$1024 \times 1024 \times 1024 \times 24$$1024?1024?1024?24?bits. The other one is that the efficiency or accuracy of image segmentation is not high enough for some images to be used in content-based image search. In order to solve the above problems, this paper proposes a new representation for multi-dimensional color image, called a $$(n\,+\,1)$$(n+1)-qubit normal arbitrary quantum superposition state (NAQSS), where $$n$$n qubits represent colors and coordinates of $${2^n}$$2n pixels (e.g., represent a three-dimensional color image of $$1024 \times 1024 \times 1024$$1024?1024?1024 only using 30?qubits), and the remaining 1?qubit represents an image segmentation information to improve the accuracy of image segmentation. And then we design a general quantum circuit to create the NAQSS state in order to store a multi-dimensional color image in a quantum system and propose a quantum circuit simplification algorithm to reduce the number of the quantum gates of the general quantum circuit. Finally, different strategies to retrieve a whole image or the target sub-image of an image from a quantum system are studied, including Monte Carlo sampling and improved Grover's algorithm which can search out a coordinate of a target sub-image only running in $$O(\sqrt{N/r} )$$O(N/r) where $$N$$N and $$r$$r are the numbers of pixels of an image and a target sub-image, respectively.

Decomposed multidimensional control grid interpolation for common consumer electronic image processing applications

Interpolation is an essential and broadly employed function of signal processing. Accordingly, considerable development has focused on advancing interpolation algorithms toward optimal accuracy. Such development has motivated a clear shift in the state-of-the art from classical interpolation to more intelligent and resourceful approaches, registration-based interpolation for example. As a natural result, many of the most accurate current algorithms are highly complex, specific, and computationally demanding. However, the diverse hardware destinations for interpolation algorithms present unique constraints that often preclude use of the most accurate available options. For example, while computationally demanding interpolators may be suitable for highly equipped image processing platforms (e.g., computer workstations and clusters), only more efficient interpolators may be practical for less well equipped platforms (e.g., smartphones and tablet computers). The latter examples of consumer electronics present a design tradeoff in this regard: high accuracy interpolation benefits the consumer experience but computing capabilities are limited. It follows that interpolators with favorable combinations of accuracy and efficiency are of great practical value to the consumer electronics industry. We address multidimensional interpolation-based image processing problems that are common to consumer electronic devices through a decomposition approach. The multidimensional problems are first broken down into multiple, independent, one-dimensional (1-D) interpolation steps that are then executed with a newly modified registration-based one-dimensional control grid interpolator. The proposed approach, decomposed multidimensional control grid interpolation (DMCGI), combines the accuracy of registration-based interpolation with the simplicity, flexibility, and computational efficiency of a 1-D interpolation framework. Results demonstrate that DMCGI provides improved interpolation accuracy (and other benefits) in image resizing, color sample demosaicing, and video deinterlacing applications, at a computational cost that is manageable or reduced in comparison to popular alternatives.

Special issue on “Multidimensional Signal and Image Analysis on HealthCare Applications”

The rapid approaching of elderly era has caused the impending needs for applying new technology for elderly and disabled people to improve health conditions and increase the quality of life. Under such a situation, novel bioengineering devices and tools are being developed to gain better understanding of human metabolic, anatomy and functions of organs. On the other hand, chronic diseases cause healthcare to be shared by daily health maintenance, triggering the impending need for intelligence of devices in which preliminary decisions need to be built into the system for providing preliminary recommendations. To achieve the development of novel and intelligent bioengineering and healthcare devices, the techniques of multidimensional signal and image processing and analysis can play one of the major roles. Compared with one dimensional signal analysis, the analysis of multidimensional signals and images provides more information. On the other hand, the high dimensionality of data may also cause large computational cost and, therefore, reduce the feasibility. Consequently, how to extract information for best estimation of health situations from the correlated or uncorrelated signals needs to be addressed. The goal of this special issue is to provide most up-to-date and recent advances of multidimensional signal and image analysis techniques for healthcare applications. This special issue serves as a forum and venue for researchers in academia, clinics and industries working in this area to share their experiences with the readers. For this special issue, we received a total of 13 submissions, each of which has gone through rigorous peer review. Based on the review results, six papers are selected for publication in this special issue, with brief description as follows. In “Enhancement of Blood Vessels in Retinal Imaging Using the Nonsubsampled Contourlet Transform”, Lee et al. propose a modified version of the nonsubsampled

4-13 多次元画像処理におけるGPU上でのメモリ利用モデルの検討(第4部門 メディア処理2)

4-13 多次元画像処理におけるGPU上でのメモリ利用モデルの検討(第4部門 メディア処理2)

Clifford Algebra Bundles to Multidimensional Image Segmentation

We present a new theoretical framework for multidimensional image processing using Clifford algebras. The aim of the paper is to detect edges by computing the first fundamental form of a surface associated to an image. We propose to construct this metric in the Clifford bundles setting. A nD image, i.e. an image of dimension n, is considered as a section of a trivial Clifford bundle $(CT(D),\widetilde{\pi},D)$ over the domain $D$ of the image and with fiber $Cl(\mathbb{R}^n,\|\|_2)$. Due to the triviality, any connection $\nabla_1$ on the given bundle is the sum of the trivial connection $\widetilde{\nabla}_0$ with $\omega$, a 1-form on $D$ with values in $End(CT(D))$. We show that varying $\omega$ and derivating well-chosen sections with respect to $\nabla_1$ provides all the information needed to perform various kind of segmentation. We present several illustrations of our results, dealing with color (n=3) and color/infrared (n=4) images. As an example, let us mention the problem of detecting regions of a given color with constraints on temperature; the segmentation results from the computation of $\nabla_1(I)=\widetilde{\nabla}_0(I)+ (dx + dy) \otimes \mu \, I$, where $I$ is the image section and $\mu$ is a vector section coding the given color.

Multidimensional Signal, Image and Video Processing and Coding [Book Reviews

This new book covers the processing and coding of multidimensional signals, in particular, still images and video. The book is accompanied by a CD-ROM and a companion Web site. The book is well designed and its material is timely. The only drawback is its lack of depth in covering color. The book would work with an introductory graduate course on still-image processing and compression or for an advanced graduate course on video processing and compression. The fact that the book takes a signal processing, rather than computer vision, viewpoint to image manipulation will make it a valuable resource for both researchers and students alike.

Wavelets with composite dilations and their MRA properties

Affine systems are reproducing systems of the form A C = { D c T k ψ ℓ : 1 ⩽ ℓ ⩽ L , k ∈ Z n , c ∈ C } , which arise by applying lattice translation operators T k to one or more generators ψ ℓ in L 2 ( R n ) , followed by the application of dilation operators D c , associated with a countable set C of invertible matrices. In the wavelet literature, C is usually taken to be the group consisting of all integer powers of a fixed expanding matrix. In this paper, we develop the properties of much more general systems, for which C = { c = a b : a ∈ A , b ∈ B } where A and B are not necessarily commuting matrix sets. C need not contain a single expanding matrix. Nonetheless, for many choices of A and B, there are wavelet systems with multiresolution properties very similar to those of classical dyadic wavelets. Typically, A expands or contracts only in certain directions, while B acts by volume-preserving maps in transverse directions. Then the resulting wavelets exhibit the geometric properties, e.g., directionality, elongated shapes, scales, oscillations, recently advocated by many authors for multidimensional signal and image processing applications. Our method is a systematic approach to the theory of affine-like systems yielding these and more general features.

3D reconstruction techniques made easy: know-how and pictures

Three-dimensional reconstructions represent a visual-based tool for illustrating the basis of three-dimensional post-processing such as interpolation, ray-casting, segmentation, percentage classification, gradient calculation, shading and illumination. The knowledge of the optimal scanning and reconstruction parameters facilitates the use of three-dimensional reconstruction techniques in clinical practise. The aim of this article is to explain the principles of multidimensional image processing in a pictorial way and the advantages and limitations of the different possibilities of 3D visualisation.

A nonlocal mean curvature flow for multi-dimensional image processing

A nonlocal mean curvature flow for multi-dimensional image processing

A high performance hybrid parallel algorithm for fast calculation of 3-D convolution/correlation using the 3-D VR FHT for multi-dimensional signal processing applications

In this paper, a fast and efficient 3-D digital filtering method based on the combination of a 3-D hybrid parallel filtering algorithm and the 3-D vector radix fast Hartley transform (3-D VR FHT) is presented. This algorithm is suitable for high resolution/high speed multi-dimensional signal and image processing. The 3-D hybrid parallel algorithm is highly parallel and efficient, as it overcomes the computations overhead and performance limitations associated with the block filtering method by eliminating the overlapping blocks and boundary conditions in parallel filtering applications. It also lifts the restrictions on the input size for high performance, as both the 3-D input data and impulse response of the system are segmented into smaller sub-sections. These sub-sections are independent and can be simultaneously processed. The algorithm's structure and mathematical derivation are given and the performance of the algorithm is tested and presented using a real multi-processor parallel system with 4-DSPs.

An interactive visualization and navigation tool for medical volume data

In order to make direct volume rendering practicable convenient visualization options and data analysis tools have to be integrated. For example, direct rendering of semi-transparent volume objects with integrated display of lighted iso-surfaces is one important challenge especially in medical applications. Furthermore, explicit use of multi-dimensional image processing operations often helps to optimize the exploration of the available data sets. On the other hand, only if interactive frame rates can be guaranteed, such visualization tools will be accepted in medical planning and surgery simulation systems. In this paper we propose a volume visualization tool for large scale medical volume data which takes advantage of hardware-assisted 3D texture interpolation and convolution operations. We demonstrate how to use the 3D texture mapping capability of high-end graphics workstations to display arbitrary iso-surfaces which can be directly illuminated to enhance the spatial relations between objects. Back-to-front 3D texture slicing is used to simultaneously display semi-transparent material densities. In order to enable on-the-fly data analysis, first approaches using hardware-assisted-convolution operations have been integrated. An implementation of the proposed method based on the OpenInventor rendering toolkit is described offering interactive frame rates at high image quality including sophisticated user interactions.

The automated interactive microscope (AIM) for investigating the functional dynamics of living cells: Status and vision

The current status and future development plans for the Automated Interactive Microscope (AIM), to better define the mechanisms of the function of living cells, is described. The development of AIM is fueled by biologists need to spatially and temporally correlate biochemical, molecular and genetic patterns to study cell functions such as division, locomotion and endocytosis. AIM will allow the investigator to use the cell as a “living microcuvette”.AIM redefines the meaning of the term “microscope” to include the functional capability of the microscope system provided by computerization, in addition to the more classical definition which emphasizes the opto-mechanical hardware. AIM is an next generation electronic light microscope imaging system which has grown out of our work in the development of the multimode light microscope. New functionality includes new experiment control facilities as well as related multidimensional image processing, image analysis and interactive data visualization. AIM is built around microscope hardware with high precision motion controls and a powerful host workstation computer linked, via a very high speed data gateway, with the high performance computing and communication (HPCC) facilities of the Pittsburgh Supercomputing Center (PSC).

特集 画像処理 ２． 画像処理の技術を拓く ２‐４ 多次元画像処理

特集 画像処理 ２． 画像処理の技術を拓く ２‐４ 多次元画像処理