Camera Hardware Research Articles

Integration of Photographic Art and Imaging Technology: Calculating Effect of Small Depth of Field

Photography teaching is inseparable from photography tools. Image processing and computing play an increasingly important role in modern photography tools. Some of the artistic effects that were originally achieved by camera hardware can now be achieved with software calculations. Background blur technology with small depth of field is a typical application. Based on the principle of optical imaging, this paper designs an image processing method to calculate the imaging effect of small depth of field, and explores the integration of imaging technology and photography art. The advancement of future imaging technology will better serve the art of photography. The integration of technology and art is a development trend, and the content of photography teaching needs to be updated simultaneously.

Automatic Measuring Approach and Device for Mature Rapeseed’s Plant Type Parameters

The plant type parameters, which can be used to track and study the morphological changes of crop organs, are of great importance for breeders and testers. Based on the structural characteristics of mature rapeseed, an automatic measuring system for rapeseed plant type parameters was studied, through which the parameters of primary branches including their quantity, position, and branching angle and the geometric parameters such as plant height, main stem width, main raceme length, and the first branch height were measured. The system hardware includes a manual cutting-short device, an image acquisition darkroom, and a control circuit, used for capturing videos of a stably rotating rapeseed plant in a darkroom with an adjustable light source. The system software is used to connect and control CCD camera and other hardware and run specific image processing algorithms to process and analyze videos to obtain plant type parameters. The human-machine interface provided by the system can be used to set working parameters and display and process measurement results. To evaluate the accuracy of this prototype, batches of rapeseed plants were tested. The results showed that the mean absolute percentage error (MAPE) for the system was about 2.74%, and the automatic measurements had a good agreement with manual measurements. The measuring efficiency was approximately 2.1 times over manual method. In conclusion, the system, which automatically extracts plant type parameters with high throughput and high precision, is of practical value as it can effectively reduce the labor intensity of researchers and provide important basic data for the research of rapeseed breeding and agricultural machinery design.

Trajectory-Based Surveillance Analysis: A Survey

Due to the advancement of camera hardware and machine learning techniques, video object tracking for surveillance has received noticeable attention from the computer vision research community. Object tracking and trajectory modeling have important applications in surveillance video analysis. For example, trajectory clustering, summarization or synopsis generation, and detection of anomalous or abnormal events in videos are mainly being exploited by the research community. However, barring one research work (which is almost a decade old), there is no recent review that emphasizes the use of video object trajectories, particularly in the perspective of visual surveillance. This paper presents a survey of trajectory-based surveillance applications with a focus on clustering, anomaly detection, summarization, and synopsis generation. The methods reviewed in this paper broadly summarize the abovementioned applications. The main purpose of this survey is to summarize the state-of-the-art video object trajectory analysis techniques used in the indoor and outdoor surveillance.

Robust Optical Spatial Localization Using a Single Image Sensor

A novel 3-D spatial localization method using a single camera with temporal-difference image processing is proposed. The proposed active localization method uses a ring of light-emitting diodes embedded on the target. The diameter and the central location of the ring's image on the image sensor are used to estimate the target location using a Volterra series expansion of the target coordinates. No knowledge of the camera hardware parameters is needed. Instead, Volterra series parameters are obtained through a prior training that needs to be performed only once. The proposed method can be implemented with low computational complexity and storage. The performance of the proposed method is compared against an earlier method that relies on prior knowledge of the camera hardware parameters. The proposed method demonstrates excellent performance even when the target is located far away from the axial direction of the camera lens. The proposed method can be applied in low-power outdoor unmanned aerial vehicle localization and indoor robotic navigation.

Effects of Varying Noise Levels and Lighting Levels on Multimodal Speech and Visual Gesture Interaction with Aerobots

This paper investigated the effects of varying noise levels and varying lighting levels on speech and gesture control command interfaces for aerobots. The aim was to determine the practical suitability of the multimodal combination of speech and visual gesture in human aerobotic interaction, by investigating the limits and feasibility of use of the individual components. In order to determine this, a custom multimodal speech and visual gesture interface was developed using CMU (Carnegie Mellon University) sphinx and OpenCV (Open source Computer Vision) libraries, respectively. An experiment study was designed to measure the individual effects of each of the two main components of speech and gesture, and 37 participants were recruited to participate in the experiment. The ambient noise level was varied from 55 dB to 85 dB. The ambient lighting level was varied from 10 Lux to 1400 Lux, under different lighting colour temperature mixtures of yellow (3500 K) and white (5500 K), and different background for capturing the finger gestures. The results of the experiment, which consisted of around 3108 speech utterance and 999 gesture quality observations, were presented and discussed. It was observed that speech recognition accuracy/success rate falls as noise levels rise, with 75 dB noise level being the aerobot’s practical application limit, as the speech control interaction becomes very unreliable due to poor recognition beyond this. It was concluded that multi-word speech commands were considered more reliable and effective than single-word speech commands. In addition, some speech command words (e.g., land) were more noise resistant than others (e.g., hover) at higher noise levels, due to their articulation. From the results of the gesture-lighting experiment, the effects of both lighting conditions and the environment background on the quality of gesture recognition, was almost insignificant, less than 0.5%. The implication of this is that other factors such as the gesture capture system design and technology (camera and computer hardware), type of gesture being captured (upper body, whole body, hand, fingers, or facial gestures), and the image processing technique (gesture classification algorithms), are more important in developing a successful gesture recognition system. Some further works were suggested based on the conclusions drawn from this findings which included using alternative ASR (Automatic Speech Recognition) speech models and developing more robust gesture recognition algorithm.

ITER upper visible/infrared wide angle viewing system: I&C design and prototyping status

The ITER Upper Visible/Infrared Wide Angle Viewing System is a key diagnostic system for divertor surface temperature monitoring and heat load calculation during plasma operation. This system will be installed after the first plasma phase in five upper ports with front-end optics under vacuum, relay optics in the interspace, back-end optics and sensors in the port-cells, and data acquisition and processing units in the Diagnostic building. The system is currently in the Preliminary Design phase which includes some prototyping activities. This paper describes the current design status of the plant system Instrumentation and Control (I&C) comprising camera pre-selection, hardware, and software architecture for camera data acquisition and image processing. The proposed design has been elaborated by following the ITER CODAC methodology for Plant System I&C design. On-going prototyping activities addressing identified R&D challenges such as camera high throughput data acquisition and real-time image processing are also presented.

An efficient face recognition system based on hybrid optimized KELM

Face recognition (FR) from video offers a challenging issue in the area of image exploration along with computer visualization, furthermore, as such recognized heaps of deem over the previous years on account of its numerous applications in the scope of domains. The chief challenges in the video centered FR are the restraint of the camera hardware, the random poses captured by means of the camera as the subject is noncooperative, and changes in the resolutions owing to disparate lighting conditions, noise along with blurriness. Numerous FR algorithms were generated in the previous decennium, although these approaches are much better, the image’s accuracy is less only. To trounce such difficulties, an efficient FR system centered on hybrid optimized Kernel ELM is proposed. The proposed work encompasses five phases, explicitly (i) preprocessing, (ii) Face detection, (iii) Feature Extraction, (iv) Feature Reduction, and (v) Classification. In the preliminary phase, the data-base video clips are converted in to the frames in which pre-processing are performed utilizing a Modified wiener filter to eliminate the noise. The succeeding phase is employed for detecting the pre-processed image via the viola–jones (V-J). With this technique, the face is identified. After that, the features are extorted. The extracted ones then will be provided as the input to the Modified PCA approach. Then, perform classification operation using hybrid (PSO-GA) optimized Kernel ELM approach. The similar process is replicated for query images (QI). At last, the recognized image is found. Experimental results contrasted with the previous ANFIS classifier and existing methods concerning precision, accuracy, recall, F-measure, sensitivity along with specificity. The proposed FR system indicates better accuracy when compared with the prevailing methods.

Pipe radius estimation using Kinect range cameras

Abstract In the heavy construction industry, pipe spool assembly is geometrically complex and suboptimal fabrication processes inevitably lead to fabrication errors and costly rework. In an attempt to mitigate fabrication risks and improve product quality, computer aided tools are being developed to provide an additional layer of control. In this paper, the data acquisition capabilities of two low-cost range cameras are investigated for the eventual purpose of pipe fitting process monitoring in a smart fabrication facility environment. Range images of various pipes are systematically taken at varying distances from the sensor. By using the proposed radius estimation algorithm, the utility of the data for accurate geometrical pipe feature detection is evaluated by a radius feature metric. Results show that the algorithm reliably extracts radius information from point clouds representing piping. In conjunction with the algorithm, the low-cost range camera hardware was able to characterize pipes, of radius ranging from 2.41 cm to 8.78 cm at a distance from the sensor ranging from 0.5 m to 3.75 m, with an average error of 18% for Kinect 1 and 10% for Kinect 2.

As-projective-as-possible bias correction for illumination estimation algorithms

Illumination estimation is the key routine in a camera's onboard auto-white-balance (AWB) function. Illumination estimation algorithms estimate the color of the scene's illumination from an image in the form of an R, G, B vector in the sensor's raw-RGB color space. While learning-based methods have demonstrated impressive performance for illumination estimation, cameras still rely on simple statistical-based algorithms that are less accurate but capable of executing quickly on the camera's hardware. An effective strategy to improve the accuracy of these fast statistical-based algorithms is to apply a post-estimate bias-correction function to transform the estimated R, G, B vector such that it lies closer to the correct solution. Recent work by Finlayson [Interface Focus8, 20180008 (2018)2042-889810.1098/rsfs.2018.0008] showed that a bias-correction function can be formulated as a projective transform because the magnitude of the R, G, B illumination vector does not matter to the AWB procedure. This paper builds on this finding and shows that further improvements can be obtained by using an as-projective-as-possible (APAP) projective transform that locally adapts the projective transform to the input R, G, B vector. We demonstrate the effectiveness of the proposed APAP bias correction on several well-known statistical illumination estimation methods. We also describe a fast lookup method that allows the APAP transform to be performed with only a few lookup operations.

Digital Foveation: An Energy-Aware Machine Vision Framework

In machine vision applications, imaging systems and analysis algorithms are generally interdependent and energy intensive. We describe a machine vision energy minimization framework in which imaging hardware and vision algorithms are co-designed and tightly integrated. Digital foveation is inspired by the human vision system, which uses a spatially varying sensing architecture to generate oculomotory feedback and capture a series of high-resolution images using the densely sampling fovea. A multiround process with bidirectional information flow between camera hardware and analysis software optimizes energy consumption while preserving accuracy. By using existing hardware mechanisms, namely, row / column skipping, random access via readout circuitry, and frame preservation, digital foveation adapts to the chosen analysis algorithm. It aims to transmit and process only the necessary parts of the scene under consideration. This framework is general across a wide range of embedded machine vision applications and enables large improvements in energy efficiency. When evaluated for an embedded license plate recognition vision application, it reduces system energy consumption by 81.3% with at most 0.65% reduction in accuracy.

Kipas Angin Pengikut Manusia Berdasarkan Wajah

Kemajuan teknologi membuat kebutuhan terhadap sistem yang otomatis meningkat, salah satunya membuat kipas dapat tracking manusia secara otomatis. Pada penelitian ini bertujuan untuk mengembangkan sebuah sistem kipas yang dapat men-tracking manusia berdasarkan wajah dan dapat membedakan wajah dengan hambatan lain disekitarnya, sehingga manusia tidak perlu mengatur secara manual dalam mengarahkan kipas agar mendapat tiupan angin dari kipas. Untuk mendeteksi wajah manusia digunakan kamera dari smartphone android yang telah terintegrasi dengan metoda deteksi Haar Cascade yang tersedia pada library OpenCV. Sedangkan untuk melakukan tracking maka digunakan pengendali PID dengan dengan metode tuning Zigler Nichols yang terintegrasi dengan motor DC. Berdasarkan hasil percobaan sistem dapat melakukan tracking terhadap wajah manusia dengan rata-rata error sebesar 4.14%, sementara pengujian terhadap deteksi wajah mempunyai tingkat keberhasilan 100% dalam keadaan wajah frontal. Sementara sistem dapat men-tracking manusia berdasarkan wajah dengan keberhasilan 95%. Adapun kesalahan diakibatkan oleh sistem mekanik gearbox motor dc dan posisi hardware kamera yang tidak lurus terhadap objek wajah.

Guest editorial: Special issue on computational image sensors and smart camera hardware

Guest editorial: Special issue on computational image sensors and smart camera hardware

High-sensitivity analysis of polarization by surface reflection

The exploitation of polarization information in the field of computer vision has become progressively more popular during the last few decades. This is primarily due to (1) the fact that polarization is a source of mostly untapped information for machine vision; (2) the relative computational ease by which geometrical information about a scene (e.g. surface normals) may be extracted from polarization data; and (3) the recent introduction of camera hardware able to capture polarization data in real time. The motivation for this paper is that a detailed quantitative study into the precision of polarization measurements with respect to expectation has yet to be performed. The paper therefore presents a detailed analysis and optimization of the key aspects of data capture necessary to acquire the most precise (as opposed to fast) results for the benefit of future research into the field of “polarization vision.” The paper mainly focuses on a rotating polarizer method as this is shown to be the most accurate for high-sensitivity measurements. Commercial polarization cameras by contrast generally sacrifice precision for the benefit of much shorter capture times. That said, the paper reviews the state of the art in polarization camera technology and quantitatively evaluates the performance of one such camera: the Fraunhofer “POLKA.”

High resolution low kV EBSD of heavily deformed and nanocrystalline Aluminium by dictionary-based indexing

We demonstrate the capability of a novel Electron Backscatter Diffraction (EBSD) dictionary indexing (DI) approach by means of orientation mapping of a highly deformed graded microstructure in a shot peened Aluminium 7075-T651 alloy. A low microscope accelerating voltage was used to extract, for the first time from a bulk sample, statistically significant orientation information from a region close to a shot crater, showing both recrystallized nano-grains and heavily deformed grains. We show that the robust nature of the DI method allows for faster acquisition of lower quality patterns, limited only by the camera hardware, compared to the acquisition speed and pattern quality required for the conventional Hough indexing (HI) approach. The proposed method paves the way for the quantitative and accurate EBSD characterization of heavily deformed microstructures at a sub-micrometer length scale in cases where the current indexing techniques largely fail.

Canine and feline fundus photography and videography using a nonpatented 3D printed lens adapter for a smartphone.

To describe an indirect funduscopy imaging technique for dogs and cats using low cost and widely available equipment: a smartphone, a three-dimensional (3D) printed indirect lens adapter, and a 40 diopters (D) indirect ophthalmoscopy lens. Fundus videography was performed in dogs and cats using a 40D indirect ophthalmoscopy lens and a smartphone fitted with a 3D printed indirect lens adapter. All animals were pharmacologically dilated with topical tropicamide 1% solution. Eyelid opening and video recording were performed using standard binocular indirect ophthalmoscopy technique. All videos were uploaded to a computer, and still images were selected and acquired for archiving purposes. Fundic images were manipulated to represent the true anatomy of the fundus. It was possible to promptly obtain good quality images from normal and diseased retinas using the nonpatented 3D printed, lens adapter for a smartphone. Fundic imaging using a smartphone can be performed with minimal investment. This simple imaging modality can be used by veterinary ophthalmologists and general practitioners to acquire, archive, and share images of the retina. The quality of images obtained will likely improve with developments in smartphone camera software and hardware.

A machine learning approach for pixel wise classification of residue and vegetation cover under field conditions

Soil cover is a crucial factor for sustainable cultivation of arable land. A certain degree of residue and vegetation cover reduces erosion significantly and has positive effects on plant development. In order to accomplish these positive effects, it is necessary to measure and control the amount of soil cover on fields. Manual measurement methods are time consuming and/or subjective. Available image analysis methods often lack of generalisation and accuracy. Many approaches only focus on residue or on vegetation cover and do not consider different camera hardware. Recent advancements in machine learning techniques are promising to overcome these issues. The proposed method, the entangled random forest, a variant of a random decision forest, classifies individual pixels into soil, residue, living plants and stones. Simple and efficient pixel-wise comparisons to neighbouring pixels are integrated as decision-features into the random forest. To validate our method, the result of the automatic classification was compared with results of manual classifications from evaluators on image grid points. The classification of soil results in a regression equation between the results of the new introduced method and a manual image classification of y = 0.99x + 2.02 (R2 = 0.93). Living plant classification results in a regression between both methods in y = 0.94x − 0.70 with (R2 = 0.98) and for dead residues in y = 1.04x − 0.64 (R2 = 0.84). It is possible to access a demo of the algorithm by using a web and a mobile application on https://soilcover.josephinum.at.

A Learning and Management System on Account of Software and Hardware

This topic is intended to describe and study a system, which is taking Jilin University as an example, based on the system feasibility and necessity of campus applications, this paper analyzes and investigates the problems existing in the process of functional design, application and development. So we propose a new system to satisfy the needs of students and teachers: including a learning-oriented multi-function mobile application, mainly about using Android Studio to build application interface, combined with Myeclipse, tomcat and MySQL database. The system equip with positioning, camera and other hardware plus web pages on the windows platform as an auxiliary system. After about 400 teachers and students using over 2 months, because of its good user experience and characteristics of learning, we gain lots of praise, now we are planning to promote it gradually.

Surveillance scene representation and trajectory abnormality detection using aggregation of multiple concepts

Use of CCTV is growing rapidly in surveillance applications. Rapid advancement in machine learning and camera hardware has opened-up adequate scopes to build next generation of expert systems aiming at understanding surveillance environments automatically by detection of trajectory abnormality through analyzing object behavior. Such intelligent surveillance systems should be able to learn and combine multiple concepts of abnormality in real-life scenario and classify the events of interest as normal or abnormal. Primary challenges of such systems are to represent and learn patterns in surveillance scenes and combine multiple concepts of abnormalities to activate the alarm system.This paper presents a graph-based representation of a given surveillance scene and learning of relevant features including origin, destination, path, speed, size, etc. These features are combined and correlated with target behaviors to detect abnormalities in moving object trajectories. We also propose an aggregation method that reduces the number of missed alarms during aggregation. Several cases using publicly available surveillance video datasets have been presented and the results indicate that the proposed method can be useful to design intelligent and expert surveillance systems.

Intuitive robot programming through environment perception, augmented reality simulation and automated program verification

The increasing complexity of products and machines as well as short production cycles with small lot sizes present great challenges to production industry. Both, the programming of industrial robots in online mode using hand-held control devices or in offline mode using text-based programming requires specific knowledge of robotics and manufacturer-dependent robot control systems. In particular for small and medium-sized enterprises the machine control software needs to be easy, intuitive and usable without time-consuming learning steps, even for employees with no in-depth knowledge of information technology. To simplify the programming of application programs for industrial robots, we extended a cloud-based, task-oriented robot control system with environment perception and plausibility check functions. For the environment perception a depth camera and pointcloud processing hardware were installed. We detect objects located in the robot’s workspace by pointcloud processing with ROS and the PCL and add them to the augmented reality user interface of the robot control. The combination of process knowledge from task-oriented application programming and information about available workpieces from automated image processing enables a plausibility check and verification of the robot program before execution. After a robot program has been approved by the plausibility check, it is tested in an augmented reality simulation for collisions with the detected objects before deployment to the physical robot hardware. Experiments were carried out to evaluate the effectiveness of the developed extensions and confirmed their functionality.

UFXC32k based camera module with a custom soft processor and USB 3.0 for large area detectors

In this paper, an X-ray camera is presented based on a UFXC32k readout chip. The camera combines high speed (≈ 28 kfps), low cost and compact size (2 cm × 15 cm × 3 cm), enabling it to be used in applications of a large area detector. The above-mentioned parameters were achieved mainly by using a USB 3.0 for measurement data transfer. As a core of the camera, a custom soft processor was used to generate readout chip control sequences. This solution allowed for a great simplification of the camera hardware and firmware development. Moreover, thanks to its universality, the soft processor can be easily applied in the cameras based on other readout chips.