Camera Link Research Articles

Latency Evaluation in the Image Acquisition System Based on MTCA.4 Architecture for Plasma Diagnostics

ITER diagnostic systems provide measurements to the Plasma Control System (PCS) in real-time. These measurements are used for plasma control and machine protection. Latency is an important parameter in the assessment of such systems. It is a time gap between capturing an external event by hardware and finishing the processing of acquired data. PCS requires the diagnostic systems to introduce a maximum total latency of 10 to 100 ms, therefore, the systems need to be tested if they meet the requirements. The system evaluated in this paper is a reference real-time image acquisition system developed as a base for ITER diagnostic systems. It consists of hardware based on the Micro Telecommunications Computing Architecture (MicroTCA) standard, developed firmware, and software. It supports cameras with various interfaces. In the paper, two cameras, with a Camera Link and 1 GigE Vision interfaces were selected to perform latency evaluation. The paper presents two methods of measuring the latency of image acquisition. The first one is based on precise time stamping consecutive stages of acquisition. This approach allows for determining which step of acquisition takes more or less time. In consequence, the software or hardware can be optimized. The other one uses LED to evaluate a particular camera, by checking the time of camera reaction to the trigger. A dedicated testing framework is developed to perform automated tests to evaluate latency. It supports collecting and analyzing the results of measurements. Besides that, a dedicated hardware is used to perform the latency tests using LED. The results and discussion of the measurements are presented in the manuscript. They show the latency evaluated using earlier proposed methods, comparing the cameras used in the image acquisition system.

Evolution of the magmatic-hydrothermal system at Lastarria volcano (Northern Chile) between 2006 and 2019: Insights from fluid geochemistry

One of the major problems in the volcanic surveillance is how data from several techniques can be correlated and used to discriminate between possible precursors of volcanic eruptions and changes related to non-eruptive processes. Gas chemical surveys and measurements of SO2 emission rates performed in the past (2006–2019) at Lastarria volcano in Northern Chile have revealed a persistent increment of magmatic sourced gas emissions since late November 2012, following a 13 years period of intense ground uplift. In this work, we provide new insights into the gas-chemical evolution of Lastarria’s fumarolic discharges obtained from direct sampling (2006–2019) and SO2 emission rates using UV camera and DOAS instruments (2018–2019) and link these to pre-existing information on ground deformation (1998–2016) in order to determine the origin of observed degassing and ground deformation processes. We revise the four mechanisms originally proposed as alternatives by Lopez et al. (Geosphere, 2018, 14 (3), 983–1007) to explain the changes observed in the fluid geochemistry and ground deformation between 2009 and 2012, in order to explain major changes in gas-geochemistry over an extended period between 1998 and 2019. We hypothesize that a continuous sequence of processes explains the evolution in the fluid geochemistry of fumarolic discharges. Two mechanisms are responsible of the changes in the gas composition during the studied period, corresponding to a 1) deep magma chamber (7–15 km depth) pressurized by volatile exsolution (1998–2020), which is responsible of the large-scale deformation; followed by 2) a crystallization-induced degassing (2001–2020) and pressurization of the hydrothermal system (2003-early November 2012), where the former process induced the changes in the gas composition from hydrothermal-dominated to magmatic-dominated, whereas the last produced the small-scale deformation at Lastarria volcano. The changes in the gas composition since late November 2012, which were strongly dominated by magmatic volatiles, produced two consecutive processes: 1) acidification (late November 2012–2020) and 2) depletion (2019–2020) of the hydrothermal system. In this work we have shown that a long-term surveillance of the chemistry of fluid discharges provides valuable insights into underlying magmatic/volcanic processes, and consequently, for forecasting future eruptions.

Adaptive Buffer Capture and Storage for High-Speed Cameras

The output speed of a high-resolution camera with a fast frame rate is high, reaching 800 MB/s or even more. Consequently, the abundant data recorded can easily cause data overflow during transmission and frame loss during storage. To address this problem of data overflow and frame loss while achieving reliable acquisition rates, herein, we design a high-speed video adaptive transmission and storage method based on the camera link transmission protocol. The system uses the camera link transmission protocol and high-speed cameras for data acquisition. During data transmission, memory space is employed as the data buffer pool, an adaptive memory buffer pool is designed using the hardware and acquisition parameters, and a circular read/write buffer, combined with a block device read/write method, is used inside the buffer pool for data transmission. For data storage, input/output completion port asynchronous storage mixed with frame sequence number filtering is employed to write data, and the adaptive buffer of the hard disk is used to achieve efficient data writing. The experimental results reveal that the proposed method can realize efficient and stable writing of data on conventional industrial cameras based on the camera link interface, effectively addressing the problem of frame loss during data storage. Moreover, the stored data can be transferred efficiently with less resource occupation, thereby meeting the application requirements of high-precision vision measurement systems and demonstrating broad application value.

Focus on lasers, imaging, microscopy, and photonics

Focus on lasers, imaging, microscopy, and photonics

On Heterogenous Sampling Rates in Origin–Destination Matrix Estimation Based on Trajectory Data and Link Counts

Being a fundamental metric of the transportation network, the origin–destination (O-D) flow matrix is a critical input for various transportation models and studies. This paper deals with the estimation of an O-D matrix of trip flows based on two kinds of data: probe trajectory data and local traffic counts. A Bayesian assignment framework is developed for demonstrating the relationship between the link probe sampling rates and the fractional contributions from the sampling rates on different O-D pairs. The unknown O-D matrix is estimated by applying cross entropy minimization using a prior matrix from the probe trajectories, along with the Bayesian assignment rules on link sample rates as the constraints. The methodology was applied using floating car data and camera link flow counts for a numerical experiment. The results show that the method can achieve a robust estimation of O-D matrices, even using different prior matrices. The issue of the heterogeneous sampling rates can be well addressed with link count constraints, effectively correcting the unknown bias in the probe sampling. The case study using real data also proves the feasibility of mining observed trajectory data to obtain the assignment fractions and estimate the O-D matrix inversely, avoiding the conventional sophisticated process of traffic assignment modeling.

Area and Power Efficient 10B6Q PAM-4 DC Balance Coder for Automotive Camera Link

We propose an area- and power-efficient four-level pulse amplitude modulation (PAM-4) encoder/decoder for an AC coupled link system that guarantees DC balance and limited run length. Configured as 10B6Q, input data of 10 bits are mapped to 5 quaternary symbols. One of four candidate codewords that have inverted symbols at predefined positions is selected that best satisfies the requirements of DC balance and the number of transitions. One quaternary symbol is added at the end to indicate which candidate codeword is selected. This coding scheme not only guarantees the PAM-4 DC balance but also increases transition density from 75% (PRBS) to 85.6% and limits the maximum run length down to six, thus facilitating timing recovery of the receiver. Although the input data width of 10 bits is used in this implementation, the proposed scheme can be extended to wider input data with higher coding efficiency. The proposed 10B6Q encoder is fabricated in the 40 nm CMOS technology and occupies an active area of 0.0009 mm <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> with a synthesized gate count of 645. It consumes 0.23 mW at the operating clock frequency of 667 MHz.

Badminton Recognition and Tracking System

Human–computer interaction systems have been developed in large numbers and quickly applied to sports. Badminton is the best sport for applying robotics because it requires quick recognition and fast movement. For the development of badminton recognition and tracking systems, it is important to accurately identify badminton, venues, and opponents. In this paper, we designed and developed a badminton recognition and tracking system using two 2 000 000-pixel high-speed cameras. The badminton tracking system has a transmission speed of 250[Formula: see text]fps and the maximum speed of the badminton resonator is 300[Formula: see text]km/h. The system uses the camera link interface Camera Link to capture images of high-speed cameras and process all captured images in real time using different regions of interest settings. In order to improve accuracy, we propose a new method for judging the center point of badminton. We have proposed a detector that detects the four corner points of the field by using the contour information of the badminton court when the approximate position of the badminton court is known. We set the sensing area according to the approximate position of the badminton court and use the histogram in the sensing area to select the point closest to the contour. Specify the intersection of the line as the corner point of the badminton court. The proposed angle detector has a high detection rate. It is more than 10 times more accurate than traditional detectors. The moving badminton is detected by an elliptical detector. We propose a method to find the center of the correct ellipse from the four candidates by selecting the four candidate contours of the ellipse. Compared to conventional circular detectors and points on three-dimensional coordinates, the proposed elliptical detector reduces the error by about 3[Formula: see text]mm.

Focus on lasers, imaging, microscopy, and nanoscience

Focus on lasers, imaging, microscopy, and nanoscience

Design and optimization of line-field optical coherence tomography at visible wavebands.

Parallel line-field Fourier-domain optical coherence tomography (LF-FDOCT) has emerged to enable relatively higher speeds than the conventional FDOCT system. In the LF-FDOCT, one B-scan is captured at a time instead of scanning the beam to acquire hundreds of A-scans. On the other hand, spectroscopic OCT using the visible waveband provides absorption information over multiple wavelengths at each voxel. This information of spectral absorption enables quantitative measurement of blood oxygenation, voxel by voxel. Here, we presented the design and optimization of a LF-FDOCT system at the visible waveband (520-620 nm), especially using a generic Camera Link area sensor (2048 × 1088 pixels). To optimize the axial resolution and depth of imaging volume, we simulated various parameters and found that two Nyquist optima can exist, the origin and implication of which has been discussed. As a result, our system acquired 1088 A-scans in parallel at the camera's frame rate of 281 frame per second, achieving an equivalent rate of over 300,000 A-scan/s, while minimizing sacrifice in the point spread function (2.8 × 3.1 × 3.2 µm3, x × y × z) and the field of view (750 × 750 × 750 µm3). As an example of application, we presented high-speed imaging of blood oxygenation in the rodent brain cortex.

Multi-Target Multi-Camera Tracking of Vehicles Using Metadata-Aided Re-ID and Trajectory-Based Camera Link Model.

In this paper, we propose a novel framework for multi-target multi-camera tracking (MTMCT) of vehicles based on metadata-aided re-identification (MA-ReID) and the trajectory-based camera link model (TCLM). Given a video sequence and the corresponding frame-by-frame vehicle detections, we first address the isolated tracklets issue from single camera tracking (SCT) by the proposed traffic-aware single-camera tracking (TSCT). Then, after automatically constructing the TCLM, we solve MTMCT by the MA-ReID. The TCLM is generated from camera topological configuration to obtain the spatial and temporal information to improve the performance of MTMCT by reducing the candidate search of ReID. We also use the temporal attention model to create more discriminative embeddings of trajectories from each camera to achieve robust distance measures for vehicle ReID. Moreover, we train a metadata classifier for MTMCT to obtain the metadata feature, which is concatenated with the temporal attention based embeddings. Finally, the TCLM and hierarchical clustering are jointly applied for global ID assignment. The proposed method is evaluated on the CityFlow dataset, achieving IDF1 76.77%, which outperforms the state-of-the-art MTMCT methods.

Camera Link Technology for Automotive Applications

Camera Link Technology for Automotive Applications

Framework for High-Performance Video Acquisition and Processing in MTCA.4 Form Factor

The presented framework enables capturing video streams from cameras with a Camera Link interface. It is designed for scalable high-reliability industrial and scientific applications. The solution performs data capture, optionally processing it, and delivers it to a host CPU. It consists of an MTCA.4-compliant frame grabber, its firmware, and a set of software components supporting several different cameras. The framework is focused on Xilinx 7-Series field-programmable gate array (FPGA) circuits. These have I/O pins fast enough to deserialize the Camera Link streams without use of any external components. The reference implementation runs on a hardware platform composed of a dual-slot FMC carrier module equipped with a modern Artix-7 FPGA and up to two Camera Link pass-through modules. The video stream is written to the host computer memory by a custom scatter-gather direct memory access (DMA) engine developed at the Department of Microelectronics and Computer Science (DMCS) of Lodz University of Technology. The framework also includes complete software support consisting of a Linux device driver, application programming interface libraries as well as console and graphical user applications. The proposed video acquisition solution is the world's first Camera Link frame grabber for the MTCA.4 architecture. It enables collecting data from the camera using the top performance 80-bit Camera Link interface, offering 6.8 Gb/s of raw image data throughput. The solution was tested with a number of commercially available cameras.

Focus on photonics, spectroscopy, and spectrometry

Focus on photonics, spectroscopy, and spectrometry

An Android Based Mobile Robot for Monitoring and Surveillance

We developed a sustainable surveillance robot that is cost effective using an Arduino microcontroller together with a motor shield and an Android smartphone that runs the Operating System. The robot consists of a video camera and wifi robot link. Smartphones come with superb hardware that satisfies the above needs. This can be leveraged upon through the use of APIs (Application Programming Interfaces) that is provided for the operating system. However, the building cost for the robot with a smartphone is greatly reduced. The robot can be remotely controlled using the wifi module and a microcontroller, smart phone interface embedded on the robot. The camera on the robot is used to capture and record real time video from the robot. The robot can be controlled based on visual feedback from the same smartphone. The four wheeled dc motors help to navigate the robot and ultrasonic sensor to avoid obstacles. The camera is attached to the wifi robot link which enables it to capture the environment or any object of concern. Experimental results with varied positions of obstacle show the flexibility of the robot to avoid it and have shown a decent performance and it is getting a communication range of nearly 50m, which is good enough for many surveillance applications.

A Method of Camera Link Image Data Processing Based on FPGA

Camera Link is based on the development of Channel Link Technology. The data is transmitted using low voltage differential signal (LVDS). It has some advantages, such as fast speed, strong anti-interference ability and so on. Camera Link is widely used in the field of robot vision applications. FPGA integrates rich gate circuit resources and has the advantages of high speed, which makes it suitable for processing parallel data. In this paper, a Camera Link image data processing method based on FPGA is proposed, which is verified by simulation and has been successfully applied to practical projects.

Online-Learning-Based Human Tracking Across Non-Overlapping Cameras

Due to the expanding scale of camera networks, Multiple Camera Tracking (MCT) of humans has received increased attention in recent years. In this paper, we present a novel approach to tracking each human within a single camera and across multiple disjoint cameras. Our framework includes a multi-object tracking and segmentation system, a two-phase feature extractor, and an online-learning-based camera link model estimation. For tracking within a single camera, we apply tracking by segmentation and local object detection with multi-kernel feedback to adaptively improve the robustness of the algorithm. In inter-camera tracking, we introduce an effective integration of appearance and context features. Couples are automatically detected, and the couple feature is also integrated with existing features. The proposed algorithm is scalable by a fully unsupervised online-learning framework. In our experiments, the proposed method outperforms all of the state-of-the-art methods in the benchmark National Laboratory of Pattern Recognition (NLPR)_MCT dataset.

Smartphone camera-based device-to-device communication using neural network-assisted high-density modulation

Device-to-device (D2D) communication has become one of the focuses for future wireless communications in conjunction with rapidly growing smartphone technologies. Smartphone camera-based D2D communication is presented as a favorable solution to supporting indoor D2D communications. As a smartphone has both camera and display, two smartphones can communicate over a relatively short range distance indoors in the form of smartphone display (transmitter) and smartphone camera (receiver) link. The smartphone camera-based wireless communication called optical camera communication (OCC), however, suffers from low data rate that is at most a hundred kilobits per second. To ameliorate this, we propose a high-density modulation that is more efficient in achieving an Mbps rate for smartphone camera-based D2D systems, by exploiting two-dimensional data transmission capability in OCC. Neural network is also implemented to improve the performance of the proposed system. To verify the proposed system, simulations and experiments were performed. It is demonstrated that a 2.66 Mbps rate of the proposed system is achieved with an acceptable bit error rate.

Research on Video Remote Transmission Technology Based on FPGA

In order to solve the problem of remote transmission technology of high speed digital video in the application process, this paper takes FPGA technology as the core, uses the optical fiber communication, Camera link interface technology, and high-speed serial parallel conversion and so on to provide a technical proposal of a high-speed digital video remote transmission. First of all, this paper overall designs the remote transmission technology of digital video, and then designs its hardware circuit and system control logic, focuses on the specific design of digital video acquisition, high-speed data caching and decoding, data sending and receiving and analog camera data. Finally, the debugging and testing of the experimental transmission technology of the remote digital video are carried out, mainly for the single board debugging, analog transmission experiment, and video transmission experiment, in which the debugging results show that the FPGA minimum system meets the requirements. Transmitting and receiving communication circuit and serial communication circuit of Camera link are correct; high-speed serial parallel conversion chip and SFP peripheral circuit are stable and reliable; simulation experiments show that in the process of image transmission, the pixel clock, line effective, field effective; and the sequence of image data is controlled reasonably, which can well achieve image transmission. The video transmission experiment results show that the transmission technology designed can achieve the remote transmission of digital video.

Non-Standard Video Protocol Transmission System Design Based on the Camera Link Interface of FPGA

With the rapid development of video communication technology, video transmission is becoming more and more important. However, in the process of video transmission, the reliability and real-time performance of transmission data is the premise of video high-speed transmission, So In order to improve the real-time video image transmission, validity, reliability, and design a Camera Link interface based on FPGA of non-standard video protocol transmission system, the system front-end by simulating the Camera to complete the video image collection, through the video format conversion chip output standard video protocol data, and uses FPGA to complete the processing of video data, complete the video image with Camera Link interface of high speed transmission, finally realizes the real-time video transmission, but also improve the reliability and effectiveness of the video image.

Methodology for the implementation of real-time image processing systems using FPGAs and GPUs and their integration in EPICS using Nominal Device Support

There is an increasing interest to improve the processing capabilities for imaging diagnostics in fusion devices. Furthermore, the continuous upgrades of the hardware systems used in these tasks require a flexible platform that obtains the maximum performance of these technologies: cameras, frame-grabbers, and parallel processing architectures using FPGAs, GPUs, and multicore CPUs. This work presents a methodology for the development of real-time image processing applications using the ITER CODAC Core System (CCS) software tools and the hardware configuration defined in the ITER fast controllers hardware catalog, by means of Camera Link cameras, Camera Link FPGA-based frame-grabbers, and NVIDIA GPUs. The integration with EPICS is provided using the Nominal Device Support approach and the IRIO tools integrated into CCS. This device support provides complete control of the camera configuration parameters, distribution of the image processing functions between the FPGA and the GPU, and efficient movement of the data between the different elements of the architecture. To validate the full development cycle, an EPICS application has been developed implementing different algorithms frequently used in fusion applications such as histogram, detection of levels, and classification of zones within an image. Results of the performance obtained are presented highlighting the differences among various configurations running the selected algorithms.