High-performance Hardware Platform Research Articles

A lightweight dense crowd density estimation network for efficient compression models

Crowd density estimation is a task of intelligent applications, and its operation efficiency is very important. However, to obtain a better density estimation performance, most of the existing works often design larger and more complex network structures, which will result in them occupying considerable memory, time and other resources at runtime, and require the support of high-performance hardware platforms, which are difficult to apply in practice. In this paper, to overcome the above problems, we propose a lightweight dense crowd estimation method based on channel attention multi-scale feature fusion. Specifically, in the process of feature extraction, an efficient and lightweight convolution module (L-weight) is designed to extract crowd features in stages, which reduces the amount of network parameters and computing costs, and we capture multi-scale crowd information through the feature extraction network of pyramid structure, which solves the problem of uneven crowd scale in video images. In the process of feature fusion, a channel attention fusion module is designed, which weights and fuses the feature information of different scales, effectively fuses multi-scale information and suppresses useless information. In addition, we design a new loss function, which enhances the sensitivity of the crowd through the pixel space loss (L2), counting loss (LC) and structural similarity loss (LS), to ensure the counting accuracy. Extensive experiments on four mainstream datasets demonstrate that compared with other state-of-the-art methods, our method achieves an optimal trade-off between counting performance and running speed, and is suitable for low-performance computing platforms such as embedded.

Le-SKT: Lightweight traffic density estimation method based on structured knowledge transfer

The traffic density estimation algorithm usually runs on an embedded platform, and its efficiency is very important to the implementation of applications. However, most of the existing methods rely on heavy backbone networks, require high-performance hardware platform support and take a long time to calculate, which are difficult prerequisites to apply in practice. In consideration of the aforementioned problems, this paper proposes a lightweight traffic density estimation method based on structured knowledge transfer (Le-SKT). This method consists of a teacher network and student network. In the teacher network, in order to solve the problem of variable vehicle scales in video images, a multiscale fusion structure is designed, which can fully fuse the low-level and high-level features of an image. To solve the problem of background interference in video images, a fusion module based on an attention mechanism is constructed to enhance the useful traffic flow feature information and suppress the background information. At the same time, the fused traffic flow feature information is dilated convoluted to expand the receptive field and obtain richer contextual vehicle feature information, which generates a trained teacher network. Finally, a lightweight student network is generated by using the structured knowledge of the teacher network, which solves the problems of long computing times and high hardware requirements. The MAE and GAME of this method are reduced to 3.92 and 5.69, respectively in the TRANCOS dataset and 3.74 and 5.57, respectively in the VisDrone2019 vehicle dataset, respectively; The parameter quantity of the student network is only 0.65% of that of the teacher network, which greatly reduces the scale of the neural network model and is suitable for an embedded computing platform with low performance.

Detection of Criminal Behavior at the Residential Unit based on Deep Convolutional Neural Network

Studies on abnormal behavior based on deep learning as a processing platform increase. Deep learning, specifically the convolutional neural network (CNN), is known for learning the features directly from the raw image. In return, CNN requires a high-performance hardware platform to accommodate its computational cost like AlexNet and VGG-16 with 62 million and 138 million parameters, respectively. Hence in this study, four CNN samplings with different architectures in detecting abnormal behavior at the gate of residential units are evaluated and validated. The forensic postures, with some other collected data, are used for the preliminary step in constructing the criminal case database. High accuracy up to 97% is obtained from the trained CNN samplings with 80% to 97% recognition rate achieved during the offline testing and 70% to 90% recognition rate recorded during the real-time testing. Results showed that the developed CNN samplings owned good performance and can be utilized in detecting and recognizing the normal and abnormal behavior at the gate of residential units.

Facial expression recognition based on dual-feature fusion and improved random forest classifier

Facial expression recognition (FER) is an important means for machines to understand the changes in the facial expression of human beings. Expression recognition using single-modal facial images, such as gray scale, may suffer from illumination changes and the lack of detailed expression-related information. In this study, multi-modal facial images, such as facial gray scale, depth, and local binary pattern (LBP), are used to recognize six basic facial expressions, namely, happiness, sadness, anger, disgust, fear, and surprise. Facial depth images are used for robust face detection initially. The deep geometric feature is represented by point displacement and angle variation in facial landmark points with the help of depth information. The local appearance feature, which is obtained by concatenating LBP histograms of expression-prominent patches, is utilized to recognize those expression changes that are difficult to capture by only the geometric changes. Thereafter, an improved random forest classifier based on feature selection is used to recognize different facial expressions. Results of comparative evaluations in benchmarking datasets show that the proposed method outperforms several state-of-the-art FER approaches that are based on hand-crafted features. The capability of the proposed method is comparable to that of the popular convolutional neural-network-based FER approach but with fewer demands for training data and a high-performance hardware platform.

Multi-video processing applications on FPGA

With the increasing needs of processing power in video and image processing for advanced media and communication applications, it is mandatory to go further than the software implementation to provide generic, real time, low cost and high performance hardware platforms. In this paper, we present a re-configurable, hardware platform for video and image processing. The proposed system uses the benefits of field programmable gate array FPGA to attain this objective. In this context, a prototype system is developed based on the Xilinx Virtex-5 FPGA with the integration of embedded processor, embedded memory, multi-port memory controller MPMC, standard interfaces, and different other resources. Our system includes different functional modules: video cut detection, video zoom-in and zoom-out. This provides the flexibility of using this system as a general video processing platform according to different application requirements. The final system utilises ∼16% of logic resource and 20% of on chip memory.

Tuning framework for stencil computation in heterogeneous parallel platforms

Image processing and computer vision applications are usually complex in terms of the large amount of processed data and high computation loads. To cope with this, optimization techniques and high-performance hardware platforms are required. Since these applications present many opportunities for parallelism, heterogeneous parallel platforms (HPPs) are an interesting choice, offering a good balance between high computation capabilities and flexibility to handle a large spectrum of application features. Applications such as image filtering and edge detection make extensive use of finite difference method to solve partial derivative equations, which computational pattern is called stencil computation. Stencil computations are known as memory-bound, so that reducing high-latency memory access becomes the biggest challenge to reach high performance. In this paper, we present our methodology as a basis of a performance tuning framework to optimize the implementation of multiple stencil computations on HPPs. Results show that our approach outperforms SDK-based methodologies, improving performance. Moreover, using the proposed approach, the developer has the ability of investigating efficiently the performance of the stencil computations before implementing actual code on the target platforms.

大数据背景下的信号处理

The rapid developments of wideband signal, high-dimensional signal, high resolution image, and multi-sensor network have made the growth rate of data acquisition amount higher than the growth rate of data storage amount and the growth rate of signal processing speed. Signal processing steps into the big data era. This paper points out the key issues of signal processing with big data. Considering the diversity and complexity of the big data produced by multi-sensor networks, it is necessary to conduct information fusion. The main information fusion models are described, and the trends of information fusion technology are analyzed. Intelligent sensor network technology can reduce the capacity of signal processing and communication, and efficiently extract valuable information from big data. The basic architecture of a smart sensor is given, and calculation methods for intelligent sensor networks are described. From viewpoint of high-speed and real-time signal processing requirements of signals with big data, we also introduce developments of high-speed digital signal processing chips and high-performance hardware platforms. The vision of core technology in high-speed signal processing is presented.

FPGA-Based Embedded Motion Estimation Sensor

Accurate real-time motion estimation is very critical to many computer vision tasks. However, because of its computational power and processing speed requirements, it is rarely used for real-time applications, especially for micro unmanned vehicles. In our previous work, a FPGA system was built to process optical flow vectors of 64 frames of640×480image per second. Compared to software-based algorithms, this system achieved much higher frame rate but marginal accuracy. In this paper, a more accurate optical flow algorithm is proposed. Temporal smoothing is incorporated in the hardware structure which significantly improves the algorithm accuracy. To accommodate temporal smoothing, the hardware structure is composed of two parts: the derivative (DER) module produces intermediate results and the optical flow computation (OFC) module calculates the final optical flow vectors. Software running on a built-in processor on the FPGA chip is used in the design to direct the data flow and manage hardware components. This new design has been implemented on a compact, low power, high performance hardware platform for micro UV applications. It is able to process 15 frames of640×480image per second and with much improved accuracy. Higher frame rate can be achieved with further optimization and additional memory space.

Dynamically Reconfigurable Computing for Wireless Communication Systems (Dynamisch rekonfigurierbares Rechnen für Mobilfunksysteme)

In this paper we will demonstrate that dynamically reconfigurable computing is the key technology to enable small and energy-efficient hardware platforms for wireless communication systems, which provide the flexibility needed for keeping up with evolving standards and protocols. Two application-specific dynamically reconfigurable systems for WLANs and wireless sensor networks will be presented. The first architecture clearly shows that highly dynamic reconfiguration approaches can provide flexible and high-performance hardware platforms that are smaller than standard ASIC solutions. The second architecture serves as a case study for the design of energy-efficient dynamically reconfigurable computing systems.