Current Input Image Research Articles

Dynamic Keypoint Detection Network for Image Matching.

Establishing effective correspondences between a pair of images is difficult due to real-world challenges such as illumination, viewpoint and scale variations. Modern detector-based methods typically learn fixed detectors from a given dataset, which is hard to extract repeatable and reliable keypoints for various images with extreme appearance changes and weakly textured scenes. To deal with this problem, we propose a novel Dynamic Keypoint Detection Network (DKDNet) for robust image matching via a dynamic keypoint feature learning module and a guided heatmap activator. The proposed DKDNet enjoys several merits. First, the proposed dynamic keypoint feature learning module can generate adaptive keypoint features via the attention mechanism, which is flexibly updated with the current input image and can capture keypoints with different patterns. Second, the guided heatmap activator can effectively fuse multi-group keypoint heatmaps by fully considering the importance of different feature channels, which can realize more robust keypoint detection. Extensive experimental results on four standard benchmarks demonstrate that our DKDNet outperforms state-of-the-art image-matching methods by a large margin. Specifically, our DKDNet can outperform the best image-matching method by 2.1% in AUC@ 3px on HPatches, 3.74% in AUC@ 5° on ScanNet, 7.14% in AUC@ 5° on MegaDepth and 12.32% in AUC@ 5° on YFCC100M.

Teacher or supervisor? Effective online knowledge distillation via guided collaborative learning

Knowledge distillation is a widely-used and effective technique to boost the performance of a lightweight student network, by having it mimic the behavior of a more powerful teacher network. This paper presents an end-to-end online knowledge distillation strategy, in which several peer students are trained together and their predictions are aggregated into a powerful teacher ensemble via an effective ensembling technique that uses an online supervisor network to determine the optimal way of combining the student logits. Intuitively, this supervisor network learns the area of expertise of each student and assigns a weight to each student accordingly►it has knowledge of the input image, the ground truth data, and the predictions of each individual student, and tries to answer the following question: “how much can we rely on each student’s prediction, given the current input image with this ground truth class?”. The proposed technique can be thought of as an inference optimization mechanism as it improves the overall accuracy over the same number of parameters. The experiments we performed show that the proposed knowledge distillation consistently improves the performance of the knowledge-distilled students vs. the independently trained students.

Visual gesture recognition based on hand key points

In recent years, computers have become a part of people's daily life, and the interaction between people and computers has increasingly become a hot spot in the field of scientific research. Gesture recognition based on vision is an indispensable part of the new generation of human-computer interaction. This paper presents a visual gesture recognition method based on hand key points, which realizes the detection of hand key points in the current input image and the recognition of defined gestures. It provides a new technical scheme for human-computer interaction application. Compared with the current mainstream solutions that directly train the defined gestures and obtain the gestures in the input image through template comparison detection, this paper first uses the rigid characteristics of the palm to design a palm detection model instead of directly detecting the entire hand. After detecting the presence of hand in the picture, the hand key point model locates 21 3D hand key point coordinates in the previously detected hand area through direct coordinate prediction. Finally, the meaning of gestures in the input image is obtained through the positional relationship between the key nodes. This solution achieves 95.7% of the accuracy of detection, and improves the FPS from 21-23 to 30-31.

Automated framework for accurate segmentation of leaf images for plant health assessment.

Leaf segmentation is significantly important in assisting ecologists to automatically detect symptoms of disease and other stressors affecting trees. This paper employs state-of-the-art techniques in image processing to introduce an accurate framework for segmenting leaves and diseased leaf spots from images. The proposed framework integrates an appearance model that visually represents the current input image with the color prior information generated from RGB color images that were formerly saved in our database. Our framework consists of four main steps: (1) Enhancing the accuracy of the segmentation at minimum time by making use of contrast changes to automatically identify the region of interest (ROI) of the entire leaf, where the pixel-wise intensity relations are described by an electric field energy model. (2) Modeling the visual appearance of the input image using a linear combination of discrete Gaussians (LCDG) to predict the marginal probability distributions of the grayscale ROI main three classes. (3) Calculating the pixel-wise probabilities of these three classes for the color ROI based on the color prior information of database images that are segmented manually, where the current and prior pixel-wise probabilities are used to find the initial labels. (4) Refining the labels with the generalized Gauss-Markov random field model (GGMRF), which maintains the continuity. The proposed segmentation approach was applied to the leaves of mangrove trees in Abu Dhabi in the United Arab Emirates. Experimental validation showed high accuracy, with a Dice similarity coefficient 90% for distinguishing leaf spot from healthy leaf area.

Fuzzy system based human behavior recognition by combining behavior prediction and recognition

With the development of intelligent surveillance systems, human behavior recognition has been extensively researched. Most of the previous methods recognized human behavior based on spatial and temporal features from (current) input image sequences, without the behavior prediction from previously recognized behaviors. Considering an example of behavior prediction, “punching” is more probable in the current frame when the previous behavior is “standing” as compared to the previous behavior being “lying down.” Nevertheless, there has been little study regarding the combination of currently recognized behavior information with behavior prediction. Therefore, we propose a fuzzy system based behavior recognition technique by combining both behavior prediction and recognition. To perform behavior recognition during daytime and nighttime, a dual camera system of visible light and thermal (far infrared light) cameras is used to capture 12 datasets including 11 different human behaviors in various surveillance environments. Experimental results along with the collected datasets and open database showed that the proposed method achieved higher accuracy of behavior recognition when compared to conventional methods.

정지 물체를 고려한 적응적 배경생성 알고리즘

배경과 현재 프레임 영상간의 차영상을 이용하여 이동 물체를 탐지하는 방법은 비디오 감시 시스템에서 가장 보편적인 방법 중 하나이지만 신뢰할 수 있는 배경의 생성은 여전히 쉽지 않은 문제이다. 본 논문에서는 정지 물체를 고려한 적응적 배경 생성 기법을 제안한다. 연속적으로 입력되는 영상들의 산술 평균을 이용하여 초기 배경을 생성한다. 배경과 현재 영상간의 차영상을 구하여 물체를 탐지한 다음, 탐지된 물체가 일정시간이상 계속 정지해 있는 경우에는 그 물체를 정지 물체로 간주하고 정지 물체 영역을 배경으로 갱신한다. 한편, 이동 물체인 경우에는 배경 갱신에서 현재 프레임을 배제함으로써 지속적으로 물체를 탐지할 수 있도록 한다. 제안된 방법은 점진적인 조명의 변화, 느리게 이동하는 물체, 정지 물체 등이 존재하는 동영상에서도 적응적으로 배경을 생성할 수 있으며 이는 실험을 통해 확인되었다. In the intelligent video surveillance system, moving objects generally are detected by calculating difference between background and input image. However formation of reliable background is known to be still challenging task because it is hard to cope with the complicated background. In this paper we propose an adaptive background formation algorithm considering stationary object. At first, the initial background is formed by averaging the initial N frames. Object detection is performed by comparing the current input image and background. If the object is at a stop for a long time, we consider the object as stationary object and background is replaced with the stationary object. On the other hand, if the object is a moving object, the pixels in the object are not reflected for background modification. Because the proposed algorithm considers gradual illuminance change, slow moving object and stationary object, we can form background adaptively and robustly which has been shown by experimental results.

Mechanisms of human place recognition

Mechanisms of human place recognition

다양한 실내 환경변수로부터 강인한 객체 검출

본 논문에서는 다양한 환경변수가 존재하는 실내에서 효율적인 객체를 검출하기 위한 방법을 제안한다. 일반적으로 실내 환경은 조명의 변화와 객체에 의해 발생된 그림자, 바닥면에 반사된 조명성분 등으로 인하여 정확한 객체 검출이 이루어지기 어려운 환경이다. 먼저 객체검출을 위한 배경영상을 생성한다. 영상 내에 객체가 존재하는 경우 이전에 생성된 배경영상과 현재 입력영상간의 유사도 비교를 통해 보정된 배경영상을 생성한다. 배경영상과 입력영상으로 생성한 평균영상과 보정된 배경영상을 이용하여 혼합영상을 생성한다. 마지막으로 혼합영상을 이용하여 입력된 영상으로부터 객체를 검출한다. 검출된 객체를 보완하기 위해 레이블링 과정을 통해 잡음 성분을 제거한 후 모폴로지 기법을 적용하여 객체영역 보완한다. 따라서 조명의 변화나 그림자와 같은 환경변수로부터 강인한 객체를 검출한다. 본 논문에서 제안한 시스템은 변형된 조명성분과 그림자 성분이 포함되어 있는 혼합영상을 사용하기 때문에 기존시스템보다 객체영역 검출이 더욱 효과적이다. In this paper, we propose a detection method of reduced computational complexity aimed at separating the moving objects from the background in a generic video sequence. In generally, indoor environments, it is difficult to accurately detect the object because environmental factors, such as lighting changes, shadows, reflections on the floor. First, the background image to detect an object is created. If an object exists in video, on a previously created background images for similarity comparison between the current input image and to detect objects through several operations to generate a mixture image. Mixed-use video and video inputs to detect objects. To complement the objects detected through the labeling process to remove noise components and then apply the technique of morphology complements the object area. Environment variable such as, lighting changes and shadows, to the strength of the object is detected. In this paper, we proposed that environmental factors, such as lighting changes, shadows, reflections on the floor, including the system uses mixture images. Therefore, the existing system more effectively than the object region is detected.

DETECTING GHOST AND LEFT OBJECTS IN SURVEILLANCE VIDEO

This paper proposes an efficient method for detecting ghost and left objects in surveillance video, which, if not identified, may lead to errors or wasted computational power in background modeling and object tracking in video surveillance systems. This method contains two main steps: the first one is to detect stationary objects, which narrows down the evaluation targets to a very small number of regions in the input image; the second step is to discriminate the candidates between ghost and left objects. For the first step, we introduce a novel stationary object detection method based on continuous object tracking and shape matching. For the second step, we propose a fast and robust inpainting method to differentiate between ghost and left objects by reconstructing the real background using the candidate's corresponding regions in the current input and background image. The effectiveness of our method has been validated by experiments over a variety of video sequences and comparisons with existing state-of-art methods.

Video Stabilization for a Camcorder Mounted on a Moving Vehicle

Vision systems play an important role in many intelligent transportation systems (ITS) applications, such as traffic monitoring, traffic law reinforcement, driver assistance, and automatic vehicle guidance. These systems installed in either outdoor environments or vehicles have often suffered from image instability. In this paper, a video stabilization technique for a camcorder mounted on a moving vehicle is presented. The proposed approach takes full advantage of the a priori information of traffic images, significantly reducing the computational and time complexities. There are four major steps involved in the proposed approach: global feature extraction, camcorder motion estimation, motion taxonomy, and image compensation. We begin with extracting the global features of lane lines and the road vanishing point from the input image. The extracted features are then combined with those detected in previous images to compute the camcorder motion corresponding to the current input image. The computed motion consists of both expected and unexpected components. They are discriminated and the expected motion component is further smoothed. The resulting motion is next integrated with a predicted motion, which is extrapolated from the previous desired camcorder motions, leading to the desired camcorder motion associated with the input image under consideration. The current input image is finally stabilized based on the computed desired camcorder motion using an image transformation technique. A series of experiments with both real and synthetic data have been conducted. The experimental results have revealed the effectiveness of the proposed technique.

A tunable algorithm to update a reference image

A change detection scheme used to detect objects in a complex real-life scene must be able to deal with illumination changes, shadows and structural variations of the environment. Several approaches are based on subtracting a reference image, representing the background, from the current input image. The most used methods estimate the background image by applying some low-pass filter on the input image sequence. Many of them require an accurate calibration phase and rely on a careful selection of critical parameters. An algorithm based on Kalman filtering is suggested here to dynamically estimate the background reference image. The approach extends former works and faces the severe problems of parameter tuning and modeling approximations. An experimental analysis on the behavior of the proposed algorithm in presence of different illumination changes is performed using noisy synthetic data. The results are used to address the choice of values for the filter parameters. The effectiveness and robustness of the algorithm are evaluated on several tests that were carried out on real-life sequences.

Correction to Construction of Panoramic Image Mosaics with Global and Local Alignment

An important element of this framework, which we exploit, is to perform the motion estimation between the current new input image and a warped (resampled) version of the mosaic. This allows us to estimate only incremental deformations of images (or equivalently, instantaneous motion), which greatly simplifies the computation of the gradients and Hessians required in our gradient descent algorithm (e.g., compare the Hessians computed below with those presented in Szeliski, 1996). Thus, to register two images I0(x) and I1(x′), where x′ is computed using some parametric motion model m, i.e., x′ = f(x; m), we first compute the warped image