Night Surveillance Research Articles

Night time Surveillance in Communication Using YOLOv3

Nighttime surveillance presents significant challenges due to low visibility, varying lighting conditions, and interference from artificial light sources. Hence, this study proposed an effective object detection framework using the YOLOv3 model to enhance real-time monitoring in night surveillance applications, specifically within communication and security systems. YOLOv3's architecture, with its multi-scale detection and use of predefined anchor boxes, enables robust detection of objects under low-light environments and amidst light interference from vehicles and streetlights. The proposed system is tested on a night surveillance dataset, where it demonstrates high precision and speed in identifying objects, making it suitable for real-time applications. With Mean Average Precision (mAP) 87.9%, YOLOv3 effectively balances detection accuracy with inference time, ensuring minimal latency in live surveillance feeds. The results indicate that YOLOv3 outperforms traditional models such as Faster RCNN, particularly in detecting small objects under poor illumination. This approach offers a reliable solution for enhancing communication and security systems in nighttime surveillance scenarios.

Multi-surveillance detection for night surveillance using modified yolo v5 algorithm based optical communication

Abstract Night vision significantly impacts our daily visual efficiency and has more impact in communications networks. This work primarily emphasises enhancing the safety and security of individuals through advancements in night vision systems. While research on night vision is crucial for addressing contemporary social challenges, there remains a scarcity of databases tailored for deep-learning investigations in this domain. To address these challenges, this study evaluates the performance of three distinct object detection models: Yolo v3 achieving a mean average precision (mAP) of 87.9 % at 45 frames per second (FPS), Yolo v5 with a mAP of 88.7 % at 20 FPS, and modified Yolo v5 with a remarkable mAP of 95.1 % at 79 FPS. From the results, it is proven that this algorithm successfully detects classes including humans, animals, and vehicles.

Research on the perception method of tiny objects in low-light and wide-field video

At present, many trackers exhibit commendable performance in well-illuminated scenarios but overlook target tracking in low-light environments. As night falls, the tracker's accuracy drops dramatically. Challenges such as high image resolution, intricate backgrounds, uneven illumination, and the resemblance between targets and backgrounds in Hawk-Eye surveillance videos make tracking small objects in low-light and wide-field scenarios exceedingly difficult for previous trackers. To address these challenges, this paper introduces an innovative approach by integrating the difference constraint method into the CF (correlation filters) tracker, which generates a change-aware mask using inter-frame difference information. In addition, a dual regression model and inter-frame difference constraint term are introduced to restrict each other for dual filter learning. In this paper, we construct a new benchmark comprising 41 night surveillance sequences captured by Hawk-Eye cameras. Exhaustive experiments are conducted on this benchmark. The results show that the proposed method maintains superior accuracy, surpasses state-of-the-art trackers in this dataset, and achieves a real-time performance of 27 fps on a single CPU, substantially advancing tiny object tracking on Hawk-Eye surveillance videos in low light and in night scenes.

Multiscale Feature-Based Infrared Ship Detection

In this paper, based on the idea of “step-by-step accuracy”, a novel multiscale feature-based infrared ship-detection method (MSFISD) is proposed. The proposed method can achieve efficient and effective infrared ship detection in complex scenarios, which may provide assistance in applications such as night surveillance. First, candidate regions (CRs) are extracted from the whole image by extracting the sea–sky line and region of interest (ROI). The real sea–sky line is extracted based on the gradient features enhanced by large-scale gradient operators. The coarse segmentation results are obtained by the optimization method and are then refined by incorporating the edge features of the ship to reduce false alarms and obtain the CRs. Second, by analyzing the shape features of ships, the feature quantity is established, and the ships in CRs are finally accurately segmented. Experimental results demonstrate that compared with the other five methods, the proposed method has higher detection accuracy with a lower false-alarm rate and performs better in complex sea scenarios.

The translation of technological advancements to enhance patient safety through nursing innovations implemented at the Point of care

The use of internet-based surveillance technology platforms is thought to benefit patients in nursing homes by enhancing their protection and the standard of care while also streamlining the job of the caregivers. A good strategy for creating and applying digital innovations and adapting the product or service is co-creation. The purpose of this research was to examine shared creation as an adoption method and practice, as well as to determine the enablers and challenges to the use of technological surveillance technologies in the residential setting for people with dementia and roaming behavior. An inductive content review of the qualitative and quantitative data was conducted after the data had been logically analyzed using an approach for the drivers of development. A greater number of resources were needed for the execution than those involved had expected for the gradual modifications since it constituted a major change. As a whole, the joint development technique stood out as the key enabler, leading to a more secure night surveillance system. The procedure of successfully implementing novel electronic surveillance technologies within care facilities is difficult and time-consuming, and it becomes even more challenging when the innovation enables medical professionals to fundamentally alter surgical procedures at the Point of care, opening up new opportunities for individuals and caregivers to co-create value. Long-term digitization of municipality medical services necessitates the direct integration of increasingly sophisticated IT skills into the administration and delivery of medical services as well as collaboratively creating values with those using the services and their families

Modified Siamese Network Based on Feature Enhancement and Dynamic Template for Low-Light Object Tracking in UAV Videos

Unmanned aerial vehicles (UAVs) visual object tracking under low-light conditions serves as a crucial component for applications, such as night surveillance, indoor searches, night combat, and all-weather tracking. However, the majority of the existing tracking algorithms are designed for optimal lighting conditions. In low-light environments, images captured by UAV typically exhibit reduced contrast, brightness, and a signal-to-noise ratio, which hampers the extraction of target features. Moreover, the target’s appearance in low-light UAV video sequences often changes rapidly, rendering traditional fixed template tracking mechanisms inadequate, and resulting in poor tracker accuracy and robustness. This study introduces a low-light UAV object tracking algorithm (SiamLT) that leverages image feature enhancement and a dynamic template-updating Siamese network. Initially, the algorithm employs an iterative noise filtering framework-enhanced low-light enhancer to boost the features of low-light images prior to feature extraction. This ensures that the extracted features possess more critical target characteristics and minimal background interference information. Subsequently, the fixed template tracking mechanism, which lacks adaptability, is enhanced by dynamically updating the tracking template through the fusion of the reference and base templates. This improves the algorithm’s capacity to address challenges associated with feature changes. Furthermore, the Average Peak-to-Correlation Energy (APCE) is utilized to filter the templates, mitigating interference from low-quality templates. Performance tests were conducted on various low-light UAV video datasets, including UAVDark135, UAVDark70, DarkTrack2021, NAT2021, and NAT2021L. The experimental outcomes substantiate the efficacy of the proposed algorithm in low-light UAV object-tracking tasks.

Retrieving Object Motions From Coded Shutter Snapshot in Dark Environment.

Video object detection is a widely studied topic and has made significant progress in the past decades. However, the feature extraction and calculations in existing video object detectors demand decent imaging quality and avoidance of severe motion blur. Under extremely dark scenarios, due to limited sensor sensitivity, we have to trade off signal-to-noise ratio for motion blur compensation or vice versa, and thus suffer from performance deterioration. To address this issue, we propose to temporally multiplex a frame sequence into one snapshot and extract the cues characterizing object motion for trajectory retrieval. For effective encoding, we build a prototype for encoded capture by mounting a highly compatible programmable shutter. Correspondingly, in terms of decoding, we design an end-to-end deep network called detection from coded snapshot (DECENT) to retrieve sequential bounding boxes from the coded blurry measurements of dynamic scenes. For effective network learning, we generate quasi-real data by incorporating physically-driven noise into the temporally coded imaging model, which circumvents the unavailability of training data and with high generalization ability on real dark videos. The approach offers multiple advantages, including low bandwidth, low cost, compact setup, and high accuracy. The effectiveness of the proposed approach is experimentally validated under low illumination vision and provide a feasible way for night surveillance.

Rapid fabrication of highly integrated and high numerical aperture chalcogenide glass microlens arrays

The fast growing of infrared imaging system for non-contact thermography, night surveillance and night driving assistance demands highly integrated optical lenses with miniaturization feature for performance improvement. Herein, we fabricated refractive type of microlens arrays (MLAs) with high numerical aperture (NA) and high integration on commercial chalcogenide glass (As2Se3) by combining photoresist thermal reflow and soft mold imprinting process. PDMS-type concave MLAs (PDMS-MLAs) soft mold is prepared by transfer printing process on photoresist (PR) MLAs fabricated by standard UV exposure and thermal reflow. The microlens on PDMS-MLAs could be further transferred to the surface of As2Se3 glass through the hot imprinting technique, forming chalcogenide glass-based MLAs (ChG-MLAs). Morphology detection reveals that microlens in the MLAs has an average aperture and sag height of approximately 13.4 and 1.7 μm. Moreover, the optical performance investigation demonstrated that the ChG-MLAs with 700 × 700 array size (10.5 × 10.5 mm) showed good structure uniformity and focusing capability, and through the FDTD simulation, it further evidenced that focal lengths of 6.3–8.1 µm with high numerical aperture of 0.64–0.73 (wavelength range: 3–5 μm) was achieved.

Review of Application of Thermal Imaging for Face Recognition

Recent advances in facial recognition utilizing infrared as a source are described. Recent research has concentrated on face identification using visible light, with the main issue being that the lighting on the face varies in outside circumstances. Recent studies employ infrared light as a source to produce infrared face pictures to overcome this and increase performance. This is known as a thermal face image, and it is extremely valuable in a variety of application systems. Night surveillance systems and military applications are two applications where night vision comes into picture. The choice of infrared, intensity fluctuation, and angle of incidence all play crucial roles in these applications. Keywords: Face recognition, multi spectral images, LBP, SIFT

Global-Local Graph Convolutional Network for cross-modality person re-identification

Visible-thermal person re-identification (VT-ReID) is an important task for retrieving pedestrian between visible and thermal modality. It makes up for the drawbacks of single modality person re-identification in night surveillance applications. Most of the existing methods extract the features of different images/parts independently which ignore the potential relationship between them. In this paper, we propose a novel Global-Local Graph Convolutional Network (GLGCN) to learn discriminative feature representation by modeling the relation through graph convolutional network. The local graph module builds the potential relation of different body parts within each modality to extract discriminative part-level features. The global module constructs the contextual relation of same identity across two modalities to reduce the modality discrepancy. By training the two modules jointly, the robustness of the model can be further improved. The experiment results on the SYSU-MM01 and RegDB datasets demonstrate that our model outperforms the state-of-the-art methods.

Visible and Near-Infrared Image Acquisition and Fusion for Night Surveillance

Image fusion combines images with different information to create a single, information-rich image. The process may either involve synthesizing images using multiple exposures of the same scene, such as exposure fusion, or synthesizing images of different wavelength bands, such as visible and near-infrared (NIR) image fusion. NIR images are frequently used in surveillance systems because they are beyond the narrow perceptual range of human vision. In this paper, we propose an infrared image fusion method that combines high and low intensities for use in surveillance systems under low-light conditions. The proposed method utilizes a depth-weighted radiance map based on intensities and details to enhance local contrast and reduce noise and color distortion. The proposed method involves luminance blending, local tone mapping, and color scaling and correction. Each of these stages is processed in the LAB color space to preserve the color attributes of a visible image. The results confirm that the proposed method outperforms conventional methods.

Photogrammetric Digital Surface Model Reconstruction in Extreme Low-Light Environments

Digital surface models (DSM) have become one of the main sources of geometrical information for a broad range of applications. Image-based systems typically rely on passive sensors which can represent a strong limitation in several survey activities (e.g., night-time monitoring, underground survey and night surveillance). However, recent progresses in sensor technology allow very high sensitivity which drastically improves low-light image quality by applying innovative noise reduction techniques. This work focuses on the performances of night-time photogrammetric systems devoted to the monitoring of rock slopes. The study investigates the application of different camera settings and their reliability to produce accurate DSM. A total of 672 stereo-pairs acquired with high-sensitivity cameras (Nikon D800 and D810) at three different testing sites were considered. The dataset includes different camera configurations (ISO speed, shutter speed, aperture and image under-/over-exposure). The use of image quality assessment (IQA) methods to evaluate the quality of the images prior to the 3D reconstruction is investigated. The results show that modern high-sensitivity cameras allow the reconstruction of accurate DSM in an extreme low-light environment and, exploiting the correct camera setup, achieving comparable results to daylight acquisitions. This makes imaging sensors extremely versatile for monitoring applications at generally low costs.

Carbon-Bridged 1,2-Bis(2-thienyl)ethylene: An Extremely Electron Rich Dithiophene Building Block Enabling Electron Acceptors with Absorption above 1000 nm for Highly Sensitive NIR Photodetectors.

The emerging donor-acceptor-donor (A-D-A)-type nonfullerene acceptors (NFAs) featuring near-infrared (NIR) photoresponsivity have greatly boosted the development of organic photovoltaics (OPVs) and display great potential for sensitive NIR organic photodetectors (OPDs). However, NIR NFAs with absorption above 1000 nm, which is of great importance for application in NIR OPDs for bioimaging, remote communication, night surveillance, etc., are still rare due to the scarcity of strong electron-rich cores. We report herein a new dithiophene building block, namely PDT, which exhibits the strongest electron-donating ability among the widely used dithiophene building blocks. By applying PDT and PDTT as the electron-donating cores and DFIC as the electron-accepting terminals, we developed two new NIR electron acceptors, PDTIC-4F and PDTTIC-4F, with optical absorptions up to 1030 nm, surpassing that of the well-known O6T-4F acceptor. In comparison with the carbon-oxygen-bridged core COi8 in O6T-4F, the synthetic complexity of PDT and PDTT is significantly reduced. Conventional OPV devices based on PM6:PDTTIC-4F display power conversion efficiencies (PCEs) of up to 10.70% with a broad external quantum efficiency (EQE) response from the ultraviolet-visible to the infrared, leading to a high short-circuit current density (Jsc) of 25.90 mA cm-2. Encouraged by these results, we investigated inverted PM6:PDTTIC-4F-based OPD devices by suppressing the dark current via modulation of the film thickness. The optimal OPD device exhibits compelling performance metrics that can compete with those of commercial silicon photodiodes: a record responsivity of 0.55 A W-1 (900 nm) among photodiode-type OPDs and excellent shot-noise-limited specific detectivity (Dsh*) of over 1013 jones.

Bright and persistent green and red light-emitting fine fibers: A potential candidate for smart textiles

The properties of smart light-emitting textile/fabric with a persistent afterglow can be beneficial for use in anti-counterfeiting, night surveillance, and security trooping among other potential applications. Keeping this in mind, this work is a step forward in designing bright green and red emitting poly vinyl alcohol (PVA) fine fibers. We first synthesized green emitting Zn2GeO4:Mn2+ (ZGOM) and red emitting ZnGa2O4:Cr3+ (ZGOC) nanoparticles using a hydrothermal process. Field Emission Scanning Electron Microscope (FESEM) images depicted the formation of nanorods and sub-10 nm nanoparticles for ZGOM and ZGOC, respectively. These nanoparticles were mixed in a PVA solution and spun into fiber using the Forcespinning® technology. FESEM images of the fiber samples show the presence of long, bead-free, defect-free, smooth surfaces with the diameters ranging from 200 nm to 1.2 μm depending on selected processing parameters; a fiber system was selected considering fiber output and fiber diameter, the selected system had fibers with average diameter of 900 nm for both ZGOM and ZGOC encapsulated PVA fine fibers (MG-PF and CR-PF). Another favorable property was that the thermal stability of the PVA fine fibers was not affected by the nanoparticles. Both MF-PF and CR-PF fine fibers depicted a bright green and red luminescence, respectively, under UV excitation. The fiber can easily accommodate loading up to 1.0% weight without doing any kind of fluorescence quenching. MG-PF fiber displayed a bright and substantial green persistence luminescence up to 500 s. In the case of CR-PF there is a quenching of trap states and therefore intrinsic red persistence luminescence is not retained beyond 100 s. We believe such green fiber if smartly woven in any kind of textile materials/fabric can be an efficient solution to deal with cloth counterfeiting and night surveillance.

Sub-Sampled Time Domain Sparse Information in Wired OFDM and STBC-OFDM Systems

A sub-sampling method developed for wired or wireless Orthogonal Frequency Division Multiplexing (OFDM) systems is presented. The proposed technique can be activated during the exchange of sparse data. The sub-sampling method is implemented with low cost and complexity hardware since it requires the customization of a few specific OFDM stages like channel interleaving, sampling control, etc, and does not require any iterative optimization procedure. Employing special Inverse-Fast Fourier Transform (I-FFT) input structures, the recovery of the original data from fewer samples than those required by the receiver FFT is possible without any other cost for general OFDM applications. A special detector initiates the sub-sampling mode when sparse information is exchanged like sensor data or uncompressed real-time video with stationary views like night surveillance cameras. Up to ¾ of the time, the system can operate in sub-sampling mode reducing the power consumed by the Analogue/Digital Converter (ADC), the FFT and the IFFT. Memory needed for sample storage, intermediate results and FFT/IFFT coefficients can be released in real time if the proposed architecture is adopted. A Bit Error Rate (BER) below 10−3 or full information recovery is achieved in most of the cases. An image reconstruction example is also tested in a wireless environment with Normalised Mean Square Error (NMSE) ranging from 10−5 to less than 0.1.

Captive balloon image object detection system using deep learning

The surveillance of large areas to ensure local security requires remote sensors with high temporal and spatial resolution. Captive balloons with infrared and visible sensors, like ALTAVE captive balloon system, can perform a long-term day–night surveillance and provide security of large areas by monitoring people and vehicles, but it is an exhaustive task for a human. In order to provide a more efficient and less arduous monitoring, a deep learning model was trained to detect people and vehicles in images from captive balloons infrared and visible sensors. Two databases containing about 700 images each, one for each sensor, were manually built. Two networks were fine-tuned from a pretrained faster region-based convolution neural network (R-CNN). The network reached accuracies of 87.1% for the infrared network and 86.1% for the visible one. Both networks were able to satisfactorily detect multiple objects in an image with a variety of angles, positions, types (for vehicles), scales, and even with some noise and overlap. Thus a faster R-CNN pretrained only in common RGB (red, green, and blue) images can be fine-tuned to work satisfactorily on visible remote sensing (RS) images and even on the infrared RS images.

Deep-learning based single object tracker for night surveillance

Tracking an object in night surveillance video is a challenging task as the quality of the captured image is normally poor with low brightness and contrast. The task becomes harder for a small object as fewer features are apparent. Traditional approach is based on improving the image quality before tracking is performed. In this paper, a single object tracking algorithm based on deep-learning approach is proposed to exploit its outstanding capability of modelling object’s appearance even during night. The algorithm uses pre-trained convolutional neural networks coupled with fully connected layers, which are trained online during the tracking so that it is able to cater for appearance changes as the object moves around. Various learning hyperparameters for the optimization function, learning rate and ratio of training samples are tested to find optimal setup for tracking in night scenarios. Fourteen night surveillance videos are collected for validation purpose, which are captured from three viewing angles. The results show that the best accuracy is obtained by using Adam optimizer with learning rate of 0.00075 and sampling ratio of 2:1 for positive and negative training data. This algorithm is suitable to be implemented in higher level surveillance applications such as abnormal behavioral recognition.

Training configuration analysis of a convolutional neural network object tracker for night surveillance application

<span lang="EN-US">Automated surveillance during the night is important as it is the period when crimes usually happened. By providing continuous monitoring, coupled with a real-time alert system, appropriate action can be taken immediately if a crime is detected. However, low lighting conditions during the night can degrade the quality of surveillance videos, where the captured images will have low contrast and less discriminative features. Consequently, these factors contribute to the problem of bad appearance representation of the object of interest in the tracking algorithm. Thus, a convolutional neural network-based object tracker for night surveillance is proposed by exploiting the deep feature strength in representing object features spatially and semantically. The proposed convolutional network consists of six layers that consist of three convolutional neural networks (CNN) and three fully connected (FC) layers. The network will be trained by using a binary classifier approach of objects and its background classes, which is updated on a fixed interval so that it fully encapsulates the changes in object appearance as it moves in the scene. The algorithm has been tested with different sets of training data configurations to find the best optimum ones with regards to VOT2015 evaluation protocols, tested on 14-night surveillance videos. The results show that the configuration of a total of 250 training samples with a sample ratio of 4:1 between positive and negative data delivers the best performance for the sequence length of [1,550]. It can be inferred that more information on the object is required compared to the background, where the background might be homogeneous due to low lighting conditions. In conclusion, this algorithm is suitable to be implemented for night surveillance application.</span>

Optical design of reflecting omnidirectional zoom optical system with peripheral half-field of view from 110° to 72° for day and night surveillance

A reflecting omnidirectional zoom optical system (ROZOS) that is simultaneously operative in the visible-light (400 − 700 nm) and long-wave infrared (8 − 12 μm) ranges is designed for day and night unmanned surveillance and security. The half-field of view (HFOV) and F-number for the developed ROZOS are 40 − 110° and 2.4, respectively, enabling omnidirectional vision. In order to expand the concentrated image in the region of the central blocking area on the image sensor, we successfully established a ROZOS by two moving mirrors to play the role of the focator/variator and compensator, respectively. The zoom magnification and maximum lower HFOV vary from 1 to 1.24 and from 110° to 72° during zooming, respectively. During zooming, the modulation transfer function (MTF) exceed 0.30 at Nyquist frequencies of 34 lp/mm and 20 lp/mm, respectively, at all maximum lower HFOVs from 110° to 72° in the visible and long-wave infrared wavelength ranges. No performance issues are encountered as the ray aberration is very small over the entire zooming range. Tolerance analysis show that when the MTF is 0.3, the cumulative probability over the entire zooming range exceed 90 %.

Objective evaluation method for advance thermal imagers based on minimum resolvable temperature difference

Advancement in thermal imaging technology has improved the day and night surveillance capability of electro-optical system. Thermal imager detects the object of interest by sensing the temperature difference. The performance evaluation of these thermal imagers plays a vital role prior to effective deployment. Key performance parameter of thermal imaging system is minimum resolvable temperature difference (MRTD), as it is able to determine the overall system performance involving a human operator. MRTD is capable of defining the minimum temperature difference required to resolve the bar target pattern of defined spatial frequencies and provides the true estimate of the range performance of a thermal imaging system. In conventional methods, 4-bar target of defined resolution is exposed to the minimum resolvable condition to compute MRTD values. Presently, subjective MRTD measurement employs an operator to observe the minimum resolvable condition, and therefore, measurement suffers from poor repeatability. During large volume production, this practice becomes a tedious procedure to evaluate large numbers of thermal imaging sights. To ease the performance evaluation procedure, an objective MRTD measurement method for the evaluation of the thermal imaging system has been presented. In the present work, objective MRTD measurement method for advance thermal imager by computing the modulation transfer function and noise equivalent temperature difference has been presented. Experiments have been conducted to measure subjective and objective MRTDs. Results for both subjective and objective MRTD measurements have been compared and presented. It is found that the proposed method is able to produce very close result within accuracy of more than 95% than that of conventional methods.