Field trials to detect drainage pipe networks using thermal and RGB data from unmanned aircraft

Person re-identification is about recognizing people who have passed by a sensor earlier. Previous work is mainly based on RGB data, but in this work we for the first time present a system where we combine RGB, depth, and thermal data for re-identification purposes. First, from each of the three modalities, we obtain some particular features: from RGB data, we model color information from different regions of the body, from depth data, we compute different soft body biometrics, and from thermal data, we extract local structural information. Then, the three information types are combined in a joined classifier. The tri-modal system is evaluated on a new RGB-D-T dataset, showing successful results in re-identification scenarios.

Sensor-based food intake monitoring has become one of the fastest-growing fields in dietary assessment. Researchers are exploring imaging-sensor-based food detection, food recognition, and food portion size estimation. A major problem that is still being tackled in this field is the segmentation of regions of food when multiple food items are present, mainly when similar-looking foods (similar in color and/or texture) are present. Food image segmentation is a relatively under-explored area compared with other fields. This paper proposes a novel approach to food imaging consisting of two imaging sensors: color (Red–Green–Blue) and thermal. Furthermore, we propose a multi-modal four-Dimensional (RGB-T) image segmentation using a k-means clustering algorithm to segment regions of similar-looking food items in multiple combinations of hot, cold, and warm (at room temperature) foods. Six food combinations of two food items each were used to capture RGB and thermal image data. RGB and thermal data were superimposed to form a combined RGB-T image and three sets of data (RGB, thermal, and RGB-T) were tested. A bootstrapped optimization of within-cluster sum of squares (WSS) was employed to determine the optimal number of clusters for each case. The combined RGB-T data achieved better results compared with RGB and thermal data, used individually. The mean ± standard deviation (std. dev.) of the F1 score for RGB-T data was 0.87 ± 0.1 compared with 0.66 ± 0.13 and 0.64 ± 0.39, for RGB and Thermal data, respectively.

Cross-modal co-feedback cellular automata for RGB-T saliency detection

High‐resolution unoccupied aerial vehicle (UAVs) data have alleviated the mismatch between the scale of ecological processes and the scale of remotely sensed data, while machine learning and deep learning methods allow new avenues for quantification in ecology. Ant nests play key roles in ecosystem functioning, yet their distribution and effects on entire landscapes remain poorly understood, in part because they and their mounds are too small for satellite remote sensing. This research maps the distribution and impact of ant mounds in a 20 ha treeline ecotone. We evaluate the detectability from UAV imagery using a deep learning model for object detection and different combinations of RGB, thermal and multispectral sensor data. We were able to detect ant mounds in all imagery using manual detection and deep learning. However, the highest precision rates were achieved by deep learning using RGB data which has the highest spatial resolution (1.9 cm) at comparable UAV flight height. While multispectral data were outperformed for detection, it allows for novel insights into the ecology of ants and their spatial impact on vegetation productivity using the normalized difference vegetation index. Scaling up, this suggests that ant mounds quantifiably impact vegetation productivity for up to 4% of our study area and up to 8% of the Betula nana vegetation communities, the vegetation type with the highest abundance of ant mounds. Therefore, they could have an overlooked role in nutrient‐limited tundra vegetation, and on the shrubification of this habitat. Further, we show the powerful combination UAV multi‐sensor data and deep learning for efficient ecological tracking and monitoring of mound‐building ants and their spatial impact.

Soybean (Glycine max (L.) Merill) productivity is severely compromised under conditions of excess water, particularly during the early growth stages. Depressions in converted paddy fields are believed to significantly contribute to excess water stress. Farmers typically use drainage systems, such as field ditches and pipe drains to improve field drainage. In this study, we aimed to evaluate the impacts of depressions and drainage systems on soybean growth during the early growth stages in farming fields. Using uncrewed aerial vehicles and imaging processing software, we observed the soybean coverage ratio at two and four weeks after sowing. We analyzed the relationship between the coverage ratio and depression depth, distance from field ditches, and distance from pipe drains. Depressions deeper than 2 cm were present in all study fields, and the coverage ratio increase from 2 to 4 weeks after sowing was negatively correlated with depression depth. Although the distance from pipe drains was negatively related to the coverage ratio increase, the distance from field ditches had a limited effect. The construction of pipe drains significantly improved the coverage ratio increase, particularly in areas with deep depressions and near pipe drains, by up to 23.0 times in this study. Therefore, constructing pipe drains near depressions is an effective way to mitigate the impact of depressions and excess water stress on soybean growth during the early stages. Our results will be useful for developing strategies to improve drainage and soybean growth during early growth stages.

Urban waterlogging control: A novel method to urban drainage pipes reconstruction, systematic and automated

A pedestrian detection method based on YoloV5 RGB and thermal image data fusion is proposed in this paper. This method uses two yolov5 branch structures to learn the features of RGB and thermal image data respectively, and finally uses multimodal features for fusion detection. It includes the following stages. Firstly, we use yolov5 network as a branch network to learn features from paired RGB and thermal image data. The two yoloV5-based backbone networks extract the features of the two modes for preliminary fusion, and then obtain the importance parameters of the two modes through feature compression and extraction. The effective information is enhanced, and redundant information is eliminated by multiplying the initial features. Finally, the fusion feature is used for target detection. Through this method, the detection effect is improved. We have done a lot of experiments on the public KAIST and VOT2019 pedestrian data set, and the experiments show that our method is better than the advanced method.

Thermal evaluation of buildings is essential to improving energy efficiency and mitigating the environmental impact of the construction sector. However, conventional methodologies face significant limitations, as they cannot integrate thermal and RGB data into a single 3D model, hindering the accurate identification of anomalies such as thermal bridges, heat losses, or areas with poor insulation. These challenges are further complicated by difficulties in aligning thermal and RGB images due to differences in optical properties and sensor resolutions, as well as the complexity of processing large datasets. This work proposes an innovative methodology to generate integrated 3D models combining thermal and RGB data, based on UAS-captured imagery using thermal cameras and RGB sensors, enabling the creation of cohesive 3D models that incorporate both the building’s geometry and its thermal characteristics. Using these models, statistical analysis of temperature distribution is conducted to identify thermal anomalies. Significant deviations are categorized into two main types: linear anomalies (e.g., thermal bridges) and surface anomalies (e.g., areas of heat loss), using techniques like Principal Component Analysis (PCA). This approach enables the semi-automatic detection of critical areas, optimizing the evaluation of thermal performance. The proposed methodology has been validated in a real-world case study, successfully detecting thermal anomalies with high precision and classifying their energy impact. The results were incorporated into digital twins that represent not only the geometry of the building but also its thermal behaviour, providing a powerful tool for energy audits and rehabilitation strategies. This approach represents a significant step forward in thermal inspection, promoting innovative solutions to optimize building energy performance and fostering more sustainable and resilient urban environments.

Deep neural networks achieve state-of-the-art performance on object detection tasks with RGB data. However, there are many advantages of detection using multi-modal imagery for defence and security operations. For example, the IR modality offers persistent surveillance and is essential in poor lighting conditions and 24hr operation. It is, therefore, crucial to create an object detection system which can use IR imagery. Collecting and labelling large volumes of thermal imagery is incredibly expensive and time-consuming. Consequently, we propose to mobilise labelled RGB data to achieve detection in the IR modality. In this paper, we present a method for multi-modal object detection using unsupervised transfer learning and adaptation techniques. We train faster RCNN on RGB imagery and test with a thermal imager. The images contain object classes; people and land vehicles and represent real-life scenes which include clutter and occlusions. We improve the baseline F1-score by up to 20% through training with an additional loss function, which reduces the difference between RGB and IR feature maps. This work shows that unsupervised modality adaptation is possible, and we have the opportunity to maximise the use of labelled RGB imagery for detection in multiple modalities. The novelty of this work includes; the use of the IR imagery, modality adaption from RGB to IR for object detection and the ability to use real-life imagery in uncontrolled environments. The practical impact of this work to the defence and security community is an increase in performance and the saving of time and money in data collection and annotation.

Semantic segmentation of urban scenes is an essential component in various applications of autonomous driving. It makes great progress with the rise of deep learning technologies. Most of the current semantic segmentation networks use single-modal sensory data, which are usually the RGB images produced by visible cameras. However, the segmentation performance of these networks is prone to be degraded when lighting conditions are not satisfied, such as dim light or darkness. We find that thermal images produced by thermal imaging cameras are robust to challenging lighting conditions. Therefore, in this article, we propose a novel RGB and thermal data fusion network named FuseSeg to achieve superior performance of semantic segmentation in urban scenes. The experimental results demonstrate that our network outperforms the state-of-the-art networks. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Note to Practitioners</i> —This article investigates the problem of semantic segmentation of urban scenes when lighting conditions are not satisfied. We provide a solution to this problem via information fusion with RGB and thermal data. We build an end-to-end deep neural network, which takes as input a pair of RGB and thermal images and outputs pixel-wise semantic labels. Our network could be used for urban scene understanding, which serves as a fundamental component of many autonomous driving tasks, such as environment modeling, obstacle avoidance, motion prediction, and planning. Moreover, the simple design of our network allows it to be easily implemented using various deep learning frameworks, which facilitates the applications on different hardware or software platforms.

Urban drainage pipe network is the backbone of urban drainage, flood control and water pollution prevention, and is also an essential symbol to measure the level of urban modernization. A large number of underground drainage pipe networks in aged urban areas have been laid for a long time and have reached or practically reached the service age. The repair of drainage pipe networks has attracted extensive attention from all walks of life. Since the Ministry of ecological environment and the national development and Reform Commission jointly issued the action plan for the Yangtze River Protection and restoration in 2019, various provinces in the Yangtze River Basin, such as Anhui, Jiangxi and Hunan, have extensively carried out PPP projects for urban pipeline restoration, in order to improve the quality and efficiency of sewage treatment. Based on the management practice of urban pipe network restoration project in Wuhu City, Anhui Province, this paper analyzes the problems of lengthy construction period and repeated operation caused by the mismatch between the design schedule of the restoration scheme and the construction schedule of the pipe network restoration in the existing project management mode, and proposes a model of urban drainage pipe network restoration scheme selection based on the improved support vector machine. The validity and feasibility of the model are analyzed and verified by collecting the data in the project practice. The research results show that the model has a favorable effect on the selection of urban drainage pipeline restoration schemes, and its accuracy can reach 90%. The research results can provide method guidance and technical support for the rapid decision-making of urban drainage pipeline restoration projects.

Research on intelligent control theory and strategy of gas drainage pipe network based on graph theory

In drainage engineering design, particularly, in analysis, check and hydraulic calculation of drainage pipe network, recognition and acquisition of drainage pipe network is the first and foremost task (Li et al., 2007).To implement the auto-generation of drainage pipe network drawing, this paper applies the graph theory to model the composition and data structure of a drainage pipe network. An effective method is proposed to automatically generate the directed drainage pipe network given a sink. The Breadth-first Search algorithm, Incidence-edge calculation algorithm and Transpose-graph algorithm from Boost Graph Library (BGL) are applied to implement the algorithm for the proposed method. The method provides greater automation and more efficiency but requires less inputs in the process of obtaining the practical drainage pipe network than other commonly used software do. It also lays a foundation for drainage pipe network analysis and related to hydraulic calculations. The accuracy and efficiency of the method is verified by the engineering practices described in this paper.

The rapid expansion of urban drainage pipe networks, driven by economic development, poses significant challenges for efficient monitoring and management. The complexity and scale of these networks make it difficult to effectively monitor and manage the discharge of urban domestic sewage, rainwater, and industrial effluents, leading to illegal discharges, leakage, environmental pollution, and economic losses. Efficient management relies on a rational layout of drainage pipe network monitoring points. However, existing research on optimal monitoring point layout is limited, primarily relying on manual analysis and fuzzy clustering methods, which are prone to human bias and ineffective monitoring data. To address these limitations, this study proposes a coupled model approach for the automatic optimization of monitoring point placement in drainage pipe networks. The proposed model integrates the information entropy index, Bayesian reasoning, the Monte Carlo method, and the stormwater management model (SWMM) to optimize monitoring point placement objectively and measurably. The information entropy algorithm is utilized to quantify the uncertainty and complexity of the drainage pipe network, facilitating the identification of optimal monitoring point locations. Bayesian reasoning is employed to update probabilities based on observed data, while the Monte Carlo method generates probabilistic distributions for uncertain parameters. The SWMM is utilized to simulate stormwater runoff and pollutant transport within the drainage pipe network. Results indicate that (1) the relative mean error of the parameter inversion simulation results of the pollution source tracking model is linearly fitted with the information entropy. The calculation shows that there is a good positive linear correlation between them, which verifies the feasibility of the information entropy algorithm in the field of monitoring node optimization; (2) the information entropy algorithm can be well applied to the optimal layout of a single monitoring node and multiple monitoring nodes, and it can correspond well to the inversion results of the tracking model parameters; (3) the constructed monitoring point optimization model can well realize the optimal layout of monitoring points of a drainage pipe network. Finally, the pollution source tracking model is used to verify the effectiveness of the optimal layout of monitoring points, and the whole process has less human participation and a high degree of automation. The automated monitoring point optimization layout model proposed in this study has been successfully applied in practical cases, significantly improving the efficiency of urban drainage network monitoring and reducing the degree of manual participation, which has important practical significance for improving the level of urban water environment management.

&amp;lt;p&amp;gt;Biocides used as film protection products to prevent algae and fungi growth on facades wash off during rain events and represent a potential risk to the environment. So far, urban monitoring studies focused mainly on large heterogeneous urban areas. Thus, little information about individual sources and entry pathways were obtained. However, this is important to understand the potential risk of biocide entry to groundwater.&amp;lt;/p&amp;gt;&amp;lt;p&amp;gt;This study investigates biocide emissions from a 2 ha residential area, 13 years after construction has ended. Investigated substances represent commonly used biocides for film protection, i.e. Terbutryn, Diuron and Octylisothiazolinone (OIT) and some of their known transformation products (TPs, Diuron-Desmethyl, Terbumeton, 2-Hydroxy-Terbutylazin and Terbutryn-Desethyl). We used existing urban infrastructure for efficient monitoring and applied a three-step approach to (a) determine the overall relevance of biocides, (b) identify source areas and long-term emission and (c) characterize entry pathways into surface- and groundwater.&amp;lt;/p&amp;gt;&amp;lt;p&amp;gt;Initial sampling in the swale system gave an integrated signal from the entire district and confirmed the relevance of biocide leaching, more than a decade after construction. Concentrations peaked at 174 ng/L for Diuron and 40 ng/L for Terbutryn during a high magnitude event and were above PNEC values. During later events, transformation products were detected, though at lower concentrations. For all substances, source areas were identified in a second step. Artificial elution experiments confirmed expected sources, i.e. fa&amp;amp;#231;ades, but we also found additional sources through sampling of rainfall downpipes from flat roofs. A small part of the roof fa&amp;amp;#231;ade was repainted two years before sampling and thereby showed a magnitude higher leaching rates than the remaining fa&amp;amp;#231;ades. Since all biocide wash-off arrived on a flat roof and was drained by rainfall down pipes, we could estimate net biocide emission and arrived at 155 mg Diuron, 17 mg Terbutryn, 12 mg OIT and 17 mg Diuron-Desmethyl from a 10 m&amp;lt;sup&amp;gt;2&amp;lt;/sup&amp;gt; painted fa&amp;amp;#231;ade area over a time period of two years. In a third step, we characterized entry pathways comparing samples from a drainage pipe that collected road runoff (surface pathway) with two others that collected infiltrated water on top of an underground garage (soil pathway). All drainage pipes showed Terbutryn, two of them also Diuron but none OIT. The drainage pipe representing the surface pathway showed a smaller number of individual transformation products but similar concentrations of parent compounds. One pipe representing the soil pathway had highest concentrations of Terbutryn and its TPs which suggests a high leaching potential of this biocide also away from concentrated infiltration in urban stormwater management infrastructure.&amp;lt;/p&amp;gt;