Object Information Research Articles

Vehicle and Pedestrian Detection Based on Improved YOLOv7-Tiny

To improve the detection accuracy of vehicles and pedestrians in traffic scenes using object detection algorithms, this paper presents modifications, compression, and deployment of the single-stage typical algorithm YOLOv7-tiny. In the model improvement section: firstly, to address the problem of small object missed detection, shallower feature layer information is incorporated into the original feature fusion branch, forming a four-scale detection head; secondly, a Multi-Stage Feature Fusion (MSFF) module is proposed to fully integrate shallow, middle, and deep feature information to extract more comprehensive small object information. In the model compression section: the Layer-Adaptive Magnitude-based Pruning (LAMP) algorithm and the Torch-Pruning library are combined, setting different pruning rates for the improved model. In the model deployment section: the V7-tiny-P2-MSFF model, pruned by 45% using LAMP, is deployed on the embedded platform NVIDIA Jetson AGX Xavier. Experimental results show that the improved and pruned model achieves a 12.3% increase in mAP@0.5 compared to the original model, with parameter volume, computation volume, and model size reduced by 76.74%, 7.57%, and 70.94%, respectively. Moreover, the inference speed of a single image for the pruned and quantized model deployed on Xavier is 9.5 ms.

Semantic Segmentation of Satellite Images for Landslide Detection Using Foreground-Aware and Multi-Scale Convolutional Attention Mechanism.

Advancements in satellite and aerial imagery technology have made it easier to obtain high-resolution remote sensing images, leading to widespread research and applications in various fields. Remote sensing image semantic segmentation is a crucial task that provides semantic and localization information for target objects. In addition to the large-scale variation issues common in most semantic segmentation datasets, aerial images present unique challenges, including high background complexity and imbalanced foreground-background ratios. However, general semantic segmentation methods primarily address scale variations in natural scenes and often neglect the specific challenges in remote sensing images, such as inadequate foreground modeling. In this paper, we present a foreground-aware remote sensing semantic segmentation model. The model introduces a multi-scale convolutional attention mechanism and utilizes a feature pyramid network architecture to extract multi-scale features, addressing the multi-scale problem. Additionally, we introduce a Foreground-Scene Relation Module to mitigate false alarms. The model enhances the foreground features by modeling the relationship between the foreground and the scene. In the loss function, a Soft Focal Loss is employed to focus on foreground samples during training, alleviating the foreground-background imbalance issue. Experimental results indicate that our proposed method outperforms current state-of-the-art general semantic segmentation methods and transformer-based methods on the LS dataset benchmark.

Research on Path of Promoting the Reform of "Three Education" in Electronic Information Major of Higher Vocational Colleges in New Era

The Ministry of Education clearly proposed the "three education reform" (the reform of teachers, teaching materials, and teaching methods) of vocational colleges in 2019. It focused on enhancing the attractiveness of vocational education and the ability to cultivate talents. In recent years, with the digital transformation of the industry, the integration of "three education" has become a critical path to promote the high-quality development of vocational education. Under the guidance of the new market demands in the new era, the reform of teachers is the leader, the reform of teaching materials is the carrier, and the reform of teaching methods is the medium. Under the premise of meeting the trend of digital and intelligent development of the industry, it can cultivate more comprehensive talents for the society. This paper takes higher vocational colleges as the research object and electronic information major as the research entry point, it analyzes the current situation of the reform of the three education, and propose measures and methods to promote the reform of the three education under the new background in the new era. It can provide support for cultivating comprehensive and professional talents to meet the position needs in the new era.

Weak Edge Target Segmentation Network Based on Dual Attention Mechanism

Segmentation of weak edge targets such as glass and plastic poses a challenge in the field of target segmentation. The detection process is susceptible to background interference and various external factors due to the transparent nature of these materials. To address this issue, this paper introduces a segmentation network for weak edge target objects (WETS-Net). To effectively extract edge information of such objects and eliminate redundant information during feature extraction, a dual-attention mechanism is employed, including the Edge Attention Extraction Module (EAEM) and the Multi-Scale Information Fusion Module (MIFM). Specifically, the EAEM combines improved edge feature extraction kernels to selectively enhance the importance of edge features, aiding in more precise target region extraction. The MIFM utilizes spatial attention mechanisms to fuse multi-scale features, reducing background and external interference. These innovations enhance the performance of WETS-Net, offering a new direction for weak edge target segmentation research. Finally, through ablation experiments, the effectiveness of each module is effectively validated. Moreover, the proposed algorithm achieves an average detection accuracy of 95.83% and 96.13% on the dataset and a self-made dataset, respectively, outperforming similar U-Net-improved networks.

Reinforcement Learning with Decoupled State Representation for Robot Manipulations

Deep reinforcement learning has significantly advanced robot manipulations by providing an alternative solution for designing control strategies using raw images as direct inputs. While images offer additional environmental information, the end-to-end policy training manner (from image to action) requires simultaneous representation and task learning by the agent. This often necessitates a substantial number of interaction samples to achieve satisfactory policy performance. Previous works has attempted to address this challenge by learning a visual representation model that encodes the entire image into a low-dimensional vector before the policy training. However, since this vector contains both robot and object information, it inevitably introduces coupling within the state, which can mislead the policy training process. In this study, a novel method called Reinforcement Learning with Decoupled State Representation is proposed to effectively decouple robot and object information within the state representation. Experimental results demonstrate that the proposed method exhibits faster learning speed and achieves superior performance compared to previous methods across various robot manipulation tasks. Moreover, with only 3096 offline images, the proposed method successfully applies to real-world robot pushing tasks, which demonstrates its high practicability.

Remote sensing image fusion method based on depth feature extraction

Abstract Remote sensing image fusion involves combining data from various remote sensing sensors or merging multiple images acquired by the same sensor but in different spectral bands to create a superior quality composite image, high resolution, multi-band, and multi-temporal remote sensing image through certain algorithms and technologies. By utilizing fusion techniques, remote sensing images of varying bands, resolutions, and phases can be combined to enhance the accuracy of ground object information. This paper introduces a remote sensing image fusion method that leverages deep learning for depth feature extraction. Swin Transformer makes full use of the characteristics of the relationship between adjacent pixels, and adaptively extracts panchromatic image (PAN) and multispectral image (MS) features through two adaptive feature extraction channels. High-quality satellite data is employed to confirm the efficacy of the suggested approach, with the findings demonstrating strong performance in ERGAS, SCC, SSIM, and PSNR, and other evaluation indexes and has good spectral information fidelity and spatial information integration degree.

SGIR-Tree: Integrating R-Tree Spatial Indexing as Subgraphs in Graph Database Management Systems

Efficient spatial query processing in Graph Database Management Systems (GDBMSs) has become increasingly important owing to the prevalence of spatial graph data. However, current GDBMSs lack effective spatial indexing, causing performance issues with complex spatial graph queries. This study proposes a spatial index called Subgraph Integrated R-Tree (SGIR-Tree) for efficient spatial query processing in GDBMSs. The SGIR-Tree integrates the hierarchical R-Tree structure with the graph structure of GDBMSs by converting R-Tree elements into graph components like nodes and edges. The Minimum Bounding Rectangle (MBR) information of spatial objects and R-Tree nodes is stored as properties of these graph elements, and the leaf nodes are directly connected to the spatial nodes. This approach combines the efficiency of spatial indexing with the flexibility of graph databases, thereby allowing spatial query results to be directly utilized in graph traversal. Experiments using OpenStreetMap datasets demonstrate that the SGIR-Tree outperforms the previous approaches in terms of query overhead and index overhead. The results are expected to improve spatial graph data processing in various fields, including location-based service and urban planning, significantly advancing spatial data management in GDBMSs.

Semantic ghost imaging based on semantic coding

To address the challenge of reconstructing high-quality images under low sampling during the Ghost Imaging (GI) procedure, we propose a semantic GI method based on semantic encoding. Through training, the obtained continuous weights of the convolution kernels are served as the speckle patterns. A processing module and a Recurrent Neural Network (RNN) are then employed to decode and reconstruct the images. Both the speckle patterns and their corresponding bucket values are optimized, so that the semantic information of the target object can be better extracted. With different gated recurrent units (GRU) layers for the target objects in different datasets, the feasibility of the proposed GI method is validated by the numerical simulations and experiments on the simple target objects (the handwriting digits “0” to “9”) and more complex target objects (“NUPT”). The results show that the target objects (the handwriting digits) can be reconstructed with higher quality even at a lower sampling rate of 1.28%. Additionally, the proposed method has applicability for more complex objects (such as “NUPT”) in real applications. In comparison with those results by using traditional ghost imaging (TGI), deep convolution auto-encoder network (DCAN), and RNN-based GI (GI-RNN), the proposed GI method shows a better performance in terms of the quality of reconstructed images and the training time, that is, the proposed method can have both good reconstructed image quality and less training time. By introducing the concept of semantic communication into GI, the proposed method provides a new idea for the model-based GI.

Highly resilient aerogel/sponge nested structures assisted multimodal tactile sensory system for robotic embodied perception

Large language models (LLMs) exhibit outstanding reasoning abilities when robots perform tasks, but are limited to the passively receiving information and lack multi-feature perception of realistic objects. Humans obtain complementary and coupled information from interactions with objects through the skin. Here, a multimodal tactile sensory system is reported to assist the LLM-equipped robots in actively perceiving and understanding multiple features of objects including material, compliance and thermal insulation. A multi-parametric tactile sensing array, where each sensor in the array comprises of a triboelectric unit, a decoupling pressure-temperature dual-mode sensing unit and a heating unit, is constructed to acquire the multiparametric tactile signals. The pressure-temperature sensing unit is fabricated from the developed aerogel/sponge nested composite structures, which exhibit stable compression resilience under the compression strain of over 90 %, impressive fatigue resistance with stable responses after 10,000 compression cycles, and linear thermoelectric responses over wide temperature gradients. The multi-parametric tactile signals are analyzed using Mamba time series network model, resulting in the high-precision clustering of multiple features (up to 98.3 %). Supported by a multimodal interactive perception framework, the LLM realizes multi-feature understanding of objects. With large-scale sample datasets and multi-task training, the LLM-equipped robots powered by the multimodal tactile sensory system are expected to realize embodied intelligence.

STORM image denoising and information extraction.

Stochastic optical reconstruction microscopy (STORM) is extensively utilized in the fields of cell and molecular biology as a super-resolution imaging technique for visualizing cells and molecules. Nonetheless, the imaging process of STORM is frequently susceptible to noise, which can significantly impact the subsequent image analysis. Moreover, there is currently a lack of a comprehensive automated processing approach for analyzing protein aggregation states from a large number of STORM images. This paper initially applies our previously proposed denoising algorithm, UNet-Att, in STORM image denoising. This algorithm was constructed based on attention mechanism and multi-scale features, showcasing a remarkably efficient performance in denoising. Subsequently, we propose a collection of automated image processing algorithms for the ultimate feature extractions and data analyses of the STORM images. The information extraction workflow effectively integrates automated methods of image denoising, objective image segmentation and binarization, and object information extraction, and a novel image information clustering algorithm specifically developed for the morphological analysis of the objects in the STORM images. This automated workflow significantly improves the efficiency of the effective data analysis for large-scale original STORM images.

Decimeter-depth and polarization addressable color 3D meta-holography

Fueled by the rapid advancement of nanofabrication, metasurface has provided unprecedented opportunities for 3D holography. Large depth 3D meta-holography not only greatly increases information storage capacity, but also enables distinguishing of the relative spatial relationship of 3D objects, which has important applications in fields like optical information storage and medical diagnosis. Although the methods based on Fresnel diffraction theory can reconstruct the real depth information of 3D objects, the maximum depth is only 2 mm. Here, we develop a 3D meta-holography based on angular spectrum diffraction theory to break through the depth limit. By developing the angular spectrum diffraction theory into meta-holography, the metasurface structure with independent polarization control is used to create a polarization multiplexing 3D meta-hologram. The fabricated amorphous silicon metasurface increases the depth range by 47.5 times and realizes 0.95 dm depth reconstruction for polarization independent and different color 3D meta-hologram in visible. Such polarization controlled large-depth color meta-holography is expected to open avenue for data storage, display, information security and virtual reality.

Experience of sexual and reproductive health service utilization among Jimma University students: a qualitative study

BackgroundAmong the limits of the university spirit are potential sexual and reproductive health issues. Sexual and reproductive health services are essential for assisting students in overcoming such challenges. However, according to many facility reports and research, students prefer private services and do not use them. Despite the difficulties, a few studies have been conducted on the experiences of Ethiopian University students. Thus, this study explored the experiences of Jimma University students who used sexual and reproductive health services.MethodsA qualitative phenomenological approach was conducted from January 22nd to March 2nd, 2019, among Jimma university students. In-depth interviews were carried out to explore the lived experiences of sexual and reproductive health service use. Students were chosen by a purposive sampling method and recruited using gatekeepers until the needed sample size was obtained and the data was theoretically saturated. Key informants required to present decent information for study objectives were selected purposefully. The data were analyzed using the thematic analysis approach with the help of the ATLAS ti software package.ResultsEleven In-depth interviews and four key informant interviews were conducted. The study revealed seven overarching themes: sexual and reproductive health awareness and perception, problems, service delivery point preferences and practices, availability and accessibility, barriers to service utilization, handling mechanisms, and suggestions.ConclusionsOur study recognized positive and negative experiences of sexual health service use among Jimma University students. These findings suggested planning a comprehensive intervention that addresses the identified challenges and problems to improve students’ sexual and reproductive health services.

An Intelligent Proofreading for Remote Skiing Actions Based on Variable Shape Basis

AbstractThe current proofreading algorithms for action regulation mainly recover the 3D structure and action information of non-rigid objects from image sequences by factorization. Most of algorithms assume that the camera model is an affine model. This assumption only holds if the size and depth of the object change very little relative to the distance from the object to the camera, which is in the case of fixed-shape basis. When the object is very close to the camera, this assumption causes a large reconstruction error. This paper solves this problem by the intelligent proofreading algorithms for remote skiing teaching actions based on variable shape basis. Firstly, the improved Retinex algorithm is used to enhance the multi-frame video images of skiing actions to make the action details more prominent. Then, measurement matrix is calculated after eliminating the translation vector by coordinate transformation. Under the condition of rank constraint, the measurement matrix is decomposed by singular value decomposition algorithm, and the correct shape basis structure of 3D action features can be obtained by using the variable shape basis. Finally, by randomly initializing a parameter, the optimized parameter and the least square algorithm are used to optimize the randomly initialized parameter further. The iteration until the convergence of the objective function can be used to calculate the deformation degree of the actions. The test results show that this algorithm improves the proofreading accuracy of action regulation in skiing teaching, and the proofreading results of various uploaded sliding actions are correct, which can be applied to remote skiing teaching and community learning.

Encryption-decryption scheme in a single-pixel system based on polarization and Laguerre-Gaussian mode modulation

Optical cryptosystems are crucial for ensuring the security of optical information transmission and storage. The indirect measurement mechanism of single-pixel imaging (SPI) offers a feasible implementation channel for optical cryptosystems. Illumination patterns are encryption keys projected onto the plaintext object, while the intensity collected by the single-pixel detector forms the ciphertext. However, the variations in the object's angular position during SPI measurement generally introduce certain inaccuracies in image reconstruction. And due to SPI's input-output linear mapping relationship, the plaintext is vulnerable to exposure. This proposes an encryption-decryption scheme in a single-pixel system based on polarization and Laguerre-Gaussian (LG) mode modulation. The inherent circular symmetry of LG mode makes the angular position of the object information that can be encrypted, while the intrinsic properties of the object can be represented by polarization. Our system characterizes various polarization parameters of samples serving as reliable plaintext with an error of less than 4.2%, including depolarization, diattenuation, and retardance. For encryption demonstration, LG modes are randomly divided into 5 groups, corresponding to an object at different rotational states. This, combined with 16 polarization modulations, constructs pattern-angle-polarization joint keys, enabling high-security encryption as well as high-fidelity decryption of the mask image, optical axis orientation, and retardance of the test sample. Experimental results demonstrate the effectiveness of our scheme in enhancing the security and information complexity of optical cryptography, offering valuable insights for optical communication and quantum information security.

Structural tensor and frequency guided semi-supervised segmentation for medical images.

The method of semi-supervised semantic segmentation entails training with a limited number of labeled samples alongside many unlabeled samples, aiming to reduce dependence on pixel-level annotations. Most semi-supervised semantic segmentation methods primarily focus on sample augmentation in spatial dimensions to reduce the shortage of labeled samples. These methods tend to ignore the structural information of objects. In addition, frequency-domain information also supplies another perspective to evaluate information from images, which includes different properties compared to the spatialdomain. In this study, we attempt to answer these two questions: (1) is it helpful to provide structural information of objects in semi-supervised semantic segmentation tasks for medical images? (2) is it more effective to evaluate the segmentation performance in the frequency domain compared to the spatial domain for semi-supervised medical image segmentation? Therefore, we seek to introduce structural and frequency information to improve the performance of semi-supervised semantic segmentation for medicalimages. We present a novel structural tensor loss (STL) to guide feature learning on the spatial domain for semi-supervised semantic segmentation. Specifically, STL utilizes the structural information encoded in the tensors to enforce the consistency of objects across spatial regions, thereby promoting more robust and accurate feature extraction. Additionally, we proposed a frequency-domain alignment loss (FAL) to enable the neural networks to learn frequency-domain information across different augmented samples. It leverages the inherent patterns present in frequency-domain representations to guide the network in capturing and aligning features across diverse augmentation variations, thereby enhancing the model's robustness for the inputting variations. We conduct our experiments on three benchmark datasets, which include MRI (ACDC) for cardiac, CT (Synapse) for abdomen organs, and ultrasound image (BUSI) for breast lesion segmentation. The experimental results demonstrate that our method outperforms state-of-the-art semi-supervised approaches regarding the Dice similaritycoefficient. We find the proposed approach could improve the final performance of the semi-supervised medical image segmentation task. It will help reduce the need for medical image labels. Our code will are available at https://github.com/apple1986/STLFAL.

A-scan fluorescence microscopy for rapid cross-sectional imaging

This paper presents a microscopy technique that can perform snapshot depth resolved optical imaging in the same manner as A-scan in ultrasound imaging and optical coherence tomography. In this technique, a laser line along the axial dimension is used to illuminate a sample to create a fluorescent line object. By transforming the line object along the axial dimension (Z) to a ring image on the lateral dimensions (X-Y) using a full cone mirror, common optics can be used to relay and acquire the ring image precisely. Then, by converting half of the ring image back to a line image using a half cone mirror, the opening side of the half cone mirror allows the line image, which contains the full depth resolved information of the line object, to be taken in one snapshot. This eliminates the requirement of axial scanning in traditional depth resolved imaging techniques such as confocal microscopy to obtain the same information. The technique is demonstrated by imaging fluorescent microspheres of different diameters. This technique offers a simple alternative to traditional depth resolved imaging techniques such as confocal microscopy and light sheet microscopy. It is particularly useful in imaging samples with multiple layers in which multiple A-scans or a few B-scans are sufficient to represent the entire sample.

Impact of Display Sub-Pixel Arrays on Perceived Gloss and Transparency.

In recent years, improvements in display image quality have made it easier to perceive rich object information, such as gloss and transparency, from images, known as shitsukan. Do the different display specifications in the world affect their appearance? Clarifying the effects of differences in pixel structure on shitsukan perception is necessary to realize shitsukan management for displays with different hardware structures, which has not been fully clarified before. In this study, we experimentally investigated the effects of display pixel arrays on the perception of glossiness and transparency. In a visual evaluation experiment, we investigated the effects of three types of sub-pixel arrays (RGB, RGBW, and PenTile) on the perception of glossiness and transparency using natural images. The results confirmed that sub-pixel arrays affect the appearance of glossiness and transparency. A general relationship of RGB > PenTile > RGBW for glossiness and RGB > RGBW > PenTile for transparency was found; however, detailed analysis, such as cluster analysis, confirmed that the relative superiority of these sub-pixel arrays may vary depending on the observer and image content.

Improved Architecture and Training Strategies of YOLOv7 for Remote Sensing Image Object Detection

The technology for object detection in remote sensing images finds extensive applications in production and people’s lives, and improving the accuracy of image detection is a pressing need. With that goal, this paper proposes a range of improvements, rooted in the widely used YOLOv7 algorithm, after analyzing the requirements and difficulties in the detection of remote sensing images. Specifically, we strategically remove some standard convolution and pooling modules from the bottom of the network, adopting stride-free convolution to minimize the loss of information for small objects in the transmission. Simultaneously, we introduce a new, more efficient attention mechanism module for feature extraction, significantly enhancing the network’s semantic extraction capabilities. Furthermore, by adding multiple cross-layer connections in the network, we more effectively utilize the feature information of each layer in the backbone network, thereby enhancing the network’s overall feature extraction capability. During the training phase, we introduce an auxiliary network to intensify the training of the underlying network and adopt a new activation function and a more efficient loss function to ensure more effective gradient feedback, thereby elevating the network performance. In the experimental results, our improved network achieves impressive mAP scores of 91.2% and 80.8% on the DIOR and DOTA version 1.0 remote sensing datasets, respectively. These represent notable improvements of 4.5% and 7.0% over the original YOLOv7 network, significantly enhancing the efficiency of detecting small objects in particular.

RFSC-net: Re-parameterization forward semantic compensation network in low-light environments

Although detectors currently perform well in well-light conditions, their accuracy decreases due to insufficient object information. In addressing this issue, we propose the Re-parameterization Forward Semantic Compensation Network (RFSC-Net). We propose the Reparameterization Residual Efficient Layer Aggregation Networks (RSELAN) for feature extraction, which integrates the concepts of re-parameterization and the Efficient Layer Aggregation Networks (ELAN). While focusing on the fusion of feature maps of the same dimension, it also incorporates upward fusion of lower-level feature maps, enhancing the detailed texture information in higher-level features. Our proposed Forward Semantic Compensation Feature Fusion (FSCFF) network reduces interference from high-level to low-level semantic information, retaining finer details to improve detection accuracy in low-light conditions. Experiments on the low-light ExDark and DarkFace datasets show that RFSC-Net improves mAP by 2% on ExDark and 0.5% on DarkFace over the YOLOv8n baseline, without an increase in parameter counts. Additionally, AP50 is enhanced by 2.1% on ExDark and 1.1% on DarkFace, with a mere 3.7 ms detection latency on ExDark.

SiamMAF: A multipath and feature-enhanced thermal infrared tracker

Thermal infrared (TIR) images are visually blurred and low in information content. Some TIR trackers focus on enhancing the semantic information of TIR features, neglecting the equally important detailed information for TIR tracking. After target localization, detailed information can assist the tracker in generating accurate prediction boxes. In addition, simple element-wise addition is not a way to fully utilize and fuse multiple response maps. To address these issues, this study proposes a multipath and feature-enhanced Siamese tracker (SiamMAF) for TIR tracking. We design a feature-enhanced module (FEM) based on complementarity, which can highlight the key semantic information of the target and preserve the detailed information of objects. Furthermore, we introduce a response fusion module (RFM) that can adaptively fuse multiple response maps. Extensive experimental results on two challenging benchmarks show that SiamMAF outperforms many existing state-of-the-art TIR trackers and runs at a steady 31FPS.