Prior Methods Research Articles

MRGazer: decoding eye gaze points from functional magnetic resonance imaging in individual space

Objective. Eye-tracking research has proven valuable in understanding numerous cognitive functions. Recently, Freyet alprovided an exciting deep learning method for learning eye movements from functional magnetic resonance imaging (fMRI) data. It employed the multi-step co-registration of fMRI into the group template to obtain eyeball signal, and thus required additional templates and was time consuming. To resolve this issue, in this paper, we propose a framework named MRGazer for predicting eye gaze points from fMRI in individual space.Approach. The MRGazer consists of an eyeball extraction module and a residual network-based eye gaze prediction module. Compared to the previous method, the proposed framework skips the fMRI co-registration step, simplifies the processing protocol, and achieves end-to-end eye gaze regression.Main results. The proposed method achieved superior performance in eye fixation regression (Euclidean error, EE = 2.04°) than the co-registration-based method (EE = 2.89°), and delivered objective results within a shorter time (∼0.02 s volume-1) than prior method (∼0.3 s volume-1).Significance. The MRGazer is an efficient, simple, and accurate deep learning framework for predicting eye movement from fMRI data, and can be employed during fMRI scans in psychological and cognitive research. The code is available athttps://github.com/ustc-bmec/MRGazer.

Building Coarse to Fine Convex Hulls With Auxiliary Vertices for Palette-Based Image Recoloring.

Constructing a convex hull for the pixel colors of an image by viewing them as 3D points can extract a set of palette colors for the image, then image recoloring can be achieved by modifying the palette colors. For better recoloring effect, the convex hull should contain more pixels (inclusive) and be more compact. Otherwise, reconstruction error would occur or the extracted palette color would be less representative, yielding wrong recoloring results or less effective edit. We observe that convex hulls constructed by prior methods can contain all the image pixels, but are far from compact. Efforts have been made to optimize the vertices of convex hull to increase the compactness but are still not perfect. In this paper, we propose a novel coarse to fine convex hull construction scheme with auxiliary vertices. We start by constructing a coarse convex hull whose vertices are directly image pixels which is thus the most compact but cannot contain all pixels. We then make a remedy by adding auxiliary vertices into the coarse convex hull to obtain a fine convex hull. More auxiliary vertices are added, more image pixels will be contained into the fine convex hull. The auxiliary vertices are image pixels too so that the compactness can still be maintained. During editing, the auxiliary vertices are not allowed to be edited for edit convenience, but deformed as-rigid-as-possible with the adjusting of other vertices. Our convex hull is both inclusive and compact. Extensive experiments validate the effectiveness of the proposed method.

3D Gamut Morphing for Non-Rectangular Multi-Projector Displays.

In a spatially augmented reality system, multiple projectors are tiled on a complex shaped surface to create a seamless display on it. This has several applications in visualization, gaming, education and entertainment. The main challenges in creating seamless and undistorted imagery on such complex shaped surfaces are geometric registration and color correction. Prior methods that provide solutions for the spatial color variation in multi-projector displays assume rectangular overlap regions across the projectors that is possible only on flat surfaces with extremely constrained projector placement. In this article, we present a novel and fully automated method for removing color variations in a multi-projector display on arbitrary shaped smooth surfaces using a general color gamut morphing algorithm that can handle any arbitrarily shaped overlap between the projectors and assures imperceptible color variations across the display surface.

HDhuman: High-Quality Human Novel-View Rendering From Sparse Views.

In this paper, we aim to address the challenge of novel view rendering of human performers that wear clothes with complex texture patterns using a sparse set of camera views. Although some recent works have achieved remarkable rendering quality on humans with relatively uniform textures using sparse views, the rendering quality remains limited when dealing with complex texture patterns as they are unable to recover the high-frequency geometry details that are observed in the input views. To this end, we propose HDhuman, which uses a human reconstruction network with a pixel-aligned spatial transformer and a rendering network with geometry-guided pixel-wise feature integration to achieve high-quality human reconstruction and rendering. The designed pixel-aligned spatial transformer calculates the correlations between the input views and generates human reconstruction results with high-frequency details. Based on the surface reconstruction results, the geometry-guided pixel-wise visibility reasoning provides guidance for multi-view feature integration, enabling the rendering network to render high-quality images at 2k resolution on novel views. Unlike previous neural rendering works that always need to train or fine-tune an independent network for a different scene, our method is a general framework that is able to generalize to novel subjects. Experiments show that our approach outperforms all the prior generic or specific methods on both synthetic data and real-world data. Source code and test data will be made publicly available for research purposes at http://cic.tju.edu.cn/faculty/likun/projects/HDhuman/index.html.

An Explainable Feature Selection Framework for Web Phishing Detection with Machine Learning

In the evolving landscape of cyber threats, phishing attacks pose significant challenges, particularly through deceptive webpages designed to extract sensitive information under the guise of legitimacy. Conventional and machine learning (ML)-based detection systems struggle to detect phishing websites owing to their constantly changing tactics. Furthermore, newer phishing websites exhibit subtle and expertly concealed indicators that are not readily detectable. Hence, effective detection depends on identifying the most critical features. Traditional feature selection (FS) methods often struggle to enhance ML model performance and instead decrease it. To combat these issues, we propose an innovative method using eXplainable AI (XAI) to enhance FS in ML models and improve the identification of phishing websites. Specifically, we employ SHapley Additive exPlanations (SHAP) for global perspective and aggregated local interpretable model-agnostic explanations (LIME) to determine specific localized patterns. The proposed SHAP and LIME-aggregated feature selection (SLA-FS) framework pinpoints the most informative features, enabling more precise, swift, and adaptable phishing detection. Applying this approach to an up-to-date web phishing dataset, we evaluate the performance of three ML models before and after FS to assess their effectiveness. Our findings reveal that random forest (RF), with an accuracy of 97.41% and XGBoost (XGB) at 97.21% significantly benefit from the SLA-FS framework, while KNN lags. Our framework increases the accuracy of RF and XGB by 0.65% and 0.41%, respectively, outperforming traditional filter or wrapper methods and any prior methods evaluated on this dataset, showcasing its potential.

Continuous measurements of respiratory muscle blood flow and oxygen consumption using noninvasive frequency-domain near-infrared spectroscopy and diffuse correlation spectroscopy.

Prior studies of muscle blood flow and muscle-specific oxygen consumption have required invasive injection of dye and magnetic resonance imaging, respectively. Such measures have limited utility for continuous monitoring of the respiratory muscles. Frequency-domain near-infrared spectroscopy and diffuse correlation spectroscopy (FD-NIRS & DCS) can provide continuous surrogate measures of blood flow index (BFi) and metabolic rate of oxygen consumption (MRO2). This study aimed to validate sternocleidomastoid FD-NIRS & DCS outcomes against electromyography (EMG) and mouth pressure (Pm) during incremental inspiratory threshold loading (ITL). Six female and six male healthy adults (means ± SD; 30 ± 7 yr, maximum inspiratory pressure 118 ± 61 cmH2O) performed incremental ITL starting at low loads (8 ± 2 cmH2O) followed by 50-g increments every 2 min until task failure. FD-NIRS & DCS continuously measured sternocleidomastoid oxygenated and deoxygenated hemoglobin + myoglobin (oxy/deoxy[Hb + Mb]), tissue saturation of oxygen (StO2), BFi, and MRO2. Ventilatory parameters including inspiratory Pm were also evaluated. Pm increased during incremental ITL (P < 0.05), reaching -47[-74 to -34] cmH2O (median [IQR: 25%-75%]) at task failure. Ventilatory parameters were constant throughout ITL (all P > 0.05). Sternocleidomastoid BFi and MRO2 increased from the start of the ITL (both P < 0.05). Deoxy[Hb + Mb] increased close to task failure, concomitantly with a constant increase in MRO2, and decreased StO2. Sternocleidomastoid deoxy[Hb + Mb], BFi, StO2, and MRO2 obtained during ITL via FD-NIRS & DCS correlated with sternocleidomastoid EMG (all P < 0.05). In healthy adults, FD-NIRS & DCS can provide continuous surrogate measures of respiratory BFi and MRO2. Increasing sternocleidomastoid oxygen consumption near task failure was associated with increased oxygen extraction and reduced tissue saturation.NEW & NOTEWORTHY This study introduces a novel approach, frequency-domain near-infrared spectroscopy and diffuse correlation spectroscopy (FD-NIRS & DCS), for noninvasive continuous monitoring of respiratory muscle blood flow and metabolic rate of oxygen consumption. Unlike prior methods involving invasive dye injection and magnetic resonance imaging, FD-NIRS & DCS offers the advantage of continuous measurement without the need for invasive procedures. It holds promise for advancing muscle physiology understanding and opens avenues for real-time monitoring of respiratory muscles.

Temporally enhanced graph convolutional network for hand tracking from an egocentric camera

We propose a robust 3D hand tracking system in various hand action environments, including hand-object interaction, which utilizes a single color image and a previous pose prediction as input. We observe that existing methods deterministically exploit temporal information in motion space, failing to address realistic diverse hand motions. Also, prior methods paid less attention to efficiency as well as robust performance, i.e., the balance issues between time and accuracy. The Temporally Enhanced Graph Convolutional Network (TE-GCN) utilizes a 2-stage framework to encode temporal information adaptively. The system establishes balance by adopting an adaptive GCN, which effectively learns the spatial dependency between hand mesh vertices. Furthermore, the system leverages the previous prediction by estimating the relevance across image features through the attention mechanism. The proposed method achieves state-of-the-art balanced performance on challenging benchmarks and demonstrates robust results on various hand motions in real scenes. Moreover, the hand tracking system is integrated into a recent HMD with an off-loading framework, achieving a real-time framerate while maintaining high performance. Our study improves the usability of a high-performance hand-tracking method, which can be generalized to other algorithms and contributes to the usage of HMD in everyday life. Our code with the HMD project will be available at https://github.com/UVR-WJCHO/TEGCN_on_Hololens2.

Railway alignment optimization in regions with densely-distributed obstacles based on semantic topological maps

Railway alignment development in a study area with densely-distributed obstacles, in which regions favorable for alignments are isolated (termed an isolated island effect, i.e., IIE), is a computation-intensive and time-consuming task. To enhance search efficiency and solution quality, an environmental suitability analysis is conducted to identify alignment-favorable regions (AFRs), focusing the subsequent alignment search on these areas. Firstly, a density-based clustering algorithm (DBSCAN) and a specific criterion are customized to distinguish AFR distribution patterns: continuously-distributed AFRs, obstructed effects, and IIEs. Secondly, a study area characterized by IIEs is represented with a semantic topological map (STM), integrating between-island and within-island paths. Specifically, between-island paths are derived through a multi-directional scanning strategy, while within-island paths are optimized using a Floyd-Warshall algorithm. To this end, the intricate alignment optimization problem is simplified into a shortest path problem, tackled with conventional shortest path algorithms (of which Dijkstra’s algorithm is adopted in this work). Lastly, the proposed method is applied to a real case in a mountainous region with karst landforms. Numerical results indicate its superior performance in both construction costs and environmental suitability compared to human designers and a prior alignment optimization method.

Improved Sparse Representation of Image from Inferred Angles of Steerable Wavelet

Objectives: Presenting images with sparse coefficients has a wide variety of real-time applications in compressive sensing. However, sparse representations of images present challenges due hidden similarities in the higher order moments. Literature suggests that the applications that involve natural images present a high level of similarity. Steerable basis, due to their rotational invariant property, have shown potential in sparse representation of natural images. Hence, the objective of the proposed study is to identify steerable basis that maximize the sparse representation of natural images. Method: Prior studies have used the angle of steerable basis either from the random assignment or derived from Hough transform. In this study, we propose the selection of steerable basis angle derived from maximum a prior method. Exploiting a steerable basis for better sparse representation requires the knowledge of proper steerable angles. Hence, we propose using MAP learning approach to identify this angle. Findings: The proposed method resulted in optimal steerable angle without the need for calculation of Hough Transform. In addition, the method also resulted in almost 10 percent improvement in sparse representation as indicated by higher Kurtosis. Novelty: We compare the measure of sparsity to evaluate the effectiveness of the proposed method. The results indicate the optimal sparsity from the proposed method as indicated from the maximum values of kurtosis compared to the previous related methods. In addition, the proposed method relaxes the requirement of manipulating Hough transform for optimal steerable angle. Keywords: Sparsity, Steerable Basis, Wavelet Pyramid Structure, Image Compression, Hough Transform

Universal Facial Encoding of Codec Avatars from VR Headsets

Faithful real-time facial animation is essential for avatar-mediated telepresence in Virtual Reality (VR). To emulate authentic communication, avatar animation needs to be efficient and accurate: able to capture both extreme and subtle expressions within a few milliseconds to sustain the rhythm of natural conversations. The oblique and incomplete views of the face, variability in the donning of headsets, and illumination variation due to the environment are some of the unique challenges in generalization to unseen faces. In this paper, we present a method that can animate a photorealistic avatar in realtime from head-mounted cameras (HMCs) on a consumer VR headset. We present a self-supervised learning approach, based on a cross-view reconstruction objective, that enables generalization to unseen users. We present a lightweight expression calibration mechanism that increases accuracy with minimal additional cost to run-time efficiency. We present an improved parameterization for precise ground-truth generation that provides robustness to environmental variation. The resulting system produces accurate facial animation for unseen users wearing VR headsets in realtime. We compare our approach to prior face-encoding methods demonstrating significant improvements in both quantitative metrics and qualitative results.

CSI-Otter: isogeny-based (partially) blind signatures from the class group action with a twist

In this paper, we construct the first provably-secure isogeny-based (partially) blind signature scheme. While at a high level the scheme resembles the Schnorr blind signature, our work does not directly follow from that construction, since isogenies do not offer as rich an algebraic structure. Specifically, our protocol does not fit into the linear identification protocol abstraction introduced by Hauck, Kiltz, and Loss (EUROCYRPT’19), which was used to generically construct Schnorr-like blind signatures based on modules such as classical groups and lattices. Consequently, our scheme is provably secure in the random oracle model (ROM) against poly-logarithmically-many concurrent sessions assuming the subexponential hardness of the group action inverse problem. In more detail, our blind signature exploits the quadratic twist of an elliptic curve in an essential way to endow isogenies with a strictly richer structure than abstract group actions (but still more restrictive than modules). The basic scheme has public key size 128 B and signature size 8 KB under the CSIDH-512 parameter sets—these are the smallest among all provably secure post-quantum secure blind signatures. Relying on a new ring variant of the group action inverse problem (rGAIP\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\ extsf{rGAIP}$$\\end{document}), we can halve the signature size to 4 KB while increasing the public key size to 512 B. We provide preliminary cryptanalysis of rGAIP\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$${\ extsf{rGAIP}} $$\\end{document} and show that for certain parameter settings, it is essentially as secure as the standard GAIP\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\ extsf{GAIP}$$\\end{document}. Finally, we show a novel way to turn our blind signature into a partially blind signature, where we deviate from prior methods since they require hashing into the set of public keys while hiding the corresponding secret key—constructing such a hash function in the isogeny setting remains an open problem.

A Hybrid Integration Method Based on SMC-PHD-TBD for Multiple High-Speed and Highly Maneuverable Targets in Ubiquitous Radar

Based on the characteristic of ubiquitous radar emitting low-gain wide beam, a method of long-time coherent integration (LTCI) is required to enhance target detection capability. However, high-speed and highly maneuverable targets can cause Doppler frequency migration (DFM), range migration (RM), and velocity ambiguity (VA), severely degrading the performance of LTCI. Additionally, the number of targets is unknown and variable, and the presence of clutter further complicates the target tracking problem. To address these challenges, we propose a hybrid integration method to achieve joint detection and estimation of multiple high-speed, and highly maneuverable targets. Firstly, we compensate for first-order RM using the keystone transform (KT) and generate corresponding sub-range-Doppler (SRD) planes with different folding factors to achieve VA compensation. These SRD planes are then stitched together to form an extended range-Doppler (ERD) plane, which covers a broader velocity range. Secondly, during the track-before-detect (TBD) process, tracking is performed directly on the ERD plane. We use the sequential Monte Carlo (SMC) approximation of the probability hypothesis density (PHD) to propagate multi-target states. Additionally, we propose an amplitude-based adaptive prior distribution method and a line spread model (LSM) observation model to compensate for DFM. Since the acceleration of the target is included in the particle state, using particles to search for DFM does not increase the computational load. To address the issue of misclassifying mirror targets as real targets in the SRD plane, we propose a particle space projection method. By stacking the SRD planes to create a folding range-Doppler (FRD) space, particles are projected along the folding factor dimension, and then, the particles are clustered to eliminate the influence of the mirror targets. Finally, through simulation experiments, the superiority of the LSM for targets with acceleration was demonstrated. In comparative experiments, the proposed method showed superior performance and robustness compared to traditional methods, achieving a balance between performance and computational efficiency. Furthermore, the proposed method’s capability to detect and track multiple high-speed and highly maneuverable targets was validated using actual data from a ubiquitous radar system.

A weighted prior tensor train decomposition method for community detection in multi-layer networks

Community detection in multi-layer networks stands as a prominent subject within network analysis research. However, the majority of existing techniques for identifying communities encounter two primary constraints: they lack suitability for high-dimensional data within multi-layer networks and fail to fully leverage additional auxiliary information among communities to enhance detection accuracy. To address these limitations, a novel approach named weighted prior tensor training decomposition (WPTTD) is proposed for multi-layer network community detection. Specifically, the WPTTD method harnesses the tensor feature optimization techniques to effectively manage high-dimensional data in multi-layer networks. Additionally, it employs a weighted flattened network to construct prior information for each dimension of the multi-layer network, thereby continuously exploring inter-community connections. To preserve the cohesive structure of communities and to harness comprehensive information within the multi-layer network for more effective community detection, the common community manifold learning (CCML) is integrated into the WPTTD framework for enhancing the performance. Experimental evaluations conducted on both artificial and real-world networks have verified that this algorithm outperforms several mainstream multi-layer network community detection algorithms.

Noise-Tolerant Hybrid Prototypical Learning with Noisy Web Data

We focus on the challenging problem of learning an unbiased classifier from a large number of potentially relevant but noisily labeled web images given only a few clean labeled images. This problem is particularly practical, because it reduces the expensive annotation costs by utilizing freely accessible web images with noisy labels. Typically, prototypes are representative images or features used to classify or identify other images. However, in the few clean and many noisy scenarios, the class prototype can be severely biased due to the presence of irrelevant noisy images. The resulting prototypes are less compact and discriminative, as previous methods do not take into account the diverse range of images in the noisy web image collections. On the other hand, the relation modeling between noisy and clean images is not learned for the class prototype generation in an end-to-end manner, which results in a suboptimal class prototype. In this paper, we introduce a similarity maximization loss named the SimNoiPro. Our SimNoiPro first generates noise-tolerant hybrid prototypes composed of clean and noise-tolerant prototypes, and then pulls them closer to each other. Our approach considers the diversity of noisy images by explicit division and overcomes the optimization discrepancy issue. This enables better relation modeling between clean and noisy images and helps extract judicious information from the noisy image set. The evaluation results on two extended few-shot classification benchmarks confirm that our SimNoiPro outperforms prior methods in measuring image relations and cleaning noisy data.

Gold‐Catalyzed Diyne‐Ene Annulation for the Synthesis of Polysubstituted Benzenes through Formal [3+3] Approach with Amide as the Critical Co‐Catalyst

The one‐step synthesis of tetra‐substituted benzenes was accomplished via gold‐catalyzed diyne‐ene annulation. Distinguished from prior modification methods, this novel strategy undergoes formal [3+3] cyclization, producing polysubstituted benzenes with exceptional efficiency. The critical factor enabling this transformation was the introduction of amides, which were reported for the first time in gold catalysis as covalent nucleophilic co‐catalysts. This interesting protocol not only offers a new strategy to achieve functional benzenes with high efficiency, but also enlightens potential new reaction pathways within gold‐catalyzed alkyne activation processes.

Production of succinate with two CO2 fixation reactions from fatty acids in Cupriavidus necator H16

BackgroundBiotransformation of CO2 into high-value-added carbon-based products is a promising process for reducing greenhouse gas emissions. To realize the green transformation of CO2, we use fatty acids as carbon source to drive CO2 fixation to produce succinate through a portion of the 3-hydroxypropionate (3HP) cycle in Cupriavidus necator H16.ResultsThis work can achieve the production of a single succinate molecule from one acetyl-CoA molecule and two CO2 molecules. It was verified using an isotope labeling experiment utilizing NaH13CO3. This implies that 50% of the carbon atoms present in succinate are derived from CO2, resulting in a twofold increase in efficiency compared to prior methods of succinate biosynthesis that relied on the carboxylation of phosphoenolpyruvate or pyruvate. Meanwhile, using fatty acid as a carbon source has a higher theoretical yield than other feedstocks and also avoids carbon loss during acetyl-CoA and succinate production. To further optimize succinate production, different approaches including the optimization of ATP and NADPH supply, optimization of metabolic burden, and optimization of carbon sources were used. The resulting strain was capable of producing succinate to a level of 3.6 g/L, an increase of 159% from the starting strain.ConclusionsThis investigation established a new method for the production of succinate by the implementation of two CO2 fixation reactions and demonstrated the feasibility of ATP, NADPH, and metabolic burden regulation strategies in biological carbon fixation.

Dark channel enhancement research on human ear images based on smartphone photography

SummaryThe experienced doctors can alleviate symptoms such as headaches, insomnia, anxiety, and depression by observing the patient's ears and massaging specific areas. In order to achieve remote ear condition diagnosis and guide patients to massage their ears independently through the network, patients can use their mobile phones to take and send photos of ears to doctors. However, due to significant differences in the clarity of photos taken by different mobile phones, as well as susceptibility to haze, lighting, jitter, and low pixels, the quality of photos is poor, which affects the accuracy of remote diagnosis by doctors. This study adopted an image preprocessing method based on He Kaiming's dark channel prior dehazing method to enhance the original ear images captured by mobile phones. The dehazing algorithm was used to remove the haze effect of the ear images, improving image quality and contrast, making the wrinkles, protrusions, pigmentation and other areas of the ear more obvious. The experiment has showed the comparison by adjusting weight from 15% to 95% between two methods—dark channel prior method and the dark channel prior method after preprocessing, which has proven the effectiveness of dehazing method in human ear images taken by mobile phones. The image quality after preprocessing and dehazing is widely recognized and accepted by doctors at hospitals in Hangzhou, China.

Multi-Task Learning of Object States and State-Modifying Actions From Web Videos.

We aim to learn to temporally localize object state changes and the corresponding state-modifying actions by observing people interacting with objects in long uncurated web videos. We introduce three principal contributions. First, we develop a self-supervised model for jointly learning state-modifying actions together with the corresponding object states from an uncurated set of videos from the Internet. The model is self-supervised by the causal ordering signal, i.e., initial object state → manipulating action → end state. Second, we explore alternative multi-task network architectures and identify a model that enables efficient joint learning of multiple object states and actions, such as pouring water and pouring coffee, together. Third, we collect a new dataset, named ChangeIt, with more than 2600 hours of video and 34 thousand changes of object states. We report results on an existing instructional video dataset COIN as well as our new large-scale ChangeIt dataset containing tens of thousands of long uncurated web videos depicting various interactions such as hole drilling, cream whisking, or paper plane folding. We show that our multi-task model achieves a relative improvement of 40% over the prior methods and significantly outperforms both image-based and video-based zero-shot models for this problem.

CTNeRF: Cross-time Transformer for dynamic neural radiance field from monocular video

The goal of our work is to generate high-quality novel views from monocular videos of complex and dynamic scenes. Prior methods, such as DynamicNeRF, have shown impressive performance by leveraging time-varying dynamic radiation fields. However, these methods have limitations when it comes to accurately modeling the motion of complex objects, which can lead to inaccurate and blurry renderings of details. To address this limitation, we propose a novel approach that builds upon a recent generalization NeRF, which aggregates nearby views onto new viewpoints. However, such methods are typically only effective for static scenes. To overcome this challenge, we introduce a module that operates in both the time and frequency domains to aggregate the features of object motion. This allows us to learn the relationship between frames and generate higher-quality images. Our experiments demonstrate significant improvements over state-of-the-art methods on dynamic scene datasets. Specifically, our approach outperforms existing methods in terms of both the accuracy and visual quality of the synthesized views. Our code is available on https://github.com/xingy038/CTNeRF.

A few-shot word-structure embedded model for bridge inspection reports learning

Intelligent bridge maintenance requires the comprehensive utilization of inspection records, as they contain valuable insights into structures’ long-term service conditions. To efficiently focus and utilize this data from extensive reports, reliable extraction methods are highly sought after. However, the heavy reliance on large amounts of manually annotated data limits the applicability and practicability of existing information extraction methods from bridge inspection reports, especially given the non-uniformity in report formats and expressions. To address this issue, this study proposed a few-shot information extraction model for bridge inspection reports understanding, comprising Word and Structure Integration Embeddings, Bi-directional Long Short-Term Memory (BiLSTM), and Condition Random Field (CRF). The model’s strength lies in its easy-to-implement word-structure embedding approach, which combines domain-specific word representations and sentence structure information. Specifically, the bridge inspection-domain pre-trained language model was further pre-trained and fine-tuned to obtain word embeddings, containing prior knowledge of the domain and tasks. Moreover, a novel encoding method was designed to generate sentence structure embeddings from dependency syntactic analysis results, providing textual representation information. Finally, the integrated word-structure embeddings, created by aligning dimensions for concatenation, were fed into the BiLSTM-CRF architecture to capture contextual dependencies and constrain extraction results. Empirical evaluations conducted on four few-shot datasets with 10, 30, 50, and 100 samples demonstrate that the proposed model achieved high accuracy and F1 score, outperforming prior methods, general domain models, and large language models. Specifically, in a dataset containing 50 sentences, our model achieved an accuracy of up to 0.9357 and an F1 score of 0.8683, representing an average increase of 38.4% higher than these methods. Ablation experiments revealed the contributions of each model component. These results suggest that the proposed model can accurately extract key information from bridge inspection reports even with limited training data scenarios, thereby facilitating applications such as structural condition evaluation and maintenance decision-making.