Backbone Design Research Articles

Arbitrary-Scale Video Super-resolution Guided by Dynamic Context

We propose a Dynamic Context-Guided Upsampling (DCGU) module for video super-resolution (VSR) that leverages temporal context guidance to achieve efficient and effective arbitrary-scale VSR. While most VSR research focuses on backbone design, the importance of the upsampling part is often overlooked. Existing methods rely on pixelshuffle-based upsampling, which has limited capabilities in handling arbitrary upsampling scales. Recent attempts to replace pixelshuffle-based modules with implicit neural function-based and filter-based approaches suffer from slow inference speeds and limited representation capacity, respectively. To overcome these limitations, our DCGU module predicts non-local sampling locations and content-dependent filter weights, enabling efficient and effective arbitrary-scale VSR. Our proposed multi-granularity location search module efficiently identifies non-local sampling locations across the entire low-resolution grid, and the temporal bilateral filter modulation module integrates content information with the filter weight to enhance textual details. Extensive experiments demonstrate the superiority of our method in terms of performance and speed on arbitrary-scale VSR.

Role of Side-Chain Free Volume on the Electrochemical Behavior of Poly(propylenedioxythiophenes).

Mixed ionic/electronic conducting polymers are versatile systems for, e.g., energy storage, heat management (exploiting electrochromism), and biosensing, all of which require electrochemical doping, i.e., the electrochemical oxidation or reduction of their macromolecular backbones. Electrochemical doping is achieved via electro-injection of charges (i.e., electronic carriers), stabilized via migration of counterions from a supporting electrolyte. Since the choice of the polymer side-chain functionalization influences electrolyte and/or ion sorption and desorption, it in turn affects redox properties, and, thus, electrochemically induced mixed conduction. However, our understanding of how side-chain versus backbone design can increase ion flow while retaining high electronic transport remains limited. Hence, heuristic design approaches have typically been followed. Herein, we consider the redox and swelling behavior of three poly(propylenedioxythiophene) derivatives, P(ProDOT)s, substituted with different side-chain motifs, and demonstrate that passive swelling is controlled by the surface polarity of P(ProDOT) films. In contrast, active swelling under operando conditions (i.e., under an applied bias) is dictated by the local side-chain free volume on the length scale of a monomer unit. Such insights deliver important design criteria toward durable soft electrochemical systems for diverse energy and biosensing platforms and new understanding into electrochemical conditioning ("break-in") in many conducting polymers.

CMNet: a novel model and design rationale based on comparison studies and synergy of CNN and MetaFormer

Convolutional- and Transformer-based backbone architecture are two dominant, widely accepted, models in computer vision. Nevertheless, it is still a challenge, thus a focus of research, to decide which backbone architecture performs better, and under which circumstances. In this paper, we conduct an in-depth investigation into the differences of the macroscopic backbone design of the CNN and Transformer models with the ultimate purpose of developing new models to combine the strengths of both types of architectures for effective image classification. Specifically, we first analyze the model structures of both models and identified four main differences, then we design four sets of ablation experiments using the ImageNet-1K dataset with an image classification problem as an example to study the impacts of these four differences on model performance. Based on the experimental results, we derive four observations as rules of thumb for designing a vision model backbone architecture. Informed by the experiment findings, we then conceive a novel model called CMNet which marries the experiment-proved best design practices of CNN and Transformer architectures. Finally, we carry out extensive experiments on CMNet using the same dataset against baseline classifiers. Initial results prove CMNet achieves the highest top-1 accuracy of 80.08% on the ImageNet-1K validation set, this is a very competitive value compared to previous classical models with similar computational complexity. Details of the implementation, algorithms and codes, are publicly available on Github: https://github.com/Arwin-Yu/CMNet.

Preparation and Properties of High-Temperature-Resistant, Lightweight, Flexible Polyimide Foams with Different Diamine Structures.

Polyimide foam (PIF) is a rising star in high-end applications such as aerospace thermal insulation and military sound absorption. However, the basic rule on molecular backbone design and uniform pore formation of PIF still need to be explored. In this work, polyester ammonium salt (PEAS) precursor powders are synthesized between alcoholysis ester of 3, 3', 4, 4'-benzophenone tetracarboxylic dianhydride (BTDE) and aromatic diamines with different chain flexibility and conformation symmetry. Then, a standard "stepwise heating" thermo-foaming approach is used to prepare PIF with comprehensive properties. A rational thermo-foaming program is designed based on in situ observation of pore formation during heating. The fabricated PIFs have uniform pore structure, and PIFBTDA-PDA shows the smallest size (147 μm) and narrow distribution. Interestingly, PIFBTDA-PDA also presents a balanced strain recovery rate (SR = 91%) and mechanical robustness (0.051 MPa at 25% strain) and its pore structure maintains regularity after 10 compression-recovery cycles, mainly due to high rigidity of the chains. Furthermore, all the PIFs possess lightweight feature (15-20 kg∙m-3), good heat resistance (Tg at 270-340 °C), thermal stability (T5% at 480-530 °C), thermal insulation properties (λ = 0.046-0.053 W∙m-1K-1 at 20 °C, λ = 0.078-0.089 W∙m-1K-1 at 200 °C), and excellent flame retardancy (LOI > 40%). The reported monomer-mediated pore-structure control strategy can provide guidelines for the preparation of high-performance PIF and its industrial applications.

Physiological signal analysis in exercise fatigue detection application based on deep learning

AbstractThis paper proposes a physiological signal analysis method in exercise fatigue detection application based on deep learning models to provide fast and accurate feedback for the player's physical status and better assist the player to perform exercise. We adopt the deep neural network as backbone model and design following strategies in our proposed method to process and extract features in signals. First, we preprocess the physiological signal, including noise reduction and segmentation. Second, we use a deep learning model to design a feature extraction method, which uses an autoencoder to label and feature the signal. Third, we perform motion fatigue detection on the fused signal features based on a long short‐term memory network model. The results prove that the method proposed has good performance.

BasicTAD: An astounding RGB-Only baseline for temporal action detection

Temporal action detection (TAD) is extensively studied in the video understanding community by generally following the object detection pipeline in images. However, complex designs are not uncommon in TAD, such as two-stream feature extraction, multi-stage training, complex temporal modeling, and global context fusion. In this paper, we do not aim to introduce any novel technique for TAD. Instead, we study a simple, straightforward, yet must-known baseline given the current status of complex design and low detection efficiency in TAD. In our simple baseline (BasicTAD), we decompose the TAD pipeline into several essential components: data sampling, backbone design, neck construction, and detection head. We extensively investigate the existing techniques in each component for this baseline and, more importantly, perform end-to-end training over the entire pipeline thanks to the simplicity of design. As a result, this simple BasicTAD yields an astounding and real-time RGB-Only baseline very close to the state-of-the-art methods with two-stream inputs. In addition, we further improve the BasicTAD by preserving more temporal and spatial information in network representation (termed as PlusTAD). Empirical results demonstrate that our PlusTAD is very efficient and significantly outperforms the previous methods on the datasets of THUMOS14 and FineAction. Meanwhile, we also perform in-depth visualization and error analysis on our proposed method and try to provide more insights into the TAD problem. Our approach can serve as a strong baseline for future TAD research. The code and model are released at https://github.com/MCG-NJU/BasicTAD.

A Reconfigurable Data Glove for Reconstructing Physical and Virtual Grasps

In this work, we present a reconfigurable data glove design to capture different modes of human hand–object interactions, which are critical in training embodied artificial intelligence (AI) agents for fine manipulation tasks. To achieve various downstream tasks with distinct features, our reconfigurable data glove operates in three modes sharing a unified backbone design that reconstructs hand gestures in real time. In the tactile-sensing mode, the glove system aggregates manipulation force via customized force sensors made from a soft and thin piezoresistive material; this design minimizes interference during complex hand movements. The virtual reality (VR) mode enables real-time interaction in a physically plausible fashion: A caging-based approach is devised to determine stable grasps by detecting collision events. Leveraging a state-of-the-art finite element method, the simulation mode collects data on fine-grained four-dimensional manipulation events comprising hand and object motions in three-dimensional space and how the object’s physical properties (e.g., stress and energy) change in accordance with manipulation over time. Notably, the glove system presented here is the first to use high-fidelity simulation to investigate the unobservable physical and causal factors behind manipulation actions. In a series of experiments, we characterize our data glove in terms of individual sensors and the overall system. More specifically, we evaluate the system’s three modes by ① recording hand gestures and associated forces, ② improving manipulation fluency in VR, and ③ producing realistic simulation effects of various tool uses, respectively. Based on these three modes, our reconfigurable data glove collects and reconstructs fine-grained human grasp data in both physical and virtual environments, thereby opening up new avenues for the learning of manipulation skills for embodied AI agents.

CDF‐net: A convolutional neural network fusing frequency domain and spatial domain features

Convolutional neural network (CNN), as a classic deep learning algorithm, has been applied to various computer vision tasks. However, most classic CNN models focus on the extraction and utilisation of spatial domain features, while ignoring the potential ability of frequency domain feature extraction. In this study, the mechanism in the backbone design is explored. Firstly, the traditional DCT formula is converted into a convolution form through mathematical derivation. On the basis of a a new type of convolution, namely the DCT Convolution is designed. It is more applicable to deep learning network architectures. Secondly, based on the DCT Convolution, a new cross-domain fusion network named CDF-Net is designed. The frequency domain and spatial domain features of the input sample are extracted and fused by the network. CDF-Net is a general network framework which can be applied to most existing prevalent networks. Finally, various experiments are conducted. On image classification task, for Imagenet2012 dataset, the method proposed was applied to ResNet50, and the accuracy of Top1 was increased by 3.684%. On object detection task, for COCO2017 dataset, the method proposed in this study was applied to ResNet50 and ResNeXt50, mAP were improved by 0.5% and 1.2% respectively.

Poly (aryl piperidinium) anion exchange membranes for acid recovery: The effect of backbone structure

The anion exchange membrane (AEM)-based diffusion dialysis is considered as an efficient strategy for the recovery of acid. Recently, the novel ether-free poly (aryl piperidinium) (PAP) AEMs modified by side chain grafting has been applied in the area of acid separation, showing excellent acid flux. However, there is little study focusing on the influence of polymer backbone structures of AEMs in acid separation. Herein, a series of PAP AEMs with different backbone arrangements based on diphenyl, m-terphenyl, and p-terphenyl were prepared for acid separation via diffusion dialysis. Compared with diphenyl and p-terphenyl, the m-terphenyl-based AEM exhibits a fantastic comprehensive performance, with a dialysis coefficient of 20.1 × 10-3 m h−1 and H+/Fe2+ selectivity of 196. The acid flux and selectivity can be kept after 7 days of acidic stability test in 1 mol/L HCl and 0.25 mol/L FeCl2. This can be attributed to its unique spatial configuration that induces a microphase-separated structure, which is beneficial to H+ conduction. Meanwhile, its high rigidity and low solubility could restrain the water uptake and swelling, and thus reduce the Fe2+ flux and enhance the selectivity. Furthermore, the stability of all PAP AEMs was tested in 2 mol/L HCl at 60 °C for 60 days, and their ionic conductivities still remained above 90 %. This study highlights a simple and reliable strategy to enhance the acid separation performance of AEMs via the facile backbone design.

Role of aggregates and microstructure of mixed-ionic-electronic-conductors on charge transport in electrochemical transistors.

Synthetic efforts have delivered a library of organic mixed ionic-electronic conductors (OMIECs) with high performance in electrochemical transistors. The most promising materials are redox-active conjugated polymers with hydrophilic side chains that reach high transconductances in aqueous electrolytes due to volumetric electrochemical charging. Current approaches to improve transconductance and device stability focus mostly on materials chemistry including backbone and side chain design. However, other parameters such as the initial microstructure and microstructural rearrangements during electrochemical charging are equally important and are influenced by backbone and side chain chemistry. In this study, we employ a polymer system to investigate the fundamental electrochemical charging mechanisms of OMIECs. We couple in situ electronic charge transport measurements and spectroelectrochemistry with ex situ X-ray scattering electrochemical charging experiments and find that polymer chains planarize during electrochemical charging. Our work shows that the most effective conductivity modulation is related to electrochemical accessibility of well-ordered, interconnected aggregates that host high mobility electronic charge carriers. Electrochemical stress cycling induces microstructural changes, but we find that these aggregates can largely maintain order, providing insights on the structural stability and reversibility of electrochemical charging in these systems. This work shows the importance of material design for creating OMIECs that undergo structural rearrangements to accommodate ions and electronic charge carriers during which percolating networks are formed for efficient electronic charge transport.

An Ultralightweight Object Detection Network for Empty-Dish Recycling Robots

The emergence of empty-dish recycling robots has alleviated problems, such as labor shortages, caused by an aging population. The detection and grasping of dishes play a crucial role in empty-dish recycling robots. However, due to the limited resources of edge devices, traditional object detection models require more space to store parameters and much computational overhead, limiting the development of empty-dish recycling robots. Therefore, this article proposes an ultralightweight dish detection model YOLO-GS for an empty-dish recycling robot. We use the modified CSPDarknet as the backbone structure and design an ultralightweight neck structure for efficient feature fusion. Meanwhile, we design a lightweight head structure for object classification and bounding box coordinate regression by combining ghost shuffle convolution (GSConv2D) and the anchor-free method. For the empty-dish recycling robot to grasp the dishes, we design a dish grasp point extraction algorithm using image processing. Finally, TensorRT is used to optimize and accelerate the model for efficient and intelligent detection of dishes on the NVIDIA Jetson Xavier NX. The experimental results show that YOLO-GS achieves 99.380% mean average precision (mAP) with a parameter amount of 0.606 M. The inference speed of the TensorRT-optimized YOLO-GS algorithm reaches 31.371 FPS, which meets the needs of real-time dish detection by the empty-dish recycling robot. The image of the empty-dish recycling robot demo is available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://www.youtube.com/watch?v=pCBo1nzm3qU&amp;t=22s</uri> .

Suppressing Water Uptake and Increasing Hydroxide Conductivity in Ring-Opened Polynorbornene Ion-Exchange Materials via Backbone Design

In this study, we aimed to decrease segmental mobility in ring-opened polynorbornene anion-exchange membranes (AEMs) as a strategy to suppress swelling. This was accomplished by copolymerization of a norbornene-anthracene cycloadduct with cationic tetraaminophosphonium norbornene monomers to afford AEMs with glass transition temperatures above 100 °C. The anthracenyl side groups appended to the poly(vinylene-1,3-cyclopentylene) chain resulted in lower water uptake for the phosphonium-functionalized AEM when compared to an analogue without the arene moiety. Though chemical aging of these AEMs was noted over time, the benefits of rigidifying the polymer backbone were clear. Additionally, we have noted that the choice of dialkylamino groups around the phosphorus center plays a key role in alkaline stability of these phosphonium-based AEMs.

Enhancing the Backbone Coplanarity of n-Type Copolymers for Higher Electron Mobility and Stability in Organic Electrochemical Transistors.

Electron-transporting (n-type) conjugated polymers have recently been applied in numerous electrochemical applications, where both ion and electron transport are required. Despite continuous efforts to improve their performance and stability, n-type conjugated polymers with mixed conduction still lag behind their hole-transporting (p-type) counterparts, limiting the functions of electrochemical devices. In this work, we investigate the effect of enhanced backbone coplanarity on the electrochemical activity and mixed ionic-electronic conduction properties of n-type polymers during operation in aqueous media. Through substitution of the widely employed electron-deficient naphthalene diimide (NDI) unit for the core-extended naphthodithiophene diimide (NDTI) units, the resulting polymer shows a more planar backbone with closer packing, leading to an increase in the electron mobility in organic electrochemical transistors (OECTs) by more than two orders of magnitude. The NDTI-based polymer shows a deep-lying lowest unoccupied molecular orbital level, enabling operation of the OECT closer to 0 V vs Ag/AgCl, where fewer parasitic reactions with molecular oxygen occur. Enhancing the backbone coplanarity also leads to a lower affinity toward water uptake during cycling, resulting in improved stability during continuous electrochemical charging and ON–OFF switching relative to the NDI derivative. Furthermore, the NDTI-based polymer also demonstrates near-perfect shelf-life stability over a month-long test, exhibiting a negligible decrease in both the maximum on-current and transconductance. Our results highlight the importance of polymer backbone design for developing stable, high-performing n-type materials with mixed ionic-electronic conduction in aqueous media.

Ig-VAE: Generative modeling of protein structure by direct 3D coordinate generation.

While deep learning models have seen increasing applications in protein science, few have been implemented for protein backbone generation—an important task in structure-based problems such as active site and interface design. We present a new approach to building class-specific backbones, using a variational auto-encoder to directly generate the 3D coordinates of immunoglobulins. Our model is torsion- and distance-aware, learns a high-resolution embedding of the dataset, and generates novel, high-quality structures compatible with existing design tools. We show that the Ig-VAE can be used with Rosetta to create a computational model of a SARS-CoV2-RBD binder via latent space sampling. We further demonstrate that the model’s generative prior is a powerful tool for guiding computational protein design, motivating a new paradigm under which backbone design is solved as constrained optimization problem in the latent space of a generative model.

Suppressing the Excitonic Effect in Covalent Organic Frameworks for Metal-Free Hydrogen Generation.

Photocatalytic hydrogen generation is a promising solution for renewable energy production and plays a role in achieving carbon neutrality. Covalent organic frameworks (COFs) with highly designable backbones and inherent pores have emerged as novel photocatalysts, yet the strong excitonic effect in COFs can impede the promotion of energy conversion efficiency. Here, we propose a facile approach to suppress the excitonic effect in COFs, which is by narrowing the band gap and increasing the dielectric screening via a rational backbone design and chemical modifications. Based on the GW-BSE method, we uncover a linear relationship between the electronic dielectric constant and the inverse square of the optical band gap of COFs of the Lieb lattice. We further demonstrate that both reduced exciton binding energy and enhanced sunlight absorption can be simultaneously realized in COFs with a narrow band gap. Specifically, we show that one of our designed COFs whose exciton binding energy is nearly half that of g-C3N4 is capable of metal-free hydrogen production under near-infrared light irradiation. Our results showcase an effective method to suppress the excitonic effect in COFs and also pave the way for their applications in photocatalytic, photovoltaic, and other related solar energy conversions.

Mimetic Neural Networks: A Unified Framework for Protein Design and Folding.

Recent advancements in machine learning techniques for protein structure prediction motivate better results in its inverse problem–protein design. In this work we introduce a new graph mimetic neural network, MimNet, and show that it is possible to build a reversible architecture that solves the structure and design problems in tandem, allowing to improve protein backbone design when the structure is better estimated. We use the ProteinNet data set and show that the state of the art results in protein design can be met and even improved, given recent architectures for protein folding.

Two-stage segmentation network with feature aggregation and multi-level attention mechanism for multi-modality heart images.

Accurate segmentation of cardiac substructures in multi-modality heart images is an important prerequisite for the diagnosis and treatment of cardiovascular diseases. However, the segmentation of cardiac images remains a challenging task due to (1) the interference of multiple targets, (2) the imbalance of sample size. Therefore, in this paper, we propose a novel two-stage segmentation network with feature aggregation and multi-level attention mechanism (TSFM-Net) to comprehensively solve these challenges. Firstly, in order to improve the effectiveness of multi-target features, we adopt the encoder-decoder structure as the backbone segmentation framework and design a feature aggregation module (FAM) to realize the multi-level feature representation (Stage1). Secondly, because the segmentation results obtained from Stage1 are limited to the decoding of single scale feature maps, we design a multi-level attention mechanism (MLAM) to assign more attention to the multiple targets, so as to get multi-level attention maps. We fuse these attention maps and concatenate the output of Stage1 to carry out the second segmentation to get the final segmentation result (Stage2). The proposed method has better segmentation performance and balance on 2017 MM-WHS multi-modality whole heart images than the state-of-the-art methods, which demonstrates the feasibility of TSFM-Net for accurate segmentation of heart images.

An optimized retroviral toolbox for overexpression and genetic perturbation of primary lymphocytes.

ABSTRACTGenetic manipulation of primary lymphocytes is crucial for both clinical purposes and fundamental research. Despite their broad use, we encountered a paucity of data on systematic comparison and optimization of retroviral vectors, the workhorses of genetic modification of primary lymphocytes. Here, we report the construction and validation of a versatile range of retroviral expression vectors. These vectors can be used for the knockdown or overexpression of genes of interest in primary human and murine lymphocytes, in combination with a wide choice of selection and reporter strategies. By streamlining the vector backbone and insert design, these publicly available vectors allow easy interchangeability of the independent building blocks, such as different promoters, fluorescent proteins, surface markers and antibiotic resistance cassettes. We validated these vectors and tested the optimal promoters for in vitro and in vivo overexpression and knockdown of the murine T cell antigen receptor. By publicly sharing these vectors and the data on their optimization, we aim to facilitate genetic modification of primary lymphocytes for researchers entering this field.

Feature Reuse and Fusion for Real-Time Semantic Segmentation

For real-time semantic segmentation, how to increase the speed while maintaining high resolution is a problem that has been discussed and solved. Backbone design and fusion design have always been two essential parts of real-time semantic segmentation. We hope to design a light-weight network based on previous design experience and reach the level of state-of-the-art real-time semantic segmentation without any pre-training. To achieve this goal, a encoder-decoder architectures are proposed to solve this problem by applying a decoder network onto a backbone model designed for real-time segmentation tasks and designed three different ways to fuse semantics and detailed information in the aggregation phase. We have conducted extensive experiments on two semantic segmentation benchmarks. Experiments on the Cityscapes and CamVid datasets show that the proposed FRFNet strikes a balance between speed calculation and accuracy. It achieves 72% Mean Intersection over Union (mIoU%) on the Cityscapes test dataset with the speed of 144 on a single RTX 1080Ti card. The Code is available at https://github.com/favoMJ/FRFNet.

Mobile Control Plane Design for Quantum Satellite Backbones

The interconnection of quantum computers through the so-called Quantum Internet is a very promising approach. The most critical issues concern the physical layer, considering that the creation of entanglement over long distances is still problematic. Given the difficulty that usually arises from fiber optics due to exponential losses, the introduction of intermediate quantum repeaters (QRs) allows mitigating the problem. A quantum satellite network based on QRs on satellites deployed over low Earth orbit could make it possible to overcome the constraints of terrestrial optical networks. Hence, the recent technological developments in terms of quantum satellite communications motivated our investigation on an ad hoc quantum satellite backbone design based on the software defined networking paradigm with a control plane directly integrated into the constellation itself. Our aim is to outline some guidelines by comparing several options. Specifically, the focus is to analyze different architectural solutions making some considerations on their feasibility, possible benefits, and costs. Finally, we performed some simulations on the architectures we considered the most promising, concluding that the integration of the control plane in the constellation itself is the most appropriate solution.