Semantic Communication Research Articles

Wireless Image Transmission With Semantic and Security Awareness

Semantic communication is an increasingly popular framework for wireless image transmission due to its high communication efficiency. With the aid of the joint-source-and-channel (JSC) encoder implemented by neural network, semantic communication directly maps original images into symbol sequences containing semantic information. Compared with the traditional separate source and channel coding design used in bit-level communication systems, semantic communication systems are known to be more efficient and accurate especially in the low signal-to-the-noise ratio (SNR) regime. This thus prompts a critical while yet to be tackled issue of security in semantic communication: it makes the eavesdropper much easier to crack the semantic information as it can be retrieved even in a highly noisy channel. In this letter, we develop a semantic communication framework that accounts for both semantic meaning decoding efficiency and its risk of privacy leakage. To this end, targeting wireless image transmission, we propose an JSC autoencoder featuring residual structure for efficient semantic meaning extraction and transmission, and the training of which is guided by a well-designed loss function that can flexibly regulate the efficiency-privacy trade-off. Extensive experimental results are provided to show the effectiveness and robustness of the proposed scheme.

Predictive and Adaptive Deep Coding for Wireless Image Transmission in Semantic Communication

Semantic communication is a newly emerged communication paradigm that exploits deep learning (DL) models to realize communication processes like source coding and channel coding. Recent advances have demonstrated that DL-based joint source-channel coding (DeepJSCC) can achieve exciting data compression and noise-resiliency performances for wireless image transmission tasks, especially in environments with low channel signal-to-noises (SNRs). However, existing DeepJSCC-based semantic communication frameworks still cannot achieve adaptive code rates for different channel SNRs and image contents, which reduces its flexibility and bandwidth efficiency. In this paper, we propose a predictive and adaptive deep coding (PADC) framework for realizing flexible code rate optimization with a given target transmission quality requirement. PADC is realized by a variable code length enabled DeepJSCC (DeepJSCC-V) model for realizing flexible code length adjustment, an Oracle Network (OraNet) model for predicting peak-signal-to-noise (PSNR) value for an image transmission task according to its contents, channel signal to noise ratio (SNR) and the compression ratio (CR) value, and a CR optimizer aims at finding the minimal data-level or instance-level CR with a PSNR quality constraint. By using the above three modules, PADC can transmit the image data with minimal CR, which greatly increases bandwidth efficiency. Simulation results demonstrate that the proposed DeepJSCC-V model can achieve similar PSNR performances compared with the state-of-the-art Attention-based DeepJSCC (ADJSCC) model, and the proposed OraNet model is able to predict high-quality PSNR values with an average error lower than 0.5dB. Results also demonstrate that the proposed PADC can use nearly minimal bandwidth consumption for wireless image transmission tasks with different channel SNR and image contents, at the same time guaranteeing the PSNR constraint for each image data.

Physical-Layer Adversarial Robustness for Deep Learning-Based Semantic Communications

End-to-end semantic communications (ESC) rely on deep neural networks (DNN) to boost communication efficiency by only transmitting the semantics of data, showing great potential for high-demand mobile applications. We argue that central to the success of ESC is the robust interpretation of conveyed semantics at the receiver side, especially for security-critical applications such as automatic driving and smart healthcare. However, robustifying semantic interpretation is challenging as ESC is extremely vulnerable to physical-layer adversarial attacks due to the openness of wireless channels and the fragileness of neural models. Toward ESC robustness in practice, we ask the following two questions: Q1: For attacks, is it possible to generate semantic-oriented physical-layer adversarial attacks that are imperceptible, input-agnostic and controllable? Q2: Can we develop a defense strategy against such semantic distortions and previously proposed adversaries? To this end, we first present MobileSC, a novel semantic communication framework that considers the computation and memory efficiency in wireless environments. Equipped with this framework, we propose SemAdv, a physical-layer adversarial perturbation generator that aims to craft semantic adversaries over the air with the abovementioned criteria, thus answering the Q1. To better characterize the real-world effects for robust training and evaluation, we further introduce a novel adversarial training method SemMixed to harden the ESC against SemAdv attacks and existing strong threats, thus answering the Q2. Extensive experiments on three public benchmarks verify the effectiveness of our proposed methods against various physical adversarial attacks. We also show some interesting findings, e.g., our MobileSC can even be more robust than classical block-wise communication systems in the low SNR regime.

Communication Beyond Transmitting Bits: Semantics-Guided Source and Channel Coding

Classical communication paradigms focus on accurately transmitting bits over a noisy channel, and Shannon theory provides a fundamental theoretical limit on the rate of reliable communications. In this approach, bits are treated equally, and the communication system is oblivious to what meaning these bits convey or how they would be used. Future communications towards intelligence and conciseness will predictably play a dominant role, and the proliferation of connected intelligent agents requires a radical rethinking of coded transmission paradigm to support the new communication morphology on the horizon. The recent concept of "semantic communications" offers a promising research direction. Injecting semantic guidance into the coded transmission design to achieve semantics-aware communications shows great potential for further breakthrough in effectiveness and reliability. This article sheds light on semantics-guided source and channel coding as a transmission paradigm of semantic communications, which exploits both data semantics diversity and wireless channel diversity together to boost the whole system performance. We present the general system architecture and key techniques, and indicate some open issues on this topic.

Exploiting Semantic Communication for Non-Orthogonal Multiple Access

A novel semantics-empowered two-user uplink non-orthogonal multiple access (NOMA) framework is proposed for resource efficiency enhancement. More particularly, a secondary far user (F-user) employs the semantic communication (SemCom) while a primary near user (N-user) employs the conventional bit-based communication (BitCom). The fundamental performance limit, namely <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">semantic-versus-bit (SvB) rate region</i> , of the proposed semantics-empowered NOMA framework is characterized. The <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">equivalent</i> SvB rate region achieved by the conventional BitCom-based NOMA is provided as the baseline scheme. It unveils that, compared to BitCom, SemCom can significantly improve the F-user’s performance when its permitted transmit power is strictly capped, but may perform worse when its permitted transmit power is high. Guided by this result, the proposed semantics-empowered NOMA framework is investigated over fading channels. An <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">opportunistic</i> SemCom and BitCom scheme is proposed, which enables the secondary F-user to participate in NOMA via the most suitable communication method at each fading state, thus striking a good tradeoff between its own achieved performance and the interference imposed on the primary N-user. Two scenarios are considered for employing the opportunistic scheme, namely on-off resource management and continuous resource management. For each scenario, the optimal communication policy over fading channels is derived for maximizing the ergodic semantic rate achieved at the secondary F-user, subject to the minimum ergodic bit rate constraint of the primary N-user. Numerical results show that: 1) The proposed opportunistic scheme in both scenarios can achieve higher communication performance for NOMA than the baseline schemes merely employing SemCom or BitCom; 2) SemCom can better guarantee the performance of the F-user admitted in NOMA than BitCom when the communication requirement of the primary N-user is high; and 3) The continuous power control at the F-user is necessary for ensuring high performance over fading channels, while the on-off time scheduling is sufficient.

A Blockchain-Based Semantic Exchange Framework for Web 3.0 Toward Participatory Economy

Web 3.0 is the next-generation Internet that enables participants to read, write, and own contents in a decentralized manner. It is mainly driven by blockchain, semantic communication, edge computing, and artificial intelligence, which can construct value networks to achieve participatory economics based on participatory decision making. Web 3.0 can capture the characteristics of blockchain, semantic extraction, and communication to achieve decentralized semantic sharing and transfer information precisely. However, current Web 3.0 solutions focus on the blockchain while overlooking other new technologies' roles in Web 3.0. To further unleash the advantages of semantic extraction and communication in Web 3.0, in this paper, we propose a blockchain-based semantic exchange framework to realize fair and efficient interactions. In this framework, we first attempt to tokenize semantic information into Non-Fungible Token (NFT) for semantic exchange. Then we utilize a Stackelberg game to maximize buying and pricing strategies for semantic trading. We also leverage Zero-Knowledge Proof to share authentic semantic information without publishing it before receiving payments, which can achieve a fair and privacy-preserving trading compared with current NFT marketplaces. A case study about urban planning is given to show clearly the proposed mechanisms. Finally, several challenges and opportunities are identified.

Toward Adaptive Semantic Communications: Efficient Data Transmission via Online Learned Nonlinear Transform Source-Channel Coding

The emerging field semantic communication is driving the research of end-to-end data transmission. By utilizing the powerful representation ability of deep learning models, learned data transmission schemes have exhibited superior performance than the established source and channel coding methods. While, so far, research efforts mainly concentrated on architecture and model improvements toward a static target domain. Despite their successes, such learned models are still suboptimal due to the limitations in model capacity and imperfect optimization and generalization, particularly when the testing data distribution or channel response is different from that adopted for model training, as is likely to be the case in real-world. To tackle this, in this paper, we propose a novel online learned joint source and channel coding approach that leverages the deep learning model’s overfitting property. Specifically, we update the off-the-shelf pre-trained models after deployment in a lightweight online fashion to adapt to the distribution shifts in source data and environment domain. We take the overfitting concept to the extreme, proposing a series of implementation-friendly methods to adapt the codec model or representations to an individual data or channel state instance, which can further lead to substantial gains in terms of the end-to-end rate-distortion performance. Accordingly, the streaming ingredients include both the semantic representations of source data and the online updated decoder model parameters. The system design is formulated as a joint optimization problem whose goal is to minimize the loss function, a tripartite trade-off among the data stream bandwidth cost, model stream bandwidth cost, and end-to-end distortion. The proposed methods enable the communication-efficient adaptation for all parameters in the network without sacrificing decoding speed. Extensive experiments, including user study, on continually changing target source data and wireless channel environments, demonstrate the effectiveness and efficiency of our approach, on which we outperform existing state-of-the-art engineered transmission scheme (VVC combined with 5G LDPC coded transmission).

Semantic Communication With Memory

Semantic Communication With Memory

Semantic Communications for Wireless Sensing: RIS-Aided Encoding and Self-Supervised Decoding

Semantic Communications for Wireless Sensing: RIS-Aided Encoding and Self-Supervised Decoding

Guest Editorial: Semantic Communications for the Metaverse

Guest Editorial: Semantic Communications for the Metaverse

Semantic Communication-Based Dynamic Resource Allocation in D2D Vehicular Networks

The semantic communication mechanism enables wireless devices in vehicular networks to communicate more effectively with the semantic meaning. However, in high-dynamic vehicular networks, the transmission of semantic information faces challenges in terms of reliability and stability. To address these challenges, a long-term robust resource allocation scheme is proposed under the Device-to-Device (D2D) vehicular (D2D-V) networks, where multiple performance indicators (user satisfaction, queue stability, and communication delay) are considered. Due to the sophisticated probabilistic form with consideration of channel fluctuations, the Bernstein approximation is introduced to acquire the deterministic constraint more efficiently. The robust resource allocation problem is proposed and separated into two independent subproblems by the Lyapunov optimization method, which includes semantic access control in the application layer and power control in the physical layer. After that, the successive convex approximation method and Karush-Kuhn-Tucher conditions are adopted to solve the subproblems, thereby proposing a robust resource allocation algorithm. The simulations reveal the trade-off relationship between user satisfaction, queue stability, and communication delay, which is on the premise of meeting the user SINR requirement. Moreover, the simulations also prove the necessity of considering channel uncertainty in high-speed mobile vehicular communication scenarios.

Quantum Semantic Communications for Metaverse: Principles and Challenges

Quantum Semantic Communications for Metaverse: Principles and Challenges

Semantic-Aware Sensing Information Transmission for Metaverse: A Contest Theoretic Approach

With the advancement of network and computer technologies, virtual cyberspace keeps evolving, and Metaverse is the main representative. As an irreplaceable technology that supports Metaverse, the sensing information transmission from the physical world to Metaverse is vital. Inspired by emerging semantic communication, in this paper, we propose a semantic transmission framework for transmitting sensing information from the physical world to Metaverse. Leveraging the in-depth understanding of sensing information, we define the semantic bases, through which the semantic encoding of sensing data is achieved for the first time. Consequently, the amount of sensing data that needs to be transmitted is dramatically reduced. Unlike conventional methods that undergo data degradation and require data recovery, our approach achieves the sensing goal without data recovery while maintaining performance. To further improve Metaverse service quality, we introduce contest theory to create an incentive mechanism that motivates users to upload data more frequently. Experimental results show that the average data amount after semantic encoding is reduced to about 27.87% of that before encoding, while ensuring the sensing performance. Additionally, the proposed contest theoretic based incentive mechanism increases the sum of data uploading frequency by 27.47% compared to the uniform award scheme.

Rethinking Semantic Image Compression: Scalable Representation With Cross-Modality Transfer

This article proposes the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">scalable cross-modality compression</i> (SCMC) paradigm, in which the image compression problem is further cast into a representation task by hierarchically sketching the image with different modalities. Herein, we adopt the conceptual organization philosophy to model the overwhelmingly complicated visual patterns, based upon the semantic, structure, and signal level representation accounting for different tasks. The SCMC paradigm that incorporates the representation at different granularities supports diverse application scenarios, such as high-level semantic communication and low-level image reconstruction. The decoder, which enables the recovery of the visual information, benefits from the scalable coding based upon the semantic, structure, and signal layers. Qualitative and quantitative results demonstrate that the SCMC can convey accurate semantic and perceptual information of images, especially at low bitrates, and promising rate-distortion performance has been achieved compared to state-of-the-art methods. The code will be available online https://github.com/ppingzhang/SCMC.

Adversarial Attacks and Defenses for Semantic Communication in Vehicular Metaverses

Adversarial Attacks and Defenses for Semantic Communication in Vehicular Metaverses

Trustworthy Semantic Communications for the Metaverse Relying on Federated Learning

Trustworthy Semantic Communications for the Metaverse Relying on Federated Learning

Semantic Communications for the Metaverse and Other Wireless Innovations

Semantic Communications for the Metaverse and Other Wireless Innovations

The Indoor Positioning Method Time Difference of Arrival with Conic Curves Utilizing a Novel Networking RFID System

At present, the demand for accurate indoor positioning at a low cost is increasing. Based on the architecture of networking passive radio frequency identification (RFID) systems, research into passive location algorithms is important for finding a location solution with ultra-low cost, easy implementation, and no required maintenance. In this paper, TDACC (time difference of arrival with conic curves) based on signal propagation time is proposed, which breaks down the positioning problem into solving the intersection of an ellipse and a hyperbola. The results indicate that this method has a positioning error of 0 m in the absence of signal interference. When the time delay fluctuates to 1 ns and 2 ns, the average errors of TDACC are 0.19 m and 0.33 m, respectively. Different from other time-based localization methods, the proposed method only requires two distribution nodes without time synchronization, which reduces the system cost. These results will help to promote the deeper semantic communication level fusion of passive RFID. By improving the coordinate positioning in the semantic prior knowledge base, this method will lead to more efficient and accurate industry applications.

A holistic framework for interpreting positive clinical outcomes in biotechnology

Clinical trials conducted by biotechnology companies are vital to not only the success of their products, but the health and safety of future patients. However, positive study results do not always lead to FDA authorization and subsequent release to the market for numerous reasons including, false positive rates, validity and reliability of outcomes, the significance of results, and the semantics of company communications. Driven by the need to compete within the marketplace, biotechnology companies have motivation to distort and/or present results to make them appear more favorable. As a result, it is important to approach positive clinical trial results with a healthy amount of skepticism and utilize a more holistic approach to evaluating results and determining whether the advertised product should be released into the market.

Network public opinion monitoring and semantic event discovery strategy in mobile edge computing scenario

AbstractBuilding a public opinion monitoring system can detect public opinion crises in advance and deal with crisis public relations in a timely manner. Semantic communication is a new type of communication technology with development potential, which reduces the amount of data required for transmission by mining semantic information in information sources. In this paper, a multi‐user semantic communication system based on federated learning deployment is used to realize data transmission in edge computing scenarios. Use the data from the client side to train the deep learning model more effectively. This paper also evaluates the performance of the proposed semantic communication system. The model trained by federated learning can achieve an effect close to centralized training and protect user privacy.