Breakthroughs In Deep Learning Research Articles

Crossing the principle–practice gap in AI ethics with ethical problem-solving

AbstractThe past years have presented a surge in (AI) development, fueled by breakthroughs in deep learning, increased computational power, and substantial investments in the field. Given the generative capabilities of more recent AI systems, the era of large-scale AI models has transformed various domains that intersect our daily lives. However, this progress raises concerns about the balance between technological advancement, ethical considerations, safety measures, and financial interests. Moreover, using such systems in sensitive areas amplifies our general ethical awareness, prompting a re-emergence of debates on governance, regulation, and human values. However, amidst this landscape, how to bridge the principle–practice gap separating ethical discourse from the technical side of AI development remains an open problem. In response to this challenge, the present work proposes a framework to help shorten this gap: ethical problem-solving (EPS). EPS is a methodology promoting responsible, human-centric, and value-oriented AI development. The framework’s core resides in translating principles into practical implementations using impact assessment surveys and a differential recommendation methodology. We utilize EPS as a blueprint to propose the implementation of an Ethics as a Service Platform, currently available as a simple demonstration. We released all framework components openly and with a permissive license, hoping the community would adopt and extend our efforts into other contexts. Available in the following URL https://nkluge-correa.github.io/ethical-problem-solving/.

A Review on Recent Deep Learning-Based Semantic Segmentation for Urban Greenness Measurement.

Accurate urban green space (UGS) measurement has become crucial for landscape analysis. This paper reviews the recent technological breakthroughs in deep learning (DL)-based semantic segmentation, emphasizing efficient landscape analysis, and integrating greenness measurements. It explores quantitative greenness measures applied through semantic segmentation, categorized into the plan view- and the perspective view-based methods, like the Land Class Classification (LCC) with green objects and the Green View Index (GVI) based on street photographs. This review navigates from traditional to modern DL-based semantic segmentation models, illuminating the evolution of the urban greenness measures and segmentation tasks for advanced landscape analysis. It also presents the typical performance metrics and explores public datasets for constructing these measures. The results show that accurate (semantic) segmentation is inevitable not only for fine-grained greenness measures but also for the qualitative evaluation of landscape analyses for planning amidst the incomplete explainability of the DL model. Also, the unsupervised domain adaptation (UDA) in aerial images is addressed to overcome the scale changes and lack of labeled data for fine-grained greenness measures. This review contributes to helping researchers understand the recent breakthroughs in DL-based segmentation technology for challenging topics in UGS research.

AI on the Road: NVIDIA Jetson Nano-Powered Computer Vision-Based System for Real-Time Pedestrian and Priority Sign Detection

Advances in information and signal processing, driven by artificial intelligence techniques and recent breakthroughs in deep learning, have significantly impacted autonomous driving by enhancing safety and reducing the dependence on human intervention. Generally, prevailing ADASs (advanced driver assistance systems) incorporate costly components, making them financially unattainable for a substantial portion of the population. This paper proposes a solution: an embedded system designed for real-time pedestrian and priority sign detection, offering affordability and universal applicability across various vehicles. The suggested system, which comprises two cameras, an NVIDIA Jetson Nano B01 low-power edge device and an LCD (liquid crystal system) display, ensures seamless integration into a vehicle without occupying substantial space and provides a cost-effective alternative. The primary focus of this research is addressing accidents caused by the failure to yield priority to other drivers or pedestrians. Our study stands out from existing research by concurrently addressing traffic sign recognition and pedestrian detection, concentrating on identifying five crucial objects: pedestrians, pedestrian crossings (signs and road paintings separately), stop signs, and give way signs. Object detection was executed using a lightweight, custom-trained CNN (convolutional neural network) known as SSD (Single Shot Detector)-MobileNet, implemented on the Jetson Nano. To tailor the model for this specific application, the pre-trained neural network underwent training on our custom dataset consisting of images captured on the road under diverse lighting and traffic conditions. The outcomes of the proposed system offer promising results, positioning it as a viable candidate for real-time implementation; its contributions are noteworthy in advancing the safety and accessibility of autonomous driving technologies.

Assessing the Impact of Communicative Artificial Intelligence Based Accounting Information Systems on Small and Medium Enterprises

Recent breakthroughs in deep learning have led to the development of cutting-edge industrial applications of Communicative Artificial Intelligence (AI), making it indispensable for businesses aiming to maintain a competitive edge. Consequently, artificial intelligence is no longer exclusive to large corporations; it now impacts businesses of all sizes, including small and medium-sized enterprises (SMEs), serving as a tool for command in the production and communication of crucial business aspects. This article delves into the extent of Communicative AI adoption by SMEs in Indonesia, shedding light on issues related to implementing industrial AI applications. To achieve this, a sample of SMEs participated in a structured online survey. Currently, AI adoption among SMEs in Indonesia is minimal. The reluctance is primarily attributed to high costs, extended duration, and inherent risks of developing proprietary applications. Instead, SMEs are heavily relying on AI-as-a-service and other cloud-based solutions. Various factors contribute to businesses' hesitancy. The slow progress in SME implementation indicates misunderstandings related to data and a lack of knowledge, influencing how these enterprises perceive the obstacles they encounter.

Complex-Valued Convolution and Frequency Global Filter for Automatic Modulation Recognition

Automatic modulation recognition (AMR) plays an important role in cognitive radio and dynamic spectrum access, which has been widely applied in military and civilian applications. Due to the breakthroughs in deep learning (DL), DL-based AMR methods are becoming extremely popular. However, most existing DL-based methods are unable to deal with complex format data, and learning the mappings from the time series or its transformed representation to the true modulation type directly is difficult. To address these difficulties, this letter presents a complex-valued convolution and frequency global filter unit (CGFU), and proposes a hybrid neural network, namely CGF-HNN, which can efficiently exploit features from different domains. We evaluate the recognition performance of the proposed model on two well-known datasets, i.e., RML2016.10a and RML2018.01a. Simulation results show that the proposed model outperforms the existing state-of-the-art models.

A unified microstructure segmentation approach via human-in-the-loop machine learning

Microstructure segmentation, a technique for extracting structural statistics from microscopy images, is an essential step for establishing quantitative structure–property relationships in a broad range of materials research areas. However, the task is challenging due to the morphological complexity and diversity of structural constituents. While recent breakthroughs in deep learning have led to significant progress in microstructure segmentation, there remain two fundamental challenges to its further diffusion: prohibitive annotation costs and an unreliable decision-making process. To tackle these challenges, we propose a human-in-the-loop machine learning framework for unified microstructure segmentation, which leverages recent advances in both weakly supervised learning and active learning techniques. The key idea behind our approach lies in the integration of human and machine capabilities to make not only precise but also reliable microstructure segmentation at minimal annotation costs. Extensive experiments demonstrate the generality of our framework across different material classes, structural constituents, and microscopic imaging modalities.

A knowledge-inherited learning for intelligent metasurface design and assembly

Recent breakthroughs in deep learning have ushered in an essential tool for optics and photonics, recurring in various applications of material design, system optimization, and automation control. Deep learning-enabled on-demand metasurface design has been the subject of extensive expansion, as it can alleviate the time-consuming, low-efficiency, and experience-orientated shortcomings in conventional numerical simulations and physics-based methods. However, collecting samples and training neural networks are fundamentally confined to predefined individual metamaterials and tend to fail for large problem sizes. Inspired by object-oriented C++ programming, we propose a knowledge-inherited paradigm for multi-object and shape-unbound metasurface inverse design. Each inherited neural network carries knowledge from the “parent” metasurface and then is freely assembled to construct the “offspring” metasurface; such a process is as simple as building a container-type house. We benchmark the paradigm by the free design of aperiodic and periodic metasurfaces, with accuracies that reach 86.7%. Furthermore, we present an intelligent origami metasurface to facilitate compatible and lightweight satellite communication facilities. Our work opens up a new avenue for automatic metasurface design and leverages the assemblability to broaden the adaptability of intelligent metadevices.

A Survey of Deep Learning Based NOMA: State of the Art, Key Aspects, Open Challenges and Future Trends

Non-Orthogonal Multiple Access (NOMA) has become a promising evolution with the emergence of fifth-generation (5G) and Beyond-5G (B5G) rollouts. The potentials of NOMA are to increase the number of users, the system's capacity, massive connectivity, and enhance the spectrum and energy efficiency in future communication scenarios. However, the practical deployment of NOMA is hindered by the inflexibility caused by the offline design paradigm and non-unified signal processing approaches of different NOMA schemes. The recent innovations and breakthroughs in deep learning (DL) methods have paved the way to adequately address these challenges. The DL-based NOMA can break these fundamental limits of conventional NOMA in several aspects, including throughput, bit-error-rate (BER), low latency, task scheduling, resource allocation, user pairing and other better performance characteristics. This article aims to provide firsthand knowledge of the prominence of NOMA and DL and surveys several DL-enabled NOMA systems. This study emphasizes Successive Interference Cancellation (SIC), Channel State Information (CSI), impulse noise (IN), channel estimation, power allocation, resource allocation, user fairness and transceiver design, and a few other parameters as key performance indicators of NOMA systems. In addition, we outline the integration of DL-based NOMA with several emerging technologies such as intelligent reflecting surfaces (IRS), mobile edge computing (MEC), simultaneous wireless and information power transfer (SWIPT), Orthogonal Frequency Division Multiplexing (OFDM), and multiple-input and multiple-output (MIMO). This study also highlights diverse, significant technical hindrances in DL-based NOMA systems. Finally, we identify some future research directions to shed light on paramount developments needed in existing systems as a probable to invigorate further contributions for DL-based NOMA system.

Intelligent identification of pavement cracks based on PSA-Net

The identification of pavement cracks is critical for ensuring road safety. Currently, manual crack detection is quite time-consuming. To address this issue, automated pavement crack-detecting technology is required. However, automatic pavement crack recognition remains challenging, owing to the intensity heterogeneity of cracks and the complexity of the backdrop, e.g., low contrast of damages and backdrop may have shadows with comparable intensity. Motivated by breakthroughs in deep learning, we present a new network architecture combining the feature pyramid with the attention mechanism (PSA-Net). In a feature pyramid, the network integrates spatial information and underlying features for crack detection. During the training process, it improves the accuracy of automatic road crack recognition by nested sample weighting to equalize the loss caused by simple and complex samples. To verify the effectiveness of the suggested technique, we used a dataset of real road cracks to test it with different crack detection methods.

Remote Sensing Image Scene Classification: Advances and Open Challenges

Deep learning approaches are gaining popularity in image feature analysis and in attaining state-of-the-art performances in scene classification of remote sensing imagery. This article presents a comprehensive review of the developments of various computer vision methods in remote sensing. There is currently an increase of remote sensing datasets with diverse scene semantics; this renders computer vision methods challenging to characterize the scene images for accurate scene classification effectively. This paper presents technology breakthroughs in deep learning and discusses their artificial intelligence open-source software implementation framework capabilities. Further, this paper discusses the open gaps/opportunities that need to be addressed by remote sensing communities.

VISHIEN-MAAT: Scrollytelling visualization design for explaining Siamese Neural Network concept to non-technical users

The past decade has witnessed rapid progress in AI research since the breakthrough in deep learning. AI technology has been applied in almost every field; therefore, technical and non-technical end-users must understand these technologies to exploit them. However existing materials are designed for experts, but non-technical users need appealing materials that deliver complex ideas in easy-to-follow steps. One notable tool that fits such a profile is scrollytelling, an approach to storytelling that provides readers with a natural and rich experience at the reader’s pace, along with in-depth interactive explanations of complex concepts. Hence, this work proposes a novel visualization design for creating a scrollytelling that can effectively explain an AI concept to non-technical users. As a demonstration of our design, we created a scrollytelling to explain the Siamese Neural Network for the visual similarity matching problem. Our approach helps create a visualization valuable for a short-timeline situation like a sales pitch. The results show that the visualization based on our novel design helps improve non-technical users’ perception and machine learning concept knowledge acquisition compared to traditional materials like online articles.

Toward Ambient Intelligence: Federated Edge Learning With Task-Oriented Sensing, Computation, and Communication Integration

With the breakthroughs in deep learning and contactless sensors, the recent years have witnessed a rise of ambient intelligence applications and services, spanning from healthcare delivery to intelligent home. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Federated edge learning</i> (FEEL), as a privacy-enhancing paradigm of collaborative learning at the network edge, is expected to be the core engine to achieve ambient intelligence. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Sensing, computation, and communication</i> (SC <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$^{2}$</tex-math></inline-formula> ) are highly coupled processes in FEEL and need to be jointly designed in a task-oriented manner to achieve the best FEEL performance under stringent resource constraints at edge devices. However, this remains an open problem as there is a lack of theoretical understanding on how the SC <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$^{2}$</tex-math></inline-formula> resources jointly affect the FEEL performance. In this paper, we address the problem of joint SC <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$^{2}$</tex-math></inline-formula> resource allocation for FEEL via a concrete case study of human motion recognition based on wireless sensing in ambient intelligence. First, by analyzing the wireless sensing process in human motion recognition, we find that there exists a thresholding value for the sensing transmit power, exceeding which yields sensing data samples with approximately the same satisfactory quality. Then, the joint SC <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$^{2}$</tex-math></inline-formula> resource allocation problem is cast to maximize the convergence speed of FEEL, under the constraints on training time, energy supply, and sensing quality of each edge device. Solving this problem entails solving two subproblems in order: the first one reduces to determine the joint sensing and communication resource allocation that maximizes the total number of samples that can be sensed during the entire training process; the second one concerns the partition of the attained total number of sensed samples over all the communication rounds to determine the batch size at each round for convergence speed maximization. The first subproblem on joint sensing and communication resource allocation is converted to a single-variable optimization problem by exploiting the derived relation between different control variables (resources), which thus allows an efficient solution via one-dimensional grid search. For the second subproblem, it is found that the number of samples to be sensed (or batch size) at each round is a decreasing function of the loss function value attained at the round. Based on this relationship, the approximate optimal batch size at each communication round is derived in closed-form as a function of the round index. Finally, extensive simulation results are provided to validate the superiority of the proposed joint SC <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$^{2}$</tex-math></inline-formula> resource allocation scheme over baseline schemes in terms of FEEL performance.

Learning to discover medicines.

Discovering new medicines is the hallmark of the human endeavor to live a better and longer life. Yet the pace of discovery has slowed down as we need to venture into more wildly unexplored biomedical space to find one that matches today's high standard. Modern AI-enabled by powerful computing, large biomedical databases, and breakthroughs in deep learning offers a new hope to break this loop as AI is rapidly maturing, ready to make a huge impact in the area. In this paper, we review recent advances in AI methodologies that aim to crack this challenge. We organize the vast and rapidly growing literature on AI for drug discovery into three relatively stable sub-areas: (a) representation learning over molecular sequences and geometric graphs; (b) data-driven reasoning where we predict molecular properties and their binding, optimize existing compounds, generate de novo molecules, and plan the synthesis of target molecules; and (c) knowledge-based reasoning where we discuss the construction and reasoning over biomedical knowledge graphs. We will also identify open challenges and chart possible research directions for the years to come.

Report of the 5th ACM SIGSPATIAL International Workshop on AI for Geographic Knowledge Discovery (GeoAI 2022)

Emerging advances in artificial intelligence, hardware accelerators, and big data processing architectures continue to reach the geospatial information sciences, with a transformative impact on many societal challenges. Recent breakthroughs in deep learning have brought forward an automated capability to learn representative features from massive and complex data, including text, images, or videos. In tandem, rapid innovations in sensing technologies are supporting the collection of geospatial data in even higher resolution and throughput, supporting the observation, mapping, and analysis of different events/phenomena over the earth's surface and in socioeconomic environments with unprecedented detail. Combined, these developments are offering the potential for breakthroughs in geographic knowledge discovery, impacting decision-making in areas such as humanitarian mapping, intelligent transportation systems, urban expansion analysis, health data analysis and epidemiology, the study of climate change, handling natural disasters, and the general monitoring of the earth's surface.

BlockCrime: Blockchain and Deep Learning-Based Collaborative Intelligence Framework to Detect Malicious Activities for Public Safety

Detecting malicious activity in advance has become increasingly important for public safety, economic stability, and national security. However, the disparity in living standards incites the minds of certain undesirable members of society to commit crimes, which may disrupt society’s stability and mental calm. Breakthroughs in deep learning (DL) make it feasible to address such challenges and construct a complete intelligent framework that automatically detects such malicious behaviors. Motivated by this, we propose a convolutional neural network (CNN)-based Xception model, i.e., BlockCrime, to detect crimes and improve public safety. Furthermore, we integrate blockchain technology to securely store the detected crime scene locations and alert the nearest law enforcement authorities. Due to the scarcity of the dataset, transfer learning has been preferred, in which a CNN-based Xception model is used. The redesigned Xception architecture is evaluated against various assessment measures, including accuracy, F1 score, precision, and recall, where it outperforms existing CNN architectures in terms of train accuracy, i.e., 96.57%.

An Efficient Deep Learning Model with Interrelated Tagging Prototype with Segmentation for Telugu Optical Character Recognition

More than 66 million people in India speak Telugu, a language that dates back thousands of years and is widely spoken in South India. There has not been much progress reported on the advancement of Telugu text Optical Character Recognition (OCR) systems. Telugu characters can be composed of many symbols joined together. OCR is the process of turning a document image into a text-editable one that may be used in other applications. It saves a great deal of time and effort by not having to start from scratch each time. There are hundreds of thousands of different combinations of modifiers and consonants when writing compound letters. Symbols joined to one another form a compound character. Since there are so many output classes in Telugu, there’s a lot of interclass variation. Additionally, there are not any Telugu OCR systems that take use of recent breakthroughs in deep learning, which prompted us to create our own. When used in conjunction with a word processor, an OCR system has a significant impact on real-world applications. In a Telugu OCR system, we offer two ways to improve symbol or glyph segmentation. When it comes to Telugu OCR, the ability to recognise that Telugu text is crucial. In a picture, connected components are collections of identical pixels that are connected to one another by either 4- or 8-pixel connectivity. These connected components are known as glyphs in Telugu. In the proposed research, an efficient deep learning model with Interrelated Tagging Prototype with Segmentation for Telugu Text Recognition (ITP-STTR) is introduced. The proposed model is compared with the existing model and the results exhibit that the proposed model’s performance in text recognition is high.

Genomics enters the deep learning era.

The tremendous amount of biological sequence data available, combined with the recent methodological breakthrough in deep learning in domains such as computer vision or natural language processing, is leading today to the transformation of bioinformatics through the emergence of deep genomics, the application of deep learning to genomic sequences. We review here the new applications that the use of deep learning enables in the field, focusing on three aspects: the functional annotation of genomes, the sequence determinants of the genome functions and the possibility to write synthetic genomic sequences.

Dali server: structural unification of protein families.

Protein structure is key to understanding biological function. Structure comparison deciphers deep phylogenies, providing insight into functional conservation and functional shifts during evolution. Until recently, structural coverage of the protein universe was limited by the cost and labour involved in experimental structure determination. Recent breakthroughs in deep learning revolutionized structural bioinformatics by providing accurate structural models of numerous protein families for which no structural information existed. The Dali server for 3D protein structure comparison is widely used by crystallographers to relate new structures to pre-existing ones. Here, we report two most recent upgrades to the web server: (i) the foldomes of key organisms in the AlphaFold Database (version 1) are searchable by Dali, (ii) structural alignments are annotated with protein families. Using these new features, we discovered a novel functionally diverse subgroup within the WRKY/GCM1 clan. This was accomplished by linking the structurally characterized SWI/SNF and NAM families as well as the structural models of the CG-1 family and uncharacterized proteins to the structure of Gti1/Pac2, a previously known member of the WRKY/GCM1 clan. The Dali server is available at http://ekhidna2.biocenter.helsinki.fi/dali. This website is free and open to all users and there is no login requirement.

X-vector anonymization using autoencoders and adversarial training for preserving speech privacy

The rapid increase in web services and mobile apps, which collect personal data from users, has also increased the risk that their privacy may be severely compromised. In particular, the increasing variety of spoken language interfaces and voice assistants empowered by the vertiginous breakthroughs in deep learning have prompted important concerns in the European Union in terms of preserving the privacy of speech data. For instance, an attacker can record speech from users and impersonate them to obtain access to systems that require voice identification. By extracting speaker, linguistic (e.g., dialect), and paralinguistic features (e.g., age) from a speech signal, the speaker profiles can also be hacked from users through existing technology.To mitigate these weaknesses, in this study, we present a speech anonymization method based on autoencoders and adversarial training. Given an utterance, we first extract an x-vector, which is a powerful utterance-level embedding used in state-of-the-art speaker recognition. This original x-vector is transformed by an autoencoder producing a new x-vector, where speaker, gender, and accent information are suppressed through adversarial training. The anonymized speech is finally generated through a neural speech synthesizer driven by the anonymized x-vector, fundamental frequency, and phoneme information extracted from the input speech.For the evaluation, we followed the VoicePrivacy Challenge framework, where anonymization or privacy is measured using automatic speaker verification and the preservation of the intelligibility is evaluated through automatic speech recognition. Our experimental results show that the proposed method achieves higher privacy than the VoicePrivacy baseline (i.e., a higher speaker verification error) while preserving a similar intelligibility for the spoken content (i.e., a similar word error rate).

CrabNet for Explainable Deep Learning in Materials Science: Bridging the Gap Between Academia and Industry

Despite recent breakthroughs in deep learning for materials informatics, there exists a disparity between their popularity in academic research and their limited adoption in the industry. A significant contributor to this “interpretability-adoption gap” is the prevalence of black-box models and the lack of built-in methods for model interpretation. While established methods for evaluating model performance exist, an intuitive understanding of the modeling and decision-making processes in models is nonetheless desired in many cases. In this work, we demonstrate several ways of incorporating model interpretability to the structure-agnostic Compositionally Restricted Attention-Based network, CrabNet. We show that CrabNet learns meaningful, material property-specific element representations based solely on the data with no additional supervision. These element representations can then be used to explore element identity, similarity, behavior, and interactions within different chemical environments. Chemical compounds can also be uniquely represented and examined to reveal clear structures and trends within the chemical space. Additionally, visualizations of the attention mechanism can be used in conjunction to further understand the modeling process, identify potential modeling or dataset errors, and hint at further chemical insights leading to a better understanding of the phenomena governing material properties. We feel confident that the interpretability methods introduced in this work for CrabNet will be of keen interest to materials informatics researchers as well as industrial practitioners alike.