Hardware Software Architecture Research Articles

Vision Systems for a UR5 Cobot on a Quality Control Robotic Station

This paper addresses the development of a vision system for the UR5 cobot and the corresponding operating algorithm of the robotic quality control station. The hardware–software architecture of the developed robotic station consisting of a UR5 cobot equipped with a web camera and a stationary industrial camera with a lighting system is presented. Image processing and analysis algorithms are described, the method of control and communication between the station components is discussed, and operating scenarios are presented as a single robotic station and as a part of the robotic line. Based on the results which were obtained, the level of measurement noise, accuracy, and repeatability of the developed vision system were estimated. A novel complete vision system architecture with hardware–software modules for the UR5 cobot quality control station is shown and discussed. The software part based on Python 3.12 language, OpenCV 4.7.0.68 libraries, and the PolyScope 1.8 environment incorporates modules for system calibration, image acquisition, preprocessing and image analysis (for objects’ location and geometric measurements) and for robotic cell communication and control. The hardware part of the UR5 cobot vision system is based on a PC with two independent and distinct cameras: one permanently affixed and the other mounted to the cobot’s flange. This innovative setup, combined with software architecture, broadens the scope of existing robotic quality control applications.

Design and Implementation of Hardware-Software Architecture Based on Hashes for SPHINCS+

Advances in quantum computing have posed a future threat to today’s cryptography. With the advent of these quantum computers, security could be compromised. Therefore, the National Institute of Standards and Technology (NIST) has issued a request for proposals to standardize algorithms for post-quantum cryptography (PQC), which is considered difficult to solve for both classical and quantum computers. Among the proposed technologies, the most popular choices are lattice-based (shortest vector problem) and hash-based approaches. Other important categories are public key cryptography (PKE) and digital signatures. Within the realm of digital signatures lies SPHINCS+. However, there are few implementations of this scheme in hardware architectures. In this article, we present a hardware-software architecture for the SPHINCS+ scheme. We utilized a free RISC-V (Reduced Instruction Set Computer) processor synthesized on a Field Programmable Gate Array (FPGA), primarily integrating two accelerator modules for Keccak-1600 and the Haraka hash function. Additionally, modifications were made to the processor to accommodate the execution of these added modules. Our implementation yielded a 15-fold increase in performance with the SHAKE-256 function and nearly 90-fold improvement when using Haraka, compared to the reference software. Moreover, it is more compact compared to related works. This implementation was realized on a Xilinx FPGA Arty S7: Spartan-7.

Enhancing Vehicle Navigation and Safety through Integration of Pre-Recorded Maps with Vehicle Control Unit

Powerful navigation and safety systems are vital to ensuring autonomous (DV) or semi-autonomous vehicles continue well in varied environmental conditions. However, in such a world those who depend on that internet connectivity to navigate their car would have problems in areas where network reliability is less than perfect. The purpose of this work is to investigate the feasibility and advantages of combining offline mapping with locally sensed positioning systems for better vehicle navigation and safety. While connected to the internet, vehicles cache offline maps and use local sensor information (such as GPS) from their inertial navigation system or computer vision without needing continuous access. The research outlines a hardware-software architecture with embedded offline map data storage, edge-based road following algorithms and an integration mechanism to advanced driving assistance systems (ADAS). In the evaluation of performance, accuracy assessments with online mapping systems are considered for offline maps and how these can have implications on end driver usability as well impacts on real-world autonomous driving technologies. The results suggest that offline mapping and on-board sensor-based localization would be able to improve vehicle contextual navigation performance in diverse driving scenarios.

A Reconfigurable Architecture for Real-time Event-based Multi-Object Tracking

Although advances in event-based machine vision algorithms have demonstrated unparalleled capabilities in performing some of the most demanding tasks, their implementations under stringent real-time and power constraints in edge systems remain a major challenge. In this work, a reconfigurable hardware-software architecture called REMOT, which performs real-time event-based multi-object tracking on FPGAs, is presented. REMOT performs vision tasks by defining a set of actions over attention units (AUs). These actions allow AUs to track an object candidate autonomously by adjusting its region of attention and allow information gathered by each AU to be used for making algorithmic-level decisions. Taking advantage of this modular structure, algorithm-architecture codesign can be performed by implementing different parts of the algorithm in either hardware or software for different tradeoffs. Results show that REMOT can process 0.43–2.91 million events per second at 1.75–5.45 W. Compared with the software baseline, our implementation achieves up to 44 times higher throughput and 35.4 times higher power efficiency. Migrating the Merge operation to hardware further reduces the worst-case latency to be 95 times shorter than the software baseline. By varying the AU configuration and operation, a reduction of 0.59–0.77 mW per AU on the programmable logic has also been demonstrated.

Extremely Sparse Networks via Binary Augmented Pruning for Fast Image Classification.

Network pruning and binarization have been demonstrated to be effective in neural network accelerator design for high speed and energy efficiency. However, most existing pruning approaches achieve a poor tradeoff between accuracy and efficiency, which on the other hand, has limited the progress of neural network accelerators. At the same time, binary networks are highly efficient, however, a large accuracy gap exists between binary networks and their full-precision counterparts. In this article, we investigate the merits of extremely sparse networks with binary connections for image classification through software-hardware codesign. More specifically, we first propose a binary augmented extremely pruning method that can achieve ~98% sparsity with small accuracy degradation. Then we design the hardware architecture based on the resulting sparse and binary networks, which extensively explores the benefits of extreme sparsity with negligible resource consumption introduced by binary branch. Experiments on large-scale ImageNet classification and field-programmable gate array (FPGA) demonstrate that the proposed software-hardware architecture can achieve a prominent tradeoff between accuracy and efficiency.

Electromyogram-Based Lip-Reading via Unobtrusive Dry Electrodes and Machine Learning Methods.

Lip-reading provides an effective speech communication interface for people with voice disorders and for intuitive human-machine interactions. Existing systems are generally challenged by bulkiness, obtrusiveness, and poor robustness against environmental interferences. The lack of a truly natural and unobtrusive system for converting lip movements to speech precludes the continuous use and wide-scale deployment of such devices. Here, the design of a hardware-software architecture to capture, analyze, and interpret lip movements associated with either normal or silent speech is presented. The system can recognize different and similar visemes. It is robust in a noisy or dark environment. Self-adhesive, skin-conformable, and semi-transparent dry electrodes are developed to track high-fidelity speech-relevant electromyogram signals without impeding daily activities. The resulting skin-like sensors can form seamless contact with the curvilinear and dynamic surfaces of the skin, which is crucial for a high signal-to-noise ratio and minimal interference. Machine learning algorithms are employed to decode electromyogram signals and convert them to spoken words. Finally, the applications of the developed lip-reading system in augmented reality and medical service are demonstrated, which illustrate the great potential in immersive interaction and healthcare applications.

Spectrum Sensing Using Optimized Deep Learning Techniques in Reconfigurable Embedded Systems

The exponential growth of Internet of Things (IoT) and 5G networks has resulted in maximum users, and the role of cognitive radio has become pivotal in handling the crowded users. In this scenario, cognitive radio techniques such as spectrum sensing, spectrum sharing and dynamic spectrum access will become essential components in Wireless IoT communication. IoT devices must learn adaptively to the environment and extract the spectrum knowledge and inferred spectrum knowledge by appropriately changing communication parameters such as modulation index, frequency bands, coding rate etc., to accommodate the above characteristics. Implementing the above learning methods on the embedded chip leads to high latency, high power consumption and more chip area utilisation. To overcome the problems mentioned above, we present DEEP HOLE Radio systems, the intelligent system enabling the spectrum knowledge extraction from the unprocessed samples by the optimized deep learning models directly from the Radio Frequency (RF) environment. DEEP HOLE Radio provides (i) an optimized deep learning framework with a good trade-off between latency, power and utilization. (ii) Complete Hardware-Software architecture where the SoC’s coupled with radio transceivers for maximum performance. The experimentation has been carried out using GNURADIO software interfaced with Zynq-7000 devices mounting on ESP8266 radio transceivers with inbuilt Omni directional antennas. The whole spectrum of knowledge has been extracted using GNU radio. These extracted features are used to train the proposed optimized deep learning models, which run parallel on Zynq-SoC 7000, consuming less area, power, latency and less utilization area. The proposed framework has been evaluated and compared with the existing frameworks such as RFLearn, Long Term Short Memory (LSTM), Convolutional Neural Networks (CNN) and Deep Neural Networks (DNN). The outcome shows that the proposed framework has outperformed the existing framework regarding the area, power and time. Moreover, the experimental results show that the proposed framework decreases the delay, power and area by 15%, 20% 25% concerning the existing RFlearn and other hardware constraint frameworks.

A System-on-Chip Based Hybrid Neuromorphic Compute Node Architecture for Reproducible Hyper-Real-Time Simulations of Spiking Neural Networks.

Despite the great strides neuroscience has made in recent decades, the underlying principles of brain function remain largely unknown. Advancing the field strongly depends on the ability to study large-scale neural networks and perform complex simulations. In this context, simulations in hyper-real-time are of high interest, as they would enable both comprehensive parameter scans and the study of slow processes, such as learning and long-term memory. Not even the fastest supercomputer available today is able to meet the challenge of accurate and reproducible simulation with hyper-real acceleration. The development of novel neuromorphic computer architectures holds out promise, but the high costs and long development cycles for application-specific hardware solutions makes it difficult to keep pace with the rapid developments in neuroscience. However, advances in System-on-Chip (SoC) device technology and tools are now providing interesting new design possibilities for application-specific implementations. Here, we present a novel hybrid software-hardware architecture approach for a neuromorphic compute node intended to work in a multi-node cluster configuration. The node design builds on the Xilinx Zynq-7000 SoC device architecture that combines a powerful programmable logic gate array (FPGA) and a dual-core ARM Cortex-A9 processor extension on a single chip. Our proposed architecture makes use of both and takes advantage of their tight coupling. We show that available SoC device technology can be used to build smaller neuromorphic computing clusters that enable hyper-real-time simulation of networks consisting of tens of thousands of neurons, and are thus capable of meeting the high demands for modeling and simulation in neuroscience.

Versatile implementation of a hardware–software architecture for development and testing of brain–computer interfaces

Brain–computer interfaces (BCI) have been focused on improving people’s lifestyles with motor or communication disabilities. However, the utilization of this technology has found news applications, such as increasing human capacities. Nowadays, several researchers are working on probing human capabilities to control several robotic devices simultaneously. The design of BCI is an intricate work that needs a long time to its implementation. For this reason, an architecture to design and implement different types of BCIs is presented in this article. The architecture has a modular design capable of reading various electroencephalography (EEG) sensors and controlling several robotic devices similar to the plug-and-play paradigm. To test the proposed architecture, a BCI was able to manage a hexapod robot and a drone was implemented. Firstly, a mobile robotic platform was designed and implemented. The BCI is based on eye blinking, where a single blinking represents a robot command. The command orders the robot to initiate or stops their locomotion for the hexapod robot. For the drone, a blink represents the takeoff or landing order. The blinking signals are obtained from the prefrontal and frontal regions of the head by EEG sensors. The signals are then filtered using temporal filters, with cutoff frequencies based on delta, theta, alpha, and beta waves. The filtered signals were labeled and used to train a classifier based on the multilayer perceptron (MLP) model. To generate the robot command, the proposal BCI used two models of MLP to ensure the classifier prediction. So, when the two classifiers make the same prediction, within a defined time interval, send the signal to the robot to start or stop its movement. The obtained results show that it is possible to get high precision to control the hexapod robot with a precision of 91.7% and an average of 81.4%.

Residue Number System-Based Solution for Reducing the Hardware Cost of a Convolutional Neural Network

Convolutional neural networks (CNNs) represent deep learning architectures that are currently used in a wide range of applications, including computer vision, speech recognition, time series analysis in finance, and many others. At the same time, CNNs are very demanding in terms of the hardware and time cost of a computing system, which considerably restricts their practical use, e.g., in embedded systems, real-time systems, and mobile volatile devices. The goal of this paper is to reduce the resources required to build and operate CNNs. To achieve this goal, a CNN architecture based on Residue Number System (RNS) and the new Chinese Remainder Theorem with fractions is proposed. The new architecture gives an efficient solution to the main problem of RNSs associated with restoring the number from its residues, which determines the main contribution to the CNN structure. In accordance with the results of hardware simulation on Kintex7 xc7k70tfbg484-2 FPGA, the use of RNS in the convolutional layer of a neural network reduces hardware cost by 32.6% compared to the traditional approach based on the binary number system. In addition, the use of the proposed hardware-software architecture reduces the average image recognition time by 37.06% compared to the software implementation.

Indoor Air Quality (IAQ) Sensing Device Prototype: Software-Hardware Architecture and Performance

In a green environment, air quality is among the prominent aspects to be considered to prevent pollution and in turn maintain safe air. Sick Building Syndrome (SBS) and Building Related Illness (BRI) are the problems that must be encountered by building occupants. Existing devices are quite expensive, and more attention is needed on monitoring air quality. Therefore, this paper discusses a software-hardware architecture proposal to overcome the existing IAQ data retrieval performance problem. The objective of this work is to analyse the most recent IAQ sensing device developed by the researcher, to evaluate the performance attribute, and to implement the software-hardware architecture based on the Performance-Driven Software Architecture Refactoring (PDSAR) method. In order to assess the overall performance of the architecture proposed, the prototype of the Indoor Air Quality (IAQ) sensing device was developed. The testing was performed during a period of six months. The results demonstrate that the average performance for 540 readings is 76.67%. This shows that the architecture can be accepted as a generic IAQ sensing device development structure. The device act as an alternative, cheaper and safer device which can be employed by all building owners in the future.

Improving the Scalability and Replicability of Embedded Systems Remote Laboratories Through a Cost-Effective Architecture

Online remote laboratories are a particularly promising tool for effective STEM education. They offer online universal access to different hardware devices in which students can experiment and can test and improve their knowledge. However, most of them have two significant limitations. First, given that most of them are developed as, or evolve from single-user proofs of concept, they have no scalability provisions other than full laboratory replication. And second, when this is done, cost efficiency is often neglected. This paper presents the requirements for the creation of a novel remote laboratory architecture focused on, but not limited to, embedded systems experimentation. An architecture, based on Redis (an open source, in-memory data structure store, which is often used as database, cache or message broker), a modular design, and hardware-sharing techniques, is proposed in order to achieve the combined requirements of high scalability and cost efficiency. This mixed hardware-software architecture serves as a basis for the development of remote laboratories, especially those focused on microcontroller-based systems experimentation and embedded devices experimentation. From a user perspective the architecture is web-based, and has provisions to be easily adaptable to different Learning Management Systems and different hardware embedded devices. A new microcontroller-oriented remote laboratory based on the architecture has been developed, with the aim of providing valid evaluation data, and has been used in a real environment. The architecture and the resulting remote laboratory have been compared with other state of the art remote laboratories and their architectures. Results suggest that the proposed architecture does indeed meet the main requirements, which are scalability through replicability and cost efficiency. Furthermore, similarly to previous architectures, it promotes usability, universal access, modularity and reliability.

A hardware-software architecture for computer vision systems

Computer vision systems are based on capture and processing of images, from which information for the application is extracted. This paper presents a hybrid platform which can be easily configured and efficiently used for a large number of applications in computer vision. It consists of a dedicated processor integrated to the capture element, connected to a general-purpose computer for the high-level application processing. The camera module consists of a DSP executing a middleware which provides all the basic functions, such as image capture, low-level image processing, control, and communication. Experiments showed that good performance is obtained with this platform, and the framework presented may simplify the development of computer vision systems. An application for the hardware-software architecture presented here would be the validation and prototyping of computer vision systems.

A Novel Packets Transmission Scheme Based on Software Defined Open Wireless Platform

The IEEE 802.11-2012, known as wireless-fidelity, offers a rich set of physical layer (PHY) features supported by advanced media access control (MAC) mechanisms. The use of off-the-shelf commercial 802.11 network interface card based (NIC-based) platforms constitutes an inexpensive prototyping approach based on reprogramming the cards software drivers. Besides, the commercial 802.11 NIC-based platform performs the functions of the MAC layer and PHY layer, respectively, by way of the firmware in the networking card and the software-driver running on the host computer, the former of which carries out the time-critical functions (Low MAC and PHY) while the latter serves the time-tolerant functions (High MAC). However, the maintenance of the cards produced in this way turns out quite troublesome. Therefore, based on a mixed real-time software-hardware architecture, i.e., a software defined open wireless platform, a novel packets transmission scheme is proposed in this paper in order to overcome the limitations of the NIC-based platform so that it becomes fully flexible and re-configurable. Furthermore, the novel scheme consists of three parts. First, a preliminary scheme is designed as follows: shift the hard-coded Low MAC and the PHY to the user space, then carry out the High MAC in the kernel space, and then combine the High MAC with the Low MAC via the Netlink. Second, based on preliminary scheme, the sending packets (i.e., the MAC frame) are reorganized in a lightweight way and the packets’ completeness and correctness are validated with experiments. Third, an M/M/1 model is exploited to create a packets transmission ring for optimizing the preliminary scheme. And the experimental results of the packets transmission concerned have proved the above-mentioned novel scheme outperforms the traditional one (i.e., the NIC-based platform) in terms of the system overhead and the number of packets sent in unit time.

A Hybrid Simulation Approach for Fast and Accurate Timing Analysis of Multi-Processor Platforms Considering Communication Resources Conflicts

In the early design phase of embedded systems, discrete-event simulation is extensively used to analyse time properties of hardware-software architectures. Improvement of simulation efficiency has become imperative for tackling the ever increasing complexity of multi-processor execution platforms. The fundamental limitation of current discrete-event simulators lies in the time-consuming context switching required in simulation of concurrent processes. In this paper, we present a new simulation approach that reduces the number of events managed by a simulator while preserving timing accuracy of hardware-software architecture models. The proposed simulation approach abstracts the simulated processes by an equivalent executable model which computes the synchronization instants with no involvement of the simulation kernel. To consider concurrent accesses to platform shared resources, a correction technique that adjusts the computed synchronization instants is proposed as well. The proposed simulation approach was experimentally validated with an industrial modeling and simulation framework and we estimated the potential benefits through various case studies. Compared to traditional lock-step simulation approaches, the proposed approach enables significant simulation speed-up with no loss of timing accuracy. A simulation speed-up by a factor of 14.5 was achieved with no loss of timing accuracy through experimentation with a system model made of 20 functions, two processors and shared communication resources. Application of the proposed approach to simulation of a communication receiver model led to a simulation speed-up by a factor of 4 with no loss of timing accuracy. The proposed simulation approach has potential to support automatic generation of efficient system models.

On the design of hardware-software architectures for frequent itemsets mining on data streams

Frequent Itemsets Mining has been applied in many data processing applications with remarkable results. Recently, data streams processing is gaining a lot of attention due to its practical applications. Data in data streams are transmitted at high rates and cannot be stored for offline processing making impractical to use traditional data mining approaches (such as Frequent Itemsets Mining) straightforwardly on data streams. In this paper, two single-pass parallel algorithms based on a tree data structure for Frequent Itemsets Mining on data streams are proposed. The presented algorithms employ Landmark and Sliding Window Models for windows handling. In the presented paper, as in other revised papers, if the number of frequent items on data streams is low then the proposed algorithms perform an exact mining process. On the contrary, if the number of frequent patterns is large the mining process is approximate with no false positives produced. Experiments conducted demonstrate that the presented algorithms outperform the processing time of the hardware architectures reported in the state-of-the-art.

Concept for Developing the Architecture of a Hardware-Software Package to Monitor Hydraulic Installation Status

This article is devoted to addressing the relevant problem of designing a hardware-software package to monitor hydraulic installations. The concept of the hardware-software package architecture provides for the integration of mathematical models of structures and structure foundations with data from information-diagnostic systems.

MLC NAND Flash memory: Aging effect and chip/channel emulation

This work presents an FPGA-based emulator that can be used for emulating NAND Flash memories, either at the chip or at the channel level, along with the effect of aging on their performance. The emulator is based on a reconfigurable hardware-software architecture, which enables accurate representation of various NAND Flash technologies, focusing especially on MLC cases. The presented architecture can be used for emulating memories at the chip and channel level, while the proposed hardware platform can be used as a valuable tool for developing and evaluating memory-related algorithms and techniques. In this paper, we analyze the architecture of the NAND Flash memory emulator and we present details about its internal functionality. Using experimental results, we demonstrate the high accuracy achieved when it is used to emulate specific MLC and TLC NAND Flash chips and we describe how this custom hardware can be used to emulate a complete NAND Flash channel, which consists of multiple NAND Flash chips that share a common data path and support the execution of pipelined commands.

Semi-virtual Plant for the Modelling, Control and Supervision of batch-processes. An example of a greenhouse irrigation system

In this work, a hardware-software architecture for control engineering education is proposed and a case of study regarding a greenhouse irrigation virtual process has been implemented. This specific process is especially useful for subjects related to industrial automation, since it is characterized by mixed dynamics (continuous/discrete) and can be controlled by a Programmable Logic Controller (PLC). In such a case, there is no need to access to the real process that is not always available for practical exercises. Moreover, the developed scheme eliminates the maintenance costs related to real process, being a very important advantage for academic institutions. The developed architecture for the industrial process simulation uses Labview, OPC, MODBUS, SolidWorks and Schneider Modicon M340 PLC for the control of virtual processes. The developed example is applied to the greenhouse fertirrigation control that enables the agricultural engineering students to design the irrigation control system. The control strategy is implemented on the PLC which interacts with the virtual process by carrying out the necessary actions in order to control the appropriate variables of the process.

Hardware-software collaboration for secure coexistence with kernel extensions

Our society is dependent upon computer systems that are the target of a never-ending siege against their resources. One powerful avenue for exploitation is the operating system kernel, which has complete control of a computer system's resources. The current methodology for kernel design, which involves loadable extensions from third parties, facilitates compromises. Most of these extensions are benign, but in general they pose a threat to system trustworthiness: they run as part of the kernel and some of them can be vulnerable or malicious. This situation is paradoxical from a security point of view: modern OSes depend, and must co-exist, with untrustworthy but needed extensions. Similarly, the immune system is continuously at war against various types of invaders and, through evolution, has developed highly successful defense mechanisms. Collaboration is one of these mechanisms, where many players throughout the body effectively communicate to share attack intelligence. Another mechanism is foreign body co-existence with its microbiota. Remarkably, these properties are not leveraged in kernel defense approaches. Security approaches at the OS and virtual machine layers do not cooperate with each other or with the hardware. This paper advocates a new paradigm for OS defense based on close collaboration between an OS and the hardware infrastructure, and describes a hardware-software architecture realizing this vision. It also discusses the architecture design at the OS and hardware levels, including experimental results from an emulator-based prototype, and aspects of an ongoing hardware implementation. The emulator-based proof-of-concept prototype, Ianus, uses Linux as the OS and the Bochs x86 emulator as the architecture layer. It successfully minimized kernel extensions interactions with the original kernel. Its security was evaluated with real rootkits and benign extensions. Ianus' performance was analyzed with system and CPU benchmarks and it caused a small overhead to the system (approximately 12%).