Paper] Lightweight Object Detection Model for a CMOS Image Sensor with Binary Feature Extraction

Binary features allow for the effective comparison, fast calculation, and compact storage in image matching and localization. Binary feature extraction algorithms, however, tend to have poor mirror invariance, and search algorithms that match binary features have a lower inlier ratio. To address these issues, we employ a scale space pyramid to simulate human eye imaging and then detect FAST (FAST feature detector) points at each level in the pyramid and calculate the FAST point feature's orientation with an image intensity centroid. We propose circumferential binary string mirror invariance rules and a circumferential binary feature (CBF) extraction algorithm to enhance the mirror invariance of binary features, and a fast calculate bitmap (FCBM) algorithm and bitmap local sensitive hash (BMLSH) to improve the inlier ratio of matching binary features. Experiments show that the CBF performs well in mirror invariance and has stronger adaptability and that BMLSH searches inliers more efficiently.

Object detection is a very well-known computer technology connected with computer vision, image processing, data collection and character recognition that focuses on detection of any object or its instances of a certain class (such as humans, flowers, animals, number plates, vehicles etc.) presented in the form of digital images and videos. There are various applications on object detection that have been well researched including face detection & recognition, character recognition and prediction, and number plate detection. Object detection and recognition are used in very vast cases and scenarios including retrieval, surveillance, detection of over speeding of vehicles and a lot more cases . In this research, various basic concepts used in object recognition and detection while making use of OpenCV library of python 3.8, increasing and improvising the efficiency & accuracy of object recognition and detection are presented

The CMOS (Complementary Metal Oxide Semiconductor) image sensor becomes increasingly competitive with respect to CCD (the charge-coupled device) except the noise performance. In order to analyze the noise sources of CMOS imagers and then find some methods to reduce noise level of CMOS imagers, a model of the CMOS imager sensor may be first set up. This paper introduces the method of modeling and simulation of CMOS imaging sensor based on SIMULINK. The CMOS imager sensor used in the simulation is LUPA4000 fabricated by Cypress Corporation. Based on this model, parameters of the sensor can be modified and simulation results are then showed clearly. The validity of the model is also confirmed in the last part of this paper. The method introduced in this paper is also suitable to establish the model of other CMOS imaging sensors.

Micro-Digital Sun Sensor is an attitude sensor which senses relative position of micro-satellites to the sun in space. It is composed of a solar cell power supply, a RF communication block and an imaging chip which is called APS+. The APS+integrates a CMOS Active Pixel Sensor (APS) of 512×512 pixels, a 12 bit Analogue to Digital Converter (ADC), digital Input and Output (I/O) circuits, timing signal generators and drivers, and digital signal processing circuits for centroid calculation. The paper describes the implementation of a prototype of the μDSS APS+ using a standard 0.18μm CMOS process. As a space application, it is particularly characterized by its low power consumption. The reduction of power consumption is mainly achieved by windowing, which is enabled by a specific active-pixel design in APS. The functions of the blocks in APS+ are tested. The test results of a test chip which contains 368×00368 pixel array will be discussed following in the paper.

The power consumption of the readout path in a CMOS image sensor is dominated by the Analog-to-digital converters (ADCs). The power consumption of an ADC is decided by the maximum noise it can tolerate. The increase in tolerable noise or reduction in resolution of the ADC decreases the power consumption. In this work, a charge based readout resulting in increased quantization voltage step compared to the conventional voltage readout for a given resolution is presented. The quantization steps and the reduction in power consumption are estimated from suitable device models. The estimated results show around 20% reduction in the power consumption of ADCs.

Neurocomputing for internet of things: Object recognition and detection strategy

For the forthcoming Internet of Things (IoT) era, it will be important to reduce the data output from sensors as well as their energy efficiency. Since conventional image sensor output data for photography are often redundant in AI applications, we propose a CMOS image sensor that can generate both RGB color images for humans and feature data for deep learning (DL). Use of feature data allows reducing the energy efficiency of the image classification system and saving storage space for imaging data. We performed experiments to demonstrate that the simulated feature data are suitable for use in image classification tasks. A five-layer convolutional neural network (CNN) classifier was trained and tested using the aggressively quantized feature data generated from a person dataset, where image classification accuracy was also improved when applying contrast enhancement. According to the experimental results, an accuracy of 95.9% was achieved using 1-bit feature data, resulting in 93.75% reduction in the amount of data compared to RGB color images.

Internet of Things (IoT) has become a fast growing research topic in recent years. Internet connected sensors and devices allow for the collection and processing of a wealth of data. This in conjunction with sensor fusion can provide greater accuracy in object recognition and detection surpassing what could be obtained by sensors operating independently. However these distributed sensors must often operate in environments with poor to no access to the internet which can greatly reduce their effectiveness. Additionally these sensors can be attached to highly dynamic platforms further complicating communication and data routing. One possible solution is to use the B.A.T.M.A.N. (Better Approach To Mobile Ad hoc Networking) routing protocol adapted for use with LoRa, a low power long range RF protocol, to route sensor data through other nodes in order to reach internet access points and allow these devices to interact with the cloud that would have otherwise been unable. Other adaptations to the algorithm will be investigated, such as including other sensors, like GPS and message signal strength to better predict route quality. This system shows promise to be an effective, fault-tolerant solution for this application.

Recently, many kinds of tremendous requirements are needed for smart Internet of Things (IoT). Among them, an intelligent and self-working CMOS image sensor (CIS) is a key component to satisfy the specifications of smart IoT. In this paper, therefore, an intelligent CIS with a deep learning algorithm is discussed. With a deep learning and a variable pixel recognition algorithm, an intelligent CIS has been implemented. Even though a low resolution image is obtained, a high resolution image can be taken by the proposed deep learning algorithm. As well as the intelligent CIS chip has been fabricated with a 90nm Samsung CIS technology, the power consumption is extremely low by about 1.0mW with a VGA graphic mode.

Internet of Things (IoT) is a rapidly growing segment and it is expected to have billions of connected devices by 2020. Cellular technology is a great foundation for IoT connectivity given its advantages of global reach, larger coverage than WiFi, reliability and security in using licensed spectrum. Extended Coverage Global System for Mobile (EC-GSM) and Narrow Band-LTE (NB-LTE) are gaining importance in 3GPP standard study item in framing the next communication support to expected huge number of low complex and low Data Rate IoT devices. These IoT devices are expected to have better battery life time i.e., more than 5 years. As EC-GSM has been considered for IoT with low Data-Rate applications, the potential techniques for power saving needs more study and research. This paper proposed a novel idea on decreasing the power consumption for an EC-GSM supported IoT device. The simulation results reveal considerable reduction in power consumption, i.e., around 40% for paging block decode over a duration of 3600 seconds in IoT devices in comparison with existing well accepted early page decode techniques.

To address the power constraints of the emerging Internet of Things (IoT) era, we propose a compression-efficient feature extraction method for a CMOS image sensor that can extract binary feature data. This sensor outputs six-channel binary feature data, comprising three channels of binarized luminance signals and three channels of horizontal edge signals, compressed via a run length encoding (RLE) method. This approach significantly reduces data transmission volume while maintaining image recognition accuracy. The simulation results obtained using a YOLOv7-based model designed for edge GPUs demonstrate that our approach achieves a large object recognition accuracy () of 60.7% on the COCO dataset while reducing the data size by 99.2% relative to conventional 8-bit RGB color images. Furthermore, the image classification results using MobileNetV3 tailored for mobile devices on the Visual Wake Words (VWW) dataset show that our approach reduces data size by 99.0% relative to conventional 8-bit RGB color images and achieves an image classification accuracy of 89.4%. These results are superior to the conventional trade-off between recognition accuracy and data size, thereby enabling the realization of low-power image recognition systems.

Automation and robotics continue to revolutionize industries, enhancing efficiency and productivity across various domains. In pursuit of this automation, the development of voice-controlled robots with object detection and picking capabilities represents a promising frontier. This project explores the convergence of hardware and software technologies to create a versatile robot that responds to voice commands, perceives its environment through object detection, and executes tasks such as object picking and placement. The hardware foundation of the project centres on the ESP32-S3 microcontroller, integrating sensors, motors, and a camera module. The software stack encompasses speech recognition for natural language voice commands and object detection powered by deep learning models. The project aims to design an intuitive user interface for remote control, providing users with the ability to command the robot seamlessly. Through a comprehensive literature review, we delve into the evolution of speech recognition, object detection, and human-robot interaction, shedding light on the theoretical and practical aspects of the project. Real-world applications across industries underscore the project's potential, from warehouse automation to healthcare assistance. However, challenges such as real-time processing, accurate object recognition, and human-robot interaction complexity are acknowledged. The project's future directions emphasize the need for ongoing research to refine the technology's capabilities and overcome existing limitations. This voice-controlled robot with object detection and picking project presents a compelling fusion of cutting-edge technology and practical utility, contributing to the ever-expanding field of robotics automation.

Advanced technologies and algorithms such as the Internet of Things (IoT), computer vision (CV), and deep learning are widely used in the healthcare industry to enhance global med-ical care. Internet of Things (IoT) has the potential to be limitless due to increased network agility, integrated artificial intelligence (AI), and the ability to deploy and automate systems. Embedded systems playa vital role in IoT due to real-time computing, low power consumption, and low maintenance cost. Object detection is a computer vision technique that aims to process and identify certain objects such as people, cars, animals, or buildings in digital images or videos. The goal of object detection is to develop computational models for computer vision applications. Recently, rapid advancement in deep learning techniques accelerated the momentum of object detection. In this study, we proposed a mechanism to perform object detection based on deep learning techniques in resource-constrained IoT devices. Due to limited computational powers in embedded systems, the performance of deep learning algorithms is not good enough. To achieve this, we compressed the video using a codec and streamed it to the amazon cloud for object detection. Video codec was used to uncompress the video in its original format so that there is no loss of video quality. A pre-trained YOLO model was deployed for object detection in medical images. The output is sent to the client using a lightweight protocol for data communication. Results indicate that the proposed mechanism worked well in a resource-constrained environment without compromising accuracy and time.

The integration of Internet of Things (IoT) and Artificial Intelligence (AI) brings us AIoT that delivers the capabilities of object detection, device localization, object tracking and re-identification on moving IoT devices to control robots/drones for smart human-machine interactions. Among the tasks, efficient objection detection plays an importance role since it acts as the foundation of many other vision-based IoT applications. One main challenge is to locate target objects fast and accurate. The paper presents the CAMDet technology that utilizes Class Activation Map (CAM) to reduce the convolution blocks and the enormous candidate bounding boxes in the detection-head stage. We have designed CAMDet and integrate it with other backbone networks. CAMDet is shown to be 2.1-2.7 times faster than the popular Tiny-YOLO/SSD methods in non-crowded scenarios when using the same backbone and feature pyramid structure. The performance study shows that our proposed methods are very attractive for real time object detection on moving IoT devices.

Metal Oxide Thin Film Transistors (MO TFTs) have garnered considerable interest in emerging Internet of Things (IoT) fields such as wearable electronics, displays, Radio Frequency Identification (RFID), and biomedical monitoring, owing to their flexibility and transparency. However, limitations in channel materials make MO TFT-based circuits unipolar. Unipolar circuits often exhibit elevated short-circuit power consumption, which restricts the development of MO TFTs in the IoT sector. This paper introduces a Capacitively Coupled Near-Threshold Biasing (CCNB) technique that leverages the unique Capacitance-Voltage (C-V) characteristics of MO TFTs to bias devices in the near-threshold region, achieving nearly a 95% reduction in power consumption compared to traditional designs with the device coupling ratio (channel capacitance/overlap capacitance) at 40. Furthermore, considering the significance of clock signals in IoT applications, we have also developed a low-power full-swing Ring Oscillator (RO) based on our CCNB technique, resulting in a 90% reduction in power consumption and a nearly 70% reduction in PDP compared to conventional low-power designs.