Camera Module Research Articles

Facial Emotional Recognition using Raspberry Pi

This facial emotion recognition project utilizes a Raspberry Pi for real-time analysis of facial expressions. Using a camera module, the system captures facial images, and a pre-trained deep learning model processes the data to classify emotions such as happiness, sadness, anger, etc. The Raspberry Pi's GPIO pins are employed to trigger specific actions or display emotion labels. This compact and efficient system demonstrates the integration of edge computing and machine learning for emotion analysis in a portable setup. Keywords- : Face, Expression, Raspberry Pi, Recognition, Feature Extraction.

A Sequential-Multi-Decision Scheme for WiFi Localization Using Vision-Based Refinement

Currently, most mobile devices have WIFI and camera modules to locate their position. However, there are two main challenges in large, highly similar indoor environments (localization accuracy and localization time). Aiming to balance these problems, we propose a sequential-multi-decision integrated system that combines WIFI and vision to acquire users' locations. This system has two phases: sequential fusion localization and adaptive multi-decision fusion localization. The former employs WIFI-based localization first, then image-based localization and fusion localization are used within the constraints of WIFI-based localization. In the WIFI-based localization phase, the gaussian process regression (GPR) model is used to construct a WIFI indoor map. Subsequently, we propose to apply the hybrid whale optimization algorithm (HWOA) to WIFI-based localization to improve its accuracy and stability. The latter uses an adaptive multi-decision fusion mechanism that integrates WIFI-based localization, image-based localization, and fusion localization to obtain the users' location finally. The experiments show the effectiveness of HWOA applied to WIFI-based localization. We also experimentally evaluate the proposed fusion algorithm with other state-of-the-art fusion algorithms (e.g., accuracy and time) in a real environment (an area larger than 10,000 <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$m^{2}$</tex-math></inline-formula> ). The experimental results show that the proposed fusion system is competitive.

Magnetically Driven Biopsy Capsule Robot with Spring Mechanism.

In recent years, capsule endoscopes (CEs) have appeared as an advanced technology for the diagnosis of gastrointestinal diseases. However, only capturing the images limits the advanced diagnostic procedures and so on in CE's applications. Herein, considering other extended functions like tissue sampling, a novel wireless biopsy CE has been presented employing active locomotion. Two permanent magnets (PMs) have been placed into the robots, which control the actuation of the capsule robot (CR) and biopsy mechanism by employing an external electromagnetic actuation (EMA) system. A spring has been attached to the biopsy mechanism to retract the biopsy tool after tissue collection. A camera module has also been attached to the front side of the CR to detect the target point and observe the biopsy process on the lesion. A prototype of CR was fabricated with a diameter of 12 mm and a length of 32 mm. A spring mechanism with a biopsy needle was placed inside the CR and sprang out around 5 mm. An in vitro experiment was conducted, which demonstrated the precise control translation (2 mm/s and 3 mm/s in the x and y directions, respectively) and desired extrusion of the biopsy mechanism (~5 mm) for sampling the tissue. A needle-based biopsy capsule robot (NBBCR) has been designed to perform the desired controlled locomotion and biopsy function by external force. This proposed active locomoted untethered NBBCR can be wirelessly controlled to perform extended function precisely, advancing the intestinal CE technique for clinical applications.

Flow analysis for lateral synthetic jets to remove contaminants from the outer surface of an automobile camera module

A test facility was constructed to investigate the flow characteristics of lateral synthetic jets generated by an actuator composed of a cavity, an orifice and a vibrating piezoelectric disc. Subsequently, 3D unsteady numerical simulations were conducted to analyze the formation of the synthetic jets and their dependence on the height of the orifice exit from the outer surface of the cavity. The time-averaged axial velocity was measured and used to validate the numerical results, and the maximum velocity reached approximately 8 m/s. In this work, it was found that the interaction of the vortex tube generated at the edge of the orifice exit and the outer surface of the cavity has a significant impact on the axial velocity distribution and characteristics of the lateral synthetic jets. Based on the experimental data and the numerical results, an attempt has been made to understand the formation of the synthetic jets and their interaction with the outer surface.

Development of an Autonomous Component Testing System with Reliability Improvement Using Computer Vision and Machine Learning

This study evaluated computer vision-based models, including Histogram Analysis, Logistic Regression, Sift-SVM, and Deep learning models, in an autonomous testing system developed for smartphone camera modules. System performance was assessed in a practical factory setting with workers operating the system, and metrics such as processing time, sensitivity, specificity, accuracy, and defect rate were evaluated. Based on the results, the Sift-SVM model demonstrated the greatest potential for enhancing the reliability of the system with a processing time of 0.01578 seconds, a sensitivity of 99.811%, and a reduction in the failure rate to 1888 PPM. The study findings suggest that Sift-SVM has the potential to be practically applied in the industry, thus improving the speed and accuracy of automatic defect detection in manufacturing and reducing the defect rate.

Solar position detection method by bionic polarized light compass.

To address the needs of polarized light navigation for accurate position information of feature points in the sky, an accurate solar position detection method based on an all-sky polarization pattern imaging system is proposed. Unlike the traditional spot-based solar position detection method, this method uses the polarization information inherent in the atmosphere to accurately measure solar position. This approach is characterized by simple detection, high accuracy, and wide application range. The optical acquisition system is composed of three miniature large-field camera modules and polarizers, which enables a more compact structure, smaller size, and lesser height. Based on this principle, the solar position solution algorithm was simulated and then verified in various weather environments using the optical acquisition system built as part of this study. Solar position was detected at different moments on the same day in clear weather, and the accuracy of the measured solar altitude and azimuth angles was 0.024° and 0.03°, respectively. The accuracy of the measured solar altitude and azimuth angles was 0.08° and 0.05°, respectively, when the sun was shielded by high-rise buildings and 0.3° and 0.1° when the sun was shielded by branches and tree leaves. Aerosol concentrations exceeding a certain amount destroyed the Rayleigh distribution pattern of polarized light, thus affecting solar position detection accuracy. It is concluded that this novel detection method can not only meet the needs of polarized light navigation for solar position, but also provide a new exploration idea for enthusiasts who are eager to explore the mysteries of the universe.

Designing hardware for a robust high-speed cryptographic key generator based on multiple chaotic systems and its FPGA implementation for real-time video encryption

Recent advancements in communication technologies have highlighted the pivotal role of information security for all individuals and entities. In response, researchers are increasingly focusing on cryptographic solutions to ensure the reliability of confidential information. Recognizing the superiority of chaotic systems preference as entropy source of cryptographic systems, this paper proposes a novel true random number generator (TRNG) design by combining four different chaotic systems outputs, tailored for real-time video encryption application. These chaotic systems are continuous-time Lorenz and fractional-order Chen-Lee systems, as well as discrete-time Logistic and Tent maps. This study generates true random bit (TRB) sequences at a high bit rate (25 Mbps) through the hardware implementations of four distinct chaotic systems to have the best statistical randomness in the resulting output. Then, the cryptographic true random key bits (8-bit at 25 MHz frequency) are employed in the post-processing with real-time video data by using the XOR operation, a fundamental post-processing algorithm. The real-time video encryption application is executed on an experimental assembly, composed of a Field Programmable Gate Array (FPGA) development kit, an OV7670 camera module, a VGA monitor, and prototype circuit boards for the chaotic systems. To evaluate the effectiveness of the proposed encryption system, several security assessments are conducted. These include NIST SP 800 − 22 statistical tests, FIPS 140-1 standards, chi-square tests, histogram and correlation analysis, and NPCR and UACI differential attack resilience tests. Consequently, the findings suggest that the presented real-time embedded cryptosystem is robust and suitable for secure communications, particularly in the realm of video transmission.

Energy-efficient image compression algorithm for wireless sensor networks in structural health monitoring applications

In Wireless Sensor Networks (WSN), transmitting an uncompressed image consumes more energy than a compressed image, and it is, therefore, the prime requirement to establish energy-aware compression methods to extend the life of the sensor node, and ultimately the network as a whole. This work suggests an image compression algorithm with a low degree of complexity for WSNs in structural health monitoring applications. This algorithm represents a pruning approach to a Discrete Cosine Transform approximation transform in which the transformation matrix is modified to reduce the series of steps and the compression ratio achieved is better compared to the actual image which makes it easy for the data storage. Because of the reduction in the number of data bits, it enhances the lifetime of the network by reducing the number of node failures caused by resource scarcity. The implementation is tested by capturing real-time images of concrete walls in buildings using Raspberry Pi3B + WSN gateway fitted with camera modules. The scheme is also investigated in terms of a variety of parameters like Peak Signal to Noise Ratio, Mean Square Error, Structural SIMilarity Index and Compression Ratio. This technique achieves the best arbitration of energy consumption and image quality.

Weed detection and removal using robotic system

Weeding is one of the most significant practices in agricultural production. Weeds are unwanted plants that grow along with the crops and compete with the crops for space, light, water, and soil nutrients. Weeds propagate themselves either through seeding or creeping rootstalk and decrease yields, increase production costs, interfere with the harvest, and lower the product quality. The use of herbicides reduces labor requirements for weed control by up to 60 percent but affects environmental quality and can be toxic to a wide range of organisms. Hence it is necessary to develop an automated system to identify and remove weeds from the vegetable fields. The objective of the proposed work is to develop a mobility level tracked bot that identifies the weeds and removes them with the help of a robotic end effector and to develop a machine learning model to identify the weeds. This functional module will be processed in a Raspberry Pi processor and by using a Raspberry Pi camera module the bot will detect the weeds in vegetable fields. We performed weed detection with different machine learning models like Haar cascade, YOLOv5, and CNN. To evaluate the performance of the machine learning models used, the performance metrics accuracy, precision, recall, and F-measure are estimated and it has been found that CNN has better accuracy, precision, and recall as compared to YOLOv5 and Haar cascade. CNN has the highest F-measure among the three algorithms at 98%. The weed removal is done using a robotic end-effector which is controlled by the Arduino UNO based on the signal from Raspberry Pi.

Impact of Blemish Artefacts on Object Detection Models in Autonomous Driving: A Study on Camera Module Manufacturing Defects

With the rise of autonomous driving technology, concerns over the safety of AI-based systems have increased, particularly regarding their effect on human life and infrastructure. The complexity and unpredictable nature of AI-enabled software functions have disrupted traditional safety evaluation methods. In such a scenario, the quality control methods used at the manufacturing environments of sensors used for deploying AI-based perception functions need to be coupled with the downstream functions. This study focuses on investigating the impact of a specific production-related defect called blemish, which is caused by debris on image sensors of camera modules. The research aims to answer two fundamental questions: 1) How can blemish artifacts be mathematically modeled, and what is the process for developing a suitable dataset that incorporates this defect? 2) How can the influence of blemish artifacts on performance of object detection models be traced back to define quality requirements of the camera modules? This work takes a novel perspective by shifting the focus from improving model resilience to improving hardware quality camera modules by studying the effects of manufacturing defects on downstream functions. This research seeks to establish a framework to derive quality targets for front camera module manufacturing which can be extended to other sensors for similar applications.

Laser Fencing Surveillance System With Email Alert

This work presents an antitheft alert system that integrates multiple technologies to offer a high level of detection, monitoring, and barrier protection through cost-effective methods and equipment. It implements a virtual LASER fence framework, enabling individuals to establish real-time surveillance at locations such as homes, farms, or any other area of interest. The proposed system detects objects that cross the vision line between the laser source and optical detector. When the laser beam is interrupted, alert data—including an instant notification via email and recorded footage of the intrusion—is sent to the security provider’s device using internet services like Gmail. The system is powered by a Raspberry Pi as the central processor, along with components like the Pi camera and laser modules. This solution addresses issues found in existing frameworks and ensures non-destructive consequences for the intruder.

Smart Trash Bin to Prevent Animal Disturbance Using Raspberry Pi and Deep Learning

A trash can is an indispensable component of human daily life. It is ubiquitous and extremely useful for keeping the city clean. However, littering is a frequent and widespread problem. Not only do humans litter, but so do animals, particularly monkeys. Animals are constantly searching for food, and when we discard our food waste, they are attracted and leave their natural habitats. Following that, the user may forget to lock the trash can, which may end up in the hands of the animal. With the aid of a deep learning system that detects and identifies human subjects, a smart trash bin with an autonomous locking mechanism will be used in this project to reduce animal disturbance. In this paper, Raspberry-Pi 4 module is being used. Moreover, this article describes a human subject identification method and its real-time displacement using the OpenCV library of programming roles, which is primarily aimed at Raspberry Pi with camera module. The smart trash bin system helps to reduce animal disturbance and minimise littering.

Maturity stages detection prototype device for classifying custard apple (Annona squamosa L) fruit using image processing approach

Custard apple (Annona squamosa L.) is commercially grown for its sweetness, delicacy, and nutraceutical benefits across the world. Fruit must be harvested from trees before the climacteric stage begins for stabilizing the maturation process and perishability, which otherwise restricts longer shelf–life. Therefore, maturity–based grading or classifying custard apple fruit in accordance with their ripening is required for commercial marketing and industrial storage purposes. This is a crucial task that agriculturists and the food processing industries must have to perform manually, which is ineffective and susceptible to human errors. In this research, we have proposed an image processing–based maturity stages detection prototype device for fruit classification of custard apple (Cv. Balanagar) based on changes in their physical, chemical and imaging features (skin color of fruit areoles). The newly developed handheld device comprises of a Raspberry Pi board, camera module, LCD touch screen, and image processing algorithms to identify five essential maturity stages i.e. 0 %, 25 %, 50 %, 75 % and 100 % areoles opening of custard apple. The images were pre–processed and segmented, after which imaging aspects of custard apple fruit including relative R, G and B components were extracted and their relationship with chemical properties such as TSS was established. For classification of captured images, K–means clustering and SVM algorithms were performed on Raspberry Pi Platform with model training and classifier codes in Python (3.9.6) to get desired results. The statistical analysis revealed a substantial difference between the maturity stages in terms of R, G and TSS. Overall, 100 % accuracy was accomplished with respect to the grading of custard apple fruit at varied maturity. This could be a low–cost device useful for farm–level custard apple fruit classification, minimising economic losses caused by faulty supply entering the marketing chain. With slight modifications in algorithms, this device could also be used to assess the maturity of other fruit crops. It could be also an alternative to several expensive approaches being used for automating the maturity grading process that is not feasible, particularly for farmers working in field conditions.

Silicon photomultipliers for the SST camera of the Cherenkov Telescope Array

The Cherenkov Telescope Array Observatory (CTAO) will be the major global observatory for gamma-ray astronomy over the next decade and beyond. It will consist of two arrays of telescopes of different sizes, one for each hemisphere, and will be sensitive to gamma rays in the energy range from a few tens of GeV to hundreds of TeV. The Small-Sized Telescopes (SSTs) are a crucial component of the southern array, as they will extend the sensitivity of the observatory to the highest energies. Their focal plane will be equipped with 2048 Silicon Photomultiplier (SiPM) pixels, each one read independently by a state-of-the-art full waveform sampling readout. These solid-state sensors offer advantages over the traditional photomultiplier tubes, such as lower operating voltage, higher photon detection efficiency, and tolerance to bright illumination. In particular, they are the best choice for a small and compact camera such as the SST one. After a detailed comparative study, LVR3-type SiPMs from Hamamatsu Photonics were selected, with an active area of 6 mm × 6 mm, a microcell of 50 μm and without a protective coating, for optimum performance. The sensors demonstrated to have a higher photon detection efficiency and a lower crosstalk compared to the competitors, alongside a low dark count rate. In this contribution, we present the selection process and the latest measurements performed on the SiPMs mounted on the SST Camera module.

ADEROS: Artificial Intelligence-Based Detection System of Critical Events for Road Security

The deployment of artificial intelligence (AI) in Intelligent Transportation Systems (ITS), especially in the field of Intelligent Transportation Cyber-Physical Systems (ITCPS) has a strong potential to achieve higher efficiency, reliability, and increased safety in both transportation and traffic. This work focuses on the real-world implementation of ITCPS, in which structure and elements in combination with advanced image processing methods increase safety and fluidity of road traffic at crossroads and railway crossings. In this work, we present a novel system called Artificial Intelligence-based Detection System for Road Security (ADEROS), which combines elements of CPS systems, object detection, and classification, computer vision (CV) which analyzes vehicle trajectory tracking, vehicle and pedestrian presence, light signaling systems, railway barriers at railway crossings, and railway warnings. The presented system is based on a camera module that is suitably positioned to capture the entire scene. The module uses graphics processing units (GPU) for accelerated image processing techniques and the YOLOv4 deep neural network model to detect traffic participants and then dangerous situations in various crossroads and railway crossings. Our improved unique detector can distinguish between individual types of road users and the status of several safety devices at crossroads and railway crossings (for example, the state of traffic lights (TL) or rail barriers). Furthermore, we present experimental implementation details of the ADEROS system, which includes a central server web interface for live traffic situation monitoring, various communication channels for the camera module, and a central server based on.NET core, Cassandra DB, and different security protocols. All data from risky situations are evaluated and transferred to the central server securely without human intervention. The central server aggregates and archives all risky situational data from connected cameras. Finally, we present our experimental results from a real-world pilot project that consists of a camera module prototype deployed in a real crossroad and an operational central web server.

Design and development of latex cup collector robot

Global rubber production has declined over the past few years as a result of a number of variables, including both environmental and human factors. In this paper, an automated robot prototype that collects latex cups was developed to improve global rubber production by assisting workers in rubber plantations. The robot was built on a mobile platform with a motorized Five Degrees of Freedom (DoF) robotic arm, Ackermann steering, double wishbone suspension, rear-wheel drive and a latex storage tank to store the latex collected from the trees. The robot was equipped with ultrasonic sensors and a camera module to locate latex cups and rubber trees so that it could move and perform tasks accurately. In addition, encoder sensor modules were used to improve the accuracy of the movements by measuring the rotational speed of the motors. The yield strength of the PLA plastic material used in developing the latex cup collector robot is 7.00 x 107N/m2 and the maximum stress of the critical parts should not exceed the maximum yield strength of the PLA plastic. In mobility test, the robot is able to travel through obstacles of 10mm and 15mm height. The topple angle of the robot in the balancing test is greater than 37°, ensuring the stability of the robot.

Adhesive ensures automotive camera modules are assembled precisely and efficiently

As consumers expect ever more sophisticated assistance systems and increasingly depend on cameras and sensors in the context of autonomous driving, the performance and reliability of these components have become critical to road users’ safety and comfort. As a follow-up to previous coverage, this focus article looks in greater detail at how the use of a single-step cure active alignment adhesive – developed by Henkel Adhesive Technologies – is helping manufacturers to produce these components not only precisely, but quickly and efficiently, whilst also reducing carbon dioxide emissions.

Fully Automated Paper Document Sorting Robot Design

A fully automated paper document sorting robot was developed in this project. This robot classifies documents efficiently and accurately. The objective of this project was to improve the efficiency of classifying or sorting paper documents, reduce costs, and save time. The robot can classify documents according to user-defined rules, such as keywords, dates, serial numbers, bar codes, and the meaning of paragraphs. Since it can classify or sort documents intelligently, it can complete large-scale document classification quickly. The robot is constructed using an aluminum profile to create a box-type truss gantry structure frame. It was built on the LubanCat 4 motherboard and controlled through Python language programming. Driven by a stepper motor to move the manipulator. The camera module is combined with an artificial intelligence algorithm to recognize paper in real time, and the text is recognized after taking pictures of the paper. The sorting function is performed by several sensors. In addition, a web-based human-computer interaction platform was developed using the Flask web framework in Python. Users could access this platform in a variety of ways, allowing them to easily and swiftly configure parameters and send operational instructions to perform various functions.

Advanced motion tracking for interactive mass spectrometry imaging (IMSI)

Mass spectrometry imaging (MSI) is a powerful analytical technique for the two-dimensional (2D) localization of chemicals on surfaces. Conventional MSI experiments require to predefine the surface of interest based on photographic or microscopic images. Typically, these boundaries can no longer be changed or adjusted once the experiment has been started. In terms of a more interactive approach we recently developed a pen-like ionization interface which is directly connected to the mass spectrometer. The device allows the user to ionize chemicals by desorption electrospray ionization (DESI) and to freely move the interface over a surface of interest. A mini camera, which is mounted on the tip of the pen, magnifies the desorption area and enables a simple positioning of the pen. The combination of optical data from the camera module and chemical data obtained by mass analysis facilitates a novel type of imaging experiment: interactive mass spectrometry imaging (IMSI). For this application, we present a novel approach for a robust, optical flow-based motion detection. While the live video stream from the camera is used to track the pen's motion across the surface a post-acquisition algorithm correlates the coordinates of the pen trajectory with respective mass spectra obtained from a simultaneous mass spectrometric data acquisition. This algorithm is no longer dependent on a single, manually applied optical marker on the sample surface, which has to be visible on all video frames throughout the analysis. The advanced DESI-IMSI method was successfully tested on inkjet-printed letters as well as mouse brain tissue samples. Validation of the results was done by comparing DESI-IMSI with standard DESI-MSI data.

A Rapid Screening Platform for Simultaneous Evaluation of Biodegradation and Therapeutic Release of an Ocular Hydrogel.

This study attempts to address the challenge of accurately measuring the degradation of biodegradable hydrogels, which are frequently employed in drug delivery for controlled and sustained release. The traditional method utilizes a mass-loss approach, which is cumbersome and time consuming. The aim of this study was to develop an innovative screening platform using a millifluidic device coupled with automated image analysis to measure the degradation of Gelatin methacrylate (GelMA) and the subsequent release of an entrapped wetting agent, polyvinyl alcohol (PVA). Gel samples were placed within circular wells on a custom millifluidic chip and stained with a red dye for enhanced visualization. A camera module captured time-lapse images of the gels throughout their degradation. An image-analysis algorithm was used to translate the image data into degradation rates. Simultaneously, the eluate from the chip was collected to quantify the amount of GelMA degraded and PVA released at various time points. The visual method was validated by comparing it with the mass-loss approach (R = 0.91), as well as the amount of GelMA eluted (R = 0.97). The degradation of the GelMA gels was also facilitated with matrix metalloproteinases 9. Notably, as the gels degraded, there was an increase in the amount of PVA released. Overall, these results support the use of the screening platform to assess hydrogel degradation and the subsequent release of entrapped therapeutic compounds.