Artificial Compound Eye Research Articles

Optical fibre based artificial compound eyes for direct static imaging and ultrafast motion detection

Natural selection has driven arthropods to evolve fantastic natural compound eyes (NCEs) with a unique anatomical structure, providing a promising blueprint for artificial compound eyes (ACEs) to achieve static and dynamic perceptions in complex environments. Specifically, each NCE utilises an array of ommatidia, the imaging units, distributed on a curved surface to enable abundant merits. This has inspired the development of many ACEs using various microlens arrays, but the reported ACEs have limited performances in static imaging and motion detection. Particularly, it is challenging to mimic the apposition modality to effectively transmit light rays collected by many microlenses on a curved surface to a flat imaging sensor chip while preserving their spatial relationships without interference. In this study, we integrate 271 lensed polymer optical fibres into a dome-like structure to faithfully mimic the structure of NCE. Our ACE has several parameters comparable to the NCEs: 271 ommatidia versus 272 for bark beetles, and 180o field of view (FOV) versus 150–180o FOV for most arthropods. In addition, our ACE outperforms the typical NCEs by ~100 times in dynamic response: 31.3 kHz versus 205 Hz for Glossina morsitans. Compared with other reported ACEs, our ACE enables real-time, 180o panoramic direct imaging and depth estimation within its nearly infinite depth of field. Moreover, our ACE can respond to an angular motion up to 5.6×106 deg/s with the ability to identify translation and rotation, making it suitable for applications to capture high-speed objects, such as surveillance, unmanned aerial/ground vehicles, and virtual reality.

All-in-Focus Three-Dimensional Reconstruction Based on Edge Matching for Artificial Compound Eye

An artificial compound eye consists of multiple apertures that allow for a large field of view (FOV) while maintaining a small size. Each aperture captures a sub-image, and multiple sub-images are needed to reconstruct the full FOV. The reconstruction process is depth-related due to the parallax between adjacent apertures. This paper presents an all-in-focus 3D reconstruction method for a specific type of artificial compound eye called the electronic cluster eye (eCley). The proposed method uses edge matching to address the edge blur and large textureless areas existing in the sub-images. First, edges are extracted from each sub-image, and then a matching operator is applied to match the edges based on their shape context and intensity. This produces a sparse matching result that is then propagated to the whole image. Next, a depth consistency check and refinement method is performed to refine the depth of all sub-images. Finally, the sub-images and depth maps are merged to produce the final all-in-focus image and depth map. The experimental results and comparative analysis demonstrate the effectiveness of the proposed method.

Stereoscopic artificial compound eyes for spatiotemporal perception in three-dimensional space.

Arthropods' eyes are effective biological vision systems for object tracking and wide field of view because of their structural uniqueness; however, unlike mammalian eyes, they can hardly acquire the depth information of a static object because of their monocular cues. Therefore, most arthropods rely on motion parallax to track the object in three-dimensional (3D) space. Uniquely, the praying mantis (Mantodea) uses both compound structured eyes and a form of stereopsis and is capable of achieving object recognition in 3D space. Here, by mimicking the vision system of the praying mantis using stereoscopically coupled artificial compound eyes, we demonstrated spatiotemporal object sensing and tracking in 3D space with a wide field of view. Furthermore, to achieve a fast response with minimal latency, data storage/transportation, and power consumption, we processed the visual information at the edge of the system using a synaptic device and a federated split learning algorithm. The designed and fabricated stereoscopic artificial compound eye provides energy-efficient and accurate spatiotemporal object sensing and optical flow tracking. It exhibits a root mean square error of 0.3 centimeter, consuming only approximately 4 millijoules for sensing and tracking. These results are more than 400 times lower than conventional complementary metal-oxide semiconductor-based imaging systems. Our biomimetic imager shows the potential of integrating nature's unique design using hardware and software codesigned technology toward capabilities of edge computing and sensing.

Artificial Compound Eye with Tunable Properties for Enhancement in Fluorescence Imaging and Raman Detection

AbstractArtificial compound eye (CE) draws inspiration from nature, offering advanced imaging capabilities and an expansive field of view. In this work, an innovative technique is developed for the creation of CE with tunable dimensions. A solution‐based process is employed that involves in situ polymerization of surface nanodroplets prior to soft lithography to produce CE consisting of millions of ommatidia. The fabricated CE comprised of a densely arranged array of microwells, each with a base radius of 5 µm. Situated on a millimeter‐sized spherical dome, the CE can be tailored to arbitrary dimensions, enhancing its adaptability with a wide angular field of view up to 118°. The CE is used to enhance signal detection in fluorescent compounds, reaching a detection limit of 107 times lower concentration than that without using CEs in bulk solution. The signal enhancement capabilities are further utilized for surface‐enhanced Raman spectroscopy by using a portable handheld device, with an enhancement factor of 2. The fabrication technique underscores the advantages of the approach in simplicity, reproducibility, and efficiency in creating CE. The potential applications of CE may be extended to various domains, such as optical sensing, light‐dependent signal enhancement, motion perception, and medical endoscopy.

Measurement Methods for Droplet Adhesion Characteristics and Micrometer-Scale Quantification of Contact Angle on Superhydrophobic Surfaces: Challenges and Opportunities.

Inspired by nature, superhydrophobic surfaces have been widely studied. Usually the wettability of a superhydrophobic surface is quantified by the macroscopic contact angle. However, this method has various limitations, especially for precision micro devices with superhydrophobic surfaces, such as biomimetic artificial compound eyes and biomimetic water strider robots. These precision micro devices with superhydrophobic surfaces proposed a higher demand for the quantification of contact angles, requiring contact angle quantification technology to have micrometer-scale measurement capabilities. In this review, it is proposed to achieve micrometer-scale quantification of superhydrophobic surface contact angles through droplet adhesion characteristics (adhesion force and contact radius). Existing contact angle quantification techniques and droplet characteristics' measurement methods were described in detail. The advancement of micrometer-scale quantification technology for the contact angle of superhydrophobic surfaces will enhance our understanding of superhydrophobic surfaces.

Compact biologically inspired camera with computational compound eye

Abstract The growing interests have been witnessed in the evolution and improvement of artificial compound eyes (CE) inspired by arthropods. However, the existing CE cameras are suffering from a defocusing problem due to the incompatibility with commercial CMOS cameras. Inspired by the CEs of South American Shrimps, we report a compact biologically inspired camera that enables wide-field-of-view (FOV), high-resolution imaging and sensitive 3D moving trajectory reconstruction. To overcome the defocusing problem, a deep learning architecture with distance regulation is proposed to achieve wide-range-clear imaging, without any hardware or complex front-end design, which greatly reduces system complexity and size. The architecture is composed of a variant of Unet and Pyramid-multi-scale attention, with designed short, middle and long distance regulation. Compared to the current competitive well-known models, our method is at least 2 dB ahead. Here we describe the high-resolution computational-CE camera with 271 ommatidia, with a weight of 5.4 g an area of 3 × 3 cm2 and 5-mm thickness, which achieves compatibility and integration of CE with commercial CMOS. The experimental result illustrates this computational-CE camera has competitive advantages in enhanced resolution and sensitive 3D live moving trajectory reconstruction. The compact camera has promising applications in nano-optics fields such as medical endoscopy, panoramic imaging and vision robotics.

Femtosecond Laser Microfabrication of Artificial Compound Eyes

Over millions of years of evolution, arthropods have intricately developed and fine-tuned their highly sophisticated compound eye visual systems, serving as a valuable source of inspiration for human emulation and tracking. Femtosecond laser processing technology has attracted attention for its excellent precision, programmable design capabilities, and advanced three-dimensional processing characteristics, especially in the production of artificial bionic compound eye structures, showing unparalleled advantages. This comprehensive review initiates with a succinct introduction to the operational principles of biological compound eyes, providing essential context for the design of biomimetic counterparts. It subsequently offers a concise overview of crucial manufacturing methods for biomimetic compound eye structures. In addition, the application of femtosecond laser technology in the production of biomimetic compound eyes is also briefly introduced. The review concludes by highlighting the current challenges and presenting a forward-looking perspective on the future of this evolving field.

Miniature bioinspired artificial compound eyes: microfabrication technologies, photodetection and applications.

As an outstanding visual system for insects and crustaceans to cope with the challenges of survival, compound eye has many unique advantages, such as wide field of view, rapid response, infinite depth of field, low aberration and fast motion capture. However, the complex composition of their optical systems also presents significant challenges for manufacturing. With the continuous development of advanced materials, complex 3D manufacturing technologies and flexible electronic detectors, various ingenious and sophisticated compound eye imaging systems have been developed. This paper provides a comprehensive review on the microfabrication technologies, photoelectric detection and functional applications of miniature artificial compound eyes. Firstly, a brief introduction to the types and structural composition of compound eyes in the natural world is provided. Secondly, the 3D forming manufacturing techniques for miniature compound eyes are discussed. Subsequently, some photodetection technologies for miniature curved compound eye imaging are introduced. Lastly, with reference to the existing prototypes of functional applications for miniature compound eyes, the future development of compound eyes is prospected.

Biomimetic Curved Artificial Compound Eyes: A Review

Natural compound eyes (NCEs) are the most abundant and successful eye designs in the animal kingdom. An NCE consists of a number of ommatidia, which are distributed along a curved surface to receive light. This curved feature is critical to the functions of NCE, and it ensures that different ommatidia point to slightly different directions and thus enables panoramic vision, depth perception, and efficient motion tracking while minimizing aberration. Consequently, biomimetic curved artificial compound eyes (BCACEs) have garnered substantial research attention in replicating the anatomical configuration of their natural counterparts by distributing ommatidia across a curved surface. The reported BCACEs could be briefly categorized into 2 groups: fixed focal lengths and tunable focal lengths. The former could be further subcategorized into simplified BCACEs, BCACEs with photodetector arrays within curved surfaces, and BCACEs with light guides. The latter encompasses other tuning techniques such as fluidic pressure modulation, thermal effects, and pH adjustments. This work starts with a simple classification of NCEs and then provides a comprehensive review of main parameters, operational mechanisms, recent advancements, fabrication methodologies, and potential applications of BCACEs. Finally, discussions are provided on future research and development. Compared with other available review articles on artificial compound eyes, our work is distinctive since we focus especially on the “curved” ones, which are difficult to fabricate but closely resemble the architecture and functions of NCEs, and could potentially revolutionize the imaging systems in surveillance, machine vision, and unmanned vehicles.

Curved fiber compound eye camera inspired by the Strepsiptera vision.

The Strepsiptera vision possesses intriguing features of a large field of view (FOV) and relatively high resolution compared to normal compound eyes. However, it presents a significant challenge of the mismatch between the curved compound eyelet lens array and the planar image sensor to image in a large FOV for artificial compound eyes (ACE). We propose what we believe to be a novel curved fiber compound eye camera (CFCEC) here, which employs coherent fiber bundles as the optical relay system to transmit sub-images curvilinearly. A total of 106 eyelets are arranged based on a scheme similar to the Goldberg polyhedron, with the advantages of uniform interval and minor edge blindness. Then, a prototype of the CFCEC is fabricated and assembled. A series of experiments are conducted to assess the FOV, contrast, resolution, and overlap rate of FOV of the prototype. The results prove that the CFCEC has a total FOV of up to 160°×160° and a total overlap rate of FOV of approximately 65%, demonstrating the promising potential of the CFCEC in various applications, such as panoramic surveillance, 3D detection, and motion tracking.

3D lithography enable ultrathin flat compound eye for moving object imaging

With a hemispherical surface, the compound eye cameras inspired by arthropod eyes provide a large field of view (FOV), low aberration, and a large depth of field. However, to match the hemisphere-imaging plane, the cameras require a hemisphere-photoelectric sensor instead of a standard planar complementary metal oxide semiconductor (CMOS) sensor. Additionally, the hemispheric shape of the imaging system restricts the miniaturization of high-performance compound eye cameras at the millimeter scale. Here, inspired by the working principle of the Fresnel lens, an ultrathin flat compound eye (UFCE) composed of an off-axis microlens array (OMLA) was developed. 3D lithography enabled the fabrication of the UFCE with a complex morphology. Compared to the traditional artificial compound eye (ACE), the thickness of the UFCE was greatly reduced. Meanwhile, a flat substrate and ultrathin structure allowed for direct integration of the UFCE into the surface of a CMOS. The resultant UFCE had a FOV of up to 110° and a resolution close to the diffraction limit and low dispersion. Furthermore, the UFCE allowed for fast imaging of mobile objects in any plane within the depth of field, which is the unique function of the UFCE. In brief, the fabricated UFCE based on OMLA reduces the complexity of constructing compound eye cameras while significantly optimizing the volume, suggesting an alternative method for creating high-performing optical elements with the possibility of simple integration and miniaturization.

The Application of Artificial Compound Eye in Precision Guided Munitions for Urban Combat

As an ideal visual perception system with miniaturization, multi-aperture, wide field of view, low aberration, infinite depth of field, high efficiency and real-time, bionic compound eye has obvious intelligent characteristics, which provides a new idea for the future intelligent development of precision guided weapons, and has also been highly concerned by the US military. Under the leadership of the military, the United States has carried out a series of studies and achieved a series of results. The strategic position of urban combat in the future is analyzed. The requirements of the U.S. Air Force for the next generation of precision-guided munitions, and also the strategic significance of developing urban combat weapons and equipment is described. The advantages and characteristics of artificial compound eye are briefly described. Mainly discussed a series of WFOV artificial compound eye seeker being studied and developed by the U.S. Air Force for urban warfare, as well as the research and development process. Finally, the development suggestion of the WFOV artificial compound eye seeker is given.

Pixelating Responsive Structural Color via a Bioinspired Morphable Concavity Array (MoCA) Composed of 2D Photonic Crystal Elastomer Actuators.

Stimuli-responsive structural coloration allows the color change of soft substrates in response to environmental stimuli such as heat, humidity, and solvents. Such color-changing systems enable smart soft devices, such as the camouflageable skin of soft robots or chromatic sensors in wearable devices. However, individually and independently programmable stimuli-responsive color pixels remain significant challenges among the existing color-changing soft materials and devices, which are crucial for dynamic display. Inspired by the dual-color concavities on butterfly wings, a morphable concavity array to pixelate the structural color of two-dimensional photonic crystal elastomer and achieve individually and independently addressable stimuli-responsive color pixels is designed. The morphable concavity can convert its surface between concave and flat upon changes in the solvent and temperature, accompanied by angle-dependent color-shifting. Through multichannel microfluidics, the color of each concavity can be controllably switched. Based on the system, the dynamic display by forming reversibly editable letters and patterns for anti-counterfeiting and encryption are demonstrated. Itis believed that the strategy of pixelating optical properties through locally altering surface topography can inspire the design of new transformable optical devices, such as artificial compound eyes or crystalline lenses for biomimetic and robotic applications.

Fabrication of a dual-focus artificial compound eye with improved imaging based on modified microprinting and air-assisted deformation.

Natural compound eyes inspire the development of artificial optical devices that feature a large field of view and fast motion detection. However, the imaging of artificial compound eyes dramatically depends on many microlenses. The single focal length of the microlens array significantly limits the actual applications of artificial optical devices, like distinguishing objects at different distances. In this study, a curved artificial compound eye for a microlens array with different focal lengths was fabricated by inkjet printing and air-assisted deformation. By adjusting the space of the microlens array, secondary microlenses were created between intervals of the primary microlens. The diameter/height of the primary and secondary microlens arrays are 75/25µm and 30/9µm, respectively. The planar-distributed microlens array was transformed into a curved configuration using air-assisted deformation. Compared with adjusting the curved base to distinguish objects at different distances, the reported technique features simplicity and is easy to operate. The applied air pressure can be used to tune the field of view of the artificial compound eye. The microlens arrays with different focal lengths could distinguish the objects at different distances without additional components. When the external objects move a small distance, they can be detected by the microlens arrays due to their different focal lengths. It could effectively improve the motion perception of the optical system. Moreover, the focusing and imaging performances of the fabricated artificial compound eye were further tested. The compound eye combines the advantages of monocular eyes and compound eyes, holding great potential for developing advanced optical devices with a large field of view and automatic variable-focus imaging.

The Development of Snapshot Multispectral Imaging Technology Based on Artificial Compound Eyes

In the present study, the advantages of multispectral imaging over hyperspectral imaging in real-time spectral imaging are briefly analyzed, and the advantages and disadvantages of snapshot spectral imaging and other spectral imaging technologies are briefly described. The technical characteristics of artificial compound eyes and multi-aperture imaging and the research significance of snapshot artificial compound eye multispectral imaging are also introduced. The classification and working principle of the snapshot artificial compound eye multispectral imaging system are briefly described. According to the realization method of the optical imaging system, the ACE snapshot multi-aperture multispectral imaging system is divided into plane and curved types. In the planar compound eye spectral imaging system, the technical progress of the multispectral imaging system based on the thin observation module by bound optics (TOMBO) architecture and the multispectral imaging system based on the linear variable spectral filter are introduced. At the same time, three curved multispectral imaging systems are introduced. Snapshot artificial compound eye multispectral imaging technology is also briefly analyzed and compared. The research results are helpful to comprehensively understand the research status of snapshot multispectral multi-aperture imaging technology based on artificial compound eyes and to lay the foundation for improving its comprehensive performance even further.

Chalcogenide Glass IR Artificial Compound Eyes Based on Femtosecond Laser Microfabrication (Adv. Mater. Technol. 2/2023)

Artificial Compound Eyes In article number 2200741, Feng Chen and co-workers present chalcogenide glass IR artificial compound eyes inspired by the fly in the nature and fabricated by femtosecond laser could capture the IR image with the advantages of large field of view and low aberration and be used in complex environments, exhibiting broad prospects in IR imaging, robot vision, IR 3D motion tracking and so on.

基于人工复眼的红外目标运动检测研究

基于人工复眼的红外目标运动检测研究

Multi-aperture optical flow estimation for artificial compound eye using non-subsampled contourlet transform and guided filter

A multi-scale optical flow estimation for the image captured by artificial compound eye (ACE) is investigated in this article. The optical flow estimation of ACE must be adapted by designing algorithms according to its unique multi-aperture characteristics. A more general filter for the regularization term, rather than a single iterative solution in the traditional variational model, is devised using the non-subsampled contourlet transform to enforce band decomposition and estimate the optical flow field. To circumvent the spillover and error of the single-aperture fringe flow field, a flow gradient weight is introduced to suppress it and enhance motion details. Furthermore, low-pass subbands adopt the Bayes threshold with the advantage of efficiently eliminating outliers. More high-pass subbands adopt guided filter with the benefit of separating important details from outliers. The prominent feature of the proposed method is that the accuracy of optical flow estimation is improved effectively by eliminating outliers. Finally, experimental results demonstrate the superiority of the examined optical flow estimation.

Mosaic-free compound eye camera based on multidirectional photodetectors and single-pixel imaging.

Compound-eye wide field-of-view (FOV) imaging generally faces the disadvantages of a complex system, low resolution, and complicated image mosaic. Single-pixel imaging has proven to very beneficial in building a high-resolution and simple wide-FOV camera, but its ability to overcome the problem of image mosaics still needs to be demonstrated. In this Letter, we propose a novel, to the best of our knowledge, kind of artificial compound eye based on multidirectional photodetectors (PDs) and demonstrate theoretically and experimentally that mosaics are unnecessary in multidirectional PD-based single-pixel imaging. In addition, we show experimentally that only nine multidirectional PDs are needed to obtain wide-angle images in a hemisphere to realize wide-FOV mosaic-free imaging. This work greatly simplifies the concept of compound-eye cameras and is very enlightening for detector design in wide-FOV single-pixel imaging, plausibly leading to the development of single-pixel endoscopic imaging.

Chalcogenide Glass IR Artificial Compound Eyes Based on Femtosecond Laser Microfabrication

AbstractThe compact and lightweight infrared (IR) optics devices are highly demanded in booming applications. However, fabrication of IR optics devices with high efficiency is still technically challenging, especially artificial compound eyes (ACE) with low aberration imaging and large field of view. In this work, a method of femtosecond laser wet etching combining with the “two‐step” precision glass molding based on chalcogenide glass is proposed to fabricate glass IR ACE. The as‐prepared consists of 6000 ommatidia (diameter of 88 µm and the sag height of 11 µm) arranged in a hexagonal manner with perfect parabolic morphology and high uniformity. The chalcogenide glass IR ACE exhibits excellent optical performance both in IR active imaging and IR passive imaging with high transmittance (60–70%) ranging from 2.5 to 15 µm. The ommatidia have a high resolution up to 20.16 lp mm−1, and imaging with large field of view up to 60° and low aberration can be achieved. Furthermore, the proposed technology shows advantages to fabricate glass IR ACE with low cost and high efficiency, and glass IR ACE shows great potential in IR imaging, robot vision, IR 3D motion tracking, and so on.