Wide-angle Camera Research Articles

Quantitative visualization of physical barriers for vulnerable pedestrians based on photogrammetry

PurposeThe purpose of this study is to propose a method for vulnerable pedestrians to visualize potential obstacles on sidewalks. In recent years, the number of vulnerable pedestrians has been increasing as Japanese society has aged. The number of wheelchair users is also expected to increase in the future. Currently, barrier-free maps and street-view applications can be used by wheelchair users to check possible routes and the surroundings of their destinations in advance. However, identifying physical barriers that pose a threat to vulnerable pedestrians en route is often difficult.Design/methodology/approachThis study uses photogrammetry to create a digital twin of the three-dimensional (3D) geometry of the existing walking space by collecting photographic images taken on sidewalks. This approach allows for the creation of high-resolution digital elevation models of the entire physical sidewalk surface from which physical barriers such as local gradients and height differences can be detected by uniform image filtering. The method can be used with a Web-based data visualization tool in a geographical information system, permitting first-person views of the ground and accurate geolocation of the barriers on the map.FindingsThe findings of this study showed that capturing the road surface with a small wide-angle camera while walking is sufficient for recording subtle 3D undulations in the road surface. The method used for capturing data and the precision of the 3D restoration results are described.Originality/valueThe proposed approach demonstrates the significant benefits of creating a digital twin of walking space using photogrammetry as a cost-effective means of balancing the acquisition of 3D data that is sufficiently accurate to show the detailed geometric features needed to navigate a walking space safely. Further, the findings showed how information can be provided directly to users through two-dimensional (2D) and 3D Web-based visualizations.

Ultra-compact 3D-printed wide-angle cameras realized by multi-aperture freeform optical design.

Simultaneous realization of ultra-large field of view (FOV), large lateral image size, and a small form factor is one of the challenges in imaging lens design and fabrication. All combined this yields an extensive flow of information while conserving ease of integration where space is limited. Here, we present concepts, correction methods and realizations towards freeform multi-aperture wide-angle cameras fabricated by femtosecond direct laser writing (fsDLW). The 3D printing process gives us the design freedom to create 180° × 360° cameras with a flat form factor in the micrometer range by splitting the FOV into several apertures. Highly tilted and decentered non-rotational lens shapes as well as catadioptric elements are used in the optical design to map the FOV onto a flat surface in a Scheimpflug manner. We present methods to measure and correct freeform surfaces with up to 180° surface normals by confocal measurements, and iterative fabrication via fsDLW. Finally, approaches for digital distortion correction and image stitching are demonstrated and two realizations of freeform multi-aperture wide-angle cameras are presented.

Stray light analysis by ray tracing simulation for the wide-angle multiband camera OROCHI onboard the Martian Moons eXploration (MMX) spacecraft

• Ray tracing analysis was conducted to reduce stray light of OROCHI onboard the MMX. • OROCHI’s optical system is simulated and its CCD reflectance was measured. • Our techniques are incorporated into the preliminary design of OROCHI. The Martian Moons eXploration (MMX) spacecraft is equipped with two cameras, i.e., the TElescopic Nadir imager for GeOmOrphology (TENGOO) and the Optical Radiometer composed Of CHromatic Imagers (OROCHI), for the scientific observation of the Martian moon Phobos. OROCHI is a wide-angle multiband camera system comprising seven cameras with different bandpass filters and one monochromatic camera in the visible and near-infrared range. Thus, OROCHI can simultaneously obtain multiband images. Previous space observations have revealed that the reflection spectra of Phobos exhibit regional differences, including a red unit and a blue unit with weak redness. Moreover, Phobos shows an absorption band near 650 nm, attributed to the red unit of Phobos. Global observations using OROCHI require the determination of such regional variations with high precision; therefore, we must reduce noise in the optical system design. In this study, to meet this requirement, we focus on stray light reduction. Stray light is a source of noise generated by reflections at surfaces of optical components (e.g., sensors, filters, lenses, and lens barrels). We design simulation models of the OROCHI optical system, measure the reflectance of its charge-coupled device (CCD) image sensor, and conduct stray light analysis via ray tracing. We estimate the ratio of the stray light intensity to the target signal intensity by deriving both the reflected (constituting stray light) and unreflected rays. We show that placing the filter away from the sensor and reducing the reflectances of the filter and lenses effectively reduce stray light. Such approaches are incorporated into the preliminary design of OROCHI.

A new method for testing wide range horizontal field angle

In order to solve the problems of small measurement range, large error and low efficiency of laboratory optical field angle testing, a high-precision, easy -operating, high-efficient, and widely used horizontal field angle test method is proposed. It comes to a conclusion that the test method can reduce the experimental error through the analysis of the principle of the field of view error and the calculation of laboratory simulation. The simulation results show that for cameras with a field of view of more than 150 degrees, the measurement error can be reduced by 37 degrees, and when the field of view of the camera under test is close to 170 degrees, the method can reduce the measurement error by nearly 54 degrees. Meanwhile, a wide-range horizontal field angle measurement method is proposed. The camera under test is moved on the supporting mobile platform to image the target test board, and then the imaging target is read by reading the scale value on the test board calculates the angle of the camera under test. This method can effectively avoid the measurement error of the angle caused by the distance between the center of the lens surface and the center of the entrance pupil, so as to quickly obtain the angle of view test results, and improve the testing accuracy, and it is also suitable for cameras that measure a wide range of field angles (wide-angle camera or fisheye camera, etc.) to solve the problem of laboratory testing a wide range of horizontal field angles.

Saudi Arabian retinopathy of prematurity national telemedicine program: Achievements and challenges.

The Saudi Arabian Retinopathy of Prematurity National Telemedicine programme (SAROP) is a product of the National Committee for Retinopathy of Prematurity (ROP). The program includes ROP telescreening, diagnosis, and management of cases requiring treatment. Digital retinal images and filled ROP software requests were uploaded from 20 level-3 neonatal intensive care units (NICU) in the Kingdom of Saudi Arabia (KSA) to the King Khaled Eye Specialist Hospital server and the ROP telemedicine website. The data were accessed and reported by qualified retinal and pediatric ophthalmologists. Currently, retinal wide-angle digital cameras are available in 20 of the 31 level-3 NICUs of the Ministry of Health, Kingdom of KSA. This telemedicine approach is practical and effective in detecting and managing ROP cases. In the first 2.5 years, encouraging results were noticed with no unfavorable outcomes in the participating NICUs. Technical challenges were resolved promptly to ensure that the program ran smoothly. Therefore, this ideal state-of-the-art ROP telemedicine program could be also applied to similar and neighboring countries.

A Novel Approach for Multiscale Lunar Crater Detection by the Use of Path-Profile and Isolation Forest Based on High-Resolution Planetary Images

Crater detection from planetary images is a challenging issue due to the complicated variations in geometry shape, illumination, and scale. An automatic crater detection algorithm (CDA) that is robust to these factors is, therefore, necessary. In this article, a novel automatic CDA that is robust to these factors is proposed to detect the multiscale craters of the Moon. The proposed method consists of two main steps: 1) in the hypothesis generation (HG) step, a novel feature operator called the path-profile, which is constructed based on the self-defined adjacency graph and a path descriptor, is presented to derive the highlight-shadow feature of craters for detecting candidate craters. 2) In the hypothesis verification (HV) step, based on the idea of anomaly detection, the isolation forest algorithm which is an unsupervised learning anomaly detection method is applied to eliminate falsely detected craters. Lunar Reconnaissance Orbiter Camera Wide Angle Camera and Narrow Angle Camera images and Chang’E-4 landing camera images were used to test the accuracy and robustness of the proposed method. The experimental results indicate that: on average, the accuracy of the detection result of the HG step is about 90%, and the HV step can further improve this by 3%–4%. The proposed method is a reliable way to detect multiscale lunar craters for various resolutions images with diameters ranging from five pixels to hundreds of pixels, and it is robust to the different terrains and illumination conditions on the Moon.

Correcting Face Distortion in Wide-Angle Videos.

Video blogs and selfies are popular social media formats, which are often captured by wide-angle cameras to show human subjects and expanded background. Unfortunately, due to perspective projection, faces near corners and edges exhibit apparent distortions that stretch and squish the facial features, resulting in poor video quality. In this work, we present a video warping algorithm to correct these distortions. Our key idea is to apply stereographic projection locally on the facial regions. We formulate a mesh warp problem using spatial-temporal energy minimization and minimize background deformation using a line-preservation term to maintain the straight edges in the background. To address temporal coherency, we constrain the temporal smoothness on the warping meshes and facial trajectories through the latent variables. For performance evaluation, we develop a wide-angle video dataset with a wide range of focal lengths. The user study shows that 83.9% of users prefer our algorithm over other alternatives based on perspective projection. The video results can be found at https://www.wslai.net/publications/video_face_correction/.

Experiments with a Semi-Trailer vehicle model of automated backward parking using wide-angle camera images

Experiments with a Semi-Trailer vehicle model of automated backward parking using wide-angle camera images

Designing an Efficient Emergency Response Airborne Mapping System with Multiple Sensors

Multisource remote sensing data have been extensively used in disaster and emergency response management. Different types of visual and measured data, such as high-resolution orthoimages, real-time videos, accurate digital elevation models, and three-dimensional landscape maps, can enable producing effective rescue plans and aid the efficient dispatching of rescuers after disasters. Generally, such data are acquired using unmanned aerial vehicles equipped with multiple sensors. For typical application scenarios, efficient and real-time access to data is more important in emergency response cases than in traditional application scenarios. In this study, an efficient emergency response airborne mapping system equipped with multiple sensors was designed. The system comprises groups of wide-angle cameras, a high-definition video camera, an infrared video camera, a LiDAR system, and a global navigation satellite system/inertial measurement unit. The wide-angle cameras had a visual field of 85° × 105°, facilitating the efficient operation of the mapping system. Numerous calibrations were performed on the constructed mapping system. In particular, initial calibration and self-calibration were performed to determine the relative pose between different wide-angle cameras to fuse all the acquired images. The mapping system was then tested in an area with altitudes of 1000 m–1250 m. The biases of the wide-angle cameras were small bias values (0.090 m, −0.018 m, and −0.046 m in the x-, y-, and z-axes, respectively). Moreover, the root-mean-square error (RMSE) along the planer direction was smaller than that along the vertical direction (0.202 and 0.294 m, respectively). The LiDAR system achieved smaller biases (0.117, −0.020, and −0.039 m in the x-, y-, and z-axes, respectively) and a smaller RMSE in the vertical direction (0.192 m) than the wide-angle cameras; however, RMSE of the LiDAR system along the planar direction (0.276 m) was slightly larger. The proposed system shows potential for use in emergency response systems for efficiently acquiring data such as images and point clouds.

Obstacle avoidance for orchard vehicle trinocular vision system based on coupling of geometric constraint and virtual force field method

Nowadays the use of visual navigation in orchard is rapidly growing, while current research mainly focuses on orchard row detection or obstacle detection separately. The economic, fast and reliable navigation vision system that completing orchard row following and obstacle avoidance simultaneously and the obstacle avoidance algorithm for vision system without complex multiple navigation systems combining are needed further research. In this paper, a trinocular vision system for orchard vehicle is set up and the obstacle avoidance algorithm based on coupling of geometric constraint and virtual force field is designed for its vision system.First of all, based on analyzing the different vision system characteristics and the detection demand of orchard obstacle avoidance the trinocular vision system for orchard vehicle is set up based on a wide-angle camera and binocular stereo vision system. Then, the image processing algorithm of orchard row and obstacle detection is designed. For orchard row detection, the trunk regions enhancement algorithm is designed based on grayscale morphology filtering the trunk regions prediction algorithm is designed based on optical flow method to improve the trunk regions detection speed and accuracy. For obstacle detection, the background equalization algorithm based on H-channel image characteristics and the interference weaken algorithm with G-channel image characteristics are designed. Based on the row and obstacle detection results the algorithm for obstacle avoidance in orchard based on coupling of geometric constraint and virtual force field is designed. In the obstacle avoidance process, the virtual force field method is used as the mainly control of the vehicle obstacle avoidance process, and the geometric constraint between visual model of trinocular vision system and obstacle position is coupling with it to fulfill the avoidance demand of the obstacle with special shape and other condition that influence the virtual force field calculation.The experimental results show that the trinocular vision system for orchard vehicle built in this paper has strong adaptability to the actual environment which is accurate and stable for orchard row detection and obstacle detection simultaneously. The average deviation is 4.76 cm and 7.05 cm at 0.5 m/s and 1.0 m/s respectively, the average deviation of obstacle distance detection in the Z-axis direction is 3.18 cm, and in the X-axis direction is 0.45 cm. And the obstacle avoidance algorithm designed in this paper is effective in the orchard which can meet the actual production requirements. The research results will lay a foundation for the development of intelligent equipment and unmanned management in orchard.

Constraints on Optical Emission of FAST-detected FRB 20181130B with GWAC Synchronized Observations

Abstract Multiwavelength simultaneous observations are essential to the constraints on the origin of fast radio bursts (FRBs). However, it is a significant observational challenge due to the nature of FRBs as transients with a radio millisecond duration, which occur randomly in the sky regardless of time and position. Here, we report the search for short-time fast optical bursts in the Ground-based Wide Angle Camera (GWAC) archived data associated with FRB 20181130B, which were detected by the Five-hundred-meter Aperture Spherical radio Telescope and recently reported. No new credible sources were detected in all single GWAC images with an exposure time of 10 s, including images with coverage of the expected arrival time in optical wavelength by taking the high dispersion measurements into account. Our results provide a limiting magnitude of 15.43 ± 0.04 mag in the R band, corresponding to a flux density of 1.66 Jy or 8.35 mag in AB system by assuming that the duration of the optical band is similar to that of the radio band of about 10 ms. This limiting magnitude makes the spectral index of α < 0.367 from optical to radio wavelength. The possible existence of longer-duration optical emission was also investigated with upper limits of 0.33 Jy (10.10 mag), 1.74 mJy (15.80 mag), and 0.16 mJy (18.39 mag) for the durations of 50 ms, 10 s, and 6060 s, respectively. This undetected scenario could be partially attributed to the shallow detection capability, as well as the high inferred distance of FRB 20181130B and the low fluence in radio wavelength. The future detectability of optical flashes associated with nearby and bright FRBs are also discussed in this paper.

Calibration method of multi-projector display system with extra-large FOV and quantitative registration accuracy analysis.

The calibration of multi-projector display with extra-large field of view (FOV) and quantitative registration analysis for realizing perfect visual splicing is crucial and difficult. In this paper, we present a novel calibration method to realize the seamless splicing for a multi-projector display system with extra-large FOV. The display consists of 24 projectors, covering the range of 360 degrees in the longitude direction and 210 degrees in the latitude direction. A wide-angle camera fixed on a rotating optical system is used to scan the entire display scene and establish point-to-point correspondence between projector pixels and spatial points using the longitude and latitude information. Local longitude table and latitude table are established on the target of the wide-angle camera. A deterministic method is proposed to locate the North Pole of the display. The local tables corresponding to different camera views can be unified based on the image of the North Pole to form global longitude and latitude tables of arbitrary free-form surface. The mapping between the projector pixels and the camera pixels is established by inverse projection technique, and then each pixel of each projector can be appointed a pair of unique longitude and latitude values. A quantitative registration accuracy analysis method is proposed for multi-projector display system, in which, three-frequency temporal unwrapping method based on coded longitude and latitude values is applied to calculate the registration accuracy. Experiments prove that the registration error of the multi-projector system is less than 0.4 pixels.

Stitching Videos from a Fisheye Lens Camera and a Wide-Angle Lens Camera for Telepresence Robots

Many telepresence robots are equipped with a forward-facing camera for video communication and a downward-facing camera for navigation. In this paper, we propose to stitch videos from the FF-camera with a wide-angle lens and the DF-camera with a fisheye lens for telepresence robots. We aim at providing a compact and efficient user interface for telepresence robots with user-friendly interactive experiences. To this end, we present a multi-homography-based video stitching method which stitches videos from a wide-angle camera and a fisheye camera. The method consists of video image alignment, seam cutting, and image blending. We directly align the wide-angle video image and the fisheye video image based on the multi-homography alignment without calibration, distortion correction, and unwarping procedures. Thus, we can obtain a stitched video with shape preservation in the non-overlapping regions and alignment in the overlapping area for telepresence. To alleviate ghosting effects caused by moving objects and/or moving cameras during telepresence robot driving, an optimal seam is found for aligned video composition, and the optimal seam will be updated in subsequent frames, considering spatial and temporal coherence. The final stitched video is created by image blending based on the optimal seam. We conducted a user study to demonstrate the effectiveness of our method and the superiority of user interfaces with a stitched video.

Geologic Mapping and Age Determinations of Tsiolkovskiy Crater

Tsiolkovskiy is a ~200 km diameter crater presenting one of the few mare deposits of the lunar far side. In this work, we perform a geological study of the crater by means of morpho-stratigraphic and color-based spectral mappings, and a detailed crater counting age determination. The work aims at characterizing the surface morphology and compositional variation observed from orbital data including the Lunar Reconnaissance Orbiter Wide Angle Camera and Clementine UVVIS Warped Color Ratio mosaics, and attempts a reconstruction of the evolutionary history of the Tsiolkovskiy crater through both relative and absolute model age determinations. The results show a clear correlation between the geologic and spectral units and an asymmetric distribution of these units reflecting the oblique impact origin of the crater. Crater counts performed using the spectral units identified on the smooth crater floor returned distinct age ranges, suggesting the occurrence of at least three different igneous events, generating units characterized by particular compositions and/or degree of maturity. This work demonstrates the scientific value of Tsiolkovskiy crater for a better understanding of the volcanic evolution of the Moon and, in particular, of its far side.

A Sensor Fused Rear Cross Traffic Detection System Using Transfer Learning.

Recent emerging automotive sensors and innovative technologies in Advanced Driver Assistance Systems (ADAS) increase the safety of driving a vehicle on the road. ADAS enhance road safety by providing early warning signals for drivers and controlling a vehicle accordingly to mitigate a collision. A Rear Cross Traffic (RCT) detection system is an important application of ADAS. Rear-end crashes are a frequently occurring type of collision, and approximately 29.7% of all crashes are rear-ended collisions. The RCT detection system detects obstacles at the rear while the car is backing up. In this paper, a robust sensor fused RCT detection system is proposed. By combining the information from two radars and a wide-angle camera, the locations of the target objects are identified using the proposed sensor fused algorithm. Then, the transferred Convolution Neural Network (CNN) model is used to classify the object type. The experiments show that the proposed sensor fused RCT detection system reduced the processing time 15.34 times faster than the camera-only system. The proposed system has achieved 96.42% accuracy. The experimental results demonstrate that the proposed sensor fused system has robust object detection accuracy and fast processing time, which is vital for deploying the ADAS system.

Science Missions of the Korean Lunar Exploration Program

The Korea Pathfinder Lunar Orbiter (KPLO), which is the Korean first lunar and space exploration spacecraft, will be launched in August 2022 and arrive in a lunar orbit in December 2022. The KPLO will carry out nominal missions while in a lunar polar orbit an ~100-km altitude for one year. The KPLO has five lunar science mission payloads and one technology demonstration payload in order to achieve their own science and technology goals. The science payloads consist of four Korean domestic instruments and one internationally collaborated science instrument for scientific investigations on the lunar surface and in a space environment. The Korean dometstic science instruments are the gamma-ray spectrometer named KGRS, the wide-angle polarimetric camera named PolCam, the fluxgate magnetometer named KMAG, and the high resolution camera named LUTI. The name of the internationally collaborated science instrument is ShandowCam, which was developed by Arizona State University, U.S., and funded and managed by NASA. The science data acquired by the science payloads will be released to the public in order to enhance scientific and educational achievements. The science data acquired by each science instrument will be archived and released through the web sites of the KPDS (KARI Planetary Data System) for the Korean science instruments and the NASA PDS (Planetary Data System) for the internationally collaborated science instrument.

자율주차를 위한 딥러닝 기반 주차영역 및 충돌위험 영역 검출

This paper proposes a detection method for parking areas and collision risk areas in parking situations. Deep learning algorithms used for area detection rely on semantic segmentation, a method of classifying pixels into semantic segmentation. The main architecture of this paper is based on a harmonic densely connected network and a cross-stage partial network. The dataset was calibrated for four 190-degree wide-angle cameras to generate 500 AVM images based on the Chungbuk National University parking lot, and the experiment was performed with this dataset. In the experimental results, we found that the available parking area can be visualized by detecting the parking line, the parking area, and the available driving area in the AVM image. We visualized the area that remained undetected, as a collision risk area, through semantic segmentation to derive the results. According to the proposed CSPHarDNet model, the experimental results obtained were 81.89% mIoU and 18.36 FPS in the NVIDIA Xavier environment.

Human Fall Detection in Surveillance Videos Using Fall Motion Vector Modeling

Representation of spatio-temporal properties of human body silhouette and human-to-ground relationship, significantly contribute to the fall detection process. So, we propose an approach to efficiently model the spatio-temporal features using fall motion vector. First, we construct a Gaussian mixture model (GMM) called fall motion mixture model (FMMM) using histogram of optical flow and motion boundary histogram features to implicitly capture motion attributes in both the fall and non-fall videos. The FMMM contains both fall and non-fall attributes resulting in a high-dimensional representation. In order to extract only the relevant attributes for a particular fall or non-fall videos, we perform factor analysis on FMMM to get a low dimensional representation known as fall motion vector. Using fall motion vector, we are able to efficiently identify fall events in varieties of scenarios, such as the narrow angle camera (Le2i dataset), wide angle camera (URFall dataset), and multiple cameras (Montreal dataset). In all these scenarios, we show that the proposed fall motion vector achieves better performance than the existing methods.

Detailed Morphologic Mapping and Traverse Planning for a Rover-based Lunar Sample Return Mission to Schrödinger Basin

Abstract Schrödinger basin, a 312 km diameter complex impact structure located near the lunar south pole, has been widely cited as a prime target for future lunar exploration. In 2020 NASA identified Schrödinger as a high-priority landing site for a 2024 mission supported by the Payloads and Research Investigations on the Surface of the Moon solicitation and the Commercial Lunar Payload Services program. Schrödinger basin hosts an uplifted peak ring that would provide a surface mission with access to materials that originated deep within the lunar crust, as well as material ejected from the larger South Pole–Aitken basin. Schrödinger basin also hosts well-preserved mare and pyroclastic deposits that could provide valuable insight into volcanic processes on the Moon. This study used high-resolution Wide Angle Camera and Narrow Angle Camera images from the Lunar Reconnaissance Orbiter, elevation data from the Lunar Orbiter Laser Altimeter, and spectral data from the Clementine mission to produce a high-resolution morphologic map of the basin center consisting of 10 distinct morphologic units. This new map was used to plan traverse paths for a rover mission to the region. The design requirements for this traverse were based on those originally developed for the multiagency Human-Enhanced Robotic Architecture and Capability for Lunar Exploration and Science (HERACLES) mission and the Canadian Space Agency Precursor to Humans And Science Rover concept. The proposed traverse path includes up to 20 sample collection stops with the goal of better understanding lunar chronology, lunar volcanism, and the impact cratering process.

Detection of Flare-associated CME Candidates on Two M-dwarfs by GWAC and Fast, Time-resolved Spectroscopic Follow-ups

Abstract The flare-associated stellar coronal mass ejections (CMEs) of solar-like and late-type stars profoundly impact the habitability of any expolanets in the systems. In this paper, we report the detection of flare-associated CMEs for two M-dwarfs, thanks to a high-cadence survey carried out by the Ground Wide-angle Camera system and fast photometric and spectroscopic follow-ups. The flare energies in the R band are determined to be 1.6 × 1035 erg and 8.1 × 1033 erg based on modeling of their light curves. The time-resolved spectroscopic observations start at about 20 and 40 minutes after the trigger in both cases. The large projected maximum velocity of ∼500–700 km s−1 suggests that the high-velocity wings of their Hα emission lines most likely result from CME events in both stars, after excluding the possibility of chromospheric evaporation and coronal rain. The masses of the CMEs are estimated to be 1.5–4.5 × 1019 g and 7.1 × 1018 g.