Single-view Approach Research Articles

AMVFNet: Attentive Multi-View Fusion Network for 3D Object Detection

Pillar-based method is significant in the field of LiDAR-based 3D object detection which could directly make use of efficient 2D backbone and save computational resources during reference. Existing methods usually sequentially project the original point clouds into the cylindrical view or the bird-eye view for feature extraction. However, the former suffers from obscured problems and the scales of instances vary greatly with distance, while the latter leads to considerable confusion problems due to the loss of semantic information caused by the sparsity of the projected point cloud. In this paper, we present a novel and efficient two-stage point-pillar hybrid architecture named Attentive Multi-View Fusion Network (AMVFNet), in which we abstract features from all cylindrical view, bird-eye view, and raw point clouds. Rather than designing more complex modules to solve the problems inherent in the single-view approach, our multi-view fusion architecture effectively combines the strengths of multiple perspectives to improve performance at a more fundamental level. Besides, to compensate for quantization distortion caused by projection operations, we propose attentive feature enhancement layers to further improve the capability of contextual information capturing. Extensive experiments on the KITTI detection benchmark illustrate that our proposed AMVFNet achieves competitive performance compared with other SOTA 3D object detectors.

Real-time, dense UAV mapping by leveraging monocular depth prediction with monocular-inertial SLAM

We present a dense and metric 3D mapping pipeline designed for embedded operation on-board UAVs, by loosely coupling deep neural networks trained to infer dense depth single images with a SLAM system that restores metric scale from sparse depth. In contrast to computationally restrictive approaches that leverage multiple views, we propose a highly efficient, single-view approach without sacrificing 3D mapping performance. This enables real-time construction of a global 3D voxel map by iterative fusion of the rescaled dense depth maps obtained via raycasting from the estimated camera poses. Quantitative and qualitative experimentations of our framework in challenging environmental conditions show comparable or superior performance with respect to state-of-the-art approaches via a better effectiveness-efficiency trade-off.

A multi-view ensemble machine learning approach for 3D modeling using geological and geophysical data

Geophysical data are often integrated into geological data for 3D modeling of underground spaces. However, the existing single-view approach means it is difficult to adequately fuse the valid information between the two types of data, and the complexity of lithological decoding and classification is high. To address this issue, a multi-view ensemble machine learning (ML) framework is proposed. Initially, the original dataset of lithology prediction is constructed by aligning geological and geophysical data with different spatial scales. Next, the dataset is divided into three datasets of structural strength, density, and moisture content according to the lithology properties of the geophysical data. The proposed framework is then used to capture the lithologic characteristics under different views to achieve the prediction of lithologic labels. In this process, a self-attentive mechanism is used to adaptively fuse the valid information under each view. To validate the proposed framework, it is applied to a project in Jiaxing, Zhejiang Province, China. Compared with existing ML methods, the proposed multi-view ensemble ML framework improves modeling accuracy and constructs models with low uncertainty. The framework can be extended to other multi-source data fusion tasks across geoscience domains.

Compositional action recognition with multi-view feature fusion.

Most action recognition tasks now treat the activity as a single event in a video clip. Recently, the benefits of representing activities as a combination of verbs and nouns for action recognition have shown to be effective in improving action understanding, allowing us to capture such representations. However, there is still a lack of research on representational learning using cross-view or cross-modality information. To exploit the complementary information between multiple views, we propose a feature fusion framework, and our framework is divided into two steps: extraction of appearance features and fusion of multi-view features. We validate our approach on two action recognition datasets, IKEA ASM and LEMMA. We demonstrate that multi-view fusion can effectively generalize across appearances and identify previously unseen actions of interacting objects, surpassing current state-of-the-art methods. In particular, on the IKEA ASM dataset, the performance of the multi-view fusion approach improves 18.1% over the performance of the single-view approach on top-1.

Monster mash

We present a new framework for sketch-based modeling and animation of 3D organic shapes that can work entirely in an intuitive 2D domain, enabling a playful, casual experience. Unlike previous sketch-based tools, our approach does not require a tedious part-based multi-view workflow with the explicit specification of an animation rig. Instead, we combine 3D inflation with a novel rigidity-preserving, layered deformation model, ARAP-L, to produce a smooth 3D mesh that is immediately ready for animation. Moreover, the resulting model can be animated from a single viewpoint --- and without the need to handle unwanted inter-penetrations, as required by previous approaches. We demonstrate the benefit of our approach on a variety of examples produced by inexperienced users as well as professional animators. For less experienced users, our single-view approach offers a simpler modeling and animating experience than working in a 3D environment, while for professionals, it offers a quick and casual workspace for ideation.

Grasp Pose Detection with Affordance-based Task Constraint Learning in Single-view Point Clouds

Learning to grasp novel objects is a challenging issue for service robots, especially when the robot is performing goal-oriented manipulation or interaction tasks whilst only single-view RGB-D sensor data is available. While some visual approaches focus on grasping that satisfy force-closure standards only, we further link affordances-based task constraints to the grasp pose on object parts, so that both force-closure standard and task constraints can be ensured. In this paper, a new single-view approach is proposed for task-constrained grasp pose detection. We propose to learn a pixel-level affordance detector based on a convolutional neural network. The affordance detector provides a fine grained understanding of the task constraints on objects, which are formulated as a pre-segmentation stage in the grasp pose detection framework. The accuracy and robustness of grasp pose detection are improved by a novel method for calculating local reference frame as well as a position-sensitive fully convolutional neural network for grasp stability classification. Experiments on benchmark datasets have shown that our method outperforms the state-of-the-art methods. We have also validated our method in real-world and task-specific grasping scenes, in which higher success rate for task-oriented grasping is achieved.

An HMM-based synthetic view generator to improve the efficiency of ensemble systems

AbstractOne of the most active areas of research in semi-supervised learning has been to study methods for constructing good ensembles of classifiers. Ensemble systems are techniques that create multiple models and then combine them to produce improved results. These systems usually produce more accurate solutions than a single model would. Specially, multi-view ensemble systems improve the accuracy of text classification because they optimize the functions to exploit different views of the same input data. However, despite being more promising than the single-view approaches, document datasets often have no natural multiple views available. This study proposes an algorithm to generate a synthetic view from a standard text dataset. The model generates a new view from the standard bag-of-words approach using an algorithm based on hidden Markov models (HMMs). To show the effectiveness of the proposed HMM-based synthetic view generation method, it has been integrated in a co-training ensemble system and tested with four text corpora: Reuters, 20 Newsgroup, TREC Genomics and OHSUMED. The results obtained are promising, showing a significant increase in the efficiency of the ensemble system compared to a single-view approach.

Breast cancer diagnosis using abnormalities on ipsilateral views of digital mammograms

Ipsilateral views of digital mammograms help radiologists to localize and confirm abnormal lesions during diagnosis of breast cancers. This study aims at developing algorithms which improve accuracy of computer-aided diagnosis (CADx) for analyzing breast abnormalities on ipsilateral views. The proposed system is a fusion of single and two view systems. Single view approach detects and characterizes suspicious lesions on craniocaudal (CC) and medio-lateral oblique (MLO) view separately using geometric and textural features. Lesions detected on each view are paired with potential lesions on another view. The proposed algorithm computes the correspondence score of each lesion pair. Single view information is fused with two views correspondence score to discriminate malignant tumours from benign masses using the SVM classifier. Performance of SVM classifier is assessed using five-fold cross validation (CV), Kappa metric and ROC analysis. Algorithms are applied to 110 pairs of mammograms from local dataset and 74 pairs from open dataset. Single view scheme yielded image-based sensitivity of 91.63% and 88.17% at 1.35 and 1.51 false positives per image (FPs/I) on local and open dataset respectively. Single view classification yielded FPs/I of 1.03 and 1.20 with sensitivity of 70%. Fusion based two views scheme using SVM classifier produced average case-based sensitivity of 75.91% at 0.69FPs/I and 73.65% at 0.72FPs/I on local and open dataset respectively. Fusion of single view features with two view correspondence score leads to improved case-based detection sensitivity. Proposed fusion based approach results into accurate and reliable diagnosis of breast abnormalities than single view approach.

Drug Target Prediction by Multi-View Low Rank Embedding.

Drug repositioning has been a key problem in drug development, and heterogeneous data sources are used to predict drug-target interactions by different approaches. However, most of studies focus on a single representation of drugs or proteins. It has been shown that integrating multi-view representations of drugs and proteins can strengthen the prediction ability. For example, a drug can be represented by its chemical structure, or by its chemical response in different cells. A protein can be represented by its sequence, or by its gene expression values in different cells. The docking of drugs and proteins based on their structure can be considered as one view (structural view), and the chemical performance of them based on gene expression and drug response can be considered as another view (chemical view). In this work, we first propose a single-view approach of SLRE based on low rank embedding for an arbitrary view, and then extend it to a multi-view approach of MLRE, which could integrate both views. Our experiments show that our methods perform significantly better than baseline methods including single-view methods and multi-view methods. We finally report predicted drug-target interactions for 30 FDA-approved drugs.

Computer‐aided detection of breast masses: Four‐view strategy for screening mammography

To improve the performance of a computer-aided detection (CAD) system for mass detection by using four-view information in screening mammography. The authors developed a four-view CAD system that emulates radiologists' reading by using the craniocaudal and mediolateral oblique views of the ipsilateral breast to reduce false positives (FPs) and the corresponding views of the contralateral breast to detect asymmetry. The CAD system consists of four major components: (1) Initial detection of breast masses on individual views, (2) information fusion of the ipsilateral views of the breast (referred to as two-view analysis), (3) information fusion of the corresponding views of the contralateral breast (referred to as bilateral analysis), and (4) fusion of the four-view information with a decision tree. The authors collected two data sets for training and testing of the CAD system: A mass set containing 389 patients with 389 biopsy-proven masses and a normal set containing 200 normal subjects. All cases had four-view mammograms. The true locations of the masses on the mammograms were identified by an experienced MQSA radiologist. The authors randomly divided the mass set into two independent sets for cross validation training and testing. The overall test performance was assessed by averaging the free response receiver operating characteristic (FROC) curves of the two test subsets. The FP rates during the FROC analysis were estimated by using the normal set only. The jackknife free-response ROC (JAFROC) method was used to estimate the statistical significance of the difference between the test FROC curves obtained with the single-view and the four-view CAD systems. Using the single-view CAD system, the breast-based test sensitivities were 58% and 77% at the FP rates of 0.5 and 1.0 per image, respectively. With the four-view CAD system, the breast-based test sensitivities were improved to 76% and 87% at the corresponding FP rates, respectively. The improvement was found to be statistically significant (p < 0.0001) by JAFROC analysis. The four-view information fusion approach that emulates radiologists' reading strategy significantly improves the performance of breast mass detection of the CAD system in comparison with the single-view approach.

Spectral Variability and Discrimination Assessment in a Tropical Peat Swamp Landscape Using CHRIS/PROBA Data

In this study, we examine seasonal aspects and the potential of multi-angle CHRIS/PROBA data, acquired at two different dates, to improve forest classification. The test site is a typical peat swamp landscape located in Central Kalimantan, Indonesia. We focus on eight specific land use/cover categories from a single view angle and from a multi-angular perspective. We show that: (1) reflectance changes from the end of the monsoon to the beginning of the dry season in the visible were small and slightly positive for the forestry classes, whereas slightly negative for grassland classes; (2) reflectance increases according to the successional stages for a given angle and were higher in the beginning of the dry season; (3) reflectance values increase in the near-infrared with decreasing leaf area index (LAI); and (4) classification results using a multi-angular approach were statistically better at a 5% level of significance from a single view approach on both selected dates, showing that anisotropy information can improve differentiation between peatland landscape classes.

Solid reconstruction from orthographic opaque views using incremental extrusion

The reconstruction of 3-D solids from 2-D projections is an important research topic in reverse engineering. The reconstruction can be grouped into two categories: single-view approach and multiple-view approach. Each approach can be classified as wireframe, BRep or CSG. However, not many CSG approaches have been reported in the literature. The methods are also restricted to uniform-thickness objects or require user interaction. The method proposed in this paper employs the CSG approach. A 3-D solid computer model is reconstructed from 2-D line drawings of six orthographic opaque views, viz. top, front, left, right, bottom and rear views. Firstly, the six views are grouped into three pairs. For each pair of views, segmented areas from one of the two views (called g-view) is incrementally extruded according to the information in the neighbouring view (called d-view). Extrusion primitive solids are generated during the incremental extrusion. All primitive solids are then unioned into an extrusion solid. Finally, all extrusion solids are intersected to give a unique 3-D solution object.

Determining robot location by cylindrical object shapes

A fast single-view approach to determining the robot location by using a known-radius cylindrical object is proposed. In the approach, no prerequisite intrinsic camera parameters are imposed, and only 3-D vector algebra is used in calculating the robot location. Therefore, alignment error can be avoided, the time of location determination is 0.3 s, and the measurement error is less than 3% on average.

Robot guidance using standard mark

A fast single-view approach to determining the robot location using the standard mark is proposed. In the approach, a standard mark with triangle pattern and location code is used as the reference target. No prerequisite condition is imposed in grabbing the image, and only simple 3-D vector algebra is used in calculating the location. Therefore, the alignment error can be avoided and the speed of location determination can be very fast.

Morphological evaluation of vascular smooth muscle cell: length and width from a single scanning electron micrograph of microvessels.

Accurate three-dimensional data on the structure of vascular smooth muscle cells is essential for understanding the microvascular system in both normal or disease conditions. The laborious serial reconstruction methods have limited the amount of data collected on the structure of individual vascular smooth muscle (VSM) cells. The circumferential viewing of whole vessel segments via scanning electron microscopy provides an alternative approach, but even this technique is highly specialized and tedious. This study presents a simplified method to determine the average cell length and width of individual VSM cells by using only one view of a microvessel (single view). The vessels do not have to be isolated for circumferential viewing and can be left in their host tissue if desired. Values for the average VSM cell length and width were obtained by both circumferential- and single-view approach on the same vessels. The average cell length and width obtained from the single-view method (using one-third circumference) duplicated the mean length and width measurements obtained by circumferential viewing.