Orthographic Views Research Articles

Activity-level construction progress monitoring through semantic segmentation of 3D-informed orthographic images

Effective progress monitoring is crucial for successful construction project delivery. Visual data, comprising images and videos of construction operations, has emerged as a valuable source for gaining comprehensive insights into project status. While various vision-based methods have been developed for automated progress monitoring at the element level (e.g., columns, beams, floors, walls), challenges persist in achieving accurate schedule activity-level (e.g., formwork, reinforcement, concrete placement) progress tracking. Existing methods often struggle to report progress status beyond binary form (built/not built). To address these limitations, this research proposes a novel framework for precisely measuring the percent completion of activities associated with under-construction building elements. The framework is implemented through an Activity-level progress monitoring system (ALPMS). ALPMS takes the input of construction site images and a four-dimensional building information model (BIM) and outputs the activity-wise completion percentage required for updating project schedules. It creates as-built point clouds from images, compares them with the as-planned BIM, generates orthographic views of under-construction elements through projective transformation or neural radiance fields (NeRF), applies deep learning-based semantic segmentation for progress reasoning, and estimates completion percentages. Finally, the activity-level rich semantic information is transferred to the as-built point cloud and BIM for three-dimensional visualization of the progress status. When the framework was applied to two building construction projects, on average, the ALPMS could report the activity-wise completion percentages with <6% mean absolute error. Future research shall focus on automating the mapping of project schedule activities and activity-level progress details for timely schedule updates and prediction of project completion dates.

MVGrasp: Real-time multi-view 3D object grasping in highly cluttered environments

Nowadays robots play an increasingly important role in our daily life. In human-centered environments, robots often encounter piles of objects, packed items, or isolated objects. Therefore, a robot must be able to grasp and manipulate different objects in various situations to help humans with daily tasks. In this paper, we propose a multi-view deep learning approach to handle robust object grasping in human-centric domains. In particular, our approach takes a point cloud of an arbitrary object as an input, and then, generates orthographic views of the given object. The obtained views are finally used to estimate pixel-wise grasp synthesis for each object. We train the model end-to-end using a synthetic object grasp dataset and test it on both simulation and real-world data without any further fine-tuning. To evaluate the performance of the proposed approach, we performed extensive sets of experiments in four everyday scenarios, including isolated objects, packed items, pile of objects, and highly cluttered scenes. Experimental results show that our approach performed very well in all simulation and real-robot scenarios. More specifically, the proposed approach outperforms previous state-of-the-art approaches and achieves a success rate of >90% in all simulated and real scenarios, except for the pile of objects which is 82%. Additionally, our method demonstrated reliable closed-loop grasping of novel objects in a variety of scene configurations. The video of our experiments can be found here: https://youtu.be/c-4lzjbF7fY.

Learning From Ambiguous Labels for Lung Nodule Malignancy Prediction.

Lung nodule malignancy prediction is an essential step in the early diagnosis of lung cancer. Besides the difficulties commonly discussed, the challenges of this task also come from the ambiguous labels provided by annotators, since deep learning models have in some cases been found to reproduce or amplify human biases. In this paper, we propose a multi-view 'divide-and-rule' (MV-DAR) model to learn from both reliable and ambiguous annotations for lung nodule malignancy prediction on chest CT scans. According to the consistency and reliability of their annotations, we divide nodules into three sets: a consistent and reliable set (CR-Set), an inconsistent set (IC-Set), and a low reliable set (LR-Set). The nodule in IC-Set is annotated by multiple radiologists inconsistently, and the nodule in LR-Set is annotated by only one radiologist. Although ambiguous, inconsistent labels tell which label(s) is consistently excluded by all annotators, and the unreliable labels of a cohort of nodules are largely correct from the statistical point of view. Hence, both IC-Set and LR-Set can be used to facilitate the training of MV-DAR. Our MV-DAR contains three DAR models to characterize a lung nodule from three orthographic views and is trained following a two-stage procedure. Each DAR consists of three networks with the same architecture, including a prediction network (Prd-Net), a counterfactual network (CF-Net), and a low reliable network (LR-Net), which are trained on CR-Set, IC-Set, and LR-Set respectively in the pretraining phase. In the fine-tuning phase, the image representation ability learned by CF-Net and LR-Net is transferred to Prd-Net by negative-attention module (NA-Module) and consistent-attention module (CA-Module), aiming to boost the prediction ability of Prd-Net. The MV-DAR model has been evaluated on the LIDC-IDRI dataset and LUNGx dataset. Our results indicate not only the effectiveness of the MV-DAR in learning from ambiguous labels but also its superiority over present noisy label-learning models in lung nodule malignancy prediction.

Machine Learning-Based View Synthesis in Fourier Lightfield Microscopy.

Current interest in Fourier lightfield microscopy is increasing, due to its ability to acquire 3D images of thick dynamic samples. This technique is based on simultaneously capturing, in a single shot, and with a monocular setup, a number of orthographic perspective views of 3D microscopic samples. An essential feature of Fourier lightfield microscopy is that the number of acquired views is low, due to the trade-off relationship existing between the number of views and their corresponding lateral resolution. Therefore, it is important to have a tool for the generation of a high number of synthesized view images, without compromising their lateral resolution. In this context we investigate here the use of a neural radiance field view synthesis method, originally developed for its use with macroscopic scenes acquired with a moving (or an array of static) digital camera(s), for its application to the images acquired with a Fourier lightfield microscope. The results obtained and presented in this paper are analyzed in terms of lateral resolution and of continuous and realistic parallax. We show that, in terms of these requirements, the proposed technique works efficiently in the case of the epi-illumination microscopy mode.

Building and Infrastructure Defect Detection and Visualization Using Drone and Deep Learning Technologies

This paper presents an accurate and stable method for object and defect detection and visualization on building and infrastructural facilities. This method uses drones and cameras to collect three-dimensional (3D) point clouds via photogrammetry, and uses orthographic or arbitrary views of the target objects to generate the feature images of points’ spectral, elevation, and normal features. U-Net is implemented in the pixelwise segmentation for object and defect detection using multiple feature images. This method was validated on four applications, including on-site path detection, pavement cracking detection, highway slope detection, and building facade window detection. The comparative experimental results confirmed that U-Net with multiple features has a better pixelwise segmentation performance than separately using each single feature. The developed method can implement object and defect detection with different shapes, including striped objects, thin objects, recurring and regularly shaped objects, and bulky objects, which will improve the accuracy and efficiency of inspection, assessment, and management of buildings and infrastructural facilities.

A comparison between analogic and digital media in Engineering teaching

Form representation is essential for any area that involves the creation of artefacts, since without it the project is just an idea in the mental field. In general terms, it is the ability to perceive and understand shapes and represent them through perspectives and orthographic views, as well as through models and geometric models. Given the issues raised, it is possible to understand the importance of knowledge related to the graphic representation of architecture for professionals involved with the Aeco Industry (Architecture, Engineering, Construction and Operation) and, also the relevance of graphic geometry disciplines on the development of Visiographic Three-dimensional Capacity. So, this paper aims to investigate the impact caused by the use of traditional and digital media in the teaching of graphic representation. To this end, two groups of students were compared, one exposed to the use of both media - analogic and digital - and another that used just digital media. From the application of questionnaires, it was possible to identify the student’s profile, their preferences of media and difficulties encountered. The work presumes that students exposed to more disciplines of graphic geometry will have less difficulties compared to those who had less contact. The main results indicated that: there is a preference, in both groups, for digital media to perform practical activities; the need to adapt the content to each specific engineering and that the most significant difficulties were related to work management, regardless of the media used.

Geometrical Assessment of Sunlit and Shaded Area of Urban Trees Based on Aligned Orthographic Views

To quantify the ecosystem services of trees in urban environments, it is necessary to assess received direct solar radiation of each tree. While the Sky View Factor (SVF) is suitable for assessing the total incoming short- and longwave radiation fluxes, its information is limited to specific points in space. For a spatial analysis, it is necessary to sample the area for SVF. A new geometrical method, Area View Factor (AVF), for the calculation of sunlit areas is proposed. AVF is the ratio of the unhidden, projected surface of an object to the whole projected surface of an object in a complex environment. Hereby, a virtual, orthographic camera is oriented in accordance to the sun’s position in the 3D model domain. The method is implemented in the microscale model SkyHelios, utilizing efficient rendering techniques to assess AVF of all urban trees in parallel. The method was applied to Rieselfeld in Freiburg, Germany. The assessed sunlit area is compared to the SVF at the top of each tree and solar altitude angle, revealing a strong relationship between sunlit areas to solar altitude angles. This study shows that AVF is an efficient methodology to assess received direct radiation of urban trees. Based on AVF, it is possible to identify urban areas with shaded and sunlit trees, but it can also be applied to other objects in complex environments. Therefore, AVF is applicable for urban architecture or energetic research questions.

Three-view generation based on a single front view image for car

The multi-view of an object can be used for 3D reconstruction. The method proposed in this paper generates the left and the top view of a target car through deep learning. The input of the method is only a front view of a 3D car and it isn’t necessary for the depth of the 3D car. Firstly, a rough orthographic views of the 3D car is gotten from an information constraint network which is constructed by considering the structural relation between one view and the other two views. And then the rough orthographic views is transformed into large-pixel block rough view through the nearest interpolation, at the same time, the large-pixel blocks are also migrated to improve the quality of the rough orthographic views. Finally, the generative adversarial network with perception loss is used to enhance the large-pixel block view. In addition, the three views generated by the network can be used to synthesize a 3D point cloud shell.

Fast character modeling with sketch-based PDE surfaces

Virtual characters are 3D geometric models of characters. They have a lot of applications in multimedia. In this paper, we propose a new physics-based deformation method and efficient character modelling framework for creation of detailed 3D virtual character models. Our proposed physics-based deformation method uses PDE surfaces. Here PDE is the abbreviation of Partial Differential Equation, and PDE surfaces are defined as sculpting force-driven shape representations of interpolation surfaces. Interpolation surfaces are obtained by interpolating key cross-section profile curves and the sculpting force-driven shape representation uses an analytical solution to a vector-valued partial differential equation involving sculpting forces to quickly obtain deformed shapes. Our proposed character modelling framework consists of global modeling and local modeling. The global modeling is also called model building, which is a process of creating a whole character model quickly with sketch-guided and template-based modeling techniques. The local modeling produces local details efficiently to improve the realism of the created character model with four shape manipulation techniques. The sketch-guided global modeling generates a character model from three different levels of sketched profile curves called primary, secondary and key cross-section curves in three orthographic views. The template-based global modeling obtains a new character model by deforming a template model to match the three different levels of profile curves. Four shape manipulation techniques for local modeling are investigated and integrated into the new modelling framework. They include: partial differential equation-based shape manipulation, generalized elliptic curve-driven shape manipulation, sketch assisted shape manipulation, and template-based shape manipulation. These new local modeling techniques have both global and local shape control functions and are efficient in local shape manipulation. The final character models are represented with a collection of surfaces, which are modeled with two types of geometric entities: generalized elliptic curves (GECs) and partial differential equation-based surfaces. Our experiments indicate that the proposed modeling approach can build detailed and realistic character models easily and quickly.

What about computational super-resolution in fluorescence Fourier light field microscopy?

Recently, Fourier light field microscopy was proposed to overcome the limitations in conventional light field microscopy by placing a micro-lens array at the aperture stop of the microscope objective instead of the image plane. In this way, a collection of orthographic views from different perspectives are directly captured. When inspecting fluorescent samples, the sensitivity and noise of the sensors are a major concern and large sensor pixels are required to cope with low-light conditions, which implies under-sampling issues. In this context, we analyze the sampling patterns in Fourier light field microscopy to understand to what extent computational super-resolution can be triggered during deconvolution in order to improve the resolution of the 3D reconstruction of the imaged data.

SVAT4: a computer program for visualization and analysis of crystal structures.

SVAT4 is a computer program for interactive visualization of three-dimensional crystal structures, including chemical bonds and magnetic moments. A wide range of functions, e.g. revealing atomic layers and polyhedral clusters, are available for further structural analysis. Atomic sizes, colors, appearance, view directions and view modes (orthographic or perspective views) are adjustable. Customized work for the visualization and analysis can be saved and then reloaded. SVAT4 provides a template to simplify the process of preparation of a new data file. SVAT4 can generate high-quality images for publication and animations for presentations. The usability of SVAT4 is broadened by a software suite for simulation and analysis of electron diffraction patterns.

Hologram conversion for speckle free reconstruction using light field extraction and deep learning.

A novel hologram conversion technique for speckle-less reconstruction is proposed. Many speckle-less reconstruction methods require holograms specially created for those techniques, limiting their applications to general pre-existing holograms. The proposed technique transforms an existing hologram with random phase distribution to new holograms for the application of the speckle-less reconstruction methods. The proposed technique first extracts a set of orthographic views from the existing hologram, then the extracted orthographic views are processed for the speckle noise removal using convolutional neural network. The processed orthographic views are finally used to synthesize new holograms with desired carrier waves by using non-hogel based computer generated hologram technique. The selection of the carrier wave is determined by the desired speckle-less reconstruction method. In this paper, we demonstrate the proposed technique with two speckle-less reconstruction methods; i.e. temporal speckle averaging of different random phase distributions and time-multiplexing of interleaved angular spectrums.

User Interface Supporting Motion Point Teaching in Off-line Industrial Robot Programming: Development and Evaluation of Orthographic Views and Pointing-based Operation Features

User Interface Supporting Motion Point Teaching in Off-line Industrial Robot Programming: Development and Evaluation of Orthographic Views and Pointing-based Operation Features

3D Model Reconstruction from Two Orthographic Views using Fuzzy Surface Analysis

The need for revision, the difficulty of visualization, and the impossibility of using 2D drawings as the input of manufacturing machines make us attempt to acquire a 3D model from 2D engineering drawings. However, the main problem in this process is that we encounter many potential uncertainties. In this paper, to overcome this problem, we used the fuzzy logic as a mathematical tool based on knowledge of an expert. One of the most outstanding advantages of using the fuzzy logic is that uncertainty in related equivalent elements in different orthographic views is reduced. Another important feature of this method is that only two orthographic views are used to reconstruct the 3D model, while in previous methods, three orthographic views were used. Reducing the limitations of reconstruction of 3D models will result lower cost because of simulation of production processes on a 3D model and direct connection to CAD/CAM Systems, conservation of natural resources and more security for documents by maintaining drawings in computer memory, improved education quality and increased health level as a result of better visualization using 3D Object which will ultimately lead to sustainable development.

Large Depth-of-Field Integral Microscopy by Use of a Liquid Lens.

Integral microscopy is a 3D imaging technique that permits the recording of spatial and angular information of microscopic samples. From this information it is possible to calculate a collection of orthographic views with full parallax and to refocus computationally, at will, through the 3D specimen. An important drawback of integral microscopy, especially when dealing with thick samples, is the limited depth of field (DOF) of the perspective views. This imposes a significant limitation on the depth range of computationally refocused images. To overcome this problem, we propose here a new method that is based on the insertion, at the pupil plane of the microscope objective, of an electrically controlled liquid lens (LL) whose optical power can be changed by simply tuning the voltage. This new apparatus has the advantage of controlling the axial position of the objective focal plane while keeping constant the essential parameters of the integral microscope, that is, the magnification, the numerical aperture and the amount of parallax. Thus, given a 3D sample, the new microscope can provide a stack of integral images with complementary depth ranges. The fusion of the set of refocused images permits to enlarge the reconstruction range, obtaining images in focus over the whole region.

Deep convolutional priors for indoor scene synthesis

We present a convolutional neural network based approach for indoor scene synthesis. By representing 3D scenes with a semantically-enriched image-based representation based on orthographic top-down views, we learn convolutional object placement priors from the entire context of a room. Our approach iteratively generates rooms from scratch, given only the room architecture as input. Through a series of perceptual studies we compare the plausibility of scenes generated using our method against baselines for object selection and object arrangement, as well as scenes modeled by people. We find that our method generates scenes that are preferred over the baselines, and in some cases are equally preferred to human-created scenes.

FIMic: design for ultimate 3D-integral microscopy of in-vivo biological samples.

In this work, Fourier integral microscope (FIMic), an ultimate design of 3D-integral microscopy, is presented. By placing a multiplexing microlens array at the aperture stop of the microscope objective of the host microscope, FIMic shows extended depth of field and enhanced lateral resolution in comparison with regular integral microscopy. As FIMic directly produces a set of orthographic views of the 3D-micrometer-sized sample, it is suitable for real-time imaging. Following regular integral-imaging reconstruction algorithms, a 2.75-fold enhanced depth of field and [Formula: see text]-time better spatial resolution in comparison with conventional integral microscopy is reported. Our claims are supported by theoretical analysis and experimental images of a resolution test target, cotton fibers, and in-vivo 3D-imaging of biological specimens.

3D geometry reconstruction from orthographic views: An improved method exploiting shading information

The demand of new tools devoted to integrate aesthetic design into classic parametric CAD suites has undergone a raising interest from the user community. Industrial tools for reconstructing the 3D geometry of an object starting from 2D representations and for surfacing wireframe models according to designers intent, assumed a high relevance and have experienced a steady evolution. The problem of surfacing wireframe models with consistent closing surface patches is still a hot topic in the computer graphics as well as in the industrial field, as demonstrated by a number of recent works. In this paper, authors propose a procedure for enhancing the surfacing process by using shading information, together with the wireframe model. In particular, this work allows to use shading, represented either in a rendered 3D model or sketched by a designer, for retrieving consistent geometrical information related to the 3D object. The procedure is accompanied by some selected case studies, to highlight its strengths and weaknesses.

Perspective as an artistic tool of composition in the engravings by Giovanni Battista Piranesi

This study is devoted to the recreation of Giovanni Battista Piranesi’s compositional thinking. The study is based on analyses of the composition of his engravings depicting architecture and space. Here Piranesi’s compositional solutions for his graphic works are compared to his predecessors and contemporaries’ beautiful paintings carrying the same motif. In order to show that the perspective was one of the artistic tools for him, the article presents the comparative analysis of one of Piranesi’s engravings (“View of Basilica di San Paolo Fuori le Mura” from the series of “Views of Rome”) with a perspective model made with the help of a SketchUp software, pictures of layouts and perspective views made with the help of an architect method (the method of the descriptive geometry). The author builds his point of view on the fact that the plane comprises only such structural elements of the existing composition, which clearly reveal its space-dimensional features when compared with each other. To confirm this, the author proposes an approach implying that all the information fixed at the plane is considered to be a professional communicative means necessary for formation of space-dimensional representations in a human mind about a composition in the overall system of the drawing or the image. In order to confirm this hypothesis, the author carried out an analysis of the Pantheon drawings made by Andrea Palladio di Pietro and Giovanni Battista Piranesi. Due to the fact that the composition information on the object is broken into pieces and is represented by the system of interrelated orthographic views on the architectural design (on the drawing), in the article we tried to find out the implied logic and the pattern of information disclosure by Giovanni Battista Piranesi with the help of projection views.

An Unsupervised Method for Uncovering Morphological Chains

Most state-of-the-art systems today produce morphological analysis based only on orthographic patterns. In contrast, we propose a model for unsupervised morphological analysis that integrates orthographic and semantic views of words. We model word formation in terms of morphological chains, from base words to the observed words, breaking the chains into parent-child relations. We use log-linear models with morpheme and word-level features to predict possible parents, including their modifications, for each word. The limited set of candidate parents for each word render contrastive estimation feasible. Our model consistently matches or outperforms five state-of-the-art systems on Arabic, English and Turkish.