Industrial Scene Research Articles

Robust fringe projection measurement based on reference phase reconstruction

Phase measurement profilometry plays an important role in industrial applications. Traditional fringe projection profilometry (FPP) requires a stable reference plane to record the reference phase for the 3D reconstruction, which makes it sensitive to the motion-induced error caused by the reference plane. Thus, it is challenging for the traditional FPP to implement a robust measurement in industrial scenarios though it exhibits high accuracy. To solve this problem, a phase reconstruction method is proposed where the reference phase is denoised and reconstructed from the background of the modulated phase by minimizing the total variation (TV) via the theory of Sylvester equation. Since this strategy achieves synchronous extraction of the reference phase and the object one, it can correct the reference phase in real-time and reduce the measurement error caused by deviation of the reference plane. The proposed method has been applied to measure the step-shape workpiece with different deviations of the reference plane and the gear workpieces on conveyer. The experimental results show competitive performance with traditional FPP in accuracy and more robustness in complex industrial scenes.

Attention U-Net with Feature Fusion Module for Robust Defect Detection

U-Net shows a remarkable performance and makes significant progress for segmentation task in medical images. Despite the outstanding achievements, the common case of defect detection in industrial scenes is still a challenging task, due to the noisy background, unpredictable environment, varying shapes and sizes of the defects. Traditional U-Net may not be suitable for low-quality images with low illumination and corruption, which are often presented in the practical collections in real-world scenes. In this paper, we propose an attention U-Net with feature fusion module for combining multi-scale features to detect the defects in noisy images automatically. Feature fusion module contains convolution kernels of different scales to capture shallow layer features and combine them with the high-dimensional features. Meanwhile, attention gates are used to enhance the robustness of skip connection between the feature maps. The proposed method is evaluated on two datasets. The best precision rate and MIoU of defect detection are 95.6% and 92.5%. The best F-score of concrete crack detection is 95.0%. Experimental results show that the proposed approach achieves promising results in both datasets. It demonstrates that our approach consistently outperforms other U-Net-based approaches for defect detection in low-quality images. Experimental results have shown the possibility of developing a mixture system that can be deployed in many applications, such as remote sensing image analysis, earthquake disaster situation assessment, and so on.

An Efficient and Stable Grasp Planning Method for Stacked Parts

In the industrial scenes, it often requires to pick up the part quickly in part stacked scenes and then place it in the specific position. This paper proposes a method of grasp evaluation function for grasp planning. It considers collision and interference between gripper and other objects, relative height, grasp angle of the object and stability of grasping the object. In addition, we use the method of voxelization to describe the scene. The research team builds a robotic arm grasping system based on ROS, UR3 and Ensenso N35. We carry out grasp experiments on the stacked scene of a single kind of object. The accuracy is 79.55%. It takes 3.6 seconds to get a grasp pose, which basically meets the requirements of industrial use.

An Improved Algorithm for Detection and Pose Estimation of Texture-Less Objects

This paper introduces an improved algorithm for texture-less object detection and pose estimation in industrial scenes. In the template training stage, a multi-scale template training method is proposed to improve the sensitivity of LineMOD to template depth. When this method performs template matching, the test image is first divided into several regions, and then training templates with similar depth are selected according to the depth of each test image region. In this way, without traversing all the templates, the depth of the template used by the algorithm during template matching is kept close to the depth of the target object, which improves the speed of the algorithm while ensuring that the accuracy of recognition will not decrease. In addition, this paper also proposes a method called coarse positioning of objects. The method avoids a lot of useless matching operations, and further improves the speed of the algorithm. The experimental results show that the improved LineMOD algorithm in this paper can effectively solve the algorithm’s template depth sensitivity problem.

Triplet-Graph Reasoning Network for Few-Shot Metal Generic Surface Defect Segmentation

Metal surface defect segmentation can play an important role in dealing with the issue of quality control during the production and manufacturing stages. There are still two major challenges in industrial applications. One is the case that the number of metal surface defect samples is severely insufficient, and the other is that the most existing algorithms can only be used for specific surface defects and it is difficult to generalize to other metal surfaces. In this work, a theory of few-shot metal generic surface defect segmentation is introduced to solve these challenges. Simultaneously, the Triplet-Graph Reasoning Network (TGRNet) and a novel dataset Surface Defects- 4 <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">i</sup> are proposed to achieve this theory. In our TGRNet, the surface defect triplet (including triplet encoder and trip loss) is proposed and is used to segment background and defect area, respectively. Through triplet, the few-shot metal surface defect segmentation problem is transformed into few-shot semantic segmentation problem of defect area and background area. For few-shot semantic segmentation, we propose a method of multi-graph reasoning to explore the similarity relationship between different images. And to improve segmentation performance in the industrial scene, an adaptive auxiliary prediction module is proposed. For Surface Defects- 4 <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">i</sup> , it includes multiple categories of metal surface defect images to verify the generalization performance of our TGRNet and adds the nonmetal categories (leather and tile) as extensions. Through extensive comparative experiments and ablation experiments, it is proved that our architecture can achieve state-of-the-art results.

An Efficient Convolutional Neural Network Model Based on Object-Level Attention Mechanism for Casting Defect Detection on Radiography Images

Automatic detection of casting defects on radiography images is an important technology to automatize digital radiography defect inspection. Traditionally, in an industrial application, conventional methods are inefficient when the detection targets are small, local, and subtle in the complex scenario. Meanwhile, the outperformance of deep learning models, such as the convolutional neural network (CNN), is limited by a huge volume of data with precise annotations. To overcome these challenges, an efficient CNN model, only trained with image-level labels, is first proposed for detection of tiny casting defects in a complicated industrial scene. Then, in this article, we present a novel training strategy which can form a new object-level attention mechanism for the model during the training phase, and bilinear pooling is utilized to improve the model capability of detecting local contrast casting defects. Moreover, to enhance the interpretability, we extend class activation maps (CAM) to bilinear CAM (Bi-CAM) which is adapted to bilinear architectures as a visualization technique to reason about the model output. Experimental results show that the proposed model achieves superior performance in terms of each quantitative metric and is suitable for most actual applications. The real-time defect detection of castings is efficiently implemented in the complex scenario.

Deep face recognition: A survey

Deep learning applies multiple processing layers to learn representations of data with multiple levels of feature extraction. This emerging technique has reshaped the research landscape of face recognition (FR) since 2014, launched by the breakthroughs of DeepFace and DeepID. Since then, deep learning technique, characterized by the hierarchical architecture to stitch together pixels into invariant face representation, has dramatically improved the state-of-the-art performance and fostered successful real-world applications. In this survey, we provide a comprehensive review of the recent developments on deep FR, covering broad topics on algorithm designs, databases, protocols, and application scenes. First, we summarize different network architectures and loss functions proposed in the rapid evolution of the deep FR methods. Second, the related face processing methods are categorized into two classes: “one-to-many augmentation” and “many-to-one normalization”. Then, we summarize and compare the commonly used databases for both model training and evaluation. Third, we review miscellaneous scenes in deep FR, such as cross-factor, heterogenous, multiple-media and industrial scenes. Finally, the technical challenges and several promising directions are highlighted.

Detection and Segmentation of Unlearned Objects in Unknown Environment

Detecting and segmenting unlearned objects in unknown environment is a very important visual perception ability to enhance industrial intelligence. In this article, we present a novel conditional random field model integrating unimodal and cross-modal terms for detecting and segmenting object instances without knowing their categories and without sampling extra proposals. This model takes a paired image and point cloud as input, from which we first develop a set of novel category-independent features to distinguish objects. Then, a set of unary, pairwise, and higher order potentials are designed according to these category-independent features, and the cross-modal potential is introduced as a novel global constraints to keep the spatial consistency in both 2-D and 3-D modalities. In this novel model, the unlearned object detection and segmentation is treated as the process of pixel labeling. Thus, adjacent or occlusion object instances can also be separated efficiently from a labeled map. By comparison with the baseline methods, experimental results on a public RGB+D dataset show that the proposed model can obtain better performance with improved precision and recall rate. Moreover, we use the proposed method in a real industrial scene and achieve satisfactory performance.

Na‐Ion Batteries—Approaching Old and New Challenges

AbstractThe last 10 years established the beginning of a post‐lithium era in the field of energy storage, with the renaissance of Na‐ion batteries (NIBs) as alternative for Li‐based systems. The development of this technology has required intense work in materials research in order to produce and optimize anodes, cathodes, and electrolytes for NIBs. The strong and weak points of the main families of compounds for each battery component are analyzed in this progress report. Taking into account the achievements made in materials for NIBs, the industrial scene is analyzed through the existing prototypes and commercial cells and also from an economical viewpoint. In this scenario, where Na‐ion technology seems to be ready for a coming second generation, the use of Na can be extended to almost the whole spectrum of electrochemical energy storage systems: the new room temperature Na–S systems, high‐energy Na–air technology, or high‐power Na‐based hybrid supercapacitors. Thus, the degree of development of NIBs, together with the promising performance of newer Na‐based energy storage systems, makes Na the key to the coming commercial post‐lithium systems.

LARGE SCALE 3D POINT CLOUD MODELING FROM CAD DATABASE IN COMPLEX INDUSTRIAL ENVIRONMENTS

Abstract. In this paper, we present a method to build Computer Aided Design (CAD) representations of dense 3D point cloud scenes by queries in a large CAD model database. This method is applied to real world industrial scenes for infrastructure modeling. The proposed method firstly relies on a region growing algorithm based on novel edge detection method. This algorithm is able to produce geometrically coherent regions which can be agglomerated in order to extract the objects of interest of an industrial environment. Each segment is then processed to compute relevant keypoints and multi-scale features in order to be compared to all CAD models from the database. The best fitting model is estimated together with the rigid six degree of freedom (6 DOF) transformation for positioning the CAD model on the 3D scene. The proposed novel keypoints extractor achieves robust and repeatable results that captures both thin geometrical details and global shape of objects. Our new multi-scale descriptor stacks geometrical information around each keypoint at short and long range, allowing non-ambiguous matching for object recognition and positioning. We illustrate the efficiency of our method in a real-world application on 3D segmentation and modeling of electrical substations.

Semantic Loop Closure Detection With Instance-Level Inconsistency Removal in Dynamic Industrial Scenes

A novel semantic loop closure detection (SLCD) method is proposed in this article for visual simultaneous localization and mapping systems. SLCD aims to relieve the instance-level semantic inconsistency issue that arose from dynamic industrial scenes (e.g., autonomous driving in big cities). As the first step in this direction, SLCD fully exploits both low- and high-level video frame information, in a coarse-to-fine way. In SLCD, we adopt a convolutional neural network based object detection to acquire object information from the consecutive frames. Meanwhile, we perform a bag of visual words based similarity calculation to narrow the frames to coarse loop closure candidates. For these candidates, we perform an object matching on them to find their semantic inconsistency cases and remove involved semantic inconsistencies according to their cases. Then, we recalculate the similarity scores for these candidates. Finally, loop closures are determined by the similarity scores and a geometrical verification. Favorable performance of the proposed method is demonstrated by comparing it to other state-of-the-art methods using data from several public datasets and our new Dynamic Scenes dataset.

A situational study on sustainable housing features in Johor

Sustainable housing brings an innovation in Malaysia housing industry scene. The trend has witnessed a surge in the demand for this offering by house buyers which in return has encouraged housing developers to incorporate it’s features in their products. The features offered by the housing developers varies while customers are encouraged by the noble causes that their purchased will make. As demand for the products increase, errant and irresponsible developers are unreasonably taking the advantage by charging exorbitantly for the product that they market. In this instance, the words “Eco” and “Green” have been used as the gimmick for marketing purposes while there are no evidences that show whether the features have been truly incorporated in the housing products sold to the customers. The situation thus called for a situational study to be carried out, which aim to clarify the situation happening in the housing industry. Accordingly, the objectives of the study are: (1) To identify the sustainable housing features provided by developers to the buyers; and (2) To identify the house occupant’s satisfaction with the sustainable features available to them. The research is an exploratory research which seeks to provide data in respond to the situation facing the industry. Therefore, both quantitative and qualitative research strategies will be employed by way of a checklist together with a satisfaction survey focusing on housing projects in Johor. Data for the study will be analyzed by employing descriptive analysis as well as thematic coding analysis. A comprehensive discussion will be offered as the possible outcome of the study. This includes the contentment of house buyers towards the sustainable features that are offered by housing developer and how it can be classified as sustainable housing in virtue of the three pillars of sustainable development.

Future trends in I&amp;M: Human-machine co-creation in the rise of AI

The emergence of Artificial Intelligence (AI) is creating new dimensions and redefining the concept and meaning of work in industrial settings. Documented success has been reported where AI is transforming industrial scenes such as scaling large operation processes, speed of execution, flexibility of processes where rigid manufacturing by dumb robots is replaced with smart individualized production following real-time customer choices, decision-making in which a huge amount of data can be quickly available at the fingertips of workers on the factory floor or even prevent problems before they happen, and personalization where AI uses data to deliver personalized user experience. According to the market research firm Trac tica, the global AI software market is expected to experience massive growth in the coming years, with revenues increasing from around US $9.5 billion in 2018 to an expected US $118.6 billion by 2025.

IFinger: Intrusion Detection in Industrial Control Systems via Register-Based Fingerprinting

Nowadays, the industrial control system (ICS) plays a vital role in critical infrastructures like the power grid. However, there is an increasing security concern that ICS devices are being vulnerable to malicious users/attackers, where any subtle changing or tampering attack would cause significant damage to industrial manufacturing. In this paper, we propose the iFinger , a novel detection approach designed to mitigate ICS attacks adapting to various industrial scenes. We take advantage of an important insight that industrial protocol packets include register status values that are used to reflect the physical characteristics of ICS controllers. The iFinger utilizes register states to generate ICS fingerprints to detect malicious attacks on industrial networks. Specifically, the boolean logic represents every register state sequence of the ICS controller, and the deterministic finite automaton (DFA) generates a device fingerprint. To discover the ICS attacks, we propose two detection approaches based on device fingerprints, including passive and active detection. We present a prototype of the iFinger and conduct real-world experiments to validate its performance. Results show that our approach achieves 97.1% F1 score in ICS device identification. Furthermore, we simulate two typical ICS attacks (replacement and code modification) to validate the effectiveness of our iFinger in industrial networks. Our device fingerprints would detect those malicious attacks within 2s latency at 98.0% recall.

SYOLO: An Efficient Pedestrian Detection

Pedestrian detection is an important branch of object detection. It plays a vital role in many fields such as intelligent monitoring systems. The premise of the pedestrian recognition algorithm in the application of the industrial scene is accurate pedestrian detection. Our paper proposes a model to solve real-time pedestrian detection with high accuracy base on YOLO v3. We provide a method to select the size and number of anchor boxes for predicting bounding boxes accurately. Then we use a modified shuffle unit to lightweight the backbone of YOLO v3, which reduces the 67.3% FLOPs and 65.1% parameters. We train and validate our model on CrowdHuman detection dataset, SYOLO gets 62.7 mAP for face and 62.0mAP person with 0.748 average IOU. Our network processes images in real-time at 185.8 FPS for network and 12.3 FPS for the entire model on CrowdHuman.

Bayer Marker Detection based on Yolov3

With the development of artificial intelligence technology, target detection is widely used in many commercial fields, such as automatic driving, security monitoring and robotics, especially the deep learning algorithm has brought about a significant improvement in target detection performance. Bay number identification has a wide range of applications in industrial scene understanding. It can be used for character positioning and is applied to driverless. The accurate detection of shell number is the basis for subsequent character recognition. Character recognition is after character detection. In this paper, yolov3 target detection algorithm is used to realize the shell number text detection. Compared with yolov2 algorithm, yolov3 uses multi-scale prediction, and adopts feature maps of different scales to extract image features to adapt to different sizes of objects, and to use better. The basic classification network (ResNet-like) and classifiers use multiple logistic classifiers instead of the sorttmax classifier to solve the problem of multi-label classification.

SmartPipe: Towards Interoperability of Industrial Applications via Computational Reflection

With the advancement of new information technologies, a revolution is being taken place to bring the industry into a new era of intelligent manufacturing. One of the key requirements of intelligent manufacturing is the interoperability of industrial applications. However, it is challenging to realize the interoperability for legacy industrial applications due to 1) the deficient semantic information of data transmitted over heterogeneous communication protocols, 2) the difficulty to understand the complex process of business logic with no source code, and 3) the high cost and potential risk of reengineering the applications. To address the issues, in this paper, we propose an approach named SmartPipe to exposing existing functionalities of an industrial application as APIs without source code while simultaneously allowing the application to remain unchanged. We design a behavioral runtime model (BRM) as the self-representation of the industrial applications, based on which a computational reflection framework is designed to flexibly construct the model and generate APIs that encapsulate specific functionalities. We validate SmartPipe on a real industrial application that controls the spin-draw winding machine. Results show that our approach is effective and more suitable for industrial scenes compared with traditional approaches.

Training with Virtual Learning Environments for Industrial Refrigeration Systems

This article describes an application for the training of engineering students and industrial refrigeration plants, based on virtual learning environments. The application developed in the Unity 3D engine allows the interaction and immersion user and environment; through the modeling of a cooling system by steam compression, the inclusion of disturbances using Matlab software that allows the manipulation of dependent variables to have a more realistic background in combination with the bilateral communication TCP / IP protocol. The exchange of information in real time between the two programs is essential because feedback between user and environment is needed, the user manipulates the system while updating its parameters according to the thermodynamic variables evaluated by Matlab. The application presents three training modalities, i) Pedagogical scene: conventional refrigeration module, ii) Industrial scene: real refrigeration installation, and iii) Failure mode; in order to relate the environments and integrate the user for that improve competencies and skills through the manipulation, supervision, and operation of processes. The result of this study is a virtual application based on the characteristics of virtual learning environments. Finally, the virtual application is validated with respect to a real one, disturbances influence the performance of the system is analyzed and the interaction with the environment is tested.

Survey of imitation learning for robotic manipulation

With the development of robotics, the application of robots has gradually evolved from industrial scenes to more intelligent service scenarios. For multitasking operations of robots in complex and uncertain environments, the traditional manual coding method is not only cumbersome but also unable to adapt to sudden changes in the environment. Imitation learning that avoids learning skills from scratch by using the expert demonstration has become the most effective way for robotic manipulation. The paper is intended to provide the survey of imitation learning of robotic manipulation and explore the future research trend. The review of the art of imitation learning for robotic manipulation involves three aspects that are demonstration, representation and learning algorithms. Towards the end of the paper, we highlight areas of future research potential.

Dual surface structured light vision system based on multidimensional parameter coding.

A dual surface structured light vision system based on the theory of multidimensional parameter coding is proposed in this paper that can provide a new high-performance and adaptive encoding robot vision perception system. Specifically, the dual surface structured light consists of an auxiliary light source and a principal structured light. Scene images from the auxiliary structured light determine the parameters of the principal structured light. The vision process system can adaptively encode the principal structured light according to types and numbers of parameters. The image formed by the encoded principal structured light can reflect the depth change details of the target object and guide the robot to locate the target object quickly and accurately. The simulation and industrial experiments show that the dual surface structured light vision system has wider application scenarios and higher accuracy than the traditional structured light vision system. According to different industrial scenes, the positioning accuracy of the vision system can reach 0.1mm to 20μm in 0.8s.