Augmented Technology Research Articles

Human Augmentation Technologies for Employee Well-Being: A Research and Development Agenda.

The COVID-19 pandemic has changed the style of work. In adapting to the changing work environment, human augmentation technologies (HAT) can provide employees with new options to support their work. However, the agenda for research and development of HAT for the new normal is still unclear. In this study, we set two research questions: (i) what type of technology demand has emerged among employees due to the COVID-19 pandemic; and (ii) what is the nature of job satisfaction experienced by employees during the COVID-19 pandemic? This study aims to clarify the technology demand and job satisfaction of employees during the COVID-19 pandemic. We analyzed data from in-depth interviews with employees based on a grounded theory approach to answer the research questions and proposed an agenda for the research and development of HAT to enhance employees’ well-being in this new normal based on the crosspoint of technology demand and job satisfaction. The theoretical contribution of this study is the development of models of technology demand and job satisfaction of employees during the COVID-19 pandemic. The practical contribution is the development of a crosspoint framework to enable the development of HAT to support work while considering their impact on employees’ well-being.

KEIROKA Assistance and Human Augmentation to Optimize Human Performance

This paper introduces two concepts, KEIROKA assistance and human augmentation, which aim to maximize and utilize human abilities, and describes the Smart Suit as a technology to realize these concepts. The expected functions of actuators used in the technologies for KEIROKA assistance and human augmentation are also described.

Research on Tibetan Speech Recognition Based on the Am-do Dialect

In China, Tibetan is usually divided into three major dialects: the Am-do, Khams and Lhasa dialects. The Am-do dialect evolved from ancient Tibetan and is a local variant of modern Tibetan. Although this dialect has its own specific historical and social conditions and development, there have been different degrees of communication with other ethnic groups, but all the abovementioned dialects developed from the same language: Tibetan. This paper uses the particularity of Tibetan suffixes in pronunciation and proposes a lexicon for the Am-do language, which optimizes the problems existing in previous research. Audio data of the Am-do dialect are expanded by data augmentation technology combining noise and reverberation, and the morphological characteristics and characteristics of the Tibetan language are further considered. According to the particularity of Tibetan grammar, grammatical features are used to optimize grammatical relationships and are combined with a language model, and the Am-do dialect is scored and rescored. Experimental results show that compared with the baseline, our proposed new lexicon and data augmentation technology yields a relative increase of approximately 3% in character error rates (CERs) and a relative increase of 3%–19% in the recognition rate of acoustic models and language models.

A Defect Detection Method for a Boiler Inner Wall Based on an Improved YOLO-v5 Network and Data Augmentation Technologies

A Defect Detection Method for a Boiler Inner Wall Based on an Improved YOLO-v5 Network and Data Augmentation Technologies

The Design and Realization of Intelligent Classification of Rock Samples Based on Convolutional Neural Network

With substantial improvement of computer hardware performance and rapid development in artificial intelligence research, artificial intelligence recognition technology has been applied in many industries, such as medicine, education, industry, commerce, exploration and other professional fields. In geological exploration, the classification and identification of rock samples is an important link in geological analysis. However, the traditional manual classification faces high economic and time costs, and is vulnerable to subjective judgment which may affect the recognition result. In order to avoid these problems, research is carried out on the specific application of computer-based artificial intelligent recognition technology in recognition of rock images and the performance difference caused by different parameters. The sample data used in the experiment are lithic fragments and core samples capture by industrial cameras at the well logging site, with a total of 350 images which cover 7 types of rock samples. In this paper, the 350 rock images are divided into training set, validation set and test set based on certain proportion. Since the data set is too small and the quantity of each type of rocks varies, data augmentation technology is used to expand the original data before training. During the expansion, the quantity of different types of rock samples are balanced. At the training stage, the open-source deep learning framework Pytorch is used to construct a multi-layer convolutional neural network for learning and classification training on the training set. The best accuracy rate in the tests reaches 82.86%. However, the recognition rate is poor for stretched images, which required further optimization of the network structure. Based on the experimental results, the optimization orientations of neural network training are summarized, including improvement of data set quality, optimization of network structure, and adjustment of training strategies. In order to display the experimental data as intuitively as possible, data visualization tools such as PlotNeuralNet and Python matplotlib are used in the experiment for specific visualization work, after which the experimental data and model effect are analyzed from the perspective with combination of data and images.

Intelligent Crack Detection and Quantification in the Concrete Bridge: A Deep Learning‐Assisted Image Processing Approach

We proposed a modified concrete bridge crack detector based on a deep learning‐assisted image processing approach. Data augmentation technology is introduced to extend the limited dataset. In our proposed method, the bounding box for the crack is detected by YOLOv5. Then, the image covered by the bounding box is processed by the image processing techniques. Compared with the conventional image processing‐based crack detection method, the deep learning‐assisted image processing approach leads to higher detection accuracy and lower computation cost. More precisely, the mask filter is employed to remove handwritten marks, and the ratio filter is adopted to eliminate speckle linear noises. When a single crack is detected by several bounding boxes, we proposed a novel fusion method to merge these bounding boxes. Furthermore, we proposed a connected component search approach based on the crack trend of the area to improve the connection accuracy. With the crack detector, the cracks that are wider than 0.15 mm can be correctly detected, quantified, and visualized. The detection absolute error of the crack width is less than 0.05 mm. Thus, we realized fast and precise detection and quantification of bridge crack based on the practical engineering dataset.

Engineering Diplomacy: How AI and Human Augmentation Could Remake the Art of Foreign Relations

Through collaborations among diplomats, scientists, and engineers, authors envision artificial intelligence (AI) paired with emerging human augmentation technologies significantly improving the bandwidth, speed, and optimality of diplomacy.

Application of transfer learning and image augmentation technology for tomato pest identification

Traditional methods of identifying pests are time consuming and limited. To solve these problems, a set of reliable prediction models should be established to correctly identify tomato pests. This study established a tomato pest image dataset of common tomato pests. Deep learning (DL) convolutional neural network (CNN) models were applied to classify eight categories of tomato pests. Transfer learning was used to reduce training time. The DL models were used to extract features, and the extracted pest features were combined with three machine learning (ML) classifiers, including discriminant analysis (DA), support vector machine, and k-nearest neighbor method (KNN). Hyper-parameters were automatically optimized through Bayesian optimization. After image augmentation, VGG16 model exhibited the highest performance than the other models, with an accuracy of 94.95 %. Regarding the CNN + ML models, the ResNet50 with discriminant analysis model achieved 97.12 % classification accuracy. Combining the advantages of DL and ML could not only simplify the feature extraction process and reduce the training time, but also provide experts and farmers with effective and immediate assistance in identifying pests, which could help reduce economic and crop yield loss.

New Online Shunt Acidification for Water Injection Increasing Technology and Its Application in Huanjiang Oilfield

The poor physical property and strong heterogeneity of Triassic Yanchang formation in Huanjiang oilfield of Ordos Basin are the main reasons for uneven water absorption, partial injection wells underinjection at high pressure, and decline of production. Previously, large numbers of conventional acidifications were used for plugging removal in the reservoir, but the effect was not so good and effective period was short. Aiming at the geological characteristics of Huanjiang oilfield, an online shunt acidification and augmented injection technology which does not stop water injection, pull original production strings out, and continuously inject acid and diverting agent has been proposed. A chelating acid COA-1S with low corrosion rate (0.3675 g/(m2·h)), good retardation capacity (hydrolysis constant = 1.2 × 10−6), and effective chelating ability (precipitation inhibition rate >95%) has been developed, as well as a diverting agent COA-1P with good dispersion in acid solution, diversion effect, and particle size (10–100 μm), which behaves well in COA-1S acid. It has been proved that the online acid system has a good diversion acidizing ability and plugging removal performance in a deep area in the laboratory core physical simulation test. The field test results show that the online shunt acidizing and augmented injection technology could reduce the injection pressure significantly (4.2 MPa) and increase water injection by 10 m3/d for the measured well (H5) and improve the water injection profile prominently. The online shunting acidification and augmented injection technology have the following advantages: simple procedures, fewer equipment needed, high efficiency of depressurization, and increasing water injection, which could effectively improve the profile of water wells, and there is a bright future of the technology.

Augmented Workforce Canvas: a management tool for guiding human-centric, value-driven human-technology integration in industry

The Fourth Industrial Revolution is transforming the roles of humans, technology and work in production systems. Although the level of automation on shop floors increases, it is not always feasible or effective to assign the execution of production tasks to autonomous systems. Where total automation is ineffective, new forms of Human-Technology Integration (HTI), Operator Assistance Systems (OAS) and Augmentation Technologies can empower the workforce and increase overall business productivity. OAS can be conceptualized as systems that interact with operators to augment their cognitive and physical capabilities whilst performing industrial work tasks, resulting in HTI. In order to achieve successful HTI, an organization must make appropriate strategic and operational decisions for its business environment while also effectively managing its operations. Identifying and executing effective managerial decisions and activities for HTI can be aided by using business support, i.e., management tools. However, in industry, the systematic guidance of practitioners in HTI activities remains challenging. This paper addresses the challenge of guiding practitioners in HTI by introducing the Augmented Workforce Canvas (Canvas). Developed through Procedural Action Research (PAR), the Canvas is a strategic technology management tool aimed at systematically guiding users through the complex transformation towards HTI and the future of work on the shop floor. The Canvas takes a value-driven, technology-neutral approach to HTI. The tool begins with an assessment by key stakeholder groups of the set of underlying problems and the required added value of the HTI. It can be used as a methodological framework for industrial researchers to identify their respective contributions to the overall context of HTI in industrial contexts. Building on 39 industry expert interviews, the Canvas was co-designed and developed in 13 workshop sessions, refined and tested in 12 working sessions by industry practitioners, and evaluated in three industry case studies. The Canvas contributes to the development of industrial human-technology systems by placing the value-added for production systems at the heart of management decisions.

Impact of video content and resolution on the cognitive dynamics of surveillance decision‐making

AbstractSecurity screeners identify targets from surveillance videos. Their performance can fluctuate as resolution or video content salience varies. Image spatial resolution may be degraded if a camera has low pixel density. Similarly, temporal resolution can be degraded due to transmission interference or low bandwidth. Both types of deteriorated resolution can increase observer uncertainty and reduce target identification accuracy. Further, these outcomes may reflect an interaction between target type and resolution. To help quantify, study, and explain possible interactions, we utilize process‐tracing methodology to understand the cognitive dynamics of surveillance decisions. These insights can improve the development of surveillance augmentation technologies and processes. Mouse tracking is a robust process‐tracing method successfully leveraged to observe continuously unfolding cognitive processes. Lateral vacillating mouse movements reflect decision uncertainty, and trajectory curvature serves measures cognitive conflict to select an unchosen alternative. We utilized mouse tracking to measure decision conflict during a video surveillance task with varying salience due to resolution or target type. Greater cognitive conflict was found when observers classified person‐related targets compared to vehicle‐related targets. Trajectories indicated stronger certainty when classifying events as having occurred, rather than when indicating events did not occur. Greater cognitive conflict was seen under degraded spatial resolution when observers were moderately confident compared to strongly confident or reported themselves as guessing. Under degraded temporal resolution, the greatest cognitive conflict occurred when observers reported they were guessing. Beyond these findings, we have demonstrated the viability of mouse tracking as an unobtrusive process‐tracing method for measuring uncertainty during realistic surveillance.

Rapid positioning method of power line inspection unmanned aerial vehicle assisted by BeiDou ground-based augmentation station

BeiDou high-precision power line inspection can improve the efficiency and automation level of inspection operations, reduce labor intensity, and save inspection costs. However, although the satellite-based augmentation technology can solve the problem of large-scale positioning, it will introduce too many parameters into the positioning system through the method of parameter estimation. As a result, the behavior of the equation deteriorates, which makes it difficult to separate some parameters, and the convergence time of ambiguity parameters is too long. Under normal circumstances, the satellite-based augmentation standard positioning service needs at least about 30 min to converge to the centimeter level, and the ground-based augmentation system can provide an accurate model. This paper proposes a fast positioning method for satellite-based augmentation assisted by the BeiDou ground-based augmentation station. The receiver adopts the least square method, combined with the extended Kalman filter positioning, to achieve the improvement of the regional positioning effect of the satellite-based augmentation system. Positioning accuracy error does not exceed 1.6 meters. The error in the three directions of X, Y, and Z can be converged to the centimeter level in about 8 min. It can significantly improve the convergence time of satellite-based augmentation, and maintain continuous, stable and high-precision positioning of unmanned aerial vehicle in power line inspection.

Special Issue on Augmenting the Human Body and Being

Information technologies, such as IoT, artificial intelligence (AI), and virtual reality (VR), have seen so much development that there is now a wide variety of digital equipment incorporated into the infrastructure of daily life. From the agrarian society (Society 1.0) through the information society (Society 4.0), humankind has created farmlands and cities by structuring natural environments physically and has built information environments by structuring them informationally. However, despite the rapid development of information environments, it may be fair to say that the perspectives of the human body have not changed at all since the industrial revolution. In the context of these recent technological developments, greater attention is being paid to human augmentation studies. These studies aim for a new embodiment of “human-computer integration,” one which can physically and informationally compensate or augment our innate sensory functions, motor functions, and intellectual processing functions by using digital equipment and information systems at will, as if they were our hands and feet. It has also been proposed that the technical systems that enable us to freely do what we want by utilizing human augmentations be called “JIZAI” (freedomization) as opposed to “automation.” The term “JIZAI body” used in these studies represents the new body image of humans who will utilize engineering and informatics technologies to act at will in the upcoming “super smart society” or “Society 5.0.” In these studies, human augmentation technologies are an important component of JIZAI, but JIZAI is not the same as human augmentation. JIZAI is different in scope from human augmentation, as it aims to enable humans to move freely among the five new human body images: “strengthened sense” (augmented perception), “strengthened physical body” (body augmentation), “separately-designed mind and body” (out of body transform), “shadow cloning,” and “assembling.” In the society of the future where JIZAI bodies widely prevail, we will use technologies that enable us to do what we have failed at or given up due to limitations of our physical bodies. We believe that a future society, one in which aging does not reduce our capabilities but instead increased options give us hope, can be realized. This special issue, consisting of two review papers and twelve research papers, deals with diverse and wide-ranging areas, including human augmentation, robotics, virtual reality, and others. We would like to express our sincere appreciation to all the authors and reviewers of the papers contributed to this special issue and to the editorial committee of the Journal of Robotics and Mechatronics for their gracious cooperation.

Automatic Identification and Quantitative Characterization of Primary Dendrite Microstructure Based on Machine Learning

Dendrites are important microstructures in single-crystal superalloys. The distribution of dendrites is closely related to the heat treatment process and mechanical properties of single-crystal superalloys. The primary dendrite arm spacing (PDAS) is an important length scale to describe the distribution of dendrites. In this work, the second-generation single crystal superalloy HT901 with a diameter of 15 mm was imaged under a metallurgical microscope. An automatic dendrite core identification and full-field quantitative statistical analysis method is proposed to automatically detect the dendrite core and calculate the local PDAS. The Faster R-CNN algorithm combined with test time augmentation (TTA) technology is used to automatically identify the dendrite cores. The local multi-directional algorithm combined with Voronoi tessellation is used to determine the local nearest neighbor dendrite and calculate the local PDAS and coordination number. The accuracy of using Faster R-CNN combined with TTA to detect the dendrite core of HT901 reaches 98.4%, which is 15.9% higher than using Faster R-CNN alone. The algorithm calculates the local PDAS of all dendrites in H901 and captures the Gaussian distribution of the local PDAS. The average PDAS determined by the Gaussian distribution is 415 μm, which is only a small difference from the average spacing λ¯ (420 μm) calculated by the traditional method. The technology analyzes the relationship between the local PDAS and the distance from the center of the sample. The local PDAS near the center of HT901 are larger than those near the edge. The results suggests that the method enables the rapid, accurate and quantitative dendritic distribution characterization.

MEAN-SSD: A novel real-time detector for apple leaf diseases using improved light-weight convolutional neural networks

Alternaria blotch, Brown spot, Mosaic, Grey spot, and Rust are 5 common apple leaf diseases that severely impact apple production and quality. At present, although many CNN methods have been proposed for apple leaf diseases, there are still lack of apple leaf disease detection models that can be applied on mobile devices, which limits their application in practical production. This paper proposes a light-weight CNN model that can be deployed on mobile devices to detect apple leaf diseases in real time. First, a dataset of apple leaf diseases composed of simple background images and complex background images, which is called AppleDisease5, is constructed via data augmentation technology and data annotation technology. Then a basic module called MEAN block(Mobile End AppleNet block) is proposed to increase the detection speed and reduce model’s size by reconstructing the common 3×3 convolution. Meanwhile, the Apple-Inception module is built by introducing GoogLeNet’s Inception module and replacing all 3×3 convolution kernels with MEAN block in Inception. Finally, a new apple leaf disease detection model, MEAN-SSD(Mobile End AppleNet based SSD algorithm), is built by using the MEAN block and Apple-Inception module. The experiment results show that MEAN-SSD can achieve the detection performance of 83.12% mAP and a speed of 12.53 FPS, which illustrates that the novel MEAN-SSD model can efficiently and accurately detect 5 common apple leaf diseases on mobile devices.

Clinical Study of Virtual Reality Augmented Technology Combined with Contrast-Enhanced Ultrasound in the Assessment of Thyroid Cancer.

Thyroid cancer has become the most common malignant tumor in the endocrine system, and its global incidence has been showing an upward trend. The diagnosis methods of thyroid cancer include ultrasound, fine-needle aspiration cytology, and neck CT, but the single ultrasound feature cannot simultaneously take into account the sensitivity and specificity of more than 85% when diagnosing thyroid cancer. The development of virtual technology can significantly improve the diagnosis of the thyroid gland. Based on this, this article proposes a clinical study of virtual reality technology combined with contrast-enhanced ultrasound in the assessment of thyroid cancer. This article uses a variety of methods, such as literature method, mathematical statistics, and experimental research, in-depth study of the theoretical cornerstones of virtual reality augmented technology, the application status of ultrasound contrast technology, and so on. And a fuzzy mean clustering algorithm was proposed to identify ultrasound images. Then, a clinical experiment of virtual reality augmented technology combined with contrast-enhanced ultrasound was designed to evaluate thyroid cancer, which included comparison of contrast-enhanced ultrasound signs, analysis of enhancement results, multifactor logistic analysis, and diagnostic efficacy analysis of ultrasound signs. The combined application of virtual reality augmented technology and contrast-enhanced ultrasound in the study of thyroid cancer has a sensitivity and specificity exceeding 85% as the diagnosis boundary changes, and the accuracy of the combined diagnosis is relatively high.

Construction of Genealogical Knowledge Graphs From Obituaries: Multitask Neural Network Extraction System.

BackgroundGenealogical information, such as that found in family trees, is imperative for biomedical research such as disease heritability and risk prediction. Researchers have used policyholder and their dependent information in medical claims data and emergency contacts in electronic health records (EHRs) to infer family relationships at a large scale. We have previously demonstrated that online obituaries can be a novel data source for building more complete and accurate family trees.ObjectiveAiming at supplementing EHR data with family relationships for biomedical research, we built an end-to-end information extraction system using a multitask-based artificial neural network model to construct genealogical knowledge graphs (GKGs) from online obituaries. GKGs are enriched family trees with detailed information including age, gender, death and birth dates, and residence.MethodsBuilt on a predefined family relationship map consisting of 4 types of entities (eg, people’s name, residence, birth date, and death date) and 71 types of relationships, we curated a corpus containing 1700 online obituaries from the metropolitan area of Minneapolis and St Paul in Minnesota. We also adopted data augmentation technology to generate additional synthetic data to alleviate the issue of data scarcity for rare family relationships. A multitask-based artificial neural network model was then built to simultaneously detect names, extract relationships between them, and assign attributes (eg, birth dates and death dates, residence, age, and gender) to each individual. In the end, we assemble related GKGs into larger ones by identifying people appearing in multiple obituaries.ResultsOur system achieved satisfying precision (94.79%), recall (91.45%), and F-1 measures (93.09%) on 10-fold cross-validation. We also constructed 12,407 GKGs, with the largest one made up of 4 generations and 30 people.ConclusionsIn this work, we discussed the meaning of GKGs for biomedical research, presented a new version of a corpus with a predefined family relationship map and augmented training data, and proposed a multitask deep neural system to construct and assemble GKGs. The results show our system can extract and demonstrate the potential of enriching EHR data for more genetic research. We share the source codes and system with the entire scientific community on GitHub without the corpus for privacy protection.

A novel framework for image-based malware detection with a deep neural network

The rapid growth in the number of malware and its variants has seriously affected the security of the Internet. In recent years, deep learning combined with visualization technology has been proven to have good results in malware detection. In this paper, we propose a novel visual malware detection framework based on deep neural networks. Firstly, executable file samples are collected and converted into bytes files and asm files through disassembly technology. In this way, a balanced experimental dataset with our labeled normal software dataset and a widely used malware dataset (BIG 2015) is constructed. Secondly, visualization technology combined with data augmentation is used to further convert the samples into three-channel RGB images, so as to extract high-dimensional intrinsic features from data samples. Finally, we present a deep neural network architecture, i.e. SERLA (SEResNet50 + Bi-LSTM + Attention) to improve the performance of the detection method. After performance evaluation, the results show that our model stands out among other neural network models and state-of-the-art methods for malware detection and classification. Furthermore, our study verifies the superiority of three-channel RGB images compared to grayscale images in malware detection, compares the contribution of different channels, and indicates that data augmentation technology can contribute to malware recognition using visualization technology. This paper provides new ideas and methods for other researchers to carry out malware detection and classification.

Adoption of virtual reality technology in higher education: An evaluation of five teaching semesters in a purpose-designed laboratory.

Virtual and augmented (VAR) technology is in the early stages of being adopted as a teaching platform in higher education. The technology can facilitate immersive learning in environments that are not usually physically accessible to students via 3D models and interactive 360° videos. To date, adoption rates of VAR technology for teaching have not been well described across a higher education institution. Further, there is an absence of information on the optimal VAR laboratory designs and cost per student. In this study, a purpose designed virtual reality laboratory was formed in 2017 at The University of Sydney, housing 26 Oculus Rift headset units. An evaluation was conducted on the design, costs, rates of teaching adoption and student experiences over five teaching periods (2.5 years). Over this period, 4833 students were taught in the laboratory across 7952 student visits. The laboratory was used most by the Faculty of Engineering (53%), followed by the Faculty of Arts & Social Science (23.8%) and Faculty of Science (23.2%). For engineering, the units of study using the laboratory represented only 1.4% of annual faculty subjects offered. This confirms that adoption was in the initial stage of innovation diffusion. The laboratory saw a 250% increase in student numbers over the period of evaluation and 71.5% of students surveyed (n = 295) reported enhanced learning outcomes. The cost per visit was only AU$ 19.50. These findings give confidence to higher education institutions that the right VAR technology infrastructure is a sound educational investment for the future.Supplementary InformationThe online version contains supplementary material available at 10.1007/s10639-021-10653-6.

Review of heat transfer augmentation methods and effect of using dimpled ribs and nanofluids

The paper offers a study of previous development and existing standing of heat transfer augmentation technologies used in various thermal energy systems consisting of solar thermal system, heat exchangers, single and two-phase flow energy systems, and jet impingement cooling systems. Furthermore, literature concerning the heat transfer practices in dimpled surfaces as well as arrangements of dimples are offered. The effect of using nano fluid particle Al2O3CuO, Fe2O3,Fe3O4, SiO2, TiO2, and ZrO2, studied by various authors is reviewed. Lastly the work on combination of dimple surface with nanofluid is reviewed having heat transfer augmentation with negligible increase in friction factor.