High-level Tasks Research Articles

Design and implementation of robot assisted arduino based object recognition and sorting

This paper deals with an automatic material handling system that coordinates the movement of a robotic arm to pick up items moving on conveyor belts. The system utilizes advanced sensors and machine learning algorithms to ensure precise and efficient manipulation of objects, enhancing the overall automation and productivity of material handling processes. It aims to organize colored objects approaching on the conveyor by picking and placing them in separate, designated locations. The robotic system employs advanced computer vision algorithms to precisely identify and manipulate the diverse array of colored objects, ensuring efficient and accurate sorting on the conveyor belt. This reduces the tedious work done by humans, ensuring accuracy and rapidity in the process. Additionally, it paves the way for more efficient utilization of human resources, allowing professionals to focus on higher-level tasks that require creativity and critical thinking. The system includes color sensors that detect the items' colors and transmit signals to the controller. Additionally, the controller processes the signals from the color sensors to facilitate accurate identification and sorting of the items. The microcontroller then guides the signal to the motor driving circuit, which operates the different motors of the robotic arm to grasp the object and place it in the correct location. Additionally, the robotic arm's sophisticated sensor feedback system ensures precise positioning and adaptability to varying environmental conditions, enhancing its overall efficiency in object manipulation tasks. Depending on the color sensed, the robotic arm goes to the correct location to release the object and returns to its normal position. Additionally, the robotic arm employs advanced computer vision algorithms to precisely identify and differentiate colors, ensuring accurate execution of tasks in diverse environments.

Security-Preserving Multi-Robot Path Planning for Boolean Specification Tasks Using Labeled Petri Nets

This letter investigates the path planning of multi-robot systems for high-level tasks described by Boolean specifications and security constraints. We assume that the behavior of each robot can be identified and partially monitored by a passive intruder. The problem aims to plan an optimal path for each robot such that the tasks expressed in conjunction, disjunction, and negation for trajectories and final states are coordinately completed. The security constraints require that the intruder should never infer the final locations of a set of robots called secret robots. In order to solve this problem, labeled Petri nets are adopted to model the mobile capability of the multi-robot systems. Then an integer linear programming problem is proposed to find an optimal solution (if it exists) such that the Boolean specification is fulfilled, while the securities of secret robots are preserved. Finally, the effectiveness of the proposed method is illustrated through several simulation studies.

Optimal Probabilistic Motion Planning With Potential Infeasible LTL Constraints

This paper studies optimal motion planning subject to motion and environment uncertainties. By modeling the system as a probabilistic labeled Markov decision process (PL-MDP), the control objective is to synthesize a finite-memory policy, under which the agent satisfies complex high-level tasks expressed as linear temporal logic (LTL) with desired satisfaction probability. In particular, the cost optimization of the trajectory that satisfies infinite horizon tasks is considered, and the trade-off between reducing the expected mean cost and maximizing the probability of task satisfaction is analyzed. The LTL formulas are converted to limit-deterministic Büchi automata (LDBA) with a reachability acceptance condition and a compact graph structure. The novelty of this work lies in considering the cases where LTL specifications can be potentially infeasible and developing a relaxed product MDP between PL- MDP and LDBA. The relaxed product MDP allows the agent to revise its motion plan whenever the task is not fully feasible and quantify the revised plan’s violation measurement. A multi- objective optimization problem is then formulated to jointly consider the probability of task satisfaction, the violation with respect to original task constraints, and the implementation cost of the policy execution. The formulated problem can be solved via coupled linear programs. This work first bridges the gap between probabilistic planning revision of potential infeasible LTL specifications and optimal control synthesis of both plan prefix and plan suffix of the trajectory over the infinite horizons. Experimental results are provided to demonstrate the effectiveness of the proposed framework.

5-6학년군 수학 교과서에 제시된 과제의 인지적 노력 수준과 발문 유형 분석

Objectives The purposes of this study were to analyze the levels of cognitive demand and questioning types in textbook tasks presented in grade 5 and 6 mathematics textbooks.
 Methods In order to do this, grade 5 and 6 mathematics textbooks in the revised 2015 curriculum were selected and the levels of cognitive demand and questioning types, answer types, and response types in the textbook tasks were analyzed. In addition, types of sub-questions in high-level tasks were also analyzed.
 Results The results of this study showed that in both grade 5 and 6 mathematics textbooks the percentage of tasks requiring mathematical reasoning was the highest and the percentage of problem solving tasks was the lowest, and the percentage of high-level tasks was considerably low. Also, the analysis of types of sub-questions of high-level tasks showed that the percentage of reasoning questions were the highest, followed by the percentage of factual questions.
 Conclusions The results indicate that the increase of opportunities to learn to develop students’ mathematical thinking is necessary by implementing tasks requiring a variety of the breadth and depth of cognitive complexity evenly. It also suggests that attention should be paid not to lower the cognitive level of the task when constructing the sub-questions of high-level mathematical task.

TOOLTANGO: Common sense Generalization in Predicting Sequential Tool Interactions for Robot Plan Synthesis

Robots assisting us in environments such as factories or homes must learn to make use of objects as tools to perform tasks, for instance, using a tray to carry objects. We consider the problem of learning common sense knowledge of when a tool may be useful and how its use may be composed with other tools to accomplish a high-level task instructed by a human. Specifically, we introduce a novel neural model, termed TOOLTANGO, that first predicts the next tool to be used, and then uses this information to predict the next action. We show that this joint model can inform learning of a fine-grained policy enabling the robot to use a particular tool in sequence and adds a significant value in making the model more accurate. TOOLTANGO encodes the world state, comprising objects and symbolic relationships between them, using a graph neural network and is trained using demonstrations from human teachers instructing a virtual robot in a physics simulator. The model learns to attend over the scene using knowledge of the goal and the action history, finally decoding the symbolic action to execute. Crucially, we address generalization to unseen environments where some known tools are missing, but unseen alternative tools are present. We show that by augmenting the representation of the environment with pre-trained embeddings derived from a knowledge-base, the model can generalize effectively to novel environments. Experimental results show at least 48.8-58.1% absolute improvement over the baselines in predicting successful symbolic plans for a simulated mobile manipulator in novel environments with unseen objects. This work takes a step in the direction of enabling robots to rapidly synthesize robust plans for complex tasks, particularly in novel settings.

Model Predictive Control of Quadruped Robot Based on Reinforcement Learning

For the locomotion control of a legged robot, both model predictive control (MPC) and reinforcement learning (RL) demonstrate powerful capabilities. MPC transfers the high-level task to the lower-level joint control based on the understanding of the robot and environment, model-free RL learns how to work through trial and error, and has the ability to evolve based on historical data. In this work, we proposed a novel framework to integrate the advantages of MPC and RL, we learned a policy for automatically choosing parameters for MPC. Unlike the end-to-end RL applications for control, our method does not need massive sampling data for training. Compared with the fixed parameters MPC, the learned MPC exhibits better locomotion performance and stability. The presented framework provides a new choice for improving the performance of traditional control.

Mal_CNN: An Enhancement for Malicious Image Classification Based on Neural Network

In image analysis and processing, image segmentation is one of the most important functions. The outcomes of segmentation have such significance on all subsequent image analysis operations, covering object tracking and description, feature measurement, and even higher-level tasks like object recognition. Malicious code detection is becoming increasingly significant, and current models must be improved. Hence forth the Image segmentation in the field of Malware image classification is a significant task. The sectional structure or region of interest must be identified and extracted during the segmentation process so that it can be evaluated independently. There are various reviews stating the traditional approach of image segmentation in various fields. The necessity of image segmentation in malicious image is extracting data for classification using CNN is discussed. In this work we use malimg_paper_dataset_imgs with 9,339 malware images. Various segmentation techniques were used to enhance the malware images. Those enhanced malicious image were applied in CNN architecture and Mal_CNN for classification and a comparative result is been discussed. The malicious images in dataset after incorporating segmentation have achieved 95% of accuracy in CNN architecture and 97% with Mal_CNN.

A learning sequence for developing technology tasks that support students’ mathematical thinking: a replication study

This article describes an instructional sequence used with inservice secondary mathematics teachers as part of a graduate course. The instructional sequence focused on designing tasks using Interactive Geometry Software (IGS). This study answers a call for replication studies to investigate the use of an established instructional sequence. Results confirm previous findings when implementing the instructional sequence with preservice teachers and demonstrate that the instructional sequence supported inservice teachers in creating high-level tasks that use IGS as a reorganizer to support student thinking. Comparison of previous and current results point to several implications for potential revisions to improve the instructional sequence, as well as revisions to the IGS Framework.

MTL-Deep-STF: A multitask learning based deep spatiotemporal fusion model for outdoor air temperature prediction in building HVAC systems

Buildings consume large quantities of energy. Reducing building energy consumption is essential to achieving carbon neutrality goals. Building energy consumption is strongly influenced by various meteorological factors for which outdoor air temperature is a proxy. This study proposes a data-driven multitask-learning deep spatiotemporal fusion model (MTL-Deep-STF) that predicts outdoor air temperature to provide baseline data against which improvements to conserve energy in heating, ventilation, and air conditioning (HVAC) systems can be proposed and assessed. MTL-Deep-STF is a generic multiple input–multiple output model that predicts temperatures for successive time points in separate high-level tasks. Each task uses a multilayer perceptron to combine meteorological data from neighboring observation stations. CNN-GRU is used to extract spatially local long-term and short-term temporal features that are shared between different tasks via CNN. Comparative validation experiments using real world datasets for complex terrains showed that the model was highly accurate in both single step and multistep predictions. One-step-ahead predictions for winter (MAE = 0.428 °C, RMSE = 0.705, Acc ≤2 °C = 97.832%) and summer (MAE = 0.467 °C, RMSE = 0.706, Acc ≤2 °C = 97.899%) datasets were particularly accurate. This accuracy was coupled with high robustness ( R 2 = 0.976 for winter and R 2 = 0.981 for summer). Use of the model can conserve building energy. • The spatiotemporal fusion model MTL-Deep-STF uses data from neighboring stations to predict air temperature at multiple future time steps. • MTL-Deep-STF treats temperature prediction for successive prediction horizons as distinct tasks. • MTL-Deep-STF extracts global spatial features of multiple stations through fully connected networks. • MTL-Deep-STF is extremely accurate and robust in predicting outdoor temperatures and is therefore suitable for controlling HVAC systems, particularly in complex terrains.

Temporal Logic Task Allocation in Heterogeneous Multirobot Systems

We consider the problem of optimally allocating tasks, expressed as global linear temporal logic (LTL) specifications, to teams of heterogeneous mobile robots of different types. Each task may require robots of multiple types. To obtain a scalable solution, we propose a hierarchical approach that first allocates specific robots to tasks using the information about the tasks contained in the nondeterministic B <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\ddot{\text{u}}$</tex-math></inline-formula> chi automaton (NBA) that captures the LTL specification and then designs low-level paths for robots that respect the high-level assignment. Specifically, motivated by “lazy collision checking” methods in robotics, we first prune and relax the NBA by removing all negative atomic propositions, which simplifies the planning problem by checking constraint satisfaction only when needed. Then, we extract sequences of subtasks from the relaxed NBA along with their temporal orders and formulate a mixed integer linear program to allocate these subtasks to robots. Finally, we define generalized multirobot path planning problems to obtain low-level paths that satisfy both the high-level task allocation and the constraints captured by the negative atomic propositions in the original NBA. We show that our method is complete for a subclass of LTL that covers a broad range of tasks and present numerical simulations demonstrating that it can generate paths with lower cost, considerably faster than existing methods.

Skill-based design of dependable robotic architectures

Software architectures for autonomous systems are generally structured with 3 layers: a decisional layer managing autonomous reasoning, a functional layer managing reactive tasks and processing, and an executive layer bridging the gap between both. The executive layer plays a central role, as it links high-level tasks with low-level processing, and is generally responsible for the robustness or the fault-tolerance of the overall system. In this paper, we propose a development process for such an executive layer that emphasizes on the dependability of this layer. To do so, we structure the executive layer using skills, that are formally defined using a specific language, and we then provide some tools to verify these models, generate some code, and a methodology to assess the fault-tolerance of the resulting architecture.

Joint Semantic Deep Learning Algorithm for Object Detection under Foggy Road Conditions

Current mainstream deep learning methods for object detection are generally trained on high-quality datasets, which might have inferior performances under bad weather conditions. In the paper, a joint semantic deep learning algorithm is proposed to address object detection under foggy road conditions, which is constructed by embedding three attention modules and a 4-layer UNet multi-scale decoding module in the feature extraction module of the backbone network Faster RCNN. The algorithm differs from other object detection methods in that it is designed to solve low- and high-level joint tasks, including dehazing and object detection through end-to-end training. Furthermore, the location of the fog is learned by these attention modules to assist image recovery, the image quality is recovered by UNet decoding module for dehazing, and then the feature representations of the original image and the recovered image are fused and fed into the FPN (Feature Pyramid Network) module to achieve joint semantic learning. The joint semantic features are leveraged to push the subsequent network modules ability, and therefore make the proposed algorithm work better for the object detection task under foggy conditions in the real world. Moreover, this method and Faster RCNN have the same testing time due to the weight sharing in the feature extraction module. Extensive experiments confirm that the average accuracy of our algorithm outperforms the typical object detection algorithms and the state-of-the-art joint low- and high-level tasks algorithms for the object detection of seven kinds of objects on road traffics under normal weather or foggy conditions.

Multi-Vehicle Tracking Based on Monocular Camera in Driver View

Multi-vehicle tracking is used in advanced driver assistance systems to track obstacles, which is fundamental for high-level tasks. It requires real-time performance while dealing with object illumination variations and deformations. To this end, we propose a novel multi-vehicle tracking algorithm based on a monocular camera in driver view. It follows the tracking-by-detection paradigm and integrates detection and appearance descriptors into a single network. The one-stage detection approach consists of a backbone, a modified BiFPN as a neck layer, and three prediction heads. The data association consists of a two-step matching strategy together with a Kalman filter. Experimental results demonstrate that the proposed approach outperforms state-of-the-art algorithms. It is also able to solve the tracking problem in driving scenarios while maintaining 16 FPS on the test dataset.

Learning from Demonstrations in Human–Robot Collaborative Scenarios: A Survey

Human–Robot Collaboration (HRC) is an interdisciplinary research area that has gained attention within the smart manufacturing context. To address changes within manufacturing processes, HRC seeks to combine the impressive physical capabilities of robots with the cognitive abilities of humans to design tasks with high efficiency, repeatability, and adaptability. During the implementation of an HRC cell, a key activity is the robot programming that takes into account not only the robot restrictions and the working space, but also human interactions. One of the most promising techniques is the so-called Learning from Demonstration (LfD), this approach is based on a collection of learning algorithms, inspired by how humans imitate behaviors to learn and acquire new skills. In this way, the programming task could be simplified and provided by the shop floor operator. The aim of this work is to present a survey of this programming technique, with emphasis on collaborative scenarios rather than just an isolated task. The literature was classified and analyzed based on: the main algorithms employed for Skill/Task learning, and the human level of participation during the whole LfD process. Our analysis shows that human intervention has been poorly explored, and its implications have not been carefully considered. Among the different methods of data acquisition, the prevalent method is physical guidance. Regarding data modeling, techniques such as Dynamic Movement Primitives and Semantic Learning were the preferred methods for low-level and high-level task solving, respectively. This paper aims to provide guidance and insights for researchers looking for an introduction to LfD programming methods in collaborative robotics context and identify research opportunities.

Ontology-based surgical workflow recognition and prediction

BackgroundSurgical context-aware systems can adapt to the current situation in the operating room and thus provide computer-aided assistance functionalities and intraoperative decision-support. To interact with the surgical team perceptively and assist the surgical process, the system needs to monitor the intraoperative activities, understand the current situation in the operating room at any time, and anticipate the following possible situations. MethodsA structured representation of surgical process knowledge is a prerequisite for any applications in the intelligent operating room. For this purpose, a surgical process ontology, which is formally based on standard medical terminology (SNOMED CT) and an upper-level ontology (GFO), was developed and instantiated for a neurosurgical use case. A new ontology-based surgical workflow recognition and a novel prediction method are presented utilizing ontological reasoning, abstraction, and explication. This way, a surgical situation representation with combined phase, high-level task, and low-level task recognition and prediction was realized based on the currently used instrument as the only input information. ResultsThe ontology-based approach performed efficiently, and decent accuracy was achieved for situation recognition and prediction. Especially during situation recognition, the missing sensor information were reasoned based on the situation representation provided by the process ontology, which resulted in improved recognition results compared to the state-of-the-art. ConclusionsIn this work, a reference ontology was developed, which provides workflow support and a knowledge base for further applications in the intelligent operating room, for instance, context-aware medical device orchestration, (semi-) automatic documentation, and surgical simulation, education, and training.

Joint Framework for Single Image Reconstruction and Super-Resolution With an Event Camera.

Event cameras sense brightness changes in each pixel and yield asynchronous event streams instead of producing intensity images. They have distinct advantages over conventional cameras, such as a high dynamic range (HDR) and no motion blur. To take advantage of event cameras with existing image-based algorithms, a few methods have been proposed to reconstruct images from event streams. However, the output images have a low resolution (LR) and are unrealistic. Low-quality outputs stem from broader applications of event cameras, where high-quality and high-resolution (HR) images are needed. In this work, we consider the problem of reconstructing and super-resolving images from LR events when no ground truth (GT) HR images and degradation models are available. We propose a novel end-to-end joint framework for single image reconstruction and super-resolution from LR event data. Our method is primarily unsupervised to handle the absence of real inputs from GT and deploys adversarial learning. To train our framework, we constructed an open dataset, including simulated events and real-world images. The use of the dataset boosts the network performance, and the network architectures and various loss functions in each phase help improve the quality of the resulting image. Various experiments showed that our method surpasses the state-of-the-art LR image reconstruction methods for real-world and synthetic datasets. The experiments for super-resolution (SR) image reconstruction also substantiate the effectiveness of the proposed method. We further extended our method to more challenging problems of HDR, sharp image reconstruction, and color events. In addition, we demonstrate that the reconstruction and super-resolution results serve as intermediate representations of events for high-level tasks, such as semantic segmentation, object recognition, and detection. We further examined how events affect the outputs of the three phases and analyze our method's efficacy through an ablation study.

The influence of reward in the Simon task: Differences and similarities to the Stroop and Eriksen flanker tasks

Previous studies have suggested that performance-contingent reward can modulate cognitive control by biasing irrelevant location-response associations in the Simon task. However, the influence of reward in the case of irrelevant words (Stroop task) or irrelevant flankers (Eriksen Flanker task) remains unclear. Across two preregistered experiments, the present study investigated the influence of reward on conflict processing with different types of distractors. Conflict effects on mean reaction time (RT) were reduced in the Simon task (Experiments 1 and 2) when incongruent versus congruent trials were rewarded, and this modulating effect of reward on conflict processing was also observed in the Eriksen flanker task (Experiment 2), but not in the Stroop task (Experiment 1). We propose that cognitive control adjustments to distractor-specific reward contingencies can be generalized across distractor types producing both perceptual-related (Flanker task) and motor-related (Simon task) conflict, but, if any, to a limited degree when distractors produce additional higher-level task conflict (Stroop task). In addition, distributional RT analyses (delta plots) revealed that rewarded distractor-response associations modulate cognitive control not only via biasing the strength (Simon and Eriksen tasks) but also the time-course of suppressing distractor processing (Eriksen task). Overall, the present study dissociated distractor-general and distractor-specific effects of reward on cognitive control.

Foreground Detection Based on Superpixel and Semantic Segmentation.

Foreground detection is an essential step in computer vision and video processing. Accurate foreground object extraction is crucial for subsequent high-level tasks such as target recognition and tracking. Although many foreground detection algorithms have been proposed, foreground detection in complex scenes is still a challenging problem. This paper presents a foreground detection algorithm based on superpixel and semantic segmentation. It first uses multiscale superpixel segmentation to obtain the initial foreground mask. At the same time, a semantic segmentation network is applied to separate potential foreground objects, and then use the defined rules to combine the results of superpixel and semantic segmentation to get the final foreground object. Finally, the background model is updated with the refined foreground result. Experiments on the CDNet2014 dataset demonstrate the effectiveness of the proposed algorithm, which can accurately segment foreground objects in complex scenes.

Lean Design of the Pediatric Intensive Care Unit Patient Room for Efficient and Safe Care Delivery.

The pediatric intensive care unit (PICU) is an environment where seriously ill children receive complex care, delivered mostly by specialty-trained nurses (registered nurses [RNs]) who must perform multiple high-level tasks. With stressors on healthcare systems at an all-time high, design that optimizes RN workflow has taken on a renewed imperative. To employ a multimodal approach (1) to identify environmental factors in the PICU patient room that contribute to caregiver workflow inefficiencies, (2) to optimize safety by identifying high-touch surfaces that cause hospital-acquired infections, (3) to develop human-centered design recommendations. This mixed-method case study was conducted in a 23-bed urban hospital PICU. The activities, movements, and workflows of 13 RNs were recorded using spatial movement mapping, behavioral mapping, and clinical activity mapping. Frequency of RN contact with surfaces was documented to assess relative infection transmission risk. Face-to-face interviews were conducted with RNs to elicit their views on care delivery and their physical work environment. Direct patient care occupied 50% of RNs' time. Of the direct patient care workflow activities recorded, 26% were to prepare for care around the bedside, while 27% were for random travel between clean and soiled areas. The surfaces most frequently touched were (1) patient bedrails, (2) intravenous pumps and poles, (3) tubing and medical equipment, and (4) vital sign monitors. Value-added tasks account for only about 20% of nurses' work. Combining technology and strategic interior design to streamline workflow and enhance infection prevention optimizes efficiency and empowers frontline providers to maximize their time at the bedside performing value-added tasks.

Two-Stage Approach of Hierarchical Deep Feature Representation and Fusion Frameworks for Brain Image Analysis

In recent decades, magnetic resonance (MR) brain images have initiated a wide range of image classification and segmentation methods. Feature representation is one of the essential aspects of medical image analysis. This paper proposes and investigates specific features that address the significance of high-level tasks with little annotation for medical images. Deep learning is a futuristic area of research in biomedical image analysis, in which the scope is moving us immediately to the goal of automating tasks in intelligent retrieval systems. This approach can incorporate many levels of feature representation to construct recognition of medical cells or images. We propose a novel approach based on the deep hierarchical features of two different convolutional neural network (CNNs) model choices to achieve competitive performance in the classification task. We explore feature representation through discriminative CNN models. The principal study of our proposed work is feature representations, feature-level fusion, and classification. Meanwhile, effective fusion frameworks were employed for brain MR image classification by using serial fusion and fusion operator strategies. The accuracy of the proposed technique is demonstrated using the Cancer Imaging Archive (TCIA) and Information eXtraction from Images (IXI) datasets. To the best of our knowledge, experiment results show that CNNs feature maps as input to the classifier and are superior to the original CNNs. The performance of the support vector machines (SVM) classifier has been used to evaluate in terms of training performance and classify subjects as either normal or abnormal.