Multistage Task Research Articles

FMB: A functional manipulation benchmark for generalizable robotic learning

In this paper, we propose a real-world benchmark for studying robotic learning in the context of functional manipulation: a robot needs to accomplish complex long-horizon behaviors by composing individual manipulation skills in functionally relevant ways. The core design principles of our Functional Manipulation Benchmark (FMB) emphasize a harmonious balance between complexity and accessibility. Tasks are deliberately scoped to be narrow, ensuring that models and datasets of manageable scale can be utilized effectively to track progress. Simultaneously, they are diverse enough to pose a significant generalization challenge. Furthermore, the benchmark is designed to be easily replicable, encompassing all essential hardware and software components. To achieve this goal, FMB consists of a variety of 3D-printed objects designed for easy and accurate replication by other researchers. The objects are procedurally generated, providing a principled framework to study generalization in a controlled fashion. We focus on fundamental manipulation skills, including grasping, repositioning, and a range of assembly behaviors. The FMB can be used to evaluate methods for acquiring individual skills, as well as methods for effectively combining and ordering such skills in order to solve complex, multi-stage manipulation tasks. We also offer an imitation learning framework that includes a suite of policies trained to solve the proposed tasks. This enables researchers to utilize our tasks as a versatile toolkit for examining various parts of the pipeline. For example, researchers could propose a better design for a grasping controller and evaluate it in combination with our baseline reorientation and assembly policies as part of a pipeline for solving multi-stage tasks. Our dataset, object CAD files, code, and evaluation videos can be found on our project website: https://functional-manipulation-benchmark.github.io .

A comparison of visual and auditory EEG interfaces for robot multi-stage task control.

Shared autonomy holds promise for assistive robotics, whereby physically-impaired people can direct robots to perform various tasks for them. However, a robot that is capable of many tasks also introduces many choices for the user, such as which object or location should be the target of interaction. In the context of non-invasive brain-computer interfaces for shared autonomy-most commonly electroencephalography-based-the two most common choices are to provide either auditory or visual stimuli to the user-each with their respective pros and cons. Using the oddball paradigm, we designed comparable auditory and visual interfaces to speak/display the choices to the user, and had users complete a multi-stage robotic manipulation task involving location and object selection. Users displayed differing competencies-and preferences-for the different interfaces, highlighting the importance of considering modalities outside of vision when constructing human-robot interfaces.

Krasnoyarsk. Transformation of the industrial territory of the EVRZ

The article deals with the territory of the industrial architectural heritage of the Krasnoyarsk Electric Railway-Car Repair Plant (EVRZ) as a place of creation of a district of modern residential development with active city life. In this case, multi-stage tasks of building and development of the territory are solved, including modern infrastructure, which will become a point of attraction for citizens of different age. The purpose of the work described in the article is to create a development strategy for the territory of the EVRZ (44.85 hectares) and further spread the experience of transformation and industrial development to all the territories adjacent to the railway line and the “rust belt” of Krasnoyarsk (2,380 hectares).

LLM for Differentiable Soft Robotic Vision Objects Manipulation

Most existing neural soft object manipulation systems rely on differentiable simu- lation as the default physics engine. However, current explicit differentiable sim- ulation solvers adopt a heuristic searching approach to manipulate the soft body, which easily gets stuck when the initial stage of the end effectors is sub-optimal on single-stage tasks or when performing multi-stage tasks that require complex hi- erarchical actions, which often leads to local-minima. Furthermore, existing deep soft object manipulation systems never consider the soft multi-body dynamics and model the dynamic system with naive explicit visual cues (e.g., image frame). To address this challenge, we propose an implicit, i.e., with energy-based models, soft objects manipulation differentiable simulator (iDSoftor) that guides the dif- ferentiable physics solver to deform the various soft object. The key idea of our work is to integrate energy-based models into the soft-object differentiable simu- lation to address the local minima in common explicit (heuristic transport-based) methods to stabilize the gradient of training differentiable simulators. On simple tasks, such as one-stage tasks, our proposed method can feasibly find a suitable initial stage based on theoretical arguments, refer to. On complex multi-stage tasks with multimodal, like implicit visual cues states, our proposed methods can iteratively adjust the manipulation trajec- tory from the perspective of energy priorities. To evaluate the feasibility of our proposed method, we formulate various potential novel soft-object manipulation tasks and demonstrate a preliminary story.

Основы документирования хищений мобильных средств связи

Introduction. The article analyses the current state and transformation of the mechanism of theft of mobile communication devices. The main ways of committing thefts are studied, as well as the techniques and methods used in this process. The relevance of the research is determined both by the high number of committed thefts of mobile communication devices and by certain aspects of legal regulation of the documentation process in their investigation.
 Methods. In the research various methods were used, including the universal scientific method of historical materialism; general scientific methods of cognition (comparison, ascent from abstract to concrete, analysis, synthesis, induction, deduction, generalisation, classification, abstraction), private scientific methods (formal-legal, comparative-legal, method of interpretation of legal norms, dogmatic), interdisciplinary method of formalisation.
 Results. The author analysed the most typical investigative situations in the investigation of mobile communication devices thefts and concluded that the process of documenting such crimes is related to their investigation being aimed at collecting, recording and analysing evidence related to the crime in order to identify the responsible parties and ensure the identification of the perpetrators of mobile communication device thefts. Certain difficulties are associated with the fact that documenting the theft of mobile communication devices is a complex integrated multi-stage task that requires careful attention and co-operation between all participants in the criminal justice process involved in the investigation of a particular criminal case.

Multi-stage two-echelon crowdsourcing logistics assignment model with future committing drivers

With the popularity of low-carbon economy, governments of all countries have introduced energy-saving and emission reduction policies. Crowdsourcing logistics, a logistics assignment model developed under the sharing economy, can effectively use transport resources from the public and thus meaningfully contribute to sustainability. It has important practical significance both for energy saving and emission reduction in last-mile logistics distributions, and hence plays an important role in tackling the challenges associated with last-mile and same-day deliveries. This paper investigated the crowdsourcing logistics task assignment problem where logistics platforms can perform multi-stage task assignment. The information of all tasks to be delivered in each stage is known a priori, however the information of crowdsourcing drivers in future stages is not known completely. A multi-stage two-echelon dynamic task assignment model (MS-2E-DAM) was developed for the problem and a heuristic which combines genetic algorithm and tabu search (GA-TS) was developed. Its performance was benchmarked with CPLEX 12.10 for small-size problems and the results demonstrated the effectiveness of the proposed heuristic approach. For large-size problems, the proposed approach can help reduce the overall cost by 1.94% over traditional assignment approaches. Sensitivity analyses on three key parameters helped identify the key factors that affect the system cost, and several management suggestions were proposed based on the results.

Cue Sources and Cue Utilization Patterns of Social Mentalizing during Two-Person Interactions.

Social mentalizing plays a crucial role in two-person interactions. Depending on the target of inference and the content being inferred, social mentalizing primarily exists in two forms: first-order mentalizing and second-order mentalizing. Our research aims to investigate the cue sources and cue utilization patterns of social mentalizing during two-person interactions. Our study created an experimental situation of a two-person interaction and used the "Spot the difference" game to reveal our research question with multi-stage tasks. Our study was divided into two experiments, Experiment 1 and Experiment 2, which examined the cue sources and cue utilization patterns of first- and second-order mentalizing, respectively. The results of the experiments showed that (1) self-performance and other performance are significant cues utilized by individuals during social mentalizing. (2) Individuals employ discrepancies to modulate the relationship between self-performance and first-order mentalizing as well as to adjust the relationship between otherperformance and second-order mentalizing. The results of this study further complement the dual-processing model of mindreading and the anchoring and adjustment hypothesis during social inference.

Orbit: A Unified Simulation Framework for Interactive Robot Learning Environments

We present <sc xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Orbit</small> , a unified and modular framework for robot learning powered by <sc xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Nvidia</small> Isaac Sim. It offers a modular design to easily and efficiently create robotic environments with photo-realistic scenes and high-fidelity rigid and deformable body simulation. With <sc xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Orbit</small> , we provide a suite of benchmark tasks of varying difficulty– from single-stage cabinet opening and cloth folding to multi-stage tasks such as room reorganization. To support working with diverse observations and action spaces, we include fixed-arm and mobile manipulators with different physically-based sensors and motion generators. <sc xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Orbit</small> allows training reinforcement learning policies and collecting large demonstration datasets from hand-crafted or expert solutions in a matter of minutes by leveraging GPU-based parallelization. In summary, we offer an open-sourced framework that readily comes with 16 robotic platforms, 4 sensor modalities, 10 motion generators, more than 20 benchmark tasks, and wrappers to 4 learning libraries. With this framework, we aim to support various research areas, including representation learning, reinforcement learning, imitation learning, and task and motion planning. We hope it helps establish interdisciplinary collaborations in these communities, and its modularity makes it easily extensible for more tasks and applications in the future. For videos, documentation, and code: <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://isaac-orbit.github.io/</uri> .

The StarCraft Multi-Agent Exploration Challenges: Learning Multi-Stage Tasks and Environmental Factors Without Precise Reward Functions

In this paper, we propose a novel benchmark called the StarCraft Multi-Agent Exploration Challenges(SMAC-Exp), where agents learn to perform multi-stage tasks and to use environmental factors without precise reward functions. The previous challenges (SMAC) recognized as a standard benchmark of Multi-Agent Reinforcement Learning are mainly concerned with ensuring that all agents cooperatively eliminate approaching adversaries only through fine manipulation with obvious reward functions. SMAC-Exp, on the other hand, is interested in the exploration capability of MARL algorithms to efficiently learn implicit multi-stage tasks and environmental factors as well as micro-control. This study covers both offensive and defensive scenarios. In the offensive scenarios, agents must learn to first find opponents and then eliminate them. The defensive scenarios require agents to use topographic features. For example, agents need to position themselves behind protective structures to make it harder for enemies to attack. We investigate a total twelve MARL algorithms under both sequential and parallel episode settings of SMAC-Exp and observe that recent approaches perform well in similar settings to the previous challenge, but we discover that current multi-agent approaches place relatively less emphasis on exploration perspectives. To a limited extent, we observe that an enhanced exploration method has a positive effect on SMAC-Exp, however there is still gap that state-of-art algorithms cannot resolve the most challenging scenarios of SMAC-Exp. Consequently, we propose a new axis for future research into Multi-Agent Reinforcement Learning studies.

Semantic Learning for Analysis of Overlapping LPI Radar Signals

The increasingly complex radio environment may cause the received low probability of intercept (LPI) radar signals to overlap in time–frequency domains. Analyzing overlapping LPI radar signals requires identifying the modulation type and estimating the parameters of each component. Prior research performs overlapping signal analysis as a multistage task, where each stage is designed to perform a part of the task. The multistage system will increase the calculation burden and cannot be optimized as a whole. Instead, this article proposes a novel framework for analyzing overlapping signals in a single stage. Specifically, we develop a joint semantic learning deep convolutional neural network (JSLCNN) that jointly learns three tasks, i.e., feature restoration, modulation classification, and parameter regression. Since the whole cognitive pipeline is a single network, it can be optimized end-to-end directly on cognitive performance. To verify the validity of the proposed JSLCNN, numerous comparative experiments are carried out in terms of modulation recognition and parameter estimation of overlapping signals. Experimental results demonstrate that the JSLCNN has desirable extensibility for identifying unseen signal combinations and robustness against unknown jamming. Furthermore, we show that the JSLCNN outperforms other existing approaches in generic real-time parameter estimation for LPI radar signals.

Involving Students in the Development and Mediation of Assessment Competences. A Case Study: Romanian as a Foreign Language. Medical Language

The cementing of mediation as an integral part of the 2020 CEFR/CV (The Common European Framework of Reference for Languages/ Companion Volume) and, consequently, of adequate assessment processes, sets the task of rethinking the Romanian language test for foreign medical students at the UMFVBT (The “Victor Babeș” University of Medicine and Pharmacy Timișoara). Mediation should be addressed with a view to integrating it into the comprehensive learning, testing and evaluation process. The paper points beyond a summative form of assessment and argues in favour of adding formative, dynamic forms of assessment during the language course on specific written mediation competences. Within this framework, we plead for partially restricting the overall authority of the teacher in matters of assessment and for involving the students in the very assessment of the mediation competences they are to acquire and develop. In order to avoid messy progress and to ensure a systematic, methodical and beneficial process, assessment requires careful preparation and monitoring. To this end, the paper suggests a hybridisation of learning and assessing, and lays out a coherent series of learning tasks adding up to a complex task suite, a multi-stage task designed to directly involve the students in as many learning and assessment stages as possible.

Prototype multi-agent robotic debris removal system: principles of development and experimental studies

Objectives. The article substantiates the relevance of the creation and the prospects of application of multi-agent robotic systems for elimination of consequences of emergency situations. The purpose of this work was to test the practical feasibility of algorithms for controlling a group of autonomous robots when performing multi-stage missions.Methods. The theses of the finite automata theory in planning actions of a multi-agent system, methods of automatic control in organizing a goal-directed movement of robots, and methods of computer vision in searching and analyzing debris geometry were used.Results. The principles of development, architecture, and composition are described for the software and algorithms of a prototype of the multi-agent robotic system created at RTU MIREA as part of integrated research for the creation of tools and methods of group control of robots. The multi-stage task of searching and removing debris in the process of eliminating the consequences of emergency situations is analyzed. A proposed algorithm for planning the actions of robotic agents determines the time sequence of the mission stages. Tasks are allocated among the performing robots according to assessments of their suitability. The autonomous functioning of robotic agents is determined by commands coming from the group control level, as well as an a priori embedded knowledge base with scenario models of appropriate actions. Compensation of local environmental uncertainties in the process of robot movement is based on a comprehensive analysis of visual and navigation information. Along with the main elements of the multi-agent system, the developed infrastructure of hardware and software for visual navigation and wireless communication is described.Conclusions. The results of the experimental studies demonstrated the efficiency of the developed approaches to the creation of intelligent technologies for group control of autonomous robots on the example of debris search and removal tasks. The feasibility of the multi-agent robotic system is demonstrated by the development and integration of a number of information management and infrastructure subsystems.

Behavior policy learning: Learning multi-stage tasks via solution sketches and model-based controllers.

Multi-stage tasks are a challenge for reinforcement learning methods, and require either specific task knowledge (e.g., task segmentation) or big amount of interaction times to be learned. In this paper, we propose Behavior Policy Learning (BPL) that effectively combines 1) only few solution sketches, that is demonstrations without the actions, but only the states, 2) model-based controllers, and 3) simulations to effectively solve multi-stage tasks without strong knowledge about the underlying task. Our main intuition is that solution sketches alone can provide strong data for learning a high-level trajectory by imitation, and model-based controllers can be used to follow this trajectory (we call it behavior) effectively. Finally, we utilize robotic simulations to further improve the policy and make it robust in a Sim2Real style. We evaluate our method in simulation with a robotic manipulator that has to perform two tasks with variations: 1) grasp a box and place it in a basket, and 2) re-place a book on a different level within a bookcase. We also validate the Sim2Real capabilities of our method by performing real-world experiments and realistic simulated experiments where the objects are tracked through an RGB-D camera for the first task.

Developing Cooperative Policies for Multi-Stage Reinforcement Learning Tasks

Many hierarchical reinforcement learning algorithms utilise a series of independent skills as a basis to solve tasks at a higher level of reasoning. These algorithms don’t consider the value of using skills that are cooperative instead of independent. This paper proposes the Cooperative Consecutive Policies (CCP) method of enabling consecutive agents to cooperatively solve long time horizon multi-stage tasks. This method is achieved by modifying the policy of each agent to maximise both the current and next agent’s critic. Cooperatively maximising critics allows each agent to take actions that are beneficial for its task as well as subsequent tasks. Using this method in a multi-room maze domain and a peg in hole manipulation domain, the cooperative policies were able to outperform a set of naive policies, a single agent trained across the entire domain, as well as another sequential HRL algorithm.

Link Load Correlation-Based Blocking Performance Analysis for Tree-Type Data Center Networks

With the explosive growth of cloud computing applications, the east-west traffic among servers has come to occupy the dominant proportion of the traffic in data center networks (DCNs). Cloud computing tasks need to be executed in a distributed manner on multiple servers, which exchange large amounts of intermediate data between the adjacent stages of each multi-stage task. Therefore, the congestion in DCNs can reduce the processing performance when conducting multi-stage tasks. To address this, the relationship between the blocking performance and the traffic load can be adopted as a theoretical basis for network planning and traffic engineering. In this paper, the traffic load correlation between edge links and aggregation links is considered, and an iterative blocking performance analysis method is proposed for two-layer tree-type DCNs. The simulation results show the good accuracy of the proposed method with respect to the theoretical results especially in the blocking rate range below 4% and with over-subscription ratio 1.5.

Interfaces for Searching and Triaging Large Document Sets: An Ontology-Supported Visual Analytics Approach

We investigate the design of ontology-supported, progressively disclosed visual analytics interfaces for searching and triaging large document sets. The goal is to distill a set of criteria that can help guide the design of such systems. We begin with a background of information search, triage, machine learning, and ontologies. We review research on the multi-stage information-seeking process to distill the criteria. To demonstrate their utility, we apply the criteria to the design of a prototype visual analytics interface: VisualQUEST (Visual interface for QUEry, Search, and Triage). VisualQUEST allows users to plug-and-play document sets and expert-defined ontology files within a domain-independent environment for multi-stage information search and triage tasks. We describe VisualQUEST through a functional workflow and culminate with a discussion of ongoing formative evaluations, limitations, future work, and summary.

Assessing the Role of Gaze Tracking in Optimizing Humans-In-The-Loop Telerobotic Operation Using Multimodal Feedback.

A key challenge in achieving effective robot teleoperation is minimizing teleoperators’ cognitive workload and fatigue. We set out to investigate the extent to which gaze tracking data can reveal how teleoperators interact with a system. In this study, we present an analysis of gaze tracking, captured as participants completed a multi-stage task: grasping and emptying the contents of a jar into a container. The task was repeated with different combinations of visual, haptic, and verbal feedback. Our aim was to determine if teleoperation workload can be inferred by combining the gaze duration, fixation count, task completion time, and complexity of robot motion (measured as the sum of robot joint steps) at different stages of the task. Visual information of the robot workspace was captured using four cameras, positioned to capture the robot workspace from different angles. These camera views (aerial, right, eye-level, and left) were displayed through four quadrants (top-left, top-right, bottom-left, and bottom-right quadrants) of participants’ video feedback computer screen, respectively. We found that the gaze duration and the fixation count were highly dependent on the stage of the task and the feedback scenario utilized. The results revealed that combining feedback modalities reduced the cognitive workload (inferred by investigating the correlation between gaze duration, fixation count, task completion time, success or failure of task completion, and robot gripper trajectories), particularly in the task stages that require more precision. There was a significant positive correlation between gaze duration and complexity of robot joint movements. Participants’ gaze outside the areas of interest (distractions) was not influenced by feedback scenarios. A learning effect was observed in the use of the controller for all participants as they repeated the task with different feedback combination scenarios. To design a system for teleoperation, applicable in healthcare, we found that the analysis of teleoperators’ gaze can help understand how teleoperators interact with the system, hence making it possible to develop the system from the teleoperators’ stand point.

Explaining Evolutionary Agent-Based Models via Principled Simplification.

Understanding how evolutionary agents behave in complex environments is a challenging problem. Agents can be faced with complex fitness landscapes derived from multi-stage tasks, interaction with others, and limited environmental feedback. Agents that evolve to overcome these can sometimes access greater fitness, as a result of factors such as cooperation and tool use. However, it is often difficult to explain why evolutionary agents behave in certain ways, and what specific elements of the environment or task may influence the ability of evolution to find goal-achieving behaviours; even seemingly simple environments or tasks may contain features that affect agent evolution in unexpected ways. We explore principled simplification of evolutionary agent-based models, as a possible route to aiding their explainability. Using the River Crossing Task (RCT) as a case study, we draw on analysis in the Minimal River Crossing (RC-) Task testbed, which was designed to simplify the original task while keeping its key features. Using this method, we present new analysis concerning when agents evolve to successfully complete the RCT. We demonstrate that the RC- environment can be used to understand the effect that a cost to movement has on agent evolution, and that these findings can be generalised back to the original RCT. Then, we present new insight into the use of principled simplification in understanding evolutionary agents. We find evidence that behaviour dependent on features that survive simplification, such as problem structure, are amenable to prediction; while predicting behaviour dependent on features that are typically reduced in simplification, such as scale, can be invalid.

Task Decomposition Based on Cloud Manufacturing Platform

As a new service-oriented manufacturing paradigm, cloud manufacturing (CMfg) realizes the optimal allocation of resources in the product manufacturing process through the network. Task decomposition is a key problem of the CMfg system for resource scheduling. A high-quality task decomposition method can shorten product development time, reduce costs for resource service providers, and provide technical support for the application of CMfg. However, a cloud manufacturing system has to manage the allocation the correct amount of manufacturing resources, complex production processes, and highly dynamic production environments. At the same time, the tasks issued by service demanders are usually asymmetric and tightly coupled. We solve the complex task decomposition problem by using the traditional methods, that are hard to complete in CMfg. To overcome the shortcomings of CMfg, this paper proposed a task decomposition method based on the cloud platform. For achieving modular production, this approach creatively divides the product production process into four stages: design, manufacturing, transportation, and maintenance. Then a hybrid method, which combines with depth-first search algorithm, fast modular optimization algorithm, and artificial bee colony algorithm, is introduced. The method can obtain a multi-stage task optimization decomposition plan in CMfg. Simulation results demonstrate the proposed method can achieve complex task optimization decomposition in a CMfg environment.

Planning of multi-stage production systems with interchangeable equipment

The multi-stage tasks of scheduling the process of manufacturing complex products are considered. Optimization criteria take into account equipment changeovers, penalties for violations of the product manufacturing deadlines, and transaction costs. A general mathematical model is constructed, within the framework of which scheduling problems are posed and algorithms for their solution are proposed. The developed algorithms are implemented in interactive software for solving the problems under consideration.