Task Definitions Research Articles

Comparative Analysis of Chatbots Using Large Language Models for Web Development Tasks

In this study, we compare the performance of five chatbots using large language models (LLMs) in handling web development tasks. Three human testers asked each chatbot nine predefined questions related to creating a simple website with a dynamic form and database integration. The questions covered tasks such as generating a web document structure, designing a layout, creating a form, and implementing database queries. The chatbots’ outputs were ranked based on accuracy, completeness, creativity, and security. The experiment reveals that conversational chatbots are adept at managing complex tasks, while programming assistants require more precisely formulated tasks or the ability to generate new responses to address irrelevant outputs. The findings suggest that conversational chatbots are more capable of handling a broader range of web development tasks with minimal supervision, whereas programming assistants need more precise task definitions to achieve comparable results. This study contributes to understanding the strengths and limitations of various LLM-based chatbots in practical coding scenarios, offering insights for their application in web development.

Assessing metarepresentational abilities in adolescence: an exploratory study on relationships between definitional competence and theory of mind.

Several developmental changes occur in adolescence, particularly in the metarepresentational domain, which allows and promotes adaptive sociality. We explored the possible relationships between theory of mind (ToM) and definitional competence, both metarepresentational, beyond age and gender effects. To reach our goals, we involved 75 adolescents (age range 14-19 years, M = 15.7, and SD = 1.36). ToM was measured through "The Reading the Mind in the Eyes Test" (RMET), and definitional competence was assessed through a new instrument, namely, the "Co.De. Scale". Attention was paid to check whether results were different when considering mental states vs. non-mental states of the scale and emotional words vs. non-emotional words. T-tests showed that older adolescents (third grade of high school) performed better than younger ones (first grade of high school) in both tasks. Only in the male group, there were no school grade differences in the ToM task. Regression analyses showed that RMET performance predicted the score of non-emotional mental states definitions and, even if marginally, of ToM word definitions. However, RMET was not a predictor of the general performance of the definitional task or emotion definitions. Connections with global adolescents' development and possible educational implications are discussed.

Pharmacists’ current and potential prescribing roles in primary care in the Netherlands: a case study

ABSTRACT In some countries, pharmacists have obtained prescribing rights to improve quality and accessibility of care and reduce physician workload. This case study explored pharmacists’ current roles in and potential for prescribing in primary care in the Netherlands, where prescribing rights for pharmacists do not exist. Participatory observations of pharmacists working in either general practice or community pharmacy were conducted, as were semi-structured interviews about current and potential practice. The latter were extended to patients and other healthcare professionals, mainly general practitioners, resulting in 34 interviews in total. Thematic analyses revealed that pharmacists, in all cases, wrote prescriptions that were then authorized by a physician before dispensing. General practice-based pharmacists often prescribed medications during patient consultations. Community pharmacists mainly influenced prescribing through (a) medication reviews where the physician and/or practice nurse often were consulted to make treatment decisions, and (b) collaborative agreements with physicians to start or substitute medications in specific situations. These findings imply that the pharmacists’ current roles in prescribing in the Netherlands resemble collaborative prescribing practices in other countries. We also identified several issues that should be addressed before formally introducing pharmacist prescribing, such as definitions of tasks and responsibilities and prescribing-specific training for pharmacists.

Protocol for identifying and characterising critical physical tasks in the military: Development and validation.

When establishing Physical Employment Standards, validity is dependent on the correct identification and characterisation of critical job tasks. To develop and validate a standardised protocol for the identification, characterisation, and documentation of critical physical job tasks in military occupational specialities in the Swedish Armed Forces (SwAF), and propose a definition of critical physical job tasks for use in the SwAF. A protocol was drafted with three content domains, including a preliminary definition. Protocol content validity was iteratively assessed in two consecutive stages where ten subject experts rated relevance and simplicity. A consensus panel revised the protocol after each stage. Content validity index (CVI) was calculated as item-CVI (I-CVI) per each feature and as scale average (S-CVI/Ave) per content domain. Acceptable content validity thresholds were 0.78 and 0.90, respectively. The validated protocol consisted of 35 items with an I-CVI≥0.90 and≥0.80 for relevance and simplicity, respectively. The S-CVI/Ave was 0.97 for relevance and 0.98 for simplicity. The protocol was language reviewed, reorganised for easy use, and approved by the consensus panel. The final protocol includes: background and aim of the protocol, the accepted generic and critical physical job task definitions, protocol instructions, subject expert-qualifications, job task source and characteristics. A standardised protocol for identification and characterisation of critical job tasks in SwAF military occupational specialties was developed. The protocol content was rated relevant and simple by experts and will be of importance in future work establishing physical requirements in the SwAF.

Beyond Collaborative Filtering: A Relook at Task Formulation in Recommender Systems

Recommender Systems (RecSys) have become indispensable in numerous applications, profoundly influencing our everyday experiences. Despite their practical significance, academic research in RecSys often abstracts the formulation of research tasks from real-world contexts, aiming for a clean problem formulation and more generalizable findings. However, it is observed that there is a lack of collective understanding in RecSys academic research. The root of this issue may lie in the simplification of research task definitions, and an overemphasis on modeling the decision outcomes rather than the decision-making process. That is, we often conceptualize RecSys as the task of predicting missing values in a static user-item interaction matrix, rather than predicting a user's decision on the next interaction within a dynamic, changing , and application-specific context. There exists a mismatch between the inputs accessible to a model and the information available to users during their decision-making process, yet the model is tasked to predict users' decisions. While collaborative filtering is effective in learning general preferences from historical records, it is crucial to also consider the dynamic contextual factors in practical settings. Defining research tasks based on application scenarios using domain-specific datasets may lead to more insightful findings. Accordingly, viable solutions and effective evaluations can emerge for different application scenarios.

STAR: Boosting Low-Resource Information Extraction by Structure-to-Text Data Generation with Large Language Models

Information extraction tasks such as event extraction require an in-depth understanding of the output structure and sub-task dependencies. They heavily rely on task-specific training data in the form of (passage, target structure) pairs to obtain reasonable performance. However, obtaining such data through human annotation is costly, leading to a pressing need for low-resource information extraction approaches that require minimal human labeling for real-world applications. Fine-tuning supervised models with synthesized training data would be a generalizable method, but the existing data generation methods either still rely on large-scale ground-truth data or cannot be applied to complicated IE tasks due to their poor performance. To address these challenges, we propose STAR, a data generation method that leverages Large Language Models (LLMs) to synthesize data instances given limited seed demonstrations, thereby boosting low-resource information extraction performance. Our approach involves generating target structures (Y) followed by generating passages (X), all accomplished with the aid of LLMs. We design fine-grained step-by-step instructions to obtain the initial data instances. We further reduce errors and improve data quality through self-reflection error identification and self-refinement with iterative revision. Our experiments show that the data generated by STAR significantly improve the performance of low-resource event extraction and relation extraction tasks, even surpassing the effectiveness of human-curated data. Human assessment of the data quality shows STAR-generated data exhibit higher passage quality and better align with the task definitions compared with the human-curated data.

Proceedings From the 2022 ACR-RSNA Workshop on Safety, Effectiveness, Reliability, and Transparency in AI

Despite the surge in artificial intelligence (AI) development for health care applications, particularly for medical imaging applications, there has been limited adoption of such AI tools into clinical practice. During a 1-day workshop in November 2022, co-organized by the ACR and the RSNA, participants outlined experiences and problems with implementing AI in clinical practice, defined the needs of various stakeholders in the AI ecosystem, and elicited potential solutions and strategies related to the safety, effectiveness, reliability, and transparency of AI algorithms. Participants included radiologists from academic and community radiology practices, informatics leaders responsible for AI implementation, regulatory agency employees, and specialty society representatives. The major themes that emerged fell into two categories: (1) AI product development and (2) implementation of AI-based applications in clinical practice. In particular, participants highlighted key aspects of AI product development to include clear clinical task definitions; well-curated data from diverse geographic, economic, and health care settings; standards and mechanisms to monitor model reliability; and transparency regarding model performance, both in controlled and real-world settings. For implementation, participants emphasized the need for strong institutional governance; systematic evaluation, selection, and validation methods conducted by local teams; seamless integration into the clinical workflow; performance monitoring and support by local teams; performance monitoring by external entities; and alignment of incentives through credentialing and reimbursement. Participants predicted that clinical implementation of AI in radiology will continue to be limited until the safety, effectiveness, reliability, and transparency of such tools are more fully addressed.

Systematic review on training models for partial nephrectomy

Robot-assisted partial nephrectomy (PN) is a complex and index procedure with a difficult learning curve that urologists need to learn how to perform safely. We systematically evaluated the development and validation evidence underpinning PN training models (TMs) by extracting and reviewing data from PubMed, Cochrane Library Central, EMBASE, MEDLINE, and Scopus databases from inception to April 2023. The level of evidence was assessed using the Oxford Center for Evidence-Based Medicine. Of the 331 screened articles, 14 cohort studies were included in the analysis. No randomized controlled trials were found, and the heterogeneous nature of the models, study groups, task definitions, and subjectivity of the metrics used were transversal to all studies. All the models were rated good for realism and usefulness as training tools. Methodological discrepancies preclude definitive conclusions regarding the construct validation. No discriminative or predictive validation evidence was reported, nor were there comparisons between an experimental group trained with a TM and a control group. The previous findings stand for the low level of evidence supporting the efficacy of the described TMs in the acquisition of skills required to safely perform PN.

Skeletal Fracture Detection with Deep Learning: A Comprehensive Review.

Deep learning models have shown great promise in diagnosing skeletal fractures from X-ray images. However, challenges remain that hinder progress in this field. Firstly, a lack of clear definitions for recognition, classification, detection, and localization tasks hampers the consistent development and comparison of methodologies. The existing reviews often lack technical depth or have limited scope. Additionally, the absence of explainable facilities undermines the clinical application and expert confidence in results. To address these issues, this comprehensive review analyzes and evaluates 40 out of 337 recent papers identified in prestigious databases, including WOS, Scopus, and EI. The objectives of this review are threefold. Firstly, precise definitions are established for the bone fracture recognition, classification, detection, and localization tasks within deep learning. Secondly, each study is summarized based on key aspects such as the bones involved, research objectives, dataset sizes, methods employed, results obtained, and concluding remarks. This process distills the diverse approaches into a generalized processing framework or workflow. Moreover, this review identifies the crucial areas for future research in deep learning models for bone fracture diagnosis. These include enhancing the network interpretability, integrating multimodal clinical information, providing therapeutic schedule recommendations, and developing advanced visualization methods for clinical application. By addressing these challenges, deep learning models can be made more intelligent and specialized in this domain. In conclusion, this review fills the gap in precise task definitions within deep learning for bone fracture diagnosis and provides a comprehensive analysis of the recent research. The findings serve as a foundation for future advancements, enabling improved interpretability, multimodal integration, clinical decision support, and advanced visualization techniques.

Text Polishing with Chinese Idiom: Task, Datasets and Pre-trained Baselines

This work presents the task of text polishing, which generates a sentence that is more graceful than the input sentence while retaining its semantic meaning. Text polishing has great value in real usage and is an important component in modern writing assistance systems. However, the task is still not well studied in the literature. Further research in this important direction requires more formal task definitions, benchmark datasets, and powerful baseline models. In this work, we formulate the task as a context-dependent text generation problem and conduct a case study on the text polishing with Chinese idiom. To circumvent the difficulties of task data annotation, we propose a semi-automatic data construction pipeline based on human-machine collaboration, and establish a large-scale text polishing dataset consisting of 1.5 million instances. We propose two types of task-specific pre-training objectives for the text polishing task and implement a series of Transformer-based models pre-trained on a massive Chinese corpus as baselines. We conduct extensive experiments with the baseline models on the constructed text polishing datasets and have some major findings. The human evaluation further reveals the polishing ability of the final system.

A tool for the semantic annotation, validation and optimization of business process models

AbstractThe adoption of the business process model notation for the description of internal workflows and procedures by both public administrations and private organizations is steadily growing, thanks to the simplicity of the standard and its consistent expressivity. However, the lack of semantic support from BPMN can pose important limitations to its efficient use, as ambiguities in task definitions and communications can lead to misinterpretations. Furthermore, there is the need to validate the BPMNs, to check their adherence to regulations, especially in public administration, and to verify their conformance to security and privacy constraints. In this work, we present SemPrAnn, a semantic annotation tool for BPMN that, exploiting domain ontologies and logical rules, provides the possibility to unambiguously identify concepts in workflows and to run inferential engines against them to enforce the rules. The manuscript presents the methodology applied for the implementation of the tool, the tool itself with its exposed functionalities, and a case study demonstrating its current capabilities.

Ergodic Shared Control: Closing the Loop on pHRI Based on Information Encoded in Motion

Advances in exoskeletons and robot arms have given us increasing opportunities for providing physical support and meaningful feedback in training and rehabilitation settings. However, the chosen control strategies must support motor learning and provide mathematical task definitions that are actionable for the actuation. Typical robot control architectures rely on measuring error from a reference trajectory. In physical human-robot interaction, this leads to low engagement, invariant practice, and few errors, which are not conducive to motor learning. A reliance on reference trajectories means that the task definition is both over-specified—requiring specific timings not critical to task success—and lacking information about normal variability. In this article, we examine a way to define tasks and close the loop using an ergodic measure that quantifies how much information about a task is encoded in the human-robot motion. This measure can capture the natural variability that exists in typical human motion, enabling therapy based on scientific principles of motor learning. We implement an ergodic hybrid shared controller (HSC) on a robotic arm as well as an error-based controller—virtual fixtures—in a timed drawing task. In a study of 24 participants, we compare ergodic HSC with virtual fixtures and find that ergodic HSC leads to improved training outcomes.

AutoFac: The Perpetual Robot Machine

Robotics currently lacks fully autonomous capabilities, especially where task knowledge is incomplete and optimal robotic solutions cannot be pre-engineered. The intersection of evolutionary robotics, artificial life and embodied artificial intelligence presents a promising paradigm for generating multitask problem-solvers suitable for adapting over extended periods in unexplored, remote and hazardous environments. To address the automation of evolving robotic systems, we propose fully autonomous, embodied artificial-life factories and laboratories, situated in various environments as multi-task problem-solvers. Such integrated factories and laboratories would be adaptive solution designers, producing fit-for-purpose physical robots with accelerated artificial evolution that experiment to continually discover new tasks. Such tasks would be stepping-stones towards accomplishing given mission objectives over extended periods (days to decades). Rather than being purely speculative, prerequisite technologies to realize such factories have been experimentally demonstrated. Currently, vast scientific and enterprise opportunities await in applications such as asteroid mining, terraforming, space and deep sea exploration, though no suitable solution exists. The proposed embodied artificial-life factories and laboratories, termed: AutoFac, use robot production equipment run by artificial evolution controllers to collect and synthesize environmental information (from robotic sensory systems). Such information is merged with current needs and mission objectives to create new robot embodiment and task definitions that are environmentally adapted and balance task-oriented behavior with exploration. AutoFac is thus generalist (deployable in many environments) but continually produces specialist solutions within such environments a perpetual robot machine.

Semantic Conceptual Framework for Environmental Monitoring and Surveillance—A Case Study on Forest Fire Video Monitoring and Surveillance

This paper presents a semantic conceptual framework and definition of environmental monitoring and surveillance and demonstrates an ontology implementation of the framework. This framework is defined in a mathematical formulation and is built upon and focused on the notation of observation systems. This formulation is utilized in the analysis of the observation system. Three taxonomies are presented, namely, the taxonomy of (1) the sampling method, (2) the value format and (3) the functionality. The definition of concepts and their relationships in the conceptual framework clarifies the task of querying for information related to the state of the environment or conditions related to specific events. This framework aims to make the observation system more queryable and therefore more interactive for users or other systems. Using the proposed semantic conceptual framework, we derive definitions of the distinguished tasks of monitoring and surveillance. Monitoring is focused on the continuous assessment of an environment state and surveillance is focused on the collection of all data relevant for specific events. The proposed mathematical formulation is implemented in the format of the computer readable ontology. The presented ontology provides a general framework for the semantic retrieval of relevant environmental information. For the implementation of the proposed framework, we present a description of the Intelligent Forest Fire Video Monitoring and Surveillance system in Croatia. We present the implementation of the tasks of monitoring and surveillance in the application domain of forest fire management.

Location-Centered House Price Prediction: A Multi-Task Learning Approach

Accurate house prediction is of great significance to various real estate stakeholders such as house owners, buyers, and investors. We propose a location-centered prediction framework that differs from existing work in terms of data profiling and prediction model. Regarding data profiling, we make an important observation as follows – besides the in-house features such as floor area, the location plays a critical role in house price prediction. Unfortunately, existing work either overlooked it or had a coarse grained measurement of locations. Thereby, we define and capture a fine-grained location profile powered by a diverse range of location data sources, including transportation profile, education profile, suburb profile based on census data, and facility profile. Regarding the choice of prediction model, we observe that a variety of approaches either consider the entire data for modeling, or split the entire house data and model each partition independently. However, such modeling ignores the relatedness among partitions, and for all prediction scenarios, there may not be sufficient training samples per partition for the latter approach. We address this problem by conducting a careful study of exploiting the Multi-Task Learning (MTL) model. Specifically, we map the strategies for splitting the entire house data to the ways the tasks are defined in MTL, and select specific MTL-based methods with different regularization terms to capture and exploit the relatedness among tasks. Based on real-world house transaction data collected in Melbourne, Australia, we design extensive experimental evaluations, and the results indicate a significant superiority of MTL-based methods over state-of-the-art approaches. Meanwhile, we conduct an in-depth analysis on the impact of task definitions and method selections in MTL on the prediction performance, and demonstrate that the impact of task definitions on prediction performance far exceeds that of method selections.

Задачи логистики: классификация и методы решения

The article provides definitions of logistics tasks and their classification. Some mathematical statements of logistics problems and methods of their solution are also given. In particular, the following combinatorial optimization problems are described: the traveling salesman problem, the resource allocation problem, the transportation problem, the maximum flow problem.

Signal Temporal Logic Task Decomposition via Convex Optimization

In this letter we focus on the problem of decomposing a global Signal Temporal Logic formula (STL) assigned to a multi-agent system to local STL tasks when the team of agents is a-priori decomposed to disjoint sub-teams. The predicate functions associated to the local tasks are parameterized as hypercubes depending on the states of the agents in a given sub-team. The parameters of the functions are, then, found as part of the solution of a convex program that aims implicitly at maximizing the volume of the zero superlevel set of the corresponding predicate function. Two alternative definitions of the local STL tasks are proposed and the satisfaction of the global STL formula is proven when the conjunction of the local STL tasks is satisfied.

Serverless Geospatial Data Processing Workflow System Design

Geospatial data and related technologies have become an increasingly important aspect of data analysis processes, with their prominent role in most of them. Serverless paradigm have become the most popular and frequently used technology within cloud computing. This paper reviews the serverless paradigm and examines how it could be leveraged for geospatial data processes by using open standards in the geospatial community. We propose a system design and architecture to handle complex geospatial data processing jobs with minimum human intervention and resource consumption using serverless technologies. In order to define and execute workflows in the system, we also propose new models for both workflow and task definitions models. Moreover, the proposed system has new Open Geospatial Consortium (OGC) Application Programming Interface (API) Processes specification-based web services to provide interoperability with other geospatial applications with the anticipation that it will be more commonly used in the future. We implemented the proposed system on one of the public cloud providers as a proof of concept and evaluated it with sample geospatial workflows and cloud architecture best practices.

Natural Language Processing and Computational Linguistics

Natural Language Processing and Computational Linguistics

Accuracy of visual inspection of flood defences

Prioritisation of flood defence maintenance is typically based on visual inspection. However, literature shows that the Probability of Detection (PoD) of visual inspection can vary significantly. Here we investigate the PoD for visual inspections of flood defence structures, the consistency of damage classification, and the influence of different variables on the PoD, such as past experience. Four flood defence sections were inspected by 22 different inspectors for a variety of damage types, such as animal burrowing and damage to block revetments. It is found that the PoD varies significantly both per damage type and inspector. Additionally, the estimated severity of damages varies significantly in comparison to the reference situation: over half of the registered damages is assigned a different severity compared to the reference, which potentially leads to incorrect maintenance measures. A likely explanation for the variation in results is the complexity of inspection guidelines and task definitions. Therefore it is advised to simplify inspection guidelines and use more focussed inspections for the most important types of damage. This likely leads to both a reduction of the number of false negatives associated with an increase in flood risk, and better risk-based asset management and maintenance prioritisation in general.