Output Tasks Research Articles

Constructing a Large Language Model to Generate Impressions from Findings in Radiology Reports.

Background The specialization and complexity of radiology makes the automatic generation of radiologic impressions (ie, a diagnosis with differential diagnosis and management recommendations) challenging. Purpose To develop a large language model (LLM) that generates impressions based on imaging findings and to evaluate its performance in professional and linguistic dimensions. Materials and Methods Six radiologists recorded imaging examination findings from August 2 to 31, 2023, at Shanghai General Hospital and used the developed LLM before routinely writing report impressions for multiple radiologic modalities (CT, MRI, radiography, mammography) and anatomic sites (cranium and face, neck, chest, upper abdomen, lower abdomen, vessels, bone and joint, spine, breast), making necessary corrections and completing the radiologic impression. A subset was defined to investigate cases where the LLM-generated impressions differed from the final radiologist impressions by excluding identical and highly similar cases. An expert panel scored the LLM-generated impressions on a five-point Likert scale (5 = strongly agree) based on scientific terminology, coherence, specific diagnosis, differential diagnosis, management recommendations, correctness, comprehensiveness, harmlessness, and lack of bias. Results In this retrospective study, an LLM was pretrained using 20 GB of medical and general-purpose text data. The fine-tuning data set comprised 1.5 GB of data, including 800 radiology reports with paired instructions (describing the output task in natural language) and outputs. Test set 2 included data from 3988 patients (median age, 56 years [IQR, 40-68 years]; 2159 male). The median recall, precision, and F1 score of LLM-generated impressions were 0.775 (IQR, 0.56-1), 0.84 (IQR, 0.611-1), and 0.772 (IQR, 0.578-0.957), respectively, using the final impressions as the reference standard. In a subset of 1014 patients (median age, 57 years [IQR, 42-69 years]; 528 male), the overall median expert panel score for LLM-generated impressions was 5 (IQR, 5-5), ranging from 4 (IQR, 3-5) to 5 (IQR, 5-5). Conclusion The developed LLM generated radiologic impressions that were professionally and linguistically appropriate for a full spectrum of radiology examinations. © RSNA, 2024 Supplemental material is available for this article.

Young beginning learners’ vocabulary learning via input and output tasks: The role of working memory

Working memory (WM) is essential to vocabulary learning. However, limited attention has been paid to young beginner learners’ vocabulary development under various task conditions from the perspective of WM. This study investigates how two types of WM – complex WM and phonological short-term memory – may influence two instructional approaches (i.e., input and output tasks) on picking up new words. 93 young learners studying English as a foreign language (EFL) participated in input and output tasks and four vocabulary assessments. These assessments functioned as a pretest, immediate posttest, and delayed posttest. The participants also took two WM tests: an operation span test for complex WM and a word repetition test for phonological short-term memory. The results demonstrated that: (1) both input and output tasks significantly influenced the learning and retention of new words, (2) complex WM did not substantially predict task effects on the learning and retention of new words, and (3) phonological WM had a notable impact on learning and retaining new words under the input and output task conditions. The findings emphasize the role of WM for EFL young learners’ vocabulary learning through tasks.

Are LLMs good at structured outputs? A benchmark for evaluating structured output capabilities in LLMs

Existing benchmarks for Large Language Models (LLMs) mostly focus on general or specific domain capabilities, overlooking structured output capabilities. We introduce SoEval, a benchmark for assessing LLMs’ ability to generate structured outputs like JSON, XML, and lists. SoEval contains 3.7K entries in Chinese and English, covering 13 types of structured output tasks across 20 subjects. In experiments, we found that while current mainstream LLMs have deficiencies in structured output, GPT-4 outperforms them in this aspect. GPT-4 achieved an average score of 0.4 on SoEval, representing a 24% enhancement over the next best-performing model. At the same time, the performance of current mainstream models on English tasks is also better than on Chinese tasks. We also report the performance of mainstream large models on different structured output types and task subjects. The benchmark construction code and SoEval dataset are open-sourced at https://github.com/MoranCoder95/SoEval.

Generalized sparse and outlier-robust broad learning systems for multi-dimensional output problems

Broad learning systems (BLSs) are becoming increasingly popular due to their fast and superior learning capabilities. However, their performances are susceptible to outliers and dense networks. The Elastic-net robust BLS (ER-BLS), which employs an ℓ1-norm loss function along with Elastic-net regularization, attempts to mitigate these issues by promoting network sparsity. Nevertheless, even ER-BLS fails to achieve satisfactory robustness and sparsity in multi-dimensional output tasks because the ℓ1-norm ignores the correlations between the output dimensions and output weights. Therefore, we propose a generalized Elastic-net robust BLS (GER-BLS) by generalizing the ℓ1-norm to the L2,1-norm. The L2,1-norm enhances robustness by treating the residuals across all output dimensions as a whole and induces node-level sparsity by penalizing all output weights connected to a single hidden node. The alternating direction method of multipliers (ADMM) algorithm was employed to solve GER-BLS efficiently, circumventing the non-differentiable issue of the objective function. Moreover, we developed DGER-BLS, a distributed variant of GER-BLS, tailored to handle data with distributed storage. The convergence analysis of our proposed algorithms and a large number of numerical experiments on multi-dimensional output datasets demonstrated the effectiveness of GER-BLS and DGER-BLS in dealing with outliers, distributed stored data, and sparsity.

A teacher-researcher snapshot of task-based peer interactions in EFL secondary school classrooms in Germany

Abstract This article reports a small classroom-based study that has explored task-based peer interactions among grade 9 secondary school students (N = 14) of middle and low relative proficiency. Drawing primarily on audio recordings of peer interactions, oral presentations, and post-task interviews, and conducted by the teacher-researcher, it investigated how students interacted on an output task that was embedded in regular EFL lessons and designed to wrap up the unit work centered around the topic Down Under. A particular focus was on exploring to what extent and how students used the language that the task was meant to trigger, namely future tense and unit-related vocabulary. The aim was to inquire whether the task worked in the way it was designed for. The findings have shown that although students were involved with the task and with each other’s contributions, the use of the targeted structure elicited by the task remained limited. Pedagogical implications are discussed.

Identification of Breathing Pattern Disorder in Athletes With Exercise-Induced Laryngeal Obstruction: A Novel Assessment Tool

To report the diagnostic utility of the novel, high-ventilatory task assessment tool called the Milstein Breathing Pattern Assessment Index (M-BPAI) for evaluation of Breathing Pattern Disorder (BPD) in athletes with and without breathing difficulty, and to evaluate the prevalence of BPD in athletes referred for Exercise Induced Laryngeal Obstruction (EILO). BPD is an abnormal respiratory biomechanical pattern caused from functional or structural factors. The presence of BPD in athletes with EILO is unknown. The current clinical evaluations of dysfunctional breathing patterns are limiting for evaluation of BPD in patients with EILO, as these only evaluate the patients in low ventilatory output tasks of rest breathing. In this prospective study, a total of 77 athletes referred to the clinic for suspected EILO and 58 athletes without any respiratory difficulty underwent M-BPAI assessment. Data collection from the experimental group also included the Dyspnea Index, and laryngeal video endoscopic provocation test. The M-BPAI score was significantly larger in the patient group compared to the control group. An overall M-BPAI score of ≥8 corresponds to the AUC of 0.87 (95% CI: 0.81-0.93) with a sensitivity of 0.862 and specificity of 0.792. A total of 60 (78%) patients had an overall M-BPAI score of ≥8. The M-BPAI has the potential to be a valuable clinical diagnostic marker for identifying BPD in patients with suspected EILO with further research and validation.

Factors of Influence for Transfer Learning Across Diverse Appearance Domains and Task Types.

Transfer learning enables to re-use knowledge learned on a source task to help learning a target task. A simple form of transfer learning is common in current state-of-the-art computer vision models, i.e., pre-training a model for image classification on the ILSVRC dataset, and then fine-tune on any target task. However, previous systematic studies of transfer learning have been limited and the circumstances in which it is expected to work are not fully understood. In this paper we carry out an extensive experimental exploration of transfer learning across vastly different image domains (consumer photos, autonomous driving, aerial imagery, underwater, indoor scenes, synthetic, close-ups) and task types (semantic segmentation, object detection, depth estimation, keypoint detection). Importantly, these are all complex, structured output tasks types relevant to modern computer vision applications. In total we carry out over 2000 transfer learning experiments, including many where the source and target come from different image domains, task types, or both. We systematically analyze these experiments to understand the impact of image domain, task type, and dataset size on transfer learning performance. Our study leads to several insights and concrete recommendations: (1) for most tasks there exists a source which significantly outperforms ILSVRC'12 pre-training; (2) the image domain is the most important factor for achieving positive transfer; (3) the source dataset should include the image domain of the target dataset to achieve best results; (4) at the same time, we observe only small negative effects when the image domain of the source task is much broader than that of the target; (5) transfer across task types can be beneficial, but its success is heavily dependent on both the source and target task types.

유의미한 출력과업과 짝활동이 영어 관계대명사절 학습에 미치는 영향

The present study conducted two experiments to investigate the effects of output tasks on learning of English relative clauses (ERCs) by Korean university students. In Experiment 1, images were used, and sixty-four students learned ERCs in either individual or pair work conditions. In Experiment 2, stories were used and ninety-seven students were assigned to one of three conditions: (i) individual, (ii) individual explanation, and (iii) pair work. The effects of groups were measured by two posttests consisting of grammaticality judgment, error correction, and sentence completion. The results showed that the pair work groups outperformed the individual groups in both experiments. The results suggest that the interaction between learners induced by the pair work promotes learners’ active participation and understanding of the target forms. It also helps them pay attention to the target forms, leading them to reconstruct their interlanguage, which facilitates the learning.

An Auto-Encoder Multitask LSTM Model for Boundary Localization

Although positioning in both indoor and outdoor environments has been investigated in-depth individually by various means, little attention has been paid to the intersected areas of both environments and the positioning accuracy in these boundary areas remains concern. For indoor positioning, several recent studies have applied machine learning algorithms based on WiFi signals. However, these methods may not be applicable to solve the boundary localization problem which is faced for positioning in boundary areas around a building. In this paper, we propose a deep learning model which concatenates auto-encoders with LSTM networks to perform multi-sensor multi-task fingerprint positioning. The adopted sensors include WiFi, GPS, cellular (fused with GPS), and magnetometer. In our model, we first employ a dense block-based auto-encoder to extract representative latent codes of fingerprints. Such latent codes are proven to be more distinguishable since they are revertible to their original inputs. Then, a sequence of latent codes are injected into an LSTM network performing three output tasks, responsible for location estimation, most suitable sensor selection, and indoor/outdoor classification, respectively. Extensive real-life experiments are performed on two campuses and the results demonstrate that our model achieves high positioning accuracy with an average Euclidean distance estimation error of about 0.43 meter.

A Review of the Effects of Different Cognitive Styles on Incidental Vocabulary Acquisition—Based on English Post-Reading Output Tasks

A Review of the Effects of Different Cognitive Styles on Incidental Vocabulary Acquisition—Based on English Post-Reading Output Tasks

The Impact of an Intensive English Reading Course Based on the Production-Oriented Approach on the L2 Motivational Self System Among Chinese University English Majors From a Dynamic Systems Theory Perspective.

This article reports on a study that took a Dynamic Systems Theory (DST) perspective to second language (L2) motivational self system (L2MSS). More specifically, it investigated the influence of an Intensive English Reading course based on the Production-Oriented Approach (POA) upon the L2MSS of Chinese university English majorsfrom the DST perspective. To this end, two intact classes composed of 50 students were assigned into experimental group (EG) (N = 23) and control group (CG) (N = 27), who responded to an L2MSS scale before and after the one-semester intervention. Eight and five students were respectively selected using the purposive sampling method from the experimental and control groups for follow-up semi-structured interviews. The quantitative results revealed that the overall and dimensional (Ideal L2 Self and L2 Learning Experience) levels of L2MSS were significantly strengthened over time in the EG while kept stable in the CG. The qualitative results suggested that the enhanced Ideal L2 Self of the participants stemmed from an attractor basin that was deepened by a number of attractors encompassing Output Tasks and Peer Performance. The interview results also showed that the increased L2 Learning Experience of the participants pertained to an attractor basin that was consolidated by an array of attractors containing Output Tasks, Teacher Guidance, Group Discussion, and Peer Assessment. The findings indicated that the attractors at the subjective and social dimensions in the POA-based course collectively worked together to cause changes in L2MSS among the participants. The implications for intervening L2 motivation from a POA approach in English as a Foreign Language (EFL) classrooms were discussed.

Effects of Sentence Output Tasks on EFL Vocabulary Learning from a Bayesian Perspective

This article builds on the growing line of inquiry into the relative effectiveness of different tasks for vocabulary learning. Specifically, the study compares the efficacy of several sentence output tasks in EFL vocabulary learning through reading. To this end, evidence was weighed for one hypothesis over the alternative from a Bayesian perspective rather than in light of the commonly used null hypothesis significance testing (NHST), which depends heavily on the <i>p</i> values for statistical conclusions. Ninety-one EFL learners were randomly assigned to one of three word-focused sentence output tasks (i.e., L2-L1 translating, paraphrasing and writing), and were subsequently tested on their initial learning and retention of newly-encountered EFL words. Both Bayes factor analysis and Bayesian parameter estimation were employed to find evidence for task effects. With respect to initial word learning, moderate evidence was in favor of no difference between translating and paraphrasing, whilst weak evidence in favor of no difference between translating and writing as well as between paraphrasing and writing. For pedagogical purposes, no good evidence was for or against task effects. Regarding word retention, moderate evidence supported no task difference, and all the tasks fared equally well pedagogically. The results partially support the involvement load hypothesis and are discussed in terms of task difficulty, context generation and semantic elaboration.

듣기 기반 출력과업이 영어 관계대명사절 학습에 미치는 영향

The present study investigated the effects of output tasks on the learning of English relative clauses (RCs). Over a 5-week period, 67 Korean students at a university were asked to learn English RCs in one of three groups: a story reconstruction (SR) group, a dictation (DT) group, and an input (IP) group. The SR group listened to stories and reconstructed them using RCs, while the DT group listened to and wrote down the stories containing RCs. The IP group underlined the RCs in the texts. The effects of groups were measured by immediate and delayed posttests. The results revealed that the output groups outperformed the IP group in the two posttests, and the differences were greater in the delayed posttests. The results suggest that the output tasks help learners focus more on the target forms than passive learning. The learners viewed output as effective in promoting learning by motivating them actively in the tasks.

Comparing the Effects of Different Post-Listening Output Tasks on Second Language Incidental Vocabulary Acquisition: Revisiting the Involvement Load Hypothesis

Abstract Based on the Involvement Load Hypothesis, the present study investigated the differential effects of three post-listening output tasks (gap-filling, translation, and sentence-making) on immediate acquisition and retention of such vocabulary dimensions as productive knowledge of orthography, receptive recall of meaning and form, and productive knowledge of grammatical functions. Ninety second-year English majors were divided into three groups to finish listening plus one of the post-listening tasks. The results showed that the post-listening output tasks had positive effects on immediate acquisition of productive vocabulary knowledge, partially in agreement with the Involvement Load Hypothesis. However, the effects on vocabulary knowledge retention were found to be largely inconsistent with the Involvement Load Hypothesis. The finding thus challenges this hypothesis in that involvement load is not the only determining factor and suggests that the theoretical construct of involvement load should be constructed with more caution.

Differential Effects of Input-based and Output-based Tasks on L2 Vocabulary Learning

This paper reports on a quasi-experimental study that explored the differential effects of input- and output-based tasks on L2 vocabulary learning. The study adopted a pretest- posttest within-participants design, with sixty Vietnamese EFL university students. Participants in the experimental group completed four input and four output tasks in an online learning environment. The target items (ten single words and ten compounds) were counterbalanced across the tasks. The study measured vocabulary gains at four levels of sensitivity: spontaneous use of the target items, form recall, meaning recall and meaning recognition. The analyses showed that the input-based tasks resulted in higher vocabulary gains at the meaning recall level, whereas the output-based tasks resulted in better scores at the form recall level. No difference was found in the spontaneous use and meaning recognition tests.

The role of depression and other psychological factors in work ability among breast cancer survivors in Australia.

Identifying modifiable factors affecting work ability among cancer survivors is important. The primary aim of the present study was to examine the effects of depression and related psychological factors on work ability among breast cancer survivors in Australia. In this cross-sectional electronic and postal survey, Australian breast cancer survivors were investigated. Work status and conditions before and after cancer treatment were analysed. Work ability was measured using the Work Limitation Questionnaire©-Short Form (WLQ-SF) with its four domains (time management, physical tasks, mental-interpersonal tasks, and output tasks). Three psychological factors were investigated: depression, fear of cancer recurrence, and demoralisation. Sociodemographic and clinical data were also collected. Multivariate regression analysis was used to identify the associations of psychological factors with WLQ-SF. Among eligible survivors, 310 (50%) responded to the survey and were analysed. Nearly one third reported their work conditions had changed after cancer treatment. The depressed group reported limited work ability in 35%-44% of the four domains of WLQ-SF, while the non-depressed group reported limited work ability in only 8%-13%. At-work productivity loss was approximately fourfold higher in the depressed group than in the non-depressed group. In multivariate analysis, at-work productivity loss was associated with depression, demoralisation, and past history of anxiety. After breast cancer treatment, work conditions changed toward lower wages and working hours. Depression, demoralisation, and past history of anxiety were associated with lower work ability. Further evaluations of work rehabilitation in breast cancer survivors are warranted.

Correlation Analysis, Optimization and Computer Simulation of Teaching Efficiency and Blended Learning of Higher Vocational English Based on POA and CoI Algorithm

Blended learning has become a popular form of teaching. In order to study a more scientific and reasonable learning plan, this paper uses CoI theory and POA theory to design the teaching plan. First of all, the use of POA theory output, input, output of the three stages of pre-class, in-class, after-class design respectively. The main task before class is to assign output tasks, preview the contents of the class and exchange doubts with students. In class, the teacher checks the students’ study before class. According to the students’ learning situation and output results, pertinently explain the knowledge points and summarize the information involved in the students’ independent output. Review the output according to what you have learned after class. At the same time, the corresponding teaching plans are made according to the three elements of CoI theory: social existence, teaching existence and cognitive existence. After that, it analyzes the correlation between teaching efficiency and teaching methods. The experimental results show that blended learning is better than traditional teaching methods. Finally, the optimization scheme is put forward, and the summary and prospect are carried out.

출력 활동이 영어 관계대명사 학습에 미치는 영향

The present study attempts to examine the effects of output on the learning of English relative clauses. Seventy Korean learners of English at a university in Korea participated in the study. They were assigned to either a Noticing or an Output condition and were asked to learn the target grammar over a 4-week period. The Output group completed stories by connecting given sentences with relative clauses, whereas the Noticing group underlined the precedents and relative pronouns in the stories. The results revealed that the Output group outperformed the Noticing group on the two post-tests (immediate and delayed). The findings indicate that the output tasks help the students focus more on the target forms and reconstruct their interlanguage by noticing the gap between the target forms and their interlanguage forms. Accordingly, this leads them to better learning outcomes. The findings also suggest learning target forms in a meaningful context rather than discrete grammar items can promote language learning.

Structured Input vs. Structured Output Task’s Effects on the Acquisition of the English Causative Forms: Discourse-Level. Structured Input vs. Structured Output Task’s Effects on the Acquisition of the English Causative Forms: Discourse-Level.

This research paper presents a quasi-experimental empirical study investigating the effects of structured input and structured output tasks on the acquisition of English causative forms. This research is framed on VanPatten’s (1996) input processing theory. The grammatical form chosen for this investigation is affected by a processing strategy called the First Noun Principle. There are three variables included that make this study significant. These variables are having participants that are young learners who speak Arabic as an L1 and using discourse-level instrumentation. These variables make this study significant because the studies that investigated the effectiveness of structured input practice with these variables are in the minority. The study’s main questions are: (i) What are the short-term effects of structured input and structured output on the acquisition of English causative forms as measured with discourse-level interpretation tasks? (ii) What are the short-term effects of structured input and structured output on the learners’ ability to acquire the English causative forms as measured with discourse-level production tasks? Participants were school-age learners (aged 12-13) from an Arabic background with Arabic as an L1 who studied English as a second language in Kuwait. A pre and post-test procedure was adopted in this study. Two instructional groups were created, which are: (i) structured input; (ii) structured output. Discourse-level tasks were used in the study to assess the effectiveness of the two instructional treatments. Results were analyzed using descriptive statistics and ANOVA. The main findings support the view that discourse-level structured input tasks are a useful pedagogical intervention in helping young L2 learners from an Arabic background with Arabic as an L1 to process, interpret and produce accurate English causative forms. The main findings have theoretical and pedagogical implications for language learning and teaching.

A Heteromodal Word-Meaning Binding Site in the Visual Word Form Area under Top-Down Frontoparietal Control.

The integral capacity of human language together with semantic memory drives the linkage of words and their meaning, which theoretically is subject to cognitive control. However, it remains unknown whether, across different language modalities and input/output formats, there is a shared system in the human brain for word-meaning binding and how this system interacts with cognitive control. Here, we conducted a functional magnetic resonance imaging experiment based on a large cohort of subjects (50 females, 50 males) to comprehensively measure the brain responses evoked by semantic processing in spoken and written word comprehension and production tasks (listening, speaking, reading, and writing). We found that heteromodal word input and output tasks involved distributed brain regions within a frontal-parietal-temporal network and focally coactivated the anterior lateral visual word form area (VWFA), which is located in the basal occipitotemporal area. Directed connectivity analysis revealed that the VWFA was invariably under significant top-down modulation of the frontoparietal control network and interacts with regions related to attention and semantic representation. This study reveals that the VWFA is a key site subserving general semantic processes linking words and meaning, challenging the predominant emphasis on this area's specific role in reading or more general visual processes. Our findings also suggest that the dynamics between semantic memory and cognitive control mechanisms during word processing are largely independent of the modalities of input or output.SIGNIFICANCE STATEMENT Binding words and their meaning into a coherent whole during retrieval requires accessing semantic memory and cognitive control, allowing our thoughts to be expressed and comprehended through mind-external tokens in multiple modalities, such as written or spoken forms. However, it is still unknown whether multimodal language comprehension and production share a common word-meaning binding system in human brains and how this system is connected to a cognitive control mechanism. By systematically measuring brain activity evoked by spoken and written verbal input and output tasks tagging word-meaning binding processes, we demonstrate a general word-meaning binding site within the visual word form area (VWFA) and how this site is modulated by the frontal-parietal control network.