Reliable Performance Measures Research Articles

A data set of Japanese corporates’ green transformation technologies patents

Abstract* In 2020, the Government of Japan declared 2050 “carbon neutral” and launched a long-term strategy to create a “virtuous cycle of economy and environment.” Japanese corporations possess many technologies that contribute to decarbonization, which is important for expanding investments in green transformation technology inventory (GXTI) development. Patent data are the most reliable measure of business performance for applied research and development activities when investigating knowledge domains or technological evolution. Our paper describes a Japanese patent dataset of Japanese corporations’ green transformation (GX) patent applications on the Japan Platform for Patent Information, using a search method of bombinating International Patent Classification (IPC) codes and keywords. The dataset contains 37,476 GX patent applications from 298 corporations during the period 1999–2022.

Behavior Among Stock Price, Financial Performance, and Financial Distress Evidence from the Australian and New Zealand Stock Exchanges

Objective: The main objective of this study is to investigate the behavior among stock price, financial performance, and financial distress in Australian and New Zealand Stock Exchanges. Similar studies have been done on United States and European markets, but none on Australian and New Zealand Stock Exchanges. Our finding suggested that PE Ratio is not a reliable measure of financial performance in Australian and New Zealand Stock Exchanges. Method: Altman’s Z-score is applied to reflect the financial distress of the large listed firms in Australian Stock Exchange and New Zealand Stock Exchange while expected return on total investment minus the market premium, which is associated with the overall risk of a firm, is used as a proxy to stock price performance. In addition, this study also investigates the various effects on profitability, financial leverage, liquidity, growth, and size factors on stock performance. Results: Regression models are applied to investigate the stock performance upon independent variables’ effects. Results of the study show that all selected variables are reliable to use to measure performance, except P/E ratio. Moreover, Z-score, current ratio, P/E ratio, and ROE generally have significant positive relationship with stock price performance, while total assets, in general, have significant negative relationship with stock price performance. Conclusion: This paper has a potential to contribute in knowledge of stock price and financial performance. It also examines financial distress of the market. The objective of the study is to examine the dynamics of stock prices using market value ratios and financial performance using financial ratios under financial distress of selective countries particularly. Using the empirical data from listed companies from the stock exchanges, this study determines the stock performance in a unique way with the combination of market value ratios and financial ratios.

Machinability of different cutting tool materials for electric discharge machining: A review and future prospects

Electric Discharge Machining (EDM) is essential for shaping and cutting tool steel. EDM’s precision in machining difficult materials and tool steel characteristics are well known. EDM efficiency requires reliable performance measurement parameters. The physical shape and mobility of the electrode tool are critical in EDM research. Layer machining is an advanced method that removes material in a sequential manner to produce intricate 3D shapes in tool steel and several other materials. The improvement in layer machining methods with precise toolpath algorithms, adaptive layer thickness management, and real-time monitoring systems is required to maximize precision and efficiency. Response surface methodology, the artificial neural network, and other techniques are necessary to optimize EDM operations and maximize performance. Many researchers experimented with electrode shapes and movement patterns to enhance the removal of material and the quality of surfaces. Investigation of complex electrode structures and innovative tool path strategies has been performed in previous studies. It was very difficult to consider various factors during the EDM operation; hence, the present review summarizes the positive outcomes of previous research. The review emphasizes optimizing pulse duration and discharge current to improve EDM efficiency. The present comprehensive review discusses research on EDM in three main areas: electrode tool geometry and motion, tool steel layer processing, and factors for measuring EDM performance. The objective of the present review is to focus on measuring material removal rates, surface roughness, tool wear, and energy usage. The present review concludes that EDM is crucial to machining tool steel and cutting tool materials. Integrating and hybrid machining technologies can improve performance, and improved optimization techniques are crucial. It also recognizes knowledge gaps and explores new frontiers in this dynamic field.

Sensor-Based Quantitative Assessment of Children's Fine Motor Competence: An Instrumented Version of the Placing Bricks Test.

The assessment of fine motor competence plays a pivotal role in neuropsychological examinations for the identification of developmental deficits. Several tests have been proposed for the characterization of fine motor competence, with evaluation metrics primarily based on qualitative observation, limiting quantitative assessment to measures such as test durations. The Placing Bricks (PB) test evaluates fine motor competence across the lifespan, relying on the measurement of time to completion. The present study aims at instrumenting the PB test using wearable inertial sensors to complement PB standard assessment with reliable and objective process-oriented measures of performance. Fifty-four primary school children (27 6-year-olds and 27 7-year-olds) performed the PB according to standard protocol with their dominant and non-dominant hands, while wearing two tri-axial inertial sensors, one per wrist. An ad hoc algorithm based on the analysis of forearm angular velocity data was developed to automatically identify task events, and to quantify phases and their variability. The algorithm performance was tested against video recordings in data from five children. Cycle and Placing durations showed a strong agreement between IMU- and Video-derived measurements, with a mean difference <0.1 s, 95% confidence intervals <50% median phase duration, and very high positive correlation (ρ > 0.9). Analyzing the whole population, significant differences were found for age, as follows: six-year-olds exhibited longer cycle durations and higher variability, indicating a stage of development and potential differences in hand dominance; seven-year-olds demonstrated quicker and less variable performance, aligning with the expected maturation and the refined motor control associated with dominant hand training during the first year of school. The proposed sensor-based approach allowed the quantitative assessment of fine motor competence in children, providing a portable and rapid tool for monitoring developmental progress.

Business Process Outsourcing 14 Cybersecurity KPIs Towards a Strategic Information Security

In today’s rapidly evolving digital landscape, the Business Process Outsourcing (BPO) industry faces an ever-increasing threat of cyber-attacks and data breaches, underscoring the critical importance of robust information security practices. To address these challenges, organizations must develop effective strategic information security, supported by reliable performance measurement tools such as Key Performance Indicators (KPIs). This research aims to investigate the information security implemented by BPO companies and their utilization of the 14 cybersecurity KPIs to track and enhance their information security measures. A mixed-methods strategy is used in the study, integrating qualitative and quantitative techniques. The qualitative component involves thematic analysis of key informant interviews, while the quantitative component entails the analysis of Likert-scale questionnaires and numerical data. The findings of this research provide valuable insights into the significance of strategic information security in achieving effective information security management within the BPO industry. The study reveals the specific elements incorporated in these plans, such as risk assessment methodologies, security controls implementation, employee training, incident response strategies, and compliance measures. The implications of this study offer valuable insights for BPO organizations to develop robust strategic information security tailored to their specific context, ensuring effective alignment with organizational objectives and risk management strategies. Moreover, the research contributes to the existing literature on information security management frameworks, emphasizing the role of cybersecurity KPIs in evaluating security performance. Keywords: cybersecurity, information security, KPI, business process outsourcing industry, information security management

PEDOT/CNT Flexible MEAs Reveal New Insights into the Clock Gene's Role in Dopamine Dynamics.

Substantial evidence has shown that the Circadian Locomotor Output Cycles Kaput (Clock) gene is a core transcription factor of circadian rhythms that regulates dopamine (DA) synthesis. To shed light on the mechanism of this interaction, flexible multielectrode arrays (MEAs) are developed that can measure both DA concentrations and electrophysiology chronically. The dual functionality is enabled by conducting polymer PEDOT doped with acid-functionalized carbon nanotubes (CNT). The PEDOT/CNT microelectrode coating maintained stable electrochemical impedance and DA detection by square wave voltammetry for 4 weeks in vitro. When implanted in wild-type (WT) and Clock mutation (MU) mice, MEAs measured tonic DA concentration and extracellular neural activity with high spatial and temporal resolution for 4 weeks. A diurnal change of DA concentration in WT is observed, but not in MU, and a higher basal DA concentration and stronger cocaine-induced DA increase in MU. Meanwhile, striatal neuronal firing rate is found to be positively correlated with DA concentration in both animal groups. These findings offer new insights into DA dynamics in the context of circadian rhythm regulation, and the chronically reliable performance and dual measurement capability of this technology hold great potentialfor a broad range of neuroscience research.

The effect of a preceptorship programme on newly hired experienced professional nurses’ self-efficacy in nursing clinical competency in Saudi Arabia

AimThis study aimed to determine if participation in a preceptorship programme affected newly hired experienced professional nurses’ self-efficacy in clinical practice and whether their perceived competence was related to respondents’ demographic profiles at a university hospital in Saudi Arabia. BackgroundFormal preceptorship programmes are initiated with the aim of improving the characteristics related to new workplace adaptations, such as reducing stress and anxiety, improving effective communication, enhancing clinical performance and competencies, and learning new clinical skills. These programmes’ effectiveness in improving nurses’ self-efficacy in clinical practice is seldom measured, especially among newly hired experienced professional nurses. MethodsA quasi-experimental, quantitative pre- and post-test one-group study design with convenient sampling was used. ResultsSelf-efficacy post-test scores were significantly higher than pre-test scores (p<0.001). The programme positively influenced the respondents’ self-efficacy scores. As perceived by the respondents, the preceptorship programme had a positive effect and improved their clinical performance in the nursing process. There was no relationship between perceived competence and respondents’ demographic profiles. ConclusionsReporting on self-efficacy in clinical practice may not be the most effective, reliable or accurate measure or evaluation of clinical performance since self-scoring may be affected by existing cultural implications and a strong sense of self-efficacy. The self-scoring results may be followed by qualitative information, like interviews, to determine respondents’ knowledge of the nursing process, direct observation of performance in the clinical area and care outcomes, and confirm if the respondents’ clinical performance matches the obtained results.

Measuring Situational Cognitive Performance in the Wild: A Psychometric Evaluation of Three Brief Smartphone-Based Test Procedures

Mobile devices provide new opportunities to draw conclusions about cognitive performance in everyday situations. To gain insights into cognitive performance patterns in healthy adult populations, we adapted three established cognitive tests for smartphone use: the Digit Symbol Substitution Task (DSST), Sustained Attention to Response Task (SART), and Psychomotor Vigilance Task (PVT). To increase their feasibility for ambulatory assessment, we identified the minimum measurement durations that provide reliable and valid state measures of cognitive performance. Over 2 weeks, 46 participants performed each test once per day at random times, along with self-reports (e.g., on concentration, mood, and mental demands). The validity and reliability of change are promising for the 30-second PVT and 90-second DSST and SART. The DSST and SART provide fruitful outcomes for ambulatory field studies linked to mood, stress, and mental demands. We provide digital versions of the adapted DSST and SART online for free.

The value of using real-world evidence as a source of clinical evidence in the European medical device regulations: a mixed methods study

ABSTRACTObjectivesThis study investigates the benefits, limitations and awareness of using Real World Evidence and Real World Data for post-market clinical follow-up studies and clinical evaluation reports in the European Medical Device Regulations.MethodologyA mixed methods study was utilized with qualitative interviews and a quantitative survey.ResultsThe findings from the study demonstrate that in the case of the Medical Device Regulations, opportunities exist for manufacturers of legacy devices to conduct Real World Evidence studies to bridge gaps in clinical evidence. The primary value of Real World Evidence lies in its ability to provide an accurate and, therefore, more reliable measure of device safety and performance. As a measure of safety and performance, it supplements clinical evidence generated from pre and post-market clinical investigations, reducing the costs associated with these studies and supporting the manufacturer’s benefit: risk conclusion.ConclusionThis study provides insight into how the medical device industry could utilize Real World Evidence and have an initiative in the EU similar to the FDA-sponsored NESTcc partnership. This would aid medical device manufacturers in transitioning to the MDR clinical evaluation requirements and mitigate the impact on medical device availability in the EU.

Development of Radiographer Performance Assessment Instruments in the Radiology Installation of Dr. H. Abdul Moeloek Lampung Province

At Abdul Moeloek Regional General Hospital in Lampung Province, evaluating radiographers' performance is crucial for ensuring quality radiology services. The current assessment tools for ongoing professional practice and work behavior lack clear, objective indicators and targets. This research uses research and development (R&amp;D) research methods with a quasi-experimental posttest-only non-equivalent control group design. Data were collected using observation, interviews, and FGD. Expert validation and design improvements are conducted, followed by a trial involving 20 radiographers. Validity is tested using corrected items to total correlation, and reliability is assessed using Cronbach's alpha. Next, expert validation tests and design improvements were carried out by 3 experts, and then a limited trial of 20 RSUDAM radiographers carried out a validity test using corrected items to total correlation. Meanwhile, the reliability test uses Cronbach's alpha. The developed instrument, based on indicators and targets, demonstrates valid and reliable measures of radiographer performance. Expert validation through FGD confirms the instrument's effectiveness. Validity test results indicate significance (&gt; 0.05), and reliability tests show Cronbach's alpha values (&gt; 0.60). Normality tests reveal non-distributed data (p-value &lt; 0.05). The Wilcoxon test, comparing old and new instruments, yields a significant Asymp.sig value (0.000) for the discipline variable, indicating improved performance assessment.

Evaluation from the Perspective of Standardized Tests (Pisa) at the Pre-University Level of the Dominican Republic

The Dominican Republic faces great challenges in pre-university education and one of them is the evaluation culture. Evaluation as a systemic process is an added value to educational quality to a large extent, to its human capital, and to achieve success in a changing world. Recent studies have shown that achieving better educational outcomes and measurements provides considerable benefits for individuals and for society as a whole (OECD 2010a, 2010c). This ensures the achievement of effective measures of student performance and development and, therefore, educational performance. Reliable and valid measures of school performance can serve as a basis for educational policies and programs and their accountability, school improvement processes, and so that families and society as a whole have more information about the effectiveness of educational systems.

A self-paced 15-minute cycling time trial is a reliable performance measure in recreationally active individuals

ABSTRACT Cycling time trial (TT) protocols have been shown to be reliable in trained cyclists, but their reproducibility in lesser-trained individuals is unknown. This study examined the reliability of a self-paced 15-minute cycling TT in recreationally active individuals. Twelve recreationally active males (age 27 ± 3 y; body mass 75.2 ± 8.9 kg; V ˙ O2peak = 51.10 ± 7.53 ml∙kg∙min−1) completed a V ˙ O2peak test and four experimental trials, separated by > 48 h. Experimental trials consisted of 10 min cycling at 60% Wmax, followed by a self-paced 15-min TT. Heart rate and work done were recorded every 5 min during the TT; and coefficient of variation (CV) was calculated. Work done was not different (P = 0.706) between trials (193.2 ± 45.3 kJ; 193.2 ± 43.5 kJ; 192.0 ± 42.3 kJ; 193.9 ± 42.8 kJ). Within participant CV ranged from 0.5–4.9% for the four TTs, with a mean CV of 2.1%. Mean CV decreased from 2.0% (range 0.1–5.0%) for the first two TTs to 1.7% (range 0.2–5.6%) for the second and third TTs, and further decreased to 1.0% (range 0.2–1.8%) for the third and fourth TTs. In conclusion, the use of a short-duration self-paced cycling TT in recreationally active individuals is a reliable performance measure.

Distribution networks reliability assessment considering distributed automation system with penetration of DG units and SOP devices

The electric utility industry is moving towards a deregulated and competitive market to meet customer expectations. So, system performance and reliability assessments are getting targeted more than before. Several performance measures of reliability are developed in the literature. In this research, the NEPLAN Simulator reliability analysis module is used to determine all the reliability indices in different cases of study considering the effect of the Advanced Distributed Automation System (ADAS). The analysis also benefits from the presence of Penetration of Distributed Generation (DG) units and Soft Open Points (SOP) to enhance system reliability alongside power quality. Therefore, this paper provides a methodology based on a cost/benefit study for distribution networks to define the best location of DG units and SOP devices that leads to better reliability indices. The objectives of the study are demonstrated and investigated through Bus 4 of the standard reliability Roy Bollington Test System (RBTS). NEPLAN uses a tool to apply the Reliability Centered Maintenance (RCM) strategy, which leads to a substantial reduction of maintenance expenses. Simulation results indicate a significant reduction in system reliability cost by 65.8% with significant enhancement in the average of all reliability indices and at each load point too such as Customer Average Interruption Duration Index (CAIDI) which is improved from 40.899 to 29.883, Energy not supplied also is declined by 52.34%. It is also worth saying that, even using DG units or SOP devices separately leads to positive results but the best outcomes are obtained with an appropriate combination of both.

Evaluation of cyclic repetition frequency based algorithm for fetal heart rate extraction from fetal phonocardiography

The passive and non-invasive approach of fetal heart rate (FHR) estimation using fetal phonocardiography (fPCG) provides an alternative to the commonly used Doppler ultrasound-based technique. The acoustic signal recorded on the mother’s abdomen is corrupted by noise, making FHR estimation difficult. Thus, an accurate and reliable FHR measurement algorithm from the fPCG signal is needed. This study evaluates a cyclic repetition frequency-based algorithm with additional FHR validation steps on the clinical recordings from three datasets with varying signal quality. The FHR from the algorithms is compared against the reference FHR measurements, and the accuracy and reliability performance measure is determined. The cyclic repetition algorithm significantly improves (p < 0.05) against the Hilbert-envelope autocorrelation algorithm for poor-quality signals. The algorithm's average reliability across three datasets is 93%, and accuracy is 4.5 beats per minute. The algorithm performs well for the gestation week ranging from 26 −40 and for the labor and antenatal ward data.

Comparison of Priority Occupational Performance of Children with Cerebral Palsy from the Perspective of Children, Parents, and Occupational Therapists in Isfahan in 2021

Background: Cerebral palsy (CP) is a non-progressive complication of the developing brain that leads to movement problems and occupational performance limitations in the activities of daily living of children with CP. Therefore, getting a sense conception of the children, parents, and occupational therapists’ occupational performance priority to providing client-centered services in performing the activities of daily living of children with CP is extremely important. Objectives: This study aimed to determine the priorities of occupational performance from the perspective of children with CP, parents, and occupational therapists and compare their priorities with each other. Methods: This descriptive-analytical comparative study was conducted in Isfahan in 2021. The research population included 115 CP children aged 6 - 12 years, their parents, and occupational therapists in the occupational therapy department of rehabilitation clinics, rehabilitation centers, and occupational therapy department of hospitals in Isfahan city. Occupational performance priorities of children with CP, their parents, and occupational therapists were assessed through the valid and reliable Canadian occupational performance measurement (COPM). Results: In this research, 115 children with CP, their parents, and occupational therapists participated. Comparison of occupational performance priorities of children with CP from the perspective of children, parents, and occupational therapists showed that the most important occupational performance priority of children with CP from the perspective of these three groups is the same and related to self-care activities and includes activities, such as walking, going up and down the stairs, toileting, and going to the bathroom (P &lt; 0.05). Conclusions: Children with CP, their parents, and occupational therapists reported occupational performance priorities in similar performance dimensions in the age groups of 6 to 12 years for children with CP, which are mainly focused on self-care activities. The attention of occupational therapists to the priorities of children and parents can increase their participation in the rehabilitation process of these children.

Prioritizing patient experience: Validation of the person-centered contraceptive counseling measure as a performance measure

ObjectiveA performance measure assessing how often patients experience patient-centered contraceptive counseling can inform quality improvement in contraceptive care and enable efforts to enhance equity in the provision of this care. We sought to test the validity and reliability of the Person-Centered Contraceptive Counseling (PCCC) measure as a performance measure, in preparation for application for endorsement from the National Quality Forum. Study designWe combined data from two research studies, a statewide quality improvement assessment, and a dedicated data collection effort at nine sites, all collected between 2009 and 2019 at 22 total sites in the United States, to evaluate the validity and reliability of the four-item PCCC measure aggregated at the provider and facility level. We scored the PCCC dichotomously as a "top-box" score of 20 versus less than 20. We assessed reliability using the Spearman-Brown coefficient, and validity by comparing scores on the PCCC to aggregated scores on single-item measures of patient satisfaction. ResultsWe included 22 facilities and 34 providers in the analyses. The average PCCC top-box score by provider and facility was 81% and 79%, respectively. We found adequate reliability with panel sizes of 20–50. PCCC scores were strongly associated with the single-item measures of satisfaction. There were notable disparities by race/ethnicity and language (Spanish vs English) in PCCC scores. DiscussionThe PCCC is a valid and reliable performance measure for use at the provider and facility level. The development and use of the measures of patient experience, like the PCCC, is critical for prioritizing patient-centeredness in reproductive health care. ImplicationsThe PCCC measure can facilitate the identification of gaps and disparities in patient-centered contraceptive counseling and enable quality improvement to promote quality, equitable contraceptive care. Using this measure provides the opportunity to enhance patient experience of care and build trust between contraceptive providers and the communities they serve.

Technology Trend Analysis of Japanese Green Vehicle Powertrains Technology Using Patent Citation Data

As automobiles are major contributors to greenhouse gas emissions, the technological shift towards vehicle powertrain systems is an attempt to lower problems such as emissions of carbon dioxide and nitrogen oxides. Patent data are the most reliable measure of business performance for applied research and development activities when investigating knowledge domains or technology evolution. This is the first study on Japanese patent citation data of the green vehicle powertrains technology industry, using the social network analysis method, which emphasizes centrality estimates and community detection. This study not only elucidates the knowledge by visualizing flow patterns but also provides a precious and congregative method for verifying important patents under the International Patent Classification system and grasping the trend of the new technology industry. This study detects leading companies, not only in terms of the number of patents but also the importance of the patents. The empirical result shows that the International Patent Classification (IPC) class that starts with “B60K”, which includes hybrid electric vehicle (HEV) and battery electric vehicle (BEV), is more likely to be the technology trend in the green vehicle powertrains industry.

Sensitivity of Travel Reliability Measures to Major System Changes

The performance-based planning and programming approach established by the Moving Ahead for Progress in the 21st Century Act (MAP-21) has necessitated the process of estimating congestion (delay) and reliability benefits from transportation system improvements. Agencies are experiencing a shift in policy focus from the average travel time to the reliability of travel time. By examining empirical data on before-and-after archived travel times at identified project locations to find evidence of observed reliability benefits, this paper addresses a gap in the literature about the overall role that reliability benefit estimations play in the decision-making process during transportation project prioritization, and the relative effects of different strategies on different reliability measures. In the process, the study investigated why and how the way the reliability measures are defined can affect their ability to capture the impacts (and benefits) of major system changes. Data for four different broad improvement categories and three different reliability performance measures (planning time index-95 [PTI95], planning time index-80 [PTI80], and level of travel time reliability [LOTTR]) revealed varying impacts of different strategies on different measures. PTI80 was better able to capture ground-level benefits experienced by most travelers and showed higher sensitivity to system improvements than PTI95. Another key finding was that LOTTR’s threshold-based definition limited its ability to capture observed reliability benefits accurately. The results provide a picture of magnitude and relative levels of observed benefits from different system improvements, which could help planners and practitioners better understand, anticipate, and incorporate the benefits expected from planned changes.

Working Vacation Policy for Load Sharing K-out-of-N: G System

In industrial systems, the K-out-of-N: G system is a prominent type of redundancy. The load sharing protects such system from malfunctioning/destroying and avoids overload problem that affects the system reliability in a significant manner. In this paper we develop a Markovian model of load-sharing K-out-of-N: G system having non-identical repairable components wherein the server may on working vacation. During his vacation period, the server repairs the failed components with different service rates rather than completely terminating service rate. The failed component gets immediately repaired by the server if not on vacation, and unequal load is distributed among remaining surviving components. The lifetime of each component is load dependent followed by non-identical exponential distribution with different failure rates. The system is failed down due to common cause with failure density which is also exponentially distributed. We suggest closed structure analytic expressions for reliability, cost estimation and other performance measures of the load-sharing K-out-of-N: G repairable system by incorporating the concept of working vacation. For the solution aspiration, Runge-Kutta method is utilized to solve the system of differential equations. Furthermore, we perform the numerical analysis for two illustrations 1-out-of-3: G system and 3-out-of-4: G system. The numerical simulation is carried out for the validation of analytical results which are exhibited and compared by giving numerical outcomes and neuro-fuzzy outcomes based on fuzzy interference system with the help of MATLAB.

Online assessment of narrative macrostructure in adult Irish-English multilinguals.

BackgroundOnline assessment of narrative production and comprehension became an important component of language assessment during the COVID-19 pandemic. This study aimed to establish quantitative measures of narrative macrostructure in the production and comprehension of adult Irish-English bilinguals in an online assessment.MethodsA total of 30 Irish-English bilingual adults participated in an online assessment of oral narrative production and comprehension. Narratives were elicited using LITMUS-MAIN for Irish and English. Story-tell elicitation method was used for all stories. Twenty participants produced Baby Birds and Baby Goats story pairs while 10 participants produced Cat and Dog story pairs. Quantitative measures of story structure, comprehension score, and the overall number of Internal State Terms (ISTs) in production and comprehension were compared across the story pairs, languages, and the output type (production vs. comprehension).ResultsA general linear model indicated no differences in either story structure or story comprehension scores across languages for both sets of stories. Combined analysis for all participants and stories indicated no difference in the story structure scores or comprehension scores across the languages or the story pairs. While the overall number of ISTs was the same across languages, a higher number of ISTs was observed in comprehension relative to production in both languages for Cat and Dog story pair only, but not for Baby birds and Baby goats' stories. The major benefit of using online assessment was the accessibility of participants. The major drawback was the inability to control the environment and the quality of the internet connection.Conclusion and implicationsWhile online assessment increased the availability of participants, which is a significant factor in rural Ireland characterized by low population density and the high percentage of Irish speakers, the availability of stable internet connection limited the applicability of online assessment. Measures of narrative macrostructure were stable across the languages and the story pairs. This is important because of high variability in exposure to Irish, frequent code-switching, and a high number of morphosyntactic errors due to rapid language change that characterizes Irish-English bilinguals. Identifying reliable measures of language performance for Irish-English adult speakers is an important step toward establishing developmental norms for Irish-English bilinguals.