Trial Data Management Research Articles

Participatory research at scale: A comparative analysis of four approaches to large-scale agricultural technology testing with farmers

Tailoring agricultural technology options to the diverse conditions of smallholder farmers requires innovative approaches for testing these technologies with farmers across varied contexts, while incorporating their feedback into learning and decision-making processes. This study compares four such approaches: the Farmer Field School on Participatory Plant Breeding (FFS-PPB), Farmer Research Network (FRN), Crowdsourcing–Triadic comparisons of technologies (Tricot), and adapted Mother–Baby Trial (MBT) as implemented by four concrete projects. The objectives are to provide detailed descriptions of these approaches and their project-specific implementations, identify and analyze common aspects and differences, and derive insights to guide future farmer-inclusive projects aiming at contextual scaling of agricultural technologies. A literature review, key informant interviews, and a systematic content analysis were conducted for the analysis. Common features include cascade training models, simple farmer-managed experiments, and the use of digital tools for data collection. Major differences lie in the extent of farmer–researcher collaboration and decision-making, as well as how technology option-by-context interactions are addressed. The FRN, FFS-PPB, and adapted MBT approaches involve farmers in decision-making throughout most stages of research, including co-learningcycles that adapt the research design and technology options to farmers’ needs. Although these approaches require more training and expertise, they increase the likelihood of achieving relevant results that farmers can implement in practice. In contrast, more standardized approaches like the Crowdsourcing–Tricot streamline the implementation, data management and analysis of large-scale trials, but have limitations in capturing the underlying reasons for farmers’ preferences. Among the studied approaches, the FRN as implemented by the Women's Fields project in Niger is particularly effective in identifying which options best suit specific farming contexts.

Optimizing Clinical Trial Eligibility Design Using Natural Language Processing Models and Real-World Data: Algorithm Development and Validation.

Clinical trials are vital for developing new therapies but can also delay drug development. Efficient trial data management, optimized trial protocol, and accurate patient identification are critical for reducing trial timelines. Natural language processing (NLP) has the potential to achieve these objectives. This study aims to assess the feasibility of using data-driven approaches to optimize clinical trial protocol design and identify eligible patients. This involves creating a comprehensive eligibility criteria knowledge base integrated within electronic health records using deep learning-based NLP techniques. We obtained data of 3281 industry-sponsored phase 2 or 3 interventional clinical trials recruiting patients with non-small cell lung cancer, prostate cancer, breast cancer, multiple myeloma, ulcerative colitis, and Crohn disease from ClinicalTrials.gov, spanning the period between 2013 and 2020. A customized bidirectional long short-term memory- and conditional random field-based NLP pipeline was used to extract all eligibility criteria attributes and convert hypernym concepts into computable hyponyms along with their corresponding values. To illustrate the simulation of clinical trial design for optimization purposes, we selected a subset of patients with non-small cell lung cancer (n=2775), curated from the Mount Sinai Health System, as a pilot study. We manually annotated the clinical trial eligibility corpus (485/3281, 14.78% trials) and constructed an eligibility criteria-specific ontology. Our customized NLP pipeline, developed based on the eligibility criteria-specific ontology that we created through manual annotation, achieved high precision (0.91, range 0.67-1.00) and recall (0.79, range 0.50-1) scores, as well as a high F1-score (0.83, range 0.67-1), enabling the efficient extraction of granular criteria entities and relevant attributes from 3281 clinical trials. A standardized eligibility criteria knowledge base, compatible with electronic health records, was developed by transforming hypernym concepts into machine-interpretable hyponyms along with their corresponding values. In addition, an interface prototype demonstrated the practicality of leveraging real-world data for optimizing clinical trial protocols and identifying eligible patients. Our customized NLP pipeline successfully generated a standardized eligibility criteria knowledge base by transforming hypernym criteria into machine-readable hyponyms along with their corresponding values. A prototype interface integrating real-world patient information allows us to assess the impact of each eligibility criterion on the number of patients eligible for the trial. Leveraging NLP and real-world data in a data-driven approach holds promise for streamlining the overall clinical trial process, optimizing processes, and improving efficiency in patient identification.

СОЦІАЛЬНО-ЕКОНОМІЧНІ АСПЕКТИ ВИКОРИСТАННЯ ТЕХНОЛОГІЇ БЛОКЧЕЙН У МЕНЕДЖМЕНТІ ДАНИХ КЛІНІЧНИХ ДОСЛІДЖЕНЬ

The article considers blockchain technology, which is one of the types of distributed ledger technology and currently has real prospects for application in the medical field, in particular for managing data arrays in clinical trials. The essence of the currently existing discussion among medical managers and other employees on the feasibility of introducing blockchain technology into clinical trials is revealed. To address this issue, the authors carried out a thorough analysis of the social and economic aspects of the application of blockchain technology in clinical trial data management. As a result of the analysis of a large number of scientific publications, social effects of the use of blockchain technology in the management of clinical trial data have been established and systematized. The article proves that the social effect of using blockchain technology in the management of clinical trial data is certainly significant. This factor clearly indicates the need to use this technology in the practice of clinical trials. The authors argue that it is important for a contractual research organization, pharmaceutical company, healthcare institution or other entity in which a clinical trial is performed to take into account the economic component of using blockchain technology to manage data sets in the process of performing clinical trials. Therefore, for the final decision on the adoption of blockchain technology in the management of clinical trial data, the article also considers the approach to taking into account the economic aspect. To determine the economic effect of the introduction of blockchain technology, an appropriate calculation method has been proposed. According to this methodology, the effect of the application of blockchain technology depends on: the proportion of conflicting disease and treatment records (drug studies) that require reconciliation; percentage reduction in the cost of issuing one conditional entry in the patient's medical card. It is indicated that the predictive assessment of the effect for a medical institution depends on a specific blockchain solution. The conditional example in the article shows the expediency of applying the proposed methodology for determining the economic effect of using blockchain technology in the management of clinical trial data.

Pharma Software - A Complete Overview

With the recent advancements in Pharmaceutical Industry, there comes an increased complexity in data and products. In a Pharmaceutical Industry, various aspects are considered simultaneously. So, nowadays various advanced softwares are used to handle these parameters. A Software is a set of programs that tells a computer what tasks it should perform. It consists of a series of code written by a programming language, which is then interpreted by a computer. This process is known as compilation. Thus, it makes our work easier by maintaining all the records and performing necessary calculation and much more. In a pharmaceutical industry variety of software are used as the tasks that needs to be performed are also of various types. For example, Record keeping of a Batch of Tablet in a manufacturing section, Quality Control Test Results, Regulatory Management, Marketing And Sales Data Handling, Clinical Trial Data Management, etc. Sometimes, a single software is used and sometimes a combination of two or more are used. There are roughly 19 types of software used in the whole Pharmaceutical Industry, and among them various companies have their wide range of products. Currently, after the implementation of GMP, GLP and GCP, the use of the software has marginally increased. Keywords: Pharmaceutical, Software, Industry, Recent software, Pharma- Industry

Blockchain-based solutions for clinical trial data management: a systematic review

Blockchain technology can reduce the need for intermediaries in various types of transactions by providing a decentralized and secure ledger that can be accessed and updated by all parties involved in the transaction. Clinical trials are essential for bringing new drugs and therapies to market, but the current clinical research process is often marred by inefficiencies, data inaccuracies, and a lack of transparency. The implementation of blockchain technology in clinical trials has the potential to address these challenges by providing a secure and transparent platform for data management. By leveraging the power of blockchain, healthcare providers can improve the integrity and accuracy of clinical trial data, enhance trust in the clinical research process, and ultimately improve patient outcomes. In this article, we propose the use of blockchain technology in clinical trials and explore its potential benefits for the healthcare. The implementation of a blockchain-based data management system for clinical trials holds significant potential to address several challenges associated with the current clinical research process. By improving the integrity and security of medical data, enhancing trust, and easing regulatory burden, such a system can promote the efficient and effective conduct of clinical trials. The adoption of a blockchain-based solution for clinical trial data management has the potential to optimize costs, contributing to the sustainability of healthcare services. It also provides a model for future research and development of blockchain-based solutions in the field of clinical research.

Identifying the association rules between adverse events and concomitant medicines in clinical trial data management using random forest

Adverse events (AEs) and concomitant medications (CMs) underreporting remains a recurrent issue in clinical trials. This study aimed to build a mapping relationship between the AE and CM using a random forest (RF) model that can be embedded in the current Electronic Data Capture (EDC) system, to enable a reliable detection of underreporting AE or CM. Current data including 524 AEs and 684 CMs for 13,905 individuals, were taken from three cancer clinical trials and six other studies that are non-public. An additional validation dataset was consisted of 448 independent patients with 269 unique AEs and 407 CMs from a cancer clinical trial on Paclitaxel. We found that the machine learning method exhibited in a learning task between AE and CM well for a common AE like hypertension that was caused by a single cause, but it was ineffective when the AE, such as the increased blood alkaline phosphatase, was caused by complex reasons or just an associated symptom of some diseases. This study suggests the potential of automatically detecting the underreported AE and CM in detail, and it will improve further safety and validity inspections from clinical trials. ABBREVIATIONS: AE, adverse event; ATC, anatomical therapeutic chemical; CM, concomitant medication; EDC, electronic data capture; FDA, Food and Drug Administration; GCP, good clinical practice; MedDRA, medical dictionary for regulatory activities; NMPA, National Medical Products Administration; PT, preferred term; RCT, randomized controlled trial; RF, random forests; SDV, source data verification; VIM, variable importance measure

Lessons learned in the Apple Heart Study and implications for the data management of future digital clinical trials

ABSTRACT The digital clinical trial is fast emerging as a pragmatic trial that can improve a trial’s design including recruitment and retention, data collection and analytics. To that end, digital platforms such as electronic health records or wearable technologies that enable passive data collection can be leveraged, alleviating burden from the participant and study coordinator. However, there are challenges. For example, many of these data sources not originally intended for research may be noisier than traditionally obtained measures. Further, the secure flow of passively collected data and their integration for analysis is non-trivial. The Apple Heart Study was a prospective, single-arm, site-less digital trial designed to evaluate the ability of an app to detect atrial fibrillation. The study was designed with pragmatic features, such as an app for enrollment, a wearable device (the Apple Watch) for data collection, and electronic surveys for participant-reported outcomes that enabled a high volume of patient enrollment and accompanying data. These elements led to challenges including identifying the number of unique participants, maintaining participant-level linkage of multiple complex data streams, and participant adherence and engagement. Novel solutions were derived that inform future designs with an emphasis on data management. We build upon the excellent framework of the Clinical Trials Transformation Initiative to provide a comprehensive set of guidelines for data management of the digital clinical trial that include an increased role of collaborative data scientists in the design and conduct of the modern digital trial.

Challenges and Gaps in Clinical Trial Genomic Data Management.

Challenges and Gaps in Clinical Trial Genomic Data Management.

Clinical Research Coordinator and Clinical Research Nurse: education and role at the Clinical Trial Center of Alessandria

The increasing complexity of clinical research makes it indispensable for an Institution that wants to conduct clinical trials to have an Organisational Structure and a dedicated multidisciplinary team. Since 2013, the public Hospital "SS Antonio and Biagio and C. Arrigo" of Alessandria (AO AL) has established a Clinical Trial Center (CTC), that has now become a centralized inter-company sector of the Research and Innovation Department. The CTC is part of the Research Infrastructure with its head office at AO AL and a further office at the Local Health Authority Territorial Hospital "S. Spirito" of Casale Monferrato (ASL AL). The CTC aims to design high standard clinical trials, ensuring better efficiency in their activation, management and data management according to the Good Clinical Practice (GCP) guidelines. Over the years, the CTC has grown and gained professional skills such as the Clinical Research Coordinator (CRC) and the Clinical Research Nurse (CRN), not yet officially recognized by the National Health System. These are skilled professional figures, that have been adequately trained even with post-graduate education through a 1st level Master in "Data Management and Coordination of Clinical Trials" which was activated by the University of Eastern Piedmont in collaboration with AO AL. The number and diversity of the activities carried out by CRCs and CRNs within the CTC has led them to develop a tool that strategically permits to recollect and record the time needed to perform these tasks with the aim of evaluating the different types of activities undertaken and assessing the needs of new staff. The organizational model adopted by the AO AL and the ASL AL through a centralized inter-company CTC represents a unique organizational model in the regional scenario and a peculiarity within the Public Health Authority not as yet recognized as Scientific Institutes for Research, Hospitalization and Healthcare.

Blockchain Integration With Digital Technology and the Future of Health Care Ecosystems: Systematic Review.

BackgroundIn the era of big data, artificial intelligence (AI), and the Internet of Things (IoT), digital data have become essential for our everyday functioning and in health care services. The sensitive nature of health care data presents several crucial issues such as privacy, security, interoperability, and reliability that must be addressed in any health care data management system. However, most of the current health care systems are still facing major obstacles and are lacking in some of these areas. This is where decentralized, secure, and scalable databases, most notably blockchains, play critical roles in addressing these requirements without compromising security, thereby attracting considerable interest within the health care community. A blockchain can be maintained and widely distributed using a large network of nodes, mostly computers, each of which stores a full replica of the data. A blockchain protocol is a set of predefined rules or procedures that govern how the nodes interact with the network, view, verify, and add data to the ledger.ObjectiveIn this article, we aim to explore blockchain technology, its framework, current applications, and integration with other innovations, as well as opportunities in diverse areas of health care and clinical research, in addition to clarifying its future impact on the health care ecosystem. We also elucidate 2 case studies to instantiate the potential role of blockchains in health care.MethodsTo identify related existing work, terms based on Medical Subject Headings were used. We included studies focusing mainly on health care and clinical research and developed a functional framework for implementation and testing with data. The literature sources for this systematic review were PubMed, Medline, and the Cochrane library, in addition to a preliminary search of IEEE Xplore.ResultsThe included studies demonstrated multiple framework designs and various implementations in health care including chronic disease diagnosis, management, monitoring, and evaluation. We found that blockchains exhibit many promising applications in clinical trial management such as smart-contract application, participant-controlled data access, trustless protocols, and data validity. Electronic health records (EHRs), patient-centered interoperability, remote patient monitoring, and clinical trial data management were found to be major areas for blockchain usage, which can become a key catalyst for health care innovations.ConclusionsThe potential benefits of blockchains are limitless; however, concrete data on long-term clinical outcomes based on blockchains powered and supplemented by AI and IoT are yet to be obtained. Nonetheless, implementing blockchains as a novel way to integrate EHRs nationwide and manage common clinical problems in an algorithmic fashion has the potential for improving patient outcomes, health care experiences, as well as the overall health and well-being of individuals.

Leveraging Blockchain Technology for Informed Consent Process and Patient Engagement in a Clinical Trial Pilot.

Despite the implementation of quality assurance procedures, current clinical trial management processes are time-consuming, costly, and often susceptible to error. This can result in limited trust, transparency, and process inefficiencies, without true patient empowerment. The objective of this study was to determine whether blockchain technology could enforce trust, transparency, and patient empowerment in the clinical trial data management process, while reducing trial cost. In this proof of concept pilot, we deployed a Hyperledger Fabric-based blockchain system in an active clinical trial setting to assess the impact of blockchain technology on mean monitoring visit time and cost, non-compliances, and user experience. Using a parallel study design, we compared differences between blockchain technology and standard methodology. A total of 12 trial participants, seven study coordinators and three clinical research associates across five sites participated in the pilot. Blockchain technology significantly reduces total mean monitoring visit time and cost versus standard trial management (475 to 7 min; P = 0.001; €722 to €10; P = 0.001 per participant/visit, respectively), while enhancing patient trust, transparency, and empowerment in 91, 82 and 63% of the patients, respectively. No difference in non-compliances as a marker of trial quality was detected. Blockchain technology holds promise to improve patient-centricity and to reduce trial cost compared to conventional clinical trial management. The ability of this technology to improve trial quality warrants further investigation.

Blockchain for COVID-19: Review, Opportunities, and a Trusted Tracking System.

The sudden development of the COVID-19 pandemic has exposed the limitations in modern healthcare systems to handle public health emergencies. It is evident that adopting innovative technologies such as blockchain can help in effective planning operations and resource deployments. Blockchain technology can play an important role in the healthcare sector, such as improved clinical trial data management by reducing delays in regulatory approvals, and streamline the communication between diverse stakeholders of the supply chain, etc. Moreover, the spread of misinformation has intensely increased during the outbreak, and existing platforms lack the ability to validate the authenticity of data, leading to public panic and irrational behavior. Thus, developing a blockchain-based tracking system is important to ensure that the information received by the public and government agencies is reliable and trustworthy. In this paper, we review various blockchain applications and opportunities in combating the COVID-19 pandemic and develop a tracking system for the COVID-19 data collected from various external sources. We propose, implement, and evaluate a blockchain-based system using Ethereum smart contracts and oracles to track reported data related to the number of new cases, deaths, and recovered cases obtained from trusted sources. We present detailed algorithms that capture the interactions between stakeholders in the network. We present security analysis and the cost incurred by the stakeholders, and we highlight the challenges and future directions of our work. Our work demonstrates that the proposed solution is economically feasible and ensures data integrity, security, transparency, data traceability among stakeholders.

Data Validation and Verification Using Blockchain in a Clinical Trial for Breast Cancer: Regulatory Sandbox.

BackgroundThe integrity of data in a clinical trial is essential, but the current data management process is too complex and highly labor-intensive. As a result, clinical trials are prone to consuming a lot of budget and time, and there is a risk for human-induced error and data falsification. Blockchain technology has the potential to address some of these challenges.ObjectiveThe aim of the study was to validate a system that enables the security of medical data in a clinical trial using blockchain technology.MethodsWe have developed a blockchain-based data management system for clinical trials and tested the system through a clinical trial for breast cancer. The project was conducted to demonstrate clinical data management using blockchain technology under the regulatory sandbox enabled by the Japanese Cabinet Office.ResultsWe verified and validated the data in the clinical trial using the validation protocol and tested its resilience to data tampering. The robustness of the system was also proven by survival with zero downtime for clinical data registration during a Amazon Web Services disruption event in the Tokyo region on August 23, 2019.ConclusionsWe show that our system can improve clinical trial data management, enhance trust in the clinical research process, and ease regulator burden. The system will contribute to the sustainability of health care services through the optimization of cost for clinical trials.

Discussion on registry study of acupuncture-moxibustion for malignant pleural effusion

Malignant pleural effusion (MPE) is one of the common complications of tumor. Acupuncture-moxibustion therapy has several advantages for treatment of MPE. Acupuncture is regarded as a complex individualized intervention, and its characteristics of TCM is difficult to be reflected by strict randomized controlled trials. The registry study provides more possibilities for the data collection of individualized diagnosis and treatment under the guidance of the overall concept and syndrome differentiation, and is more suitable for data management and collection of large samples and multi-center trials in the real-world study. It has become an opportunity to carry out real-world study of acupuncture for MPE. There are many challenges in the registry study of acupuncture for MPE. However, it is of great significance to collect real-world data of acupuncture for MPE to improve the clinical effect of MPE and provide a new clinical research method for acupuncture in tumors and related complications.

Data management of international collaborative clinical trials for childhood solid tumors

Data management of international collaborative clinical trials for childhood solid tumors

Clinical Trial Data Management in Environmental Health Tailored for an African Setting.

Clinical trial data management tools are widely available—some free to access and others relatively expensive, particularly for low- and middle-income countries. Such tools also do not always permit adaptation for local conditions nor include options to capture environmental and meteorological data. In the context of climate change and pressing environmental health threats, more studies that aim to assess the impacts of environmental change on public health are being carried out. Here, using freely available software, we tailor-made a clinical trial data management tool that managed all aspects of an intervention-based clinical trial to assess the impact of personal solar ultraviolet radiation exposure on vaccine effectiveness. Data captured and associated procedures included patient data, scheduling, reporting, analysis and data management. Moreover, patient enrolment, recruitment, follow-up and decision-making in response to patient data were managed. Given the multidisciplinary study approach, the tool also managed all environmental and meteorological data for the rural African study site. Application of the tool ensured efficient communication between rural sites, a relatively high overall participant response rate (87%) and minimal loss to follow-up. This study suggests that it is possible to tailor-make a clinical trial data management tool for environmental and public health studies.

Clinical Trial Data Management Software: A Review of the Technical Features.

Data management is an important, complex and multidimensional process in clinical trials. The execution of this process is very difficult and expensive without the use of information technology. A clinical data management system is software that is vastly used for managing the data generated in clinical trials. The objective of this study was to review the technical features of clinical trial data management systems. Related articles were identified by searching databases, such as Web of Science, Scopus, Science Direct, ProQuest, Ovid and PubMed. All of the research papers related to clinical data management systems which were published between 2007 and 2017 (n=19) were included in the study. Most of the clinical data management systems were web-based systems developed based on the needs of a specific clinical trial in the shortest possible time. The SQL Server and MySQL databases were used in the development of the systems. These systems did not fully support the process of clinical data management. In addition, most of the systems lacked flexibility and extensibility for system development. It seems that most of the systems used in the research centers were weak in terms of supporting the process of data management and managing clinical trial's workflow. Therefore, more attention should be paid to design a more complete, usable, and high quality data management system for clinical trials. More studies are suggested to identify the features of the successful systems used in clinical trials.

A survey for the core work activity of clinical research coordinator in China

Objective Sorting out work activity of China Clinical Research Coordinator(CRC), defining the core work activity of China CRC, and making comparison with foreign CRC’s. Providing basic data to establish training system and occupational skill assessment system for China CRC. Methods This is a cross-section design survey. The method for sampling will be the convenient sampling combine the snowball sampling. A self-administered questionnaire was introduced. Data managed by Excel and analyzed by SPSS. The frequency, percentage, mean and SD will used for the description of the CRC’s demographics information and vocation characteristics. The core work activities of CRCs were defined with the method which used to define core work activities of CRCs in foreign literatures. Results Totally 227 questionnaires were returned, 223 effective questionnaires were used for analysis, the effective rate is 98%. Around 89% of CRCs are female. The mean age is 27.5±3.8 years. Around 72% of CRCs owned bachelor degree, 24% are junior college degree, 4% are master degree. Around 43% CRCs are nurse background, 34% are pharmacy background, 16% are medical background. Around 55% of CRCs owned 3 000-5 000RMB income. More than 50% of the CRCs had worked in this position for less than 1.5 years. Totally 10 items of work activities are defined as the core work activity of China CRCs, mainly focus on clinical trial data collection and management. Comparing to foreign CRC, the involvement of China CRC on CRF completion, query solution, study document management, assisting Clinical Research Associate (CRA) on monitoring, AE and SAE reporting are lower than foreign CRC, and the involvement on assisting investigator on patient screening, lab-test report collection and management, sample delivery and management, assisting investigator on patient visit arrangement are higher than foreign CRC. Conclusions The CRC in China has now involved in all work activities of the whole process of clinical trials, their core work activities mainly in the phase of clinical trial conduct, they play an important role in clinical trial operation. Due to the immature training system and the unclear job description, there are differences Between Foreign CRC and China CRC on working activity. Key words: Clinical research coordinator; Clinical trial coordinator; Site coordinator; Core work activity; Cross-sectional survey

Prototype of running clinical trials in an untrustworthy environment using blockchain

Monitoring and ensuring the integrity of data within the clinical trial process is currently not always feasible with the current research system. We propose a blockchain-based system to make data collected in the clinical trial process immutable, traceable, and potentially more trustworthy. We use raw data from a real completed clinical trial, simulate the trial onto a proof of concept web portal service, and test its resilience to data tampering. We also assess its prospects to provide a traceable and useful audit trail of trial data for regulators, and a flexible service for all members within the clinical trials network. We also improve the way adverse events are currently reported. In conclusion, we advocate that this service could offer an improvement in clinical trial data management, and could bolster trust in the clinical research process and the ease at which regulators can oversee trials.

Using Blockchain Technology to Manage Clinical Trials Data: A Proof-of-Concept Study.

BackgroundBlockchain technology is emerging as an innovative tool in data and software security.ObjectiveThis study aims to explore the role of blockchain in supporting clinical trials data management and develop a proof-of-concept implementation of a patient-facing and researcher-facing system.MethodsBlockchain-based Smart Contracts were built using the Ethereum platform.ResultsWe described BlockTrial, a system that uses a Web-based interface to allow users to run trials-related Smart Contracts on an Ethereum network. Functions allow patients to grant researchers access to their data and allow researchers to submit queries for data that are stored off chain. As a type of distributed ledger, the system generates a durable and transparent log of these and other transactions. BlockTrial could be used to increase the trustworthiness of data collected during clinical research with benefits to researchers, regulators, and drug companies alike. In addition, the system could empower patients to become more active and fully informed partners in research.ConclusionsBlockchain technology presents an opportunity to address some of the common threats to the integrity of data collected in clinical trials and ensure that the analysis of these data comply with prespecified plans. Further technical work is needed to add additional functions. Policies must be developed to determine the optimal models for participation in the system by its various stakeholders.