Interface and Features for an Automatic ‘C' Program Evaluation System

Objectives. To evaluate the feasibility of implementing a statewide children with special health care needs (CSHCN) program evaluation, case management, and surveillance system using a standardized instrument and protocol that operationalized the United States Health and Human Services CSHCN National Performance Measures. Methods. Public health nurses in local public health agencies in Washington State jointly developed and implemented the standardized system. The instrument was the Omaha System. Descriptive statistics were used for the analysis of standardized data. Results. From the sample of CSHCN visit reports (n = 127), 314 problems and 853 interventions were documented. The most common problem identified was growth and development followed by health care supervision, communication with community resources, caretaking/parenting, income, neglect, and abuse. The most common intervention category was surveillance (60%), followed by case management (24%) and teaching, guidance, and counseling (16%). On average, there were 2.7 interventions per problem and 6.7 interventions per visit. Conclusions. This study demonstrates the feasibility of an approach for statewide CSHCN program evaluation, case management, and surveillance system. Knowledge, behavior, and status ratings suggest that there are critical unmet needs in the Washington State CSHCN population for six major problems.

TEACHER quality is a central pillar of the No Child Left Behind Act for good reason. Without high-quality teachers, we cannot realize the goal of high achievement for all students. In turn, if preservice teachers do not receive the highest-quality teacher preparation, then we cannot meet the demand for high-quality teachers. Few would argue with this logic, but it does invite the question, How can teacher preparation institutions determine whether their programs effectively prepare graduates to help all students achieve academically? To help answer this question, in 2000 the U.S. Department of Education (ED) established the National Awards Program for Effective Teacher Preparation. The awards are to recognize teacher preparation programs that have compelling evidence of their effectiveness in readying teachers to help all students meet high academic standards. ED also envisioned the awards program as encouraging further discussion of the elements of effective teacher preparation. To that end, the awards specifically require programs to describe how they systematically gather and use a variety of data for monitoring their activities and improving their efforts. Winning programs detailed their size, focus, goals, objectives, components (such as courses and field experiences), and criteria for judging effectiveness. Of the 19 programs that applied for the award in 2000, four were selected: * Alverno College, Elementary Education Program, Milwaukee, Wis. * East Carolina University, Middle Grades Mathematics Program, Greenville, N.C. * Fordham University Graduate School of Education, Initial Teacher Education, Elementary Program, New York, N.Y. * Samford University, Elementary Education Program, Birmingham, Ala. AN OVERVIEW OF THE PROGRAMS Whether part of a large public institution or a small private one, each of these programs has specific structures and processes in place to ensure that it continually improves. The programs share a commitment to monitoring and evaluating their effectiveness, yet each is unique in terms of size, focus, and how its evaluation system evolved. Alverno College. This small private college for women prepares teachers for urban settings. The curriculum focuses on eight abilities (such as communicating effectively through a variety of media or working collaboratively with others), and all of Alverno's programs use performance assessments to determine students' proficiency in these skills. Faculty members across the seven major divisions of the college collaborate on the curriculum, performance assessments, and evaluations. The evaluation system began and continues as part of Alverno's systematic, campuswide improvement efforts. East Carolina University. The award-winning teacher preparation program at this large public institution focuses on preparing middle school mathematics teachers. This program relies on strong collaboration with P-12 partners, and many program graduates teach in rural schools. The program's evaluation system is more highly developed than that of other teacher education programs at the university. Fordham University. Fordham, a relatively large private institution, offers three graduate teacher education programs. The award-winning program prepares teachers for urban settings. Faculty members had a twofold reason for developing the program's evaluation system: not only did they recognize a need for change, but they also wanted to meet accreditation requirements. Samford University. This small private university uses a problem-based curriculum, and its evaluation system, which is in place throughout the university, is built on the principles of total quality management. The school of education developed the Quality Education Comprehensive Assessment Program as its model for planning and evaluation. Faculty members and advisory groups that include candidates, graduates, practitioners, and employers provide input into the planning process. …

The Ohio State University College of Medicine

Summary form only given. Industry has been wrestling with the complexity of managing business systems for years. The challenge stems from the variety of application and IT resource providers that enterprises use to build their business systems. A range of management systems co-exist to manage the breadth of resources. The management industry and customers have an opportunity to take advantage of the industry trend towards using Web services for business integration and moving to service oriented architectures for business. It is now possible to garner these same advantages seen in business for management. Building manageable resources and management systems on a Web services foundation is going to cause a profound shift in how enterprises and vendors manage their IT resources in the future. Embracing this shift is going to create more flexible IT infrastructures, better integration of business and IT objectives, and greater end to end management of both IT infrastructures and business processes. This presentation provides a bottoms-up tutorial of Web services distributed management (WSDM), the new OASIS Standard that provides the first step in solving this classic management integration problem. The session will begin with an overview of the SOA based management architecture and WSDM's place in that architecture relative to other industry standards and initiatives. The technical tutorial will begin with an introduction on WSDL and WS-addressing, specifications on which WSDM depends. The presenters will build on this with an overview of the Web services for resource framework (WSRF) and Web services notification (WSN) OASIS specifications and discuss how they are used by WSDM. Finally, the session will explore WSDM components, management using WS (MUWS) and WSDM management of Web services (MOWS). MUWS defines how to represent and access the manageability interfaces of any IT resource as Web services. MOWS defines how to manage Web services as resources and how to describe and access that manageability using MUWS. Concrete customer issues solved by WSDM will also be highlighted as well as how CIM modeled resources can be accessed using WSDM. Apache MUSE and IBM Developerworks Toolkits will be used to illustrate the programming techniques

This article describes a model and implementation of a system for automatically testing, evaluating, grading and providing critical feedback for submitted programming assignments. Complete automation of the evaluation process, with proper attention towards monitoring student’s progress and performing a finegrained analysis is addressed. The tool provides online support to both the instructors and students and is designed for service-oriented integration with a course management system using Web services.

A systems model for safety program evaluation

Performance ImprovementVolume 39, Issue 9 p. 16-19 Building the Endowment Assessing and developing organizational capacity Sheila E. Murphy PhD, Sheila E. Murphy PhD As President of Sheila Murphy Associates since 1993, Dr. Sheila Murphy specializes in organizational analysis, program evaluation, and performance system design and implementation, as well as planning and executive coaching with boards of directors and senior executives. The primary focus of her work is on capacity building designed to enhance the competitive strength of the organization. Her firm also provides services in the areas of instructional design, facilitation, and training. Dr. Murphy previously served as Vice President of Worldwide Reservations, maintaining responsibility for generating approximately one-third of system revenues, and as Director of Ramada Management Institute, the training division of the global franchise organization. In addition, she was Executive Director of Business and Management Programs at the corporate office of the University of Phoenix, responsible for faculty and curriculum in business/management programs for the University's adult-centered campuses. Dr. Murphy is the author of Practical Motivation Handbook and numerous articles in the areas of board, executive, and organizational development, performance measurement, and core skills for government and private sector executives. She may be reached at shemurph@aol.com.Search for more papers by this author Sheila E. Murphy PhD, Sheila E. Murphy PhD As President of Sheila Murphy Associates since 1993, Dr. Sheila Murphy specializes in organizational analysis, program evaluation, and performance system design and implementation, as well as planning and executive coaching with boards of directors and senior executives. The primary focus of her work is on capacity building designed to enhance the competitive strength of the organization. Her firm also provides services in the areas of instructional design, facilitation, and training. Dr. Murphy previously served as Vice President of Worldwide Reservations, maintaining responsibility for generating approximately one-third of system revenues, and as Director of Ramada Management Institute, the training division of the global franchise organization. In addition, she was Executive Director of Business and Management Programs at the corporate office of the University of Phoenix, responsible for faculty and curriculum in business/management programs for the University's adult-centered campuses. Dr. Murphy is the author of Practical Motivation Handbook and numerous articles in the areas of board, executive, and organizational development, performance measurement, and core skills for government and private sector executives. She may be reached at shemurph@aol.com.Search for more papers by this author First published: 08 February 2007 https://doi.org/10.1002/pfi.4140390908Citations: 1 AboutPDF ToolsRequest permissionExport citationAdd to favoritesTrack citation ShareShare Give accessShare full text accessShare full-text accessPlease review our Terms and Conditions of Use and check box below to share full-text version of article.I have read and accept the Wiley Online Library Terms and Conditions of UseShareable LinkUse the link below to share a full-text version of this article with your friends and colleagues. Learn more.Copy URL Share a linkShare onFacebookTwitterLinkedInRedditWechat Citing Literature Volume39, Issue9October 2000Pages 16-19 RelatedInformation

The Centers for Disease Control and Prevention (CDC) Division of HIV/AIDS Prevention (DHAP) has responded to the need for accurate and timely data to monitor and evaluate federally funded HIV prevention programs by designing and implementing the Program Evaluation and Monitoring System (PEMS). PEMS is a national data reporting system that includes a standardized set of HIV prevention data variables, Web-based software for data entry and management, data collection and evaluation guidance and training, and software implementation support services. This article discusses the purposes of evaluation and the need for and development of PEMS and also describes how PEMS strengthens the monitoring and evaluation of HIV prevention services nationally and program planning, management, and monitoring locally. PEMS data may be used by prevention stakeholders at all levels to examine program fidelity, adaptation and tailoring, and key program health service utilization and behavioral outcomes. PEMS provides a strong foundation to bring a program closer to its intended goals and to serve the needs of the clients and the community, provides a means for agencies to fulfill their reporting mandates and funding obligations, and enables the CDC to monitor HIV prevention efforts in a consistent, efficient, and effective manner across the United States.

Treatment Delayed Is Treatment Denied!

Agencies providing services to individuals with developmental disabilities must demonstrate quality programing as evidenced through a high level of staff compliance to organizational, licensing, and accreditation standards. In addition, economic and practical considerations impose the need for cost‐effective procedures for evaluating and improving staff performance. This study utilized a combination of checklists, as an evaluative tool to measure staff performance, and private feedback to improve staff performance. A multiple baseline design was used to determine the effects of providing staff with (a) orientation training, (b) task‐analyzed checklists of job skills, (c) private feedback from the agency trainer, and (d) private feedback from their immediate supervisor. Results indicate that training alone does not ensure competency; checklists alone do not ensure competency; private feedback from trainers improves competency; and private feedback from direct supervisors improves competency even further.

This chapter assesses the link between democracy and program evaluation from a public management perspective. It commences with a succinct summary of the nature of government and governance and the origins and nature of democracy. It then explains the relevance of systematic evaluation of governmental programs for the consolidation of democracy from an evidence-informed perspective on public policymaking. It is concluded that evaluation is an integral higher order public management function that needs to be regularly undertaken in order to ensure the optimal efficiency and effectiveness of democratic policymaking. The democratic nature of the South African state is then summarized and the role of the Government-Wide Monitoring and Evaluation System in supporting the consolidation of democracy in South Africa assessed. The final conclusion is that systematic program evaluation is increasingly seen as a crucial, intrinsic element of good democratic management and governance in 21st century governments.

Objective Resource Facilitation is a lesser-known service line within the field of brain injury rehabilitation and has been described as similar to case management, care coordination, and neuronavigation. The purpose of this project was to evaluate current Resource Facilitation programs and provide a comprehensive summary of program characteristics to work toward a common definition of Resource Facilitation services and inform future program development and evaluation frameworks. Materials and Methods An online survey was sent to all known Resource Facilitation programs in the US (N = 70). Data Collection Completed through Survey Monkey including information related to program history, structure, population served, and outcomes measurement. StatisticalAnalysis Descriptive statistical analyses were applied. Results Twenty-four of the 70 programs completed the survey, resulting in a 34% completion rate. This snapshot of the current definitions and activities demonstrated high variability across programs regarding structure, funding, eligibility, and data collection. Only 33% of programs reported having a program evaluation system. Conclusions This project provided a comprehensive summary of Resource Facilitation program characteristics and associated outcome metrics. The findings may be used to begin the formulation of a common definition of Resource Facilitation services, a program evaluation framework, and aid in establishing a common data set across programs.

Educational program evaluation can improve the quality of the curriculum, instructional methods, and resources and provide useful data for making educational decisions and policies. Developing and implementing a program evaluation system is essential in competency-based medical education. The purpose of this study was to explore and establish an educational program evaluation system adapting an integrated program evaluation model to promote competency-based medical education. First, an Educational Evaluation Committee was organized, consisting of faculty, staff members, and students. The committee established an integrated program evaluation model, combining Stufflebeam’s Context, Input, Process, and Product (CIPP) model of a process-oriented approach and Kirkpatrick’s four-level model of an outcome-oriented approach. Kirkpatrick’s model was applied to the product evaluation of the CIPP model. The committee then developed evaluation criteria, indicators, and data collection methods according to the components of the CIPP model and the four levels (reaction, learning, behavior, and results) of Kirkpatrick’s model, and collected and analyzed data. Finally, the committee reported the results of evaluation to a Medical Education Quality Improvement Committee, and the results were used to improve the curriculum and student selection. To enhance the quality of education, identifying educational deficiencies and developing various elements of education in a balanced way through educational evaluation will be needed. Furthermore, it will be necessary to listen to opinions of various stakeholders, work with all members involved in education, and communicate with decision-makers in the process of educational evaluation.

One of the interesting aspects of the Web 2.0 'evolution' is the wide-availability of various Web applications as APIs or Web services. These APIs expose informational services on the Web and take many forms of remote invocation of functions using standard Web protocols and XML for data representations, e.g., REST, SOAP/WSDL, XML-RPC, and other approaches. The services (or APIs) are also usually accompanied by user facing Web applications for human-consumption. Canonical examples are Google Maps, Yahoo! Flykr and del.icio.us, EVDB's Eventful's application and API, Amazon.com's S3, ECS, Alexa, and many others. The Ruby programming language and its Rails framework are ideal for programming Web applications and services in the Web 2.0. Ruby's modern and dynamic features make it an excellent language for rapid prototyping and integration of various Web services. Rails' superb support for rapid Web application development, database access, and AJAX, make it well suited for creating front-ends and back-ends to the next generation of Web applications and services. In this tutorial we will take a hands-on deep-dive into the Ruby and Rails platform and learn how they can be used to: (1) create Web applications backed by a relational database, (2) consume Web services, (3) create and deploy APIs or Web services, and (4) mashup of existing Web services and applications. No a priori knowledge of Ruby or Rails is required - although some programming in a modern OO language and Web application development are definite plus.

Educators in graduate medical education (GME) regularly make decisions about adding, deleting, or revising some aspect of a curriculum, rotation, or other program activity. At the core of each decision is the activity's value to the learners, faculty, program, and sponsoring institution, and/or to meeting accreditation/licensure requirements. When these decisions are informed by systematic program evaluation, the results may be generalizable and have value to others in GME beyond their local worth. Yet how does an educator know if their work might be relevant to others or fulfill scholarship criteria? These are particularly vexing challenges for evaluation, as every program is local and has unique settings, trainees, faculty, and resources. Thus, disseminating evaluation as scholarship is a balancing act: staying true to the local nature of the program while addressing concerns of skeptics who may say, “That's not relevant…it wouldn't work here.”Applying standards of program evaluation can tip the balance toward becoming a scholarly activity. Other Rip Outs1,2 in this series outlined the American Evaluation Association (AEA) standards (accuracy, feasibility, integrity [ie, propriety], and utility)3 and suggested their application to program planning and implementation. Viewing these same standards in a checklist format can guide evaluation as a scholarly activity for authors and reviewers. Reminder: standards apply to evaluation, not to the project (eg, asking if the evaluation is feasible, not if the project is feasible). Standards and sample items include:As evaluators, readers, and editors, we like checklists ordered to match the standard introduction/aim, methods, results, discussion/conclusions (IMRD) framework. However, evaluation standards underpin the entire program evaluation endeavor; thus, one standard can—and often should—appear in multiple sections. See online supplementary data for the complete Evaluation as Scholarly Activity Checklist by AEA Standards.