Experiences With Auto-Grading in a Systems Course

Accessibility has been recognized as a key aspect of using technology for education. Yet educational software, including web-based systems, and even the main web-pages of top institutions often do not meet accessibility standards. When addressing accessibility issues, some creators of educational technology attempt to merely retrofit accessibility. Such an approach of “accessibility as an afterthought" often leads to low quality or even ineffective user interfaces (UI) and degraded user experience (UX) for all users. In order to avoid the aforementioned common pitfalls, we pursued an agile development process with an accessibility expert integrated into the process. The agile method has been used to develop a web-based automatic testing and grading software system called CODE. This paper presents our agile method and experiences with designing and developing an accessible software system. We discuss the software design and the some of the accessibility improvements recommended to meet accessibility standards. We present statistical analysis of 3,300 submissions from 47 students and their experiences. Our experience shows that even if software tools are able to identify accessibility issues, they are not effective in suggesting appropriate solutions to address them. A key “lesson learned" is that integrating an accessibility expert into software development teams is an effective approach to ensure effective and accessible UI/UX for educational technologies.

Automatic testing software in the multi-mode terminal tester is to complete the automatic tests of multi-mode terminals so as to achieve the performance indexes and consistency condition. This paper firstly explains the multi-mode terminal tester automatic test system, and then introduces the automated testing software architecture and its design goals. Finally, it completes the overall design of the automatic testing software based on multimode terminal tester and describes the testing results.

Nowadays, the bioelectromagnetic field has been accepted, and more and more scientists are concerned about the bioelectromagnetic effect from external electromagnetic field. But few scientists study the bioelectromagnetic spectrum, which features may be used to identify the living beings. In this paper, we give a method to measure the bioelectromagnetic field. At first, we study the theory of measuring the bioelectromagnetic signal. Furthermore, we designed an automatic test system (ATS) based on the theory. The automatic test software, an important part of the ATS, which can control the instruments such as spectrum analyzer, poisoning controllers, etc, is introduced by the diagram. Finally, the measurement data are calculated and recorded to a text file, which can be used to analyze the bioelectro-magnetic spectrum. We also display the 3D graph of the bioelec-tromagnetic field vs. each frequency, to study and view the direction of bioelectromagnetic field. The automatic system has been applied in our research about mice's bioelectromegnetic field test, and works well.

Nowadays, the bioelectromagnetic field has been accepted, and more and more scientists are concerned about the bioelectromagnetic effect from external electromagnetic field. But few scientists study the bioelectromagnetic spectrum, which features may be used to identify the living beings. In this paper, we give a method to measure the bioelectromagnetic field. At first, we study the theory of measuring the bioelectromagnetic signal. Furthermore, we designed an automatic test system (ATS) based on the theory. The automatic test software, an important part of the ATS, which can control the instruments such as spectrum analyzer, poisoning controllers, etc, is introduced by the diagram. Finally, the measurement data are calculated and recorded to a text file, which can be used to analyze the bioelectro-magnetic spectrum. We also display the 3D graph of the bioelec-tromagnetic field vs. each frequency, to study and view the direction of bioelectromagnetic field. The automatic system has been applied in our research about mice's bioelectromegnetic field test, and works well.

INTUITEL is a research project aiming to offer a personalized learning environment. The INTUITEL approach includes an Intelligent Tutoring System that gives students recommendations and feedback about what the best learning path is for them according to their profile, learning progress, context and environmental influences. INTUITEL combines efficient pedagogical-based recommendations with freedom of choice and it introduces this tutoring support in different Learning Management Systems. During the INTUITEL project various software and pedagogical testing procedures were defined to provide the development teams with feedback, both summative and formative. The current paper describes the initial user test, which was conducted at the University of Valladolid for the course “Network Design”. The experiment was focused on real learners’ reactions to INTUITEL recommendations received by an INTUITEL-enabled LMS. Nineteen students participated in a two phase testing procedure in order to analyze the learners’ behavior with INTUITEL, as well as obtaining information about how learners perceive the influence and usefulness of the tutoring system in online learning courses. Results show that students with INTUITEL follow learning paths that are more suitable for them. Besides, the general satisfaction level of participants is high. Most learners appreciate INTUITEL, would follow its recommendations and consider the messages shown by INTUITEL as useful and caring.

To achieve miniaturization and function integration of the telemetry radar antenna, multi-channel analog and digital circuits are integrated into a single receiver unit called Digital Receiver Module (DRM). To fulfill the real-time transmission of baseband data, optical fiber is applied as the transmission medium. The above two points make it impossible to measure key indicators of DRM rely only on traditional microwave instruments. Automatic Test System (ATS) based on LXI architecture with functions of command sending, baseband data receiving and instrument control is designed as a solution. Multi-channel signal routing link is established by dividers and microwave switches, so that all channels of DRM can be tested in parallel. Optical fiber transceiver is exploited not only to send synchronous commands, but also to receive and record baseband data. Signal processing methods dedicated to the specific characters of DRM are designed to analyze key indicators from baseband data, such as receiving gain, noise figure and instantaneous dynamic range. Automatic test software and signal processing methods are implemented using VS2005. Through practical application, the obtained results are completely consistent with the design baseline. Besides, on the premise that communication interference and temperature change are included in the test results, the maximum deviation of receiving gain is ±0.21dB and the maximum deviation of noise figure is ±0.23dB within five discrete measurements, which could demonstrate the accuracy and effectiveness of the ATS.

The assessment of programming assignments is a costly task. Several tools have been proposed and developed in order to automate the repetitive tasks performed by instructors in the assessment of programming assignments and provide a faster and more adequate feedback to students. However, adding a new tool increases the overload of new information and environments that students have to deal with. Similarly, the assessment tool is one more resource that the instructor has to configure, maintain and teach students to use, spending time and effort that could be used in other pedagogical activities. For this reason, several works have been conducted to integrate assessment tools for programming assignments in learning management systems (LMSs). The integration of assessment tools into LMSs promotes their adoption in computing courses, since they will be in agreement with the LMS already familiar to students and instructors, without the need to adopt, learn and manage the submission and correction of assignments in another environment. However, these works consist only of abstract conceptual models, do not propose integration with several LMSs, or do not address certain specific difficulties faced when integrating assessment tools. In this perspective, this work aims at the proposition of the IMPACTLE architecture, a solution that allows the integration of different assessment tools for programming assignments in LMSs. The idea is that students and teachers can access the features of the tools through the LMSs they are already used to, without the need to learn how to use and adopt a new tool. Architecture prototypes were instantiated and experiments were carried out involving the use of different assessment tools through LMSs. In general, we noticed that the use of IMPACTLE enables instructors and students perform tasks related to programming activities in a more efficient, efficacious and effective way through the LMSs.

With the development of testing bus technology, PXI/VXI bus is implemented in navigation signal simulator. The automatic test software mostly communicates with the simulator by using direct I/O. Due to a lack of uniform standards, it's not convenient for function upgrading and extension. A navigation signal simulator solution based on LXI(LAN eXtensions for Instrumentation) bus is proposed. Through the analysis of technique features of LXI bus standard, the hardware architecture of navigation signal simulator is designed. Meanwhile, the implementation of VXI-11-based remote control is mainly discussed. The simulator is controlled by driver interface in line with IVI (Interchangeable Virtual Instrumentation) specification to unify software programming interfaces of different simulators, so ATS (Automatic Test System) only needs to change the simulator driver after simulator upgrading or replacement which can greatly improve the reusability of automatic test software.

Current industrial robotics technology is often not well integrated with the enterprise’s on-site environment and actual working conditions and small and medium-sized enterprises are unable to achieve product automation due to production cost constraints. In order to meet the medium and small scale production of the slide valve body of OCV (Oil Control Valve) of a certain enterprise and its special process requirements, the automatic test system and sorting system based on the production environment of the enterprise are studied and designed. Firstly, according to the production conditions and process requirements of the enterprise, the overall design scheme of the automatic production line is put forward based on the existing automatic assembly system. Secondly, the test description is further improved by analysing and interpreting the test requirements of the products in detail and the automatic test system and test process are designed. Finally, according to the sorting process requirements, a Cartesian coordinate robot sorting system with two-terminal manipulators parallel operation is designed and its sorting motion scheme is optimized. The automatic test system and sorting system are seamlessly connected with the automatic assembly system, which can efficiently complete the automatic test and sorting of products and meet the production cycle time.

The article adopts the bus technology to build the automatic test equipment, including the intelligent interface, the data acquisition system composed of the bus module instrument and the automatic test software with VEE as the development environment. The intelligent interface successfully solves the mutual isolation of the detection signals provided by the test object; the use of the bus module instrument greatly improves the hardware reliability of the entire automatic test equipment; the use of VEE to develop automatic test software reduces the complexity of software development, Shorten the development cycle and ensure the reliability of the automatic test software.

Automatic grading models are valued for the time and effort saved during the instruction of large student bodies. Especially with the increasing digitization of education and interest in large-scale standardized testing, the popularity of automatic grading has risen to the point where commercial solutions are widely available and used. However, for short answer formats, automatic grading is challenging due to natural language ambiguity and versatility. While automatic short answer grading models are beginning to compare to human performance on some datasets, their robustness, especially to adversarially manipulated data, is questionable. Exploitable vulnerabilities in grading models can have far-reaching consequences ranging from cheating students receiving undeserved credit to undermining automatic grading altogether—even when most predictions are valid. In this paper, we devise a black-box adversarial attack tailored to the educational short answer grading scenario to investigate the grading models’ robustness. In our attack, we insert adjectives and adverbs into natural places of incorrect student answers, fooling the model into predicting them as correct. We observed a loss of prediction accuracy between 10 and 22 percentage points using the state-of-the-art models BERT and T5. While our attack made answers appear less natural to humans in our experiments, it did not significantly increase the graders’ suspicions of cheating. Based on our experiments, we provide recommendations for utilizing automatic grading systems more safely in practice.

Automatic grading of object‐oriented programming (OOP) assignments is an important problem from practical, theoretical, and educational viewpoints. Apart from computing a specific grade, an effective grading method needs to provide systematic feedback comments to both the design and code elements. Existing works have proposed grading methods that make various assumptions about the design. However, none of these methods have considered using a design language for the program model. A challenge here is to use a language that eases learning and application in program design. In this paper, we propose a novel grading method, named OOPGRADER, which automatically grades OOP assignments constructed based on an essentially detailed program model. This model is defined in an embedded design language, which directly uses the annotation feature of OOP language to express the essential design rules. We explain how programming assignments can be designed with this language and propose a program checker and a grader for both student and teacher to use in working on the assignments and in grading them. We implement these components in a tool and develop an Eclipse plugin for this tool. We evaluate our method using a combination of qualitative and quantitative techniques. The main result is that our method helps students effectively learn to program through detailed feedback comments based on a program model. The tool is adaptable, has a good performance, and helps enhance the productivity of both students and teachers through IDE integration.

Objective To investigate the feasibility of the automatic cystocele severity grading software for quantitative evaluation of prolapse of bladder posterior wall by transperineal ultrasound. Methods One hundred and seventy transperineal ultrasound video clips were recorded when the female patients performing the Valsalva maneuver and those clips were divided into training group (85 cases) and test group (85 cases) randomly, then the ralated structures of the images from the training group offline were marked. Through machine learning algorithm, the computer had learned and was able to analyzed the marking information, then the automatic cystocele severity grading software was obtained. And later the software was ran to mark the structures and get the cystocele severity grading in the images from the test group. Meanwhile, the same structures of the same images manually were marked and after an interval of more than two weeks the process were repeated by 3 doctors. Finally the grading results obtained from the software and the measurers of the 3 doctors were compared. Results The intelligent identification and automatic measurement software obtained from the machine learning algorithm was able to identify the related structures. The grading results of each measurer were of good consistency (κ: 0.72-0.78; ICC: 0.980-0.990). The grading results between different measurers were of good consistency (κ: 0.65-0.75; ICC: 0.985-0.992). The grading results between automatic software and three different measurers were of good consistency (κ: 0.63-0.67; ICC: 0.967-0.969; r=0.936, 0.943, 0.936, all P<0.01). Conclusions The automatic cystocele severity grading software is able to identify the related structures in the images and reliable to apply the software in pelvic floor ultrasound. Key words: Ultrasonography, transperineal; Posterior bladder wall prolapse; Intelligent identification; Automatic measurement; Machine learning algorithm

SR is a language for concurrent programming. This article describes the SR language, presents some examples of SR programs in the context of an undergraduate operating systems course, and provides some programming assignments that can be used in an open laboratory. The SR language can be used by instructors of operating systems courses to give students experience in writing concurrent programs that use multiple processes, semaphores, message passing, and the rendezvous. These examples and programming assignments have been used successfully in undergraduate operating systems courses at Drexel University in Philadelphia and Trinity University in San Antonio.

Aiming at the problems of a large volume of current electrical five-parameter standard source devices, inconvenience in carrying them to the test site, and difficulty in conducting automatic metrology tests, a PXI board-type electrical five-parameter calibration source device is designed. Together with the driver control and automatic test software, workers can complete the on-site automated verification and calibration of electrical parameter test equipment in automatic test systems and related equipment tests by using the on-site PXI chassis, without the need to carry chassis and desktop devices or disassemble the devices to be tested, which greatly improves work efficiency and realizes portable, channel-expandable, and integrated automatic calibration.