Reinforcement Learning in Materials Science: Recent Advances, Methodologies and Applications

Bu araştırma, fen bilimleri öğretmenlerinin EğitimBilişim Ağıında (EBA) yer alan öğretim materyalleri ve yararlandıklarıdiğer eğitim platformlarında yer alan öğretim materyallerinin karşılaştırmasınıamaçlamaktadır. EBA; kullanışlılık, yeterlilik, müfredat uygunluğu, tasarım, içerikdoluluğu, özgünlük gibi konularda incelenerek diğer eğitim platformları ilekarşılaştırmalar yapılmıştır. Araştırmada görüşme tekniği ve amaçlı örneklemeyöntemi kullanılmıştır. Katılımcılar çeşitli şehirlerde görev yapan 20 fen bilimleriöğretmenidir. Veri toplama aracı olarak yarı yapılandırılmış görüşme formu;elde edilen verilerin analizinde içerik analizi yöntemi kullanılmıştır. EBA platformundayer alan materyaller diğer dijital öğretim materyalleri ilekarşılaştırıldığında yeterliliğin henüz beklenen seviyede olmadığı görülmüştür.EBA platformunda yer alan öğretim materyalleri diğer dijital öğretimmateryallerine bakarak daha az kullanılmaktadır. Kullanışlılık açısındankarşılaştırmalar yapılmış ve görsel bakımdan düzenlemeler olması gerektiğibelirtilmiştir. Öğretmenler sık kullandıkları materyalleri sıralarken; dersplanı yapabilme, planı kaydetme, aradığını kolay bulabilme gibi ölçütler önplana çıkarmaktadırlar. İçerik güncelliği ve müfredat uygunluğu öğretmenlerce,diğer dijital öğretim materyallerine göre yeterli bulunmamıştır.Haberler ve süreli dergiler bölümleri, diğer dijital öğretim materyallerindeçok sık rastlanmayan, Eğitim Bilişim Ağı’na özgü beğenilen ve en çok dilegetirilen kısımlar olmuştur. Eğitim Bilişim Ağı platformunun tanıtımı yetersizbulunmuştur. Eğitim Bilişim Ağı üzerine yapılan geliştirme çalışmalarıöğretmenler tarafından beğenilmektedir.

본 연구의 목적은 2009 개정 교육과정에 맞추어 개편된 고등학교 '과학' 교과서에 새롭게 도입된 '신소재'에 대한 예비 고등학생들과 고등학교 과학 교사들의 인식을 조사하여 신소재 수업을 효과적으로 진행하기 위해 필요한 기초자료를 제시하는 것이다. 서울, 인천, 경기 지역의 예비 고등학생 1,499명과 과학 교사 123명을 대상으로 설문을 진행하였으며 그 결과는 다음과 같다. 첫째, 학생들과 교사들의 신소재에 대한 태도는 긍정적이었지만 신소재에 대한 관심은 매우 낮았다. 학생들은 개정 과학 교과서의 신소재 단원에 대해 높은 관심을 보이지 않았고, 교사들도 신소재 단원 수업을 어렵게 생각하는 것으로 나타났다. 둘째, 학생들과 교사들의 신소재에 대한 인지도와 신소재 관련 지식에 대한 이해도는 낮은 편이었다. 학생과 교사 모두 다양한 경로를 통하여 신소재에 대해 접했던 것으로 나타났으나 신소재에 대해 잘 알고 있다고 응답한 학생과 교사는 많지 않았으며, 신소재에 대한 구체적인 정보를 올바르게 이해하고 있지 못한 교사들도 다수 있었다. 셋째, 학생과 교사 모두 신소재 수업이 필요하다는 점에 대해서는 공감하고 있었다. 신소재의 용도에 대해 알 필요가 있으며 매체 활용 수업을 가장 선호한다는 공통점을 나타냈지만 학생들은 교사들보다 체험학습을 선호하는 등 교사와 학생 입장에서 원하는 수업 진행 방식에는 차이점도 있었다. 이러한 연구 결과를 바탕으로 신소재 수업을 진행하는 교사들이 필요로 하는 것과 이들이 수업을 계획하고 진행할 때 고려할 점들을 제안하였다. Prospective high school students and science teachers' perceptions of 'advanced material', which was first introduced in the science textbooks of the 2009 revised curriculum, were surveyed. One thousand four hundred and ninety nine students and 123 teachers from Seoul, Incheon, and the Kyeonggi areas participated in this survey. The results are as follows. First, the attitude of students and teachers towards 'advanced material' was positive, but their interests in 'advanced material' was low. Also, some teachers mentioned that 'advanced material' was one of the difficult subjects to teach. Second, the perception of 'advanced material' was relatively low for both students and teachers. Both of them had heard of 'advanced material' through various routes, however not many of them thought that they knew what 'advanced material' was exactly. Also there were some teachers who didn't understand the detailed information of 'advanced material'. Third, both students and teachers agreed that 'advanced material' was worthwhile to learn and teach. However, each party had their own desire of 'what to learn', 'what to teach', 'how to learn' and 'how to teach'. Based on the results, some suggestions were made for effective teaching of this new subject.

This comprehensive analysis highlights the potential of Reinforcement Learning (RL) to transform intelligent decision-making systems by examining its techniques and applications in a variety of disciplines. The study offers a thorough examination of the advantages and disadvantages of several reinforcement learning (RL) approaches, such as Q-Learning, Deep Q-Networks (DQN), Policy Gradient Methods, and Model-Based RL. The paper explores RL applications in several domains, including robotics, autonomous systems, and healthcare, showcasing its adaptability in handling intricate decision-making assignments. RL has demonstrated promise in the field of healthcare for managing clinical resources, identifying chronic diseases, and improving patient therapy. Robotics uses reinforcement learning (RL) to create autonomous navigation and adaptive motor skills. The study highlights the advantages of reinforcement learning (RL) in managing high-dimensional state spaces, delayed rewards, and model-free learning, but they also point out certain drawbacks, including sample inefficiency and the exploration-exploitation trade-off. The paper highlights the flexibility and potential effect of reinforcement learning (RL) across industries, providing practitioners and academics looking to exploit RL in intelligent systems with insightful information. The future of adaptive decision-making in real-world scenarios may be shaped by RL's integration with other AI approaches, such as deep learning and transfer learning, which could further broaden its applicability to increasingly complicated domains as it continues to advance.

This study used the Relevance of Science Education (ROSE) survey (Sjoberg & Schreiner, 2004) to examine topics of interest and perspectives of secondary science students in a large school district in the southwestern U.S. A situated learning perspective was used to frame the project. The research questions of this study focused on (a) perceptions students have about themselves and their science classroom and how these beliefs may influence their participation in the community of practice of science; (b) consideration of how a future science classroom where the curriculum is framed by the Next Generation Science Standards might foster students' beliefs and perceptions about science education and their legitimate peripheral participation in the community of practice of science; and (c) reflecting on their school science interests and perspectives, what can be inferred about students' identities as future scientists or STEM field professionals? Data were collected from 515 second year science students during a 4-week period in May of 2012 using a Web-based survey. Data were disaggregated by gender and ethnicity and analyzed descriptively and by statistical comparison between groups. Findings for Research Question 1 indicated that boys and girls showed statistically significant differences in scientific topics of interest. There were no statistical differences between ethnic groups although. For Research Question 2, it was determined that participants reported an increase in their interest when they deemed the context of the content to be personally relevant. Results for Research Question 3 showed that participants do not see themselves as youthful scientists or as becoming scientists. While participants value the importance of science in their lives and think all students should take science, they do not aspire to careers in science. Based on this study, a need for potential future work has been identified in three areas: (a) exploration of the perspectives and interests of non-mainstream students and urban students whose representation in this study was limited; (b) investigation of topics where students expressed low interests topics; and (c) development and design of authentic communities of practice in the science classroom.

ABSTRACTSpatial intelligence plays an important role in the success of nanoscience students specific to their visual ability to perceive structures in three dimensions. The NSF-funded IDEAS project makes use of a unique interactive 3D visualization system, based on immersive environment technology, for research and learning in Materials Science and Engineering (MSE) at UC Merced. In order to determine the effectiveness of the immersive system on nanoscience learning, a pilot project was conducted with undergraduate students, which showed the success of immersive systems in the science learning process. Overall, the immersive environment provided complete control in the construction and analysis of carbon-based nanostructure models. Results also showed the 3D visualization system benefited students with low spatial abilities. To facilitate a better understanding of the structure and properties of nanostructures, the IDEAS project has recently been expanded to allow accelerated simulations for materials research. It is important to integrate these new applications into undergraduate level courses in order to strengthen materials science education, recruit and retain future students, and to adapt modern technologies for future materials science educators. The expansion of the IDEAS project relies on the flexibility of this system to serve as a research tool as well as an innovative resource for science education. To adapt the 3D visualization and computing system and help engage students early in engineering research, our research group gathered practical technical documentation geared towards education of science users, based on both Cognitive Science and MSE Education (MSE-Ed) research. The work presented here involves developing educational resources through the design of audio-visual manuals for effective nanoscience learning. The manuals are being created using commercial software to produce interactive electronic books (ebooks). During the planning of the audio-visual manuals, we discovered that it is imperative to provide adequate educational tools as well as efficient guiding principles for the large number of visual, inductive, and active learners in general engineering education. This interdisciplinary project combines fundamental concepts from materials science and cognitive science, particularly project-based learning and active processing, while considering the concepts of overloading, and the unreliability of natural language, among other topics. This investigation will serve society by enhancing materials science research and education, as well as influencing engineering, chemistry, computer science and cognitive science fields, among others.

Introduction Reinforcement learning formalizes the concept of learning from interactions.1 Broadly, reinforcement learning focuses on a setting in which an agent (decision maker) sequentially interacts with an environment that is partially unknown to them. At each stage, the agent takes an action and receives a reward. The objective of the agent is to maximize rewards accumulated in the long run. There are many situations in health care where decisions are made sequentially for which reinforcement learning approaches could prove useful for decision making. Throughout this article, we consider treatment prescription as an archetypical example to connect reinforcement learning concepts to a health care setting. In this setting, the care provider, the prescribed treatment, and the patients can be viewed as the agent, the action, and the environment, respectively, as depicted in Figure 1.Figure 1: Sequential treatment of AKI or CKD complications modeled as a reinforcement learning problem.Background In this section, with the objective of making reinforcement learning literature more accessible to a clinical audience, we briefly introduce related fundamental concepts and approaches. We refer the interested reader to Sutton and Barto1 for a comprehensive introduction to reinforcement learning. Markov Decision Processes Markov decision processes (MDPs) are a formalism of the sequential decision-making problem that has been central to the theoretical and practical advancements of reinforcement learning. In each stage of an MDP, the agent observes the state of the environment and takes an action, which, in turn, results in a change of the state. This change of state is assumed to be probabilistic with the next state being determined only by the preceding state, the chosen action, and the transition probability. The agent also receives a reward that is a function of the taken action, the preceding state, and the subsequent state. In an MDP, the objective of the agent is to maximize the return defined as the reward accumulated over a time horizon. In some applications, it is common to consider the horizon to be infinite, in which case the future rewards are discounted by a factor smaller than one. The selection of action by the agent on the basis of the observed state is known as the policy. More formally, a policy is a probabilistic mapping from states to each possible action. Because the policy and the reward are a function of the state, it is critical to estimate the utility of being in a certain state. More specifically, the value function is defined as the expected return starting from a given state under the chosen policy. Under this formalism, the objective of the agent is to find the optimal policy that maximizes the value function for all states. Reinforcement Learning Methods Action-value methods are a class of reinforcement learning methods in which the actions are chosen on the basis of the estimation of their long-term value. A prominent example of an action-value method is Q-learning in which the agent iteratively takes actions with the highest estimated values and updates the action-state value function on the basis of new observations. Policy gradient methods are another class of reinforcement learning methods that seek to optimize the policy directly instead of choosing actions on the basis of their respective estimated value. Such methods could be advantageous in health care applications that entail a large number of possible actions, e.g., when recommending a wide range of drug dosages or treatment options. Clinical Applications Reinforcement learning frameworks and methods are broadly applicable to clinical settings in which decisions are made sequentially. A prominent clinical application of reinforcement learning is for treatment recommendation, which has been studied across a variety of diseases and treatments including radiation and chemotherapy for cancer, brain stimulation for epilepsy, and treatment strategies for sepsis.2–5 In such treatment recommendation settings, a policy is commonly known as a dynamic treatment regime. There are various other clinical applications of reinforcement learning including diagnosis, medical imaging, and decision support tools (see refs. 2–5 and the references therein). Reinforcement Learning in Nephrology Although there have been recent applications of machine learning in nephrology,6,7 to the best of the authors' knowledge, the application of reinforcement learning to nephrology has been primarily limited to optimizing the erythropoietin dosage in hemodialysis patients.8,9 However, there are other settings where reinforcement learning has the potential to improve patient care in nephrology. For example, reinforcement learning methods can be adopted in the treatment of the complications of AKI or CKD (Figure 1). In this problem, the state models the conditions of the patient (e.g., vital signs, laboratory test results including urine and blood tests, and urine output measurements). The action refers to the treatment options (e.g., the dosage of medications such as sodium polystyrene sulfonate, and hemodialysis). The reward models the improvement in patient conditions. Similarly, reinforcement learning can help automate and optimize the dosage of immunosuppressive drugs in kidney transplants. Challenges and Opportunities Despite the success of reinforcement learning in several simplified clinical settings, their large-scale application to patient care faces several open challenges. The complexity of human biology complicates modeling clinical decision making as a reinforcement learning problem. The state space in such settings is often enormous, which could make a purely computational approach infeasible. Moreover, modeling all potential objectives a priori as a reward function may not be feasible. To overcome these challenges and realize the potential of reinforcement learning, clinical insight can play a pivotal role. More specifically, restricting the state space to only include highly relevant clinical variables could greatly reduce the computational complexity. Furthermore, using inverse reinforcement learning,2 relevant reward functions can be learned from retrospective studies assuming the optimality of clinical decisions. Another critical challenge is addressing moral and ethical concerns. It is imperative to ensure that reinforcement learning methods do not cause harm to the patient. To this end, there exists a need for a thorough validation of such methods before their use in patient care. Hence, there is a need to go beyond retrospective studies that have been used for the proof of concept of most existing reinforcement learning methods in health care applications.2,3 The lessons learned from the success of reinforcement learning in other application areas (e.g., self-driving cars) can help navigate the path to realizing its potential in health care. Accessible open-source simulation environments that enable researchers to compare various approaches are essential to the field of reinforcement learning. OpenAI Gym is currently the leading toolkit containing a wide range of simulated environments, e.g., surgical robotics.10 The development of high-quality and reliable simulation environments for nephrology and other health care applications can facilitate the development and validation of reinforcement learning methods beyond limited retrospective studies. The adoption of methods validated in such simulation environments in actual clinical settings will require clinicians' oversight. Similar to how self-driving cars require a human driver to ensure collision avoidance, clinicians' oversight is critical to ensure the safety of the patients, especially in the early stages of the adoption of reinforcement learning methods. The data from clinicians' decisions (e.g., overruling the automated treatment recommendation) can be used to improve the reliability of autonomous systems over time and reduce the burden of clinicians' oversight.

A vision for the next phase of JRST

Enhancing the process-structure-property (PSP) loop plays an important role in the field of materials engineering for creating materials with specific characteristics which enhances manufacturing process efficiency. Standard approaches towards developing materials primarily depend according to experimentation evaluation and error, which might be economical & time-saving. Systematically building predictive models for complicated material networks merged with Machine Learning (ML) has shown significant potential in automating and speeding up the improvement in material operations and features with the rise of data-driven innovations. The goal of this study is to construct a model for machine learning designed to enhance material engineering's Process-Structure Property interactions. Different machine learning approaches such as reinforcement learning, deep learning & supervised learning are implemented in the technique to simulate the PSP loop. The models are trained using an enormous array comprising microstructural attributes, process parameters and properties of the material. The architecture integrates data extraction, data preparation & model evaluation protocols to ensure accurate predictions. Material qualities for polymers, metals & ceramics were accurately anticipated using an ML-based optimization methodology. It required quite less time and resources to produce materials compared with earlier strategies. Additionally, the structure proposed appropriate conditions for processing by increasing the material's durability as well as decreasing flaws. The use of machine learning may transform material creation and manufacturing by adapting high-performance developing materials faster and inexpensive. KeywordsMachine Learning, Material engineering, PSP loop, Data-driven innovations, Reinforcement learning, deep learning and supervised learning.

Forty years ago there were essentially no academic departments with titles of “Materials Science” or “Materials Engineering.” There were, of course, many materials departments. They were called “Metallurgy,” “Metallurgical Engineering,” “Mining and Metallurgy,” and other permutations and combinations. There were also a small number of “Ceramic” or “Ceramic Engineering” departments. Essentially none included “polymers.” Over the years titles have evolved via a route that frequently followed “Mining and Metallurgy,” to “Metallurgical Engineering,” to “Materials Science and Metallurgical Engineering,” and finally to “Materials Science and Engineering.” The evolution was driven by recognition of the commonality of material structure-property correlations and the concomitant broadening of faculty interests to include other materials. However, the issue is not department titles but whether a single degree option in materials science and engineering best serves the needs of students.Few proponents of materials science and engineering dispute the necessity for understanding the relationships between processing (including synthesis), structure, and properties (including performance) of materials. However, can a single BS degree in materials science and engineering provide the background in these relationships for all materials and satisfy the entire market now served by several different materials degrees?The issue is not whether “Materials Science and Engineering” departments or some other academic grouping of individuals with common interests should or should not exist.

This research aims to develop a digital Science instructional material based on the 5E learning cycle to enhance elementary school students' learning outcomes. The research method employed Research & Development with the ADDIE approach, consisting of Analysis of Needs, Design of instructional material, Development of instructional material, Implementation, and Evaluation stages. Data collection was conducted through interviews and questionnaires to gather information from the analysis of the needs of 20 elementary school students and 3 class teachers. The analysis results indicate that 66.7% of teachers state the necessity of digital instructional material to improve students' learning outcomes. Meanwhile, interviews with students reveal the need for easily accessible, interactive, and engaging instructional materials. Both students and teachers responded positively to the development of digital instructional materials in Science learning. In conclusion, the analysis underscores the crucial need for digital Science instructional material based on the 5E learning cycle as additional teaching resources. keywords: digital material;science; 5E learning cycle; elementary school

The study presents the results of design-based rese arch on the influence of connectivism on science education, with the emphasis on an international co llaboration among/between teachers and students fro m different countries. Science and technology educati on is a very important part of culture as a knowled ge background of society. Very fast ICT development strongly influences education. The pedagogical theory of connectivism was born as a response to this ICT dev elopment. Thus a need occurred to examine these connectivistic influences on science and technology education. This study presents a design-based rese arch which is focussed on the following issues: identifi cation of connectivistic factors and their influenc e on science education; creation of connectivistic educa tional methods; implementation of connectivistic educational methods into teaching/learning and teac her’s training. These methods were created within t he frame of collaborative action research based on ICT which can be used as a vehicle for international collaboration with effective exploitation of ICT. T he collaborative action research based on ICT was c arried out by two collaborating teachers and their student s in the Czech Republic and in Portugal. Concrete scenarios and strategic planning of the collaborati ve connectivistic teaching/learning are presented o n the topic photosynthesis. Our design-based research res ults verify that implementation of connectivism in science education is reality. We identify the set o f connectivistic factors which influence science ed ucation: selection of topic, selection of students, use of Information and Communication Technologies (ICT), collaboration schedule and elaboration of materials for teaching and learning. Connectivistic educatio nal methods in science education are also presented. Co nnectivistic teaching/learning methods have a very positive influence on science education. This conne ctivistic approach can contribute to reducing the g ap between educational research and school practice. I t will be important to implement our research resul ts into pre-service and especially in-service science teach er training.

Graduate and postgraduate education in science and engineering in the U.S. has long been recognized as outstanding. In materials science and engineering, graduate and postgraduate education is not only outstanding, but as a result of its intrinsic multidisciplinary nature, it is having a positive pervasive impact upon the culture within major research universities in the U.S. Although generally excellent, this paper focuses on areas in which new initiatives are being taken to more broadly enhance education and training in materials science and engineering. We briefly discuss the issues of training modes, undergraduate materials education and research, vocational and “shop floor” level training, as well as some fundamental infrastructure issues that will impact our ability to broadly enhance materials science and engineering education in the U.S.

New Perspectives on Materials Education in the U.S.

Without materials, there will be no meaningful engineering.This idea should be grasped by all engineering students.Understanding of materials science and engineering (MSE) and how it interacts with other engineering fields will ultimately affect potential as competent engineer.To ensure prospective engineers' success, instructors within an engineering program should prioritize working in unison to educate students on MSE.The core MSE and non-MSE courses are discussed from first year through fourth year to illustrate how courses build on previously taught concepts.The courses discussed are MSE and related courses that provide relevant curriculums and instructing methods.It is vital for engineering students to recognize the importance of MSE, and the roles materials play in engineering.MSE education, instruction, and relation to MSE and non-MSE courses are dependent on the engineering program unison.Based on the analysis, it was concluded that the education and application of MSE courses are most effective when key MSE concepts, principles, and knowledge threads are continuously introduced, reviewed, and reintroduced for students in all levels of courses.For better outcomes, it is recommended that MSE instructors highlight materials' importance through application and explain MSE's connections via courses on different subjects whenever possible.

The review delves into creating an understandable framework for machine learning in robotics. It stresses the significance of machine learning in materials science and robotics highlighting how it can transform industries by boosting efficiency and deepening our knowledge of materials on levels. The review also discusses the hurdles posed by attacks on machine learning and the increasing relevance of machine learning in software development. It outlines the approach used in the review, including the search strategy criteria for inclusion and exclusion and the process for selecting studies, including adherence to research published in English only. The classification section organizes the chosen studies into six areas: reinforcement learning, adversarial techniques, applications of learning, and image recognition. In the Discussion section, challenges like critical learning models in robotics unsupervised learning, adversarial attacks on datasets, and limited data for polyp detection are identified. Recommendations for research are provided along with insights into motivations behind these studies; topics covered include reinforcement learning, adversarial examples, domain alignment, and world adversarial attacks on industrial systems.