Current Control System Research Articles

The Application of PID Control in DC Motor Control Systems

Abstract. The application of Proportional-Integral-Derivative (PID) control in direct current (DC) motor control systems is essential for achieving high levels of precision and reliability in both industrial and robotic applications. As DC motors are widely utilized in environments requiring accurate speed and torque control, this study seeks to address common challenges such as overshoot, instability, and sluggish response times that can compromise performance. The research is centered on the development and implementation of a PID-based control strategy using MATLAB/Simulink as the primary tool for simulation and testing. The study involves detailed simulation models and experimental validation, using data collected from controlled lab environments to evaluate the effectiveness of various tuning methods. These methods include adjusting the proportional, integral, and derivative gains to optimize motor responsiveness and system stability. The results indicate that with appropriate tuning, PID controllers can significantly enhance DC motor performance, leading to reduced overshoot, faster response times, and greater overall system reliability. The research concludes that adaptive PID control strategies, which dynamically adjust parameters in response to changing operating conditions, provide a robust solution for improving DC motor control in complex, dynamic, and nonlinear environments. This study lays the groundwork for future innovations in automation and precision-driven industries, offering insights that could be applied across a wide range of applications.

Integrating spoken instructions into flight trajectory prediction to optimize automation in air traffic control

The booming air transportation industry inevitably burdens air traffic controllers’ workload, causing unexpected human factor-related incidents. Current air traffic control systems fail to consider spoken instructions for traffic prediction, bringing significant challenges in detecting human errors during real-time traffic operations. Here, we present an automation paradigm integrating controlling intent into the information processing loop through the spoken instruction-aware flight trajectory prediction framework. A 3-stage progressive multi-modal learning paradigm is proposed to address the modality gap between the trajectory and spoken instructions, as well as minimize the data requirements. Experiments on a real-world dataset show the proposed framework achieves flight trajectory prediction with high predictability and timeliness, obtaining over 20% relative reduction in mean deviation error. Moreover, the generalizability of the proposed framework is also confirmed by various model architectures. The proposed framework can formulate full-automated information processing in real-world air traffic applications, supporting human error detection and enhancing aviation safety.

Energy consumption analysis of trains based on multi-mode virtual coupling operation control strategies

Urban rail transit, as a main mode of travel, has always been concerned with the balanced relationship between energy consumption and transport efficiency. The advent of the virtually coupled train set (VCTS) has significantly enhanced the efficiency of existing railway lines. Both energy consumption and comfort during the operation of VCTS control technology have emerged as significant challenges for the development of VCTS since the inception of onsite testing. A review of the current research status of VCTS control systems has revealed that the majority of current formation control systems are focused on the safe operation of formation trains, the stability of train control systems, and safe train separation models. However, a case study has shown there is a strong correlation between train operating intervals, train control strategies, and train operating energy consumption. Therefore, it is necessary to design and plan the operation modes of formation trains for different train operation requirements. This paper presents four distinct control modes, developed through the integration of the most prevalent VCTS control strategies, operational modes, and operational requirements. The design of these control modes is guided by considerations of energy consumption, safety, and comfort during train operation. A case study of Beijing Subway Line 11 revealed a significant correlation between energy consumption, control mode, and safety under different control modes. In particular, the correlation between train running intervals and energy consumption in a high-precision tracking state provides valuable technical support for the future operation of VCTS technology in different periods, scenarios, and environments.

UAV Trajectory Tracking Using Proportional-Integral-Derivative-Type-2 Fuzzy Logic Controller with Genetic Algorithm Parameter Tuning.

Unmanned Aerial Vehicle (UAV)-type Quadrotors are highly nonlinear systems that are difficult to control and stabilize outdoors, especially in a windy environment. Many algorithms have been proposed to solve the problem of trajectory tracking using UAVs. However, current control systems face significant hurdles, such as parameter uncertainties, modeling errors, and challenges in windy environments. Sensitivity to parameter variations may lead to performance degradation or instability. Modeling errors arise from simplifications, causing disparities between assumed and actual behavior. Classical controls may lack adaptability to dynamic changes, necessitating adaptive strategies. Limited robustness in handling uncertainties can result in suboptimal performance. Windy environments introduce disturbances, impacting system dynamics and precision. The complexity of wind modeling demands advanced estimation and compensation strategies. Tuning challenges may necessitate frequent adjustments, posing practical limitations. Researchers have explored advanced control paradigms, including robust, adaptive, and predictive control, aiming to enhance system performance amidst uncertainties in a scientifically rigorous manner. Our approach does not require knowledge of UAVs and noise models. Furthermore, the use of the Type-2 controller makes our approach robust in the face of uncertainties. The effectiveness of the proposed approach is clear from the obtained results. In this paper, robust and optimal controllers are proposed, validated, and compared on a quadrotor navigating an outdoor environment. First, a Type-2 Fuzzy Logic Controller (FLC) combined with a PID is compared to a Type-1 FLC and Backstepping controller. Second, a Genetic Algorithm (GA) is proposed to provide the optimal PID-Type-2 FLC tuning. The Backstepping, PID-Type-1 FLC, and PID-Type-2 FLC with GA optimization are validated and evaluated with real scenarios in a windy environment. Deep robustness analysis, including error modeling, parameter uncertainties, and actuator faults, is considered. The obtained results clearly show the robustness of the optimal PID-Type-2 FLC compared to the Backstepping and PID-Type-1 FLC controllers. These results are confirmed by the numerical index of each controller compared to the PID-type-2 FLC, with 12% for the Backstepping controller and 51% for the PID-Type-1 FLC.

Research on constant tension yarn transfer control technology based on AGA-PID and FOC algorithm

To solve the problems of poor real-time tension tracking, large overshoot, long adjustment time, and large tension fluctuation error that exist in the current control system for yarn feeding in the circular weft knitting machine, this paper presents an optimized and improved adaptive genetic algorithm (AGA) for adjusting the PID (Proportional Integral Derivative) parameters of the tension control part of the yarn-conveying system (AGA-PID). The algorithm is combined with the field-oriented control technology of the permanent magnet synchronous motor to design a closed-loop control strategy for the yarn-conveying system of a circular weft knitting machine. The MATLAB-Simulink simulation model was used to analyze the external tension loop-based AGA-PID controller and permanent magnet synchronous motor dual closed-loop control strategy of the yarn transport control system. The experimental verification was conducted by constructing an experimental platform that simulates the real process of yarn transportation. The outcomes of system simulations and experiments have consistently demonstrated that the AGA-PID control algorithm outperforms the GA-PID and PID control algorithms in terms of overshooting, response speed, stability, and anti-jamming ability. The yarn transport control system, which is designed to operate based on an AGA-PID control algorithm, is capable of reducing the fluctuations in tension that occur during the transportation of yarn and exhibits enhanced control accuracy, effectively stabilizing yarn tension control during the conveying process. The application of this algorithm can enhance the overall elasticity uniformity and surface flatness of knitted fabrics, thereby facilitating the realization of adjustable and controllable local elasticity in fabrics.

CSNS-II multi-harmonic LLRF upgrade and commissioning

The second phase of the China Spallation Neutron Source (CSNS-II) will employ a dual-harmonic RF system to mitigate the significant space charge effects associated with 500 kW beam power. To address this, three magnetic alloy loaded cavities, characterized by their wideband and high-gradient properties, will be integrated as multi-harmonic cavities in the upgrade project. Additionally, the current LLRF control system hardware, based on the CPCI platform, is encountering issues related to discontinuation and performance limitations. This paper presents the testing and verification of hardware upgrades and multi-harmonic control system algorithms for the magnetic alloy loaded cavity LLRF control system, ensuring the effective implementation of the multi-harmonic control algorithm within the new hardware framework.

An open-source data storage and visualization platform for collaborative qubit control.

Developing collaborative research platforms for quantum bit control is crucial for driving innovation in the field, as they enable the exchange of ideas, data, and implementation to achieve more impactful outcomes. Furthermore, considering the high costs associated with quantum experimental setups, collaborative environments are vital for maximizing resource utilization efficiently. However, the lack of dedicated data management platforms presents a significant obstacle to progress, highlighting the necessity for essential assistive tools tailored for this purpose. Current qubit control systems are unable to handle complicated management of extensive calibration data and do not support effectively visualizing intricate quantum experiment outcomes. In this paper, we introduce Qubit Control Storage and Visualization (QubiCSV), a platform specifically designed to meet the demands of quantum computing research, focusing on the storage and analysis of calibration and characterization data in qubit control systems. As an open-source tool, QubiCSV facilitates efficient data management of quantum computing, providing data versioning capabilities for data storage and allowing researchers and programmers to interact with qubits in real time. The insightful visualization are developed to interpret complex quantum experiments and optimize qubit performance. QubiCSV not only streamlines the handling of qubit control system data but also improves the user experience with intuitive visualization features, making it a valuable asset for researchers in the quantum computing domain.

Application of improved fish school algorithm in variable frequency speed control system

In order to solve the problems of long adjustment time and poor stability of the commonly used speed current variable frequency negative feedback speed control system in engineering, the author proposes an improved fish school algorithm application method in variable frequency speed control systems. A fish swarm algorithm optimized based on the arena method is applied to a variable frequency speed control system, and the optimization algorithm is applied to the speed control system to screen the PI parameters that meet the requirements of the speed control system. The simulation results show that compared with manual parameter tuning, the controller parameters optimized by the improved fish school algorithm have better control performance. In the absence of overshoot starting, the starting time was shortened by 0.21s, and the system response speed was improved. When a sudden load of 6 N m is applied, the speed drop is reduced by 3 r · min−1, and the recovery time is shortened by 0.15 s, resulting in stronger anti-interference performance of the system. Conclusion: The new algorithm shortens the control time and improves the robustness of the system.

Study of New Material Management System in Construction Sector

Construction is a major component, which forms 45% to 50% of the country’s development activity. Construction materials and components contribute about 50% to 60% of the total value of construction. There will be some difficulty in defining construction materials. Some are exclusively used in construction and others in other industries also. Some form heavy percentage and some very minor percentage, but critical in their nature and affect the construction activity. The management and control of inventory is a problem common to all organizations in any sector of the economy. The wealth of shareholders also lies in the warehouse. More than 60% of working capital is normally being invested in the inventory. There can be disadvantages in holding either too much or too little inventory. Therefore inventory management is primarily concerned with obtaining correct balance between two extremes. Inventory management involves the development and administration of policies, system and procedures which will minimizes total cost relative to inventory decisions and related functions such as customer service requirement, production scheduling, purchasing etc.., Viewed in that perspective, inventory management has a broad scope and affects a great or number of activities in an Organization. For the past few years the concept of inventory system has gained more importance in our country. It is due to intense competition in the market, which has forced organizations to search for proper inventory control technique to reduce investment in inventories and thereby reducing overall cost. In this study the study of current inventory system is carried out and by combining the different inventory system the one new inventory system developed .By comparing the cost benefit analysis between current inventory control system and new inventory system by taking suitable actual case study will be carried out

User-Centered Evaluation of the Wearable Walker Lower Limb Exoskeleton; Preliminary Assessment Based on the Experience Protocol.

Using lower limb exoskeletons provides potential advantages in terms of productivity and safety associated with reduced stress. However, complex issues in human-robot interactions are still open, such as the physiological effects of exoskeletons and the impact on the user's subjective experience. In this work, an innovative exoskeleton, the Wearable Walker, is assessed using the EXPERIENCE benchmarking protocol from the EUROBENCH project. The Wearable Walker is a lower-limb exoskeleton that enhances human abilities, such as carrying loads. The device uses a unique control approach called Blend Control that provides smooth assistance torques. It operates two models simultaneously, one in the case in which the left foot is grounded and another for the grounded right foot. These models generate assistive torques combined to provide continuous and smooth overall assistance, preventing any abrupt changes in torque due to model switching. The EXPERIENCE protocol consists of walking on flat ground while gathering physiological signals, such as heart rate, its variability, respiration rate, and galvanic skin response, and completing a questionnaire. The test was performed with five healthy subjects. The scope of the present study is twofold: to evaluate the specific exoskeleton and its current control system to gain insight into possible improvements and to present a case study for a formal and replicable benchmarking of wearable robots.

Применение машинного обучения и интеллектуального анализа данных в автоматизированной системе неразрушающего вихретокового контроля поверхностного слоя деталей подшипников

Changing the properties of the material during physical and mechanical processing can significantly reduce the working life of the manufactured product, therefore it is important to control the quality of the surface layer of the parts. To solve this problem, non-destructive testing techniques such as etching, visual, capillary, magnetic powder, ultrasonic, vibration, eddy current methods are used at bearing enterprises. The article discusses the physical foundations of the presented techniques and provides their comparative analysis. Machine vision and digital signal processing approaches can be used to automate the processing of the results of non-destructive testing of the surface layer of bearing parts within the framework of the Industry 4.0 concept. From the point of view of productivity and the possibility of integration into the production system, the eddy current method is the most promising, the result of surface control in this way is an array of digital values. The development of modern methods of information analysis makes it possible to efficiently process a large amount of data, and machine learning makes it possible to solve problems of classification, regression, etc. This article provides methodological support for the development and application of an automated eddy current control system using machine learning and data mining methods. The works of scientists devoted to the processing of the results of the shock control of various objects, including bearing parts, are considered, it is noted that previously attention had not been paid to the issue of a reasonable choice of a machine learning model for recognizing defects on the surface of parts. The possibility of using the median polishing method to transform the eddy current signal is shown. The development and implementation of a bearing defect recognition system based on the methodological support presented in this paper can significantly improve the efficiency of product quality control and optimize the technological process.

Deep learning-based prediction of 3-dimensional silver contact shapes enabling improved quality control in solar cell metallization

The industrial metallization of Si solar cells predominantly relies on screen printing, with silver as the preferred electrode material. However, the design of commercial screens often leads to suboptimal silver usage and increased electrical resistance due to print-related inhomogeneities like mesh marks, constrictions and spreading. Real-time monitoring of quality parameters during production has thus become increasingly critical. Current inline optical quality control systems usually only include 2D visualizations of the printed layout, which limits their effectiveness in quality control. Options that allow 3D measurements are usually slow, expensive, and therefore not worth considering in most cases. This research focuses on the development of a model that can estimate the three-dimensional shape of printed contact fingers from a single 2D image without the need of additional hardware using deep learning. Furthermore, a workflow for the generation of training data, which involves the creation of image pairs from a 2D microscope and a 3D confocal laser scanning microscope (CLSM) to accurately represent solar cell fingers, is presented. After model training, the predicted height maps are compared with the ground truth height maps, and the robustness of the model with respect to a paste variation and screen parameter variation is examined. The results confirm the feasibility and reliability of deep learning-based 3D shape estimation, extending its applicability to new, previously unseen data from screen-printed contact fingers. With a structural similarity index (SSIM) score of 0.76, a strong correlation between the estimated and ground truth height maps is established. In summary, our deep learning-based approach for height map estimation offers an effective and reliable solution for fast inline detection and analysis of the cross-sectional area of the printed contact fingers.

Load torque observation and compensation for permanent magnet synchronous motor based on sliding mode observer

Background Permanent magnet synchronous motors (PMSM) are widely used in various industries. However, in practical applications, the time-varying nature of load torque may lead to speed fluctuations, negatively impacting the motor's control performance and stability. To mitigate these issues, this paper proposes a load torque observation method for PMSMs based on a sliding mode observer. Methods The sliding mode observer is designed to estimate the load torque and convert it into a current, which is fed back as a feedforward compensation to the q-axis current in the current closed-loop control system. The observer's dynamic performance is optimized using a genetic algorithm to minimize the Integral of Time-weighted Absolute Error (ITAE) between the observer and the actual system state. Experimental tests are conducted on a motor torque testing platform. After stabilizing the motor at 800rpm, a sudden torque of 0.5Nm is applied. Results Compared to the situation without load torque compensation, the motor speed fluctuations are reduced by approximately 60% after adding load torque compensation. Conclusions This enhancement improves the system's speed control performance during torque variations and increases the system's robustness and disturbance rejection capability.

Power Trading and Access Control Scheme Based on IPFS and Blockchain

In recent years, a large number of renewable energy power plants have been built all over the country, which has led to a sharp increase in the pressure on the current centralized electricity trading platform. There are data storage bottlenecks and data privacy problems in the current blockchain-based power transaction and access control system. In this paper, the interstellar file system is proposed to store the data of the new distributed power trading platform in a distributed way, so as to improve the storage capacity of the physical nodes; An improved ciphertext strategy attribute encryption scheme, which is lightweight, traceable and supports outsourced decryption combined with state secret algorithm, is adopted to integrate the user’s unique identity into the user’s private key and part of the expensive decryption calculation is outsourced to the cloud server to realize fine-grained data access control in the electricity market. The experimental results show that compared to the basic CP-ABE scheme, when the encryption attribute is 100, the decryption time of the proposed scheme is reduced from 556[Formula: see text]ms to 1.1[Formula: see text]ms; Compared to blockchain-based solutions alone, when storing data of 15[Formula: see text]MB, GAS consumption decreases from 2968738133 to 89360. The scheme can greatly improve the storage capacity of the system, improve the operational efficiency and meet the actual performance requirements.

Predictive Controller Strategies for Electrical Drives System using Inverter System

Advanced control strategies in power electronics include Predictive controller of current (P CURRENT CONTROL) and Predictive controller of torque (P TORQUE CONTROL). In order to operate a SRM or an induction machine, the Predictive controller of torque (P TORQUE CONTROL) approach analyses the stator flux and electromagnetic torque in the cost function (IM), and the Predictive controller of current (P CURRENT CONTROL) method [1,2] takes errors between the current reference and the measured current into account in the cost function. The switching vector selected for usage in IGBTs reduces the error between the references and the predicted values. The system restrictions are easy to include [4, 5]. The weighting component is not required. Together with the P TORQUE CONTROL and P CURRENT CONTROL systems, the SRM method is the most practicable direct control technique since it doesn't require a modulator and offers 10% to 30% more power than an induction motor [3]. With the same current, an induction motor can only generate between 70 and 90 percent of the force generated by an SRM due to its lagging power factor. SRM approach decreases 23% more THD in torque, speed, and stator current when P CURRENT CONTROL and P TORQUE CONTROL method with 15-level H-bridge multilevel inverter is compared to P CURRENT CONTROL and P TORQUE CONTROL method with 15-level H-bridge multilevel inverter utilising induction motor [21]. The transistors are only swapped when necessary to maintain the limits of torque and flux, which minimises switching losses. To improve the efficiency of a multilevel inverter, semiconductor switches are switched in a specific pattern. In contrast to the P TORQUE CONTROL and P CURRENT CONTROL approaches using a 2-level voltage source inverter, the 15-level H-bridge multilevel inverter employed in this study, coupled with SRM and IM, gives outstanding torque and flux responses and achieves robust and stable operation. This unique strategy quickly caught the interest of academics due to its simple algorithm and high performances in both steady and transient modes [8].

A Cell-Type Selective Stimulation and Recording System for Retinal Ganglion Cells.

Future retinal implants will require a stimulation selectivity between different sub-types of Retinal Ganglion Cells (RGCs) to evoke natural perceptions rather than phosphenes in patients. To achieve this, a cell-type specific stimulation pipeline is required that identifies target RGC sub-types from recorded input images and extracts the specific stimulation parameters to activate this cell-type selectively. Promising biological experiments showed that ON-/OFF- sustained/transient RGCs could be selectively activated by modulating repetition rate and amplitude of an electrical stimulation current in the kilohertz range. This research presents a 42 channel current controlled stimulation and recording system on chip (SoC) with parameter input from a real time target RGC selection algorithm. The SoC is able to stimulate retinal tissue with sinusoidal frequencies higher than 1 kHz at amplitudes of up to 200 μA at a supply voltage of 1.8 V. It also includes tunable recording units with an integrated action potential detection pipeline that are able to amplify signals between 1 Hz and 50 kHz. The required area of one stimulator is 0.0071 mm2, while one recording unit consumes an area of 0.0092 mm2. The application of sinusoidal stimulation currents in the kilohertz range towards retinal tissue leads to a suppressive response of only certain RGC sub-types that has not been oberved before, using electrical stimulation. Because this response is very similar to the natural light response of RGCs, this stimulation approach can lead to a more genuine visual perception for patients using retinal implants.

Control system of the electric drive of a robot with a dc motor

Problem. Robotics and flexible production systems are the most important technical basis for the intensification of production. The use of robotic systems is expanding every year. This paper considers a two-circuit servo control system using the example of a DC motor current and speed control system and examines the functional diagram of the third circuit, a digital angle control system. Using the Matlab + SIMULINK software package, the servo drive and the digital system for controlling the angular position of the robot manipulator were modelled. Goal. As a result of the analysis, the purpose of the study was highlighted: the development of a digital system for adjusting the angular position of the manipulator, improve the quality of its positioning. Tasks: to study work management systems; to study the drives of robots; to get a mathematical model of an electric motor; to develop a machine diagram of a digital electric drive; to synthesize a three-loop robot positioning control system. Methodology. The research problems should be solved by methods of analysis and synthesis of automatic control systems. Results. In the work, a three-circuit control system of the electric drive of a robot with a direct current motor was developed. Thus, as a result of modeling the operation of a digital electric drive with three control circuits for current, speed and positioning, an aperiodic transient process was provided with quality indicators. Originality. Thus, the results of simulation of the two-loop servo control system showed that the dynamic characteristics of the electric motor in terms of current and speed of rotation, which meet the requirements of the task, are provided by the PI controller in the current control circuit and the PID controller in the speed control circuit. Thus, as a result of modeling the operation of a digital electric drive with three control circuits for current, speed and positioning, an aperiodic transient process was provided with quality indicators. Practical value. The following results were obtained during the research: the best control law for the current circuit is the proportional-integral (PI) law; the best control law for the speed contour is the proportional-integral-differential (PID) law; the best control law for a digital positioning control loop is the proportional (P) law.

Tecuide. Telemonitorización de pacientes con deterioro cognitivo y sus cuidadores

IntroductionGiven the growing increase in dementia, the need to control these patients, together with the rise of new technologies, makes a change in the current control system imperative. Material and methodWe have carried out a single-center, clinical study with two groups, a control group of 72 patients/caregivers, who followed the usual controls in consultations, and another telematic group of 76 patients/caregivers, who followed the controls through of the Tecuide platform. The platform had a survey part to detect problems in patients and caregiver claudication, another training part and another chat for direct communication when the caregiver needed it and also served to respond when a problem was detected. ResultsAfter a year of monitoring with the platform we have obtained: a)in patients, reduce behavioral disorders and use of drugs, increase physical exercise and delay institutionalization (DS not found); b)in caregivers there is an improvement in satisfaction with respect to the control of patients with cognitive impairment, and c)in terms of resources, visits to emergency services and dementia consultations have decreased, although admissions to the psychogeriatric unit have increased. ConclusionsThe use of Tecuide as a telematic tool in the control of patients with cognitive impairment does not seem to be inferior to the usual controls in consultations and improves caregiver satisfaction.

Fuzzy Neural Network PID-Based Constant Deceleration Control for Automated Mine Electric Vehicles Using EMB System.

It is urgent for automated electric transportation vehicles in coal mines to have the ability of self-adaptive tracking target constant deceleration to ensure stable and safe braking effects in long underground roadways. However, the current braking control system of underground electric trackless rubber-tired vehicles (UETRVs) still adopts multi-level constant braking torque control, which cannot achieve target deceleration closed-loop control. To overcome the disadvantages of lower safety and comfort, and the non-precise stopping distance, this article describes the architecture and working principle of constant deceleration braking systems with an electro-mechanical braking actuator. Then, a deceleration closed-loop control algorithm based on fuzzy neural network PID is proposed and simulated in Matlab/Simulink. Finally, an actual brake control unit (BCU) is built and tested in a real industrial field setting. The test illustrates the feasibility of this constant deceleration control algorithm, which can achieve constant decelerations within a very short time and maintain a constant value of -2.5 m/s2 within a deviation of ±0.1 m/s2, compared with the deviation of 0.11 m/s2 of fuzzy PID and the deviation of 0.13 m/s2 of classic PID. This BCU can provide electric and automated mine vehicles with active and smooth deceleration performance, which improves the level of electrification and automation for mine transport machinery.

Malicious Packet Detection Technology Using Reinforcement Learning

In the present day, the advancement of 5G and Internet of Things (IoT) technology has resulted in the interconnectedness of everyday objects through networks. However, attempts to exploit networked computers for malicious purposes continue to rise while the attacks utilizing malicious codes to compromise user information's confidentiality and integrity are becoming increasingly sophisticated and intelligent. To counter these evolving threats, researches have been conducted on a method to identify malicious network packets using a combination of security control systems and Artificial Intelligence (AI) technology, especially supervised learning. Unfortunately, the current cybersecurity control systems suffer from inefficiencies in terms of both manpower and cost. Moreover, the surge in remote work has created challenges in responding swiftly to security incidents. Furthermore, the existing AI technology based on supervised learning has limitations, particularly in detecting new variants of malicious code, and its accuracy in identifying malicious code depends heavily on the quantity and quality of available data. In light of these challenges, this study, reinforcement learning is employed to overcome the limitations of the original supervised learning-based malicious packet detection system, such as high dependency on training data and the failure to detect variant malicious packets. This research proposes a malicious packet detection technology capable of addressing new malicious packets or variant types effectively.