Static Allocation Method Research Articles

Dynamic berth allocation under uncertainties based on deep reinforcement learning towards resilient ports

With the evolving global trade landscape and the post-pandemic effects, the resilience of ports has become paramount. The unforeseen disturbances bring substantial challenges, especially in berth allocation, a vital task ensuring seamless resilient port operations. The unpredictability of vessel arrivals and the variability in loading/unloading times intensify these issues, pushing traditional static allocation methods beyond their limits. Fortunately, the advent of smart ports has led in an era of big data availability, enabling the application of advanced deep reinforcement learning (DRL) techniques. To capitalize on this shift, this research presents a DRL-based methodology specially designed to solve the berth allocation problem with the uncertainties in vessel arrival and container handling time to enhance port resilience. A Markov Decision Process model (MDP) of the berth allocation problem is established to minimize the mean waiting time with tailored state space, rule-based action space, and reward function to address the issue. An offline training method is designed to train the agent in selecting the optimal action based on the current state of the port berth system at each decision point even in uncertain environments, deep Q-network (DQN) is implemented for this problem. Comprehensive experiments across different problem scales are conducted to validate the effectiveness and generality of the proposed method in solving berth allocation challenges under uncertain conditions. Furthermore, the trained model also performs better than other methods in different vessel congestion levels through learning.

OPTIMIZING EFFICIENCY AND PERFORMANCE IN 5G NETWORKS THROUGH A DYNAMIC RESOURCE ALLOCATION ALGORITHMIC FRAMEWORK

With the exponential growth of data demand and the advent of 5G networks, the need for efficient resource allocation algorithms has become paramount. This study presents a dynamic resource allocation algorithmic framework aimed at optimizing efficiency and performance in 5G networks. The framework focuses on frequency reuse at the edges while employing fractional pilots for enhanced spectrum utilization. 5G networks promise unprecedented speeds and low latency, enabling a wide array of applications from IoT to augmented reality. However, the efficient allocation of resources remains a challenge, especially at the network edges where interference is high. Traditional static resource allocation schemes fail to adapt to dynamic network conditions, leading to suboptimal performance. The main challenge lies in effectively managing resources to meet the diverse demands of various applications while mitigating interference and maximizing spectral efficiency. The proposed framework employs a dynamic resource allocation algorithm that adapts to changing network conditions in real-time. Leveraging fractional pilots, the algorithm optimizes frequency reuse at the network edges, thereby enhancing spectral efficiency. The framework integrates stochastic learning for predictive analytics to anticipate resource demands and interference patterns. Simulation results demonstrate significant improvements in spectral efficiency and network performance compared to traditional static allocation methods. The utilization of fractional pilots effectively reduces interference, enabling higher throughput and lower latency, especially at the network edges. The dynamic nature of the algorithm ensures adaptability to varying traffic loads, leading to enhanced overall network efficiency.

A novel machine learning-based framework for channel bandwidth allocation and optimization in distributed computing environments

Efficient utilization of network resources, particularly channel bandwidth allocation, is critical for optimizing the overall system performance and ensuring fair resource allocation among multiple distributed computing nodes. Traditional methods for channel bandwidth allocation, based on fixed allocation schemes or static heuristics, often need more adaptability to dynamic changes in the network and may not fully exploit the system’s potential. To address these limitations, we employ reinforcement learning algorithms to learn optimal channel allocation policies by intermingling with the environment and getting feedback on the outcomes of their actions. This allows devices to adapt to changing network conditions and optimize resource usage. Our proposed framework is experimentally evaluated through simulation experiments. The results demonstrate that the framework consistently achieves higher system throughput than conventional static allocation methods and state-of-the-art bandwidth allocation techniques. It also exhibits lower latency values, indicating faster data transmission and reduced communication delays. Additionally, the hybrid approach shows improved resource utilization efficiency, efficiently leveraging the strengths of both Q-learning and reinforcement learning for optimized resource allocation and management.

Experience in dynamic tape drive allocation to manage scientific data

The main computing and storage facility of INFN (Italian Institute for Nuclear Physics) running at CNAF hosts and manages tens of Petabytes of data produced by the LHC (Large Hadron Collider) experiments at CERN and other scientific collaborations in which INFN is involved. Most of these data are stored on tape resources of different technologies. All the tape drives can be used for administrative tasks (as repack, audit, space reclamation), as well to write and read data of all the experiments. Moreover, the usage of tape resources by scientific communities will become considerably more intense in the next years and the amount of data on tape will double by 2025. For these reasons, the issue of the concurrent access to tape drives is significant. We designed a software solution to optimize the efficiency of the shared usage of tape drives in our environment and put it in production in January 2020. In this paper we present the experience with such dynamic tape resources allocation in production. Comparing it with the previous static allocation method, we observed an improvement in reading throughput up to 85%. Moreover, we describe the new features added to our solution to optimize the efficiency of the shared usage of tape drives of different technologies.

Dynamic allocation of human resources: case study in the metal 4.0 manufacturing industry

Industry 4.0 concepts make it possible to rethink human resources allocation, even for more traditional environments like metal machining. While parts machining on Computer Numerical Control (CNC) machines is automated, some manual tasks must still be executed by operators. The current approach is typically that operators are statically allocated to one or many machines. This causes avoidable bottlenecks. We propose an optimisation model to dynamically assign tasks to the operators with the objective of minimising production delays. Three different scenarios are compared; one representing the current widely used static allocation method and two others that allow more flexibility in the operators’ allocation. The dynamic task assignment problem is solved using a constraint programming model. The model was applied to a case study from a high-precision metal manufacturing job shop. Experimental results show that switching from a static allocation to a dynamic one reduces by 76% the average production delays caused by human operators. Supposing more versatile operators under the dynamic allocation leads to further improvements.

Joint Allocation of Radio and Fronthaul Resources in Multi-Wavelength-Enabled C-RAN Based on Reinforcement Learning

Cloud radio access network (C-RAN) is a key technology for the new-generation mobile network (5G). Multi-wavelength passive optical networks (PONs) such as wavelength division multiplexing (WDM) and time wavelength division multiplexing (TWDM) PONs are outstanding solutions for providing a sufficient bandwidth to support mobile fronthaul in C-RAN-based 5G architecture. In this paper, a joint allocation framework for multi-wavelength PONs fronthaul uplink resources and C-RAN radio interface uplink resources is presented. From the principle that the uplink resource allocation in mobile networks is an NP-hard optimization problem, this paper proposes a novel uplink scheduling algorithm based on reinforcement learning (RL). The performance of the algorithm is evaluated with numerical simulations and compared with two other algorithms from the literature; namely genetic algorithm (GA) and tabu search (TS) algorithm. The simulation results show that the new algorithm achieves higher system throughput, faster convergence and lower scheduling latency compared to the two other algorithms. Also, show that RL-based adaptive allocation of fronthaul transport block size based on actual radio resource block capacity can significantly reduce the capacity required on fronthaul link and decrease the total end to end uplink scheduling latency compared to the static allocation method.

Analysis and comparison of the two sectoring approaches in elevator traffic systems

Sectoring is a group control algorithm that is used in elevator traffic control systems by grouping passengers that have common destinations or common origins into elevator cars that serve these floors. The building is split into sectors usually comprising contiguous floors. Two different alternative algorithms for sectoring are discussed in this paper. The first approach is based on dynamic allocation with equal sector allocation. The second approach is based on static allocation with unequal sector sizes. Under static allocation, the same elevator car is allocated to the same sector in every round trip. Under dynamic allocation, each elevator car is allocated to a different sector in each round trip. Under the dynamic allocation scheme suggested in this paper, the elevator cars are sequenced to the various sectors in the buildings in a round-robin fashion. It is important to note that under both schemes, the provided (relative) handling capacity of different sectors is equalised. Five different buildings have been analysed using the two suggested sectoring algorithms. The building is first designed by finding the required number and speed of elevators assuming conventional control. Each building is then analysed using one of the two suggested sectoring algorithms. In order to compare the performance of the two sectoring algorithms, the provided (relative) handling capacity is calculated. The provided (relative) handling capacity of the two suggested algorithms is then compared. Very little difference was found between the two algorithms. The dynamic sectoring with equal sector sizes offers the convenience of having equal sector sizes. The static sectoring with unequal sector sizes is more convenient for passengers that are familiar with the building. Practical application: This paper analyses two different options for sectoring the control system of an elevator system in a building. Each of the two sectoring methods is suitable for different situations. The dynamic allocation method is more suitable for destination group control systems and offers the group controller more flexibility. The static allocation method with unequal sector sizes ensures that the passengers remain familiar with the elevators that they use to get to their floors. The programmer of the elevator group controller can programme both methods in the controller and allow it to switch to the most suitable algorithm depending on the prevailing conditions.

A Predictive Workload Balancing Algorithm in Cloud Services

Performance of dynamic clouds depends on the efficiency of its load balancing and resource allocation. This paper is an exploratory study on the predictive approach for dynamic resource distribution of cloud services. Efficient cloud resource management can be achieved by simulating cloud services based on the predictions of incoming workloads, which can be more efficient than static allocation methods. This paper introduces a rule-based workload-balancing algorithm based on the predictions of an end-to-end system called Cicada. A simulation of cloud services can be achieved by a cloud service simulator called CloudSim and it will be used to achieve an algorithm with lower computational demand and a faster workload balancing. The final result will demonstrate the effectiveness of a predictive workload balancing approach that can achieve faster workload balancing with a lower computational power usage.

Multi-User MAC Protocol for WLANs in MmWave Massive MIMO Systems With Mobile Edge Computing

Merging mobile edge computing (MEC) service with the wireless local area networks (WLANs) provides enormous benefits such as intensive computation capabilities. Access to high-speed WLANs has become an urgent challenge due to the fast rate of application development and user adoption. Providing increased wireless resources is one solution, but millimeter (mmWave) multi-user massive Multiple-Input Multiple-Output (MIMO) technology has also attracted attention due to a series of features such as high throughput, strong anti-interference ability, and small fading. Hence, exploring the potential of mmWave MIMO systems for WLAN could play a significant role in improving network performance, especially throughput. In this paper, we introduce mmWave and massive MIMO into traditional WLAN with MEC, which enables a large portion of spatial resources to be allocated among users, and mitigates inter-user interference. Moreover, we propose a novel media access control (MAC) protocol, named as LSMWN-MAC, to adapt to a series of unique characteristics of mmWave and massive MIMO, which adopts space division multiple access (SDMA) as one of the resource methods, taking full advantage of spatial resources. In addition, we modify the protocol and propose the MLSMWN-MAC protocol to allocate resources dynamically, a solution that is particularly suitable for scenes with 30-50 users. Simulation results show that compared to 802.11ad, the proposed two protocols can increase the saturated throughput by 4.7Gb/s and 5.4Gb/s respectively, which are close to the throughput upper limit of the mmWave MIMO system. Meanwhile, we observe that when the number of users exceeds the threshold, the performance of the LSMWN-MAC protocol using a static allocation method is superior to the MLSMWN-MAC protocol using a dynamic method. In other cases, the dynamic protocol has better performances.

On-line Energy Allocation Based on Approximate Dynamic Programming for Iron and Steel Industry

Energy allocation in iron and steel industry is the assignment of available energy to various production users. With the increasing price of energy, a perfect allocation plan should ensure that nothing gets wasted and no shortage. This is challenging because the energy demand is dynamic due to the changes of orders, production environment, technological level, etc. This paper try to realize on-line energy resources allocation under the situation of dynamic production plan and environment based on typical energy consumption process of steel enterprises. Without definite analytical model, it is a tough task to make the energy allocation plan tracks the dynamic change of production environment in real time. This paper proposes to deal with dynamic energy allocation problem by interactive learning with time-varying environment using Approximate Dynamic Programming method. The problem is formulated as a dynamic model with variable right-hand items, which is an updated energy demand obtained by on-line learning. Reinforcement learning method is designed to learn the energy consumption principle from the historical data to predict energy consumption level corresponding to current production environment and the production plan in future horizon. Using the prediction results, on-line energy allocation plan is made and its performance is demonstrated by comparison with static allocation method.

Dynamic Thrust Allocation of Dynamic Positioning Vessel Based on Model Predictive Control

The dynamic positioning (DP) vessel maintains its position and heading by active thrusters. For safety reasons, DP vessels are typically designed with redundancy thrusters more than needed for motion control. Optimization theories are useful in finding thrust allocation solutions that minimize fuel consumption and reduce “wear and tear” on a thruster. But several challenges exist such as uncertain thruster model, thruster dynamics characters and the individual limitations of the thrusters. In this paper, a dynamic thrust allocation scheme is presented based on model predictive control (MPC) that directly takes thrusters with dynamics characters and various constraints into account. It is shown in simulations that the MPC dynamic thrust allocation scheme performs better in comparison with an existing static allocation method. Its main advantage is the ability to handle thruster dynamics characters and various constraints.

CAPACITY IMPROVEMENT OF MIMO-OFDMA SYSTEM USING ADAPTIVE RESOURCE ALLOCATION ALGORITHM

The user demands for wireless communication are increasing but the existing networks are not able to fulfill the high data rate of future communication services. More bandwidth can support high data rate wireless access. Due to spectral limitations, it is very expensive to increase. Therefore, the available bandwidth is needed to use efficiently. Orthogonal Frequency Division Multiplexing (OFDM) has been explored as a promising air interface technology to utilize the bandwidth effectively. OFDM can be exploited as a multiple access technique, called Orthogonal Frequency Division Multiple Access (OFDMA) that can support both frequency and multiuser diversity. In order to serve the user population, one of the most sophisticated techniques, resource allocation technique, can be integrated into the OFDMA system. Multiple Input Multiple Output (MIMO) technology can increase the spectral efficiency because multiple data streams can be simultaneously transmitted over the channel. The combination of MIMO technology and OFDMA system can achieve both higher spectral efficiency and better Quality of Service (QoS) such as capacity, fairness, and Bit Error Rate (BER). In this paper, a MIMO-OFDMA system is developed for downlink transmission within one single cell. Two optimization methods such as subcarrier allocation and power allocation are used for resource allocation. To decompose the MIMO channel into parallel single channels, Eigen Value Decomposition (EVD) is used. Subcarrier selection is based on the priority of the user by using the dominant eigen channels. Water-filling algorithm is adapted for power distribution among users. The aim is to maximize the system capacity subject to constraints on total power and proportional fairness. The performance of the system is evaluated by comparing the static allocation method (equal subcarrier allocation with equal power) and two adaptive allocation methods (adaptive subcarrier allocation with equal power and adaptive subcarrier allocation with adaptive power).

Resource Allocation in Downlink MIMO-OFDMA with Proportional Fairness

A new scheme for MIMO-OFDMA (Multiple Input Multiple Output - Orthogonal Frequency Division Multiple Access) downlink resource allocation is presented in this paper. It has an objective of maximizing the total system capacity while having constraints on the total available power and proportional fairness. Dominant eigen-channels obtained from MIMO state matrix are used to formulate this scheme with low complexity. It gives a tradeoff strategy among system capacity, computational complexity and proportional fairness. Simulation results show that this new adaptive allocation scheme can provide much better capacity gain than static allocation methods while achieve near optimal fairness of rate distribution with a linear complexity. As a result, it may be suitable for fulfilling diverse QoS requirements of MIMO-OFDMA systems.

Transmit Selection Diversity for Unitary Precoded Multiuser Spatial Multiplexing Systems With Linear Receivers

Multiuser spatial multiplexing is a downlink transmission technique that uses linear transmit precoding to multiplex multiple users and precancel interuser interference. In such a system, the spatial degrees of freedom are used for interference mitigation and generally come at the expense of diversity gain. This paper proposes two precoding methods that use extra transmit antennas, beyond the minimum required, to provide additional degrees of diversity. The approach taken is to solve for a unitary transmit precoder, under a zero interuser interference constraint, that minimizes an upper bound on the symbol error rate (SER) for each user. Solutions where all transmit antennas are employed, as well as subsets of antennas (to reduce analog components), are described. Numerical results confirm a dramatic improvement in terms of SER and mutual information over single-user multiple-input multiple-output (MIMO) methods and static allocation methods. For example, the proposed techniques achieve a signal-to-noise ratio (SNR) improvement of 6-10 dB at an uncoded SER of 10-3, with only one extra transmit antenna