Location
Hilton Hawaiian Village, Honolulu, Hawaii
Event Website
https://hicss.hawaii.edu/
Start Date
3-1-2024 12:00 AM
End Date
6-1-2024 12:00 AM
Description
This paper proposes a robust Multi-Agent Reinforcement Learning (MARL) approach to optimize the charge schedule and price offered by EV charging stations competing to maximize profits, i.e. the differences between the payments collected by the charging stations and the electricity price set from a distribution system operator. It is assumed that, to prevent energy congestion on the distribution grid, each charging station pays the locational marginal price (LMP) of electricity to serve its customer, determined to be the dual variable of the optimal power flow (OPF) problem. Our proposed RL algorithm trains multiple agents to make optimal charging and pricing decisions at each time step, based solely on past event observations. Additionally, the algorithm takes into account the randomness caused by user behavior, such as travel and wait times, and user flexibility. We observe that, when they are profit maximizing, competing agents vie for higher profits. This intense competition can often lead agents to adopt inefficient policies, mainly due to the disruptions caused by the actions of their competitors. To address this issue, we incorporate constant-sum game theory in the RL policy training. This approach utilizes the minimax policy gradient to maximize the reward of a robust agent, while considering the worst-case scenarios created by competing agents. Simulation results validate that robust agents are capable of generating greater profits than competing agents that do not undergo minimax training and that their presence stabilizes the training.
Recommended Citation
Surani, Adrian-Petru; Wu, Tong; and Scaglione, Anna, "Competitive Reinforcement Learning for Real-Time Pricing and Scheduling Control in Coupled EV Charging Stations and Power Networks" (2024). Hawaii International Conference on System Sciences 2024 (HICSS-57). 9.
https://aisel.aisnet.org/hicss-57/es/monitoring/9
Competitive Reinforcement Learning for Real-Time Pricing and Scheduling Control in Coupled EV Charging Stations and Power Networks
Hilton Hawaiian Village, Honolulu, Hawaii
This paper proposes a robust Multi-Agent Reinforcement Learning (MARL) approach to optimize the charge schedule and price offered by EV charging stations competing to maximize profits, i.e. the differences between the payments collected by the charging stations and the electricity price set from a distribution system operator. It is assumed that, to prevent energy congestion on the distribution grid, each charging station pays the locational marginal price (LMP) of electricity to serve its customer, determined to be the dual variable of the optimal power flow (OPF) problem. Our proposed RL algorithm trains multiple agents to make optimal charging and pricing decisions at each time step, based solely on past event observations. Additionally, the algorithm takes into account the randomness caused by user behavior, such as travel and wait times, and user flexibility. We observe that, when they are profit maximizing, competing agents vie for higher profits. This intense competition can often lead agents to adopt inefficient policies, mainly due to the disruptions caused by the actions of their competitors. To address this issue, we incorporate constant-sum game theory in the RL policy training. This approach utilizes the minimax policy gradient to maximize the reward of a robust agent, while considering the worst-case scenarios created by competing agents. Simulation results validate that robust agents are capable of generating greater profits than competing agents that do not undergo minimax training and that their presence stabilizes the training.
https://aisel.aisnet.org/hicss-57/es/monitoring/9