› › Mixed Platoon Control Strategy Of Connected And Automated Vehicles Based On Physics Informed Deep Reinforcement Learning

Mixed Platoon Control Strategy of Connected and Automated Vehicles Based on Physics-informed Deep Reinforcement Learning

Author : Haotian Shi (Ph.D.)

Publisher :

Page : 0 pages

File Size : 30,80 MB

Release : 2023

Category :

ISBN :

Get eBook

Book Description

This dissertation presents a distributed platoon control strategy of connected and automated vehicles (CAVs) based on physics-informed Deep Reinforcement Learning (DRL) for mixed traffic of CAVs and human-driven vehicles (HDVs). The dissertation will mainly consist of three parts: (i) generic DRL-based CAV control framework for the mixed traffic flow; (ii) DRL-based CAV distributed control under communication failure for the fully connected automated environment; (iii) distributed CAVs control for the mixed traffic flow, under real-time aggregated macroscopic car-following behavior estimation based on DRL. For the first part, we first discussed the current challenges for CAV control in mixed traffic flow. For distributed CAV control, we categorize the local downstream environment into two broad traffic scenarios based on the composition of CAVs and HDVs to accommodate any possible CAV-HDV platoon configuration: (i) a fully connected automated environment, where all local downstream vehicles are CAVs, forming a CAV-CAVs topology; and (ii) a mixed local downstream environment, comprising the closest downstream CAV followed by one or more HDVs, creating a CAV-HDVs-CAV topology. This generic control framework effectively accommodates any CAV-HDV platoon topology that may emerge within the mixed traffic platoon. This part is discussed in Section 3. For the second part, this study proposes a deep reinforcement learning (DRL) based distributed longitudinal control strategy for connected and automated vehicles (CAVs) under communication failure to stabilize traffic oscillations. Specifically, the Signal-Interference-plus-Noise Ratio (SINR) based vehicle-to-vehicle (V2V) communication is incorporated into the DRL training environment to reproduce the realistic communication and time-space varying information flow topologies (IFTs). A dynamic information fusion mechanism is designed to smooth the high-jerk control signal caused by the dynamic IFTs. Based on that, each CAV controlled by the DRL-based agent was developed to receive the real-time downstream CAVs' state information and take longitudinal actions to achieve the equilibrium consensus in the multi-agent system. Simulated experiments are conducted to tune the communication adjustment mechanism and further validate the control performance, oscillation dampening performance and generalization capability of our proposed algorithm. This part is discussed in Section 4. The third part proposes an innovative distributed longitudinal control strategy for connected automated vehicles (CAVs) in the mixed traffic environment of CAV and human-driven vehicles (HDVs), incorporating high-dimensional platoon information. For mixed traffic, the traditional CAV control method focuses on microscopic trajectory information, which may not be efficient in handling the HDV stochasticity (e.g., long reaction time; various driving styles) and mixed traffic heterogeneities. Different from traditional methods, our method, for the first time, characterizes consecutive HDVs as a whole (i.e., AHDV) to reduce the HDV stochasticity and utilize its macroscopic features to control the following CAVs. The new control strategy takes advantage of platoon information to anticipate the disturbances and traffic features induced downstream under mixed traffic scenarios and greatly outperforms the traditional methods. In particular, the control algorithm is based on deep reinforcement learning (DRL) to fulfill car-following control efficiency and further address the stochasticity for the aggregated car following behavior by embedding it in the training environment. To better utilize the macroscopic traffic features, a general platoon of mixed traffic is categorized as a CAV-HDVs-CAV pattern and described by corresponding DRL states. The macroscopic traffic flow properties are built upon the Newell car-following model to capture the characteristics of aggregated HDVs' joint behaviors. Simulated experiments are conducted to validate our proposed strategy. The results demonstrate that the proposed control method has outstanding performances in terms of oscillation dampening, eco-driving, and generalization capability. This part is discussed in Section 5.



Predictive Control Strategy for Automated Driving Systems Under Mixed Traffic Lane Change Conditions

Author : Kunsong Shi

Publisher :

Page : 0 pages

File Size : 37,55 MB

Release : 2023

Category :

ISBN :

Get eBook

Book Description

With the recent development of technologies, automated vehicles and connectedautomated vehicles (CAVs) have been researched and developed. However, mass deployment of fully automated vehicles is very difficult to achieve in the near future because of the high cost of high level autonomous vehicles. Automated driving system (ADS) like the Connected and automated vehicle highway (CAVH) system that can utilize roadside infrastructure is one of the best approaches for large scale deployment for CAVs because the system can reduce the workload and cost of a single vehicle. However, mass deployment of ADS will still take some time. Therefore, in the near future, mixed traffic conditions containing CAVs and human driven vehicles will be the predominant condition. Safe and efficient control for autonomous vehicles under mixed is still a very challenging task for the automated driving system. In this research, we present a predictive control strategy for automated driving systems under mixed traffic lane change conditions. To achieve this goal, we first proposed a deep learning based lane change prediction module that considers a new lane change prediction scenario that is more realistic by considering more surrounding vehicles. Then we developed a deep learning based integrated two dimensional vehicle trajectory prediction module. This integrated model can predict combined behaviors of car-following and lane change. Then we created a predictive deep reinforcement learning based CAV controller that can utilize the predicted information to generate safe and efficient longitudinal control for CAVs under mixed traffic lane change conditions. Several experiments are conducted using the trajectory data Next Generation Simulation (NGSIM) dataset to evaluate the effectiveness of the proposed modules. The experiment result shows that our lane change prediction module can accurately predict human lane change behavior under the defined lane change condition. Moreover, the experiment result demonstrates that the proposed integrated two dimensional trajectory prediction model can accurately predict both lane change trajectories and car-following trajectories. In addition, experiments for the deep reinforcement learning-based CAV controller showed that the proposed controller can improve traffic safety and efficiency of CAVs under mixed traffic lane change conditions.



Reinforcement Learning in Eco-driving for Connected and Automated Vehicles

Author : Zhaoxuan Zhu

Publisher :

Page : 0 pages

File Size : 21,67 MB

Release : 2021

Category : Automated vehicles

ISBN :

Get eBook

Book Description

Connected and Automated Vehicles (CAVs) can significantly improve transportation efficiency by taking advantage of advanced connectivity technologies. Meanwhile, the combination of CAVs and powertrain electrification, such as Hybrid Electric Vehicles (HEVs) and Plug-in Hybrid Electric Vehicles (PHEVs), offers greater potential to improve fuel economy due to the extra control flexibility compared to vehicles with a single power source. In this context, the eco-driving control optimization problem seeks to design the optimal speed and powertrain components usage profiles based upon the information received by advanced mapping or Vehicle-to-Everything (V2X) communications to minimize the energy consumed by the vehicle over a given itinerary. To overcome the real-time computational complexity and embrace the stochastic nature of the driving task, the application and extension of state-of-the-art (SOTA) Deep Reinforcement Learning (Deep RL, DRL) algorithms to the eco-driving problem for a mild-HEV is studied in this dissertation. For better training and a more comprehensive evaluation, an RL environment, consisting of a mild HEV powertrain and vehicle dynamics model and a large-scale microscopic traffic simulator, is developed. To benchmark the performance of the developed strategies, two causal controllers, namely a baseline strategy representing human drivers and a deterministic optimal-control-based strategy, and the non-causal wait-and-see solution are implemented. In the first RL application, the eco-driving problem is formulated as a Partially Observable Markov Decision Process, and a SOTA model-free DRL (MFDRL) algorithm, Proximal Policy Optimization with Long Short-term Memory as function approximator, is used. Evaluated over 100 trips randomly generated in the city of Columbus, OH, the MFDRL agent shows a 17% fuel economy improvement against the baseline strategy while keeping the average travel time comparable. While showing performance comparable to the optimal-control-based strategy, the actor of the MFDRL agent offers an explicit control policy that significantly reduces the onboard computation. Subsequently, a model-based DRL (MBDRL) algorithm, Safe Model-based Off-policy Reinforcement Learning (SMORL) is proposed. The algorithm addresses the following issues emerged from the MFDRL development: a) the cumbersome process necessary to design the rewarding mechanism, b) the lack of the constraint satisfaction and feasibility guarantee and c) the low sample efficiency. Specifically, SMORL consists of three key components, a massively parallelizable dynamic programming trajectory optimizer, a value function learned in an off-policy fashion and a learned safe set as a generative model. Evaluated under the same conditions, the SMORL agent shows a 21% reduction on the fuel consumption over the baseline and the dominant performance over the MFDRL agent and the deterministic optimal-control-based controller.



Deep Long Short-term Memory Network Embedded Connected Automated Car-following Model Predictive Control Strategy

Author : Zhen Zhang

Publisher :

Page : 0 pages

File Size : 41,51 MB

Release : 2019

Category :

ISBN :

Get eBook

Book Description

Recent years, autonomous vehicle (AV) technology, which is expected to solve critical issues, such as traffic efficiency, capacity, and safety, has been put a lot of efforts and making considerable progress. It utilizes data from various sensors for sensing, prediction, and control tasks. Another related technology that also has significant impacts on transportation is connected vehicle (CV). With the assistance of dedicated short-range communication devices, CV communicates with other vehicles in the system or roadside infrastructure to get valuable information about surroundings. Combining these technologies together, connected and automated vehicle (CAV) can further enhance the AV benefits in various ways, such as safety and efficiency. Towards to fully automation, one of most important areas is the advanced driver-assistance systems, especially the longitudinal control. Since the manual vehicles will still dominate the road for a long time, how to perform the longitudinal control for a CAV is a critical problem to be solved for mixed traffic consisting of CAVs and manual vehicles. Model Predictive Control (MPC) is a modern control framework that has been extensively studied across various fields. There is also plenty of research applying MPC to control the vehicle in full CAV environments. However, due to the lack of communication with the preceding manual vehicle, CAV is not able to attain the planning of the leading vehicle's control actions, which is critically needed by MPC controller. The emerging deep learning techniques have demonstrated promising capability in various domains, including traffic prediction. This research focuses on developing a novel car-following control strategy for a platoon of CAVs and manual vehicles. Specifically, it controls those CAVs following another manual vehicle in this platoon and enhance the stability. The proposed longitudinal control strategy is designed in MPC manner, embedded with deep-learning enhanced prediction. This dissertation first conducts a comprehensive review on car-following models and MPC theories and applications on vehicle control. Then a novel control strategy is developed to enhance the efficiency and stability of controlling CAVs in mixed traffic. There are two major parts in this strategy. One is trajectory prediction model, and the other is MPC controller. Two different deep long-short-term-memory (LSTM) based models are designed and evaluated for two potential control scenarios, taking advantages of new deep learning technology. Embedded with deep learning models, MPC controller is formulated with consideration of safety, efficiency, and driving comfort. Several experiments are carried out to analyze the performance of trajectory prediction models and proposed control strategy and results show promising potential.



Vehicular Platoon System Design

Author : Hui Zhang

Publisher : Elsevier

Page : 316 pages

File Size : 18,89 MB

Release : 2024-08-23

Category : Technology & Engineering

ISBN : 0443298580

Get eBook

Book Description

Vehicular Platoon System Design: Fundamentals and Robustness provides a comprehensive introduction to connected and automated vehicular platoon system design. Platoons decrease the distances between cars or trucks using electronic, and possibly mechanical, coupling. This capability allows many cars or trucks to accelerate or brake simultaneously. It also allows for a closer headway between vehicles by eliminating reacting distance needed for human reaction. The book considers the key issues of robustness and cybersecurity, with optimization-based model predictive control schemes applied to control vehicle platoon. In the controller design part, several practical problems, such as constraint handling, optimal control performance, robustness against disturbance, and resilience against cyberattacks are reviewed. In addition, the book provides detailed theoretical analysis of the stability of the platoon under different control schemes. Provides a comprehensive introduction to the state-of-the-art development of connected and automated vehicular platoon systems Covers the advanced, robust and stochastic model predictive control algorithm design methods for constraint handling and robustness improvement Introduces rigorous theoretical stability analysis from the robust tube-based distributedMPC (Model Predictive Control) and stochastic tube-based distributed MPC perspectives Offers various filter-based inter-vehicle attack detection methods and event-based resilient vehicle platoon control design methods



Deep Learning for Autonomous Vehicle Control

Author : Sampo Kuutti

Publisher : Springer Nature

Page : 70 pages

File Size : 41,76 MB

Release : 2022-06-01

Category : Technology & Engineering

ISBN : 3031015029

Get eBook

Book Description

The next generation of autonomous vehicles will provide major improvements in traffic flow, fuel efficiency, and vehicle safety. Several challenges currently prevent the deployment of autonomous vehicles, one aspect of which is robust and adaptable vehicle control. Designing a controller for autonomous vehicles capable of providing adequate performance in all driving scenarios is challenging due to the highly complex environment and inability to test the system in the wide variety of scenarios which it may encounter after deployment. However, deep learning methods have shown great promise in not only providing excellent performance for complex and non-linear control problems, but also in generalizing previously learned rules to new scenarios. For these reasons, the use of deep neural networks for vehicle control has gained significant interest. In this book, we introduce relevant deep learning techniques, discuss recent algorithms applied to autonomous vehicle control, identify strengths and limitations of available methods, discuss research challenges in the field, and provide insights into the future trends in this rapidly evolving field.



Networked Control Systems for Connected and Automated Vehicles

Author : Alexander Guda

Publisher : Springer Nature

Page : 2004 pages

File Size : 16,21 MB

Release : 2022-11-15

Category : Technology & Engineering

ISBN : 303111051X

Get eBook

Book Description

Control of large-scale distributed energy systems over communication networks is an important topic with many application domains. The book presents novel concepts of distributed control for networked and cyber-physical systems (CPS), such as smart industrial production lines, smart energy grids, and autonomous vehicular systems. It focuses on new solutions in managing data and connectivity to support connected and automated vehicles (CAV). The book compiles original research papers presented at the conference “Networked Control Systems for Connected and Automated Vehicles” (Russia). The latest connected and automated vehicle technologies for next generation autonomous vehicles are presented. The book sets new goals for the standardization of the scientific results obtained and the advancement to the level of full autonomy and full self-driving (FSD). The book presents the latest research in artificial intelligence, assessing virtual environments, deep learning systems, and sensor fusion for automated vehicles. Particular attention is paid to new safety standards, safety and security systems, and control of epidemic spreading over networks. The issues of building modern transport infrastructure facilities are also discussed in the articles presented in this book. The book is of considerable interest to scientists, researchers, and graduate students in the field of transport systems, as well as for managers and employees of companies using or producing equipment for these systems.



Distributed Robust Connected Automated Car Following Strategy to Stabilize Mixed Traffic

Author : Yang Zhou (Ph.D.)

Publisher :

Page : 98 pages

File Size : 25,46 MB

Release : 2019

Category :

ISBN :

Get eBook

Book Description

This paper presents a robust CAV longitudinal control strategy to stabilize mixed platoons considering the uncertainties for vehicular dynamics and communication delay. This mathematically guarantee of the robust stability based on a distributed linear controller for pure CAV sub-platoons. Based on that, we formulated an H-infinity control problem that aims to find the control parameters (feedback and feedforward gains) that essentially minimize the H-infinity norm of the transfer function (the maximum disturbance propagation ratio in the frequency domain) within the predominant acceleration frequency range based on extracted human-driven car following characteristics and vehicular dynamics and communication delay uncertainties boundaries. The proposed control was tested via simulation experiments utilizing the field-tested parameters and the NGSIM data. To show the effectiveness of the dampening disturbance, two uncertainties scenarios were tested, and from experimental results, the proposed method is capable of handling uncertainties and dampen the disturbance effectively.



A Simulation Framework for Exploring the Impacts of Vehicle Platoons on Mixed Traffic Under Connected and Autonomous Environment

Author : Dian Yuan

Publisher :

Page : 0 pages

File Size : 39,20 MB

Release : 2023

Category :

ISBN :

Get eBook

Book Description

Vehicle platooning, first studied as an application of Intelligent Transportation Systems (ITS), is increasingly gaining attention in recent years as autonomous driving and connected vehicle technologies advance. When being platooned, vehicles communicate within the platoon and operate with coordination to maintain a relatively steady state status with each other and with the outside. The major goal of this study is to build a conceptual simulation framework to help with exploring the impacts of connected and autonomous vehicle platoons on the existing traffic. The first part of this work effort is reviewing autonomous and connected vehicle technologies for depicting the functional structure of a platooning-ready connected and autonomous vehicle (CAV) platform. Then models and simulation tools are reviewed to break down the simulation framework into two levels - vehicle level and traffic level. The vehicle-level model provides in-depth modeling of CAVs and platooning modules. The traffic-level simulator provides the simulation of the existing traffic with the built CAV platoons. The simulation framework has been developed by integration and usage of GIS, MATLAB/Simulink, SUMO, and OMNeT++. GIS tools are used to gather the necessary traffic data. MATLAB/Simulink serves as the platform for vehicle-level modeling and simulation. SUMO and OMNeT++ are used to build the traffic and communication simulations, respectively. The completed model was used to conduct two case studies based on a section of the US Interstate Highway in order to explore the impacts of CAV platoons on existing traffic. The results indicate that, with the existing traffic pattern and infrastructure design, traffic can be improved after the introduction of CAV platoons, even after taking into consideration the rate of traffic growth. Moreover, deploying dedicated lanes and separating CAV platoon traffic from the non-platooning traffic can benefit the traffic using such output as the travel speed/time and delay measures. However, using such new traffic patterns and infrastructure designs is not recommended for a low percentage of CAV platoon traffic.



Novel Off-board Decision-making Strategy for Connected and Autonomous Vehicles

Author : Zine El Abidine Kherroubi

Publisher :

Page : 0 pages

File Size : 20,61 MB

Release : 2020

Category :

ISBN :

Get eBook

Book Description

Merging in the highway on-ramp is a significant challenge toward realizing fully automated driving (level 4 of autonomous driving). The combination of communication technology and autonomous driving technology, which underpins the notion of Connected Autonomous Vehicles (CAVs), may improve greatly safety performances when performing highway on-ramp merging. However, even with the emergence of CAVs vehicles, some keys constraints should be considered to achieve a safe on-ramp merging. First, human-driven vehicles will still be present on the road, and it may take decades before all the commercialized vehicles will be fully autonomous and connected. Also, on-board vehicle sensors may provide inaccurate or incomplete data due to sensors limitations and blind spots, especially in such critical situations. To resolve these issues, the present thesis introduces a novel solution that uses an off-board Road-Side Unit (RSU) to realize fully automated highway on-ramp merging for connected and automated vehicles. Our proposed approach is based on an Artificial Neural Network (ANN) to predict drivers' intentions. This prediction is used as an input state to a Deep Reinforcement Learning (DRL) agent that outputs the longitudinal acceleration for the merging vehicle. To achieve this, we first show how the road-side unit may be used to enhance perception in the on-ramp zone. We then propose a driver intention model that can predict the behavior of the human-driven vehicles in the main highway lane, with 99% accuracy. We use the output of this model as an input state to train a Twin Delayed Deep Deterministic Policy Gradients (TD3) agent that learns « safe » and « cooperative » driving policy to perform highway on-ramp merging. We show that our proposed decision-making strategy improves performance compared to the solutions proposed previously.