Learning Augmented Model-Based Optimization Controls (2022-2025)

Problem statement: Off-road autonomy missions face many unknown terrains and dynamics where nominal models for mobility controls do not suffice. Can we utilize reinforcement learning to enable fast adaptation of MPC and life-long learning for performant vehicle controls?

Approach: Vision informed residual learning

• Reducing data dependency by leveraging nominal models and increasing generalization of the controller across different driving conditions
• Formulating and investigating hybrid reinforcement learning control architecture to handle modeling mismatches and unmodeled system dynamics
  
  
• The results show that learning augmented contols not only outperform model based (MPC) an purely learnt (AC) controllers across prevviously unseen scenarios, but also improve smootheness of control.
  
  
• Using vision foundation models to inform offline and online life-long learning for adaptation to tasks and terrains
• Validating in simulation and on a drive-by-wire Polaris RZR vehicle on off-road terrains
  

Advisor: Dr. Yunyi Jia, Associate Professor, Clemson University

Funding: Department of Defense through VIPR-GS

Energy Efficient Cooperative Autonomous Driving (2022-2024)

Problem statement: How can we improve energy efficiency for urban driving corridors using V2X?

Approach: Constrianed optimal control.

• Research aimed to improve energy efficiency through V2V, V2I connectivity by informing lane switching and acceleration strategy on connected traffic corridors
• Experimental results on an in-house drive-by-wire Mazda CX7 showed up to 36% improvements
• Expert skills for on-vehicle control deployment using real-time solvable MPC with efficient vehicle model and other state-of-the-art controllers
  
  
• Collaborators: This work was led by many researchers including Jihun Han, Tyler Ard, Tony Wang

Advisors: Dr. Yunyi Jia, Dr. Ardalan Vahidi, Clemson University

Funding: Department of Energy through Argonne National Laboratory

Physics Conditioned GAN for Video Frame Prediction

Problem statement: How can physics constraints improve generative predicitons for urban autonomous driving?

Approach: Physics conditioned GAN.

We build upon the ideas from Retrospective Cycle GAN (Kwon et al). They established great performance compared to the SOTA with their forward and backward temporal consistency idea for training the generator. However, they do not consider any conditioning on physics or restrict the movement of pixels expicitly. We ask the following quesiton: "Can we improve blurring in longer term predictions through the use of physics constraints?"

Future Video Frame Prediciton performance with conditional Generative Adverserial Network (GAN) is shown below. More details can be found on the project webpage

Tools used: PyTorch, sklearn etc.

Deep Orange 13 (2021-2022)

• Utilized systems engineering principles to arrive at engineering requirements for mission critical needs.
• Implemented fully autonomous navigation for off-road driving using cameras, lidars, GNSS over ROS (Robot Operating Software) on small and full-scale platforms
• Designed and developed in-house Drive-by-Wire vehicle controls architecture using New Eagle Raptor controller and ROS with C++ and Python
• Model based control and logic development in MATLAB/Simulink for embedded deployment on-board
• H-i-L and S-i-L testing for vehicle controls
• Communications and vehicle networking using CAN, Ethernet, wireless short-range communications
• Autonomous Driving:
• Manned Autonomous:

Advisor: Dr. Chris Paredis, Professor Emeritus, Clemson University

Funding: Department of Defense through VIPR-GS

Collaborative Monitoring of Road Conditions (2021-2022)

• Research aims to utilize deep learning and cloud architecture to augment road condition monitoring
• Object detection neural networks were trained to identify road defects that need attention using just smartphone sensors like IMU and camera.
  Best Poster Award
• Research poster:
  

Advisor: Dr. Yunyi Jia, Clemson University

Funding: SC Department of Transportation

Hexapod Inverse Kinematics (2021)

Advisor: Dr. Venkat Krovi, Professor, Clemson University

• Implemented a planar cable robot and designed a controller using Quanser’s Hexapod hardware in HiL simulation
• Formulated inverse and forward kinematics for the cable robot