Back to Main Page
Autonomous Machine Computing: Mission

A myriad of autonomous machines such as drones, robots, and self-driving cars are on the cusp of becoming an integral part of our life. The continuous proliferation of autonomous machines, however, depends critically on an efficient and resileint computing substrate, driven by higher performance and safety requirements and the miniaturization of machine form factors.

My research into Autonomous Machine Computing (AMC) explores cross-layer computing systems, architecture, and hardware for autonomous machines, with the vision to advance their performance, efficiency, resilience, and trustworthy, and pave the way for deploying autonomous machines in mission-critical environments.

Book
Robotic Computing on FPGAs
Shaoshan Liu, Zishen Wan, Bo Yu, Yu Wang
Editor: Natalie Enright Jerger
Synthesis Lectures on Computer Architecture (Morgan & Claypool Publishers), pp.1-218, Jun 2021
Book

This book provides a thorough overview of the state-of-the-art FPGA-based robotic computing accelerator designs and summarizes their adopted optimized techniques. This book consists of ten chapters, delving into the details of how FPGAs have been utilized in robotic perception, localization, planning, and multi-robot collaboration tasks. In addition to individual robotic tasks, this book provides detailed descriptions of how FPGAs have been used in robotic products, including commercial autonomous vehicles and space exploration robots. Some key observations of this book has been published as a survey paper in IEEE Circuits and Systems Magazine, 2021.

Publications

(Research Topics: Reliability of AMC / Hardware Architecture of AMC / Benchmarking and Design Flow of AMC)

Reliability of Autonomous Machine Computing

The Vulnerability-Adaptive Protection Paradigm Toward Reliable Autonomous Machines
Zishen Wan*, Yiming Gan*, Bo Yu, Shaoshan Liu, Arijit Raychowdhury, Yuhao Zhu
Communications of the ACM (CACM), 2024
Paper (to appear) / Media

We characterize the inherent resilience of different compute kernels in autonomous vehicles and drones systems. We analyze the protection design landscape and propose the lightweight Vulnerable-Adaptive Protection (VAP) paradigm for resilient autonomous machines.

MulBERRY: Enabling Bit-Error Robustness for Energy-Efficient Multi-Agent Autonomous Systems
Zishen Wan, Nandhini Chandramoorthy, Karthik Swaminathan, Pin-Yu Chen, Kshitij Bhardwaj, Vijay Janapa Reddi, Arijit Raychowdhury
ACM Inter Conf on Architectural Support for Programming Languages and Operating Systems (ASPLOS), 2024
Best Poster Award, IBM IEEE AI Compute Symposium 2023
Paper (to appear) / Media

We propose MulBERRY, a multi-agent robust learning framework to enhance bit error robustness and energy efficiency for autonomous swarm systems.

Silent Data Corruption in Robot Operating System: A Case for End-to-End System-Level Fault Analysis Using Autonomous UAVs
Yu-Shun Hsiao*, Zishen Wan*, Tianyu Jia, Radhika Ghosal, Abdulrahman Mahmoud, Arijit Raychowdhury, David Brooks, Gu-Yeon Wei, Vijay Janapa Reddi
IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems (TCAD), 2023
Paper / Code

We introduce ROSFI, the first Robot Operating System (ROS) resilience analysis methodology, to assess the effect of silent data corruption (SDC) on safety-critical applications.

BERRY: Bit Error Robustness for Energy-Efficient Reinforcement Learning-Based Autonomous Systems
Zishen Wan, Nandhini Chandramoorthy, Karthik Swaminathan, Pin-Yu Chen, Vijay Janapa Reddi, Arijit Raychowdhury
ACM/IEEE Design Automation Conference (DAC), 2023
Paper / Slide / Poster

We propose BEERY, a robust learning framework to improve bit error robustness and energy efficiency for RL autonomous systems. BEERY enables robust low-voltage operation on UAVs, leading to high energy savings in both compute-level operation and system-level quality-of-flight.

MAVFI: An End-to-End Fault Analysis Framework with Anomaly Detection and Recovery for Micro Aerial Vehicles
Yu-Shun Hsiao*, Zishen Wan*, Tianyu Jia, Radhika Ghosal, Abdulrahman Mahmoud, Arijit Raychowdhury, David Brooks, Gu-Yeon Wei, Vijay Janapa Reddi
Design, Automation and Test in Europe Conference (DATE), 2023
Paper / Slide / Poster / Code

We build a ROS-based end-to-end fault analysis framework to understand the resilience of Micro Aerial Vehicles (MAVs) system, and propose two low overhead anomaly-based transient fault detection and recovery schemes.

Analyzing and Improving Resilience and Robustness of Autonomous Systems
Zishen Wan, Karthik Swaminathan, Pin-Yu Chen, Nandhini Chandramoorthy, Arijit Raychowdhury
IEEE/ACM International Conference on Computer-Aided Design (ICCAD), 2022
Paper / Media

We explore the various originations of fault sources across the computing stack of autonomous systems, and discuss the diverse fault impacts and fault mitigation techniques of different scales of autonomous systems.

FRL-FI: Transient Fault Analysis for Federated Reinforcement Learning-Based Navigation Systems
Zishen Wan, Aqeel Anwar, Abdulrahman Mahmoud, Tianyu Jia, Yu-Shun Hsiao, Vijay Janapa Reddi, Arijit Raychowdhury
Design, Automation and Test in Europe Conference (DATE), 2022
Paper / Slide

We characterize the hardware transient fault impact on federated reinforcement learning system, a swarm intelligence paradigm in autonomous machines. We further propose application-aware cost-effective fault detection and mitigation scheme to enable autonomy reliability.

Analyzing and Improving Fault Tolerance of Learning-Based Navigation System
Zishen Wan, Aqeel Anwar, Yu-Shun Hsiao, Tianyu Jia, Vijay Janapa Reddi, Arijit Raychowdhury
ACM/IEEE Design Automation Conference (DAC), 2021
Best Presentation Award as DAC Young Fellow
Paper / Slide / Video / Media

We evaluate the resilience of learning-based navigation systems to transient and permanent hardware faults. We further propose two efficient fault mitigation techniques for both RL training and inference.


Hardware Architecture of Autonomous Machine Computing

ORIANNA: An Accelerator Generation Framework for Optimization-Based Robotic Applications
Yuhui Hao, Yiming Gan, Bo Yu, Qiang Liu, Yinhe Han, Zishen Wan, Shaoshan Liu
ACM Inter Conf on Architectural Support for Programming Languages and Operating Systems (ASPLOS), 2024
Paper (to appear)

We propose ORIANNA, a framework leverageing a common abstraction factor graph to generate accelerators for diverse robotic applications (e.g., manipulators, vehicles, drones) containing multiple optimization-based algorithms (e.g., localization, planning).

A Heterogeneous RRAM In-Memory and SRAM Near-Memory SoC for Fused Frame and Event-Based Target Identification and Tracking
Ashwin Lele*, Muya Chang*, Samuel Spetalnick, Brian Crafton, Shota Konna, Zishen Wan, Ashwin Bhat, Win-San Khwa, Yu-der Chih, Meng-Fan Chang, Arijit Raychowdhury
IEEE Journal of Solid-State Circuits (JSSC), 2023
Paper

We present a heterogeneous programmable ARM Cortex-based SoC with power-efficient RRAM compute-in-memory for CNN and high-speed SRAM compute-near-memory for SNN for the modality-matched acceleration of the hybrid vision.

Real-Time Fully Unsupervised Domain Adaptation for Lane Detection in Autonomous Driving
Kshitij Bhardwaj, Zishen Wan, Arijit Raychowdhury, Ryan Goldhahn
Design, Automation and Test in Europe Conference (DATE), 2023
Paper

We propose a lightweight, fully unsupervised and real-time adaptation algorithm for safety-critical lane detection of autonomous driving, and demonstrate its state-of-the-art performance on Nvidia Jetson Orin.

A 73.53TOPS/W 14.74TOPS Heterogeneous RRAM In-Memory and SRAM Near-Memory SoC for Hybrid Frame and Event-Based Target Tracking
Muya Chang*, Ashwin Lele*, Samuel Spetalnick, Brian Crafton, Shota Konna, Zishen Wan, Ashwin Bhat, Win-San Khwa, Yu-der Chih, Meng-Fan Chang, Arijit Raychowdhury
IEEE International Solid-State Circuits Conference (ISSCC), 2023
Paper

We propose a fully-programmable heterogeneous ARM Cortex-based SoC with an in-memory low-power RRAM-based CNN and a near-memory high-speed SRAM-based SNN in a hybrid architecture, for high-speed target identification and tracking applications.

An Energy-Efficient and Runtime-Reconfigurable FPGA-Based Accelerator for Robotic Localization Systems
Qiang Liu*, Zishen Wan*, Bo Yu*, Weizhuang Liu, Shaoshan Liu, Arijit Raychowdhury
IEEE Custom Integrated Circuits Conference (CICC), 2022
Paper / Slide

We present an energy-efficient and runtime-reconfigurable FPGA-based accelerator for robotic localization tasks. We exploit SLAM-specific data locality, sparsity, reuse, and parallelism, and achieve >5x performance improvement over state-of-the-art.

Robotic Computing on FPGAs: Current Progress, Research Challenges, and Opportunities
Zishen Wan, Ashwin Lele, Bo Yu, Shaoshan Liu, Yu Wang, Vijay Janapa Reddi, Cong (Callie) Hao, Arijit Raychowdhury
IEEE International Conference on Artificial Intelligence Circuits and Systems (AICAS), 2022
Paper / Slide / Video

We present the cross-layer robotic computing stack, illustrate the current progress and key design techniques. We summarize and highlight the challenges, research opportunities, and roadmap for the next-generation FPGA-based robotic computing systems.

Circuit and System Technologies for Energy-Efficient Edge Robotics
Zishen Wan, Ashwin Lele, Arijit Raychowdhury
Asia and South Pacific Design Automation Conference (ASP-DAC), 2022 (Invited Paper)
Paper / Slide

We present a series of ultra-low-power accelerator and system designs on enabling the intelligence in edge robotic platforms, with an emphasis on mixed-signal circuit, neuro-inspired computing, benchmarking, software infrastructure, and algorithm-hardware co-design.

A Survey of FPGA-Based Robotic Computing
Zishen Wan*, Bo Yu*, Thomas Yuang Li, Jie Tang, Yuhao Zhu, Yu Wang, Arijit Raychowdhury, Shaoshan Liu
IEEE Circuits and Systems Magazine (CAS-M), 2021
Paper

We provide an overview of recent work on FPGA-based robotic accelerators. An analysis of software and hardware optimization techniques and main technical issues is presented, along with some commercial and space applications.

An Energy-Efficient Quad-Camera Visual System for Autonomous Machines on FPGA Platform
Zishen Wan*, Yuyang Zhang*, Arijit Raychowdhury, Bo Yu, Yanjun Zhang, Shaoshan Liu
IEEE International Conference on Artificial Intelligence Circuits and Systems (AICAS), 2021
Paper / Slide / Summary Video / Long Video

We present a real-time and energy-efficient ORB (Oriented-Fast and Rotated-BRIEF) based visual system on FPGAs. Compared to Nvidia TX1 and Intel i7 CPU, our FPGA-based implementation achieves 5.6x and 3.4x speedup, as well as 3.0x and 34.6x power reduction, respectively.

iELAS: An ELAS-Based Energy-Efficient Accelerator for Real-Time Stereo Matching on FPGA Platform
Tian Gao*, Zishen Wan*, Yuyang Zhang, Bo Yu, Yanjun Zhang, Shaoshan Liu, Arijit Raychowdhury
IEEE International Conference on Artificial Intelligence Circuits and Systems (AICAS), 2021
Paper

We present an energy-efficient hardware architecture for real-time ELAS (Efficient Large-scale Stereo) based stereo matching on FPGAs. Compared to SOTA SoC and Intel i7 CPU, our FPGA design achieves 3.32x and 38.4x speedup, and 1.13x and 27.1x power reduction, respectively.


Benchmarking and Design Flow of Autonomous Machine Computing

RobotPerf: An Open-Source, Vendor-Agnostic, Benchmarking Suite for Evaluating Robotics Computing System Performance
Victor Mayoral-Vilches, Jason Jabbour, Yu-Shun Hsiao, Zishen Wan, Alejandra Martinez-Farina, Martino Crespo-Alvarez, Matthew Stewart, Juan Manuel Reina-Munoz, Prateek Nagras, Gaurav Vikhe, Mohammad Bakhshalipour, Martin Pinzger, Stefan Rass, Smruti Panigrahi, Giulio Corradi, Niladri Roy, Phillip B. Gibbons, Sabrina M. Neuman, Brian Plancher, Vijay Janapa Reddi
IEEE International Conference on Robotics and Automation (ICRA), 2024
Best Paper Award, IROS Robotics Benchmarking Workshop 2023
Paper / Poster / Code / Project Page / Media

We introduce RobotPerf, a benchmarking suite to evaluate robotics computing performance across a diverse range of hardware platforms. As an open-source initiative, RobotPerf remains committed to evolving with community input to advance the future of hardware-accelerated robotics.

Intelligence in Robotic Computing: Agile Design Flows for Building Efficient and Resilient Autonomous Machines
Zishen Wan, Vijay Janapa Reddi, Arijit Raychowdhury
ACM Student Research Competition (SRC), Grand Final, 2023
First Place, ACM/SIGBED Student Research Competition (SRC)
Paper / Media / Media

This report summarizes our recent efforts in facilitating the development of scalable, efficient, adaptive, and reliable autonomous machine computing, including automatic domain-specific SoC exploration, software-hardware co-design, and performance-efficiency-resilience co-optimization.

Automatic Domain-Specific SoC Design for Autonomous Unmanned Aerial Vehicles
Srivatsan Krishnan, Zishen Wan, Kshitij Bhardwaj, Paul Whatmough, Aleksandra Faust, Sabrina M. Neuman, Gu-Yeon Wei, David Brooks, Vijay Janapa Reddi
IEEE/ACM International Symposium on Microarchitecture (MICRO), 2022
IEEE Micro Top Picks 2023 Honorable Mention
Paper / arXiv

We propose a machine learning-based design space exploration framework, Autopilot, that can automate the full system cyber-physical co-design for aerial robots. AutoPilot consistently outperforms general-purpose processors and specialized accelerators built for drones.

Roofline Model for UAVs: A Bottleneck Analysis Tool for Onboard Compute Characterization of Autonomous Unmanned Aerial Vehicles
Srivatsan Krishnan, Zishen Wan, Kshitij Bhardwaj, Ninad Jadhav, Aleksandra Faust, Vijay Janapa Reddi
IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS), 2022
Paper / Skyline Tool

We present a bottleneck analysis tool, Skyline, for designing compute systems for autonomous Unmanned Aerial Vehicles (UAV). The tool provides insights by exploiting the fundamental relationships between various components in the autonomous UAV such as sensor, compute, body dynamics.

AutoSoC: Automating Algorithm-SoC Co-design for Aerial Robots
Srivatsan Krishnan, Thierry Tambe, Zishen Wan, Vijay Janapa Reddi
Pre-print, 2021
Paper / Poster

We present AutoSoC, a algorithms-hardware co-design framework for end-to-end learning-based aerial autonomous machines. AutoSoC runs the ASIC flow of place and route and generates layouts of the floor-planed accelerators with varying performance, area, and power consumption.

The Sky Is Not the Limit: A Visual Performance Model for Cyber-Physical Co-Design in Autonomous Machines
Srivatsan Krishnan, Zishen Wan, Kshitij Bhardwaj, Paul Whatmough, Aleksandra Faust, Gu-Yeon Wei, David Brooks, Vijay Janapa Reddi
IEEE Computer Architecture Letters (CAL), 2020
Best Paper Award
Presented at International Symposium on High-Performance Computer Architecture (HPCA), 2021
Paper / Tool Website

We introduce a roofline-like model to understand the role of computing in aerial autonomous machines. The model provides insights by exploiting the fundamental relationships between various components in an aerial robot, such as sensor framerate, compute performance, and body dynamics.

Acknowledgements

Srivatsan Krishnan, Yiming Gan, Yu-Shun Hsiao, Jason Jabbour, Ashwin Lele, Dr. Shaoshan Liu, Dr. Bo Yu, Dr. Karthik Swaminathan, Dr. Nandhini Chandramoorthy, Dr. Pin-Yu Chen, Dr. Kshitij Bhardwaj, Dr. Aqeel Anwar, Dr. Abdulrahman Mahmoud, Dr. Paul Whatmough, Dr. Aleksandra Faust, Dr. Muya Chang, Prof. Tianyu Jia, Prof. Sabrina M. Neuman, Prof. Brian Plancher, Prof. Yuhao Zhu, Prof. David Brooks, Prof. Gu-Yeon Wei, Prof. Tushar Krishna, Prof. Vijay Janapa Reddi, Prof. Arijit Raychowdhury, and many more:)

Last Update: Feb. 2024

Back to Main Page