ECE Distinguished Seminar: Sarma Vrudhula
Friday, October 20, 2023 — 11 a.m.
Sarma Vrudhula
Professor
School of Computing and Augmented Intelligence
Arizona State University
"A Holistic Approach for Energy Efficient Computing"
ECE 530 | Zoom link
Abstract
Deep neural networks have become the dominant algorithm framework in machine learning due to their remarkable success in numerous applications. They are also some of the most computationally and energetically intensive algorithms that perform trillions of floating point multiply-accumulate operations on very large dimensional datasets, some involving tens of billions of parameters. The energy cost and the ensuing environmental damage caused by training and running DNN models using software on even the most highly optimized Von Neumann architectures is fast becoming unsustainable and has the potential to negate many of their societal benefits. For this not to happen, at least three to four orders of magnitude improvement in energy efficiency and performance are required.
This talk will cover a new approach to co-design of domain-specific hardware accelerators, that includes co-design of devices and circuits and of circuits and architecture. Co-design at device and circuit level involves new logic structures that combine CMOS devices with configurable non-volatile devices. The new logic structures are artificial neurons (AN) that are used to compute logic and arithmetic operations. ANs are designed as standard cells to make them compatible with existing ASIC design tools so that they can be automatically embedded during synthesis. This new approach to automatic ASIC design leads to substantial reductions in power and area of performance optimized CMOS ASICs, without sacrificing performance. The ANs are used in the design of a scalable neural processing element (NPE), that serves as a basic compute element which can be configured instantaneously to perform all basic operations common to neural networks. An NPE is at least an order of magnitude smaller than a corresponding MAC unit used in traditional NN accelerators. This area and power advantage is exploited in the design of an ASIC array processor, named TULIP, that is operated in SIMD mode to implement quantized neural networks. TULIP implemented in 40nm achieves 30-50X improvement in energy efficiency compared to the SoA equivalent ASIC design without any penalty in performance, area or accuracy. TULIP also permits run-time tradeoffs between accuracy and energy efficiency. The talk will next address how NPEs can be non-intrusively integrated with a DRAM to create a processor-in-memory (PIM) architecture, named CIDAN. The talk will cover several variations of CIDAN, from the simplest which has an NPE with just one AN, to larger capacity NPEs. Each of these is demonstrated to achieve substantial improvements in energy efficiency and performance for various vector operations in NNs and image processing. The range of applications of CIDAN has also been extended to graph neural networks, demonstrating three to four orders of magnitude improvement in energy efficiency and latency compared to CPU/GPU based implementations. The talk will conclude with a summary of ongoing work that aims to extend the scope of applications to Large Language Models.
Biography
Sarma Vrudhula is a professor in the School of Computing and Augmented Intelligence at Arizona State University, Tempe AZ, and the director of the NSF IUCRC Center of Intelligent, Distributed, Embedded Applications and Systems (IDEAS). His areas of research and teaching are centered around VLSI design and Computer-Aided Design and Design Automation for VLSI, focusing on low power design and energy management of circuits and systems. He has co-authored more than 250 papers in journals and conferences and holds 20 patents. In addition to research and teaching, he has led several research teams. In 1996 he established the NSF Center for Low Power Electronics (CLPE) at the University of Arizona, in collaboration with the EE Department of Arizona State University. CLPE was supported by the NSF, the State of Arizona, and many leading companies in the microelectronics industry. Through CLPE, he has facilitated the research of twenty faculty and over 40 graduate students across both campuses. He was director of the NSF I/UCRC Center for Embedded Systems at ASU from 2009-2022. He was made fellow of the IEEE in 2016 for “contributions to low power and energy-efficient design of digital circuits and systems” and elected as member of National Academy of Inventors in 2017. He received the B.Math (Honors) from the University of Waterloo, Ontario, Canada, and MS and Ph.D. degrees in electrical engineering from the University of Southern California. Prior to joining ASU, he was a professor in the ECE Department at the University of Arizona.