MEMSYS 2023

Ever wondered why SRAM, DRAM, and Flash are the only three successful memory technologies today, despite several emerging technologies being hyped over decades with outrageous claims and promises? These new emerging technologies either failed to deliver, or overpromised, or misrepresented their benefits. In this talk we will first discuss salient memory attributes such as energy, performance, persistence, and endurance. Then describe how to measure these attributes for existing memory technologies, exposing some of the myths and hype. Next, we will establish a system level value metric for comparison and evaluate different memory technologies, comparing their attributes using the established metric, and conclude on what is required for a memory technology to succeed!

Shekhar Borkar is a Sr. Director of Technology at Qualcomm Inc. He started his career with Intel Corp, worked on the 8051 family of microcontrollers, supercomputers, and high performance & low power digital circuits research. He has authored over 100 peer reviewed publications in conferences and journals, over 60 invited papers and keynotes, five book chapters, and has more than 60 issued patents. His research interests are low power, high performance digital circuits and system level optimization.

Modern machine learning workloads are increasing in scale at a rapid rate. For example, GPT-3 has 175 billion parameters, requiring 570 GB of storage, and GPT-4 is even larger.   ML workloads are typically memory bandwidth constrained.  For example, a saturated Google TPU can support four HBM channels operating at full capacity.  The Cerebras Wafer Scale Engine is designed for ML workloads and supports 20 PBps of memory bandwidth.  The problem is not just in inference but also in training.  The cost of training is a serious issue especially for large models and can benefit from acceleration.  Compute is made more difficult due the irregular nature of training and Transformer workloads.  Processor near memory (PnM) paradigms with high density memory stacked on logic are especially attractive options.

We describe multiple memory alternatives to address these issues and resulting opporunities.  All are 3DIC technology enabled with aggressive use of high density Through Silicon Vias (TSVs) and two-sided hybrid bonding.  First, we modified the design of the Tezzaron 64 Gb diRAM memory to expose over 130 Tbps of peak memory bandwidth. Second, we revisit the DRAM stack using more conventional bank designs and aggressive use of 3DIC technologies. Finally, we explore a mix of non-volatile and SRAM to enable a stack that can be built with accessible foundry technologies.  In each instance the memory array(s) are matched with  a network layer and array of SIMD processors to create a high performance, high capacity power-efficient solution.   The resulting solution is very area efficient and has the best power efficiency out of programmable solutions. 

Paul D. Franzon is currently the Cirrus Logic Distinguished Professor and the Director of Graduate programs in the Department of Electrical and Computer Engineering at North Carolina State University.  He is also a site Director for the Center for Advanced Electronics through Machine Learning (CAEML).  He earned his Ph.D. from the University of Adelaide, Adelaide, Australia.  He has also worked at AT&T Bell Laboratories, DSTO Australia, Australia Telecom, Rambus, and four companies he cofounded, Communica, LightSpin Technologies, Polymer Braille Inc. and Indago Technologies.  His current interests include applying machine learning to EDA, building AI accelerators, RFID, advanced packaging, heterogeneous integration, 2.5D and 3DICs and secure chip design. He has lead several major efforts and published over 300 papers in these areas.  In 1993 he received an NSF Young Investigators Award, in 2001 was selected to join the NCSU Academy of Outstanding Teachers, in 2003, selected as a Distinguished Undergraduate Alumni Professor, received the Alcoa Research Award in 2005, the Board of Governors Teaching Award in 2014, and the Distinguished Graduate Alumni Professor in 2021.  He has been awarded faculty awards from Qualcomm, IBM, Synopsys, and Google.  He served with the Australian Army Reserve for 13 years as an Infantry Soldier and Officer.  He is a Fellow of the IEEE.