CSR: 3D-PFLOPS: 3D Integrated Parallel Fabrics enabling Layered Opto-electronic Processors for Near-memory AI Computing

Artificial Intelligence (AI) is becoming an integral part of our society and is being widely used across sectors such as technology, agriculture, manufacturing, healthcare, and finance. Supporting this growth also demands enormous amounts of electricity and expensive infrastructure. However, a significant portion of the energy cost and overhead of AI computing stems from the bottleneck in moving large amounts of data between the processing and memory chips within the AI hardware. This project aims to address this limitation by designing transformative AI hardware that assembles light-based (photonic) and electronic computing chips closer to memory chips via vertical (3D) assembly, significantly reducing data transfer costs for AI computing. Additionally, by assigning different AI computing phases (i.e., training and inference) to the chip best suited for that task, the proposed project will be able to optimize both the computational and data communication costs of AI computing compared to existing technologies. This project creates an architectural simulation framework for phase-aware heterogeneous processors that include electronic and photonic processing units, integrated near a shared memory. The research enhances core principles of computer architecture related to data transfer, arithmetic operations, and thermal constraints through three specific objectives: (1) co-design an electro-optical application-specific integrated circuit, (2) develop a thermal-aware high-bandwidth near-memory 3D integration strategy, and (3) create a topology-aware, localized data orchestration and task-scheduling framework that enables efficient execution across heterogeneous compute fabrics. This research aims to develop a cutting-edge electro-photonic near-memory computing framework that could revolutionize the AI computing industry and research. An open-source architectural simulator created in this project will be accessible to all Americans, promoting sustainable, transparent, and reproducible investigations of energy and performance trade-offs in AI systems and electronic-photonic integration. These efforts will bolster the AI and semiconductor workforce and foster broader participation across various institutional levels. At Boise State University, the findings will enhance courses on edge artificial intelligence hardware and memory-centered computer architecture, reaching approximately sixty to eighty undergraduate and graduate students annually through an interdisciplinary teaching approach and industry collaborations. Additionally, in partnership with the Micron Student Success Center, interactive demonstrations will introduce K-12 students in the Greater Boise Metropolitan area to core AI and hardware concepts, building educational infrastructure and providing timely pathways into the semiconductor industry. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.