3D-PLANE: A 3D-stacked DRAM-based Programmable SLM Accelerator Capable of Near-Memory and Energy-Efficient Parallel Processing

Small Language Model (SLM) processing is characterized by extensive matrix multiplication workloads, which are memory-bound and data-intensive in nature, leading to a significant amount of data movement overhead and consequent energy inefficiency in traditional processors such as GPUs and TPUs. To minimize these overheads, we propose 3D-PLANE, a novel memory-centric accelerator for SLM processing that incorporates programmable processing logic within the memory to alleviate the data movement overheads. Our proposed architecture is integrated on a 3D-stacked DRAM packaging where each stacked DRAM chip is enhanced with programmable computing logic to facilitate a competitive compute-bandwidth for fast processing of SLMs. The area-efficient programmable processing elements, accompanied by workload-aware dynamic power gating, near-memory computation, and adaptive dataflow scheduling, enable us to minimize energy consumption without compromising performance. We evaluate the effectiveness of 3D-PLANE through hardware simulations and system-level analytical modeling across multiple configurations, using decoder-only variants of SLMs such as Phi-3 Mini and TinyLLaMA.