M3D-MDA: New scratchpad memory for enhancing GPU performance and energy efficiency

Applications in various fields, such as deep learning and scientific computing, naturally exhibit data access patterns along both the row and column dimensions of static random access memory (SRAM). However, traditional SRAM architectures only support asymmetric access, typically in the row dimension. Accordingly, to perform operations that require both row-wise and column-wise data, SRAM must activate multiple rows sequentially to obtain column-wise data. Such time-consuming operations significantly degrade not only the performance but also the energy efficiency of graphics processing units (GPUs). In this study, we exploited monolithic 3D (M3D) integration to construct a large-scale SRAM architecture supporting accessing data in both the row and column dimensions (that is, multi-dimensional access [MDA]). When our M3D-MDA memory is utilized for GPU scratchpad memory, it provides average performance improvements of 68% and 23.4% for fundamental operations and workloads that exhibit MDA access patterns, respectively, compared with traditional 2D memory. The M3D-MDA memory also substantially reduces energy consumption.