DeepGEMM-Ascend

deepseek-ai/DeepGEMM-Ascend is ranked 3,201 among these 8 peers; humancomputerlab/LeLamp is ranked 3,199.

DeepGEMM Ascend DeepGEMM Ascend is a port of DeepGEMM to the HUAWEI Ascend platform. It is fully API-compatible with DeepGEMM and supports BF16, FP8, FP4 GEMM, MQA logits, and MegaMoE. On Ascend platforms, users can simply install the package and use the same APIs and development workflow as DeepGEMM on other supported platforms. DeepGEMM Ascend provides a lightweight abstraction over the Ascend MAD (matrix multiply-add) primitives, hiding much of the complexity associated with fractal layouts, alignment constraints, address

calculations, and verbose low-level parameters. This enables GEMM kernels to remain both concise and efficient. DeepGEMM Ascend makes extensive use of Ascend-specific optimization techniques, such as sparse data loading and coroutine-based pipelining, to approach the performance limits of the Ascend hardware. These implementations can also serve as references for extreme performance optimization on the Ascend platform. Despite its lightweight codebase, DeepGEMM Ascend can achieve peak hardware performance across a wide range of matrix shapes. 30: Initial release

of DeepGEMM Ascend, with support for Ascend 950 devices. The kernels are designed to achieve near-peak hardware performance. Quick Start Requirements HUAWEI Ascend NPU (developed and validated on the Ascend 950 series) CANN 9. 20 toolkit providing , Torch NPU package Python 3. 10 or higher Compilers and standard libraries with C++20 support , used by the HC prenorm kernel (declared as a package dependency) and , used to generate Python type stubs when building from source Development Installation Interfaces Kernel

Interface Please refer to DeepGEMM's interfaces for the kernel APIs. NOTE] The scaling factor format on Ascend differs from NVIDIA's: each pair of UE8M0 scaling factors along the K dimension is packed into an , and the packed values are stored in MN-major order for optimal hardware efficiency. Dense BF16, FP8, FP4 and mixed FP8×FP4 GEMMs accept with shape or and the same dtype as . A contiguous bias is loaded and added in the AIV epilogue before the output

cast. The and interfaces take and : Utilities The library provides some utility functions besides the above kernels: / : set/get the number of AI cores the kernels may use / : override/query the NPU architecture used for JIT compilation, queries the device / : set/get the group-level M/K alignment for contiguous layout : get the theoretical minimum M/K alignment / : enable/disable deterministic algorithms : transform scaling factors into the required layout : transform K-grouped scaling factors into the required layout : build the

scheduling metadata for the paged MQA kernels and : ACLNN reference GEMMs used to cross-check the kernels in tests Environment Variables Ascend home path is given by or . Each variable falls back to the corresponding global variable when unset, and is snapshotted when the JIT runtime is first created. General JIT cache Compiler output and artifacts Debugging Performance Measured on Ascend 950DT (CANN 9. Shapes and cases follow the DeepGEMM test suite and cover inference and training workloads of the

DeepSeek model series; see for reproduction. Dense GEMM Dense GEMM reaches up to **99. 3 | | FP8×FP4 | 4096 | 7168 | 16384