Repository Details
Shared by
HelloGitHub Rating
0 ratings
MLA Kernel Optimization Based on Hopper GPU
Past 6 days Received 10 stars ✨
Use arrow keys to explore dates, Home and End for the first and last date, Escape to dismiss.| Date | Star | Daily change |
|---|---|---|
| 周日 | 13,256 | — |
| 周一 | 13,257 | +1 |
| 周二 | 13,261 | +4 |
| 周三 | 13,261 | 0 |
| 周四 | 13,263 | +2 |
| 周五 | 13,266 | +3 |
13.3k
Stars
No
Chinese
C++
Language
Yes
Active
20
Contributors
149
Issues
Yes
Organization
None
Latest
1k
Forks
MIT
License
This is an efficient MLA decoding kernel designed specifically for Hopper architecture GPUs, aiming to improve the inference efficiency of large language models (LLMs). Developed in C++ and CUDA, it addresses the performance bottlenecks of traditional methods when handling variable-length sequences by leveraging NVIDIA's CUTLASS library and paginated caching techniques, significantly enhancing memory bandwidth and computational efficiency.
Comments
Rating:
No comments yet