Repository Details
Shared by
HelloGitHub Rating
10.0
1 ratings
Microsoft's Open-Source 1-bit Large Model Inference Framework
Past 7 days Received 73 stars ✨
Use arrow keys to explore dates, Home and End for the first and last date, Escape to dismiss.| Date | Star | Daily change |
|---|---|---|
| 周二 | 41,516 | — |
| 周三 | 41,526 | +10 |
| 周四 | 41,563 | +37 |
| 周五 | 41,575 | +12 |
| 周六 | 41,580 | +5 |
| 周日 | 41,585 | +5 |
| 周一 | 41,589 | +4 |
41.6k
Stars
No
Chinese
C++
Language
No
Active
23
Contributors
379
Issues
Yes
Organization
None
Latest
4k
Forks
MIT
License

This is an inference framework designed by Microsoft specifically for CPU-based local inference and extreme compression (low-bit) of large models. It supports efficient and low-power inference for 1-bit/1.58-bit quantized models, compatible with models such as BitNet, Llama3-8B-1.58, and Falcon3. It is suitable for running large model inference tasks locally or on edge devices without the need for GPUs.
Comments
Rating:
No comments yet