Repository Details
Shared by
HelloGitHub Rating
0 ratings
Run 100B+ Parameter Models Without a GPU
Past 7 days Received 1.8k stars ✨
Use arrow keys to explore dates, Home and End for the first and last date, Escape to dismiss.| Date | Star | Daily change |
|---|---|---|
| 周二 | 49,754 | — |
| 周三 | 49,967 | +213 |
| 周四 | 50,427 | +460 |
| 周五 | 50,848 | +421 |
| 周六 | 51,119 | +271 |
| 周日 | 51,354 | +235 |
| 周一 | 51,573 | +219 |
51.6k
Stars
No
Chinese
C
Language
Yes
Active
220
Contributors
198
Issues
No
Organization
None
Latest
6k
Forks
Apache-2.0
License

This is a mixture-of-experts inference engine written in pure C, with zero dependencies and no GPU required. It keeps only the parts the model needs on every run in memory, and reads the remaining expert weights from disk on demand. With as little as 8GB of RAM, it can run open-source large models such as GLM, Kimi, DeepSeek, and Qwen, and provides OpenAI- and Anthropic-compatible APIs. It works on Windows, Linux, and macOS.
Comments
Rating:
No comments yet