下拉刷新
Repository Details
Shared bynavbar_avatar
repo_avatar
HelloGitHub Rating
0 ratings
Run 100B+ Parameter Models Without a GPU
Free•Apache-2.0
Claim
Collect
Share
44.2k
Stars
No
Chinese
C
Language
Yes
Active
199
Contributors
148
Issues
No
Organization
None
Latest
5k
Forks
Apache-2.0
License
More
colibri image
This is a mixture-of-experts inference engine written in pure C, with zero dependencies and no GPU required. It keeps only the parts the model needs on every run in memory, and reads the remaining expert weights from disk on demand. With as little as 8GB of RAM, it can run open-source large models such as GLM, Kimi, DeepSeek, and Qwen, and provides OpenAI- and Anthropic-compatible APIs. It works on Windows, Linux, and macOS.
Included in:
Vol.127
Tags:
推理框架
C

Comments

Rating:
No comments yet