Repository Details
Shared by


HelloGitHub Rating
0 ratings
Free•Apache-2.0
Claim
Discuss
Collect
Share
8.5k
Stars
No
Chinese
Rust
Language
Yes
Active
67
Contributors
286
Issues
Yes
Organization
2025.7.0
Latest
542
Forks
Apache-2.0
License
More

This is an open-source enterprise-level LLM application platform that integrates functions such as LLM gateway, monitoring, optimization, fine-tuning, evaluation and experimentation. It supports one-stop access and unified management of multiple mainstream large models (such as OpenAI, Anthropic, Llama, etc.), and provides multiple functions such as streaming inference, multimodal, structured response, caching, etc. Built with Rust at the bottom, it achieves high throughput and low latency and is suitable for production deployment of large model access and complex AI scenarios.
Comments
Rating:
No comments yet