Repository Details
Shared by


HelloGitHub Rating
10.0
6 ratings
Simple and Easy-to-Use Multi-end AI Inference Deployment Framework
Past 7 days Received 28 stars ✨
Free•Apache-2.0
Discuss
Collect
Share
992
Stars
Yes
Chinese
C++
Language
Yes
Active
38
Contributors
3
Issues
Yes
Organization
1.0.0.0
Latest
126
Forks
Apache-2.0
License
More

This is a simple, easy-to-use, high-performance, and multi-end AI inference deployment framework. It is designed based on directed acyclic graphs, abstracting preprocessing, inference, and postprocessing as nodes of the graph, and supporting optimization methods such as pipeline parallelism and task parallelism. It is compatible with multiple inference backends such as TensorRT, OpenVINO, and MNN, and is adapted to mainstream text-to-image, large language, detection and other models, realizing one-code multi-end deployment.
Comments
Rating:
No comments yet