Official inference framework for 1-bit LLMs

BitNet

该项目是微软专为 CPU 本地推理和极致压缩（低比特）大模型设计的推理框架。它支持对 1-bit/1.58-bit 量化模型进行高效、低能耗的推理，兼容 BitNet、Llama3-8B-1.58、Falcon3 等模型，适用于在本地或边缘设备上运行大模型推理任务，无需 GPU。

This is an inference framework designed by Microsoft specifically for CPU-based local inference and extreme compression (low-bit) of large models. It supports efficient and low-power inference for 1-bit/1.58-bit quantized models, compatible with models such as BitNet, Llama3-8B-1.58, and Falcon3. It is suitable for running large model inference tasks locally or on edge devices without the need for GPUs.

microsoft/BitNet

microsoft/BitNet

评论