Repository Details
Shared by
HelloGitHub Rating
10.0
4 ratings
Train a Small Language Model from Scratch
Past 6 days Received 188 stars ✨
Use arrow keys to explore dates, Home and End for the first and last date, Escape to dismiss.| Date | Star | Daily change |
|---|---|---|
| 周日 | 65,019 | — |
| 周一 | 65,066 | +47 |
| 周二 | 65,098 | +32 |
| 周三 | 65,129 | +31 |
| 周四 | 65,174 | +45 |
| 周五 | 65,207 | +33 |
65.2k
Stars
No
Chinese
Python
Language
Yes
Active
27
Contributors
63
Issues
No
Organization
None
Latest
9k
Forks
Apache-2.0
License

This is not only an implementation of a mini-language model, but also an introductory tutorial for LLMs, aimed at lowering the barrier to learning and getting started with LLMs. It provides the full process code and tutorials from data preprocessing to model training, fine-tuning, and inference. The smallest model has a parameter count of only 0.02B, which can be easily run on a regular GPU.
Comments
Rating:
No comments yet