As everyone knows, training large language models (LLM) is a 'time-consuming and costly' task. This project reduces the time and cost required for training by leveraging ZeRO++ technology, which segments the model state onto each GPU, thereby increasing throughput.