This project is a summary of the author's experience in training open-source BLOOM-176B large-scale models and IDEFICS-80B multimodal models, offering a wealth of ready-to-use code and scripts to assist you in successfully training large language models and multimodal models.