2024-03-18_strategy_to_pretrain_a_model.md

Pre-train a model

Resources

• Colossal AI return of experience to continue the pre-training of LLama2
• Colossal AI Making large AI models cheaper, faster and more accessible
• Colossal AI Website

url: https://colossalai.org/
title: "Colossal-AI"
description: "Making big AI models cheaper, easier, and scalable"
host: colossalai.org
favicon: https://colossalai.org/img/favicon.ico
image: https://colossalai.org/img/social-card.png

url: https://github.com/hpcaitech/ColossalAI/tree/main/applications/Colossal-LLaMA-2
title: "ColossalAI/applications/Colossal-LLaMA-2 at main · hpcaitech/ColossalAI"
description: "Making large AI models cheaper, faster and more accessible - hpcaitech/ColossalAI"
host: github.com
favicon: https://github.githubassets.com/favicons/favicon.svg
image: https://repository-images.githubusercontent.com/422274596/bf53ba49-de0d-4308-bc38-9c0ef56f4eeb

• Simple and Scalable Strategies to Continually Pre-train Large Language Models
• Yi: Open Foundation Models by 01.AI