---
title: README
emoji: 🚀
colorFrom: indigo
colorTo: blue
sdk: static
pinned: false
---

# Foundation Model Stack

Foundation Model Stack (fms) is a collection of components for development, inference, training, and tuning of foundation models leveraging PyTorch native components. 
For inference optimizations we aim to support PyTorch compile, accelerated transformers, and tensor parallelism. 
At training time we aim to support FSDP, accelerated transformers, and PyTorch compile.

Foundation Model Stack is split up into a few main repositories:

- [foundation-model-stack](https://github.com/foundation-model-stack/foundation-model-stack): Main repository for which all fms models are based
- [fms-extras](https://github.com/foundation-model-stack/fms-extras): New features staged to be integrated with foundation-model-stack
- [fms-fsdp](https://github.com/foundation-model-stack/fms-fsdp): Pre-Training Examples using FSDP wrapped foundation models
- [fms-hf-tuning](https://github.com/foundation-model-stack/fms-hf-tuning): Basic Tuning scripts for fms models leveraging SFTTrainer