Spaces:
Running
Running
File size: 1,095 Bytes
3893779 2009ed1 62d29da 2009ed1 6b2edfb 2009ed1 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 |
---
title: README
emoji: 🚀
colorFrom: indigo
colorTo: blue
sdk: static
pinned: false
---
# Foundation Model Stack
Foundation Model Stack (fms) is a collection of components for development, inference, training, and tuning of foundation models leveraging PyTorch native components.
For inference optimizations we aim to support PyTorch compile, accelerated transformers, and tensor parallelism.
At training time we aim to support FSDP, accelerated transformers, and PyTorch compile.
Foundation Model Stack is split up into a few main repositories:
- [foundation-model-stack](https://github.com/foundation-model-stack/foundation-model-stack): Main repository for which all fms models are based
- [fms-extras](https://github.com/foundation-model-stack/fms-extras): New features staged to be integrated with foundation-model-stack
- [fms-fsdp](https://github.com/foundation-model-stack/fms-fsdp): Pre-Training Examples using FSDP wrapped foundation models
- [fms-hf-tuning](https://github.com/foundation-model-stack/fms-hf-tuning): Basic Tuning scripts for fms models leveraging SFTTrainer
|