SQFT: Low-cost Model Adaptation in Low-precision Sparse Foundation Models
Paper
•
2410.03750
•
Published
•
1
SQFT Models (SQFT: Low-cost Model Adaptation in Low-precision Sparse Foundation Models)
Note SQFT base models for Mistral-7B-v0.3 (Sparsity 50%)
Note SQFT fine-tuned models for Mistral-7B-v0.3 on GSM8K (Sparsity 50%)
Note SQFT fine-tuned models for Mistral-7B-v0.3 on Math Instruction Tuning (Sparsity 50%)
Note SQFT base models for Phi-3-mini-4k-instruct (Sparsity 50%)
Note SQFT fine-tuned models for Phi-3-mini-4k-instruct on Math Instruction Tuning (Sparsity 50%)
Note SQFT fine-tuned models for Phi-3-mini-4k-instruct on Commonsense Reasoning (Sparsity 50%) Below are some other models.