|
--- |
|
library_name: transformers |
|
tags: [] |
|
--- |
|
|
|
# OFT model for PEFT testing purposes |
|
|
|
Use PEFT version 0.12.0. |
|
|
|
```python |
|
from transformers import AutoModelForCausalLM |
|
from peft import OFTConfig, get_peft_model |
|
|
|
model_id = "hf-internal-testing/tiny-random-OPTForCausalLM" |
|
model = AutoModelForCausalLM.from_pretrained(model_id) |
|
config = OFTConfig(init_weights=False, target_modules="all-linear") |
|
model = get_peft_model(model, config) |
|
|
|
hub_id = "peft-internal-testing/OFT-tiny-random-OPTForCausalLM" |
|
model.push_to_hub(hub_id, token=...) |
|
``` |
