File size: 2,321 Bytes
1689bb8 eeb5be0 1689bb8 dc6c541 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 |
---
license: llama3.2
---
## Llama-3.2-SFT-Vision-Arena Model Card
### Model Details
Llama-3.2-SFT-Vision-Arena is a chat assistant trained by fine-tuning Llama-3.2-11B-Vision on user-shared conversations collected from Chatbot Arena.
- Developed by: LMArena
- Model type: An auto-regressive vision language model based on the transformer architecture
- License: Llama 3.2 Community License Agreement
- Finetuned from model: Llama-3.2-11B-Vision
### Model Sources
- Repository: https://github.com/lm-sys/FastChat
- Paper: https://arxiv.org/abs/2412.08687
### Sample Inference Code
```
import requests
import torch
from PIL import Image
from transformers import MllamaForConditionalGeneration, AutoProcessor
model_id = "lmarena-ai/llama-3.2-sft-vision-arena"
model = MllamaForConditionalGeneration.from_pretrained(
model_id,
torch_dtype=torch.bfloat16,
device_map="auto",
)
processor = AutoProcessor.from_pretrained(model_id)
url = "https://huggingface.co/datasets/huggingface/documentation-images/resolve/0052a70beed5bf71b92610a43a52df6d286cd5f3/diffusers/rabbit.jpg"
image = Image.open(requests.get(url, stream=True).raw)
messages = [
{"role": "user", "content": [
{"type": "image"},
{"type": "text", "text": "Write a haiku about this image: "}
]}
]
input_text = processor.tokenizer.apply_chat_template(messages, add_generation_prompt=True, tokenize=False)
inputs = processor(
image,
input_text,
add_special_tokens=False,
return_tensors="pt"
).to(model.device)
output = model.generate(**inputs, max_new_tokens=30)
print(processor.decode(output[0]))
```
### Uses
The primary use of Llama-3.2-SFT-Vision-Arena is research on vision language models and chatbots. The primary intended users of the model are researchers and hobbyists in natural language processing, machine learning, and artificial intelligence.
### BibTex
```
@misc{chou2024visionarena,
title={VisionArena: 230K Real World User-VLM Conversations with Preference Labels},
author={Christopher Chou and Lisa Dunlap and Koki Mashita and Krishna Mandal and Trevor Darrell and Ion Stoica and Joseph E. Gonzalez and Wei-Lin Chiang},
year={2024},
eprint={2412.08687},
archivePrefix={arXiv},
primaryClass={cs.LG},
url={https://arxiv.org/abs/2412.08687},
}
``` |