Update README.md
Browse files
README.md
CHANGED
@@ -8,14 +8,14 @@ pipeline_tag: image-text-to-text
|
|
8 |
[![arXiv](https://img.shields.io/badge/Arxiv-2402.14289-b31b1b.svg?logo=arXiv)](https://arxiv.org/abs/2402.14289)[![Github](https://img.shields.io/badge/Github-Github-blue.svg)](https://github.com/TinyLLaVA/TinyLLaVA_Factory)[![Demo](https://img.shields.io/badge/Demo-Demo-red.svg)](http://8843843nmph5.vicp.fun/#/)
|
9 |
|
10 |
|
11 |
-
Here, we introduce TinyLLaVA-Qwen2-0.5B
|
12 |
|
13 |
### Usage
|
14 |
Execute the following test code:
|
15 |
```python
|
16 |
from transformers import AutoTokenizer, AutoModelForCausalLM
|
17 |
|
18 |
-
hf_path = 'Zhang199/TinyLLaVA-Qwen2-0.5B
|
19 |
model = AutoModelForCausalLM.from_pretrained(hf_path, trust_remote_code=True)
|
20 |
model.cuda()
|
21 |
config = model.config
|
@@ -34,7 +34,7 @@ print('runing time:', genertaion_time)
|
|
34 |
| [bczhou/TinyLLaVA-3.1B](https://huggingface.co/bczhou/TinyLLaVA-3.1B) (our legacy model) | 79.9 | 62.0 | 69.1 | 59.1 | 32.0 | 86.4 | 1464.9 | - |
|
35 |
| [tinyllava/TinyLLaVA-Gemma-SigLIP-2.4B](https://huggingface.co/tinyllava/TinyLLaVA-Gemma-SigLIP-2.4B) | 78.4 | 61.6 | 64.4 | 53.6 | 26.9 | 86.4 | 1339.0 | 31.7 |
|
36 |
| [tinyllava/TinyLLaVA-Phi-2-SigLIP-3.1B](https://huggingface.co/tinyllava/TinyLLaVA-Phi-2-SigLIP-3.1B) | 80.1 | 62.1 | 73.0 | 60.3 | 37.5 | 87.2 | 1466.4 | 38.4 |
|
37 |
-
| [Zhang199/TinyLLaVA-Qwen2-0.5B
|
38 |
|
39 |
P.S. [TinyLLaVA Factory](https://github.com/TinyLLaVA/TinyLLaVA_Factory) is an open-source modular codebase for small-scale LMMs with a focus on simplicity of code implementations, extensibility of new features, and reproducibility of training results. This code repository provides standard training&evaluating pipelines, flexible data preprocessing&model configurations, and easily extensible architectures. Users can customize their own LMMs with minimal coding effort and less coding mistake.
|
40 |
|
|
|
8 |
[![arXiv](https://img.shields.io/badge/Arxiv-2402.14289-b31b1b.svg?logo=arXiv)](https://arxiv.org/abs/2402.14289)[![Github](https://img.shields.io/badge/Github-Github-blue.svg)](https://github.com/TinyLLaVA/TinyLLaVA_Factory)[![Demo](https://img.shields.io/badge/Demo-Demo-red.svg)](http://8843843nmph5.vicp.fun/#/)
|
9 |
|
10 |
|
11 |
+
Here, we introduce TinyLLaVA-Qwen2-SigLIP-0.5B , which is trained by the [TinyLLaVA Factory](https://github.com/TinyLLaVA/TinyLLaVA_Factory) codebase. For LLM and vision tower, we choose [Qwen-2-0.5B](https://huggingface.co/Qwen/Qwen2-0.5B) and [siglip-so400m-patch14-384](https://huggingface.co/google/siglip-so400m-patch14-384), respectively.
|
12 |
|
13 |
### Usage
|
14 |
Execute the following test code:
|
15 |
```python
|
16 |
from transformers import AutoTokenizer, AutoModelForCausalLM
|
17 |
|
18 |
+
hf_path = 'Zhang199/TinyLLaVA-Qwen2-SigLIP-0.5B'
|
19 |
model = AutoModelForCausalLM.from_pretrained(hf_path, trust_remote_code=True)
|
20 |
model.cuda()
|
21 |
config = model.config
|
|
|
34 |
| [bczhou/TinyLLaVA-3.1B](https://huggingface.co/bczhou/TinyLLaVA-3.1B) (our legacy model) | 79.9 | 62.0 | 69.1 | 59.1 | 32.0 | 86.4 | 1464.9 | - |
|
35 |
| [tinyllava/TinyLLaVA-Gemma-SigLIP-2.4B](https://huggingface.co/tinyllava/TinyLLaVA-Gemma-SigLIP-2.4B) | 78.4 | 61.6 | 64.4 | 53.6 | 26.9 | 86.4 | 1339.0 | 31.7 |
|
36 |
| [tinyllava/TinyLLaVA-Phi-2-SigLIP-3.1B](https://huggingface.co/tinyllava/TinyLLaVA-Phi-2-SigLIP-3.1B) | 80.1 | 62.1 | 73.0 | 60.3 | 37.5 | 87.2 | 1466.4 | 38.4 |
|
37 |
+
| [Zhang199/TinyLLaVA-Qwen2-SigLIP-0.5B](https://huggingface.co/Zhang199/TinyLLaVA-Qwen2-0.5B-siglip-so400m-patch14-384-base) | 72.33 | 55.84 | 60.14 | 45.17 | 19.5 | 86.59 | 1153 | 29.7 |
|
38 |
|
39 |
P.S. [TinyLLaVA Factory](https://github.com/TinyLLaVA/TinyLLaVA_Factory) is an open-source modular codebase for small-scale LMMs with a focus on simplicity of code implementations, extensibility of new features, and reproducibility of training results. This code repository provides standard training&evaluating pipelines, flexible data preprocessing&model configurations, and easily extensible architectures. Users can customize their own LMMs with minimal coding effort and less coding mistake.
|
40 |
|