Spaces:
Configuration error
Configuration error
| language: | |
| - en | |
| license: apache-2.0 | |
| # FOR HF VERSION: https://huggingface.co/qresearch/llama-3-vision-alpha-hf | |
| # llama3-vision-alpha | |
| projection module trained to add vision capabilties to Llama 3 using SigLIP. built by [@yeswondwerr](https://x.com/yeswondwerr) and [@qtnx_](https://x.com/qtnx_) | |
| **usage** | |
| ``` | |
| pip install -r requirements.txt | |
| ``` | |
| ``` | |
| python __main__.py -i image_path | |
| ``` | |
| **examples** | |
| | Image | Examples | | |
| | --- | --- | | |
| | <img src="assets/demo-1.jpg" width="200"/> | **What is the title of this book? answer briefly**<br>The title of the book is "The Little Book of Deep Learning".<br><br>**Where is the person standing? answer briefly**<br>The person is standing on the balcony.<br><br>**Describe the image**<br>The image shows a person holding a book with a cityscape visible through the window behind them. The book has a cover with a title that reads "The Little Book of Deep Learning" in bold letters. | | |
| | <img src="assets/demo-2.jpg" width="200"/> | **What type of food is the girl holding? answer briefly**<br>A hamburger!<br><br>**What color is the woman's hair? answer briefly**<br>It's white!<br><br>**Describe the image**<br>The image is of a young girl with short, curly hair and a sweet smile, holding a giant hamburger in her hand. She's sitting at a table with a festive dinner setting, surrounded by candles and a warm glow. Her eyes are shining with excitement and contentment as she takes a big bite of the burger. | | |
| **acknowledgements** | |
| - Liu et al. : [LLaVA](https://arxiv.org/abs/2304.08485) | |
| - Moon et al. : [AnyMAL](https://arxiv.org/abs/2309.16058) | |
| - vikhyatk : moondream, test images | |
| ``` | |
| .x+=:. | |
| z` ^% .uef^" | |
| .u . . <k .u . :d88E | |
| .u@u .d88B :@8c .u .@8Ned8" .u u .d88B :@8c . `888E | |
| .zWF8888bx ="8888f8888r ud8888. .@^%8888" ud8888. us888u. ="8888f8888r .udR88N 888E .z8k | |
| .888 9888 4888>'88" :888'8888. x88: `)8b. :888'8888. .@88 "8888" 4888>'88" <888'888k 888E~?888L | |
| I888 9888 4888> ' d888 '88%" 8888N=*8888 d888 '88%" 9888 9888 4888> ' 9888 'Y" 888E 888E | |
| I888 9888 4888> 8888.+" %8" R88 8888.+" 9888 9888 4888> 9888 888E 888E | |
| I888 9888 .d888L .+ 8888L @8Wou 9% 8888L 9888 9888 .d888L .+ 9888 888E 888E | |
| `888Nx?888 ^"8888*" '8888c. .+ .888888P` '8888c. .+ 9888 9888 ^"8888*" ?8888u../ 888E 888E | |
| "88" '888 "Y" "88888% ` ^"F "88888% "888*""888" "Y" "8888P' m888N= 888> | |
| 88E "YP' "YP' ^Y" ^Y' "P' `Y" 888 | |
| 98> J88" | |
| '8 @% | |
| ` :" | |
| ``` |