|
--- |
|
language: en |
|
datasets: |
|
- laion2b |
|
--- |
|
|
|
# OpenFlamingo-9B (Deprecated) |
|
|
|
**This early checkpoint was part of an initial release. It has since been deprecated in favor of [other checkpoints](https://huggingface.co/openflamingo/OpenFlamingo-9B-vitl-mpt7b) as part of the OpenFlamingo v2 release. However, it is possible to continue using this older checkpoint in the new codebase.** |
|
|
|
---- |
|
|
|
[Blog post](https://laion.ai/blog/open-flamingo/) | [Code](https://github.com/mlfoundations/open_flamingo) | [Demo](https://7164d2142d11.ngrok.app) |
|
|
|
OpenFlamingo is an open source implementation of DeepMind's [Flamingo](https://www.deepmind.com/blog/tackling-multiple-tasks-with-a-single-visual-language-model) models. |
|
OpenFlamingo-9B is built off of [CLIP ViT-L/14](https://huggingface.co/openai/clip-vit-large-patch14) and [LLaMA-7B](https://ai.facebook.com/blog/large-language-model-llama-meta-ai/). Before using this model please familiarize yourself with our [terms and conditions](https://github.com/mlfoundations/open_flamingo/blob/main/TERMS_AND_CONDITIONS.md). |
|
|
|
## Model Details |
|
We freeze the pretrained vision encoder and language model, and then we train connecting Perceiver modules and cross-attention layers, following the original Flamingo paper. |
|
|
|
Our training data is a mixture of [LAION 2B](https://huggingface.co/datasets/laion/laion2B-en) and a large interleaved image-text dataset called Multimodal C4, which will be released soon. |
|
|
|
The current model is an early checkpoint of an ongoing effort. This checkpoint has seen 5 million interleaved image-text examples from Multimodal C4. |
|
|
|
## Uses |
|
OpenFlamingo-9B is intended to be used **for academic research purposes only.** Commercial use is prohibited, in line with LLaMA's non-commercial license. |
|
|
|
### Bias, Risks, and Limitations |
|
This model may generate inaccurate or offensive outputs, reflecting biases in its training data and pretrained priors. |
|
|
|
In an effort to mitigate current potential biases and harms, we have deployed a content filter on model outputs in the OpenFlamingo demo. We continue to red-team the model to understand and improve its safety. |
|
|
|
## Evaluation |
|
We've evaluated this checkpoint and report validation performance for two vision-language tasks: COCO captioning and VQAv2. Results are displayed below. |
|
|
|
**COCO (CIDEr)** |
|
|
|
|0-shot|4-shot|8-shot|16-shot|32-shot| |
|
|--|--|--|--|--| |
|
|65.52|74.28|79.26|81.84|84.52| |
|
|
|
|
|
**VQAv2 (VQA accuracy)** |
|
|
|
|0-shot|4-shot|8-shot|16-shot|32-shot| |
|
|---|---|---|---|---| |
|
|43.55|44.05|47.5|48.87|50.34| |
