Spaces:
Runtime error
Runtime error
File size: 1,068 Bytes
c2c4571 ffe19d9 fb3c77c |
1 2 3 4 5 6 7 8 9 10 11 |
- This demo loads the `FlaxCLIPVisionBertForMaskedLM` present in the `model` directory of this repository. The checkpoint is loaded from [`flax-community/clip-vision-bert-cc12m-70k`](https://huggingface.co/flax-community/clip-vision-bert-cc12m-70k) which is pre-trained checkpoint with 70k steps.
- 100 random validation set examples are present in the `cc12m_data/vqa_val.tsv` with respective images in the `cc12m_data/images_data` directory.
- You can get a random example by clicking on `Get a random example` button. The caption is tokenized and a random token is masked by replacing it with `[MASK]`.
- We provide `English Translation` of the caption for users who are not well-acquainted with the other languages. This is done using `mtranslate` to keep things flexible enough and needs internet connection as it uses the Google Translate API.
- The model predicts the scores for tokens from the `bert-base-multilingual-uncased` checkpoint.
- The top-5 predictions are displayed below and their respective confidence scores are shown in form of a bar plot. |