File size: 1,814 Bytes
a5d86ce 0471e58 b1d5c08 9372ae4 b1d5c08 7b8e5bb b1d5c08 43f8bb1 b1d5c08 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 |
---
license: apache-2.0
---
# MLM-Filter-13b Model Card
## Model details
**Model type:**
MLM-Filter-13b is an open-source MLLM trained to assess the data quality of image-text paired data. It can generate 4 quality metrics for image-text data: Image Text Matching, Object Detail Fulfillment, Caption Text Quality, and Semantic Understanding.
**Model date:**
MLM-Filter-13B was trained in Dec 2023.
**Paper or resources for more information:**
https://mlm-filter.github.io/
```
@article{wang2024finetuned,
title={Finetuned Multimodal Language Models Are High-Quality Image-Text Data Filters},
author={Wang, Weizhi and Mrini, Khalil and Yang, Linjie and Kumar, Sateesh and Tian, Yu and Yan, Xifeng and Wang, Heng},
journal={arXiv preprint arXiv:2403.02677},
year={2024}
}
```
## License
Llama 2 is licensed under the LLAMA 2 Community License,
Copyright (c) Meta Platforms, Inc. All Rights Reserved.
**Where to send questions or comments about the model:**
https://github.com/Victorwz/MLM_Filter/issues
## Intended use
**Primary intended uses:**
MLM-Filter can be used as a drop-in replacement for CLIPScore in these tasks:
1. Score image-text data in large-scale pre-training dataset and then filter high-quality subsets based on the scores (For training MLLMs or VLMs, please consider to jointly use the Image-Text Matching score and the Object Detail Fulfillment score);
2. Evaluate the image-text alignment for image2text or text2image generation models;
3. Any potential applications with the need to calculate the image-text alignment.
## Training dataset
- 46k instruction sampled from LLaVA-1.5 665k data.
- 4k instructions on image-text data quality assessment tasks ranging across 4 metrics.
## Usage Sample
Please follow the instructions in https://github.com/Victorwz/MLM_Filter. |