Spaces:
Sleeping
tags:
- visual-question-answering-for-fashion-context
sdk: gradio
license: apache-2.0
widget:
- text: VQA system
src: testing.jpg
This is a simple VQA system using Hugging Face, PyTorch and VQA models
In this repository we created a simple VQA system capable of recognize spatial and context information of fashion images (e.g. clothes color and details).
The project was based in this paper FashionVQA: A Domain-Specific Visual Question Answering System [1]. We also used the VQA pre-trained model from BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation [] to make the model finetuning the two new models.
We used the datasets Deep Fashion with Masks available in https://huggingface.co/datasets/SaffalPoosh/deepFashion-with-masks and the Control Net Dataset available in https://huggingface.co/datasets/ldhnam/deepfashion_controlnet.
References
[1] Min Wang and Ata Mahjoubfar and Anupama Joshi, 2022 FashionVQA: A Domain-Specific Visual Question Answering System
[2] Junnan Li and Dongxu Li and Caiming Xiong and Steven Hoi, 2022 BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation