Merge branch 'main' of https://huggingface.co/spaces/flax-community/Multilingual-VQA into main

app.py

@@ -9,7 +9,7 @@ def main():
     st.set_page_config(
         page_title="Multilingual VQA",
         layout="wide",
-        initial_sidebar_state="collapsed",
+        initial_sidebar_state="auto",
         page_icon="./misc/mvqa-logo-3-white.png",
     )
 

apps/article.py

@@ -61,11 +61,11 @@ def app(state=None):
     st.write(read_markdown("limitations.md"))
     
     toc.header("Conclusion, Future Work, and Social Impact")
-    # toc.subheader("Conclusion")
-    # st.write(read_markdown("conclusion_future_work/conclusion.md"))
-    # toc.subheader("Future Work")
-    # st.write(read_markdown("conclusion_future_work/future_work.md"))
-    # toc.subheader("Social Impact")
+    toc.subheader("Conclusion")
+    st.write(read_markdown("conclusion_future_work/conclusion.md"))
+    toc.subheader("Future Work")
+    st.write(read_markdown("conclusion_future_work/future_work.md"))
+    toc.subheader("Social Impact")
     st.write(read_markdown("conclusion_future_work/social_impact.md"))
     
     toc.header("References")

sections/conclusion_future_work/conclusion.md

@@ -0,0 +1 @@
+In this project, we presented Proof-of-Concept with our CLIP Vision + BERT model baseline which leverages a multilingual checkpoint with pre-trained image encoders in four languages - **English, French, German, and Spanish**. We hope to improve this in the future by using better translators (for e.g. Google Translate API) to get more multilingual data, especially in low-resource languages.

sections/conclusion_future_work/future_work.md

@@ -0,0 +1,5 @@
+We hope to improve this project in the future by using:
+- Superior translation model: Translation has a very huge impact on how the end model would perform. Better translators (for e.g. Google Translate API) and language specific seq2seq models for translation are able to generate better data, both for high-resource and low-resource languages.
+- Checking translation quality: Inspecting quality of translated data is as important as the translation model itself. For this we'll either require native speakers to manually inspect a sample of translated data or devise some unsupervised translation quality metrics for the same.
+- More data: Currently we are using only 2.5M images of Conceptual 12M for image captioning. We plan to include other datasets like Conceptual Captions 3M, subset of YFCC100M dataset etc.
+- Low resource languages: With better translation tools we also wish to train our model in low resource languages which would further democratize the image captioning solution and help people realise the potential of language systems.