Update article

apps/article.py

@@ -3,9 +3,8 @@ from apps.utils import read_markdown
 
 def app(state):
     st.write(read_markdown("abstract.md"))
-    st.write(read_markdown("caveats.md"))
     st.write("## Methodology")
-    col1, col2 = st.beta_columns([1, 1])
+    col1, col2 = st.beta_columns([1,2])
     col1.image(
         "./misc/article/Multilingual-VQA.png",
         caption="Masked LM model for Image-text Pretraining.",
@@ -13,6 +12,7 @@ def app(state):
     col2.markdown(read_markdown("pretraining.md"))
     st.markdown(read_markdown("finetuning.md"))
     st.write(read_markdown("challenges.md"))
+    st.write(read_markdown("limitations.md"))
     st.write(read_markdown("social_impact.md"))
     st.write(read_markdown("references.md"))
     st.write(read_markdown("checkpoints.md"))

sections/abstract.md

@@ -1,22 +1,17 @@
-Visual Question Answering (VQA) is a task where we expect the AI to answer a question about a given image.
+Visual Question Answering (VQA) is a task where we expect the AI to answer a question about a given image. VQA has been an active area of research for the past 4-5 years, with most datasets using natural images found online. Two examples of such datasets: [VQAv2](https://visualqa.org/challenge.html), [GQA](https://cs.stanford.edu/people/dorarad/gqa/about.html). VQA is a particularly interesting multi-modal machine learning challenge because it has several interesting applications across several domains including healthcare chatbots, interactive-agents, etc. **However, most VQA challenges or datasets deal with English-only captions and questions.** 
 
-VQA has been an active area of research for the past 4-5 years, with most datasets using natural images found online. Two examples of such datasets: [VQAv2](https://visualqa.org/challenge.html), [GQA](https://cs.stanford.edu/people/dorarad/gqa/about.html). VQA is a particularly interesting multi-modal machine learning challenge because it has several interesting applications across several domains including healthcare chatbots, interactivw-agents, etc. 
+In addition, even recent **approaches that have been proposed for VQA generally are obscure** due to the reasons that CNN-based object detectors are relatively difficult and more complex. For example, a FasterRCNN approach uses the following steps:
+- a FPN (Feature Pyramid Net) over a ResNet backbone, and
+- then a RPN (Regision Proposal Network) layer detects proposals in those features, and
+- then the ROI (Region of Interest) heads get the box proposals in the original image, and
+- the the boxes are selected using a NMS (Non-max suppression),
+- and then the features for selected boxes.
 
-However, most VQA challenges or datasets deal with English-only captions and questions.
-In addition, even recent approaches that have been proposed for VQA generally are obscure due to the reasons that CNN-based object detectors are relatively difficult and more complex.
-
-For example, a FasterRCNN approach uses the following steps:
-    - a FPN (Feature Pyramid Net) over a ResNet backbone, and
-    - then a RPN (Regision Proposal Network) layer detects proposals in those features, and
-    - then the ROI (Region of Interest) heads get the box proposals in the original image, and
-    - the the boxes are selected using a NMS (Non-max suppression),
-    - and then the features for selected boxes.
-
-A major advantage that comes from using transformers is their simplicity and their accessibility - thanks to HuggingFace team, ViT and Transformers authors. For ViT models, for example, all one needs to do is pass the normalized images to the transformer.
+A major **advantage that comes from using transformers is their simplicity and their accessibility** - thanks to HuggingFace team, ViT and Transformers authors. For ViT models, for example, all one needs to do is pass the normalized images to the transformer.
 
 While building a low-resource non-English VQA approach has several benefits of its own, a multilingual VQA task is interesting because it will help create a generic approach/model that works decently well across several languages.
 
-With the aim of democratizing such an obscure yet interesting task, in this project, we focus on Mutilingual Visual Question Answering (MVQA). Our intention here is to provide a Proof-of-Concept with our simple CLIP Vision + BERT baseline which leverages a multilingual checkpoint with pre-trained image encoders. Our model currently supports for four languages - English, French, German and Spanish.
+With the aim of democratizing such an challenging yet interesting task, in this project, we focus on Mutilingual Visual Question Answering (MVQA). Our intention here is to provide a Proof-of-Concept with our simple CLIP Vision + BERT baseline which leverages a multilingual checkpoint with pre-trained image encoders. Our model currently supports for four languages - English, French, German and Spanish.
 
 We follow the two-staged training approach, our pre-training task being text-only Masked Language Modeling (MLM). Our pre-training dataset comes from Conceptual-12M dataset where we use mBART-50 for translation. Our fine-tuning dataset is taken from the VQAv2 dataset and its translation is done using MarianMT models.
 

sections/caveats.md

@@ -1 +0,0 @@
-**Caveats**: The best fine-tuned model only achieves 0.49 accuracy on the multilingual validation data that we create. This could be because of not-so-great quality translations, sub-optimal hyperparameters and lack of ample training. In future, we hope to improve this model by addressing such concerns.

sections/limitations.md

@@ -0,0 +1,2 @@
+## Limitations and Bias
+- Our best fine-tuned model only achieves 0.49 accuracy on the multilingual validation data that we create. This could be because of not-so-great quality translations, sub-optimal hyperparameters and lack of ample training. In future, we hope to improve this model by addressing such concerns.