Update README.md

README.md

@@ -1,3 +1,38 @@
----
-license: apache-2.0
----
+---
+license: apache-2.0
+---
+
+# AskVideos-VideoCLIPv0.3
+Like it's image-only counterpart, CLIP, VideoCLIP enables you to compute a single embedding for videos that can be used to compute similarity with text.
+
+VideoCLIP uses a Video Q-Former to aggregate frame-level embeddings temporally into a single embedding, maintaining relevance of the underlying content. The resulting embedding is then trained with contrastive loss + captioning loss to match it's corresponding text.
+
+This is the latest version of the VideoCLIP model, incorporating more diverse and high quality data. Compared to v0.2, this model performs better on a larger distribution of data and works better on long range retrieval tasks.
+
+# Usage
+Link to github to run the model: [link](https://github.com/AskYoutubeAI/AskVideos-VideoCLIP).
+```
+# Load model.
+import video_clip
+eval_config = 'eval_configs/video_clip.yaml'
+model, vis_processor = video_clip.load_model(eval_config)
+
+# Compute video embeddings.
+# video_embs: float matrix of size [num_videos, clip_dim_size, query_tokens] containing VideoCLIP embeddings.
+# In this model, clip_dim_size=1024 and query_tokens=32.
+video_embs = video_clip.get_all_video_embeddings(videos, model, vis_processor)
+
+# Compute Video-Text similarity.
+# v2t_sim: float matrix of size [num_videos, num_texts] indicating similarity.
+v2t_sim = video_clip.compute_sim(model, texts, video_embs)
+
+# Compute Text-Video similarity.
+# t2v_sim: float matrix of size [num_texts, num_videos] indicating similarity.
+t2v_sim = v2t_sim.T
+
+# Compute Video-Video distance.
+# v2v_dists: float vector of size [1, num_videos] indicating distance to query video embedding.
+v2v_dists = video_clip.compute_dist_videoq(model, video_embs[0], video_embs)
+```
+
+For a more detailed demo of how to use the model, see the [colab](https://colab.research.google.com/drive/1TfEIqzEq_ppVSQHfEHXvbIrh0MTn9vpX).