Update README.md

README.md

@@ -1,12 +1,21 @@
+---
+language:
+- en
+base_model:
+- lmsys/vicuna-7b-v1.5
+- openai/clip-vit-large-patch14
+- laion/CLIP-convnext_large_d.laion2B-s26B-b102K-augreg
+pipeline_tag: image-text-to-text
+tags:
+- chatrex
+- upn
+---
+
 
 <div align=center>
   <img src="assets/teaser.jpg" width=600 >
 </div>
 
-
-
-<div align=center>
-
 ----
 
 # 1. Introduction 📚
@@ -25,13 +34,15 @@ understanding.
 ```bash
 conda install -n chatrex python=3.9
 pip install torch==2.1.2 torchvision==0.16.2 --index-url https://download.pytorch.org/whl/cu121
+git clone https://github.com/IDEA-Research/ChatRex.git
+cd ChatRex
 pip install -v -e .
 # install deformable attention for universal proposal network
 cd chatrex/upn/ops
 pip install -v -e .
 ```
 
-## 2.1 Download Pre-trained Models
+## 2.1 Download Pre-trained UPN Models
 We provide model checkpoints for both the ***Universal Proposal Network (UPN)*** and the ***ChatRex model***. You can download the pre-trained models from the following links:
 - [UPN Checkpoint](https://github.com/IDEA-Research/ChatRex/releases/download/upn-large/upn_large.pth)
 - [ChatRex-7B Checkpoint](https://huggingface.co/IDEA-Research/ChatRex-7B)
@@ -42,10 +53,6 @@ mkdir checkpoints
 mkdir checkpoints/upn
 # download UPN checkpoint
 wget -O checkpoints/upn/upn_large.pth https://github.com/IDEA-Research/ChatRex/releases/download/upn-large/upn_large.pth
-# download ChatRex checkpoint from huggingface IDEA-Research/ChatRex-7B
-# Download ChatRex checkpoint from Hugging Face
-git lfs install
-git clone https://huggingface.co/IDEA-Research/ChatRex-7B checkpoints/chatrex
 ```
 
 ## 2.2 Verify Installation
@@ -165,8 +172,6 @@ Please detect person in the car; cat below the table in this image. Answer the q
 <details close>
 <summary><strong>Example Code</strong></summary>
 
-- [Example Code in python file](tests/test_chatrex_detection.py)
-
 ```python
 import torch
 from PIL import Image
@@ -283,8 +288,6 @@ Can you provide me with a one sentence of <obji>? Answer the question with one s
 <details close>
 <summary><strong>Example Code</strong></summary>
 
-- [Example Code in python file](tests/test_chatrex_region_caption.py)
-
 ```python
 import torch
 from PIL import Image
@@ -377,8 +380,6 @@ Please provide a detailed description of the image and detect all the mentioned
 <details close>
 <summary><strong>Example Code</strong></summary>
 
-- [Example Code in python file](tests/test_chatrex_grounded_image_caption.py)
-
 ```python
 import torch
 from PIL import Image
@@ -481,8 +482,6 @@ Answer the question in Grounded format. Question
 <details close>
 <summary><strong>Example Code</strong></summary>
 
-- [Example Code in python file](tests/test_chatrex_grounded_conversation.py)
-
 ```python
 import torch
 from PIL import Image