Upload folder using huggingface_hub

README.md

@@ -0,0 +1,44 @@
+---
+base_model:
+- openai/clip-vit-base-patch32
+datasets:
+- nateraw/rendered-sst2
+metrics:
+- accuracy
+---
+
+# Model Card
+
+## Model Details
+
+- Architecture: ViT-Base with patch size 32
+- Training Data: rendered-sst2 dataset
+
+## Training Details
+
+  Adam Optimizer with a constant learning rate 1e-5 for 4000 steps training (batch_size=32).
+  Only the vision encoder is fine-tuned.
+
+## Evaluation Results
+
+- pre-trained: 0.5856747031211853
+- fine-tuned: 0.6964021921157837
+
+## Usage
+
+load vision model
+
+```python
+from transformers import CLIPVisionModel
+
+vision_model = CLIPVisionModel.from_pretrained('tanganke/clip-vit-base-patch32_rendered-sst2')
+```
+
+substitute the vision encoder of clip
+
+```python
+from transformers import CLIPModel
+
+clip_model = CLIPModel.from_pretrained("openai/clip-vit-base-patch32")
+clip_model.vision_model.load_state_dict(vision_model.vision_model.state_dict())
+```

config.json

@@ -0,0 +1,23 @@
+{
+  "_name_or_path": "results/temp/",
+  "architectures": [
+    "CLIPVisionModel"
+  ],
+  "attention_dropout": 0.0,
+  "dropout": 0.0,
+  "hidden_act": "quick_gelu",
+  "hidden_size": 768,
+  "image_size": 224,
+  "initializer_factor": 1.0,
+  "initializer_range": 0.02,
+  "intermediate_size": 3072,
+  "layer_norm_eps": 1e-05,
+  "model_type": "clip_vision_model",
+  "num_attention_heads": 12,
+  "num_channels": 3,
+  "num_hidden_layers": 12,
+  "patch_size": 32,
+  "projection_dim": 512,
+  "torch_dtype": "float32",
+  "transformers_version": "4.39.1"
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1eb80c18ede1b025c857ac5d68e09c85e37b4b675ebed3c245cdde0c8e228fef
+size 349847824