Image-Text-to-Text
PEFT
Safetensors
English
æLtorio commited on
Commit
5767cfc
1 Parent(s): 7875fd5
Files changed (1) hide show
  1. README.md +19 -8
README.md CHANGED
@@ -14,6 +14,12 @@ pipeline_tag: image-to-text
14
 
15
  ![Stage](https://img.shields.io/badge/stage-early%20development-yellow)![License](https://img.shields.io/badge/license-Apache%202.0-blue)![Contributors Welcome](https://img.shields.io/badge/contributors-welcome-brightgreen)[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/#fileId=https://huggingface.co/eltorio/IDEFICS3_ROCO/blob/main/ROCO-idefics3.ipynb)
16
 
 
 
 
 
 
 
17
  ## A Fine-tuned Radiology-focused Model based on Hugging Face's Idefics3 Model
18
 
19
  This repository contains a fine-tuned version of the Hugging Face [Idefics3-8B-Llama3](https://huggingface.co/HuggingFaceM4/Idefics3-8B-Llama3) model, built on top of the Meta Llama 3.1 8B architecture. Our model, `IDEFICS3_ROCO`, has been fine-tuned on the [Radiology Objects in Context (ROCO)](https://huggingface.co/datasets/eltorio/ROCO-radiology) dataset, a large-scale medical and multimodal imaging collection.
@@ -23,14 +29,14 @@ This repository contains a fine-tuned version of the Hugging Face [Idefics3-8B-L
23
  * **Base Model:** Idefics3-8B-Llama3
24
  * **Fine-tuning Dataset:** Radiology Objects in Context (ROCO)
25
  * **License:** Apache-2.0
26
- * **Current Status:** Fine-tuning process is currently halted at checkpoint 2350 (out of 12,267) (in branch bug-restart) due to limitations with Colab Free T4 GPU unit. Contributions to complete the fine-tuning process are welcome!
27
 
28
  ### Training Progress Status
29
 
30
- * Current checkpoint: 2350/12267 (~19% completed) (in branch bug-restart)
31
- * Estimated remaining GPU time: ~57 hours
32
  * Hardware requirements: T4 GPU with >16GB VRAM
33
- * Last update: november, 8th 2024
34
 
35
  ### Fine-tuning Code
36
 
@@ -60,7 +66,7 @@ If you use this model in your work, please cite the original Idefics3 model and
60
 
61
  2. **Getting Started**
62
  * Fork the repository
63
- * Resume from checkpoint 2350/12267 (in branch bug-restart)
64
  * Follow instructions in [ROCO-idefics3.ipynb](https://huggingface.co/eltorio/IDEFICS3_ROCO/blob/main/ROCO-idefics3.ipynb) [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/#fileId=https://huggingface.co/eltorio/IDEFICS3_ROCO/blob/main/ROCO-idefics3.ipynb)
65
 
66
  3. **Contact**
@@ -68,14 +74,19 @@ If you use this model in your work, please cite the original Idefics3 model and
68
 
69
  ### Docker Image
70
 
71
- A AI training docker image is available for this model. The image and includes all necessary dependencies to run the fine-tuning process. The image is available on Docker Hub:
 
 
 
 
72
 
73
  ```bash
74
- docker run --user=42420:42420 -it sctg/roco-idefics3:latest /start.sh hf_TOKEN
 
75
  ```
76
 
77
  The Dockerfile is available in the [IDEFICS_ROCO repository](https://huggingface.co/eltorio/IDEFICS3_ROCO/blob/main/Dockerfile).
78
 
79
  ### Acknowledgments
80
 
81
- This work was made possible by the [Hugging Face Transformers](https://huggingface.co/) library and the [ROCO-radiology dataset](https://huggingface.co/datasets/eltorio/ROCO-radiology).
 
14
 
15
  ![Stage](https://img.shields.io/badge/stage-early%20development-yellow)![License](https://img.shields.io/badge/license-Apache%202.0-blue)![Contributors Welcome](https://img.shields.io/badge/contributors-welcome-brightgreen)[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/#fileId=https://huggingface.co/eltorio/IDEFICS3_ROCO/blob/main/ROCO-idefics3.ipynb)
16
 
17
+ ## Star the project
18
+
19
+ **If you appreciate my work, please consider giving it a star! 🤩**
20
+ **I'm also looking for donations of free GPU time to complete the fine-tuning process.**
21
+ **Please contact me if you can help! 🙏**
22
+
23
  ## A Fine-tuned Radiology-focused Model based on Hugging Face's Idefics3 Model
24
 
25
  This repository contains a fine-tuned version of the Hugging Face [Idefics3-8B-Llama3](https://huggingface.co/HuggingFaceM4/Idefics3-8B-Llama3) model, built on top of the Meta Llama 3.1 8B architecture. Our model, `IDEFICS3_ROCO`, has been fine-tuned on the [Radiology Objects in Context (ROCO)](https://huggingface.co/datasets/eltorio/ROCO-radiology) dataset, a large-scale medical and multimodal imaging collection.
 
29
  * **Base Model:** Idefics3-8B-Llama3
30
  * **Fine-tuning Dataset:** Radiology Objects in Context (ROCO)
31
  * **License:** Apache-2.0
32
+ * **Current Status:** Fine-tuning process is finished. Contributions to complete the fine-tuning / vallidation / test processes are welcome!
33
 
34
  ### Training Progress Status
35
 
36
+ * Current checkpoint: 12267 (100% completed)
37
+ * Estimated remaining GPU time: 0 hours
38
  * Hardware requirements: T4 GPU with >16GB VRAM
39
+ * Last update: november, 12th 2024
40
 
41
  ### Fine-tuning Code
42
 
 
66
 
67
  2. **Getting Started**
68
  * Fork the repository
69
+ * Resume from checkpoint 12267
70
  * Follow instructions in [ROCO-idefics3.ipynb](https://huggingface.co/eltorio/IDEFICS3_ROCO/blob/main/ROCO-idefics3.ipynb) [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/#fileId=https://huggingface.co/eltorio/IDEFICS3_ROCO/blob/main/ROCO-idefics3.ipynb)
71
 
72
  3. **Contact**
 
74
 
75
  ### Docker Image
76
 
77
+ A AI training docker image is available for this model. The image and includes all necessary dependencies to run the fine-tuning process.
78
+ You need to set the `HF_TOKEN` environment variable to your Hugging Face API token.
79
+ You also need to have NVidia Docker container runtime installed.
80
+ Finnaly, you need to run the container with GPU support with `--gpus all` option.
81
+ The image is available on Docker Hub:
82
 
83
  ```bash
84
+ export HF_TOKEN=hf_some_token
85
+ docker run --gpus all --user=42420:42420 -e HF_TOKEN=$HF_TOKEN -it sctg/roco-idefics3:latest bash -i /start.sh $HF_TOKEN
86
  ```
87
 
88
  The Dockerfile is available in the [IDEFICS_ROCO repository](https://huggingface.co/eltorio/IDEFICS3_ROCO/blob/main/Dockerfile).
89
 
90
  ### Acknowledgments
91
 
92
+ This work was made possible by the [Hugging Face Transformers](https://huggingface.co/) library and the [ROCO-radiology dataset](https://huggingface.co/datasets/eltorio/ROCO-radiology).