Hugging Face's logo

eltorio
/
IDEFICS3_ROCO

Image-Text-to-Text
PEFT
Safetensors
English
Model card Files Community
IDEFICS3_ROCO / README.md
eltorio's picture
eltorio
Create README.md
7f849e1 verified
preview code
|
raw
history blame
3.88 kB
metadata
license: apache-2.0
datasets:
  - eltorio/ROCO-radiology
language:
  - en
  - fr
base_model:
  - HuggingFaceM4/Idefics3-8B-Llama3

IDEFICS3_ROCO

StageLicenseContributors WelcomeOpen In Colab

A Fine-tuned Radiology-focused Model based on Hugging Face's Idefics3 Model

This repository contains a fine-tuned version of the Hugging Face Idefics3-8B-Llama3 model, built on top of the Meta 3.1 8B architecture. Our model, IDEFICS3_ROCO, has been fine-tuned on the Radiology Objects in Context (ROCO) dataset, a large-scale medical and multimodal imaging collection.

Model Information

  • Base Model: Idefics3-8B-Llama3
  • Fine-tuning Dataset: Radiology Objects in Context (ROCO)
  • License: Apache-2.0
  • Current Status: Fine-tuning process is currently halted at checkpoint 640 (out of 24,000) due to limitations with Colab Free T4 GPU unit. Contributions to complete the fine-tuning process are welcome!

Training Progress Status

  • Current checkpoint: 620-640/24000 (~2.7% completed)
  • Estimated remaining GPU time: ~57 hours
  • Hardware requirements: T4 GPU with >16GB VRAM
  • Last update: november, 7th 2021

Fine-tuning Code

The fine-tuning code is available as a Jupyter Notebook in the ROCO-radiology dataset repository on Hugging Face:

The Junyper Notebook Open In Colab contains the code to fine-tune the Idefics3-8B-Llama3 model on the ROCO dataset. The fine-tuning process is currently halted at checkpoint 640 (out of 24,000) due to limitations with Colab Free T4 GPU unit. Contributions to complete the fine-tuning process are welcome!

Contributions Welcome

If you have the resources to complete the fine-tuning process, we would appreciate your contribution. Please fork this repository, finish the fine-tuning process, and submit a pull request with your updates.

Citation

If you use this model in your work, please cite the original Idefics3 model and our fine-tuned model:

Contribution Guide

  1. Technical Requirements

    • Access to powerful GPU (T4, V100, A100 or equivalent)
    • Python environment with PyTorch
    • Disk space: ~50GB

  2. Getting Started

  3. Contact

    • For questions: [link to issues/discussions]

Acknowledgments

This work was made possible by the Hugging Face Transformers library and the ROCO-radiology dataset.