|
--- |
|
license: apache-2.0 |
|
--- |
|
|
|
# InsectSAM: Insect Segmentation and Monitoring |
|
|
|
<p align="left"> |
|
<a href="" rel="noopener"> |
|
<img width=200px height=200px src="https://i.imgur.com/hjWgAN9.png alt="Project logo"></a> |
|
</p> |
|
|
|
## Overview |
|
|
|
InsectSAM is an advanced machine learning model tailored for the https://diopsis.eu camera systems and https://www.arise-biodiversity.nl/, dedicated to Insect Biodiversity Detection and Monitoring in the Netherlands. Built on Meta AI's `segment-anything` model, InsectSAM is fine-tuned to be accurate at segmenting insects from complex backgrounds, enhancing the accuracy and efficiency of biodiversity monitoring efforts. |
|
|
|
## Purpose |
|
|
|
This model has been meticulously trained to identify and segment insects against a variety of backgrounds that might otherwise confuse traditional algorithms. It is specifically designed to adapt to future changes in background environments, ensuring its long-term utility in the DIOPSIS / ARISE project. |
|
|
|
## Model Architecture |
|
|
|
InsectSAM utilizes the advanced capabilities of the `segment-anything` architecture, enhanced by our custom training on an insect-centric dataset. The model is further refined by integrating with GroundingDINO, improving its ability to distinguish fine details and subtle variations in insect appearances. |
|
|
|
## Quick Start |
|
|
|
### Prerequisites |
|
|
|
- Python |
|
- Hugging Face Transformers |
|
- PyTorch |
|
|
|
### Usage |
|
|
|
#### Install |
|
``` bash |
|
!pip install --upgrade -q git+https://github.com/huggingface/transformers |
|
!pip install torch |
|
``` |
|
#### Load model directly via HF Transformers 🤗 |
|
``` bash |
|
from transformers import AutoProcessor, AutoModelForMaskGeneration |
|
|
|
processor = AutoProcessor.from_pretrained("martintmv/InsectSAM") |
|
model = AutoModelForMaskGeneration.from_pretrained("martintmv/InsectSAM") |
|
``` |
|
|
|
### Notebooks |
|
|
|
Three Jupyter notebooks are provided to demonstrate the model's capabilities and its integration with GroundingDINO: |
|
|
|
- **InsectSAM.ipynb**: Covers the training process, from data preparation to model evaluation. |
|
- **InsectSAM_GroundingDINO.ipynb**: Demonstrates how InsectSAM is combined with GroundingDINO for enhanced segmentation performance. |
|
- **Run_InsectSAM_Inference_Transformers.ipynb**: Run InsectSAM using Transformers. |
|
|
|
Check out the notebooks on RB-IBDM's GitHub page - https://github.com/martintmv-git/RB-IBDM/tree/main/InsectSAM |
