Fine-tuning Florence-2 - Microsoft's Cutting-edge Vision Language Models
•
179
Piotr Skalski PRO
SkalskiP
AI & ML interests
Computer Vision | Multimodality
Recent Activity
liked
a model
10 days ago
SkalskiP/paligemma2_latex_ocr_v5
updated
a model
11 days ago
SkalskiP/paligemma2_latex_ocr_v5
updated
a model
11 days ago
SkalskiP/paligemma2_latex_ocr_v4
Articles
Organizations
Posts
2
Post
YOLO-World: Real-Time, Zero-Shot Object Detection 🔥 🔥 🔥
YOLO-World was designed to solve a limitation of existing zero-shot object detection models: speed. Whereas other state-of-the-art models use Transformers, a powerful but typically slower architecture, YOLO-World uses the faster CNN-based YOLO architecture.
YOLO-World provides three models: small with 13M (re-parametrized 77M), medium with 29M (re-parametrized 92M), and large with 48M (re-parametrized 110M) parameters.
The YOLO-World team benchmarked the model on the LVIS dataset and measured their performance on the V100 without any performance acceleration mechanisms like quantization or TensorRT.
According to the paper, YOLO-World reached 35.4 AP with 52.0 FPS for the L version and 26.2 AP with 74.1 FPS for the S version. While the V100 is a powerful GPU, achieving such high FPS on any device is impressive.
- 🔗 YOLO-World arXiv paper: https://lnkd.in/ddRBKCCX
- 🔗 my YOLO-World technical report: https://blog.roboflow.com/what-is-yolo-world
- 🤗 YOLO-World space: SkalskiP/YOLO-World
YOLO-World was designed to solve a limitation of existing zero-shot object detection models: speed. Whereas other state-of-the-art models use Transformers, a powerful but typically slower architecture, YOLO-World uses the faster CNN-based YOLO architecture.
YOLO-World provides three models: small with 13M (re-parametrized 77M), medium with 29M (re-parametrized 92M), and large with 48M (re-parametrized 110M) parameters.
The YOLO-World team benchmarked the model on the LVIS dataset and measured their performance on the V100 without any performance acceleration mechanisms like quantization or TensorRT.
According to the paper, YOLO-World reached 35.4 AP with 52.0 FPS for the L version and 26.2 AP with 74.1 FPS for the S version. While the V100 is a powerful GPU, achieving such high FPS on any device is impressive.
- 🔗 YOLO-World arXiv paper: https://lnkd.in/ddRBKCCX
- 🔗 my YOLO-World technical report: https://blog.roboflow.com/what-is-yolo-world
- 🤗 YOLO-World space: SkalskiP/YOLO-World
Post
Real-Time Vehicle Speed Estimation Tutorial 🚗💨💨💨
TL;DR: Watch the tutorial here: https://www.youtube.com/watch?v=uWP6UjDeZvY
Key Steps:
1. Vehicle Detection: Before we jump into speed estimation, we begin by detecting moving vehicles. I demonstrate this using YOLOv8, deployed through the Inference pip package.
2. Tracking with ByteTrack: For effective object tracking, ByteTrack is my tool of choice. It assigns a unique ID to each vehicle, which is essential for accurately monitoring the distance each car travels. This forms the cornerstone of our speed calculation process.
3. Distance Calculation Complexities: Calculating traveled distance can be tricky due to perspective distortion from the camera. A car moving at a constant speed will appear to move a different number of pixels in the image, depending on its distance from the camera.
4. Vehicle Positioning: We can accurately pinpoint each vehicle's position within our monitored area. By representing each vehicle with x and y coordinates in meters, we can compare its current and past positions, paving the way for calculating its speed.
5. We store the position of each car in the last second, calculate the offset, and divide it by the time delta to get the local speed.
- 🔗 tutorial: https://www.youtube.com/watch?v=uWP6UjDeZvY
- 🔗 code: https://github.com/roboflow/supervision/tree/develop/examples/speed_estimation
TL;DR: Watch the tutorial here: https://www.youtube.com/watch?v=uWP6UjDeZvY
Key Steps:
1. Vehicle Detection: Before we jump into speed estimation, we begin by detecting moving vehicles. I demonstrate this using YOLOv8, deployed through the Inference pip package.
2. Tracking with ByteTrack: For effective object tracking, ByteTrack is my tool of choice. It assigns a unique ID to each vehicle, which is essential for accurately monitoring the distance each car travels. This forms the cornerstone of our speed calculation process.
3. Distance Calculation Complexities: Calculating traveled distance can be tricky due to perspective distortion from the camera. A car moving at a constant speed will appear to move a different number of pixels in the image, depending on its distance from the camera.
4. Vehicle Positioning: We can accurately pinpoint each vehicle's position within our monitored area. By representing each vehicle with x and y coordinates in meters, we can compare its current and past positions, paving the way for calculating its speed.
5. We store the position of each car in the last second, calculate the offset, and divide it by the time delta to get the local speed.
- 🔗 tutorial: https://www.youtube.com/watch?v=uWP6UjDeZvY
- 🔗 code: https://github.com/roboflow/supervision/tree/develop/examples/speed_estimation
Collections
3
spaces
13
models
8
SkalskiP/paligemma2_latex_ocr_v5
Updated
•
9
•
2
SkalskiP/paligemma2_latex_ocr_v4
Updated
•
4
SkalskiP/paligemma2_latex_ocr_v3
Updated
•
4
SkalskiP/paligemma2_latex_ocr_v2
Updated
•
9
SkalskiP/paligemma2_latex_ocr
Updated
•
8
•
2
SkalskiP/paligemma-numbers-finetune
Updated
•
2
SkalskiP/paligemma-finetune-2
Updated
SkalskiP/paligemma-finetune
Updated
•
2
datasets
None public yet