File size: 3,048 Bytes
de7115c e99cf64 de7115c e99cf64 de7115c e99cf64 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 |
---
title: BlinkBlur
emoji: 👀
sdk: gradio
sdk_version: 3.17.1
app_file: app.py
---
# Eye Blink / Blurriness detectors
### Try [BlinkBlur on HuggingFace 🤗](https://huggingface.co/spaces/ArnoBen/BlinkBlur)
## Assumptions
- Input images are in RGB format, in a format readable by OpenCV/PIL
- For live webcam detection, the user has at least 1 camera
- The person's face is not rotated
- The person's face is not too close to the sides of the frame (this can lead to an error during cropping)
## Decisions
- If several persons appear, only the first one detected will be processed
- I use MediaPipe's Face Detection to get eyes coordinates, then crop the eyes region and feed them into a custom model.
- The blurriness estimation is based on an edge detector (variance of laplacian).
The result is based on a manual and arbitrary threshold of 100.
## Packages
```requirements.txt
gradio==3.17.1
matplotlib==3.5.2
mediapipe==0.9.0.1
numpy==1.21.5
opencv-contrib-python==4.6.0.66
opencv-python==4.6.0.66
pandas==1.3.5
pillow==9.3.0
pyqt5==5.15.8
pytorch-lightning==1.9.0
scikit-learn==1.0.2
seaborn==0.12.2
torch==1.13.1
torchvision==0.14.1
tqdm==4.64.1
```
## How to use
```commandline
usage: main.py [-h] [-w] [-p PATH] [-e] [-b]
optional arguments:
-h, --help show this help message and exit
-w, --webcam Runs eye blink detection on the user's webcam.
Overrides other options.
-p PATH, --path PATH path of image to process
-e, --eyes Detects eye blink on the input image
-b, --blur Estimates image blur
```
You can try with images available in `examples/`.
Command example:\
```commandline
python main.py --path examples/close.jpg --eyes
python main.py -p examples/open_far.jpg -e
python main.py -p examples/blurry_1.jpg --blur
python main.py -p examples/sharp_1.jpg -b
```
## Metrics
### Closed-eyes detector
Here are some evaluation metrics on the model, calculated with the code in `closed_eyes_detection/training/test.py`.
The confusion matrix has been calculated on a classification threshold of 0.49,
this corresponds to the red cross on the curve.
My assumption is that we want to make sure we don't keep a picture where the user is blinking
so we need to be more strict on it to maximize recall.
### Blurriness estimator
The blurriness estimator uses the variance of the laplacian to assess the blurriness.
Here are some metrics and statistics on the blurriness detection task,
calculated with the code in `blurriness_estimation/calculate_metrics.py`.
<img src="assets/distribution.png" width="400"/>
The red vertical line indicates the chosen threshold, given by the following metrics:
## Datasets
- I trained the closed eye detection model on [this kaggle dataset](https://www.kaggle.com/datasets/tauilabdelilah/mrl-eye-dataset).
- The blurriness estimator was tested on [this kaggle dataset](https://www.kaggle.com/datasets/kwentar/blur-dataset) |