Hugging Face's logo

Spaces:
ArnoBen
/
BlinkBlur
Running

App Files Community
BlinkBlur / README.md
ArnoBen's picture
ArnoBen
Upload 31 files
e99cf64
preview code
|
raw
history blame
3.05 kB
metadata
title: BlinkBlur
emoji: 👀
sdk: gradio
sdk_version: 3.17.1
app_file: app.py

Eye Blink / Blurriness detectors

Try BlinkBlur on HuggingFace 🤗

Assumptions

  • Input images are in RGB format, in a format readable by OpenCV/PIL
  • For live webcam detection, the user has at least 1 camera
  • The person's face is not rotated
  • The person's face is not too close to the sides of the frame (this can lead to an error during cropping)

Decisions

  • If several persons appear, only the first one detected will be processed
  • I use MediaPipe's Face Detection to get eyes coordinates, then crop the eyes region and feed them into a custom model.
  • The blurriness estimation is based on an edge detector (variance of laplacian). The result is based on a manual and arbitrary threshold of 100.

Packages 

gradio==3.17.1
matplotlib==3.5.2
mediapipe==0.9.0.1
numpy==1.21.5
opencv-contrib-python==4.6.0.66
opencv-python==4.6.0.66
pandas==1.3.5
pillow==9.3.0
pyqt5==5.15.8
pytorch-lightning==1.9.0
scikit-learn==1.0.2
seaborn==0.12.2
torch==1.13.1
torchvision==0.14.1
tqdm==4.64.1

How to use 

usage: main.py [-h] [-w] [-p PATH] [-e] [-b]

optional arguments:
  -h, --help            show this help message and exit
  -w, --webcam          Runs eye blink detection on the user's webcam.
                        Overrides other options.
  -p PATH, --path PATH  path of image to process
  -e, --eyes            Detects eye blink on the input image
  -b, --blur            Estimates image blur

You can try with images available in examples/.

Command example:\

python main.py --path examples/close.jpg --eyes
python main.py -p examples/open_far.jpg -e
python main.py -p examples/blurry_1.jpg --blur
python main.py -p examples/sharp_1.jpg -b

Metrics

Closed-eyes detector

Here are some evaluation metrics on the model, calculated with the code in closed_eyes_detection/training/test.py.

Metrics

The confusion matrix has been calculated on a classification threshold of 0.49, this corresponds to the red cross on the curve.

My assumption is that we want to make sure we don't keep a picture where the user is blinking so we need to be more strict on it to maximize recall.

Blurriness estimator

The blurriness estimator uses the variance of the laplacian to assess the blurriness.

Here are some metrics and statistics on the blurriness detection task, calculated with the code in blurriness_estimation/calculate_metrics.py.

The red vertical line indicates the chosen threshold, given by the following metrics:

Metrics

Datasets