Spaces:
Running
Per-tag thresholds?
As I understand it, the model consists of independent classifiers, one for each tag. At the same time, different tags have different amounts of (positive) images in the training data, and the model seems to give lower scores for less frequently used tags.
Based on that, would it make sense to use different thresholds for different tags in order to increase the accuracy?
Yeah if you have enough images for a particular tag it would be possible to calibrate the output using something like the ml-insights package.
The EVA02 model actually has some funky stuff going on in the bias weights of the last dense layer to this effect, you can check out how that works for you to begin with. The "calibration" on that model was targeted towards giving an equilibrium between Precision and Recall for all tags at around 0.5 threshold, meaning higher than 0.5 you're skewing towards more precision, lower than 0.5 you're going towards higher recall.