🚩 Report: Ethical issue(s)
Matt and Hugging Face,
Given the claims and counterclaims on this model, given Matt's silence and inaction to address reports on these models, it's overdue for both Hugging Face and Matt to resolve the problem immediately and either fix the problems, delete the bad/duplicate repos and clear up the confusion regarding this.
It looks bad for both HF and Matt. I personally give Matt the benefit of the doubt as to intention, but as to results and his inaction given the 3rd party testing results, I am disappointed that Matt has not done anything to resolve the issues.
You shouldn't be allowed to make the claims, publish the results you have published and have so many independent tests show that this appears to be fraudulent work.
I am asking for both parties to please resolve this, one way or another, quickly. If Matt actually has a functioning model, delete the trash repos and upload a model that functions as claimed. Assuming Matt has a functional model that scores per his claims, he should place on the functional repo model card an explanation of the problems with the early uploads.
Given the claims, HF should benchmark the model that is supposed to be an advancement and if it under-performs Matt's claims, delete the repo and take appropriate actions.
Matt would you do your part to clear things up? The inaction on your part, the poor test scores by 3rd parties, gives credence to the claim that this is fraudulent work.