I am trying to use this for some coding tasks (in particular for Java language) and I am getting borderline nonsensical results. Has someone else faced this issue?
#72
by
anshumankmr
- opened
Using Phi3 Mini 128k, I am trying to generate readme file using Load QA Chain (Map Reduce) + some codebases in Java and I am getting very poor and slow results. Can someone confirm if they have faced similar such issues/
Not surprising. Documentation shows Phi-3 favors Python.
https://huggingface.co/microsoft/Phi-3-mini-128k-instruct#responsible-ai-considerations
Limited Scope for Code: Majority of Phi-3 training data is based in Python and use common packages such as "typing, math, random, collections, datetime, itertools". If the model generates Python scripts that utilize other packages or scripts in other languages, we strongly recommend users manually verify all API uses.
This is really unfortunate. But thanks for the link.