Some notes/feedback
#15
by
thomwolf
HF staff
- opened
Maybe we could:
- Move examples to the right of the input boxes?
- Have an option to give a custom generation model handle on the hub?
Hi @thomwolf ,
Thanks for the feedback.
- I agree that would be nice. @sdiazlor is working on a slight redesign and improvement of the task.
- FYI
@sdiazlor
is also working on adding
evaluation/judgingto the space. (this task will use the proposed space next to the input boxes to preview the dataset to evaluate, hence we had not moved them there at first. - Adding custom generation model handles is nice but somewhat complex due to the fact there is no clear selection of models that are available as serverless inference endpoints with a high enough rate limit (70B is lower than 7B and might not scale). Additionally, we need a specific query template-logic to work with Magpie generation. We can for sure add an easy dropdown to add some Qwen an other llama models because these have already been implemented.
- After this redesign, I will for sure add a more robust choice of models and stores because I want to push this through GH and a pip install as a more formal "package" to easier get traction through socials and SEO.