TokisakiKurumi

transZ

AI & ML interests

NLP

Recent Activity

Organizations

None yet

transZ's activity

reacted to davanstrien's post with πŸ‘ 9 months ago
view post
Post
KTO offers an easier way to preference train LLMs (only πŸ‘πŸ‘Ž ratings are required). As part of #DataIsBetterTogether, I've written a tutorial on creating a preference dataset using Argilla and Spaces.

Using this approach, you can create a dataset that anyone with a Hugging Face account can contribute to 🀯

See an example of the kind of Space you can create following this tutorial here: davanstrien/haiku-preferences

πŸ†• New tutorial covers:
πŸ’¬ Generating responses with open models
πŸ‘₯ Collecting human feedback (do you like this model response? Yes/No)
πŸ€– Preparing a TRL-compatible dataset for training aligned models

Check it out here: https://github.com/huggingface/data-is-better-together/tree/main/kto-preference
  • 2 replies
Β·
updated a model about 1 year ago
New activity in transZ/reword about 1 year ago