Fine Tuning
Can fine tuning be done on this model, which is a falcon-7b fine tuning? Is there documentation about it? Examples?
Hey @carlosmtoro ! Some helpful resources about fine-tuning Falcon that helped me: https://huggingface.co/blog/falcon and https://github.com/rmihaylov/falcontune.
The tunes appear to work very well
Can you explain a bit more how the different variants are to be chosen from ? (sft-mix etc)
Also you have a system prompt but it's not documented, is that just reserved for the future or is it part of the fine tuning ?
@cmp-nct
(system prompt) https://github.com/LAION-AI/Open-Assistant/blob/main/inference/worker/chat_chain_prompts.py
Example: (how I understand)prompt="""
<|system|>
You're a helpful assistant
<|endoftext|>
<|prompter|>
How to be a successful CEO?
<|endoftext|>
<|assistant|>
"""