Salesforce

/

xgen-7b-8k-base

like 316

help on this would be very useful. Thanks!

following this.

Hopefully we get a resource for a straight-forward fine-tuning process for XGen-7b like in llama.

Hi everyone

As x-gen has the same architecture as Llama you can easily extend this gist: https://gist.github.com/younesbelkada/9f7f75c94bdc1981c8ca5cc937d4a4da to make use of x-gen and run it on your custom dataset.
Since the model is a 7b model you can also fine-tune it easily on a free-tier google colab instance by following for example this tutorial: https://colab.research.google.com/drive/1DNenc5BpdqaS10prtklYyIe9qW_7gUnb but run it on xgen instead.

Thanks.