metadata

datasets:
  - tatsu-lab/alpaca
  - the_pile
language:
  - en
library_name: transformers
tags:
  - peft
  - lora
  - instruct
  - alpaca
  - gptj

Instruct-GPT-J "Vicuña"

A demo that runs in free Google Colab can be run here: https://bit.ly/3K1P4PQ just change the model dropdown to the name of this model.

The EleutherAI/gpt-j-6B model finetuned on the Alpaca instruction dataset with low rank adaptation. This is not a model from Eleuther but a personal project.

Don't knock LoRA, all it is is finetuning how the internal representations should change (simplified, the residual of the weights) instead of finetuning just the internal representations! All the previous weights are in tact meaning LoRA tuning makes the model less likely to forget what it was trained on, and also less likely to push the model into mode collapse. Check table 2 of the LoRA paper and you can see that LoRA many times outperforms traditional finetuning as well.

Use:

import torch
from peft import PeftModel, PeftConfig
from transformers import AutoModelForCausalLM, AutoTokenizer

peft_model_id = "crumb/Instruct-GPT-J"
config = PeftConfig.from_pretrained(peft_model_id)
model = AutoModelForCausalLM.from_pretrained(config.base_model_name_or_path, return_dict=True, load_in_8bit=True, device_map='auto', revision='sharded')
tokenizer = AutoTokenizer.from_pretrained(config.base_model_name_or_path)

# Load the Lora model
model = PeftModel.from_pretrained(model, peft_model_id)

# This example is in the alpaca training set
batch = tokenizer("Below is an instruction that describes a task. Write a response that appropriately completes the request. ### Instruction: How can we reduce air pollution? ### Response:", return_tensors='pt')

with torch.cuda.amp.autocast():
  output_tokens = model.generate(**batch, max_new_tokens=50)

print(tokenizer.decode(output_tokens[0], skip_special_tokens=True))
# One way to reduce air pollution is to reduce the amount of emissions from vehicles. This can be done by implementing stricter emission standards and increasing the use of electric vehicles. Another way to reduce air pollution is to reduce the amount of waste produced by industries.

A function to turn an instruction into a prompt for the model could be written as follows

def prompt(instruction, input=''):
  if input=='':
    return f"Below is an instruction that describes a task. Write a response that appropriately completes the request. ### Instruction: {instruction} ### Response: "
  return f"Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request. ### Instruction: {instruction} ### Input: {input} ### Response: "

Where input would be an input for the model to act on based on the instruction.

citations

@misc{gpt-j,
  author = {Wang, Ben and Komatsuzaki, Aran},
  title = {{GPT-J-6B: A 6 Billion Parameter Autoregressive Language Model}},
  howpublished = {\url{https://github.com/kingoflolz/mesh-transformer-jax}},
  year = 2021,
  month = May
}

@misc{alpaca,
  author = {Rohan Taori and Ishaan Gulrajani and Tianyi Zhang and Yann Dubois and Xuechen Li and Carlos Guestrin and Percy Liang and Tatsunori B. Hashimoto },
  title = {Stanford Alpaca: An Instruction-following LLaMA model},
  year = {2023},
  publisher = {GitHub},
  journal = {GitHub repository},
  howpublished = {\url{https://github.com/tatsu-lab/stanford_alpaca}},
}