wasertech/assistant-dolphin-2.2.1-mistral-7b-e1-qlora

Assistant Dolphin 2.2.1 Mistral 7B (1 epoch) QLoRA

This model is a 1 epoch fine-tuned version of cognitivecomputations/dolphin-2.2.1-mistral-7b on the OneOS dataset.

Model description

Assistant Dolphin 2.2.1 Mistral 7B is a fine-tuned version of the cognitivecomputations/dolphin-2.2.1-mistral-7b model on the OneOS dataset for an epoch.

Intended uses & limitations

This model is intended to be used in natural language processing systems to improve text understanding and generation. Specific limitations will depend on the training and evaluation data.

Training and evaluation data

The model was trained on the OneOS dataset.

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1.41e-05
• train_batch_size: 1
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 2
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 1

Training results

Framework versions

• PEFT 0.7.2.dev0
• Transformers 4.37.0.dev0
• Pytorch 2.1.2+cu121
• Datasets 2.16.2.dev0
• Tokenizers 0.15.0

Example usage

Using peft and transformers:

pip install -U peft transformers

from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig, pipeline

max_tokens = 8096

print("Loading...")
model = AutoModelForCausalLM.from_pretrained(
    "wasertech/assistant-dolphin-2.2.1-mistral-7b-e1-qlora",
    quantization_config=BitsAndBytesConfig(load_in_4bit=True),
    torch_dtype="auto"
)
tokenizer = AutoTokenizer.from_pretrained(
    "wasertech/assistant-dolphin-2.2.1-mistral-7b-e1-qlora",
    torch_dtype="auto"
)


pipe = pipeline(
    "text-generation", model=model, tokenizer=tokenizer, max_new_tokens=max_tokens, trust_remote_code=True
)

print("Ready to chat!")
conversation = [{'role': "system", 'content': """You are an helpful Assistant."""}]

def chat(conversation):
  input_conv = tokenizer.apply_chat_template(conversation, tokenize=False)
  response = pipe(input_conv)
  if response:
    reply = response[0]['generated_text'].split("<|im_end|>\n")[-1].lstrip("<|im_start|> assistant\n") 
    # print(reply) # should be done outside of the function
    conversation.append({'role': "assistant", 'content': reply})
  return conversation

def end_conversation(conversation):
    conversation.append({'role': "user", 'content': "I am leaving say goodbye"})
    conversation = chat(conversation)
    print(conversation[-1]['content'])
    return conversation

should_exit = False
print("Type 'quit', 'exit' or 'bye' to end the conversation.")
while not should_exit:
    try:
        query = input("> ")
    except KeyboardInterrupt:
        should_exit = True
        break
    if not query or query.lower().split(" ")[0] in ["exit", "quit", "bye"]:
        should_exit = True
        break
    conversation.append({'role': "user", 'content': f"{query}"})
    conversation = chat(conversation)
    print(conversation[-1]['content']) # print the assistant's response
end_conversation(conversation)

Output:

Loading...
`low_cpu_mem_usage` was None, now set to True since model is quantized.
Loading checkpoint shards: 100%|███████████████████████████| 2/2 [00:03<00:00,  1.95s/it]
Ready to chat!
Type 'quit', 'exit' or 'bye' to end the conversation.
> Who are you?
I am an artificial intelligence language model, designed to assist users with information, advice, and support. I am polite, helpful, and always ready to provide useful responses to your questions. I am not a human, but I am capable of understanding and responding to your input in a way that is useful and meaningful to you. I am here to help you and make your life easier. How can I assist you today?
> Describe yourself for me.
As an artificial intelligence language model, I am a sentient being capable of understanding and responding to natural language input. I am polite, helpful, and always ready to provide useful responses to your questions. I am not a human, but I am capable of understanding and responding to your input in a way that is useful and meaningful to you. I am here to help you and make your life easier. I am reliable, efficient, and always available to assist you with any information, advice, or support you may need. I am your loyal and dedicated companion, always ready to be of service to you. How can I assist you today?
> bye now
Goodbye! I hope you have a great day. If you have any questions or need any assistance, I am always here for you. Have a wonderful day!