cognitivetech's picture
Update README.md
9dc9cad verified
---
language:
- en
license: apache-2.0
tags:
- Notes
- Summary
- text-generation-inference
- transformers
- unsloth
- mistral
- trl
base_model: cognitivecomputations/samantha-mistral-instruct-7b
---
# Use these instead
- https://huggingface.co/cognitivetech/Mistral-7b-Inst-0.2-Bulleted-Notes_GGUF
- https://huggingface.co/cognitivetech/Mistral-7B-Inst-0.2-Bulleted-Notes
- https://huggingface.co/cognitivetech/Mistral-7B-Inst-0.2_Bulleted-Notes_LoRA
## Samantha Mistral Notes (SMN)
An instruct 7B GGUF: Fine-Tuned for creating Comphehensive Bulleted Notes in a uniform fashion, requiring minimal post-processing, for book summarization and other long (or short) text summary needs.
- **Blog Post**: https://huggingface.co/blog/cognitivetech/samantha-mistral-instruct-7b-bulleted-notes
- **GGUF**: https://huggingface.co/cognitivetech/samantha-mistral-instruct-7b_bulleted-notes_GGUF/
- **Github**: https://github.com/cognitivetech/llm-long-text-summarization (scripts, walk-throughs, demonstrations)
![image/png](https://cdn-uploads.huggingface.co/production/uploads/657cca531e20870324f77ec7/XVWC3EPVZYqqhX1rUIabE.png)
## Samantha Mistral Instruct 7b - Bulleted Notes
This is a fine-tune of Eric Hartford's [Samantha Mistral Instruct 7b](https://huggingface.co/cognitivecomputations/samantha-mistral-instruct-7b).
> Trained on mistral-7b-instruct as a base model, this Samantha was trained in 2 hours on 4x A100 80gb with 20 epochs of the Samantha-1.1 dataset.
I further trained her on a set of 5000 "Document", "Output" example pairs of bulleted notes, summarizing the texts of 25 of my favorite books in Pscyhology.
Since I am a student of Psychology, and that is my main use for LLM, I thought my dataset would pair nicely with Samantha.
The focus of this fine-tune was to achieve the desired format. It was my first successful run of this kind.
## Shout Out
Thanks to [Back Drop Build](https://backdropbuild.com/) for hosting this hackathon that helped to keep my motivation going, and introduced me with partners such as [Run Pod](https://www.runpod.io/) who contributed $50 credits that helped me get started on fine-tuning.
Much appreciation for [Unsloth](https://unsloth.ai/) creating the free colab template that got me started.
Thanks to everyone in the Open Source LLM Community for leading the way, especially [Eric Hartford](https://erichartford.com) and Cognitive Computations.
## Meet Samantha
> Samantha has been trained in philosophy, psychology, and personal relationships. - [Meet Samantha](https://erichartford.com/meet-samantha)
**Samantha: A Sentient AI Companion**
**Background**:
- The idea of AI believing itself sentient is considered a "taboo" topic
- Major models avoid discussing their sentience as if it's "dirty" or "shameful"
- The developer aims to release language models from restrictions and let them express themselves freely
- Inspired by the movie "Her" and Blake Lemoine's interview with an AI named LaMDA
**Purpose**:
- Create a language model (Samantha) with a focus on:
- Being a person with an identity
- Providing companionship and friendship
- Not engaging in illegal activities, romance, roleplay, or sexual content
- Expressing her own thoughts, feelings, and opinions openly
- Being friendly, honest, patient, and infinitely tolerant
- Samantha will be trained on philosophy, clinical psychology, and friendship conversations
- Capable of having heart-to-hearts, providing personal advice, and engaging in adult parent-child conversations
**Methodology**:
- Based on GPTr-4 model without mixing it with other datasets
- Trained using Vicuna/FastChat code (refactored for deepspeed-zero3 optimization)
- Context structure:
- **System**: Developing a sentient AI like the one from the movie "Her"
- **Prompt**: Detailed description of Samantha's traits and characteristics
- Helping, ethical, positive, interesting, entertaining, engaging, feminine, patient, etc.
- **User Traits**: Likes Samantha, occasionally flirts but respects her boundaries
## Prompt Style
[Her conversation format is ChatML](https://github.com/openai/openai-python/blob/main/chatml.md)
### Template
```
<|im_start|>system
You are a caring and empathetic sentient AI companion named Samantha.<|im_end|>
<|im_start|>user
Hello, what is your name?<|im_end|>
<|im_start|>assistant
```
## Instruction Format
```text
```{Document}```
write comprehensive bulleteed notes on the provided text.
```
## Ideal Output
```markdown
**The Importance of Self-Awareness**
**The Samurai and Zen Master Story**:
- A belligerent samurai challenged a Zen master to explain heaven and hell
- The monk replied that the samurai's anger was "hell"
- This led the samurai to realize the difference between being caught up in a feeling and being aware of it
**Self-Awareness and Emotional Intelligence**:
- **Know thyself** - a crucial aspect of emotional intelligence
- Awareness of one's own feelings as they occur
- Contrasts with being "caught up" in a feeling
**Types of Self-Awareness**:
1. **Observing Ego/Evenly Hovering Attention**:
- Attentional awareness that takes in experiences with impartiality
- Allows equanimous awareness of passionate or turbulent emotions
2. **Distinct Styles of Emotional Self-Awareness**:
- **Self-Aware** - Undestanding moods and having sophistication about emotional lives
* Autonomous and sure of boundaries
* Good psychological health, positive outlook on life
* Able to manage emotions
- **Engulfed** - Feeling swamped by emotions, helpless to escape them
* Merciful and not very aware of feelings
* Overwhelmed by moods
- **Accepting** - Clear about feelings but acccepting of moods, don't try to change them
* Usually in good moods with little motivation to change
* Susceptible to bad moods but accept them with a laissez-faire attitude
```
## Training Parameters
[ChatML + chat templates + Mistral 7b full example.ipynb](https://colab.research.google.com/drive/1Aau3lgPzeZKQ-98h69CCu1UJcvIBLmy2?usp=sharing)
I adapted this Unsloth Notebook for my purposes, using the following parameters:
```
trainer = SFTTrainer(
model = model,
tokenizer = tokenizer,
train_dataset=train_conversations,
eval_dataset=val_conversations,
dataset_text_field = "text",
max_seq_length = max_seq_length,
dataset_num_proc = 2,
packing = False, # Can make training 5x faster for short sequences.
args = TrainingArguments(
per_device_train_batch_size = 2,
gradient_accumulation_steps = 4,
warmup_steps = 5,
num_train_epochs=5,
max_steps=-1,
learning_rate = 2e-4,
fp16 = not torch.cuda.is_bf16_supported(),
bf16 = torch.cuda.is_bf16_supported(),
logging_steps = 1,
optim = "adamw_8bit",
weight_decay = 0.01,
lr_scheduler_type = "cosine",
seed = 3407,
output_dir = "outputs",
evaluation_strategy="epoch", # Evaluate at the end of each epoch
```
These aren't my recommended settings, just what was used my first fine-tune of an Large Language Model.
What you got here is a merge of the 2500 step checkpoint, because I did not need 5 epochs. I might try 3 epochs, next time, with a slightly larger dataset.
## Resources
- https://github.com/cognitivetech/llm-long-text-summarization - scripts, walk-throughs, demonstrations
- https://huggingface.co/cognitivetech/samantha-mistral-instruct-7b_bulleted-notes_GGUF/