See axolotl config
axolotl version: 0.4.1
base_model: mistralai/Mistral-7B-v0.3
model_type: AutoModelForCausalLM
tokenizer_type: AutoTokenizer
tokenizer_use_fast: false
load_in_8bit: false
load_in_4bit: false
strict: false
model_config:
datasets:
- path: /home/migel/ai_datasets/tess-v1.5b-chatml.jsonl
type: sharegpt
conversation: chatml
- path: /home/migel/ai_datasets/Tess-3.0/Tess-3.0-multi_turn_chatml.jsonl
type: sharegpt
conversation: chatml
- path: /home/migel/ai_datasets/Tess-3.0/Tess-3.0-single_turn_chatml.jsonl
type: sharegpt
conversation: chatml
chat_template: chatml
dataset_prepared_path: last_run_prepared_mistral
val_set_size: 0.0
output_dir: /home/migel/tess-2.5-mistral-7B-phase-1
resume_from_checkpoint: /home/migel/tess-2.5-mistral-7B-phase-1/checkpoint-440
auto_resume_from_checkpoints: true
sequence_len: 16384
sample_packing: true
pad_to_sequence_len: true
gradient_accumulation_steps: 4
micro_batch_size: 4
num_epochs: 1
logging_steps: 1
optimizer: adamw_8bit
lr_scheduler: constant
learning_rate: 1e-6
wandb_project: mistral-7b
wandb_watch:
wandb_run_id:
wandb_log_model:
train_on_inputs: false
group_by_length: false
bf16: auto
fp16:
tf32: false
gradient_checkpointing: true
gradient_checkpointing_kwargs:
use_reentrant: false
early_stopping_patience:
resume_from_checkpoint:
local_rank:
logging_steps: 1
xformers_attention:
flash_attention: true
saves_per_epoch: 10
evals_per_epoch: 10
save_total_limit: 3
save_steps:
eval_sample_packing: false
debug:
deepspeed: /home/migel/axolotl/deepspeed_configs/zero3_bf16.json
weight_decay: 0.0
fsdp:
fsdp_config:
special_tokens:
bos_token: "<|im_start|>"
eos_token: "<|im_end|>"
pad_token: "<|end_of_text|>"
Tess-3-7B-SFT
Tess-3-7B is a finetuned version of the Mistral-7B-v0.3 base model. This version is the first phase of the final Tess-3 model, and have been trained with supervised fine-tuning (SFT) on a curated dataset of ~500K samples. The total SFT dataset contains about 1B tokens.
This model has 32K context length.
Sample code to run inference
Note that this model uses ChatML prompt format.
import torch, json
from transformers import AutoModelForCausalLM, AutoTokenizer
from stop_word import StopWordCriteria
model_path = "migtissera/Tess-3-7B-SFT"
output_file_path = "/home/migel/conversations.jsonl"
model = AutoModelForCausalLM.from_pretrained(
model_path,
torch_dtype=torch.float16,
device_map="auto",
load_in_4bit=False,
trust_remote_code=True,
)
tokenizer = AutoTokenizer.from_pretrained(model_path, trust_remote_code=True)
terminators = [
tokenizer.convert_tokens_to_ids("<|im_end|>")
]
def generate_text(instruction):
tokens = tokenizer.encode(instruction)
tokens = torch.LongTensor(tokens).unsqueeze(0)
tokens = tokens.to("cuda")
instance = {
"input_ids": tokens,
"top_p": 1.0,
"temperature": 0.75,
"generate_len": 1024,
"top_k": 50,
}
length = len(tokens[0])
with torch.no_grad():
rest = model.generate(
input_ids=tokens,
max_length=length + instance["generate_len"],
use_cache=True,
do_sample=True,
top_p=instance["top_p"],
temperature=instance["temperature"],
top_k=instance["top_k"],
num_return_sequences=1,
pad_token_id=tokenizer.eos_token_id,
eos_token_id=terminators,
)
output = rest[0][length:]
string = tokenizer.decode(output, skip_special_tokens=True)
return f"{string}"
conversation = f"""<|im_start|>system\nYou are Tesoro, a helful AI assitant. You always provide detailed answers without hesitation.<|im_end|>\n<|im_start|>user\n"""
while True:
user_input = input("You: ")
llm_prompt = f"{conversation}{user_input}<|im_end|>\n<|im_start|>assistant\n"
answer = generate_text(llm_prompt)
print(answer)
conversation = f"{llm_prompt}{answer}\n"
json_data = {"prompt": user_input, "answer": answer}
with open(output_file_path, "a") as output_file:
output_file.write(json.dumps(json_data) + "\n")
Join My General AI Discord (NeuroLattice):
Limitations & Biases:
While this model aims for accuracy, it can occasionally produce inaccurate or misleading results.
Despite diligent efforts in refining the pretraining data, there remains a possibility for the generation of inappropriate, biased, or offensive content.
Exercise caution and cross-check information when necessary. This is an uncensored model.
Open LLM Leaderboard Evaluation Results
Detailed results can be found here
Metric | Value |
---|---|
Avg. | 17.10 |
IFEval (0-Shot) | 39.46 |
BBH (3-Shot) | 24.12 |
MATH Lvl 5 (4-Shot) | 3.32 |
GPQA (0-shot) | 2.80 |
MuSR (0-shot) | 10.28 |
MMLU-PRO (5-shot) | 22.60 |
- Downloads last month
- 67
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social
visibility and check back later, or deploy to Inference Endpoints (dedicated)
instead.