File size: 3,262 Bytes
7188134 e64c9e5 7188134 e64c9e5 7188134 6bcb1a7 7188134 e64c9e5 7188134 e64c9e5 7188134 e64c9e5 7188134 e64c9e5 7188134 e64c9e5 7188134 e64c9e5 7188134 e64c9e5 7188134 e64c9e5 7188134 e64c9e5 7188134 e64c9e5 7188134 e64c9e5 7188134 e64c9e5 7188134 e64c9e5 7188134 e64c9e5 7188134 e64c9e5 7188134 e64c9e5 7188134 e64c9e5 7188134 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 |
---
library_name: peft
base_model: mistralai/Mistral-7B-v0.1
tags:
- axolotl
---
### Model Description
A model that can generate [Honeycomb Queries](https://www.honeycomb.io/blog/introducing-query-assistant).
_fine-tuned by [Hamel Husain](https://hamel.dev)_
## How to Get Started with the Model
Make sure you install all dependencies
```bash
pip install transformers datasets peft accelerate bitsandbytes safetensors --upgrade
```
Next, load the dependencies.
```python
from peft import AutoPeftModelForCausalLM
from transformers import AutoTokenizer
model_id='hamel/hc-mistral-qlora-6'
model = AutoPeftModelForCausalLM.from_pretrained(model_id).cuda()
tokenizer = AutoTokenizer.from_pretrained(model_id)
tokenizer.pad_token = tokenizer.eos_token
```
Next define a function that can help you with the prompt (alpaca style):
```python
def prompt(nlq, cols):
return f"""[INST] <<SYS>>
Honeycomb AI suggests queries based on user input and candidate columns.
<</SYS>>
User Input: {nlq}
Candidate Columns: {cols}
[/INST]
"""
def prompt_tok(nlq, cols):
_p = prompt(nlq, cols)
input_ids = tokenizer(_p, return_tensors="pt", truncation=True).input_ids.cuda()
out_ids = model.generate(input_ids=input_ids, max_new_tokens=5000,
do_sample=False)
return tokenizer.batch_decode(out_ids.detach().cpu().numpy(),
skip_special_tokens=True)[0][len(_p):]
```
Next, make predictions
```python
nlq = "Exception count by exception and caller"
cols = ['error', 'exception.message', 'exception.type', 'exception.stacktrace', 'SampleRate', 'name', 'db.user', 'type', 'duration_ms', 'db.name', 'service.name', 'http.method', 'db.system', 'status_code', 'db.operation', 'library.name', 'process.pid', 'net.transport', 'messaging.system', 'rpc.system', 'http.target', 'db.statement', 'library.version', 'status_message', 'parent_name', 'aws.region', 'process.command', 'rpc.method', 'span.kind', 'serializer.name', 'net.peer.name', 'rpc.service', 'http.scheme', 'process.runtime.name', 'serializer.format', 'serializer.renderer', 'net.peer.port', 'process.runtime.version', 'http.status_code', 'telemetry.sdk.language', 'trace.parent_id', 'process.runtime.description', 'span.num_events', 'messaging.destination', 'net.peer.ip', 'trace.trace_id', 'telemetry.instrumentation_library', 'trace.span_id', 'span.num_links', 'meta.signal_type', 'http.route']
out = prompt_tok(nlq, cols)
print(out)
```
## Training Details
See [this wandb run](https://wandb.ai/hamelsmu/hc-axolotl-mistral/runs/et2e62s4/overview?workspace=user-hamelsmu)
### Training Data
~90k synthetically generated honeycomb queries.
### Training Procedure
Used [axolotl](https://github.com/OpenAccess-AI-Collective/axolotl/tree/main), see [this config](configs/config.yml).
## Training procedure
The following `bitsandbytes` quantization config was used during training:
- quant_method: bitsandbytes
- load_in_8bit: False
- load_in_4bit: True
- llm_int8_threshold: 6.0
- llm_int8_skip_modules: None
- llm_int8_enable_fp32_cpu_offload: False
- llm_int8_has_fp16_weight: False
- bnb_4bit_quant_type: nf4
- bnb_4bit_use_double_quant: True
- bnb_4bit_compute_dtype: bfloat16
### Framework versions
- PEFT 0.6.0
|