vagmi commited on
Commit
a6be58c
·
1 Parent(s): 54faafd

Update readme with usage instructions.

Browse files
Files changed (1) hide show
  1. README.md +42 -1
README.md CHANGED
@@ -5,4 +5,45 @@ datasets:
5
  language:
6
  - en
7
  library_name: transformers
8
- ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
5
  language:
6
  - en
7
  library_name: transformers
8
+ ---
9
+ # Generate SQL from text - Squeal
10
+
11
+
12
+ Please use the code below as an example for how to use this model.
13
+
14
+ This model was streamed on Youtube.
15
+
16
+ https://www.youtube.com/watch?v=PNFhAfxR_d8
17
+
18
+ ```python
19
+ import torch
20
+ from transformers import pipeline, AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
21
+
22
+ def load_model(model_name):
23
+ # Load tokenizer and model with QLoRA configuration
24
+ compute_dtype = getattr(torch, 'float16')
25
+
26
+ bnb_config = BitsAndBytesConfig(
27
+ load_in_4bit=True,
28
+ bnb_4bit_quant_type='nf4',
29
+ bnb_4bit_compute_dtype=compute_dtype,
30
+ bnb_4bit_use_double_quant=False,
31
+ )
32
+
33
+ model = AutoModelForCausalLM.from_pretrained(
34
+ model_name,
35
+ device_map={"": 0},
36
+ quantization_config=bnb_config
37
+ )
38
+
39
+
40
+ # Load Tokenizer
41
+ tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True)
42
+ tokenizer.pad_token = tokenizer.eos_token
43
+ tokenizer.padding_side = "right"
44
+
45
+ return model, tokenizer
46
+
47
+ model, tokenizer = load_model('vagmi/squeal')
48
+ ```
49
+