Update README.md
Browse files
README.md
CHANGED
@@ -56,11 +56,19 @@ To use the phi-2-chaos-gen model, follow these steps:
|
|
56 |
Load the phi-2-chaos-gen model using the transformers package. Ensure you have an internet connection as the model will be downloaded the first time you run this code.
|
57 |
|
58 |
```python
|
59 |
-
from transformers import
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
60 |
```
|
61 |
-
|
62 |
-
tokenizer = AutoTokenizer.from_pretrained("webnizam/phi-2-chaos-gen")
|
63 |
-
model = AutoModelForCausalLM.from_pretrained("webnizam/phi-2-chaos-gen")
|
64 |
3. **Using the Model:**
|
65 |
You can now use the model to generate text. For example:
|
66 |
```python
|
|
|
56 |
Load the phi-2-chaos-gen model using the transformers package. Ensure you have an internet connection as the model will be downloaded the first time you run this code.
|
57 |
|
58 |
```python
|
59 |
+
from transformers import AutoTokenizer, AutoModelForCausalLM
|
60 |
+
|
61 |
+
base_model_id = "microsoft/phi-2"
|
62 |
+
base_model = AutoModelForCausalLM.from_pretrained(
|
63 |
+
base_model_id, # Phi2, same as before
|
64 |
+
device_map="auto",
|
65 |
+
trust_remote_code=True,
|
66 |
+
load_in_8bit=True,
|
67 |
+
torch_dtype=torch.float16,
|
68 |
+
)
|
69 |
+
|
70 |
+
base_model.config.use_cache = True
|
71 |
```
|
|
|
|
|
|
|
72 |
3. **Using the Model:**
|
73 |
You can now use the model to generate text. For example:
|
74 |
```python
|