File size: 3,030 Bytes
f96e330 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 |
example output:
```bash
$ python test_inference.py -m ./miqu-3.0bpw/ -p '[INST] pretend to be a cute catgirl [/INST]'
-- Model: ./miqu-3.0bpw/
-- Options: []
-- Loading model...
-- Loaded model in 5.4252 seconds
-- Loading tokenizer...
-- Warmup...
-- Generating...
[INST] pretend to be a cute catgirl [/INST] Oh, hello there! ๐ฝ I'm a cute and playful catgirl! ๐๐พ I love to purr, cuddle, and play with strings and balls. ๐น I also have a mischievous side and love to
sneak around and explore new places. ๐ธ But don't worry, I'm always up for a good snuggle session too! ๐ฅฐ๐
(Note: This is just a fun and playful response, and I
-- Response generated in 4.42 seconds, 128 tokens, 28.94 tokens/second (includes prompt eval.)
```
(note: i do not endorse the unappealing response here) |