Hugging Face's logo

tensorblock
/
Phi-3-medium-4k-instruct-GGUF

Text Generation
GGUF
multilingual
nlp
code
TensorBlock
GGUF
Inference Endpoints
conversational
Model card Files Community
morriszms commited on
Commit
e61bef0
·
verified ·
1 Parent(s): e5f1ca9

Upload folder using huggingface_hub

Browse files
Files changed (13) hide show
  1. Phi-3-medium-4k-instruct-Q2_K.gguf +2 -2
  2. Phi-3-medium-4k-instruct-Q3_K_L.gguf +2 -2
  3. Phi-3-medium-4k-instruct-Q3_K_M.gguf +2 -2
  4. Phi-3-medium-4k-instruct-Q3_K_S.gguf +2 -2
  5. Phi-3-medium-4k-instruct-Q4_0.gguf +2 -2
  6. Phi-3-medium-4k-instruct-Q4_K_M.gguf +2 -2
  7. Phi-3-medium-4k-instruct-Q4_K_S.gguf +2 -2
  8. Phi-3-medium-4k-instruct-Q5_0.gguf +2 -2
  9. Phi-3-medium-4k-instruct-Q5_K_M.gguf +2 -2
  10. Phi-3-medium-4k-instruct-Q5_K_S.gguf +2 -2
  11. Phi-3-medium-4k-instruct-Q6_K.gguf +2 -2
  12. Phi-3-medium-4k-instruct-Q8_0.gguf +2 -2
  13. README.md +23 -17
Phi-3-medium-4k-instruct-Q2_K.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:ab411a477d13d0fb61923b77d26960c8783272d66dc05d05a631048bceb9697b
3
- size 5204447392
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a96f461dfd5bfd908f64fb2d0fb734804f22d6d9c630853a985da9ee2d8855d3
3
+ size 5143000160
Phi-3-medium-4k-instruct-Q3_K_L.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:bbf9cd9c899b6000e3e7a35cadd55b5cf6ab0d117e5bde0dae077c587355af8d
3
- size 7348992672
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f59873658558de8924e5f024b0a1746616f03a3a04cbc82a4e886aa06c71267f
3
+ size 7490297440
Phi-3-medium-4k-instruct-Q3_K_M.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:fb883ac66623fd8ea6dc7ee5617cc475357a9c6d28cca12fe78f58197fe7be28
3
- size 6750976672
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:769294b835781a4f6942cc898d2d2e159b15905b0fbc7ad100d0ac9435c7e38f
3
+ size 6923411040
Phi-3-medium-4k-instruct-Q3_K_S.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:d391d17d965b4c8fb769218c9ae1cafd994a54a642f57b3e70d8cce2ee0c1ec0
3
- size 6064896672
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3cbe67d04a9dd2b653d8589befca9646cfae3c3c4c56717bb6596e034ac82a4f
3
+ size 6064889440
Phi-3-medium-4k-instruct-Q4_0.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:3357c77c2fec18e9b0655703171316f977713c0e436db29550de25553c89c16e
3
- size 7897132192
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:040918d94736cec2bdeb3ae115fb4c48c75d965344d676e7bf89aa7a9f050152
3
+ size 7897124960
Phi-3-medium-4k-instruct-Q4_K_M.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:1b9ded40380f3e2e1cb219b52c57cef992b423847382cd1faa1a2f96be82e81b
3
- size 8404012192
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e11a39b7942b501011d5e1f75e530dfadb55b2f95451de7d9e5025bc44d1b594
3
+ size 8566820960
Phi-3-medium-4k-instruct-Q4_K_S.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:11fc6f23100e2b10f70db92d7f569450fe04bac349866ec69051784e3c4c6500
3
- size 7957752992
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:087b8d51e139abeb48baae8bcb64d296dd77ea2e3e2f602c50e1473b8db07a2c
3
+ size 7954468960
Phi-3-medium-4k-instruct-Q5_0.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:1db29faccce31ff1f25309f3dd4d6e42e067a9dae368bf51c5898b3c92d9d66c
3
- size 9621589152
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4fb2e6c4f14392d4663d5f2036634f96e3ff44727b0ea0459d2e7737f89c3729
3
+ size 9621581920
Phi-3-medium-4k-instruct-Q5_K_M.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:563814fddc4e61c385ed8f6436a51a1269595dd570760e34c5468fd1b726ccab
3
- size 9882709152
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:97f96065bfdbc3d038cdb0545d83c48bc06bc97a7e55d4b35b668f91150c3d09
3
+ size 10074189920
Phi-3-medium-4k-instruct-Q5_K_S.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:4d18e7e96f2a3683531cbe8226d50bdf1bfdad86c610fd02b3c567658ac65e29
3
- size 9621589152
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:17936bf9f0e05e60b32dde370bbd5f69c0071cc63e7dc62687e49bce50c03b06
3
+ size 9621581920
Phi-3-medium-4k-instruct-Q6_K.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:0c81399cf50b6b293f67c8db62eed2b666a83fb79d3c5b2a123fdd378dffbc72
3
- size 11453824672
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:64b940aa79b3282311b292beee66cd31efda19a75de50fd70d5c47cc6b63444c
3
+ size 11453817440
Phi-3-medium-4k-instruct-Q8_0.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:7cd0cda6efadc5641f715166b69f611b4dc684a0e06df905d90fb93641e92e5f
3
- size 14834719392
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:45f486f3314808fe34110561621d9aafd7bdf820645c85b2fa40df41ad0467e9
3
+ size 14834712160
README.md CHANGED
@@ -1,16 +1,22 @@
1
  ---
2
- language:
3
- - en
4
- library_name: transformers
5
  license: mit
 
 
 
 
6
  tags:
7
- - unsloth
8
- - phi3
9
- - transformers
10
- - phi
11
  - TensorBlock
12
  - GGUF
13
- base_model: unsloth/Phi-3-medium-4k-instruct
 
 
 
 
 
 
 
14
  ---
15
 
16
  <div style="width: auto; margin-left: auto; margin-right: auto">
@@ -24,11 +30,11 @@ base_model: unsloth/Phi-3-medium-4k-instruct
24
  </div>
25
  </div>
26
 
27
- ## unsloth/Phi-3-medium-4k-instruct - GGUF
28
 
29
- This repo contains GGUF format model files for [unsloth/Phi-3-medium-4k-instruct](https://huggingface.co/unsloth/Phi-3-medium-4k-instruct).
30
 
31
- The files were quantized using machines provided by [TensorBlock](https://tensorblock.co/), and they are compatible with llama.cpp as of [commit b4242](https://github.com/ggerganov/llama.cpp/commit/a6744e43e80f4be6398fc7733a01642c846dce1d).
32
 
33
  <div style="text-align: left; margin: 20px 0;">
34
  <a href="https://tensorblock.co/waitlist/client" style="display: inline-block; padding: 10px 20px; background-color: #007bff; color: white; text-decoration: none; border-radius: 5px; font-weight: bold;">
@@ -48,16 +54,16 @@ The files were quantized using machines provided by [TensorBlock](https://tensor
48
 
49
  | Filename | Quant type | File Size | Description |
50
  | -------- | ---------- | --------- | ----------- |
51
- | [Phi-3-medium-4k-instruct-Q2_K.gguf](https://huggingface.co/tensorblock/Phi-3-medium-4k-instruct-GGUF/blob/main/Phi-3-medium-4k-instruct-Q2_K.gguf) | Q2_K | 5.204 GB | smallest, significant quality loss - not recommended for most purposes |
52
  | [Phi-3-medium-4k-instruct-Q3_K_S.gguf](https://huggingface.co/tensorblock/Phi-3-medium-4k-instruct-GGUF/blob/main/Phi-3-medium-4k-instruct-Q3_K_S.gguf) | Q3_K_S | 6.065 GB | very small, high quality loss |
53
- | [Phi-3-medium-4k-instruct-Q3_K_M.gguf](https://huggingface.co/tensorblock/Phi-3-medium-4k-instruct-GGUF/blob/main/Phi-3-medium-4k-instruct-Q3_K_M.gguf) | Q3_K_M | 6.751 GB | very small, high quality loss |
54
- | [Phi-3-medium-4k-instruct-Q3_K_L.gguf](https://huggingface.co/tensorblock/Phi-3-medium-4k-instruct-GGUF/blob/main/Phi-3-medium-4k-instruct-Q3_K_L.gguf) | Q3_K_L | 7.349 GB | small, substantial quality loss |
55
  | [Phi-3-medium-4k-instruct-Q4_0.gguf](https://huggingface.co/tensorblock/Phi-3-medium-4k-instruct-GGUF/blob/main/Phi-3-medium-4k-instruct-Q4_0.gguf) | Q4_0 | 7.897 GB | legacy; small, very high quality loss - prefer using Q3_K_M |
56
- | [Phi-3-medium-4k-instruct-Q4_K_S.gguf](https://huggingface.co/tensorblock/Phi-3-medium-4k-instruct-GGUF/blob/main/Phi-3-medium-4k-instruct-Q4_K_S.gguf) | Q4_K_S | 7.958 GB | small, greater quality loss |
57
- | [Phi-3-medium-4k-instruct-Q4_K_M.gguf](https://huggingface.co/tensorblock/Phi-3-medium-4k-instruct-GGUF/blob/main/Phi-3-medium-4k-instruct-Q4_K_M.gguf) | Q4_K_M | 8.404 GB | medium, balanced quality - recommended |
58
  | [Phi-3-medium-4k-instruct-Q5_0.gguf](https://huggingface.co/tensorblock/Phi-3-medium-4k-instruct-GGUF/blob/main/Phi-3-medium-4k-instruct-Q5_0.gguf) | Q5_0 | 9.622 GB | legacy; medium, balanced quality - prefer using Q4_K_M |
59
  | [Phi-3-medium-4k-instruct-Q5_K_S.gguf](https://huggingface.co/tensorblock/Phi-3-medium-4k-instruct-GGUF/blob/main/Phi-3-medium-4k-instruct-Q5_K_S.gguf) | Q5_K_S | 9.622 GB | large, low quality loss - recommended |
60
- | [Phi-3-medium-4k-instruct-Q5_K_M.gguf](https://huggingface.co/tensorblock/Phi-3-medium-4k-instruct-GGUF/blob/main/Phi-3-medium-4k-instruct-Q5_K_M.gguf) | Q5_K_M | 9.883 GB | large, very low quality loss - recommended |
61
  | [Phi-3-medium-4k-instruct-Q6_K.gguf](https://huggingface.co/tensorblock/Phi-3-medium-4k-instruct-GGUF/blob/main/Phi-3-medium-4k-instruct-Q6_K.gguf) | Q6_K | 11.454 GB | very large, extremely low quality loss |
62
  | [Phi-3-medium-4k-instruct-Q8_0.gguf](https://huggingface.co/tensorblock/Phi-3-medium-4k-instruct-GGUF/blob/main/Phi-3-medium-4k-instruct-Q8_0.gguf) | Q8_0 | 14.835 GB | very large, extremely low quality loss - not recommended |
63
 
 
1
  ---
 
 
 
2
  license: mit
3
+ license_link: https://huggingface.co/microsoft/Phi-3-medium-4k-instruct/resolve/main/LICENSE
4
+ language:
5
+ - multilingual
6
+ pipeline_tag: text-generation
7
  tags:
8
+ - nlp
9
+ - code
 
 
10
  - TensorBlock
11
  - GGUF
12
+ inference:
13
+ parameters:
14
+ temperature: 0.7
15
+ widget:
16
+ - messages:
17
+ - role: user
18
+ content: Can you provide ways to eat combinations of bananas and dragonfruits?
19
+ base_model: microsoft/Phi-3-medium-4k-instruct
20
  ---
21
 
22
  <div style="width: auto; margin-left: auto; margin-right: auto">
 
30
  </div>
31
  </div>
32
 
33
+ ## microsoft/Phi-3-medium-4k-instruct - GGUF
34
 
35
+ This repo contains GGUF format model files for [microsoft/Phi-3-medium-4k-instruct](https://huggingface.co/microsoft/Phi-3-medium-4k-instruct).
36
 
37
+ The files were quantized using machines provided by [TensorBlock](https://tensorblock.co/), and they are compatible with llama.cpp as of [commit ec7f3ac](https://github.com/ggerganov/llama.cpp/commit/ec7f3ac9ab33e46b136eb5ab6a76c4d81f57c7f1).
38
 
39
  <div style="text-align: left; margin: 20px 0;">
40
  <a href="https://tensorblock.co/waitlist/client" style="display: inline-block; padding: 10px 20px; background-color: #007bff; color: white; text-decoration: none; border-radius: 5px; font-weight: bold;">
 
54
 
55
  | Filename | Quant type | File Size | Description |
56
  | -------- | ---------- | --------- | ----------- |
57
+ | [Phi-3-medium-4k-instruct-Q2_K.gguf](https://huggingface.co/tensorblock/Phi-3-medium-4k-instruct-GGUF/blob/main/Phi-3-medium-4k-instruct-Q2_K.gguf) | Q2_K | 5.143 GB | smallest, significant quality loss - not recommended for most purposes |
58
  | [Phi-3-medium-4k-instruct-Q3_K_S.gguf](https://huggingface.co/tensorblock/Phi-3-medium-4k-instruct-GGUF/blob/main/Phi-3-medium-4k-instruct-Q3_K_S.gguf) | Q3_K_S | 6.065 GB | very small, high quality loss |
59
+ | [Phi-3-medium-4k-instruct-Q3_K_M.gguf](https://huggingface.co/tensorblock/Phi-3-medium-4k-instruct-GGUF/blob/main/Phi-3-medium-4k-instruct-Q3_K_M.gguf) | Q3_K_M | 6.923 GB | very small, high quality loss |
60
+ | [Phi-3-medium-4k-instruct-Q3_K_L.gguf](https://huggingface.co/tensorblock/Phi-3-medium-4k-instruct-GGUF/blob/main/Phi-3-medium-4k-instruct-Q3_K_L.gguf) | Q3_K_L | 7.490 GB | small, substantial quality loss |
61
  | [Phi-3-medium-4k-instruct-Q4_0.gguf](https://huggingface.co/tensorblock/Phi-3-medium-4k-instruct-GGUF/blob/main/Phi-3-medium-4k-instruct-Q4_0.gguf) | Q4_0 | 7.897 GB | legacy; small, very high quality loss - prefer using Q3_K_M |
62
+ | [Phi-3-medium-4k-instruct-Q4_K_S.gguf](https://huggingface.co/tensorblock/Phi-3-medium-4k-instruct-GGUF/blob/main/Phi-3-medium-4k-instruct-Q4_K_S.gguf) | Q4_K_S | 7.954 GB | small, greater quality loss |
63
+ | [Phi-3-medium-4k-instruct-Q4_K_M.gguf](https://huggingface.co/tensorblock/Phi-3-medium-4k-instruct-GGUF/blob/main/Phi-3-medium-4k-instruct-Q4_K_M.gguf) | Q4_K_M | 8.567 GB | medium, balanced quality - recommended |
64
  | [Phi-3-medium-4k-instruct-Q5_0.gguf](https://huggingface.co/tensorblock/Phi-3-medium-4k-instruct-GGUF/blob/main/Phi-3-medium-4k-instruct-Q5_0.gguf) | Q5_0 | 9.622 GB | legacy; medium, balanced quality - prefer using Q4_K_M |
65
  | [Phi-3-medium-4k-instruct-Q5_K_S.gguf](https://huggingface.co/tensorblock/Phi-3-medium-4k-instruct-GGUF/blob/main/Phi-3-medium-4k-instruct-Q5_K_S.gguf) | Q5_K_S | 9.622 GB | large, low quality loss - recommended |
66
+ | [Phi-3-medium-4k-instruct-Q5_K_M.gguf](https://huggingface.co/tensorblock/Phi-3-medium-4k-instruct-GGUF/blob/main/Phi-3-medium-4k-instruct-Q5_K_M.gguf) | Q5_K_M | 10.074 GB | large, very low quality loss - recommended |
67
  | [Phi-3-medium-4k-instruct-Q6_K.gguf](https://huggingface.co/tensorblock/Phi-3-medium-4k-instruct-GGUF/blob/main/Phi-3-medium-4k-instruct-Q6_K.gguf) | Q6_K | 11.454 GB | very large, extremely low quality loss |
68
  | [Phi-3-medium-4k-instruct-Q8_0.gguf](https://huggingface.co/tensorblock/Phi-3-medium-4k-instruct-GGUF/blob/main/Phi-3-medium-4k-instruct-Q8_0.gguf) | Q8_0 | 14.835 GB | very large, extremely low quality loss - not recommended |
69