KeyserSoze1 commited on
Commit
c4517f0
1 Parent(s): e4d8bfa

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +87 -0
README.md ADDED
@@ -0,0 +1,87 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model: utrobinmv/t5_translate_en_ru_zh_large_1024_v2
3
+ datasets:
4
+ - ccmatrix
5
+ language:
6
+ - ru
7
+ - zh
8
+ - en
9
+ license: apache-2.0
10
+ metrics:
11
+ - sacrebleu
12
+ tags:
13
+ - translation
14
+ - llama-cpp
15
+ - gguf-my-repo
16
+ widget:
17
+ - example_title: translate zh-ru
18
+ text: 'translate to ru: 开发的目的是为用户提供个人同步翻译。
19
+
20
+ '
21
+ - example_title: translate ru-en
22
+ text: 'translate to en: Цель разработки — предоставить пользователям личного синхронного
23
+ переводчика.
24
+
25
+ '
26
+ - example_title: translate en-ru
27
+ text: 'translate to ru: The purpose of the development is to provide users with
28
+ a personal synchronized interpreter.
29
+
30
+ '
31
+ - example_title: translate en-zh
32
+ text: 'translate to zh: The purpose of the development is to provide users with
33
+ a personal synchronized interpreter.
34
+
35
+ '
36
+ - example_title: translate zh-en
37
+ text: 'translate to en: 开发的目的是为用户提供个人同步解释器。
38
+
39
+ '
40
+ - example_title: translate ru-zh
41
+ text: 'translate to zh: Цель разработки — предоставить пользователям личного синхронного
42
+ переводчика.'
43
+ ---
44
+
45
+ # KeyserSoze1/t5_translate_en_ru_zh_large_1024_v2-Q4_K_M-GGUF
46
+ This model was converted to GGUF format from [`utrobinmv/t5_translate_en_ru_zh_large_1024_v2`](https://huggingface.co/utrobinmv/t5_translate_en_ru_zh_large_1024_v2) using llama.cpp via the ggml.ai's [GGUF-my-repo](https://huggingface.co/spaces/ggml-org/gguf-my-repo) space.
47
+ Refer to the [original model card](https://huggingface.co/utrobinmv/t5_translate_en_ru_zh_large_1024_v2) for more details on the model.
48
+
49
+ ## Use with llama.cpp
50
+ Install llama.cpp through brew (works on Mac and Linux)
51
+
52
+ ```bash
53
+ brew install llama.cpp
54
+
55
+ ```
56
+ Invoke the llama.cpp server or the CLI.
57
+
58
+ ### CLI:
59
+ ```bash
60
+ llama-cli --hf-repo KeyserSoze1/t5_translate_en_ru_zh_large_1024_v2-Q4_K_M-GGUF --hf-file t5_translate_en_ru_zh_large_1024_v2-q4_k_m.gguf -p "The meaning to life and the universe is"
61
+ ```
62
+
63
+ ### Server:
64
+ ```bash
65
+ llama-server --hf-repo KeyserSoze1/t5_translate_en_ru_zh_large_1024_v2-Q4_K_M-GGUF --hf-file t5_translate_en_ru_zh_large_1024_v2-q4_k_m.gguf -c 2048
66
+ ```
67
+
68
+ Note: You can also use this checkpoint directly through the [usage steps](https://github.com/ggerganov/llama.cpp?tab=readme-ov-file#usage) listed in the Llama.cpp repo as well.
69
+
70
+ Step 1: Clone llama.cpp from GitHub.
71
+ ```
72
+ git clone https://github.com/ggerganov/llama.cpp
73
+ ```
74
+
75
+ Step 2: Move into the llama.cpp folder and build it with `LLAMA_CURL=1` flag along with other hardware-specific flags (for ex: LLAMA_CUDA=1 for Nvidia GPUs on Linux).
76
+ ```
77
+ cd llama.cpp && LLAMA_CURL=1 make
78
+ ```
79
+
80
+ Step 3: Run inference through the main binary.
81
+ ```
82
+ ./llama-cli --hf-repo KeyserSoze1/t5_translate_en_ru_zh_large_1024_v2-Q4_K_M-GGUF --hf-file t5_translate_en_ru_zh_large_1024_v2-q4_k_m.gguf -p "The meaning to life and the universe is"
83
+ ```
84
+ or
85
+ ```
86
+ ./llama-server --hf-repo KeyserSoze1/t5_translate_en_ru_zh_large_1024_v2-Q4_K_M-GGUF --hf-file t5_translate_en_ru_zh_large_1024_v2-q4_k_m.gguf -c 2048
87
+ ```