Hugging Face's logo

RichardErkhov
/
zjunlp_-_OceanGPT-2B-v0.1-gguf

GGUF
Inference Endpoints
conversational
Model card Files Community
RichardErkhov commited on
Commit
4575669
·
verified ·
1 Parent(s): ff2816f

uploaded readme

Browse files
Files changed (1) hide show
  1. README.md +168 -0
README.md ADDED
@@ -0,0 +1,168 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ Quantization made by Richard Erkhov.
2
+
3
+ [Github](https://github.com/RichardErkhov)
4
+
5
+ [Discord](https://discord.gg/pvy7H8DZMG)
6
+
7
+ [Request more models](https://github.com/RichardErkhov/quant_request)
8
+
9
+
10
+ OceanGPT-2B-v0.1 - GGUF
11
+ - Model creator: https://huggingface.co/zjunlp/
12
+ - Original model: https://huggingface.co/zjunlp/OceanGPT-2B-v0.1/
13
+
14
+
15
+ | Name | Quant method | Size |
16
+ | ---- | ---- | ---- |
17
+ | [OceanGPT-2B-v0.1.Q2_K.gguf](https://huggingface.co/RichardErkhov/zjunlp_-_OceanGPT-2B-v0.1-gguf/blob/main/OceanGPT-2B-v0.1.Q2_K.gguf) | Q2_K | 1.12GB |
18
+ | [OceanGPT-2B-v0.1.IQ3_XS.gguf](https://huggingface.co/RichardErkhov/zjunlp_-_OceanGPT-2B-v0.1-gguf/blob/main/OceanGPT-2B-v0.1.IQ3_XS.gguf) | IQ3_XS | 1.21GB |
19
+ | [OceanGPT-2B-v0.1.IQ3_S.gguf](https://huggingface.co/RichardErkhov/zjunlp_-_OceanGPT-2B-v0.1-gguf/blob/main/OceanGPT-2B-v0.1.IQ3_S.gguf) | IQ3_S | 1.26GB |
20
+ | [OceanGPT-2B-v0.1.Q3_K_S.gguf](https://huggingface.co/RichardErkhov/zjunlp_-_OceanGPT-2B-v0.1-gguf/blob/main/OceanGPT-2B-v0.1.Q3_K_S.gguf) | Q3_K_S | 1.26GB |
21
+ | [OceanGPT-2B-v0.1.IQ3_M.gguf](https://huggingface.co/RichardErkhov/zjunlp_-_OceanGPT-2B-v0.1-gguf/blob/main/OceanGPT-2B-v0.1.IQ3_M.gguf) | IQ3_M | 1.32GB |
22
+ | [OceanGPT-2B-v0.1.Q3_K.gguf](https://huggingface.co/RichardErkhov/zjunlp_-_OceanGPT-2B-v0.1-gguf/blob/main/OceanGPT-2B-v0.1.Q3_K.gguf) | Q3_K | 1.38GB |
23
+ | [OceanGPT-2B-v0.1.Q3_K_M.gguf](https://huggingface.co/RichardErkhov/zjunlp_-_OceanGPT-2B-v0.1-gguf/blob/main/OceanGPT-2B-v0.1.Q3_K_M.gguf) | Q3_K_M | 1.38GB |
24
+ | [OceanGPT-2B-v0.1.Q3_K_L.gguf](https://huggingface.co/RichardErkhov/zjunlp_-_OceanGPT-2B-v0.1-gguf/blob/main/OceanGPT-2B-v0.1.Q3_K_L.gguf) | Q3_K_L | 1.46GB |
25
+ | [OceanGPT-2B-v0.1.IQ4_XS.gguf](https://huggingface.co/RichardErkhov/zjunlp_-_OceanGPT-2B-v0.1-gguf/blob/main/OceanGPT-2B-v0.1.IQ4_XS.gguf) | IQ4_XS | 1.45GB |
26
+ | [OceanGPT-2B-v0.1.Q4_0.gguf](https://huggingface.co/RichardErkhov/zjunlp_-_OceanGPT-2B-v0.1-gguf/blob/main/OceanGPT-2B-v0.1.Q4_0.gguf) | Q4_0 | 1.5GB |
27
+ | [OceanGPT-2B-v0.1.IQ4_NL.gguf](https://huggingface.co/RichardErkhov/zjunlp_-_OceanGPT-2B-v0.1-gguf/blob/main/OceanGPT-2B-v0.1.IQ4_NL.gguf) | IQ4_NL | 1.51GB |
28
+ | [OceanGPT-2B-v0.1.Q4_K_S.gguf](https://huggingface.co/RichardErkhov/zjunlp_-_OceanGPT-2B-v0.1-gguf/blob/main/OceanGPT-2B-v0.1.Q4_K_S.gguf) | Q4_K_S | 1.57GB |
29
+ | [OceanGPT-2B-v0.1.Q4_K.gguf](https://huggingface.co/RichardErkhov/zjunlp_-_OceanGPT-2B-v0.1-gguf/blob/main/OceanGPT-2B-v0.1.Q4_K.gguf) | Q4_K | 1.68GB |
30
+ | [OceanGPT-2B-v0.1.Q4_K_M.gguf](https://huggingface.co/RichardErkhov/zjunlp_-_OceanGPT-2B-v0.1-gguf/blob/main/OceanGPT-2B-v0.1.Q4_K_M.gguf) | Q4_K_M | 1.68GB |
31
+ | [OceanGPT-2B-v0.1.Q4_1.gguf](https://huggingface.co/RichardErkhov/zjunlp_-_OceanGPT-2B-v0.1-gguf/blob/main/OceanGPT-2B-v0.1.Q4_1.gguf) | Q4_1 | 1.64GB |
32
+ | [OceanGPT-2B-v0.1.Q5_0.gguf](https://huggingface.co/RichardErkhov/zjunlp_-_OceanGPT-2B-v0.1-gguf/blob/main/OceanGPT-2B-v0.1.Q5_0.gguf) | Q5_0 | 1.78GB |
33
+ | [OceanGPT-2B-v0.1.Q5_K_S.gguf](https://huggingface.co/RichardErkhov/zjunlp_-_OceanGPT-2B-v0.1-gguf/blob/main/OceanGPT-2B-v0.1.Q5_K_S.gguf) | Q5_K_S | 1.81GB |
34
+ | [OceanGPT-2B-v0.1.Q5_K.gguf](https://huggingface.co/RichardErkhov/zjunlp_-_OceanGPT-2B-v0.1-gguf/blob/main/OceanGPT-2B-v0.1.Q5_K.gguf) | Q5_K | 1.9GB |
35
+ | [OceanGPT-2B-v0.1.Q5_K_M.gguf](https://huggingface.co/RichardErkhov/zjunlp_-_OceanGPT-2B-v0.1-gguf/blob/main/OceanGPT-2B-v0.1.Q5_K_M.gguf) | Q5_K_M | 1.9GB |
36
+ | [OceanGPT-2B-v0.1.Q5_1.gguf](https://huggingface.co/RichardErkhov/zjunlp_-_OceanGPT-2B-v0.1-gguf/blob/main/OceanGPT-2B-v0.1.Q5_1.gguf) | Q5_1 | 1.93GB |
37
+ | [OceanGPT-2B-v0.1.Q6_K.gguf](https://huggingface.co/RichardErkhov/zjunlp_-_OceanGPT-2B-v0.1-gguf/blob/main/OceanGPT-2B-v0.1.Q6_K.gguf) | Q6_K | 2.2GB |
38
+ | [OceanGPT-2B-v0.1.Q8_0.gguf](https://huggingface.co/RichardErkhov/zjunlp_-_OceanGPT-2B-v0.1-gguf/blob/main/OceanGPT-2B-v0.1.Q8_0.gguf) | Q8_0 | 2.7GB |
39
+
40
+
41
+
42
+
43
+ Original model description:
44
+ ---
45
+ license: mit
46
+ pipeline_tag: text-generation
47
+ tags:
48
+ - ocean
49
+ - text-generation-inference
50
+ - oceangpt
51
+ language:
52
+ - en
53
+ - zh
54
+ datasets:
55
+ - zjunlp/OceanInstruct
56
+ ---
57
+
58
+
59
+ <div align="center">
60
+ <img src="logo.jpg" width="300px">
61
+
62
+ **OceanGPT(沧渊): A Large Language Model for Ocean Science Tasks**
63
+
64
+ <p align="center">
65
+ <a href="https://github.com/zjunlp/OceanGPT">Project</a> •
66
+ <a href="https://arxiv.org/abs/2310.02031">Paper</a> •
67
+ <a href="https://huggingface.co/collections/zjunlp/oceangpt-664cc106358fdd9f09aa5157">Models</a> •
68
+ <a href="http://oceangpt.zjukg.cn/">Web</a> •
69
+ <a href="#quickstart">Quickstart</a> •
70
+ <a href="#citation">Citation</a>
71
+ </p>
72
+
73
+
74
+ </div>
75
+
76
+ OceanGPT-2B-v0.1 is based on MiniCPM-2B and has been trained on a bilingual dataset in the ocean domain, covering both Chinese and English.
77
+
78
+ - ❗**Disclaimer: This project is purely an academic exploration rather than a product. Please be aware that due to the inherent limitations of large language models, there may be issues such as hallucinations.**
79
+
80
+
81
+ ## ⏩Quickstart
82
+ ### Download the model
83
+
84
+ Download the model: [OceanGPT-2B-v0.1](https://huggingface.co/zjunlp/OceanGPT-2B-v0.1)
85
+
86
+ ```shell
87
+ git lfs install
88
+ git clone https://huggingface.co/zjunlp/OceanGPT-2B-v0.1
89
+ ```
90
+ or
91
+ ```
92
+ huggingface-cli download --resume-download zjunlp/OceanGPT-2B-v0.1 --local-dir OceanGPT-2B-v0.1 --local-dir-use-symlinks False
93
+ ```
94
+ ### Inference
95
+
96
+ ```python
97
+ from transformers import AutoModelForCausalLM, AutoTokenizer
98
+ import torch
99
+ device = "cuda" # the device to load the model onto
100
+ path = 'YOUR-MODEL-PATH'
101
+ model = AutoModelForCausalLM.from_pretrained(
102
+ path,
103
+ torch_dtype=torch.bfloat16,
104
+ device_map="auto"
105
+ )
106
+ tokenizer = AutoTokenizer.from_pretrained(path)
107
+
108
+ prompt = "Which is the largest ocean in the world?"
109
+ messages = [
110
+ {"role": "system", "content": "You are a helpful assistant."},
111
+ {"role": "user", "content": prompt}
112
+ ]
113
+ text = tokenizer.apply_chat_template(
114
+ messages,
115
+ tokenize=False,
116
+ add_generation_prompt=True
117
+ )
118
+ model_inputs = tokenizer([text], return_tensors="pt").to(device)
119
+
120
+ generated_ids = model.generate(
121
+ model_inputs.input_ids,
122
+ max_new_tokens=512
123
+ )
124
+ generated_ids = [
125
+ output_ids[len(input_ids):] for input_ids, output_ids in zip(model_inputs.input_ids, generated_ids)
126
+ ]
127
+
128
+ response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]
129
+ ```
130
+
131
+ ## 📌Models
132
+
133
+ | Model Name | HuggingFace | WiseModel | ModelScope |
134
+ |-------------------|-----------------------------------------------------------------------------------|----------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------|
135
+ | OceanGPT-14B-v0.1 (based on Qwen) | <a href="https://huggingface.co/zjunlp/OceanGPT-14B-v0.1" target="_blank">14B</a> | <a href="https://wisemodel.cn/models/zjunlp/OceanGPT-14B-v0.1" target="_blank">14B</a> | <a href="https://modelscope.cn/models/ZJUNLP/OceanGPT-14B-v0.1" target="_blank">14B</a> |
136
+ | OceanGPT-7B-v0.2 (based on Qwen) | <a href="https://huggingface.co/zjunlp/OceanGPT-7b-v0.2" target="_blank">7B</a> | <a href="https://wisemodel.cn/models/zjunlp/OceanGPT-7b-v0.2" target="_blank">7B</a> | <a href="https://modelscope.cn/models/ZJUNLP/OceanGPT-7b-v0.2" target="_blank">7B</a> |
137
+ | OceanGPT-2B-v0.1 (based on MiniCPM) | <a href="https://huggingface.co/zjunlp/OceanGPT-2B-v0.1" target="_blank">2B</a> | <a href="https://wisemodel.cn/models/zjunlp/OceanGPT-2b-v0.1" target="_blank">2B</a> | <a href="https://modelscope.cn/models/ZJUNLP/OceanGPT-2B-v0.1" target="_blank">2B</a> |
138
+
139
+
140
+ ## 🌻Acknowledgement
141
+
142
+ OceanGPT(沧渊) is trained based on the open-sourced large language models including [Qwen](https://huggingface.co/Qwen), [MiniCPM](https://huggingface.co/collections/openbmb/minicpm-2b-65d48bf958302b9fd25b698f), [LLaMA](https://huggingface.co/meta-llama). Thanks for their great contributions!
143
+
144
+ ## Limitations
145
+
146
+ - The model may have hallucination issues.
147
+
148
+ - We did not optimize the identity and the model may generate identity information similar to that of Qwen/MiniCPM/LLaMA/GPT series models.
149
+
150
+ - The model's output is influenced by prompt tokens, which may result in inconsistent results across multiple attempts.
151
+
152
+ - The model requires the inclusion of specific simulator code instructions for training in order to possess simulated embodied intelligence capabilities (the simulator is subject to copyright restrictions and cannot be made available for now), and its current capabilities are quite limited.
153
+
154
+
155
+ ### 🚩Citation
156
+
157
+ Please cite the following paper if you use OceanGPT in your work.
158
+
159
+ ```bibtex
160
+ @article{bi2023oceangpt,
161
+ title={OceanGPT: A Large Language Model for Ocean Science Tasks},
162
+ author={Bi, Zhen and Zhang, Ningyu and Xue, Yida and Ou, Yixin and Ji, Daxiong and Zheng, Guozhou and Chen, Huajun},
163
+ journal={arXiv preprint arXiv:2310.02031},
164
+ year={2023}
165
+ }
166
+
167
+ ```
168
+