|
--- |
|
library_name: transformers |
|
license: cc-by-nc-4.0 |
|
language: |
|
- ko |
|
- en |
|
--- |
|
|
|
# spow12/POLAR-14B_4.3_very_big_sft |
|
|
|
<!-- Provide a quick summary of what the model is/does. --> |
|
|
|
<!--This modelcard aims to be a base template for new models. It has been generated using [this raw template](https://github.com/huggingface/huggingface_hub/blob/main/src/huggingface_hub/templates/modelcard_template.md?plain=1).--> |
|
|
|
|
|
### Model Description |
|
|
|
<!-- Provide a longer summary of what this model is. --> |
|
|
|
This model is a Supervised fine-tuned version of [x2bee/POLAR-14B-v0.2](https://huggingface.co/x2bee/POLAR-14B-v0.2) with DeepSpeed and trl for korean. |
|
|
|
### Trained Data |
|
|
|
- Trained with public data and private data and Generated data (about 50k) |
|
|
|
### Usage |
|
```python |
|
from transformers import TextStreamer, pipeline, AutoTokenizer, AutoModelForCausalLM |
|
|
|
model_id = 'spow12/POLAR-14B_4.3_very_big_sft' |
|
tokenizer = AutoTokenizer.from_pretrained(model_id) |
|
# %% |
|
model = AutoModelForCausalLM.from_pretrained( |
|
model_id, |
|
torch_dtype=torch.bfloat16, |
|
attn_implementation="flash_attention_2", |
|
device_map='auto', |
|
) |
|
model.eval() |
|
|
|
pipe = pipeline("text-generation", model=model, tokenizer=tokenizer, device_map='auto') |
|
|
|
streamer = TextStreamer(tokenizer) |
|
|
|
generation_configs = dict( |
|
max_new_tokens=2048, |
|
num_return_sequences=1, |
|
temperature=0.1, |
|
# early_stopping=True, |
|
repetition_penalty=1.2, |
|
num_beams=1, |
|
do_sample=True, |
|
top_k=20, |
|
top_p=0.9, |
|
eos_token_id=tokenizer.eos_token_id, |
|
pad_token_id=tokenizer.eos_token_id, |
|
streamer=streamer |
|
) |
|
|
|
sys_message = """λΉμ μ μΉμ ν μ±λ΄μΌλ‘μ μλλ°©μ μμ²μ μ΅λν μμΈνκ³ μΉμ νκ² λ΅ν΄μΌν©λλ€. |
|
μ¬μ©μκ° μ 곡νλ μ 보λ₯Ό μΈμ¬νκ² λΆμνμ¬ μ¬μ©μμ μλλ₯Ό μ μνκ² νμ
νκ³ κ·Έμ λ°λΌ λ΅λ³μ μμ±ν΄μΌν©λλ€. |
|
|
|
νμ λ§€μ° μμ°μ€λ¬μ΄ νκ΅μ΄λ‘ μλ΅νμΈμ.""" |
|
|
|
message = [ |
|
{ |
|
'role': "system", |
|
'content': sys_message |
|
}, |
|
{ |
|
'role': 'user', |
|
'content': "νμ¬μ κ²½μ μν©μ λν΄ μ΄λ»κ² μκ°ν΄?." |
|
} |
|
] |
|
conversation = pipe(message, **generation_configs) |
|
conversation[-1] |
|
``` |
|
|
|
### License |
|
|
|
This model is licensed under the cc-by-nc-4.0. which allows others to share and adapt the model for non-commercial purposes. |
|
|
|
Here is Original Readme.md |
|
|