spow12's picture
Update README.md
d0a3f1a verified
metadata
library_name: transformers
license: cc-by-nc-4.0
language:
  - ko
  - en

spow12/POLAR-14B_4.3_very_big_sft

Model Description

This model is a Supervised fine-tuned version of x2bee/POLAR-14B-v0.2 with DeepSpeed and trl for korean.

Trained Data

  • Trained with public data and private data and Generated data (about 50k)

Usage

from transformers import TextStreamer, pipeline, AutoTokenizer, AutoModelForCausalLM

model_id = 'spow12/POLAR-14B_4.3_very_big_sft'
tokenizer = AutoTokenizer.from_pretrained(model_id)
# %%
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    attn_implementation="flash_attention_2", 
    device_map='auto',
)
model.eval()

pipe = pipeline("text-generation", model=model, tokenizer=tokenizer, device_map='auto')

streamer = TextStreamer(tokenizer)

generation_configs = dict(
    max_new_tokens=2048,
    num_return_sequences=1, 
    temperature=0.1,
    # early_stopping=True,
    repetition_penalty=1.2,
    num_beams=1,
    do_sample=True,
    top_k=20,
    top_p=0.9,
    eos_token_id=tokenizer.eos_token_id,
    pad_token_id=tokenizer.eos_token_id,
    streamer=streamer
)

sys_message = """당신은 μΉœμ ˆν•œ μ±—λ΄‡μœΌλ‘œμ„œ μƒλŒ€λ°©μ˜ μš”μ²­μ— μ΅œλŒ€ν•œ μžμ„Έν•˜κ³  μΉœμ ˆν•˜κ²Œ λ‹΅ν•΄μ•Όν•©λ‹ˆλ‹€. 
μ‚¬μš©μžκ°€ μ œκ³΅ν•˜λŠ” 정보λ₯Ό μ„Έμ‹¬ν•˜κ²Œ λΆ„μ„ν•˜μ—¬ μ‚¬μš©μžμ˜ μ˜λ„λ₯Ό μ‹ μ†ν•˜κ²Œ νŒŒμ•…ν•˜κ³  그에 따라 닡변을 μƒμ„±ν•΄μ•Όν•©λ‹ˆλ‹€.  

항상 맀우 μžμ—°μŠ€λŸ¬μš΄ ν•œκ΅­μ–΄λ‘œ μ‘λ‹΅ν•˜μ„Έμš”."""

message = [
    {
        'role': "system",
        'content': sys_message
    },
    {
        'role': 'user',
        'content': "ν˜„μž¬μ˜ κ²½μ œμƒν™©μ— λŒ€ν•΄ μ–΄λ–»κ²Œ 생각해?."
    }
]
conversation = pipe(message, **generation_configs)
conversation[-1]

License

This model is licensed under the cc-by-nc-4.0. which allows others to share and adapt the model for non-commercial purposes.

Here is Original Readme.md