Hugging Face's logo

Himalayan-wildcat
/
gemma-2-9b-finetune

Transformers
Safetensors
Japanese
Inference Endpoints
Model card Files Community
gemma-2-9b-finetune
File size: 2,816 Bytes 
33e67ab
 
fcd8c0b
 
 
 
 
 
 
33e67ab
 
fcd8c0b
33e67ab
 
 
 
 
fcd8c0b
 
 
19762d9
 
33e67ab
 
 
497c689
 
 
 
fcd8c0b
 
 
 
 
 
 
 
 
 
 
 
 
 
 
150e2f5
fcd8c0b
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1fc86a1
fcd8c0b
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
 
---
library_name: transformers
license: mit
language:
- ja
base_model:
- google/gemma-2-9b
datasets:
- DeL-TaiseiOzaki/Tengentoppa-sft-v1.0
---

## Model Description

<!-- Provide a longer summary of what this model is. -->

This is the model card of a πŸ€— transformers model that has been pushed on the Hub. This model card has been automatically generated.

- **Developed by:** Hiroaki Hara(@Himalayan-wildcat)
- **Language(s) (NLP):** ja
- **License:** MIT
- **Finetuned from model:** Himalayan-wildcat/gemma-2-9b-finetune
- **Datasets:** DeL-TaiseiOzaki/Tengentoppa-sft-v1.0

## Uses

```
pip install peft~=0.14 tqdm~=4.67 transformers~=4.47
```

```Python
import json
import re

import torch
from peft import PeftModel
from tqdm import tqdm
from transformers import (
    AutoModelForCausalLM,
    AutoTokenizer,
    BitsAndBytesConfig,
)


model_id = "Himalayan-wildcat/gemma-2-9b-finetune"
hf_token = "/YOUR_HUGGING_FACE_TOKEN/"
test_jsonl_data = "elyza-tasks-100-TV_0.jsonl"

bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.bfloat16)

model = AutoModelForCausalLM.from_pretrained(
    model_id,
    quantization_config=bnb_config,
    device_map="auto",
    token = hf_token)

tokenizer = AutoTokenizer.from_pretrained(
    model_id,
    trust_remote_code=True,
    token=hf_token)

datasets = []
with open(test_jsonl_data) as f:
    item = ""
    for line in f:
      line = line.strip()
      item += line
      if item.endswith("}"):
        datasets.append(json.loads(item))
        item = ""

results = []
for data in tqdm(datasets):
    input_: str = data["input"]

    prompt = f"""
[要仢]
- δΈŽγˆγ‚‰γ‚ŒγŸθ³ͺε•γ¨εŒγ˜θ¨€θͺžγ§ε›žη­”をしてください。
- ε›žη­”γŒεˆ†γ‹γ‚‰γͺγ„ε ΄εˆγ―γ€θ™šε½γ‚’γ›γšγ€γ€Œεˆ†γ‹γ‚ŠγΎγ›γ‚“γ€‚γ€γ¨ε›žη­”γ‚’γ—γ¦γγ γ•γ„γ€‚

[θ³ͺ問]
{input_}

[ε›žη­”]"""

    tokenized_input = tokenizer(prompt, return_tensors="pt").to("cuda")
    
    with torch.no_grad():
        generated_ids = model.generate(
            tokenized_input.input_ids,            
            attention_mask=tokenized_input.attention_mask,
            max_new_tokens=500,
            do_sample=False,
            repetition_penalty=1.2,
            pad_token_id=tokenizer.eos_token_id)

    generated_ids = [
        output_ids[len(input_ids):] for input_ids, output_ids in zip(tokenized_input.input_ids, generated_ids)
    ]
    output = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]
    results.append({"task_id": data["task_id"], "input": input_, "output": output})

jsonl_id = re.sub(".*/", "", model_id)
with open(f"./{jsonl_id}-outputs.jsonl", 'w', encoding='utf-8') as f:
    for result in results:
        json.dump(result, f, ensure_ascii=False)
        f.write('\n')
```