---
base_model: meta-llama/Llama-2-7b-hf
library_name: peft
license: llama2
tags:
- generated_from_trainer
model-index:
- name: legal_llama
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# legal_llama

This model is a fine-tuned version of [meta-llama/Llama-2-7b-hf](https://huggingface.co/meta-llama/Llama-2-7b-hf) on an unknown dataset.
It achieves the following results on the evaluation set:
- Loss: 0.8815
- Law Precision: 0.4949
- Law Recall: 0.6622
- Law F1: 0.5665
- Law Number: 74
- Violated by Precision: 0.5116
- Violated by Recall: 0.6197
- Violated by F1: 0.5605
- Violated by Number: 71
- Violated on Precision: 0.2373
- Violated on Recall: 0.28
- Violated on F1: 0.2569
- Violated on Number: 50
- Violation Precision: 0.2670
- Violation Recall: 0.4405
- Violation F1: 0.3325
- Violation Number: 597
- Overall Precision: 0.3011
- Overall Recall: 0.4672
- Overall F1: 0.3662
- Overall Accuracy: 0.9060

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0001
- train_batch_size: 16
- eval_batch_size: 16
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 100

### Training results

| Training Loss | Epoch | Step | Validation Loss | Law Precision | Law Recall | Law F1 | Law Number | Violated by Precision | Violated by Recall | Violated by F1 | Violated by Number | Violated on Precision | Violated on Recall | Violated on F1 | Violated on Number | Violation Precision | Violation Recall | Violation F1 | Violation Number | Overall Precision | Overall Recall | Overall F1 | Overall Accuracy |
|:-------------:|:-----:|:----:|:---------------:|:-------------:|:----------:|:------:|:----------:|:---------------------:|:------------------:|:--------------:|:------------------:|:---------------------:|:------------------:|:--------------:|:------------------:|:-------------------:|:----------------:|:------------:|:----------------:|:-----------------:|:--------------:|:----------:|:----------------:|
| No log        | 1.0   | 45   | 0.6132          | 0.0           | 0.0        | 0.0    | 74         | 0.0                   | 0.0                | 0.0            | 71                 | 0.0                   | 0.0                | 0.0            | 50                 | 0.0046              | 0.0084           | 0.0059       | 597              | 0.0046            | 0.0063         | 0.0053     | 0.7722           |
| No log        | 2.0   | 90   | 0.5228          | 0.0170        | 0.0405     | 0.0240 | 74         | 0.0                   | 0.0                | 0.0            | 71                 | 0.0                   | 0.0                | 0.0            | 50                 | 0.0242              | 0.0519           | 0.0330       | 597              | 0.0231            | 0.0429         | 0.0301     | 0.8191           |
| No log        | 3.0   | 135  | 0.4312          | 0.1231        | 0.1081     | 0.1151 | 74         | 0.0526                | 0.0423             | 0.0469         | 71                 | 0.0                   | 0.0                | 0.0            | 50                 | 0.0549              | 0.1240           | 0.0761       | 597              | 0.0575            | 0.1073         | 0.0749     | 0.8525           |
| No log        | 4.0   | 180  | 0.4691          | 0.2281        | 0.1757     | 0.1985 | 74         | 0.3333                | 0.0986             | 0.1522         | 71                 | 0.0                   | 0.0                | 0.0            | 50                 | 0.0651              | 0.1106           | 0.0819       | 597              | 0.0787            | 0.1086         | 0.0912     | 0.8512           |
| No log        | 5.0   | 225  | 0.3555          | 0.2388        | 0.2162     | 0.2270 | 74         | 0.3226                | 0.1408             | 0.1961         | 71                 | 0.0                   | 0.0                | 0.0            | 50                 | 0.1525              | 0.3166           | 0.2059       | 597              | 0.1603            | 0.2715         | 0.2016     | 0.8840           |
| No log        | 6.0   | 270  | 0.3772          | 0.2842        | 0.3649     | 0.3195 | 74         | 0.5818                | 0.4507             | 0.5079         | 71                 | 0.1875                | 0.06               | 0.0909         | 50                 | 0.1148              | 0.2010           | 0.1462       | 597              | 0.1503            | 0.2298         | 0.1817     | 0.8852           |
| No log        | 7.0   | 315  | 0.3975          | 0.4342        | 0.4459     | 0.44   | 74         | 0.5                   | 0.5634             | 0.5298         | 71                 | 0.0952                | 0.12               | 0.1062         | 50                 | 0.1262              | 0.2278           | 0.1624       | 597              | 0.1658            | 0.2715         | 0.2058     | 0.8885           |
| No log        | 8.0   | 360  | 0.3774          | 0.4146        | 0.6892     | 0.5178 | 74         | 0.3636                | 0.6761             | 0.4729         | 71                 | 0.125                 | 0.16               | 0.1404         | 50                 | 0.1783              | 0.3199           | 0.2290       | 597              | 0.2144            | 0.3763         | 0.2731     | 0.8927           |
| No log        | 9.0   | 405  | 0.4564          | 0.4071        | 0.6216     | 0.4920 | 74         | 0.2740                | 0.5634             | 0.3687         | 71                 | 0.12                  | 0.18               | 0.144          | 50                 | 0.2046              | 0.3400           | 0.2555       | 597              | 0.2247            | 0.3763         | 0.2814     | 0.8942           |
| No log        | 10.0  | 450  | 0.4914          | 0.51          | 0.6892     | 0.5862 | 74         | 0.4018                | 0.6338             | 0.4918         | 71                 | 0.1443                | 0.28               | 0.1905         | 50                 | 0.1878              | 0.3250           | 0.2380       | 597              | 0.2265            | 0.3838         | 0.2849     | 0.8925           |
| No log        | 11.0  | 495  | 0.5342          | 0.3684        | 0.5676     | 0.4468 | 74         | 0.4737                | 0.6338             | 0.5422         | 71                 | 0.1633                | 0.16               | 0.1616         | 50                 | 0.1859              | 0.3266           | 0.2369       | 597              | 0.2219            | 0.3662         | 0.2763     | 0.8937           |
| 0.3436        | 12.0  | 540  | 0.5091          | 0.4545        | 0.6757     | 0.5435 | 74         | 0.3133                | 0.6620             | 0.4253         | 71                 | 0.1494                | 0.26               | 0.1898         | 50                 | 0.2422              | 0.4405           | 0.3125       | 597              | 0.2603            | 0.4710         | 0.3353     | 0.8975           |
| 0.3436        | 13.0  | 585  | 0.5243          | 0.4630        | 0.6757     | 0.5495 | 74         | 0.4034                | 0.6761             | 0.5053         | 71                 | 0.1558                | 0.24               | 0.1890         | 50                 | 0.2190              | 0.4372           | 0.2918       | 597              | 0.2480            | 0.4684         | 0.3243     | 0.8950           |
| 0.3436        | 14.0  | 630  | 0.5990          | 0.3836        | 0.7568     | 0.5091 | 74         | 0.3511                | 0.6479             | 0.4554         | 71                 | 0.1647                | 0.28               | 0.2074         | 50                 | 0.1954              | 0.3384           | 0.2477       | 597              | 0.2278            | 0.4015         | 0.2907     | 0.8948           |
| 0.3436        | 15.0  | 675  | 0.5916          | 0.4483        | 0.7027     | 0.5474 | 74         | 0.3701                | 0.6620             | 0.4747         | 71                 | 0.2209                | 0.38               | 0.2794         | 50                 | 0.2165              | 0.3869           | 0.2776       | 597              | 0.25              | 0.4407         | 0.3190     | 0.8966           |
| 0.3436        | 16.0  | 720  | 0.5755          | 0.4490        | 0.5946     | 0.5116 | 74         | 0.4713                | 0.5775             | 0.5190         | 71                 | 0.1970                | 0.26               | 0.2241         | 50                 | 0.2228              | 0.4020           | 0.2867       | 597              | 0.2545            | 0.4268         | 0.3189     | 0.8999           |
| 0.3436        | 17.0  | 765  | 0.6197          | 0.4815        | 0.7027     | 0.5714 | 74         | 0.4608                | 0.6620             | 0.5434         | 71                 | 0.2727                | 0.42               | 0.3307         | 50                 | 0.2051              | 0.3484           | 0.2582       | 597              | 0.2521            | 0.4141         | 0.3134     | 0.8956           |
| 0.3436        | 18.0  | 810  | 0.6345          | 0.5402        | 0.6351     | 0.5839 | 74         | 0.5152                | 0.4789             | 0.4964         | 71                 | 0.22                  | 0.22               | 0.22           | 50                 | 0.2322              | 0.4037           | 0.2948       | 597              | 0.2683            | 0.4205         | 0.3276     | 0.9011           |
| 0.3436        | 19.0  | 855  | 0.6967          | 0.3590        | 0.5676     | 0.4398 | 74         | 0.4167                | 0.5634             | 0.4790         | 71                 | 0.2206                | 0.3                | 0.2542         | 50                 | 0.1922              | 0.3300           | 0.2429       | 597              | 0.2251            | 0.3712         | 0.2803     | 0.8940           |
| 0.3436        | 20.0  | 900  | 0.6172          | 0.4486        | 0.6486     | 0.5304 | 74         | 0.4831                | 0.6056             | 0.5375         | 71                 | 0.2286                | 0.32               | 0.2667         | 50                 | 0.2313              | 0.4305           | 0.3009       | 597              | 0.2643            | 0.4596         | 0.3356     | 0.8989           |
| 0.3436        | 21.0  | 945  | 0.6819          | 0.4717        | 0.6757     | 0.5556 | 74         | 0.4059                | 0.5775             | 0.4767         | 71                 | 0.2586                | 0.3                | 0.2778         | 50                 | 0.2106              | 0.3601           | 0.2658       | 597              | 0.2496            | 0.4053         | 0.3090     | 0.9004           |
| 0.3436        | 22.0  | 990  | 0.6565          | 0.4444        | 0.6486     | 0.5275 | 74         | 0.4725                | 0.6056             | 0.5309         | 71                 | 0.2807                | 0.32               | 0.2991         | 50                 | 0.2531              | 0.4439           | 0.3224       | 597              | 0.2855            | 0.4697         | 0.3551     | 0.9044           |
| 0.0088        | 23.0  | 1035 | 0.6754          | 0.4949        | 0.6622     | 0.5665 | 74         | 0.5176                | 0.6197             | 0.5641         | 71                 | 0.25                  | 0.28               | 0.2642         | 50                 | 0.2547              | 0.4322           | 0.3205       | 597              | 0.2913            | 0.4609         | 0.3570     | 0.9049           |
| 0.0088        | 24.0  | 1080 | 0.6782          | 0.48          | 0.6486     | 0.5517 | 74         | 0.5119                | 0.6056             | 0.5548         | 71                 | 0.2281                | 0.26               | 0.2430         | 50                 | 0.2673              | 0.4523           | 0.3360       | 597              | 0.2990            | 0.4722         | 0.3661     | 0.9062           |
| 0.0088        | 25.0  | 1125 | 0.6931          | 0.4902        | 0.6757     | 0.5682 | 74         | 0.5                   | 0.6197             | 0.5535         | 71                 | 0.2632                | 0.3                | 0.2804         | 50                 | 0.2667              | 0.4472           | 0.3342       | 597              | 0.3013            | 0.4747         | 0.3686     | 0.9051           |
| 0.0088        | 26.0  | 1170 | 0.6949          | 0.4554        | 0.6216     | 0.5257 | 74         | 0.5422                | 0.6338             | 0.5844         | 71                 | 0.2407                | 0.26               | 0.25           | 50                 | 0.2689              | 0.4523           | 0.3373       | 597              | 0.3011            | 0.4722         | 0.3677     | 0.9052           |
| 0.0088        | 27.0  | 1215 | 0.7091          | 0.4752        | 0.6486     | 0.5486 | 74         | 0.5181                | 0.6056             | 0.5584         | 71                 | 0.2414                | 0.28               | 0.2593         | 50                 | 0.2604              | 0.4405           | 0.3273       | 597              | 0.2939            | 0.4646         | 0.3601     | 0.9055           |
| 0.0088        | 28.0  | 1260 | 0.7128          | 0.5           | 0.6892     | 0.5795 | 74         | 0.4944                | 0.6197             | 0.55           | 71                 | 0.2586                | 0.3                | 0.2778         | 50                 | 0.2657              | 0.4456           | 0.3329       | 597              | 0.3008            | 0.4747         | 0.3683     | 0.9055           |
| 0.0088        | 29.0  | 1305 | 0.7147          | 0.4653        | 0.6351     | 0.5371 | 74         | 0.5057                | 0.6197             | 0.5570         | 71                 | 0.2456                | 0.28               | 0.2617         | 50                 | 0.2659              | 0.4489           | 0.3340       | 597              | 0.2977            | 0.4710         | 0.3648     | 0.9058           |
| 0.0088        | 30.0  | 1350 | 0.7223          | 0.4653        | 0.6351     | 0.5371 | 74         | 0.5                   | 0.6056             | 0.5478         | 71                 | 0.2414                | 0.28               | 0.2593         | 50                 | 0.2639              | 0.4456           | 0.3315       | 597              | 0.2953            | 0.4672         | 0.3619     | 0.9056           |
| 0.0088        | 31.0  | 1395 | 0.7284          | 0.4752        | 0.6486     | 0.5486 | 74         | 0.4945                | 0.6338             | 0.5556         | 71                 | 0.2069                | 0.24               | 0.2222         | 50                 | 0.2696              | 0.4489           | 0.3369       | 597              | 0.2998            | 0.4710         | 0.3664     | 0.9059           |
| 0.0088        | 32.0  | 1440 | 0.7287          | 0.49          | 0.6622     | 0.5632 | 74         | 0.5287                | 0.6479             | 0.5823         | 71                 | 0.2667                | 0.32               | 0.2909         | 50                 | 0.2654              | 0.4472           | 0.3331       | 597              | 0.3017            | 0.4773         | 0.3697     | 0.9055           |
| 0.0088        | 33.0  | 1485 | 0.7377          | 0.48          | 0.6486     | 0.5517 | 74         | 0.5357                | 0.6338             | 0.5806         | 71                 | 0.2069                | 0.24               | 0.2222         | 50                 | 0.2676              | 0.4456           | 0.3344       | 597              | 0.3002            | 0.4684         | 0.3659     | 0.9059           |
| 0.0003        | 34.0  | 1530 | 0.7451          | 0.4747        | 0.6351     | 0.5434 | 74         | 0.5357                | 0.6338             | 0.5806         | 71                 | 0.2143                | 0.24               | 0.2264         | 50                 | 0.2694              | 0.4472           | 0.3363       | 597              | 0.3016            | 0.4684         | 0.3670     | 0.9062           |
| 0.0003        | 35.0  | 1575 | 0.7493          | 0.48          | 0.6486     | 0.5517 | 74         | 0.5116                | 0.6197             | 0.5605         | 71                 | 0.1754                | 0.2                | 0.1869         | 50                 | 0.2739              | 0.4506           | 0.3407       | 597              | 0.3029            | 0.4684         | 0.3679     | 0.9058           |
| 0.0003        | 36.0  | 1620 | 0.7565          | 0.48          | 0.6486     | 0.5517 | 74         | 0.5116                | 0.6197             | 0.5605         | 71                 | 0.2069                | 0.24               | 0.2222         | 50                 | 0.2660              | 0.4389           | 0.3312       | 597              | 0.2978            | 0.4621         | 0.3622     | 0.9060           |
| 0.0003        | 37.0  | 1665 | 0.7559          | 0.48          | 0.6486     | 0.5517 | 74         | 0.5116                | 0.6197             | 0.5605         | 71                 | 0.2069                | 0.24               | 0.2222         | 50                 | 0.2691              | 0.4422           | 0.3346       | 597              | 0.3004            | 0.4646         | 0.3649     | 0.9058           |
| 0.0003        | 38.0  | 1710 | 0.7632          | 0.5           | 0.6486     | 0.5647 | 74         | 0.5357                | 0.6338             | 0.5806         | 71                 | 0.2069                | 0.24               | 0.2222         | 50                 | 0.2676              | 0.4389           | 0.3325       | 597              | 0.3016            | 0.4634         | 0.3654     | 0.9062           |
| 0.0003        | 39.0  | 1755 | 0.7659          | 0.4747        | 0.6351     | 0.5434 | 74         | 0.5057                | 0.6197             | 0.5570         | 71                 | 0.1905                | 0.24               | 0.2124         | 50                 | 0.2621              | 0.4338           | 0.3268       | 597              | 0.2926            | 0.4571         | 0.3568     | 0.9057           |
| 0.0003        | 40.0  | 1800 | 0.7705          | 0.4898        | 0.6486     | 0.5581 | 74         | 0.5                   | 0.6197             | 0.5535         | 71                 | 0.2034                | 0.24               | 0.2202         | 50                 | 0.2642              | 0.4355           | 0.3289       | 597              | 0.2962            | 0.4596         | 0.3602     | 0.9058           |
| 0.0003        | 41.0  | 1845 | 0.7740          | 0.4948        | 0.6486     | 0.5614 | 74         | 0.5244                | 0.6056             | 0.5621         | 71                 | 0.2069                | 0.24               | 0.2222         | 50                 | 0.2704              | 0.4439           | 0.3361       | 597              | 0.3024            | 0.4646         | 0.3664     | 0.9062           |
| 0.0003        | 42.0  | 1890 | 0.7735          | 0.4752        | 0.6486     | 0.5486 | 74         | 0.4944                | 0.6197             | 0.55           | 71                 | 0.2034                | 0.24               | 0.2202         | 50                 | 0.2666              | 0.4439           | 0.3331       | 597              | 0.2969            | 0.4659         | 0.3627     | 0.9059           |
| 0.0003        | 43.0  | 1935 | 0.7785          | 0.49          | 0.6622     | 0.5632 | 74         | 0.4944                | 0.6197             | 0.55           | 71                 | 0.2                   | 0.24               | 0.2182         | 50                 | 0.2654              | 0.4405           | 0.3312       | 597              | 0.2968            | 0.4646         | 0.3622     | 0.9060           |
| 0.0003        | 44.0  | 1980 | 0.7821          | 0.4898        | 0.6486     | 0.5581 | 74         | 0.5294                | 0.6338             | 0.5769         | 71                 | 0.2                   | 0.24               | 0.2182         | 50                 | 0.2652              | 0.4372           | 0.3302       | 597              | 0.2983            | 0.4621         | 0.3626     | 0.9062           |
| 0.0001        | 45.0  | 2025 | 0.7853          | 0.4848        | 0.6486     | 0.5549 | 74         | 0.5238                | 0.6197             | 0.5677         | 71                 | 0.2034                | 0.24               | 0.2202         | 50                 | 0.2667              | 0.4422           | 0.3327       | 597              | 0.2987            | 0.4646         | 0.3636     | 0.9065           |
| 0.0001        | 46.0  | 2070 | 0.7897          | 0.4898        | 0.6486     | 0.5581 | 74         | 0.5294                | 0.6338             | 0.5769         | 71                 | 0.2203                | 0.26               | 0.2385         | 50                 | 0.2640              | 0.4355           | 0.3287       | 597              | 0.2983            | 0.4621         | 0.3626     | 0.9062           |
| 0.0001        | 47.0  | 2115 | 0.7899          | 0.48          | 0.6486     | 0.5517 | 74         | 0.5172                | 0.6338             | 0.5696         | 71                 | 0.2097                | 0.26               | 0.2321         | 50                 | 0.2670              | 0.4405           | 0.3325       | 597              | 0.2990            | 0.4659         | 0.3643     | 0.9065           |
| 0.0001        | 48.0  | 2160 | 0.7959          | 0.48          | 0.6486     | 0.5517 | 74         | 0.5116                | 0.6197             | 0.5605         | 71                 | 0.2203                | 0.26               | 0.2385         | 50                 | 0.2682              | 0.4389           | 0.3329       | 597              | 0.3003            | 0.4634         | 0.3644     | 0.9064           |
| 0.0001        | 49.0  | 2205 | 0.7945          | 0.49          | 0.6622     | 0.5632 | 74         | 0.5057                | 0.6197             | 0.5570         | 71                 | 0.2295                | 0.28               | 0.2523         | 50                 | 0.2685              | 0.4439           | 0.3346       | 597              | 0.3012            | 0.4697         | 0.3670     | 0.9066           |
| 0.0001        | 50.0  | 2250 | 0.7989          | 0.48          | 0.6486     | 0.5517 | 74         | 0.5116                | 0.6197             | 0.5605         | 71                 | 0.25                  | 0.3                | 0.2727         | 50                 | 0.2672              | 0.4422           | 0.3331       | 597              | 0.3006            | 0.4684         | 0.3662     | 0.9061           |
| 0.0001        | 51.0  | 2295 | 0.8055          | 0.48          | 0.6486     | 0.5517 | 74         | 0.5238                | 0.6197             | 0.5677         | 71                 | 0.2414                | 0.28               | 0.2593         | 50                 | 0.2627              | 0.4338           | 0.3272       | 597              | 0.2972            | 0.4609         | 0.3614     | 0.9060           |
| 0.0001        | 52.0  | 2340 | 0.8055          | 0.49          | 0.6622     | 0.5632 | 74         | 0.5116                | 0.6197             | 0.5605         | 71                 | 0.2333                | 0.28               | 0.2545         | 50                 | 0.2636              | 0.4389           | 0.3294       | 597              | 0.2976            | 0.4659         | 0.3632     | 0.9064           |
| 0.0001        | 53.0  | 2385 | 0.8095          | 0.4898        | 0.6486     | 0.5581 | 74         | 0.5233                | 0.6338             | 0.5732         | 71                 | 0.2241                | 0.26               | 0.2407         | 50                 | 0.2665              | 0.4405           | 0.3321       | 597              | 0.3002            | 0.4659         | 0.3652     | 0.9066           |
| 0.0001        | 54.0  | 2430 | 0.8113          | 0.49          | 0.6622     | 0.5632 | 74         | 0.5233                | 0.6338             | 0.5732         | 71                 | 0.2                   | 0.24               | 0.2182         | 50                 | 0.2684              | 0.4405           | 0.3335       | 597              | 0.3010            | 0.4659         | 0.3657     | 0.9065           |
| 0.0001        | 55.0  | 2475 | 0.8160          | 0.4706        | 0.6486     | 0.5455 | 74         | 0.5116                | 0.6197             | 0.5605         | 71                 | 0.2069                | 0.24               | 0.2222         | 50                 | 0.2636              | 0.4372           | 0.3289       | 597              | 0.2953            | 0.4609         | 0.3600     | 0.9062           |
| 0.0001        | 56.0  | 2520 | 0.8221          | 0.4949        | 0.6622     | 0.5665 | 74         | 0.5116                | 0.6197             | 0.5605         | 71                 | 0.2167                | 0.26               | 0.2364         | 50                 | 0.2673              | 0.4405           | 0.3327       | 597              | 0.3002            | 0.4659         | 0.3652     | 0.9058           |
| 0.0001        | 57.0  | 2565 | 0.8214          | 0.5102        | 0.6757     | 0.5814 | 74         | 0.5233                | 0.6338             | 0.5732         | 71                 | 0.2295                | 0.28               | 0.2523         | 50                 | 0.2685              | 0.4439           | 0.3346       | 597              | 0.3036            | 0.4722         | 0.3696     | 0.9061           |
| 0.0001        | 58.0  | 2610 | 0.8212          | 0.49          | 0.6622     | 0.5632 | 74         | 0.5116                | 0.6197             | 0.5605         | 71                 | 0.2295                | 0.28               | 0.2523         | 50                 | 0.2672              | 0.4422           | 0.3331       | 597              | 0.3004            | 0.4684         | 0.3661     | 0.9060           |
| 0.0001        | 59.0  | 2655 | 0.8258          | 0.4848        | 0.6486     | 0.5549 | 74         | 0.5116                | 0.6197             | 0.5605         | 71                 | 0.2281                | 0.26               | 0.2430         | 50                 | 0.2681              | 0.4456           | 0.3348       | 597              | 0.3006            | 0.4684         | 0.3662     | 0.9062           |
| 0.0001        | 60.0  | 2700 | 0.8291          | 0.49          | 0.6622     | 0.5632 | 74         | 0.5233                | 0.6338             | 0.5732         | 71                 | 0.2203                | 0.26               | 0.2385         | 50                 | 0.2662              | 0.4405           | 0.3319       | 597              | 0.3001            | 0.4672         | 0.3654     | 0.9060           |
| 0.0001        | 61.0  | 2745 | 0.8327          | 0.4848        | 0.6486     | 0.5549 | 74         | 0.5172                | 0.6338             | 0.5696         | 71                 | 0.2333                | 0.28               | 0.2545         | 50                 | 0.2678              | 0.4405           | 0.3331       | 597              | 0.3013            | 0.4672         | 0.3663     | 0.9063           |
| 0.0001        | 62.0  | 2790 | 0.8328          | 0.4851        | 0.6622     | 0.5600 | 74         | 0.5294                | 0.6338             | 0.5769         | 71                 | 0.2414                | 0.28               | 0.2593         | 50                 | 0.2683              | 0.4422           | 0.3340       | 597              | 0.3029            | 0.4697         | 0.3683     | 0.9064           |
| 0.0001        | 63.0  | 2835 | 0.8346          | 0.49          | 0.6622     | 0.5632 | 74         | 0.5233                | 0.6338             | 0.5732         | 71                 | 0.2241                | 0.26               | 0.2407         | 50                 | 0.2701              | 0.4439           | 0.3359       | 597              | 0.3037            | 0.4697         | 0.3689     | 0.9060           |
| 0.0001        | 64.0  | 2880 | 0.8394          | 0.4653        | 0.6351     | 0.5371 | 74         | 0.5116                | 0.6197             | 0.5605         | 71                 | 0.2414                | 0.28               | 0.2593         | 50                 | 0.2706              | 0.4456           | 0.3367       | 597              | 0.3021            | 0.4684         | 0.3673     | 0.9063           |
| 0.0001        | 65.0  | 2925 | 0.8417          | 0.4752        | 0.6486     | 0.5486 | 74         | 0.5116                | 0.6197             | 0.5605         | 71                 | 0.2069                | 0.24               | 0.2222         | 50                 | 0.2667              | 0.4405           | 0.3323       | 597              | 0.2981            | 0.4634         | 0.3628     | 0.9060           |
| 0.0001        | 66.0  | 2970 | 0.8386          | 0.5           | 0.6757     | 0.5747 | 74         | 0.5116                | 0.6197             | 0.5605         | 71                 | 0.2241                | 0.26               | 0.2407         | 50                 | 0.2695              | 0.4456           | 0.3359       | 597              | 0.3030            | 0.4710         | 0.3688     | 0.9061           |
| 0.0001        | 67.0  | 3015 | 0.8435          | 0.5102        | 0.6757     | 0.5814 | 74         | 0.4828                | 0.5915             | 0.5316         | 71                 | 0.2105                | 0.24               | 0.2243         | 50                 | 0.2682              | 0.4439           | 0.3344       | 597              | 0.3               | 0.4659         | 0.3650     | 0.9061           |
| 0.0001        | 68.0  | 3060 | 0.8453          | 0.5           | 0.6622     | 0.5698 | 74         | 0.5233                | 0.6338             | 0.5732         | 71                 | 0.2542                | 0.3                | 0.2752         | 50                 | 0.2663              | 0.4389           | 0.3314       | 597              | 0.3024            | 0.4684         | 0.3675     | 0.9061           |
| 0.0001        | 69.0  | 3105 | 0.8487          | 0.4898        | 0.6486     | 0.5581 | 74         | 0.4886                | 0.6056             | 0.5409         | 71                 | 0.2333                | 0.28               | 0.2545         | 50                 | 0.2704              | 0.4439           | 0.3361       | 597              | 0.3018            | 0.4672         | 0.3667     | 0.9061           |
| 0.0001        | 70.0  | 3150 | 0.8510          | 0.5           | 0.6622     | 0.5698 | 74         | 0.4886                | 0.6056             | 0.5409         | 71                 | 0.2333                | 0.28               | 0.2545         | 50                 | 0.2696              | 0.4439           | 0.3354       | 597              | 0.3019            | 0.4684         | 0.3671     | 0.9062           |
| 0.0001        | 71.0  | 3195 | 0.8522          | 0.4949        | 0.6622     | 0.5665 | 74         | 0.4943                | 0.6056             | 0.5443         | 71                 | 0.2241                | 0.26               | 0.2407         | 50                 | 0.2657              | 0.4405           | 0.3314       | 597              | 0.2982            | 0.4646         | 0.3633     | 0.9058           |
| 0.0001        | 72.0  | 3240 | 0.8543          | 0.5155        | 0.6757     | 0.5848 | 74         | 0.5057                | 0.6197             | 0.5570         | 71                 | 0.2105                | 0.24               | 0.2243         | 50                 | 0.2686              | 0.4422           | 0.3342       | 597              | 0.3023            | 0.4672         | 0.3671     | 0.9062           |
| 0.0001        | 73.0  | 3285 | 0.8577          | 0.4653        | 0.6351     | 0.5371 | 74         | 0.5                   | 0.6197             | 0.5535         | 71                 | 0.2295                | 0.28               | 0.2523         | 50                 | 0.2693              | 0.4439           | 0.3352       | 597              | 0.2998            | 0.4672         | 0.3653     | 0.9059           |
| 0.0001        | 74.0  | 3330 | 0.8584          | 0.5051        | 0.6757     | 0.5780 | 74         | 0.5                   | 0.6197             | 0.5535         | 71                 | 0.25                  | 0.3                | 0.2727         | 50                 | 0.2678              | 0.4405           | 0.3331       | 597              | 0.3027            | 0.4697         | 0.3681     | 0.9060           |
| 0.0001        | 75.0  | 3375 | 0.8609          | 0.5053        | 0.6486     | 0.5680 | 74         | 0.4886                | 0.6056             | 0.5409         | 71                 | 0.2281                | 0.26               | 0.2430         | 50                 | 0.2703              | 0.4456           | 0.3365       | 597              | 0.3023            | 0.4672         | 0.3671     | 0.9063           |
| 0.0001        | 76.0  | 3420 | 0.8586          | 0.5102        | 0.6757     | 0.5814 | 74         | 0.5057                | 0.6197             | 0.5570         | 71                 | 0.2333                | 0.28               | 0.2545         | 50                 | 0.2675              | 0.4422           | 0.3333       | 597              | 0.3019            | 0.4697         | 0.3676     | 0.9061           |
| 0.0001        | 77.0  | 3465 | 0.8621          | 0.5052        | 0.6622     | 0.5731 | 74         | 0.4831                | 0.6056             | 0.5375         | 71                 | 0.2295                | 0.28               | 0.2523         | 50                 | 0.2685              | 0.4439           | 0.3346       | 597              | 0.3006            | 0.4684         | 0.3662     | 0.9061           |
| 0.0           | 78.0  | 3510 | 0.8638          | 0.5           | 0.6622     | 0.5698 | 74         | 0.5                   | 0.6197             | 0.5535         | 71                 | 0.2542                | 0.3                | 0.2752         | 50                 | 0.2675              | 0.4422           | 0.3333       | 597              | 0.3019            | 0.4697         | 0.3676     | 0.9062           |
| 0.0           | 79.0  | 3555 | 0.8642          | 0.4949        | 0.6622     | 0.5665 | 74         | 0.4944                | 0.6197             | 0.55           | 71                 | 0.25                  | 0.3                | 0.2727         | 50                 | 0.2706              | 0.4456           | 0.3367       | 597              | 0.3038            | 0.4722         | 0.3697     | 0.9060           |
| 0.0           | 80.0  | 3600 | 0.8647          | 0.48          | 0.6486     | 0.5517 | 74         | 0.5056                | 0.6338             | 0.5625         | 71                 | 0.2295                | 0.28               | 0.2523         | 50                 | 0.2677              | 0.4422           | 0.3335       | 597              | 0.3002            | 0.4684         | 0.3659     | 0.9059           |
| 0.0           | 81.0  | 3645 | 0.8701          | 0.4747        | 0.6351     | 0.5434 | 74         | 0.5057                | 0.6197             | 0.5570         | 71                 | 0.2373                | 0.28               | 0.2569         | 50                 | 0.2691              | 0.4422           | 0.3346       | 597              | 0.3010            | 0.4659         | 0.3657     | 0.9063           |
| 0.0           | 82.0  | 3690 | 0.8683          | 0.5052        | 0.6622     | 0.5731 | 74         | 0.5116                | 0.6197             | 0.5605         | 71                 | 0.25                  | 0.3                | 0.2727         | 50                 | 0.2680              | 0.4422           | 0.3338       | 597              | 0.3029            | 0.4697         | 0.3683     | 0.9063           |
| 0.0           | 83.0  | 3735 | 0.8703          | 0.5052        | 0.6622     | 0.5731 | 74         | 0.5                   | 0.6197             | 0.5535         | 71                 | 0.2542                | 0.3                | 0.2752         | 50                 | 0.2688              | 0.4422           | 0.3344       | 597              | 0.3034            | 0.4697         | 0.3687     | 0.9060           |
| 0.0           | 84.0  | 3780 | 0.8720          | 0.5155        | 0.6757     | 0.5848 | 74         | 0.5                   | 0.6197             | 0.5535         | 71                 | 0.2586                | 0.3                | 0.2778         | 50                 | 0.2657              | 0.4389           | 0.3310       | 597              | 0.3019            | 0.4684         | 0.3671     | 0.9062           |
| 0.0           | 85.0  | 3825 | 0.8745          | 0.49          | 0.6622     | 0.5632 | 74         | 0.5057                | 0.6197             | 0.5570         | 71                 | 0.2373                | 0.28               | 0.2569         | 50                 | 0.2660              | 0.4389           | 0.3312       | 597              | 0.2998            | 0.4659         | 0.3648     | 0.9056           |
| 0.0           | 86.0  | 3870 | 0.8757          | 0.5           | 0.6622     | 0.5698 | 74         | 0.5                   | 0.6197             | 0.5535         | 71                 | 0.25                  | 0.3                | 0.2727         | 50                 | 0.2649              | 0.4389           | 0.3304       | 597              | 0.2996            | 0.4672         | 0.3651     | 0.9058           |
| 0.0           | 87.0  | 3915 | 0.8754          | 0.49          | 0.6622     | 0.5632 | 74         | 0.5057                | 0.6197             | 0.5570         | 71                 | 0.2586                | 0.3                | 0.2778         | 50                 | 0.2678              | 0.4405           | 0.3331       | 597              | 0.3024            | 0.4684         | 0.3675     | 0.9059           |
| 0.0           | 88.0  | 3960 | 0.8744          | 0.5           | 0.6622     | 0.5698 | 74         | 0.4944                | 0.6197             | 0.55           | 71                 | 0.2333                | 0.28               | 0.2545         | 50                 | 0.2670              | 0.4405           | 0.3325       | 597              | 0.3003            | 0.4672         | 0.3656     | 0.9058           |
| 0.0           | 89.0  | 4005 | 0.8789          | 0.5052        | 0.6622     | 0.5731 | 74         | 0.5057                | 0.6197             | 0.5570         | 71                 | 0.2333                | 0.28               | 0.2545         | 50                 | 0.2696              | 0.4439           | 0.3354       | 597              | 0.3032            | 0.4697         | 0.3685     | 0.9058           |
| 0.0           | 90.0  | 4050 | 0.8788          | 0.4949        | 0.6622     | 0.5665 | 74         | 0.5                   | 0.6197             | 0.5535         | 71                 | 0.2542                | 0.3                | 0.2752         | 50                 | 0.2677              | 0.4422           | 0.3335       | 597              | 0.3019            | 0.4697         | 0.3676     | 0.9058           |
| 0.0           | 91.0  | 4095 | 0.8795          | 0.5           | 0.6757     | 0.5747 | 74         | 0.4889                | 0.6197             | 0.5466         | 71                 | 0.25                  | 0.3                | 0.2727         | 50                 | 0.2674              | 0.4439           | 0.3338       | 597              | 0.3014            | 0.4722         | 0.3679     | 0.9058           |
| 0.0           | 92.0  | 4140 | 0.8818          | 0.5052        | 0.6622     | 0.5731 | 74         | 0.4943                | 0.6056             | 0.5443         | 71                 | 0.2333                | 0.28               | 0.2545         | 50                 | 0.2688              | 0.4439           | 0.3348       | 597              | 0.3016            | 0.4684         | 0.3670     | 0.9059           |
| 0.0           | 93.0  | 4185 | 0.8807          | 0.5           | 0.6622     | 0.5698 | 74         | 0.5057                | 0.6197             | 0.5570         | 71                 | 0.2373                | 0.28               | 0.2569         | 50                 | 0.2696              | 0.4439           | 0.3354       | 597              | 0.3032            | 0.4697         | 0.3685     | 0.9062           |
| 0.0           | 94.0  | 4230 | 0.8810          | 0.5           | 0.6622     | 0.5698 | 74         | 0.4886                | 0.6056             | 0.5409         | 71                 | 0.2167                | 0.26               | 0.2364         | 50                 | 0.2677              | 0.4422           | 0.3335       | 597              | 0.2995            | 0.4659         | 0.3646     | 0.9059           |
| 0.0           | 95.0  | 4275 | 0.8803          | 0.5           | 0.6622     | 0.5698 | 74         | 0.5057                | 0.6197             | 0.5570         | 71                 | 0.2373                | 0.28               | 0.2569         | 50                 | 0.2675              | 0.4422           | 0.3333       | 597              | 0.3014            | 0.4684         | 0.3668     | 0.9059           |
| 0.0           | 96.0  | 4320 | 0.8808          | 0.5102        | 0.6757     | 0.5814 | 74         | 0.5057                | 0.6197             | 0.5570         | 71                 | 0.2333                | 0.28               | 0.2545         | 50                 | 0.2662              | 0.4405           | 0.3319       | 597              | 0.3009            | 0.4684         | 0.3664     | 0.9059           |
| 0.0           | 97.0  | 4365 | 0.8830          | 0.5           | 0.6622     | 0.5698 | 74         | 0.5057                | 0.6197             | 0.5570         | 71                 | 0.2414                | 0.28               | 0.2593         | 50                 | 0.2686              | 0.4422           | 0.3342       | 597              | 0.3026            | 0.4684         | 0.3677     | 0.9059           |
| 0.0           | 98.0  | 4410 | 0.8807          | 0.5052        | 0.6622     | 0.5731 | 74         | 0.5057                | 0.6197             | 0.5570         | 71                 | 0.2373                | 0.28               | 0.2569         | 50                 | 0.2669              | 0.4422           | 0.3329       | 597              | 0.3011            | 0.4684         | 0.3666     | 0.9059           |
| 0.0           | 99.0  | 4455 | 0.8822          | 0.5052        | 0.6622     | 0.5731 | 74         | 0.5057                | 0.6197             | 0.5570         | 71                 | 0.2414                | 0.28               | 0.2593         | 50                 | 0.2682              | 0.4439           | 0.3344       | 597              | 0.3024            | 0.4697         | 0.3680     | 0.9058           |
| 0.0           | 100.0 | 4500 | 0.8815          | 0.4949        | 0.6622     | 0.5665 | 74         | 0.5116                | 0.6197             | 0.5605         | 71                 | 0.2373                | 0.28               | 0.2569         | 50                 | 0.2670              | 0.4405           | 0.3325       | 597              | 0.3011            | 0.4672         | 0.3662     | 0.9060           |


### Framework versions

- PEFT 0.12.0
- Transformers 4.44.0
- Pytorch 2.4.0+cu121
- Datasets 2.20.0
- Tokenizers 0.19.1