---
base_model: meta-llama/Llama-3.2-3B-Instruct
library_name: sft
datasets:
- lianghsun/tw-emergency-medicine-bench
- lianghsun/tw-legal-nlp
- lianghsun/tw-structured-law-article
- lianghsun/tw-legal-synthetic-qa
- lianghsun/tw-law-article-qa
- lianghsun/tw-judgment-qa
- lianghsun/tw-bar-examination-2020-chat
- lianghsun/tw-law-exam-choice
tags:
- legal
- TW
- Taiwan
- ROC
license: llama3.2
language:
- zh
pipeline_tag: text-generation
---

# Model Card for Model lianghsun/Llama-3.2-Taiwan-Legal-3B-Instruct

基於 [meta-llama/Llama-3.2-3B-Instruct](meta-llama/Llama-3.2-3B-Instruct) 模型，透過中華民國台灣法律條文及判決書等相關資料集進行微調。

## Model Details

### Model Description

基於 [meta-llama/Llama-3.2-3B-Instruct](meta-llama/Llama-3.2-3B-Instruct) 模型，此微調過程使用了來自中華民國台灣的法律條文與相關判決書資料集，以提升模型在法律領域的專業知識與應用能力。這些資料集涵蓋了法律條文的結構、判決書的格式，以及法庭上常見的法律語言與術語，使模型能夠更準確地理解和處理與台灣法律體系相關的問題。經過這些微調，模型將能夠更好地為法律專業人士提供幫助，並在台灣法制框架內提供更精準的回應與建議。

- **Developed by:** [Huang Liang Hsun](https://www.linkedin.com/in/lianghsunhuang)
- **Model type:** LlamaForCausalLM
- **Language(s) (NLP)**: 主要處理繁體中文（zh-tw），針對中華民國台灣的法律用語與判決書進行微調。
- **License**: [llama3.2](https://huggingface.co/meta-llama/Llama-3.2-1B/blob/main/LICENSE.txt)
- **Finetuned from model**: [meta-llama/Llama-3.2-3B-Instruct](meta-llama/Llama-3.2-3B-Instruct)

### Model Sources [optional]

- **Repository:** [lianghsun/Llama-3.2-Taiwan-Legal-3B-Instruct](https://huggingface.co/lianghsun/Llama-3.2-Taiwan-Legal-3B-Instruct)
- **Demo [optional]:** (WIP)

## Uses

<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->

### Direct Use

<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->

[More Information Needed]

### Downstream Use [optional]

<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->

[More Information Needed]

### Out-of-Scope Use

<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->

[More Information Needed]

## Bias, Risks, and Limitations

模型在生成法律條文和判決書內容時，可能會生成虛構或不存在的法條或判決書內容，這是模型的內在限制之一。使用者在參考這些資料時，應謹慎檢查生成的內容，並避免將模型輸出視為法律依據。建議在實際應用中，將模型生成的結果與可靠的法律見解和來源進行比對，確保準確性、合法性和適用性。

### Recommendations

<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->

Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.

## How to Get Started with the Model

Use the code below to get started with the model.

[More Information Needed]

## Training Details

### Training Data

- [lianghsun/tw-legal-nlp](https://huggingface.co/datasets/lianghsun/tw-legal-nlp)
- [lianghsun/tw-structured-law-article](https://huggingface.co/datasets/lianghsun/tw-structured-law-article)
- [lianghsun/tw-legal-synthetic-qa]
- [lianghsun/tw-law-article-qa]
- [lianghsun/tw-judgment-qa]
- [lianghsun/tw-bar-examination-2020-chat]
- [lianghsun/tw-law-exam-choice]
- [lianghsun/tw-emergency-medicine-bench]


### Training Procedure

<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->

#### Preprocessing

無。基本上我們並沒有針對 [meta-llama/Llama-3.2-3B-Instruct](meta-llama/Llama-3.2-3B-Instruct) 做任何的預訓練或更改其模型架構；Tokenizer 也是採用原生所提供的。

#### Training Hyperparameters

- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->

#### Speeds, Sizes, Times [optional]

<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->

[More Information Needed]

## Evaluation

<!-- This section describes the evaluation protocols and provides the results. -->

### Testing Data, Factors & Metrics

#### Testing Data

<!-- This should link to a Dataset Card if possible. -->

[More Information Needed]

#### Factors

<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->

[More Information Needed]

#### Metrics

<!-- These are the evaluation metrics being used, ideally with a description of why. -->

[More Information Needed]

### Results

[More Information Needed]

#### Summary

WIP

## Model Examination [optional]

<!-- Relevant interpretability work for the model goes here -->

[More Information Needed]

## Environmental Impact

- **Hardware Type:** 8 x NVIDIA A100 40GB
- **Hours used:** 6.03 hours 
- **Cloud Provider:** Google Cloud Platform
- **Compute Region:** us-central1-c
- **Carbon Emitted:** `0.86 kgCO$_2$eq`                        

## Technical Specifications [optional]

### Model Architecture and Objective

[More Information Needed]

### Compute Infrastructure

[More Information Needed]

#### Hardware

- 8 x NVIDIA A100 40GB

#### Software

- [hiyouga/LLaMA-Factory](https://github.com/hiyouga/LLaMA-Factory)

## Citation

無。

## Glossary [optional]

<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->

[More Information Needed]

## More Information [optional]

[More Information Needed]

## Model Card Authors [optional]

[Huang Liang Hsun](https://www.linkedin.com/in/lianghsunhuang)

## Model Card Contact

[Huang Liang Hsun](https://www.linkedin.com/in/lianghsunhuang)

### Framework versions

- PEFT 0.12.0