Upload README.md with huggingface_hub
Browse files
README.md
CHANGED
@@ -26,7 +26,7 @@ license: llama3.1
|
|
26 |
### Model Overview
|
27 |
This is the SFT-version of the NB-Llama-models. This means the model has gone through supervised finetuning, and it now understands a basic template. Note that this model has not yet been aligned, so it will behave fairly unpredictable. It is most suited for additional fine tuning.
|
28 |
|
29 |
-
**NB-Llama-3.1-70B-sft** is part of the **NB-Llama-3.1** series of models, trained on top of [NB-Llama-3.1-70B](https://huggingface.co/
|
30 |
|
31 |
The basic idea with this model series was to explore how current state-of-the-art models could be improved for Norwegian by training only on publicly available data. While these models are trained by the National Library of Norway, they do not include data only available through legal deposit. They do, however, contain public data like governmental reports that are both publicly available and legally deposited.
|
32 |
|
@@ -87,7 +87,7 @@ Please note tht this is still a research project, and the purpose of releasing t
|
|
87 |
import torch
|
88 |
from transformers import pipeline
|
89 |
|
90 |
-
model_id = "
|
91 |
pipe = pipeline(
|
92 |
"text-generation",
|
93 |
model=model_id,
|
@@ -123,7 +123,10 @@ Parts of the following publicly available datasets were used:
|
|
123 |
|
124 |
### Data Selection
|
125 |
|
126 |
-
To ensure the highest quality training data, only a small subset of the original raw data was used.
|
|
|
|
|
|
|
127 |
|
128 |
- **Categorization Methods:**
|
129 |
- Inspired by the [FineWeb](https://example.com/FineWeb) project.
|
@@ -143,6 +146,6 @@ The model is released under the [Llama 3.1 Community License](https://github.com
|
|
143 |
---
|
144 |
|
145 |
### Citing & Authors
|
146 |
-
The model was trained and documentation written by Per Egil Kummervold.
|
147 |
|
148 |
---
|
|
|
26 |
### Model Overview
|
27 |
This is the SFT-version of the NB-Llama-models. This means the model has gone through supervised finetuning, and it now understands a basic template. Note that this model has not yet been aligned, so it will behave fairly unpredictable. It is most suited for additional fine tuning.
|
28 |
|
29 |
+
**NB-Llama-3.1-70B-sft** is part of the **NB-Llama-3.1** series of models, trained on top of [NB-Llama-3.1-70B](https://huggingface.co/NbAiLab/Llama-3.1-70B). This multilingual generative model was fine-tuned specifically to support Norwegian Bokmål, Norwegian Nynorsk, and English, with partial support for Swedish and Danish.
|
30 |
|
31 |
The basic idea with this model series was to explore how current state-of-the-art models could be improved for Norwegian by training only on publicly available data. While these models are trained by the National Library of Norway, they do not include data only available through legal deposit. They do, however, contain public data like governmental reports that are both publicly available and legally deposited.
|
32 |
|
|
|
87 |
import torch
|
88 |
from transformers import pipeline
|
89 |
|
90 |
+
model_id = "NbAiLab/nb-llama-3.1-70B-sft"
|
91 |
pipe = pipeline(
|
92 |
"text-generation",
|
93 |
model=model_id,
|
|
|
123 |
|
124 |
### Data Selection
|
125 |
|
126 |
+
To ensure the highest quality training data, only a small subset of the original raw data was used. [Corpus Quality Classifiers](https://huggingface.
|
127 |
+
co/collections/NbAiLab/corpus-quality-classifier-673f15926c2774fcc88f23aa) built on [nb-bert-base](https://huggingface.co/NbAiLab/nb-bert-base) were
|
128 |
+
trained to evaluate both educational value and linguistic quality of the training samples. These models are released along with the NB-Llama-3.x models, and are considered the main output from this initiative.
|
129 |
+
|
130 |
|
131 |
- **Categorization Methods:**
|
132 |
- Inspired by the [FineWeb](https://example.com/FineWeb) project.
|
|
|
146 |
---
|
147 |
|
148 |
### Citing & Authors
|
149 |
+
The model was trained and documentation written by Per Egil Kummervold as part of the NoTraM-project.
|
150 |
|
151 |
---
|