pere commited on
Commit
e5c2754
1 Parent(s): 1e23866

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +7 -4
README.md CHANGED
@@ -26,7 +26,7 @@ license: llama3.1
26
  ### Model Overview
27
  This is the SFT-version of the NB-Llama-models. This means the model has gone through supervised finetuning, and it now understands a basic template. Note that this model has not yet been aligned, so it will behave fairly unpredictable. It is most suited for additional fine tuning.
28
 
29
- **NB-Llama-3.1-70B-sft** is part of the **NB-Llama-3.1** series of models, trained on top of [NB-Llama-3.1-70B](https://huggingface.co/north/Llama-3.1-70B). This multilingual generative model was fine-tuned specifically to support Norwegian Bokmål, Norwegian Nynorsk, and English, with partial support for Swedish and Danish.
30
 
31
  The basic idea with this model series was to explore how current state-of-the-art models could be improved for Norwegian by training only on publicly available data. While these models are trained by the National Library of Norway, they do not include data only available through legal deposit. They do, however, contain public data like governmental reports that are both publicly available and legally deposited.
32
 
@@ -87,7 +87,7 @@ Please note tht this is still a research project, and the purpose of releasing t
87
  import torch
88
  from transformers import pipeline
89
 
90
- model_id = "north/nb-llama-3.1-70B-sft"
91
  pipe = pipeline(
92
  "text-generation",
93
  model=model_id,
@@ -123,7 +123,10 @@ Parts of the following publicly available datasets were used:
123
 
124
  ### Data Selection
125
 
126
- To ensure the highest quality training data, only a small subset of the original raw data was used. An encoder-only classifier built on [nb-bert-base](https://huggingface.co/NbAiLab/nb-bert-base) was trained to evaluate both educational value and linguistic quality of the training samples.
 
 
 
127
 
128
  - **Categorization Methods:**
129
  - Inspired by the [FineWeb](https://example.com/FineWeb) project.
@@ -143,6 +146,6 @@ The model is released under the [Llama 3.1 Community License](https://github.com
143
  ---
144
 
145
  ### Citing & Authors
146
- The model was trained and documentation written by Per Egil Kummervold.
147
 
148
  ---
 
26
  ### Model Overview
27
  This is the SFT-version of the NB-Llama-models. This means the model has gone through supervised finetuning, and it now understands a basic template. Note that this model has not yet been aligned, so it will behave fairly unpredictable. It is most suited for additional fine tuning.
28
 
29
+ **NB-Llama-3.1-70B-sft** is part of the **NB-Llama-3.1** series of models, trained on top of [NB-Llama-3.1-70B](https://huggingface.co/NbAiLab/Llama-3.1-70B). This multilingual generative model was fine-tuned specifically to support Norwegian Bokmål, Norwegian Nynorsk, and English, with partial support for Swedish and Danish.
30
 
31
  The basic idea with this model series was to explore how current state-of-the-art models could be improved for Norwegian by training only on publicly available data. While these models are trained by the National Library of Norway, they do not include data only available through legal deposit. They do, however, contain public data like governmental reports that are both publicly available and legally deposited.
32
 
 
87
  import torch
88
  from transformers import pipeline
89
 
90
+ model_id = "NbAiLab/nb-llama-3.1-70B-sft"
91
  pipe = pipeline(
92
  "text-generation",
93
  model=model_id,
 
123
 
124
  ### Data Selection
125
 
126
+ To ensure the highest quality training data, only a small subset of the original raw data was used. [Corpus Quality Classifiers](https://huggingface.
127
+ co/collections/NbAiLab/corpus-quality-classifier-673f15926c2774fcc88f23aa) built on [nb-bert-base](https://huggingface.co/NbAiLab/nb-bert-base) were
128
+ trained to evaluate both educational value and linguistic quality of the training samples. These models are released along with the NB-Llama-3.x models, and are considered the main output from this initiative.
129
+
130
 
131
  - **Categorization Methods:**
132
  - Inspired by the [FineWeb](https://example.com/FineWeb) project.
 
146
  ---
147
 
148
  ### Citing & Authors
149
+ The model was trained and documentation written by Per Egil Kummervold as part of the NoTraM-project.
150
 
151
  ---