jon-tow commited on
Commit
95292b8
1 Parent(s): 2724cd4

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +5 -1
README.md CHANGED
@@ -10,4 +10,8 @@ Pythia-2.8B Deduped 4K is a [Pythia-2.8B Deduped](https://huggingface.co/Eleuthe
10
  Training resumed from their 143,000 step checkpoint and continued on The Pile v1 Deduped (threshold=0.87).
11
  This particular model is from a checkpoint captured at step 175,500 for an extra 134,217,728,000 tokens of training.
12
 
13
- Note: Sequence length warmup was not used to move up from 2048 but, in hindsight, should have been applied.
 
 
 
 
 
10
  Training resumed from their 143,000 step checkpoint and continued on The Pile v1 Deduped (threshold=0.87).
11
  This particular model is from a checkpoint captured at step 175,500 for an extra 134,217,728,000 tokens of training.
12
 
13
+ Note: Sequence length warmup was not used to move up from 2048 but, in hindsight, should have been applied.
14
+
15
+ ## Acknoweldgements
16
+
17
+ This work would not have been possible without the support of [Stability AI](https://stability.ai/).