Commit
•
83c1908
1
Parent(s):
93de90d
Update README.md
Browse files
README.md
CHANGED
@@ -9,6 +9,8 @@ Model Hyperparams:
|
|
9 |
- LR: 3e-4
|
10 |
- Batch Size: 64
|
11 |
- Grad Accumulation: 8 steps
|
|
|
12 |
- Total steps: 5,000
|
|
|
13 |
|
14 |
![output.png](https://s3.amazonaws.com/moonup/production/uploads/1672153152960-6262d89f63f73be3d2f6b7c1.png)
|
|
|
9 |
- LR: 3e-4
|
10 |
- Batch Size: 64
|
11 |
- Grad Accumulation: 8 steps
|
12 |
+
- Effectibe Batch Size: 512
|
13 |
- Total steps: 5,000
|
14 |
+
- Linear Beta Schedule: 1000 Steps
|
15 |
|
16 |
![output.png](https://s3.amazonaws.com/moonup/production/uploads/1672153152960-6262d89f63f73be3d2f6b7c1.png)
|