Commit History

Merge pull request #122 from borisdayma/feat-acccum
c91ceb7
unverified

boris commited on

feat(data): support accumulation in non-streaming
88c8e06

boris commited on

refactor(train): cleanup
274ba73

boris commited on

feat: custom gradient accumulation
2d07559

boris commited on

fix: style
df01fa8

boris commited on

feat(train): use MultiSteps for gradient accumulation
4fa53a5

boris commited on

Merge pull request #118 from borisdayma/feat-optim
193c88c
unverified

boris commited on

fix: style
25862e8

boris commited on

feat: add more config of distributed_shampoo
89cf9ea

boris commited on

fix(data): no shuffling of validation data
ddcbc6a

boris commited on

feat(train): refactor learning rate params
e2781bc

boris commited on

fix(train): handle seed_dataset
8b72ed8

boris commited on

feat: refactor TrainingArguments
adbdff9

boris commited on

fix: push_to_hub deprecated
23389f6

boris commited on

feat: support pypi
f5dba1e

boris commited on

doc: update contributions
e3b1b56

boris commited on

Merge pull request #117 from borisdayma/fix-inference
ef985be
unverified

boris commited on

fix(inference): use float32 + flatten logits
71c4de3

boris commited on

Merge pull request #115 from borisdayma/feat-shampoo
3a3d375
unverified

boris commited on

feat: update inference pipeline
af807f7

boris commited on

style: isort
531cd78

boris commited on

style: apply to distributed_shampoo
e669c1b

boris commited on

feat: add best_effort_memory_usage_reduction
4d518c7

boris commited on

doc: add reference to Distributed Shampoo
db882b8

boris commited on

feat: update distributed_shampoo
b90198c

boris commited on

fix: weight decay Adam + speed logging
7143593

boris commited on

feat: add micro config
e501f71

boris commited on

fix: shampoo -> distributed shampoo
edae62d

boris commited on

feat: update params
604a65d

boris commited on

feat: add shampoo optimizer
0b87452

boris commited on

feat: update sweep
e1555d4

boris commited on

feat: create config files
dc5c024

boris commited on

feat: allow abstract_init
772415c

boris commited on

fix: typo
5c84978

boris commited on

feat: log more metrics
1b757dc

boris commited on

feat: shard by host is optional
901ff72

boris commited on

feat: load data first
fdf7698

boris commited on

feat: display local TPU's
15993e3

boris commited on

fix: check local TPU instances only
87fed1b

boris commited on

feat: add config
3f13951

boris commited on

feat(setup): require braceexpand for dev
98f1db7

boris commited on

fix(data): type
c6ebb14

boris commited on

feat(train): handle multi-hosts
5b533b5

boris commited on

feat: split shards by host
ed93c8a

boris commited on

style
a6252c9

boris commited on

feat(data): accept braceexpand notation
5ee6e60

boris commited on

feat: minor improvements
53dade7

boris commited on

fix(config): set min/max for generation
eb24dbc

boris commited on

fix: update model name
61c93f2

boris commited on

fix(train): update model name
b257ca8

boris commited on