Model loading#25

Merged

tigranfah merged 14 commits intomainfrom model_loading

Sep 24, 2024

+251-64

Commits on Sep 11, 2024

save aim experiments in the dump directory
tigranfah
committed

Commits on Sep 14, 2024

prepare the hparams of galactica-1.3b for reproduction training, add torch compile, a single dataloader num worker
tigranfah
committed

Commits on Sep 16, 2024

tune 1.3b model config for optimal training throughput, and importantly add gradient cliping after each forward pass to avoid overflows
tigranfah
committed

Commits on Sep 20, 2024

Merge branch 'main' into model_loading
tigranfah
committed

Commits on Sep 21, 2024

fix the logging of the arguments to the aim experiment
tigranfah
committed

Commits on Sep 22, 2024

Merge branch 'aim' into model_loading
tigranfah
committed

Commits on Sep 23, 2024

pass and set the log_level to dataloader logging
tigranfah
committed