Code

WIP Kron#843

Code

Draft

dlwh wants to merge 66 commits intostanford-crfm:mainstanford-crfm/levanter:mainfrom evanatyourservice:kronevanatyourservice/levanter:kronCopy head branch name to clipboard

+1,296-6

Commits

Commits on Jan 1, 2024

• logs

  

  Kaiyue Wen

  committed
  86d6556

  Copy full SHA for 86d6556

• try dataset path

  WhenWen

  committed
  8b0900f

  Copy full SHA for 8b0900f

Commits on Oct 12, 2024

• use the jax serialization manager for deser in an attempt to fix crash. (it doesn't)

  dlwh

  committed
  5bf8cd0

  Copy full SHA for 5bf8cd0

Commits on Oct 13, 2024

• cleaner data loader but ti doesn't help :(

  dlwh

  committed
  20a4568

  Copy full SHA for 20a4568

Commits on Oct 14, 2024

• ok this maybe fixed it?

  dlwh

  committed
  2747705

  Copy full SHA for 2747705

• cleanup

  dlwh

  committed
  538f0ed

  Copy full SHA for 538f0ed

• fix tests

  dlwh

  committed
  2c5ee4b

  Copy full SHA for 2c5ee4b

• fix what is probably the underlying problem

  dlwh

  committed
  ab543d6

  Copy full SHA for ab543d6

• wip

  dlwh

  committed
  6395305

  Copy full SHA for 6395305

Commits on Dec 3, 2024

• Merge branch 'main' of github.com:WhenWen/levanter-2024 into main

  

  Kaiyue Wen

  committed
  b8f28f4

  Copy full SHA for b8f28f4

• Implement MARS (tested) and Muon (have bug in saving), example config in config/llama2_100M_{mars/muon}.yaml

  

  Kaiyue Wen

  committed
  f98e376

  Copy full SHA for f98e376

• Implement MARS (tested) and Muon (have bug in saving), example config in config/llama2_100M_{mars/muon}.yaml

  

  Kaiyue Wen

  committed
  e961c95

  Copy full SHA for e961c95

• wip

  dlwh

  committed
  2b80af7

  Copy full SHA for 2b80af7

• enough device puts and we're good

  dlwh

  committed
  87d7665

  Copy full SHA for 87d7665

• ok we're good

  dlwh

  committed
  074d0ec

  Copy full SHA for 074d0ec

• Merge remote-tracking branch 'origin/main' into WhenWen/main

  dlwh

  committed
  5668289

  Copy full SHA for 5668289

• Merge branch 'use_manager_deser' into WhenWen/main

  dlwh

  committed
  d2d310e

  Copy full SHA for d2d310e

• fix tree leaf stuff

  dlwh

  committed
  722edaf

  Copy full SHA for 722edaf

Commits on Dec 4, 2024

• add map_flattened_linear_layers use in muon

  dlwh

  committed
  5692611

  Copy full SHA for 5692611

• Merge remote-tracking branch 'origin/main' into muon

  dlwh

  committed
  2f119bd

  Copy full SHA for 2f119bd

• adding kron file to optim

  evanatyourservice

  committed
  0f41ebb

  Copy full SHA for 0f41ebb

Commits on Dec 5, 2024

• testing 123

  evanatyourservice

  committed
  fe3ecc9

  Copy full SHA for fe3ecc9

• Update kron.py

  evanatyourservice

  committed
  37452c7

  Copy full SHA for 37452c7

• Update llama2_100M_kron_test.yaml

  evanatyourservice

  committed
  701956d

  Copy full SHA for 701956d

• Update llama2_100M_kron_test.yaml

  evanatyourservice

  committed
  aac1cee

  Copy full SHA for aac1cee

• Update llama2_100M_kron_test.yaml

  evanatyourservice

  committed
  e44e7fa

  Copy full SHA for e44e7fa

Commits on Dec 6, 2024

• Update kron.py

  evanatyourservice

  committed
  476ba36

  Copy full SHA for 476ba36

Commits on Dec 7, 2024

• expose precond lr and init

  evanatyourservice

  committed
  966b80e

  Copy full SHA for 966b80e

• Merge branch 'stanford-crfm:main' into kron

  evanatyourservice

  authored
  6408d23

  Copy full SHA for 6408d23

Commits on Dec 10, 2024

• Update kron.py

  evanatyourservice

  committed
  91e29e7

  Copy full SHA for 91e29e7

Commits on Dec 14, 2024

• Merge remote-tracking branch 'upstream/main' into kron

  evanatyourservice

  committed
  6b9ce26

  Copy full SHA for 6b9ce26

• Update llama2_100M_kron_test.yaml

  evanatyourservice

  committed
  53efaa3

  Copy full SHA for 53efaa3

• Update README.md

  evanatyourservice

  committed
  311da92

  Copy full SHA for 311da92

• Update kron.py

  evanatyourservice

  committed
  33c17f4

  Copy full SHA for 33c17f4

• Update kron.py

  evanatyourservice

  committed
  8da6e34

  Copy full SHA for 8da6e34

• Update kron.py

  evanatyourservice

  committed
  5607cec

  Copy full SHA for 5607cec

Commits on Dec 15, 2024

• trust remote code

  evanatyourservice

  committed
  2fb6c34

  Copy full SHA for 2fb6c34

• settings defaults

  evanatyourservice

  committed
  336e1e1

  Copy full SHA for 336e1e1

• no key, deterministic, pass all into cond, more sharding

  evanatyourservice

  committed
  a5ff351

  Copy full SHA for a5ff351

• set key in state

  evanatyourservice

  committed
  3a06e1c

  Copy full SHA for 3a06e1c

• whoops

  evanatyourservice

  committed
  f7f2382

  Copy full SHA for f7f2382

• small fix

  evanatyourservice

  committed
  07781e6

  Copy full SHA for 07781e6

• Update kron.py

  evanatyourservice

  committed
  9ef0869

  Copy full SHA for 9ef0869

• Update kron.py

  evanatyourservice

  committed
  ed50cce

  Copy full SHA for ed50cce

• settings

  evanatyourservice

  committed
  1dc0f43

  Copy full SHA for 1dc0f43

Commits on Dec 16, 2024

• small fix in init sharding

  evanatyourservice

  committed
  f1c1b38

  Copy full SHA for f1c1b38

• trying repl only

  evanatyourservice

  committed
  7a6f501

  Copy full SHA for 7a6f501

• Revert "trying repl only"

  Show description for 3473eed
  evanatyourservice

  committed
  3473eed

  Copy full SHA for 3473eed

• trying while loop

  evanatyourservice

  committed
  7684702

  Copy full SHA for 7684702

Commits on Dec 19, 2024

• trying more simple psgd kron version

  evanatyourservice

  committed
  d284518

  Copy full SHA for d284518

• Update kron.py

  evanatyourservice

  committed
  0c920b0

  Copy full SHA for 0c920b0

• Merge branch 'stanford-crfm:main' into kron

  evanatyourservice

  authored
  2feff32

  Copy full SHA for 2feff32

• trying simple version

  evanatyourservice

  committed
  6a2e19f

  Copy full SHA for 6a2e19f

• take out unavailable args

  evanatyourservice

  committed
  5108be0

  Copy full SHA for 5108be0

• no extra args

  evanatyourservice

  committed
  975a2d7

  Copy full SHA for 975a2d7

• trying this

  evanatyourservice

  committed
  3fa70ab

  Copy full SHA for 3fa70ab

• Update kron.py

  evanatyourservice

  committed
  b62963e

  Copy full SHA for b62963e

• Revert "Update kron.py"

  Show description for 5bafdcf
  evanatyourservice

  committed
  5bafdcf

  Copy full SHA for 5bafdcf

• small fix

  evanatyourservice

  committed
  c47c4c5

  Copy full SHA for c47c4c5

• settings

  evanatyourservice

  committed
  ee747c0

  Copy full SHA for ee747c0

Commits on Dec 22, 2024

• small changes/moving to remote

  evanatyourservice

  committed
  59f2c10

  Copy full SHA for 59f2c10

• Merge remote-tracking branch 'upstream/main' into kron

  evanatyourservice

  committed
  aa43e4f

  Copy full SHA for aa43e4f

• simplified kron is working, need to test on larger pod

  evanatyourservice

  committed
  25a2c20

  Copy full SHA for 25a2c20

Commits on Dec 23, 2024

• Update kron.py

  evanatyourservice

  committed
  88d49ed

  Copy full SHA for 88d49ed

Commits on Dec 31, 2024

• get rid of norming and clipping in lieu of rms clip, retouches

  evanatyourservice

  committed
  4d630d8

  Copy full SHA for 4d630d8

• Merge remote-tracking branch 'upstream/main' into kron

  evanatyourservice

  committed
  964cf19

  Copy full SHA for 964cf19