Code

Tweaks to Muon branch#831

Code

Open

dlwh wants to merge 24 commits intomainstanford-crfm/levanter:mainfrom muonstanford-crfm/levanter:muonCopy head branch name to clipboard

+452-3

Commits

Commits on Jan 1, 2024

• logs

  

  Kaiyue Wen

  committed
  86d6556

  Copy full SHA for 86d6556

• try dataset path

  WhenWen

  committed
  8b0900f

  Copy full SHA for 8b0900f

Commits on Oct 12, 2024

• use the jax serialization manager for deser in an attempt to fix crash. (it doesn't)

  dlwh

  committed
  5bf8cd0

  Copy full SHA for 5bf8cd0

Commits on Oct 13, 2024

• cleaner data loader but ti doesn't help :(

  dlwh

  committed
  20a4568

  Copy full SHA for 20a4568

Commits on Oct 14, 2024

• ok this maybe fixed it?

  dlwh

  committed
  2747705

  Copy full SHA for 2747705

• cleanup

  dlwh

  committed
  538f0ed

  Copy full SHA for 538f0ed

• fix tests

  dlwh

  committed
  2c5ee4b

  Copy full SHA for 2c5ee4b

• fix what is probably the underlying problem

  dlwh

  committed
  ab543d6

  Copy full SHA for ab543d6

• wip

  dlwh

  committed
  6395305

  Copy full SHA for 6395305

Commits on Dec 3, 2024

• Merge branch 'main' of github.com:WhenWen/levanter-2024 into main

  

  Kaiyue Wen

  committed
  b8f28f4

  Copy full SHA for b8f28f4

• Implement MARS (tested) and Muon (have bug in saving), example config in config/llama2_100M_{mars/muon}.yaml

  

  Kaiyue Wen

  committed
  f98e376

  Copy full SHA for f98e376

• Implement MARS (tested) and Muon (have bug in saving), example config in config/llama2_100M_{mars/muon}.yaml

  

  Kaiyue Wen

  committed
  e961c95

  Copy full SHA for e961c95

• wip

  dlwh

  committed
  2b80af7

  Copy full SHA for 2b80af7

• enough device puts and we're good

  dlwh

  committed
  87d7665

  Copy full SHA for 87d7665

• ok we're good

  dlwh

  committed
  074d0ec

  Copy full SHA for 074d0ec

• Merge remote-tracking branch 'origin/main' into WhenWen/main

  dlwh

  committed
  5668289

  Copy full SHA for 5668289

• Merge branch 'use_manager_deser' into WhenWen/main

  dlwh

  committed
  d2d310e

  Copy full SHA for d2d310e

• fix tree leaf stuff

  dlwh

  committed
  722edaf

  Copy full SHA for 722edaf

Commits on Dec 4, 2024

• add map_flattened_linear_layers use in muon

  dlwh

  committed
  5692611

  Copy full SHA for 5692611

• Merge remote-tracking branch 'origin/main' into muon

  dlwh

  committed
  2f119bd

  Copy full SHA for 2f119bd

Commits on Dec 12, 2024

• The training succeed and the model seems to be properly saved but have error in loading model.

  Show description for 271479f
  WhenWen

  committed
  271479f

  Copy full SHA for 271479f

• Merge remote-tracking branch 'origin/main' into muon

  dlwh

  committed
  f36ff81

  Copy full SHA for f36ff81

• fix deser for muon

  dlwh

  committed
  144e20f

  Copy full SHA for 144e20f

• Merge remote-tracking branch 'origin/muon' into muon

  dlwh

  committed
  083835d

  Copy full SHA for 083835d