Code

Add "blocked"/"flash" cross entropy#790

Code

Merged

dlwh merged 46 commits intomainstanford-crfm/levanter:mainfrom blocked_cross_entropystanford-crfm/levanter:blocked_cross_entropyCopy head branch name to clipboardNov 6, 2024

+883-47

Commits

Commits on Jul 17, 2024

• Add llama fineweb yaml

  Ivan-Zhou

  committed
  fa9fd25

  Copy full SHA for fa9fd25

• small modification

  Ivan-Zhou

  committed
  ac80e57

  Copy full SHA for ac80e57

• pre commit checks

  Ivan-Zhou

  committed
  8b9bd78

  Copy full SHA for 8b9bd78

• mypy

  Ivan-Zhou

  committed
  525f4a6

  Copy full SHA for 525f4a6

• add fire

  Ivan-Zhou

  committed
  92425ca

  Copy full SHA for 92425ca

• add more html

  Ivan-Zhou

  committed
  1d9f6f8

  Copy full SHA for 1d9f6f8

• add more md urls

  Ivan-Zhou

  committed
  b1c0905

  Copy full SHA for b1c0905

• delete get_files_on_gcs.py

  Ivan-Zhou

  committed
  89b6192

  Copy full SHA for 89b6192

• CC-MAIN-*/*/*_processed_md.jsonl.gz

  Ivan-Zhou

  committed
  0b34522

  Copy full SHA for 0b34522

Commits on Jul 18, 2024

• Adding configs related to DCLM

  abhinavg4

  committed
  4aa2f2c

  Copy full SHA for 4aa2f2c

Commits on Jul 19, 2024

• Adding configs related to DCLM

  abhinavg4

  committed
  dde9ed0

  Copy full SHA for dde9ed0

• Adding Z loss

  abhinavg4

  committed
  b991e29

  Copy full SHA for b991e29

• pre commit changes

  abhinavg4

  committed
  bb674bb

  Copy full SHA for bb674bb

• Adding z_loss as part of train_lm.py

  abhinavg4

  committed
  6c99dfb

  Copy full SHA for 6c99dfb

• Reverting changes to llama.py for z_loss

  abhinavg4

  committed
  24469e7

  Copy full SHA for 24469e7

Commits on Aug 20, 2024

• Merge remote-tracking branch 'origin/dclm' into dclm

  dlwh

  committed
  e12c1b6

  Copy full SHA for e12c1b6

• match specs in dclm

  dlwh

  committed
  c9ebc88

  Copy full SHA for c9ebc88

Commits on Aug 21, 2024

• publish dev build

  dlwh

  committed
  7727696

  Copy full SHA for 7727696

• wip

  dlwh

  committed
  55e4d98

  Copy full SHA for 55e4d98

Commits on Aug 22, 2024

• fix imports and such

  dlwh

  committed
  de51236

  Copy full SHA for de51236

• get default zone from gcloud config

  dlwh

  committed
  7863989

  Copy full SHA for 7863989

• factor out docker command, build

  dlwh

  committed
  a550bb5

  Copy full SHA for a550bb5

• Merge remote-tracking branch 'origin/main' into dclm

  dlwh

  committed
  6341252

  Copy full SHA for 6341252

Commits on Aug 28, 2024

• Merge remote-tracking branch 'origin/main' into dclm

  dlwh

  committed
  e9ca517

  Copy full SHA for e9ca517

Commits on Aug 29, 2024

• wip

  dlwh

  committed
  d674dd9

  Copy full SHA for d674dd9

• wip

  dlwh

  committed
  06dc304

  Copy full SHA for 06dc304

Commits on Sep 5, 2024

• bump equinox

  dlwh

  committed
  f13cfde

  Copy full SHA for f13cfde

Commits on Sep 6, 2024

• wip

  dlwh

  committed
  8d3dfe0

  Copy full SHA for 8d3dfe0

• 768

  dlwh

  committed
  8ecb7ea

  Copy full SHA for 8ecb7ea

Commits on Oct 14, 2024

• Merge remote-tracking branch 'origin/main' into dclm

  dlwh

  committed
  0ea3eb4

  Copy full SHA for 0ea3eb4

Commits on Oct 21, 2024

• wip

  dlwh

  committed
  2f53923

  Copy full SHA for 2f53923

Commits on Oct 30, 2024

• wip

  dlwh

  committed
  9050258

  Copy full SHA for 9050258

• Merge remote-tracking branch 'origin/main' into blocked_cross_entropy

  dlwh

  committed
  b15e5d3

  Copy full SHA for b15e5d3

• wip

  dlwh

  committed
  d6a3ded

  Copy full SHA for d6a3ded

Commits on Nov 4, 2024

• it works?!?

  dlwh

  committed
  2e25357

  Copy full SHA for 2e25357

Commits on Nov 5, 2024

• tuning. just about there

  dlwh

  committed
  2390058

  Copy full SHA for 2390058

• wip

  dlwh

  committed
  795fd08

  Copy full SHA for 795fd08

Commits on Nov 6, 2024

• Merge remote-tracking branch 'origin/main' into blocked_cross_entropy

  dlwh

  committed
  05afef0

  Copy full SHA for 05afef0

• pre-commit

  dlwh

  committed
  0de1482

  Copy full SHA for 0de1482

• remove stray files

  dlwh

  committed
  fc01b9e

  Copy full SHA for fc01b9e

• implement lm_head

  dlwh

  committed
  d907aa4

  Copy full SHA for d907aa4

• implement lm_head

  dlwh

  committed
  eee3ecd

  Copy full SHA for eee3ecd

• misc test fixes

  dlwh

  committed
  879d5c0

  Copy full SHA for 879d5c0

• Merge remote-tracking branch 'origin/blocked_cross_entropy' into blocked_cross_entropy

  dlwh

  committed
  6fe0fb8

  Copy full SHA for 6fe0fb8

• increase tolerances

  dlwh

  committed
  5cf5f17

  Copy full SHA for 5cf5f17

• increase tolerances

  dlwh

  committed
  db08341

  Copy full SHA for db08341