Code

Partially-sharded-data-parallel#588

Code

Merged

dlwh merged 48 commits intomainstanford-crfm/levanter:mainfrom psdpstanford-crfm/levanter:psdpCopy head branch name to clipboardMay 28, 2024

+163-66

Commits

Commits on May 12, 2024

• refactor parallelism

  blahBlahhhJ

  committed
  79815c0

  Copy full SHA for 79815c0

• small fix

  blahBlahhhJ

  committed
  69ce2e7

  Copy full SHA for 69ce2e7

• add config

  blahBlahhhJ

  committed
  e1d1b51

  Copy full SHA for e1d1b51

• small fix

  blahBlahhhJ

  committed
  0112e87

  Copy full SHA for 0112e87

• small fix

  blahBlahhhJ

  committed
  d119af4

  Copy full SHA for d119af4

• small fix

  blahBlahhhJ

  committed
  c0e25fd

  Copy full SHA for c0e25fd

• allow splitting logical axes

  blahBlahhhJ

  committed
  d71e9ca

  Copy full SHA for d71e9ca

• flatten splash attention axes

  blahBlahhhJ

  committed
  54f407e

  Copy full SHA for 54f407e

• add print deebug

  blahBlahhhJ

  committed
  e0df652

  Copy full SHA for e0df652

• remove debug

  blahBlahhhJ

  committed
  ea86cf7

  Copy full SHA for ea86cf7

• simple mesh

  blahBlahhhJ

  committed
  61b70fd

  Copy full SHA for 61b70fd

• revert auto mesh

  blahBlahhhJ

  committed
  2ecf8c6

  Copy full SHA for 2ecf8c6

Commits on May 15, 2024

• patch for non-TP

  blahBlahhhJ

  committed
  71f7d5b

  Copy full SHA for 71f7d5b

• remove print

  blahBlahhhJ

  committed
  8ecd091

  Copy full SHA for 8ecd091

• small fix

  blahBlahhhJ

  committed
  67cf17f

  Copy full SHA for 67cf17f

• patch process groups size as well

  blahBlahhhJ

  committed
  e2f7794

  Copy full SHA for e2f7794

• small fix

  blahBlahhhJ

  committed
  ec7e41c

  Copy full SHA for ec7e41c

• tries simple fix

  blahBlahhhJ

  committed
  07a8b17

  Copy full SHA for 07a8b17

• revert

  blahBlahhhJ

  committed
  4d61503

  Copy full SHA for 4d61503

Commits on May 17, 2024

• use normal mesh for sharding data loader

  blahBlahhhJ

  committed
  89d18c8

  Copy full SHA for 89d18c8

• bump haliax version

  blahBlahhhJ

  committed
  803df3d

  Copy full SHA for 803df3d

Commits on May 20, 2024

• fix shard loader logic

  blahBlahhhJ

  committed
  02a1ed5

  Copy full SHA for 02a1ed5

• small fix

  blahBlahhhJ

  committed
  479d318

  Copy full SHA for 479d318

• small fix

  blahBlahhhJ

  committed
  019d516

  Copy full SHA for 019d516

• small fix

  blahBlahhhJ

  committed
  c8d48f5

  Copy full SHA for c8d48f5

Commits on May 23, 2024

• small fix

  blahBlahhhJ

  committed
  642a377

  Copy full SHA for 642a377

• add print

  blahBlahhhJ

  committed
  8f9c884

  Copy full SHA for 8f9c884

• add print

  blahBlahhhJ

  committed
  0f7cced

  Copy full SHA for 0f7cced

• add print

  blahBlahhhJ

  committed
  7e4eee1

  Copy full SHA for 7e4eee1

• add print

  blahBlahhhJ

  committed
  cebd0b3

  Copy full SHA for cebd0b3

• add print

  blahBlahhhJ

  committed
  b9e3eda

  Copy full SHA for b9e3eda

• small fix

  blahBlahhhJ

  committed
  27eb7fa

  Copy full SHA for 27eb7fa

• remove print

  blahBlahhhJ

  committed
  2a459e7

  Copy full SHA for 2a459e7

Commits on May 24, 2024

• add process mapping

  blahBlahhhJ

  committed
  5db35c3

  Copy full SHA for 5db35c3

Commits on May 25, 2024

• small fix

  blahBlahhhJ

  committed
  6536c54

  Copy full SHA for 6536c54

• small fix

  blahBlahhhJ

  committed
  abcd5d2

  Copy full SHA for abcd5d2

• small fix

  blahBlahhhJ

  committed
  3313002

  Copy full SHA for 3313002

• small fix

  blahBlahhhJ

  committed
  c34a573

  Copy full SHA for c34a573

• small fix

  blahBlahhhJ

  committed
  2331ae8

  Copy full SHA for 2331ae8

• small fix

  blahBlahhhJ

  committed
  8b7b804

  Copy full SHA for 8b7b804

Commits on May 26, 2024

• rename model axis size

  blahBlahhhJ

  committed
  130c212

  Copy full SHA for 130c212

• clean up

  blahBlahhhJ

  committed
  68ed94e

  Copy full SHA for 68ed94e

• clean up

  blahBlahhhJ

  committed
  3bdf5ba

  Copy full SHA for 3bdf5ba

• modify tests

  blahBlahhhJ

  committed
  f7dfc64

  Copy full SHA for f7dfc64

• clean up

  blahBlahhhJ

  committed
  3787610

  Copy full SHA for 3787610

Commits on May 27, 2024

• clean up

  blahBlahhhJ

  committed
  24fbf37

  Copy full SHA for 24fbf37

• Merge branch 'main' into psdp

  blahBlahhhJ

  committed
  4a63e09

  Copy full SHA for 4a63e09

• clean up

  blahBlahhhJ

  committed
  843f5a9

  Copy full SHA for 843f5a9