Code

Add sampling#7

Code

Open

kingoflolz wants to merge 34 commits intomainEleutherAI/DALLE-mtf:mainfrom sampleEleutherAI/DALLE-mtf:sampleCopy head branch name to clipboard

+718-196

Commits

Commits on Jan 13, 2021

• add basic sampling code

  

  sid

  committed
  99d11e1

  Copy full SHA for 99d11e1

• Merge remote-tracking branch 'origin/main' into sample

  

  sid

  committed
  8bb3322

  Copy full SHA for 8bb3322

• add prediction input / output fns

  

  sid

  committed
  b9fd039

  Copy full SHA for b9fd039

Commits on Jan 14, 2021

• get sample_autoregressive working

  ConnorJL

  committed
  cf76c6c

  Copy full SHA for cf76c6c

• truncate text tokens properly

  ConnorJL

  committed
  c7ff6c4

  Copy full SHA for c7ff6c4

• log model params to tensorboard

  ConnorJL

  committed
  4d51fd9

  Copy full SHA for 4d51fd9

• add vae decoding and write to jpeg

  kingoflolz

  committed
  346871f

  Copy full SHA for 346871f

• unshift image outputs at decode time

  kingoflolz

  committed
  d13c330

  Copy full SHA for d13c330

• dirty hack to use vae decoder params when training dalle

  kingoflolz

  committed
  f8a7449

  Copy full SHA for f8a7449

Commits on Jan 17, 2021

• Move initialize_vae_weights to after lowering

  leogao2

  authored
  ff56d12

  Copy full SHA for ff56d12

• fix vae checkpoint load in training

  ConnorJL

  committed
  4c4e0e0

  Copy full SHA for 4c4e0e0

Commits on Jan 18, 2021

• fix parameter count logging

  ConnorJL

  committed
  2c14bde

  Copy full SHA for 2c14bde

• fix image vocab size

  ConnorJL

  committed
  130c26e

  Copy full SHA for 130c26e

Commits on Apr 2, 2021

• add missing dep

  lucidrains

  committed
  42e7677

  Copy full SHA for 42e7677

Commits on Apr 3, 2021

• ready tests for rewiring

  lucidrains

  committed
  fcab065

  Copy full SHA for fcab065

• no 3.9 for tests

  lucidrains

  committed
  3858108

  Copy full SHA for 3858108

• make DALLE-mtf work with text and image logits created with separate projections

  lucidrains

  committed
  002343a

  Copy full SHA for 002343a

• add axial positional embedding

  lucidrains

  committed
  037aefb

  Copy full SHA for 037aefb

• cleanup

  lucidrains

  committed
  e203de4

  Copy full SHA for e203de4

• shift by text vocab size (not text seq len)

  lucidrains

  committed
  5ecb496

  Copy full SHA for 5ecb496

• make sure sampling can be forced to never start below a certain minimum starting position, useful when the text tokens contain padding tokens of 0

  lucidrains

  committed
  ec5c298

  Copy full SHA for ec5c298

• add unique pad token ids, which obviates the need to remove attention and cross entropy loss from text padding tokens

  lucidrains

  committed
  3492d80

  Copy full SHA for 3492d80

Commits on Apr 4, 2021

• fix text_vocab_dim error

  sdtblck

  authored
  3b76849

  Copy full SHA for 3b76849

• Update models.py

  sdtblck

  authored
  ef96537

  Copy full SHA for ef96537

• fix bug

  lucidrains

  committed
  a1634d4

  Copy full SHA for a1634d4

• fix bug with shift

  lucidrains

  authored
  a3d2115

  Copy full SHA for a3d2115

• Add adam weight decay optimizer

  sdtblck

  authored
  5f279ad

  Copy full SHA for 5f279ad

• fix args.steps_per_checkpoint

  sdtblck

  authored
  8db0d55

  Copy full SHA for 8db0d55

• add variable scope

  lucidrains

  committed
  d3112ca

  Copy full SHA for d3112ca

• do an early copy of inputs for labels

  lucidrains

  committed
  33de203

  Copy full SHA for 33de203

• tweak

  lucidrains

  committed
  4dbf727

  Copy full SHA for 4dbf727

Commits on Apr 5, 2021

• fix initial positions at text_seq_len

  lucidrains

  committed
  10dc024

  Copy full SHA for 10dc024

• more cleanup

  lucidrains

  committed
  9dacd35

  Copy full SHA for 9dacd35

Commits on Apr 6, 2021

• make sure axial positional embedding is correctly shifted by one due to <bos>

  lucidrains

  committed
  c073fbc

  Copy full SHA for c073fbc