Diffusampler

[htoyryla]

Diffusampler.py has been added.

python diffusampler.py --dir samples --name un1bhs --ema --modelSize 512 --mults 1 1 2 2 4 4 8 --epoch 180 --nsamples 4 --batchSize 2 --load un1bhs4/model --model unet1

--dir directory where to store the samples
--name base name for the samples (script will add epoch and sample numbers to the saved image names)
--epoch first epoch to sample
--epoch2 last epoch to sample
--estep step between epochs to sample, use 1 for every epoch
--nsamples total number of samples per epoch
--batchSize batch size for (parallel) sampling
--load use the path to the model but leave out the epoch number and extension (e.g. un1b/model instead of un1b/model-45.pt)

As in training
--ema
--modelSize
--mults
--model

[metaphorz]

Looks great but I cannot get it to work (yet). Here is the command:

!python diffusampler.py --dir samples --name un1bhs --ema --modelSize 512 --mults 1 1 2 2 4 4 8 --nsamples 4 --batchSize 2 --load {pretrained_models}/Diffusion-Model-55.pt --model unet0k5 # --epoch 180

I commented out -- epoch 180 since I am not sure what this is. My model is Diffusion-Model-55.pt.

Here is the error:

FileNotFoundError: [Errno 2] No such file or directory: '/content/drive/MyDrive/pretrained/Diffusion-Model-55.pt-0.pt'

Where is this extra '-0.pt' coming from?

[htoyryla]

The script supports sampling with multiple epochs/checkpoints with a single command. Therefore one must give, not the full filename of the model, but leave out the end, in your example "-55.pt". Like I wrote above:

--load use the path to the model but leave out the epoch number and extension (e.g. un1b/model instead of un1b/model-45.pt)

So in your case

--load {pretrained_models}/Diffusion-Model --epoch 55

and with

--load {pretrained_models}/Diffusion-Model --epoch 55 --epoch2 65 --estep

you can sample from Diffusion-Model-55.pt to Diffusion-Model-65.pt

with a single command.

This is something I have been doing a lot with my GANs. Even interpolating between saved models, like from Diffusion-Model-55.pt to Diffusion-Model-56.pt in 10 steps, but that is not supposed here yet.

[htoyryla]

I guess it added to the confusion that I used the word epoch here, after explaining that we don't really use epochs. The term was just carried over from my earlier scripts. Anyway, we are saving numbered models, and here separate the base name and the number to be able to sample from multiple models in sequence.

[metaphorz]

I am halfway there. Thanks for the explicit suggestion. Now my command reads:

!python diffusampler.py --dir samples --name un1bhs --ema --modelSize 512 --mults 1 1 2 2 4 4 8 --nsamples 4 --batchSize 2 --load {pretrained_models}/Diffusion-Model --model unet0k5 --epoch 55

but I get this error now:

FileNotFoundError: [Errno 2] No such file or directory: 'samples/name-55-0.png'

There is no 'samples' subdirectory in the repo. I can try adding this directory and rerun.

[htoyryla]

No need to remove it all. I guess what annoyed me immediately was the beginning. "Steps means the diffusion steps inside the model. 50 is very low, will probably not work." Kind of starting with the wrong foot :)

Why not simply "Steps means the diffusion steps and should be left at 1000."

Also this line

"Trainsteps derives from the original code, perhaps we should count the number of such rounds (1000 batches in the example)."

is development speculation and out of place in a colab, because with the present code one has to give trainsteps and not cycles.

I have now gone through my scripts and edited the argparse help texts.

BTW, the name of the repo is now minidiffusion and the url, correspondingly, https://github.com/htoyryla/minidiffusion
The old url still works but probably better to use the new one.

In addition, it would probably be correct to give credit also to https://github.com/lucidrains/denoising-diffusion-pytorch which is the original codebase, based on which I have built my application.

[metaphorz]

I cut out many of the citation comments but kept those that I hope are informative. Also got rid of the two "-h" cells since the user should be able to figure this out.

I updated the credit for lucidrains repo

I updated the name/code base to minidiffusion

[htoyryla]

Thanks! I perhaps reacted a bit strongly... but I really felt that those comments were not informative to a newcomer. Thinking further, if I had understood from the beginning that you intend to use my code as it is and use colab just as a front end, I would probably have answered differently. But the colabs I had seen had been major rewrites of earlier code and I believed that you were about to do something like that and starting thinking what could be done differently (like the train steps thing).

Not sure I get the part about "the two -h cells" though.

[htoyryla]

Oh, I see you have removed the argument lists (which I have now corrected). Anyway, it was probably a bit too cryptic. Have a look at the one I wrote for diffusampler in the first post of this discussion. That again, may be problematic because of my concept of sampling multiple models sequentially. But otherwise, there I have tried to explain the different options better.

The discussion miniDiffusion techniques again describes diffudiver and its various uses, yet without going into detailed level of argument settings. I could expand on that. It would be a natural continuation of what I am doing with this project.

So there is already material here and I am gradually doing more.

[metaphorz]

If you could expand as you suggest that would be helpful

[metaphorz]

It was a good exercise for me — to simplify the notebook. Yes I view colab as a front end

…

-p Paul Fishwick, PhD Distinguished University Chair of Arts, Technology, and Emerging Communication Professor of Computer Science Director, Creative Automata Laboratory The University of Texas at Dallas Arts & Technology 800 West Campbell Road, AT10 Richardson, TX 75080-3021 Home: https://atec.utdallas.edu/content/fishwick-paul/ Media: ***@***.*** Modeling: digest.sigsim.org Twitter: @PaulFishwick ONLINE: Webex,Collaborate, TEAMS, Zoom, Skype, Hangout

On Jun 4, 2022, at 3:02 AM, Hannu Töyrylä ***@***.***> wrote:  Thanks! I perhaps reacted a bit strongly... but I really felt that those comments were not informative to a newcomer. Thinking further, if I had understood from the beginning that you intend to use my code as it is and use colab just as a front end, I would probably have answered differently. But the colabs I had seen had been major rewrites of earlier code and I believed that you were about to do something like that and starting thinking what could be done differently (like the train steps thing). Not sure I get the part about "the two -h cells" though. — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you commented.

[metaphorz]

My position is if you want and can edit it, this would be welcome. This solves your “disliking of the situation”. I am not editing it.

…

-p Paul Fishwick, PhD Distinguished University Chair of Arts, Technology, and Emerging Communication Professor of Computer Science Director, Creative Automata Laboratory The University of Texas at Dallas Arts & Technology 800 West Campbell Road, AT10 Richardson, TX 75080-3021 Home: utdallas.edu/atec/fishwick Media: ***@***.*** Modeling: digest.sigsim.org Twitter: @PaulFishwick ONLINE: Webex,Collaborate, TEAMS, Zoom, Skype, Hangout From: Hannu Töyrylä ***@***.***> Reply-To: htoyryla/minidiffusion ***@***.***> Date: Thursday, June 2, 2022 at 2:20 PM To: htoyryla/minidiffusion ***@***.***> Cc: Paul Fishwick ***@***.***>, Comment ***@***.***> Subject: Re: [htoyryla/minidiffusion] Diffusampler (Discussion #8) They helped you in the process. However, "50 will not work" instead of "stick to 1000" is not helpful. I also noticed that the list of options is from the code where the help texts are often unedited after a copy-paste. There is better info on the settings in the readme and in discussions (diffudiver, diffusampler), but even that is incomplete. So... I don't like the situation but I am absolutely not willing to become a maintainer. — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you commented.Message ID: ***@***.***>

[htoyryla]

Frankly, I barely remember what the problem was. Haven't touched this project on 2 1/2 months now.

I guess one problem was that this was evolving a lot while you were trying to deploy it.
Updating the documentation during ongoing development was simply too much for me.

Especially there weren't any knowledgable contributors to fill in gaps left when I had no time. This was simply of too little interest compared to the easy to use fancy diffusion apps available.

I learned to lot about diffusion though. And even if minimal models are not so easy as I might have thought, they are not useless either.

Especially since I learned how to do proper guidance, which is implemented in diffumaker, resulting in much better results from text prompts. At the moment it should be the preferred script for inference instead of diffudiver, if you are doing mainly text2image.

But you may have to train a model for days, even if sometimes only a few hours is enough.

During all this, my two linux boxes started to fail, having been in use since 2016. It took time and money keep everything working.

And now the energy situation in Europe is such that I simply cannot afford training runs taking several days (even if my new machines would be ideal for it).

I guess what I was concerned of was whether the instructions to the colab end-user were useful or confusing. I don't know if it matters any longer. Anyhow it is difficult to do it well while a system is still taking form. It usually helps if there are a couple active, interested of contributors in the repo.

If there is something you expect from me, we need to be more specific. Otherwise I don't care.

[metaphorz]

This is so old I don't remember either. I did create create a notebook

[htoyryla]

I mean... I received the above post today, which is why I responded. Maybe it was stuck somewhere and arrived a bit late.

[metaphorz]

Oh ok. No issues

…

Sent from my iPhone Paul Fishwick, PhD Distinguished University Chair of Arts, Technology & Emerging Communication Professor of Computer Science The University of Texas at Dallas Art, Technology & Emerging Communication 800 West Campbell Road, AT10 Richardson, TX 75080-3021 Blog 1: ***@***.*** Blog 2: digest.sigsim.org LinkedIn: metaphorz Twitter: @PaulFishwick

On Oct 11, 2022, at 2:19 PM, Hannu Töyrylä ***@***.***> wrote:  I mean... I received the above post today, which is why I responded. Maybe it was stuck somewhere and arrived a bit late. — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you commented.