You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Hi! I'm doing a deep dive into the implementation and have some questions.
I see that the key caching-related codes are in DeepCache.sd.pipeline_stable_diffusion.py under StableDiffusionPipeline's call() around lines 726 to 740 (It says "7. Denoising loop" at the top, so I assume it must be around here). However, I am struggling to see how/where you are storing the cached up-sample block output? Are you doing this through sample_from_quad_center()? But this function is only for non-uniform sampling right?
This question is related to Q1: So far all the hyperparameters belonging to the actual caching part of deepcache are: cache_interval, cache_layer_id (I believe this is the skip connection layer), cache_block_id (this should be the up-sampling block), center, and power (these are for non-uniform sampling). However, if we specify which block we want to cache, shouldn't that also specify which skip connection is being connected to the output? So, why do we need to specify both?
Is there anywhere else I should check for stuff you have changed from the original stable_diffusion pipeline (outside of pipeline_stable_diffusion.py)?
There are parts of the paper I'm not sure about, so I apologize if my questions seem dumb. Thank you!
The text was updated successfully, but these errors were encountered:
Q1: The variable responsible for storing cached features is named as prv_features at Line 723 and undergoes updates at Line 753. To understand the mechanism of feature caching, you can delve into the inference code of the UNet architecture. The function sample_from_quad_center exclusively serves to adjust the caching schedule.
Q2: Each combination of cache_layer_id and cache_block_id corresponds uniquely to a skip connection.
Q3: I documented the modifications compared to the original pipeline in ReadMe, available here. You can find detailed information there.
Hi! I'm doing a deep dive into the implementation and have some questions.
DeepCache.sd.pipeline_stable_diffusion.py
under StableDiffusionPipeline's call() around lines 726 to 740 (It says "7. Denoising loop" at the top, so I assume it must be around here). However, I am struggling to see how/where you are storing the cached up-sample block output? Are you doing this throughsample_from_quad_center()
? But this function is only for non-uniform sampling right?There are parts of the paper I'm not sure about, so I apologize if my questions seem dumb. Thank you!
The text was updated successfully, but these errors were encountered: