ComfyUI/comfy
Jedrzej Kosinski 6c9bd11fa3
Hooks Part 2 - TransformerOptionsHook and AdditionalModelsHook (#6377)
* Add 'sigmas' to transformer_options so that downstream code can know about the full scope of current sampling run, fix Hook Keyframes' guarantee_steps=1 inconsistent behavior with sampling split across different Sampling nodes/sampling runs by referencing 'sigmas'

* Cleaned up hooks.py, refactored Hook.should_register and add_hook_patches to use target_dict instead of target so that more information can be provided about the current execution environment if needed

* Refactor WrapperHook into TransformerOptionsHook, as there is no need to separate out Wrappers/Callbacks/Patches into different hook types (all affect transformer_options)

* Refactored HookGroup to also store a dictionary of hooks separated by hook_type, modified necessary code to no longer need to manually separate out hooks by hook_type

* In inner_sample, change "sigmas" to "sampler_sigmas" in transformer_options to not conflict with the "sigmas" that will overwrite "sigmas" in _calc_cond_batch

* Refactored 'registered' to be HookGroup instead of a list of Hooks, made AddModelsHook operational and compliant with should_register result, moved TransformerOptionsHook handling out of ModelPatcher.register_all_hook_patches, support patches in TransformerOptionsHook properly by casting any patches/wrappers/hooks to proper device at sample time

* Made hook clone code sane, made clear ObjectPatchHook and SetInjectionsHook are not yet operational

* Fix performance of hooks when hooks are appended via Cond Pair Set Props nodes by properly caching between positive and negative conds, make hook_patches_backup behave as intended (in the case that something pre-registers WeightHooks on the ModelPatcher instead of registering it at sample time)

* Filter only registered hooks on self.conds in CFGGuider.sample

* Make hook_scope functional for TransformerOptionsHook

* removed 4 whitespace lines to satisfy Ruff,

* Add a get_injections function to ModelPatcher

* Made TransformerOptionsHook contribute to registered hooks properly, added some doc strings and removed a so-far unused variable

* Rename AddModelsHooks to AdditionalModelsHook, rename SetInjectionsHook to InjectionsHook (not yet implemented, but at least getting the naming figured out)

* Clean up a typehint
2025-01-11 12:20:23 -05:00
..
cldm Replace print with logging (#6138) 2024-12-20 16:24:55 -05:00
comfy_types Fix custom node type-hinting examples (#6281) 2024-12-31 03:41:09 -05:00
extra_samplers Fix and enforce no trailing whitespace. 2024-12-31 03:16:37 -05:00
k_diffusion Make a few more samplers deterministic. 2025-01-07 02:12:32 -05:00
ldm WIP support for Nvidia Cosmos 7B and 14B text to world (video) models. 2025-01-10 09:14:16 -05:00
sd1_tokenizer Initial commit. 2023-01-16 22:37:14 -05:00
t2i_adapter Controlnet refactor. 2024-06-27 18:43:11 -04:00
taesd Improvements to the TAESD3 implementation. 2024-06-16 02:04:24 -04:00
text_encoders WIP support for Nvidia Cosmos 7B and 14B text to world (video) models. 2025-01-10 09:14:16 -05:00
checkpoint_pickle.py Remove pytorch_lightning dependency. 2023-06-13 10:11:33 -04:00
cli_args.py Add option to log non-error output to stdout (#6243) 2024-12-27 14:40:05 -05:00
clip_config_bigg.json Fix potential issue with non clip text embeddings. 2024-07-30 14:41:13 -04:00
clip_model.py Support new flux model variants. 2024-11-21 08:38:23 -05:00
clip_vision_config_g.json Add support for clip g vision model to CLIPVisionLoader. 2023-08-18 11:13:29 -04:00
clip_vision_config_h.json Add support for unCLIP SD2.x models. 2023-04-01 23:19:15 -04:00
clip_vision_config_vitl_336.json support clip-vit-large-patch14-336 (#4042) 2024-07-17 13:12:50 -04:00
clip_vision_config_vitl.json Add support for unCLIP SD2.x models. 2023-04-01 23:19:15 -04:00
clip_vision_siglip_384.json Support new flux model variants. 2024-11-21 08:38:23 -05:00
clip_vision.py Add a way to disable cropping in the CLIPVisionEncode node. 2024-11-28 20:24:47 -05:00
conds.py Fix bug with cosxl edit model. 2024-06-10 16:00:03 -04:00
controlnet.py Remove some trailing white space. 2024-12-27 18:02:30 -05:00
diffusers_convert.py Basic Hunyuan Video model support. 2024-12-16 19:35:40 -05:00
diffusers_load.py load_unet -> load_diffusion_model with a model_options argument. 2024-08-12 23:20:57 -04:00
float.py Clamp output when rounding weight to prevent Nan. 2024-10-19 19:07:10 -04:00
gligen.py Lint and fix undefined names (1/N) (#6028) 2024-12-12 18:55:26 -05:00
hooks.py Hooks Part 2 - TransformerOptionsHook and AdditionalModelsHook (#6377) 2025-01-11 12:20:23 -05:00
latent_formats.py Fast latent preview for Cosmos. 2025-01-11 04:41:24 -05:00
lora_convert.py Support new flux model variants. 2024-11-21 08:38:23 -05:00
lora.py Add PixArt model support (#6055) 2024-12-20 15:25:00 -05:00
model_base.py WIP support for Nvidia Cosmos 7B and 14B text to world (video) models. 2025-01-10 09:14:16 -05:00
model_detection.py WIP support for Nvidia Cosmos 7B and 14B text to world (video) models. 2025-01-10 09:14:16 -05:00
model_management.py Remove old unused function. 2025-01-04 07:20:54 -05:00
model_patcher.py Hooks Part 2 - TransformerOptionsHook and AdditionalModelsHook (#6377) 2025-01-11 12:20:23 -05:00
model_sampling.py Make timestep ranges more usable on rectified flow models. 2024-12-05 16:40:58 -05:00
ops.py Make fast fp8 take a bit less peak memory. 2024-12-24 18:05:19 -05:00
options.py Only parse command line args when main.py is called. 2023-09-13 11:38:20 -04:00
patcher_extension.py Add ruff rule for empty line with trailing whitespace. 2024-12-28 05:23:08 -05:00
sample.py Auto reshape 2d to 3d latent for single image generation on video model. 2024-12-29 02:26:49 -05:00
sampler_helpers.py Hooks Part 2 - TransformerOptionsHook and AdditionalModelsHook (#6377) 2025-01-11 12:20:23 -05:00
samplers.py Hooks Part 2 - TransformerOptionsHook and AdditionalModelsHook (#6377) 2025-01-11 12:20:23 -05:00
sd1_clip_config.json Fix potential issue with non clip text embeddings. 2024-07-30 14:41:13 -04:00
sd1_clip.py Replace print with logging (#6138) 2024-12-20 16:24:55 -05:00
sd.py WIP support for Nvidia Cosmos 7B and 14B text to world (video) models. 2025-01-10 09:14:16 -05:00
sdxl_clip.py Long CLIP L support for SDXL, SD3 and Flux. 2024-09-15 07:59:38 -04:00
supported_models_base.py Mixed precision diffusion models with scaled fp8. 2024-10-21 18:12:51 -04:00
supported_models.py WIP support for Nvidia Cosmos 7B and 14B text to world (video) models. 2025-01-10 09:14:16 -05:00
utils.py Document get_attr and get_model_object (#6357) 2025-01-06 20:12:22 -05:00