1364 Commits

Author	SHA1	Message	Date
comfyanonymous	507199d9a8	Uni pc sampler now works with audio and video models.	2025-01-18 05:27:58 -05:00
comfyanonymous	2f3ab40b62	Add warning when using old pytorch versions.	2025-01-17 18:47:27 -05:00
comfyanonymous	0aa2368e46	Fix some cosmos fp8 issues.	2025-01-16 17:45:37 -05:00
comfyanonymous	cca96a85ae	Fix cosmos VAE failing with videos longer than 121 frames.	2025-01-16 16:30:06 -05:00
comfyanonymous	31831e6ef1	Code refactor.	2025-01-16 07:23:54 -05:00
comfyanonymous	88ceb28e20	Tweak hunyuan memory usage factor.	2025-01-16 06:31:03 -05:00
comfyanonymous	23289a6a5c	Clean up some debug lines.	2025-01-16 04:24:39 -05:00
comfyanonymous	9d8b6c1f46	More accurate memory estimation for cosmos and hunyuan video.	2025-01-16 03:48:40 -05:00
comfyanonymous	6320d05696	Slightly lower hunyuan video memory usage.	2025-01-16 00:23:01 -05:00
comfyanonymous	25683b5b02	Lower cosmos diffusion model memory usage.	2025-01-15 23:46:42 -05:00
comfyanonymous	4758fb64b9	Lower cosmos VAE memory usage by a bit.	2025-01-15 22:57:52 -05:00
comfyanonymous	008761166f	Optimize first attention block in cosmos VAE.	2025-01-15 21:48:46 -05:00
comfyanonymous	cba58fff0b	Remove unsafe embedding load for very old pytorch.	2025-01-15 04:32:23 -05:00
comfyanonymous	2feb8d0b77	Force safe loading of files in torch format on pytorch 2.4+ ... If this breaks something for you make an issue.	2025-01-15 03:50:27 -05:00
Pam	c78a45685d	Rewrite res_multistep sampler and implement res_multistep_cfg_pp sampler. (#6462)	2025-01-14 18:20:06 -05:00
comfyanonymous	3aaabb12d4	Implement Cosmos Image/Video to World (Video) diffusion models. ... Use CosmosImageToVideoLatent to set the input image/video.	2025-01-14 05:14:10 -05:00
comfyanonymous	1f1c7b7b56	Remove useless code.	2025-01-13 03:52:37 -05:00
comfyanonymous	90f349f93d	Add res_multistep sampler from the cosmos code. ... This sampler should work with all models.	2025-01-12 03:10:07 -05:00
Jedrzej Kosinski	6c9bd11fa3	Hooks Part 2 - TransformerOptionsHook and AdditionalModelsHook (#6377) ... * Add 'sigmas' to transformer_options so that downstream code can know about the full scope of current sampling run, fix Hook Keyframes' guarantee_steps=1 inconsistent behavior with sampling split across different Sampling nodes/sampling runs by referencing 'sigmas' * Cleaned up hooks.py, refactored Hook.should_register and add_hook_patches to use target_dict instead of target so that more information can be provided about the current execution environment if needed * Refactor WrapperHook into TransformerOptionsHook, as there is no need to separate out Wrappers/Callbacks/Patches into different hook types (all affect transformer_options) * Refactored HookGroup to also store a dictionary of hooks separated by hook_type, modified necessary code to no longer need to manually separate out hooks by hook_type * In inner_sample, change "sigmas" to "sampler_sigmas" in transformer_options to not conflict with the "sigmas" that will overwrite "sigmas" in _calc_cond_batch * Refactored 'registered' to be HookGroup instead of a list of Hooks, made AddModelsHook operational and compliant with should_register result, moved TransformerOptionsHook handling out of ModelPatcher.register_all_hook_patches, support patches in TransformerOptionsHook properly by casting any patches/wrappers/hooks to proper device at sample time * Made hook clone code sane, made clear ObjectPatchHook and SetInjectionsHook are not yet operational * Fix performance of hooks when hooks are appended via Cond Pair Set Props nodes by properly caching between positive and negative conds, make hook_patches_backup behave as intended (in the case that something pre-registers WeightHooks on the ModelPatcher instead of registering it at sample time) * Filter only registered hooks on self.conds in CFGGuider.sample * Make hook_scope functional for TransformerOptionsHook * removed 4 whitespace lines to satisfy Ruff, * Add a get_injections function to ModelPatcher * Made TransformerOptionsHook contribute to registered hooks properly, added some doc strings and removed a so-far unused variable * Rename AddModelsHooks to AdditionalModelsHook, rename SetInjectionsHook to InjectionsHook (not yet implemented, but at least getting the naming figured out) * Clean up a typehint	2025-01-11 12:20:23 -05:00
comfyanonymous	ee8a7ab69d	Fast latent preview for Cosmos.	2025-01-11 04:41:24 -05:00
comfyanonymous	2ff3104f70	WIP support for Nvidia Cosmos 7B and 14B text to world (video) models.	2025-01-10 09:14:16 -05:00
comfyanonymous	129d8908f7	Add argument to skip the output reshaping in the attention functions.	2025-01-10 06:27:37 -05:00
comfyanonymous	ff838657fa	Cleaner handling of attention mask in ltxv model code.	2025-01-09 07:12:03 -05:00
comfyanonymous	2307ff6746	Improve some logging messages.	2025-01-08 19:05:22 -05:00
comfyanonymous	d0f3752e33	Properly calculate inner dim for t5 model. ... This is required to support some different types of t5 models.	2025-01-07 17:33:03 -05:00
comfyanonymous	4209edf48d	Make a few more samplers deterministic.	2025-01-07 02:12:32 -05:00
Chenlei Hu	d055325783	Document get_attr and get_model_object (#6357) ... * Document get_attr and get_model_object * Update model_patcher.py * Update model_patcher.py * Update model_patcher.py	2025-01-06 20:12:22 -05:00
comfyanonymous	916d1e14a9	Make ancestral samplers more deterministic.	2025-01-06 03:04:32 -05:00
Jedrzej Kosinski	c496e53519	In inner_sample, change "sigmas" to "sampler_sigmas" in transformer_options to not conflict with the "sigmas" that will overwrite "sigmas" in _calc_cond_batch (#6360)	2025-01-06 01:36:47 -05:00
comfyanonymous	d45ebb63f6	Remove old unused function.	2025-01-04 07:20:54 -05:00
comfyanonymous	9e9c8a1c64	Clear cache as often on AMD as Nvidia. ... I think the issue this was working around has been solved. If you notice that this change slows things down or causes stutters on your AMD GPU with ROCm on Linux please report it.	2025-01-02 08:44:16 -05:00
Andrew Kvochko	0f11d60afb	Fix temporal tiling for decoder, remove redundant tiles. (#6306) ... This commit fixes the temporal tile size calculation, and removes a redundant tile at the end of the range when its elements are completely covered by the previous tile. Co-authored-by: Andrew Kvochko <a.kvochko@lightricks.com>	2025-01-01 16:29:01 -05:00
comfyanonymous	79eea51a1d	Fix and enforce all ruff W rules.	2025-01-01 03:08:33 -05:00
blepping	c0338a46a4	Fix unknown sampler error handling in calculate_sigmas function (#6280) ... Modernize calculate_sigmas function	2024-12-31 17:33:50 -05:00
Jedrzej Kosinski	1c99734e5a	Add missing model_options param (#6296)	2024-12-31 14:46:55 -05:00
filtered	67758f50f3	Fix custom node type-hinting examples (#6281) ... * Fix import in comfy_types doc / sample * Clarify docstring	2024-12-31 03:41:09 -05:00
comfyanonymous	b7572b2f87	Fix and enforce no trailing whitespace.	2024-12-31 03:16:37 -05:00
blepping	a90aafafc1	Add kl_optimal scheduler (#6206) ... * Add kl_optimal scheduler * Rename kl_optimal_schedule to kl_optimal_scheduler to be more consistent	2024-12-30 05:09:38 -05:00
comfyanonymous	d9b7cfac7e	Fix and enforce new lines at the end of files.	2024-12-30 04:14:59 -05:00
Jedrzej Kosinski	3507870535	Add 'sigmas' to transformer_options so that downstream code can know about the full scope of current sampling run, fix Hook Keyframes' guarantee_steps=1 inconsistent behavior with sampling split across different Sampling nodes/sampling runs by referencing 'sigmas' (#6273)	2024-12-30 03:42:49 -05:00
comfyanonymous	a618f768e0	Auto reshape 2d to 3d latent for single image generation on video model.	2024-12-29 02:26:49 -05:00
comfyanonymous	b504bd606d	Add ruff rule for empty line with trailing whitespace.	2024-12-28 05:23:08 -05:00
comfyanonymous	d170292594	Remove some trailing white space.	2024-12-27 18:02:30 -05:00
filtered	9cfd185676	Add option to log non-error output to stdout (#6243) ... * nit * Add option to log non-error output to stdout - No change to default behaviour - Adds CLI argument: --log-stdout - With this arg present, any logging of a level below logging.ERROR will be sent to stdout instead of stderr	2024-12-27 14:40:05 -05:00
comfyanonymous	4b5bcd8ac4	Closer memory estimation for hunyuan dit model.	2024-12-27 07:37:00 -05:00
comfyanonymous	ceb50b2cbf	Closer memory estimation for pixart models.	2024-12-27 07:30:09 -05:00
comfyanonymous	160ca08138	Use python 3.9 in launch test instead of 3.8 ... Fix ruff check.	2024-12-26 20:05:54 -05:00
Huazhong Ji	c4bfdba330	Support ascend npu (#5436) ... * support ascend npu Co-authored-by: YukMingLaw <lymmm2@163.com> Co-authored-by: starmountain1997 <guozr1997@hotmail.com> Co-authored-by: Ginray <ginray0215@gmail.com>	2024-12-26 19:36:50 -05:00
comfyanonymous	ee9547ba31	Improve temporal VAE Encode (Tiled) math.	2024-12-26 07:18:49 -05:00
comfyanonymous	19a64d6291	Cleanup some mac related code.	2024-12-25 05:32:51 -05:00
comfyanonymous	b486885e08	Disable bfloat16 on older mac.	2024-12-25 05:18:50 -05:00
comfyanonymous	0229228f3f	Clean up the VAE dtypes code.	2024-12-25 04:50:34 -05:00
comfyanonymous	99a1fb6027	Make fast fp8 take a bit less peak memory.	2024-12-24 18:05:19 -05:00
comfyanonymous	73e04987f7	Prevent black images in VAE Decode (Tiled) node. ... Overlap should be minimum 1 with tiling 2 for tiled temporal VAE decoding.	2024-12-24 07:36:30 -05:00
comfyanonymous	5388df784a	Add temporal tiling to VAE Encode (Tiled) node.	2024-12-24 07:10:09 -05:00
comfyanonymous	bc6dac4327	Add temporal tiling to VAE Decode (Tiled) node. ... You can now do tiled VAE decoding on the temporal direction for videos.	2024-12-23 20:03:37 -05:00
comfyanonymous	15564688ed	Add a try except block so if torch version is weird it won't crash.	2024-12-23 03:22:48 -05:00
Simon Lui	c6b9c11ef6	Add oneAPI device selector for xpu and some other changes. (#6112) ... * Add oneAPI device selector and some other minor changes. * Fix device selector variable name. * Flip minor version check sign. * Undo changes to README.md.	2024-12-23 03:18:32 -05:00
comfyanonymous	e44d0ac7f7	Make --novram completely offload weights. ... This flag is mainly used for testing the weight offloading, it shouldn't actually be used in practice. Remove useless import.	2024-12-23 01:51:08 -05:00
comfyanonymous	56bc64f351	Comment out some useless code.	2024-12-22 23:51:14 -05:00
zhangp365	f7d83b72e0	fixed a bug in ldm/pixart/blocks.py (#6158)	2024-12-22 23:44:20 -05:00
comfyanonymous	80f07952d2	Fix lowvram issue with ltxv vae.	2024-12-22 23:20:17 -05:00
comfyanonymous	57f330caf9	Relax minimum ratio of weights loaded in memory on nvidia. ... This should make it possible to do higher res images/longer videos by further offloading weights to CPU memory. Please report an issue if this slows down things on your system.	2024-12-22 03:06:37 -05:00
comfyanonymous	da13b6b827	Get rid of meshgrid warning.	2024-12-20 18:02:12 -05:00
comfyanonymous	c86cd58573	Remove useless code.	2024-12-20 17:50:03 -05:00
comfyanonymous	b5fe39211a	Remove some useless code.	2024-12-20 17:43:50 -05:00
comfyanonymous	e946667216	Some fixes/cleanups to pixart code. ... Commented out the masking related code because it is never used in this implementation.	2024-12-20 17:10:52 -05:00
Chenlei Hu	d7969cb070	Replace print with logging (#6138) ... * Replace print with logging * nit * nit * nit * nit * nit * nit	2024-12-20 16:24:55 -05:00
City	bddb02660c	Add PixArt model support (#6055) ... * PixArt initial version * PixArt Diffusers convert logic * pos_emb and interpolation logic * Reduce duplicate code * Formatting * Use optimized attention * Edit empty token logic * Basic PixArt LoRA support * Fix aspect ratio logic * PixArtAlpha text encode with conds * Use same detection key logic for PixArt diffusers	2024-12-20 15:25:00 -05:00
comfyanonymous	418eb7062d	Support new LTXV VAE.	2024-12-20 04:38:29 -05:00
comfyanonymous	cac68ca813	Fix some more video tiled encode issues. ... The downscale_ratio formula for the temporal had issues with some frame numbers.	2024-12-19 23:14:03 -05:00
comfyanonymous	52c1d933b2	Fix tiled hunyuan video VAE encode issue. ... Some shapes like 1024x1024 with tile_size 256 and overlap 64 had issues.	2024-12-19 22:55:15 -05:00
comfyanonymous	2dda7c11a3	More proper fix for the memory issue.	2024-12-19 16:21:56 -05:00
comfyanonymous	3ad3248ad7	Fix lowvram bug when using a model multiple times in a row. ... The memory system would load an extra 64MB each time until either the model was completely in memory or OOM.	2024-12-19 16:04:56 -05:00
comfyanonymous	c441048a4f	Make VAE Encode tiled node work with video VAE.	2024-12-19 05:31:39 -05:00
comfyanonymous	9f4b181ab3	Add fast previews for hunyuan video.	2024-12-18 18:24:23 -05:00
comfyanonymous	cbbf077593	Small optimizations.	2024-12-18 18:23:28 -05:00
comfyanonymous	ff2ff02168	Support old diffusion-pipe hunyuan video loras.	2024-12-18 06:23:54 -05:00
comfyanonymous	4c5c4ddeda	Fix regression in VAE code on old pytorch versions.	2024-12-18 03:08:28 -05:00
comfyanonymous	37e5390f5f	Add: --use-sage-attention to enable SageAttention. ... You need to have the library installed first.	2024-12-18 01:56:10 -05:00
comfyanonymous	a4f59bc65e	Pick attention implementation based on device in llama code.	2024-12-18 01:30:20 -05:00
comfyanonymous	ca457f7ba1	Properly tokenize the template for hunyuan video.	2024-12-17 16:22:02 -05:00
comfyanonymous	cd6f615038	Fix tiled vae not working with some shapes.	2024-12-17 16:22:02 -05:00
comfyanonymous	e4e1bff605	Support diffusion-pipe hunyuan video lora format.	2024-12-17 07:14:21 -05:00
comfyanonymous	d6656b0c0c	Support llama hunyuan video text encoder in scaled fp8 format.	2024-12-17 04:19:22 -05:00
comfyanonymous	f4cdedea62	Fix regression with ltxv VAE.	2024-12-17 02:17:31 -05:00
comfyanonymous	39b1fc4ccc	Adjust used dtypes for hunyuan video VAE and diffusion model.	2024-12-16 23:31:10 -05:00
comfyanonymous	bda1482a27	Basic Hunyuan Video model support.	2024-12-16 19:35:40 -05:00
comfyanonymous	19ee5d9d8b	Don't expand mask when not necessary. ... Expanding seems to slow down inference.	2024-12-16 18:22:50 -05:00
Raphael Walker	61b50720d0	Add support for attention masking in Flux (#5942) ... * fix attention OOM in xformers * allow passing attention mask in flux attention * allow an attn_mask in flux * attn masks can be done using replace patches instead of a separate dict * fix return types * fix return order * enumerate * patch the right keys * arg names * fix a silly bug * fix xformers masks * replace match with if, elif, else * mask with image_ref_size * remove unused import * remove unused import 2 * fix pytorch/xformers attention This corrects a weird inconsistency with skip_reshape. It also allows masks of various shapes to be passed, which will be automtically expanded (in a memory-efficient way) to a size that is compatible with xformers or pytorch sdpa respectively. * fix mask shapes	2024-12-16 18:21:17 -05:00
comfyanonymous	e83063bf24	Support conv3d in PatchEmbed.	2024-12-14 05:46:04 -05:00
comfyanonymous	4e14032c02	Make pad_to_patch_size function work on multi dim.	2024-12-13 07:22:05 -05:00
Chenlei Hu	563291ee51	Enforce all pyflake lint rules (#6033) ... * Enforce F821 undefined-name * Enforce all pyflake lint rules	2024-12-12 19:29:37 -05:00
Chenlei Hu	2cddbf0821	Lint and fix undefined names (1/N) (#6028)	2024-12-12 18:55:26 -05:00
Chenlei Hu	60749f345d	Lint and fix undefined names (3/N) (#6030)	2024-12-12 18:49:40 -05:00
Chenlei Hu	d9d7f3c619	Lint all unused variables (#5989) ... * Enable F841 * Autofix * Remove all unused variable assignment	2024-12-12 17:59:16 -05:00
comfyanonymous	fd5dfb812c	Set initial load devices for te and model to mps device on mac.	2024-12-12 06:00:31 -05:00
comfyanonymous	7a7efe8424	Support loading some checkpoint files with nested dicts.	2024-12-11 08:04:54 -05:00
comfyanonymous	44db978531	Fix a few things in text enc code for models with no eos token.	2024-12-10 23:07:26 -05:00
comfyanonymous	1c8d11e48a	Support different types of tokenizers. ... Support tokenizers without an eos token. Pass full sentences to tokenizer for more efficient tokenizing.	2024-12-10 15:03:39 -05:00