298977887/ComfyUI
1
0
Fork 0
mirror of https://github.com/comfyanonymous/ComfyUI.git synced 2025-01-11 10:25:16 +00:00
Code Issues Packages Projects Releases Wiki Activity
2,399 Commits 2 Branches 32 Tags 1.1 GiB
a5f4292f9f
Commit Graph

228 Commits

Author SHA1 Message Date
comfyanonymous
a7874d1a8b Add support for the stable diffusion x4 upscaling model. 
This is an old model.

Load the checkpoint like a regular one and use the new
SD_4XUpscale_Conditioning node.
 2024-01-03 03:37:56 -05:00
comfyanonymous
5eddfdd80c Refactor VAE code. 
Replace constants with downscale_ratio and latent_channels.
 2024-01-02 13:24:34 -05:00
comfyanonymous
824e4935f5 Add dtype parameter to VAE object. 2023-12-12 12:03:29 -05:00
comfyanonymous
ba07cb748e Use faster manual cast for fp8 in unet. 2023-12-11 18:24:44 -05:00
comfyanonymous
9ac0b487ac Make --gpu-only put intermediate values in GPU memory instead of cpu. 2023-12-08 02:35:45 -05:00
comfyanonymous
983ebc5792 Use smart model management for VAE to decrease latency. 2023-11-28 04:58:51 -05:00
comfyanonymous
c45d1b9b67 Add a function to load a unet from a state dict. 2023-11-27 17:41:29 -05:00
comfyanonymous
871cc20e13 Support SVD img2vid model. 2023-11-23 19:41:33 -05:00
comfyanonymous
410bf07771 Make VAE memory estimation take dtype into account. 2023-11-22 18:17:19 -05:00
comfyanonymous
6a491ebe27 Allow model config to preprocess the vae state dict on load. 2023-11-21 16:29:18 -05:00
comfyanonymous
cd4fc77d5f Add taesd and taesdxl to VAELoader node. 
They will show up if both the taesd_encoder and taesd_decoder or taesdxl
model files are present in the models/vae_approx directory.
 2023-11-21 12:54:19 -05:00
comfyanonymous
0cf4e86939 Add some command line arguments to store text encoder weights in fp8. 
Pytorch supports two variants of fp8:
--fp8_e4m3fn-text-enc (the one that seems to give better results)
--fp8_e5m2-text-enc
 2023-11-17 02:56:59 -05:00
comfyanonymous
0a6fd49a3e Print leftover keys when using the UNETLoader. 2023-11-07 22:15:55 -05:00
comfyanonymous
2455aaed8a Allow model or clip to be None in load_lora_for_models. 2023-11-01 20:27:20 -04:00
comfyanonymous
2a134bfab9 Fix checkpoint loader with config. 2023-10-27 22:13:55 -04:00
comfyanonymous
6ec3f12c6e Support SSD1B model and make it easier to support asymmetric unets. 2023-10-27 14:45:15 -04:00
comfyanonymous
430a8334c5 Fix some potential issues. 2023-10-18 19:48:36 -04:00
comfyanonymous
d44a2de49f Make VAE code closer to sgm. 2023-10-17 15:18:51 -04:00
comfyanonymous
9a55dadb4c Refactor code so model can be a dtype other than fp32 or fp16. 2023-10-13 14:41:17 -04:00
comfyanonymous
72188dffc3 load_checkpoint_guess_config can now optionally output the model. 2023-10-06 13:48:18 -04:00
City
9bfec2bdbf Fix quality loss due to low precision 2023-10-04 15:40:59 +02:00
comfyanonymous
71713888c4 Print missing VAE keys. 2023-09-28 00:54:57 -04:00
comfyanonymous
7d401ed1d0 Add ldm format support to UNETLoader. 2023-09-11 16:36:50 -04:00
comfyanonymous
7931ff0fd9 Support SDXL inpaint models. 2023-09-01 15:22:52 -04:00
comfyanonymous
fe4c07400c Fix "Load Checkpoint with config" node. 2023-08-29 23:58:32 -04:00
comfyanonymous
1300a1bb4c Text encoder should initially load on the offload_device not the regular. 2023-08-28 15:08:45 -04:00
comfyanonymous
f92074b84f Move ModelPatcher to model_patcher.py 2023-08-28 14:51:31 -04:00
comfyanonymous
c77f02e1c6 Move controlnet code to comfy/controlnet.py 2023-08-25 17:33:04 -04:00
comfyanonymous
15a7716fa6 Move lora code to comfy/lora.py 2023-08-25 17:11:51 -04:00
comfyanonymous
ec96f6d03a Move text_projection to base clip model. 2023-08-24 23:43:48 -04:00
comfyanonymous
51dde87e97 Try to free enough vram for control lora inference. 2023-08-24 17:20:54 -04:00
comfyanonymous
cc44ade79e Always shift text encoder to GPU when the device supports fp16. 2023-08-23 21:45:00 -04:00
comfyanonymous
00c0b2c507 Initialize text encoder to target dtype. 2023-08-23 21:01:15 -04:00
comfyanonymous
f081017c1a Save memory by storing text encoder weights in fp16 in most situations. 
Do inference in fp32 to make sure quality stays the exact same.
 2023-08-23 01:08:51 -04:00
comfyanonymous
afcb9cb1df All resolutions now work with t2i adapter for SDXL. 2023-08-22 16:23:54 -04:00
comfyanonymous
85fde89d7f T2I adapter SDXL. 2023-08-22 14:40:43 -04:00
comfyanonymous
cf5ae46928 Controlnet/t2iadapter cleanup. 2023-08-22 01:06:26 -04:00
comfyanonymous
763b0cf024 Fix control lora not working in fp32. 2023-08-21 20:38:31 -04:00
comfyanonymous
199d73364a Fix ControlLora on lowvram. 2023-08-21 00:54:04 -04:00
comfyanonymous
d08e53de2e Remove autocast from controlnet code. 2023-08-20 21:47:32 -04:00
comfyanonymous
c9b562aed1 Free more memory before VAE encode/decode. 2023-08-19 12:13:13 -04:00
comfyanonymous
d6e4b342e6 Support for Control Loras. 
Control loras are controlnets where some of the weights are stored in
"lora" format: an up and a down low rank matrice that when multiplied
together and added to the unet weight give the controlnet weight.

This allows a much smaller memory footprint depending on the rank of the
matrices.

These controlnets are used just like regular ones.
 2023-08-18 11:59:51 -04:00
comfyanonymous
c28db1f315 Fix potential issues with patching models when saving checkpoints. 2023-08-17 11:07:08 -04:00
comfyanonymous
89a0767abf Smarter memory management. 
Try to keep models on the vram when possible.

Better lowvram mode for controlnets.
 2023-08-17 01:06:34 -04:00
comfyanonymous
53f326a3d8 Support diffusers mini controlnets. 2023-08-16 12:28:01 -04:00
comfyanonymous
c8a23ce9e8 Support for yet another lora type based on diffusers. 2023-08-11 13:04:21 -04:00
comfyanonymous
c20583286f Support diffuser text encoder loras. 2023-08-10 20:28:28 -04:00
comfyanonymous
d8e58f0a7e Detect hint_channels from controlnet. 2023-08-06 14:08:59 -04:00
comfyanonymous
c5d7593ccf Support loras in diffusers format. 2023-08-05 01:40:24 -04:00
comfyanonymous
4b957a0010 Initialize the unet directly on the target device. 2023-07-29 14:51:56 -04:00
comfyanonymous
727588d076 Fix some new loras. 2023-07-25 16:39:15 -04:00
comfyanonymous
5f75d784a1 Start is now 0.0 and end is now 1.0 for the timestep ranges. 2023-07-24 18:38:17 -04:00
comfyanonymous
7ff14b62f8 ControlNetApplyAdvanced can now define when controlnet gets applied. 2023-07-24 17:50:49 -04:00
comfyanonymous
22f29d66ca Try to fix memory issue with lora. 2023-07-22 21:38:56 -04:00
comfyanonymous
12a6e93171 Del the right object when applying lora. 2023-07-22 11:25:49 -04:00
comfyanonymous
78e7958d17 Support controlnet in diffusers format. 2023-07-21 22:58:16 -04:00
comfyanonymous
09386a3697 Fix issue with lora in some cases when combined with model merging. 2023-07-21 21:27:27 -04:00
comfyanonymous
58b2364f58 Properly support SDXL diffusers unet with UNETLoader node. 2023-07-21 14:38:56 -04:00
comfyanonymous
0115018695 Print errors and continue when lora weights are not compatible. 2023-07-20 19:56:22 -04:00
comfyanonymous
3ded1a3a04 Refactor of sampler code to deal more easily with different model types. 2023-07-17 01:22:12 -04:00
comfyanonymous
5f57362613 Lower lora ram usage when in normal vram mode. 2023-07-16 02:59:04 -04:00
comfyanonymous
490771b7f4 Speed up lora loading a bit. 2023-07-15 13:25:22 -04:00
comfyanonymous
50b1180dde Fix CLIPSetLastLayer not reverting when removed. 2023-07-15 01:41:21 -04:00
comfyanonymous
6fb084f39d Reduce floating point rounding errors in loras. 2023-07-15 00:53:00 -04:00
comfyanonymous
91ed2815d5 Add a node to merge CLIP models. 2023-07-14 02:41:18 -04:00
comfyanonymous
6ad0a6d7e2 Don't patch weights when multiplier is zero. 2023-07-09 17:46:56 -04:00
comfyanonymous
a9a4ba7574 Fix merging not working when model2 of model merge node was a merge. 2023-07-08 22:31:10 -04:00
comfyanonymous
e7bee85df8 Add arguments to run the VAE in fp16 or bf16 for testing. 2023-07-06 23:23:46 -04:00
comfyanonymous
ddc6f12ad5 Disable autocast in unet for increased speed. 2023-07-05 21:58:29 -04:00
comfyanonymous
af7a49916b Support loading unet files in diffusers format. 2023-07-05 17:38:59 -04:00
comfyanonymous
acf95191ff Properly support SDXL diffusers loras for unet. 2023-07-04 21:15:23 -04:00
comfyanonymous
c3e96e637d Pass device to CLIP model. 2023-07-03 16:09:37 -04:00
comfyanonymous
2c4e0b49b7 Switch to fp16 on some cards when the model is too big. 2023-07-02 10:00:57 -04:00
comfyanonymous
1c1b0e7299 --gpu-only now keeps the VAE on the device. 2023-07-01 15:22:40 -04:00
comfyanonymous
3b6fe51c1d Leave text_encoder on the CPU when it can handle it. 2023-07-01 14:38:51 -04:00
comfyanonymous
b6a60fa696 Try to keep text encoders loaded and patched to increase speed. 
load_model_gpu() is now used with the text encoder models instead of just
the unet.
 2023-07-01 13:28:07 -04:00
comfyanonymous
97ee230682 Make highvram and normalvram shift the text encoders to vram and back. 
This is faster on big text encoder models than running it on the CPU.
 2023-07-01 12:37:23 -04:00
comfyanonymous
5a9ddf94eb LoraLoader node now caches the lora file between executions. 2023-06-29 23:40:51 -04:00
comfyanonymous
62db11683b Move unet to device right after loading on highvram mode. 2023-06-29 20:43:06 -04:00
comfyanonymous
2c7c14de56 Support for SDXL text encoder lora. 2023-06-28 02:22:49 -04:00
comfyanonymous
9b93b920be Add CheckpointSave node to save checkpoints. 
The created checkpoints contain workflow metadata that can be loaded by
dragging them on top of the UI or loading them with the "Load" button.

Checkpoints will be saved in fp16 or fp32 depending on the format ComfyUI
is using for inference on your hardware. To force fp32 use: --force-fp32

Anything that patches the model weights like merging or loras will be
saved.

The output directory is currently set to: output/checkpoints but that might
change in the future.
 2023-06-26 12:22:27 -04:00
comfyanonymous
b72a7a835a Support loras based on the stability unet implementation. 2023-06-26 02:56:11 -04:00
comfyanonymous
20f579d91d Add DualClipLoader to load clip models for SDXL. 
Update LoadClip to load clip models for SDXL refiner.
 2023-06-25 01:40:38 -04:00
comfyanonymous
b7933960bb Fix CLIPLoader node. 2023-06-24 13:56:46 -04:00
comfyanonymous
05676942b7 Add some more transformer hooks and move tomesd to comfy_extras. 
Tomesd now uses q instead of x to decide which tokens to merge because
it seems to give better results.
 2023-06-24 03:30:22 -04:00
comfyanonymous
8607c2d42d Move latent scale factor from VAE to model. 2023-06-23 02:33:31 -04:00
comfyanonymous
30a3861946 Fix bug when yaml config has no clip params. 2023-06-23 01:12:59 -04:00
comfyanonymous
9e37f4c7d5 Fix error with ClipVision loader node. 2023-06-23 01:08:05 -04:00
comfyanonymous
9f83b098c9 Don't merge weights when shapes don't match and print a warning. 2023-06-22 19:08:31 -04:00
comfyanonymous
f87ec10a97 Support base SDXL and SDXL refiner models. 
Large refactor of the model detection and loading code.
 2023-06-22 13:03:50 -04:00
comfyanonymous
51581dbfa9 Fix last commits causing an issue with the text encoder lora. 2023-06-20 19:44:39 -04:00
comfyanonymous
8125b51a62 Keep a set of model_keys for faster add_patches. 2023-06-20 19:08:48 -04:00
comfyanonymous
45beebd33c Add a type of model patch useful for model merging. 2023-06-20 17:34:11 -04:00
comfyanonymous
8883cb0f67 Add a way to set patches that modify the attn2 output. 
Change the transformer patches function format to be more future proof.
 2023-06-18 22:58:22 -04:00
comfyanonymous
fb4bf7f591 This is not needed anymore and causes issues with alphas_cumprod. 2023-06-18 03:18:25 -04:00
comfyanonymous
f7edcfd927 Add a --gpu-only argument to keep and run everything on the GPU. 
Make the CLIP model work on the GPU.
 2023-06-15 15:38:52 -04:00
comfyanonymous
6b774589a5 Set model to fp16 before loading the state dict to lower ram bump. 2023-06-14 12:48:02 -04:00
comfyanonymous
388567f20b sampler_cfg_function now uses a dict for the argument. 
This means arguments can be added without issues.
 2023-06-13 16:10:36 -04:00
comfyanonymous
ff9b22d79e Turn on safe load for a few models. 2023-06-13 10:12:03 -04:00
comfyanonymous
f0a2b81cd0 Cleanup: Remove a bunch of useless files. 2023-06-13 02:19:08 -04:00