Preface:
Learning ComfyUI is a long battle, and ComfyUI Impact is a huge module node library, built-in many very useful and powerful nodes, such as Detector, Detail Enhancer, Preview Bridge, Wildcard, Hook, Picture Sender, Picture Receiver and so on. Through the combination of these nodes, we can achieve a lot of work, such as automatic face detection and optimization repair, area enhancement, local repainting, crowd control, hairstyling, changing the model's clothes and so on. ComfyUI Impact is a big river that everyone can't bypass on the way of ComfyUI advancement, so this post will lead you to understand and learn to use these nodes. I wish you all good luck in your studies and become a master of ComfyUI soon!
catalogs
I. Installation
II. CLIPSegDetectorProvider node
III. Decompose (SEGS) / From_SEG_ELT node
IV. PixelKSampleUpscalerProviderPipe node
V. lterative Upscale (Latent/on Pixel Space) Node
VI. Make Tile SEGS nodes
VII. Picker (SEGS) nodes
I. Installation
Method 1: Installation via ComfyUI Manager (recommended)
Open the Manager interface
Method 2: Installation with git clone command
Enter cmd in the ComfyUI/custom_nodes directory and press Enter to enter the computer terminal.
Enter the following line of code in the terminal to start the download
git clone /ltdrdata/ComfyUI-Impact-Pack
II. CLIPSegDetectorProvider node
This node focuses on image segmentation and object detection using CLIPSeg model. This node can segment objects or regions in an image that match the description based on the input image and text description.
Parameters:
text → input the noun of the part to be detected, e.g. hand, face, wing, etc.
blur → denotes the blur region that determines whether some noise is retained or discarded for each detection frame detection
threshold → select the threshold of the final mask, the detection model detects not just one mask, but many detection frames, the model will ultimately choose the best one from the many masks based on the threshold value
dilation_factor → dilation factor that extends the boundary so that it completely covers the detection target
Note: The range of the blur parameter is 0-15, and an error will be reported if it is exceeded.
Output:
BBOX_DETECTOR → bounding box detection model with specific hints
Note: To use this node, you need to install the ComfyUI-CLIPSeg node extension, the installation method is: open the manager manager, click Install Custom Nodes, in the search box, type CLIPSeg to install.
Example:
caveat
- Text description accuracy: Ensure that the input text description is accurate and clear so that it can be properly understood and processed by the CLIPSeg model.
- Model Parameter Configuration: Adjust the parameters of the CLIPSeg model according to specific needs to get the best segmentation and detection results.
- Processing performance: Image segmentation and detection may require high computational resources, ensure that the system performance is sufficient to support the processing requirements.
By using the CLIPSegDetectorProvider node, image segmentation and detection based on natural language descriptions can be implemented in the image processing workflow to improve the intelligence and accuracy of image processing.
III. Decompose (SEGS) / From_SEG_ELT node
SEGS contains a lot of data after target detection, and these two nodes are convenient for us to take one of the data.
Input:
segs → segs data input
Output:
SEGS_HEADER → header information of segs data, containing meta information about the data such as data source, data version, data generation time, etc.
SEG_ELT → segs separates out the remainder of the header information and contains the main content of the segs data
cropped_image → cropped image
cropped_mask → crop_image_mask
crop_region → description of the region cropped out of the original image, indicating the location and size information of the segmented element in the original image
bbox → detection box
control_net_wrapper → control_net data
confidence → confidence of detection
label → detects the label of the target
Example:
caveat
- Data format consistency: Ensure that the input segmentation data and segmentation elements are formatted consistently so that the nodes can process and transfer the data correctly.
- Extraction Parameter Configuration: Configure the extraction parameters of the From_SEG_ELT node according to the specific needs to obtain the required features and information.
- Processing performance: Decomposition and feature extraction operations may require high computational resources, ensure that the system performance is sufficient to support the processing requirements.
By using Decompose(SEGS) and From_SEG_ELT nodes, fine-grained deconstruction and information extraction of segmented data can be realized in the image processing workflow to enhance the fineness and accuracy of image processing and analysis.
IV. PixelKSampleUpscalerProviderPipe node
This node focuses on image upsampling using the PixelKSample algorithm. With upsampling, an image can be boosted from a lower resolution to a higher resolution while maintaining the quality and detail of the image.
Input:
basic_pipe → Pipes with information about models, cues, etc.
upscale_model_opt → upscale model
pk_hook_opt → hook detail control
Parameters:
scale_mothod → the way to zoom in, with nearest-neighbor interpolation, bilinear interpolation, Lanczos interpolation, and area interpolation
seed → random seed
control_after_generate → control_after_seed_change_after_each_execution
steps → steps
cfg → cue guidance coefficient, i.e., the magnitude of the effect of the cue on the result **too high to have a negative effect**
sampler_name → select sampler
scheduler → Select scheduler
denoise → denoising amplitude **the larger the value, the greater the effect and change on the image**
use_tiled_vae → select whether to use chunking **Chunking produces higher resolution images by dividing the image into multiple chunks and encoding and decoding each chunk separately, which helps to process larger images while being able to retain more details and features **
tile_size → the size of the block used in the image enlargement process **When the image size is large, in order to improve the computational efficiency and reduce the memory consumption, the image can be divided into multiple small blocks to deal with**
Output:
UPSCALER → Image enlargement data processed by the enlargement operation can be further used for subsequent processing of the image or other operations.
caveat
- Up-sampling multiplier: Select the appropriate up-sampling multiplier according to the specific needs to get the best resolution enhancement effect.
- Algorithm Parameter Configuration: Adjust the parameters of the PixelKSample algorithm to ensure that the upsampling effect is as expected.
- Processing Performance: Up-sampling processing may require high computational resources, ensure that system performance is sufficient to support processing requirements.
- Input Image Quality: The effectiveness of upsampling depends on the quality of the input image, ensuring that the input image is as sharp as possible.
By using the PixelKSampleUpscalerProviderPipe node, it is possible to achieve efficient image upsampling in image processing workflows, improve image resolution and detail performance, and meet a variety of image optimization and processing needs.
V. lterative Upscale (Latent/on Pixel Space) Node
This node incrementally increases the image resolution during image processing by means of an iterative upsampling method. This method can be performed in different spaces depending on the setting of configuration parameters.
Input:
samples → latent space images
upscaler → Image enlargement data processed by the enlargement operation.
Parameters:
upscale_factor → upscale_factor, the number represents how many times it is scaled up.
steps → number of repetitions of the zoom operation
temp_prefix → parameter for setting the prefix of the temp file name
Output:
latent → latent space image after processing is complete
vae → vae model
This node and the third node can be used in combination and multiple such combinations can be connected for better zoom. A workflow is given below to facilitate the understanding of these two nodes:
caveat
- Iteration number: choose the appropriate number of iterations according to the specific needs, the more iterations, the more obvious the effect of up-sampling, but the consumption of computing resources will also increase.
- Processing space selection: up-sampling processing in potential space or pixel space is selected according to the specific application scenario. Latent space processing may be more suitable for deep learning models, while pixel space processing is suitable for traditional image processing methods.
- Algorithm Parameter Configuration: adjust the parameters of iterative up-sampling to ensure that the up-sampling effect is as expected.
- Processing performance: Iterative upsampling processing requires high computational resources, ensuring that system performance is sufficient to support processing requirements.
- Input Image Quality: The effectiveness of upsampling depends on the quality of the input image, ensuring that the input image is as sharp as possible.
By using the Iterative Upscale (Latent/on Pixel Space) node, efficient iterative upsampling can be achieved in the image processing workflow to gradually improve the resolution and detail performance of the image to meet various image optimization and processing needs.
VI. Make Tile SEGS nodes
The main function of this node is to split the input segmented image into multiple small chunks. This approach helps to reduce memory consumption while processing large images and allows for finer processing and analysis of each small chunk.
Input:
images → Mask images
filter_in_segs_opt → If this input is provided, the mask for each block will be generated by overlapping with the mask of filter_in_segs_opt when creating the block
filter_out_segs_opt → if this input is provided, the masks of each block will exclude any overlap with the mask of filter_in_segs_opt when creating the block
Note: Tiles with empty masks will not be created as SEGS
Parameters:
bbox_size → size of the block
crop_factor → the crop factor or crop ratio to apply when cropping an image
min_overlap → the smallest overlap area, related to the parameters filter_in_segs_opt and filter_out_segs_opt
filter_segs_dilation → parameter for the dilation operation on images in image processing
mask_irregularity → a metric used in image processing to characterize or measure the degree of irregularity of a mask ** the larger the boundary the more irregular it is**
irregular_mask_mode → method mode for generating irregular masks, options are fast multiplexing, mass multiplexing, fully randomized fast and fully randomized mass
Output:
SEGS → Output SEGS data
Example:
caveat
- Block size: Select the appropriate block size according to the specific needs, to ensure that each block contains enough information, but not too large to lead to a decline in processing efficiency.
- Overlapping sections: Configuring overlapping sections between chunks prevents important information from being lost during the slicing process, especially in edge areas.
- Processing performance: Slicing operations may require high computational resources, ensure that system performance is sufficient to support processing requirements.
- Splice reorganization: If there is a subsequent need to reorganize small pieces of the processed image back into a complete image, take care to record and maintain the positional information of each small piece.
By using the Make Tile SEGS node, it is possible to realize efficient slicing and processing of large-size images in the image processing workflow, and improve the fineness and flexibility of image processing.
VII. Picker (SEGS) nodes
This node focuses on selecting specific segmentation regions or objects from the image segmentation results. By configuring the selection criteria, the user can extract the portion of the segmentation mask that needs to be processed.
Input:
segs → segs containing multiple images
fallback_image_opt → Alternative image, when an alternative image exists the node scales the image in the segs to match the size or shape of the alternative image. Doing so ensures that the image segments are aligned with the alternative image so that appropriate operations can be performed during processing.
Parameters:
picks → Select Image
Output:
SEGS → Output segs data of selected images
The following diagrams illustrate the usage of the Picker (SEGS) node:
caveat
- Configuration of selection conditions: Configure selection conditions according to specific needs to ensure that the selected segmentation area meets expectations.
- Data format consistency: Ensure that the input segmentation data format is compatible with the selection conditions so that the node can process and output the data correctly.
- Processing performance: Selecting an operation may involve a large number of computations, ensure that the system performance is sufficient to support the processing requirements.
By using the Picker (SEGS) node, accurate selection and extraction of segmentation results can be realized in the image processing workflow, improving the efficiency and accuracy of image processing and analysis.
As this paper basically gives a separate workflow for each node to explain, coupled with the correlation between the nodes in this paper is not so strong, so this does not provide a workflow that encompasses all the nodes, you can learn to understand the workflow corresponding to each of the above nodes.
**To go beyond oneself is to strive for excellence. Perseverance is the key to success. **