No matter how good your tips and models are, it's rare to get a perfect image all at once.
An indispensable method for fixing small defects is image repair (inpainting). In this article, I'll walk through some basic examples of how to use theImage Restorationto fix the defect.
desired software
We will use the AUTOMATIC1111 Stable Diffusion GUI to create the image.
Basic image restoration settings
In this section, I'll show you step-by-step how to use image repair to fix small defects.
We'll start by creating an image using the tips below:
Positive Cue Words:
masterpiece,best quality,masterpiece,best quality,official art,extremely detailed CG unity 8k wallpaper,a beautiful woman,full body,
Negative Cue Words:
lowers,monochrome,grayscales,skin spots,acnes,skin blemishes,age spot,6 more fingers on one hand,deformity,bad legs,error legs,bad feet,malformed limbs,extra limbs,
We can get the image below:
While this image looks pretty good overall, there are still some issues.
Like the face and hands.
So how do we fix it next?
Select the corresponding model
If you're a frequent visitor to the C site, you can see that for some models there will be a model specifically for redrawing, a Stable Diffusion model trained specifically for image restoration.
You can use it if you want to get the best results. However, it is usually OK to use the same model that generated the image for image restoration.
We download the corresponding model and place it in a folder:
stable-diffusion-webui/models/Stable-diffusion
In AUTOMATIC1111, click the refresh icon next to the checkpoint selection dropdown box in the upper left corner to see the model you just downloaded.
Creating Image Repair Masks
In the AUTOMATIC1111 GUI, select theimg2imgtab and selectInpaintSubtab. Uploads the image to the Image Repair Canvas.
Or select send img to inpaint in the txt2img tag.
We will fix both the hands and the face. Use the brush tool to create ashieldsThis is the area where you want Stable Diffusion to recreate the image. This is the area where you want Stable Diffusion to regenerate the image.
Settings for image restoration
image size
The image needs to be resized to be the same as the original image. (768 x 512 in this case).
Facial Restoration
If you are restoring faces, you can open restore faces. select the corresponding face restoration model:CodeFormer.
Some of you may ask, why there is no restore faces option on the top of my page?
If you don't have this option, you need to go to user interface inside settings and add the following two settings:
Note that this option may generate an unnatural appearance. It may also generate content that is inconsistent with the style of the model.
Mask content
The next important setting isMasked Content。
If you want the results to be guided by the colors and shapes of the original content, select theoriginal。
original is commonly used for facial image restorationBecause the general shape and anatomy is correct. We just want it to look a little different.
In most cases, you will use theoriginalfurthermoredenoising intensityto achieve different effects.
If you want to completely recreate something from the original image, such as removing a limb or hiding a hand, you can use thelatent noisemaybelatent nothing。
These options initialize the mask area with something different than the original image. It will produce something completely different.
denoising intensity
denoising intensityControls how much change will be made compared to the original image. When you set it to 0, nothing will change. When you set it to 1, you get an irrelevant image. 0.75 is usually a good starting point. If you want less change, lower it.
batch size
Make sure to generate a few images at a time so you can choose the best ones. Place thetorrentSet to -1 so that each image is different.
Image Restoration Results
Here are some of the restored images.
As you can see the fourth one is still good, but not perfect. So we might consider another round of fixes.
Another round of image restoration
Post the last image generated above back into inpait to fix it again.
We can get the following result:
Image restoration is an iterative process. You can apply it as many times as necessary to refine the image.
If it doesn't work once, we might consider doing it a few more times.
Adding a new object
Sometimes you may want to add something new to an image.
Let's try to add a sword to the image.
First, upload the image to the Image Repair canvas and add a mask to the hand location.
Add "holding a sword" to the beginning of the original prompt. The prompt for image restoration is
(holding a sword:1.5),masterpiece,best quality,masterpiece,best quality,official art,extremely detailed CG unity 8k wallpaper,a beautiful woman,full body,
Adding new objects to the original tip ensures a consistent style. You can adjust the keyword weights (1.5 above) to make the sword display.
commander-in-chief (military)Mask contentset topotential noise。
aligndenoising intensitycap (a poem)CFG ratioto fine-tune the restored image.
After some experimentation, our mission was accomplished:
Explanation of image restoration parameters
denoising intensity
Denoising Intensity controls the similarity between the final image and the original content. Setting it to 0 changes nothing. Set it to 1 and you get an irrelevant image.
If you want small changes, set it to a low value; if you want large changes, set it to a high value.
CFG scale
Similar to its use in text-to-image, theCFG scaleis a parameter that controls the correlation between the model and your cue word.
1 - Largely ignore your prompts.
3 - Be more creative.
7 - Strike a good balance between following tips and freedom.
15 - Follow tips more often.
30 - Strictly follow the tips.
Mask content
Mask content controls how the mask area is initialized.
fill: Initialize with a highly blurred version of the original image.
Original: Unmodified.
latent noise: The masked area is covered with thepaddingInitialize and add random noise to the potential space.
latent nothing: Like potential noise, only without the added noise in the potential space.
Tips for image restoration
Successful image restoration requires patience and skill. Here are some pointers for using image restoration:
- Repair one small area at a time.
- Try differentMask contentto see which one works best.
- Multiple attempts can be made to fix it.
- If nothing works in the AUTOMATIC1111 setup, use an image editing software like Photoshop or GIMP to draw the area of interest in the approximate shape and color you want. Upload that image and perform image restoration with the original content.
Click on me for more highlights.