Inpainting Stable Diffusion locally (uncensored stacks, 2026)
Inpainting is where the magic happens for me. You generate something 90% right, then fix that one hand or swap that background - and suddenly it's exactly what you imagined. No rerolling the entire image. No praying the next seed nails it. Just mask the problem, dial in denoise, and paint over exactly the part that bugs you.
This guide covers local uncensored inpainting in Forge and ComfyUI on your own hardware. "Uncensored" just means no cloud moderation layer - you still need proper masks, a matched VAE, and controlled denoise to get clean results. I'll walk you through the full workflow: preparing your source image, painting masks, setting denoise for different edit types, configuring the inpaint pipeline, generating, and refining until the edit disappears into the surrounding image. VRAM requirement is roughly the same as your base model plus 10-15% overhead for the mask encoding pass.
The Quick Answer
Key Takeaway - May 2026
Local inpainting = mask the region + VAE encode + denoise inside the mask. In Forge, use the dedicated Inpaint tab with "Only masked" selected. In ComfyUI, wire a VAE Encode (for Inpainting) node for inpainting-trained models or a Set Latent Noise Mask node for standard checkpoints. Keep denoise in the right band for your edit type:
- Face fixes: 0.4-0.6 (preserve identity)
- Clothing swaps: 0.6-0.8 (enough freedom for new fabric)
- Background replacement: 0.7-0.9 (near-full regeneration is fine)
Not sure which checkpoint to load? See "Which inpainting model for NSFW edits" below.
LocalForge AI is our own product: Forge pre-configured with zero setup, for anyone who would rather not install it by hand. DIY installs of Forge or ComfyUI follow the same steps in this guide.
What You Need
- GPU: NVIDIA with 6+ GB VRAM (RTX 2060 or better). 8 GB handles SD 1.5 inpainting comfortably; 12 GB for SDXL inpainting models.
- Frontend: Forge (recommended for most users) or ComfyUI (node-level control).
- Checkpoint: Either an inpainting-specific model (sd-v1-5-inpainting.ckpt, Realistic Vision V5.1 Inpainting) or your normal checkpoint with the Set Latent Noise Mask approach.
- VAE: Matched to your checkpoint. Mismatched VAE is the #1 cause of color shift after inpainting.
- Source image: PNG format, native resolution. Don't upscale before inpainting - match your generation resolution.
Which inpainting model for NSFW edits
For SD 1.5 work, use an inpainting-trained checkpoint with VAE Encode (for Inpainting); for SDXL or Pony work, use an SDXL inpainting checkpoint; for any other checkpoint, use Set Latent Noise Mask at a lower denoise (0.4-0.8, as in Step 4). The inpainting pipeline is the same whatever the content, so the choice comes down to the base model your image was made on and how much of the original you want to keep.
| Model | Base | What it is good for | How to wire it |
|---|---|---|---|
stable-diffusion-v1-5/stable-diffusion-inpainting |
SD 1.5 | The standard inpainting-trained SD 1.5 checkpoint. Use it for edits on 512px SD 1.5 images. | VAE Encode (for Inpainting), KSampler at denoise 1.0 |
diffusers/stable-diffusion-xl-1.0-inpainting-0.1 (Stable Diffusion XL inpainting) |
SDXL | The inpainting-trained SDXL checkpoint. Use it for edits on 1024px SDXL images. | Inpaint node as in Step 4, denoise 1.0 |
| LUSTIFY! [SDXL NSFW] inpainting | SDXL | An SDXL inpainting model built for NSFW work, for SDXL images that need an NSFW-capable inpaint checkpoint. | Same as the SDXL row |
| Inpaint_SDXL_Pony | Pony | An inpainting checkpoint for Pony-style SDXL images. Its listing is marked for Automatic1111 / WebForge, so run it in Forge. | Inpaint tab in Forge |
| Your regular NSFW checkpoint | SD 1.5, SDXL or Pony | Any checkpoint you already use. Keeps more of the original features, which is why it suits face fixes. | Set Latent Noise Mask, denoise 0.4-0.8 |
How to choose: match the model to the base of the image you are editing. An SD 1.5 image goes through an SD 1.5 inpainting checkpoint, and an SDXL or Pony image goes through an SDXL one. If you would rather keep your favorite checkpoint's look, skip the dedicated model and use the noise-mask route with a lower denoise. If the face stops matching at denoise 1.0 on an inpainting model, switch to the noise-mask route (see Troubleshooting).
Looking for which base checkpoints to start from? See our ranked NSFW checkpoints for ComfyUI, or the ranked NSFW checkpoints for Stable Diffusion in general.
Use adult or fictional subjects only, and follow the laws that apply where you live.
Step 1 - Prepare Your Source Image
Start with a PNG at native generation resolution (512x512 for SD 1.5, 1024x1024 for SDXL). JPEG compression introduces artifacts that bleed into inpainted regions. If you're fixing a previously generated image, use the original output file - not a screenshot or resized copy.
Keep the working file in one color profile end-to-end. Round-tripping through external editors with different color spaces causes hue drift that looks like a VAE problem but isn't.
Step 2 - Paint Your Mask
This is where most people go wrong. The mask tells the model exactly which pixels to regenerate.
- Mask blur/feather: 8-12 px at 512px resolution, 16-24 px at 1024px. Too low = hard sticker edges. Too high = bleeding into areas you wanted untouched.
- Tight masks for detail work: Face fixes, jewelry, eyes. Paint just the problem area.
- Generous masks for replacement: Clothing swaps, backgrounds. Include some surrounding context so the model sees color temperature transitions.
- Hair and fur: Need wider feather than skin because strands disappear under hard mask edges. Paint conservatively, then expand feather until flyaways blend without halo.
Forge: Use the built-in mask painter on the Inpaint tab. Keep mask preview enabled while you paint so you can see the red overlay in real time.
ComfyUI: Load your mask as a separate image or paint it in an external editor. Feed it into the mask input of your inpaint node. Name your mask files descriptively - mask_face_v02_feather12.png beats final2.png.
Step 3 - Set Denoise Strength
Denoise controls how much the masked region changes. Too low and nothing happens. Too high and you lose the original context entirely.
- 0.3-0.5: Subtle corrections. Skin smoothing, minor color fixes, small texture adjustments.
- 0.4-0.6: Face fixes. Enough to correct structure while preserving identity. Start at 0.45, bump by 0.05 increments.
- 0.6-0.8: Clothing and outfit changes. The model needs freedom to generate new fabric patterns and folds.
- 0.7-0.9: Background replacement. Near-full regeneration with just enough context from surrounding pixels to match lighting.
Lock your seed while tuning denoise. Change one variable at a time. If textures crawl or identity drifts, you're too high for your mask size - either lower denoise or widen the mask.
Step 4 - Configure Inpaint Settings
Forge path:
- Load your image in the Inpaint tab (under img2img).
- Paint your mask directly on the canvas.
- Set Inpaint area to "Only masked" - this regenerates just the masked region at full resolution.
- Set Masked content to "Original" (preserves surrounding context for the model to reference).
- Match your resolution to the source image. Don't upscale inside inpaint.
ComfyUI path (inpainting model):
- Load your image and mask separately.
- Connect both to a VAE Encode (for Inpainting) node. This fills masked pixels with gray (0.5, 0.5, 0.5) before encoding - required by inpainting checkpoints.
- Set
grow_mask_byto 6-12 pixels for edge blending. - Run the KSampler at denoise 1.0 (the inpainting model handles the rest internally).
ComfyUI path (standard model):
- Encode your image normally with VAE Encode.
- Add a Set Latent Noise Mask node between the latent output and your KSampler.
- Feed your mask into the noise mask input.
- Set denoise on the KSampler to your target range (0.4-0.8 depending on edit type).
Label your nodes clearly - mask_in, vae_encode, latent_out. When graphs fork into multiple paths, disable unused branches cleanly.
Step 5 - Generate
Run your first pass at slightly lower denoise than you think you need. It's easier to bump up than to recover from an over-cooked generation that destroyed context.
Preview at lower resolution first if you're working on large images. A 512px test pass takes 3-5 seconds versus 15-20 seconds at 1024px. Once your mask and denoise feel right at preview scale, commit to the full-res render.
Save the output alongside your mask file. Keep both so you can iterate without losing your reference point.
Step 6 - Refine with Multiple Passes
One pass rarely gives you a perfect result. The two-pass approach works reliably:
- Pass 1: Wide soft mask, moderate denoise (0.5-0.6). Fix gross structure - wrong anatomy, bad lighting, missing features.
- Pass 2: Tight mask on fine detail only (eyes, lips, fabric pattern), lower denoise (0.3-0.45). Fix micro-detail without disturbing the structure you just fixed.
Screenshot between passes. If pass 2 goes wrong, you can revert to your pass 1 output without starting over.
Write down the settings for each pass so any result can be repeated, and so you can change one thing at a time between passes:
- Checkpoint: the exact model file, matched to the base of your image (see the model table above).
- Sampler and steps: the same for both passes unless you are testing them.
- CFG scale: keep it the same across both passes so only the mask and denoise change.
- Seed: locked, as in Step 3.
- Denoise: pass 1 in the 0.5-0.6 band, pass 2 in the 0.3-0.45 band.
- Mask blur and resolution: the Step 2 values for your resolution; the wide mask in pass 1, the tight mask in pass 2.
- Positive prompt: describe what the masked area should contain, not the whole image.
- Negative prompt: list the flaws you are fixing, such as extra fingers or blurry skin, and reuse it unchanged in pass 2.
Verify It Works
Your inpaint is clean when:
- No visible seam at mask boundary (zoom to 200% and check)
- Color temperature matches surrounding pixels - no warm patch on cool skin or vice versa
- Texture grain is consistent across masked and unmasked regions
- Lighting direction inside the mask matches the rest of the image
If any of these fail, check feather width and denoise before touching prompts or samplers.
Troubleshooting
Color shift after inpainting: The masked region has different hues than surrounding areas. Almost always a VAE issue. Verify you're using the same VAE that generated the original image. If you switched checkpoints, the baked-in VAE may differ. Fix: load an explicit VAE that matches your original generation, or re-encode the full image through the same VAE before inpainting.
Hard edges (sticker effect): The inpainted region looks pasted on. Your mask feather is too low. Increase mask blur from 4 to 12-16 px (at 512 res) or 16-24 px (at 1024 res). Also check that "Inpaint area" is set to "Only masked" in Forge - "Whole picture" can cause edge mismatches at lower denoise.
Identity loss (face doesn't match): Denoise is too high for a face edit. Drop to 0.4-0.5 and use a tighter mask. If you're using an inpainting-specific model at denoise 1.0, try switching to a standard model with Set Latent Noise Mask at 0.45-0.55 instead - it preserves more of the original features.
Bottom Line
Inpainting is cosmetic surgery for your generations: small masks, controlled denoise, matched VAE. Start conservative, iterate in passes, and check your edges at 200% zoom. The technique rewards patience over brute-force prompting - and once you get the feel for mask + denoise together, you'll fix images in 30 seconds that used to cost 20 rerolls.
