LocalForge AILocalForge AI
LibraryBlogFAQ

Inpainting Stable Diffusion locally (uncensored stacks, 2026)

Inpainting is where the magic happens for me. You generate something 90% right, then fix that one hand or swap that background - and suddenly it's exactly what you imagined. No rerolling the entire image. No praying the next seed nails it. Just mask the problem, dial in denoise, and paint over exactly the part that bugs you.

This guide covers local uncensored inpainting in Forge and ComfyUI on your own hardware. "Uncensored" just means no cloud moderation layer - you still need proper masks, a matched VAE, and controlled denoise to get clean results. I'll walk you through the full workflow: preparing your source image, painting masks, setting denoise for different edit types, configuring the inpaint pipeline, generating, and refining until the edit disappears into the surrounding image. VRAM requirement is roughly the same as your base model plus 10-15% overhead for the mask encoding pass.

The Quick Answer

Key Takeaway - May 2026

Local inpainting = mask the region + VAE encode + denoise inside the mask. In Forge, use the dedicated Inpaint tab with "Only masked" selected. In ComfyUI, wire a VAE Encode (for Inpainting) node for inpainting-trained models or a Set Latent Noise Mask node for standard checkpoints. Keep denoise in the right band for your edit type:

  • Face fixes: 0.4-0.6 (preserve identity)
  • Clothing swaps: 0.6-0.8 (enough freedom for new fabric)
  • Background replacement: 0.7-0.9 (near-full regeneration is fine)

Or use LocalForge AI for Forge pre-configured with zero setup alongside DIY installs.

What You Need

  • GPU: NVIDIA with 6+ GB VRAM (RTX 2060 or better). 8 GB handles SD 1.5 inpainting comfortably; 12 GB for SDXL inpainting models.
  • Frontend: Forge (recommended for most users) or ComfyUI (node-level control).
  • Checkpoint: Either an inpainting-specific model (sd-v1-5-inpainting.ckpt, Realistic Vision V5.1 Inpainting) or your normal checkpoint with the Set Latent Noise Mask approach.
  • VAE: Matched to your checkpoint. Mismatched VAE is the #1 cause of color shift after inpainting.
  • Source image: PNG format, native resolution. Don't upscale before inpainting - match your generation resolution.

Step 1 - Prepare Your Source Image

Start with a PNG at native generation resolution (512x512 for SD 1.5, 1024x1024 for SDXL). JPEG compression introduces artifacts that bleed into inpainted regions. If you're fixing a previously generated image, use the original output file - not a screenshot or resized copy.

Keep the working file in one color profile end-to-end. Round-tripping through external editors with different color spaces causes hue drift that looks like a VAE problem but isn't.

Step 2 - Paint Your Mask

This is where most people go wrong. The mask tells the model exactly which pixels to regenerate.

  • Mask blur/feather: 8-12 px at 512px resolution, 16-24 px at 1024px. Too low = hard sticker edges. Too high = bleeding into areas you wanted untouched.
  • Tight masks for detail work: Face fixes, jewelry, eyes. Paint just the problem area.
  • Generous masks for replacement: Clothing swaps, backgrounds. Include some surrounding context so the model sees color temperature transitions.
  • Hair and fur: Need wider feather than skin because strands disappear under hard mask edges. Paint conservatively, then expand feather until flyaways blend without halo.

Forge: Use the built-in mask painter on the Inpaint tab. Keep mask preview enabled while you paint so you can see the red overlay in real time.

ComfyUI: Load your mask as a separate image or paint it in an external editor. Feed it into the mask input of your inpaint node. Name your mask files descriptively - mask_face_v02_feather12.png beats final2.png.

Step 3 - Set Denoise Strength

Denoise controls how much the masked region changes. Too low and nothing happens. Too high and you lose the original context entirely.

  • 0.3-0.5: Subtle corrections. Skin smoothing, minor color fixes, small texture adjustments.
  • 0.4-0.6: Face fixes. Enough to correct structure while preserving identity. Start at 0.45, bump by 0.05 increments.
  • 0.6-0.8: Clothing and outfit changes. The model needs freedom to generate new fabric patterns and folds.
  • 0.7-0.9: Background replacement. Near-full regeneration with just enough context from surrounding pixels to match lighting.

Lock your seed while tuning denoise. Change one variable at a time. If textures crawl or identity drifts, you're too high for your mask size - either lower denoise or widen the mask.

Step 4 - Configure Inpaint Settings

Forge path:

  1. Load your image in the Inpaint tab (under img2img).
  2. Paint your mask directly on the canvas.
  3. Set Inpaint area to "Only masked" - this regenerates just the masked region at full resolution.
  4. Set Masked content to "Original" (preserves surrounding context for the model to reference).
  5. Match your resolution to the source image. Don't upscale inside inpaint.

ComfyUI path (inpainting model):

  1. Load your image and mask separately.
  2. Connect both to a VAE Encode (for Inpainting) node. This fills masked pixels with gray (0.5, 0.5, 0.5) before encoding - required by inpainting checkpoints.
  3. Set grow_mask_by to 6-12 pixels for edge blending.
  4. Run the KSampler at denoise 1.0 (the inpainting model handles the rest internally).

ComfyUI path (standard model):

  1. Encode your image normally with VAE Encode.
  2. Add a Set Latent Noise Mask node between the latent output and your KSampler.
  3. Feed your mask into the noise mask input.
  4. Set denoise on the KSampler to your target range (0.4-0.8 depending on edit type).

Label your nodes clearly - mask_in, vae_encode, latent_out. When graphs fork into multiple paths, disable unused branches cleanly.

Step 5 - Generate

Run your first pass at slightly lower denoise than you think you need. It's easier to bump up than to recover from an over-cooked generation that destroyed context.

Preview at lower resolution first if you're working on large images. A 512px test pass takes 3-5 seconds versus 15-20 seconds at 1024px. Once your mask and denoise feel right at preview scale, commit to the full-res render.

Save the output alongside your mask file. Keep both so you can iterate without losing your reference point.

Step 6 - Refine with Multiple Passes

One pass rarely gives you a perfect result. The two-pass approach works reliably:

  • Pass 1: Wide soft mask, moderate denoise (0.5-0.6). Fix gross structure - wrong anatomy, bad lighting, missing features.
  • Pass 2: Tight mask on fine detail only (eyes, lips, fabric pattern), lower denoise (0.3-0.45). Fix micro-detail without disturbing the structure you just fixed.

Screenshot between passes. If pass 2 goes wrong, you can revert to your pass 1 output without starting over.

Verify It Works

Your inpaint is clean when:

  • No visible seam at mask boundary (zoom to 200% and check)
  • Color temperature matches surrounding pixels - no warm patch on cool skin or vice versa
  • Texture grain is consistent across masked and unmasked regions
  • Lighting direction inside the mask matches the rest of the image

If any of these fail, check feather width and denoise before touching prompts or samplers.

Troubleshooting

Color shift after inpainting: The masked region has different hues than surrounding areas. Almost always a VAE issue. Verify you're using the same VAE that generated the original image. If you switched checkpoints, the baked-in VAE may differ. Fix: load an explicit VAE that matches your original generation, or re-encode the full image through the same VAE before inpainting.

Hard edges (sticker effect): The inpainted region looks pasted on. Your mask feather is too low. Increase mask blur from 4 to 12-16 px (at 512 res) or 16-24 px (at 1024 res). Also check that "Inpaint area" is set to "Only masked" in Forge - "Whole picture" can cause edge mismatches at lower denoise.

Identity loss (face doesn't match): Denoise is too high for a face edit. Drop to 0.4-0.5 and use a tighter mask. If you're using an inpainting-specific model at denoise 1.0, try switching to a standard model with Set Latent Noise Mask at 0.45-0.55 instead - it preserves more of the original features.

Bottom Line

Inpainting is cosmetic surgery for your generations: small masks, controlled denoise, matched VAE. Start conservative, iterate in passes, and check your edges at 200% zoom. The technique rewards patience over brute-force prompting - and once you get the feel for mask + denoise together, you'll fix images in 30 seconds that used to cost 20 rerolls.

What to Do Next

Set up ControlNet for anatomy-accurate inpainting

Build a full ComfyUI inpainting workflow with Flux

Try Pony Diffusion for stylized inpainting

FAQ

What denoise strength should I use for face fixes? +
Start at 0.45 and adjust by 0.05 increments. The 0.4-0.6 range preserves identity while fixing structure. Go higher and you risk generating a different face entirely.
Why does my inpainted area have different colors than the rest? +
VAE mismatch. The masked region re-encodes through VAE differently than untouched pixels. Fix: use the same explicit VAE that generated the original image, not a different checkpoint's baked-in VAE.
Should I use an inpainting-specific checkpoint or a regular model? +
Inpainting checkpoints (sd-v1-5-inpainting, Realistic Vision Inpainting) work at denoise 1.0 with VAE Encode for Inpainting. Standard models use Set Latent Noise Mask with lower denoise (0.4-0.8). Standard models give finer control; inpainting models handle large replacements better.
How much VRAM does inpainting need? +
Same as your base model plus roughly 10-15% overhead for the mask encoding pass. SD 1.5 inpainting runs fine on 6 GB VRAM. SDXL inpainting needs 10-12 GB comfortably.
Does uncensored inpainting mean anything different technically? +
No. 'Uncensored' means no cloud-side content filter blocking your generations. The inpainting pipeline itself is identical - mask, encode, denoise, decode. You still follow applicable laws for anything you distribute.
Why do hair and fur edges look bad after inpainting? +
Hair strands are high-frequency detail that disappears under hard masks. Use wider feather (20-24 px at 1024 res) and paint the mask conservatively first, then expand until flyaways blend without creating a halo.