Kohya LoRA Training Guide: From Dataset to Tested LoRA

Step 1 — Install the Current Kohya_ss Build

Clone the official bmaltais/kohya_ss repository rather than downloading an old bundle from a tutorial. On Windows, the current README documents both uv and pip installation paths and recommends trying uv first. Linux and macOS have separate instructions.

Keep the installation in a short path without unusual characters. After installation, launch the GUI once and confirm it detects your GPU before adding a large model or dataset.

Don't blindly run git pull in the middle of a successful project. Export your configuration, record the commit or release you used, and update between experiments. A dependency or sd-scripts change can alter memory use or defaults.

Step 2 — Choose the Exact Base Model

A LoRA learns an adapter for a specific model architecture. Select the checkpoint you actually plan to use at inference, then confirm its family.

• Match architecture: SD 1.5, SDXL, and Flux LoRAs aren't interchangeable.
• Keep a clean base: Don't switch checkpoints halfway through an experiment and compare results as if only the LoRA changed.
• Check the license: Confirm the base model permits your intended use and distribution before training.
• Record the source: Save the model name, source URL, file hash, and architecture in a project note.

For a general-purpose LoRA, training against the base architecture can improve portability. For a LoRA intended for one specialized checkpoint, train and test against that checkpoint and say so when publishing it.

Step 3 — Prepare Images and Captions

Remove near-duplicates, watermarks you don't want learned, broken images, and examples that contradict the concept. More images aren't automatically better; a smaller coherent set beats a noisy scrape.

Pair each image with a text caption when your dataset method expects captions. Describe what should vary, and use a consistent trigger token for the identity or style you want the LoRA to learn. Don't caption every permanent subject trait so aggressively that the trigger becomes redundant.

Keep a validation set out of training. It helps you detect whether the LoRA learned the concept or merely memorized the training compositions. Our dataset and captioning guide covers selection, caption cleanup, trigger tokens, and duplicate checks in detail.

Step 4 — Build the Dataset Configuration

Point Kohya at the correct image and output directories, then set the architecture-appropriate resolution and bucketing options. Bucketing preserves varied aspect ratios by grouping images into compatible dimensions instead of forcing every source into one crop.

Check repeats, epochs, and total steps together. Ten repeats and ten epochs don't mean “ten passes”; their combined effect depends on image count, batch size, and other settings. Use the step estimate shown by the trainer and write it into your experiment log.

Start without aggressive augmentation. Random flips can be useful for symmetric concepts but wrong for text, asymmetrical clothing, logos, or a subject with a defining side. Caption dropout and shuffle settings also change what the model associates with your trigger.

Step 5 — Set a Conservative LoRA Baseline

Pick a known-good preset for the correct architecture, then change only the fields your project requires. Rank and alpha control adapter capacity and scaling, but a higher rank isn't a free quality upgrade. It increases file size and can make memorization easier.

Use an optimizer and learning rate as a pair. A rate copied from an AdamW run may not translate to Adafactor or another optimizer. Keep batch size, accumulation, scheduler, precision, and trained components in the record so the result can be reproduced.

For SDXL, the Kohya LoRA resource guide recommends 1024×1024, at least 12 GB VRAM, and UNet-only training to avoid unexpected results from the dual text encoders. That's a sound baseline, not a rule that proves every dataset should use identical settings.

Step 6 — Configure Samples and Saves

Create a small prompt file that tests the concept from several angles. Use fixed seeds and include:

• Direct trigger test: A plain prompt that makes the learned concept easy to see.
• Variation test: Different lighting, backgrounds, framing, or poses absent from the closest training image.
• Prompt adherence test: A composition where the trigger must coexist with other requested details.
• Leakage test: A prompt without the trigger to see whether the LoRA changes unrelated output too strongly.

Save intermediate checkpoints often enough to catch the useful point before overtraining. Sample files are evidence; loss alone can't tell you whether identity, style strength, flexibility, and anatomy are improving.

Step 7 — Run a Short Smoke Test

Run a deliberately short training job before committing hours to the full schedule. Confirm that the model loads, captions are found, cache creation completes, samples generate, and checkpoints open in your inference UI.

Read the console output. A run that silently skipped captions or loaded the wrong model type isn't a successful smoke test. Stop immediately if sample images are blank, corrupted, or unrelated to the dataset.

When memory fails, change one pressure point at a time. Lower batch size, verify mixed precision support, use documented caching, or reduce resolution only when the target architecture and quality goal allow it.

Step 8 — Train, Compare, and Stop on Evidence

Start the full run and inspect each saved checkpoint with the same evaluation grid. Compare at several LoRA strengths rather than testing only at weight 1.0; a useful adapter should have a controllable range.

Stop when later checkpoints gain training-image resemblance but lose flexibility. Common overtraining signs include repeated backgrounds, locked poses, baked-in clothing, harsh texture, color drift, and failure to follow new prompts.

If every checkpoint is weak, don't automatically double the steps. First inspect caption loading, trigger consistency, base-model compatibility, learning rate, and whether the dataset actually shows the concept clearly.

Step 9 — Validate Outside the Training UI

Copy a candidate checkpoint into the inference application you actually use. Test it on the exact base model and on any compatible checkpoint you plan to claim.

Build a contact sheet with fixed prompts, seeds, sampler, dimensions, and LoRA weights. Include at least one prompt that differs strongly from the training set. Keep the winning checkpoint, config, prompt file, dataset manifest, and model hash together.

Don't publish a LoRA with vague compatibility. State the base family, tested checkpoint, trigger, recommended weight range, known failures, and whether text or logos in the dataset may appear in output.

Common Kohya Training Mistakes

• Wrong model family: A technically completed run can still be useless if the architecture selection doesn't match the checkpoint.
• Preset worship: Presets are starting points. They can't repair a weak dataset or determine the correct stopping point.
• No checkpoint samples: Waiting for the final file hides overtraining that happened much earlier.
• Changing five variables: You can't learn from a comparison if rank, optimizer, captions, repeats, and model all changed.
• Treating loss as quality: Loss is a training signal, not a complete visual evaluation.
• Skipping provenance: Unlicensed images and unclear base-model terms become distribution problems after the expensive work is done.

Troubleshooting

Out of memory at startup: Close other GPU applications, lower batch size, confirm the correct precision, and use memory options documented for your model family. A larger Windows page file can help specific allocation failures, but it doesn't turn system RAM into fast VRAM.

LoRA has no visible effect: Confirm the trigger appears in captions and prompts, the LoRA is loaded at a useful weight, and the inference checkpoint matches the training architecture. Check early and late checkpoints before retraining.

LoRA copies training images: Reduce total training exposure, improve dataset variety, lower capacity, or stop at an earlier checkpoint. Near-duplicates are a frequent cause.

Output quality collapses: Check learning rate, precision support, architecture selection, corrupted files, and captions. Restore a saved config instead of trying to remember the working fields.

Next Steps

Use the overview pillar to understand the planned workflow, then finish the separate dataset and captioning checklists before your first paid or overnight run. Follow the dedicated Flux requirements or SDXL guide for architecture-specific settings, and use the existing local Stable Diffusion setup article to build a clean inference test environment.

After one baseline run, change only one variable. A disciplined second run teaches you more than five undocumented experiments.