LoRA Dataset Folder Structure

Sidecar Caption Structure

Many LoRA workflows use a text file beside each image. The stem must match exactly:

subject_001.png
subject_001.txt
subject_002.jpg
subject_002.txt

The trainer determines the caption extension. kohya-ss sd-scripts exposes a caption_extension setting, so .txt is common but should not be assumed when a configuration says otherwise.

Validate these rules:

• Every approved image has one expected sidecar.
• Every sidecar has one image with the same stem.
• No sidecar is blank unless blank captions are deliberate.
• Extensions and case match the configuration.
• Text encoding is UTF-8.
• Temporary files such as subject_001 - Copy.txt are absent.

A missing caption can cause a loader error, fall back to a filename, or silently train without intended text, depending on the tool. Treat it as a hard preflight failure.

ImageFolder With metadata.jsonl

Hugging Face Datasets supports local image datasets through the ImageFolder builder. Captioned training examples can be described in metadata.jsonl, with one JSON object per line mapping an image path to text.

A simplified file can look like:

{"file_name": "subject_001.png", "text": "tokperson, close portrait, outdoor light"}
{"file_name": "subject_002.png", "text": "tokperson, full body, black jacket, city street"}

The field names must match what the loading and training script expects. JSON Lines is not a JSON array. Each line must be valid JSON on its own, paths must resolve from the documented location, and duplicate records should be rejected.

Do not keep a stale metadata.jsonl after renaming images. Generate or validate the metadata from the final directory state.

Repeat Folders in kohya-Style Workflows

Some kohya training workflows use a subfolder name such as:

train/
  10_tokperson person/
    subject_001.png
    subject_001.txt

In this convention, the leading number can represent repeats and the rest of the folder name can carry class or identifier information, depending on the script and mode. This is a loader convention, not a universal LoRA standard.

Newer or more explicit workflows can define datasets and subsets in a TOML configuration. Explicit configuration is easier to review when you have multiple concepts, regularization data, distinct repeat values, or several roots.

Never assume a folder prefix is active just because a tutorial used it. Confirm which script parses it. If the trainer uses a dataset configuration, the visible folder number may do nothing or conflict with explicit settings.

One Concept Per Explicit Subset

If a project has multiple source groups, represent them as named concepts or subsets in the trainer configuration instead of mixing everything in one directory.

Useful separations include:

• Subject images versus regularization images.
• High-confidence core images versus a lower-repeat supplemental set.
• Different caption formats required for controlled experiments.
• Distinct concepts that receive different repeats.

Do not split folders merely by camera angle if every subset receives identical settings. Excess hierarchy can make pairing and auditing harder. A directory boundary should correspond to a real loader or review decision.

Keep Regularization Data Separate

When a workflow uses class or regularization images, place them in a separate root and map that root explicitly. They are not captions, validation references, or ordinary concept examples.

A clear layout is:

dataset/
  concept/
  regularization/

Confirm whether the selected method actually uses regularization images. Do not copy an empty folder or thousands of unrelated class images into every project by habit.

Avoid Recursive Scan Surprises

Trainers differ in whether they read only direct files or recurse through subdirectories. Before starting, determine:

• Which root is scanned.
• Whether scanning is recursive.
• Which image extensions are accepted.
• Whether hidden files are ignored.
• Whether captions are loaded from sidecars or metadata.
• Whether multiple files with the same stem are ambiguous.

Operating systems can add files such as Thumbs.db, .DS_Store, or cloud synchronization placeholders. These should not be treated as images, but a clean training root makes loader behavior easier to understand.

Keep archives outside the scan root. A ZIP file is not a backup if the trainer or a preprocessing script unexpectedly traverses or extracts it into place.

Add a Dataset Manifest

A manifest is a review aid, even when the trainer does not consume it. Store it outside the active training folder unless the loader safely ignores it.

Record:

• Stable image filename.
• Caption or metadata record presence.
• Width and height.
• File hash.
• Approval state.
• Source or rights note.
• Preprocessing note.

The manifest lets you prove which exact files went into a run. If a later cleanup changes the dataset, a hash comparison shows the change instead of relying on memory.

Windows Path and Sync Cautions

Long nested paths can still cause trouble in some Windows tools, Python environments, and archive utilities. Keep the project root reasonably short. Avoid characters that need special escaping in TOML, JSON, command lines, or shell scripts.

Cloud-synced folders can expose placeholder files before content is downloaded. Ensure every training file is local and readable. Pause synchronization during mass renames if the service creates conflict copies.

Use relative paths in portable configuration when the trainer supports them. If absolute paths are required, document them as machine-specific and do not assume another computer can reuse the configuration unchanged.

Preflight the Folder Contract

Run these checks before opening the trainer:

1. List every image the configured scan should find.
2. Confirm each image has exactly one caption or metadata record.
3. Reject orphan captions and orphan metadata rows.
4. Reject duplicate stems, including case-only collisions.
5. Confirm all paths resolve and all files can be read.
6. Confirm the loader recognizes every intended extension.
7. Verify repeat values from the actual configuration.
8. Confirm validation and output files are outside the scan root.
9. Save a manifest or file listing with the run configuration.

Then use the trainer's dataset preview or cache step. A successful preview is stronger evidence than a tidy-looking directory.

Common Structure Failures

Images and captions have different stems

subject-01.png does not pair with subject_01.txt. Rename deliberately and rerun the pairing check.

Rejected images remain in a subfolder

If scanning is recursive, a folder called rejects inside the dataset may still train. Move it outside the root.

A repeat prefix is treated as universal

Folder naming conventions are tool-specific. Read the selected loader's behavior.

Metadata points to old filenames

Renaming files without regenerating metadata.jsonl creates missing records or wrong mappings.

Outputs are written under the dataset root

Previews and checkpoints can pollute later scans and backups. Give them a separate destination.

Bottom Line

A strong folder structure is a verifiable contract between your files and one trainer. Separate originals, working assets, final data, validation material, configuration, and outputs. Use stable names and exact pairings.

Do not optimize the tree for appearance. Optimize it so you can state exactly which files, captions, repeats, and metadata records the loader will use.