LoRA Caption Format Explained

Metadata JSON Lines

Hugging Face Datasets ImageFolder documents metadata.jsonl for attaching captions and other metadata to local images. JSON Lines means each line is a complete JSON object. It is not one large JSON array.

Example:

{"file_name": "0001.png", "text": "A side-view photo of zvpack backpack on a desk."}

{"file_name": "0002.png", "text": "A front-view photo of zvpack backpack being worn outdoors."}

The documented file_name value links the row to the image. Paths are relative to the metadata file's dataset context. The text field becomes a dataset column; the training script must select the correct caption column.

JSONL rules that matter

• one valid JSON object per physical line;
• double quotes around JSON keys and string values;
• proper escaping for quotes, backslashes, and control characters;
• a filename entry that resolves to an actual image;
• no trailing comma between lines;
• the exact metadata filename expected by the loader;
• a caption column name accepted or configured by the script.

A file that begins with [ and ends with ] is a JSON array, not JSON Lines. A human may read it easily while the loader rejects it.

Tag Caption Syntax

Tag captions use separate descriptors, commonly divided by commas:

ohwxperson, woman, red sweater, sitting, side view, indoors

Each item should describe one useful concept or a deliberate phrase. Multiword tags such as side view remain one comma-delimited item.

Tag strengths:

• compact;
• easy to search and normalize;
• compatible with tag-oriented base models;
• suitable for item shuffling;
• easy to preserve a fixed leading trigger item.

Tag limitations:

• weak expression of relationships;
• ambiguity about which subject owns an attribute;
• vocabulary can become noisy;
• delimiter choices affect processing.

Do not add commas inside a tag unless the loader supports quoting or an alternate separator. Most simple caption processors treat each comma as a boundary.

Natural-Language Caption Syntax

Natural-language captions use factual sentences:

A side-view photo of ohwxperson wearing a red sweater while sitting indoors.

Sentence strengths:

• clear relationships;
• readable editing;
• useful for models trained on descriptive language;
• flexible for scenes and products outside a fixed tag vocabulary.

Sentence limitations:

• inconsistent phrasing is harder to normalize;
• filler can consume prompt length;
• ordinary comma shuffling breaks grammar;
• captioners may invent plausible details.

Punctuation belongs to the sentence. Do not configure a comma-based shuffler merely because the sentence contains commas.

Hybrid Formats

A hybrid combines a stable structured prefix with descriptive text:

zvpack backpack, a rear view with the main compartment open on a table

Hybrid text can be useful, but only when the trainer's parser and base model handle it as intended. It can also create ambiguity: is the second section one item, several comma items, or prose that will be shuffled?

If using a hybrid:

• define the boundary explicitly;
• use separators documented by the trainer;
• decide which section can shuffle;
• inspect the processed caption if the tool exposes it;
• apply one pattern across the dataset.

Randomly mixing tags, sentences, and filename fragments is not a hybrid strategy.

Caption Separators

sd-scripts documents caption processing options including a caption separator, a secondary separator, and a keep-token separator. These controls determine how text is divided for shuffling and dropout.

The default mental model is comma-delimited items:

trigger, class, clothing, pose, background

But configuration can change the delimiter or identify a fixed prefix. Use only syntax supported by the current script version. Decorative characters copied from another user's preset can become literal training text when the option is missing or misconfigured.

Test separators on a tiny dataset and inspect logs or cached metadata where possible.

Shuffling and Fixed Items

shuffle_caption in sd-scripts randomizes separated caption items. It helps reduce dependence on one fixed tag order.

keep_tokens preserves a chosen number of leading items. Despite the name, think carefully about the parser's separated items, not only tokenizer subwords.

Example:

ohwxperson, woman, red coat, outdoors

With a comma separator and one kept item, ohwxperson remains first. woman does not remain fixed.

If the intended prefix is:

ohwxperson woman, red coat, outdoors

then ohwxperson woman is one comma-delimited item, though the text encoder may tokenize it into multiple pieces.

Do not use tag shuffling for:

• complete sentences;
• clauses whose order defines relationships;
• product descriptions where modifiers must remain attached;
• untested hybrid syntax.

Prefixes and Suffixes

Dataset and caption tools can add text before or after stored captions. This is useful for applying a trigger or class phrase, but it creates hidden input if you inspect only the sidecars.

OneTrainer's caption-tool documentation notes that prefix and postfix text are added directly, so spacing and punctuation must be supplied correctly.

Check for:

• zvpackopen backpack from a missing separator;
• doubled commas;
• duplicate triggers already present in files;
• a suffix attached after terminal punctuation;
• a UI preview that differs from the training loader.

Prefer storing final captions visibly unless dynamic insertion has a clear operational benefit.

Caption Dropout Is Processing, Not Storage

Caption dropout can remove the complete caption on selected examples. Tag dropout can remove separated items. The exact names and behavior depend on the trainer.

These settings do not change the source file. They change conditioning seen during training. That makes debugging difficult if the configuration is not recorded.

A dropped variable tag can push its visual detail toward the trigger association. Start with clean, readable captions and no clever dropout. Add dropout only for a tested reason, preserve required prefix items, and compare fixed validation prompts.

Character Encoding and Line Breaks

Use UTF-8 unless the trainer documents another encoding. Avoid smart punctuation that your pipeline cannot reproduce consistently. A byte-order mark may be tolerated by some loaders and exposed as text by others.

For simple sidecars, use one logical caption per file. Unexpected line breaks can be joined, truncated, or interpreted differently across scripts.

Check for:

• empty first lines;
• invisible control characters;
• nonbreaking spaces;
• copied tabs;
• mixed newline conventions;
• encoding replacement characters.

Plain text reduces operational uncertainty.

Trigger Placement and Tokenization

The typed trigger belongs in the caption syntax, but the text encoder decides how it is tokenized. A made-up string may split into several tokenizer pieces. That does not alter the sidecar pairing rules or the comma-item count.

Keep these layers separate:

• filename stem pairs the caption with the image;
• delimiters divide caption items;
• keep_tokens preserves leading parsed items;
• the tokenizer converts final text to model inputs;
• the text encoder has a finite usable sequence length.

Put important concept text early and keep it consistent. Do not claim a trigger is one true token just because it has no spaces.

Format Examples by Workflow

sd-scripts tag sidecar

Image:

train/014.png

Caption:

train/014.txt

Content:

zvpack backpack, open, top view, wooden desk

Relevant configuration may specify .txt, caption shuffling, and the number of fixed leading items.

Sentence sidecar

Image:

train/014.png

Caption content:

A top-view photo of zvpack backpack open on a wooden desk.

Disable comma-based shuffling for sentence semantics.

ImageFolder metadata

File:

train/metadata.jsonl

Row:

{"file_name": "014.png", "text": "A top-view photo of zvpack backpack open on a wooden desk."}

Confirm the script uses text as its caption column.

How to Prove Captions Were Loaded

Do not infer success from the presence of files.

Use the strongest evidence the trainer provides:

1. dataset summary reports the expected captioned image count;
2. logs show the intended extension or metadata source;
3. a caption preview displays real file text;
4. warnings show no missing sidecars or columns;
5. a tiny dry run or debug dataset uses intentionally distinct captions;
6. cached text outputs are rebuilt after caption changes.

If a tool supports caption display after shuffling or prefix insertion, inspect it. That is closer to the text encoder input than the source file alone.

Mechanical Validation Checklist

Before a full run:

• count included images;
• count nonblank captions or metadata rows;
• compare image identifiers;
• confirm extension and case;
• parse every JSONL line;
• confirm caption column selection;
• search exact trigger spelling;
• inspect delimiter and fixed-item behavior;
• disable sentence shuffling;
• remove orphan files;
• rebuild stale caption or text caches;
• check the dataset summary.

Then inspect caption meaning separately. Structural validity does not make a false caption true.

LoRA Studio can help organize paired local data. sd-scripts offers explicit dataset and caption processing controls, OneTrainer supports sidecar-driven caption workflows, and Hugging Face Datasets provides ImageFolder metadata mapping.

Bottom Line

Caption format is a contract among your files, loader, processor, tokenizer, and base model. The contract must be explicit.

Choose sidecars or metadata according to the script, choose tags or sentences according to the model and task, and configure processing according to the actual syntax. Prove captions loaded before spending hours training.