LoRA Captioning Guide
LoRA captions aren't alt text, creative writing, or a list of every visible pixel. They are training controls. A good caption separates the concept you want the adapter to learn from details you still want to change later. A bad caption quietly binds a person to one jacket, a product to one background, or a style to one subject.
This guide explains the reasoning behind useful captions across subject, character, object, clothing, and style datasets. You'll learn when to use tags or natural language, what a trigger token does, how caption shuffling changes tag order, why auto-captioning needs human review, and how to audit a finished caption set for missing or inconsistent information. The goal is simple: cleaner learning and more control when you prompt.
Quick Answer
Caption visible details that should remain controllable. Don't caption the core concept with ordinary descriptive tokens if your goal is for the LoRA or trigger token to carry that concept.
Put a unique trigger token first when your workflow uses one. Then describe variable attributes such as clothing, pose, expression, camera view, background, lighting, and other subjects. Keep wording consistent, remove false tags, and review every automatic caption.
What Captions Do During Training
A text-conditioned image trainer pairs visual evidence with text. Repeated visual information that isn't explained by ordinary caption terms can become associated with the trigger token or adapter behavior.
Suppose every image shows the same person:
- If captions say
ohwxperson, black jacketonly where the jacket appears, the model gets evidence that the jacket is separate from identity. - If the jacket appears repeatedly but is never captioned, it can leak into the learned identity.
- If captions describe the person's fixed facial traits in every file, those traits may be distributed across common words instead of being carried cleanly by the trigger.
Captioning is about assigning responsibility.
Step 1: Decide What the LoRA Owns
Write two lists:
The LoRA should own
- identity;
- a product's fixed design;
- a garment's construction;
- a character's defining features;
- a visual style's rendering language.
The prompt should still control
- pose and expression;
- clothing not central to the concept;
- background and location;
- lighting and camera angle;
- color when it isn't fixed;
- other people or objects.
Caption the second list when those details are visible. Avoid redundantly describing the first list unless your specific model and caption strategy require class words or known attributes.
Step 2: Pick Tags or Natural Language
Tag captions
Tags are comma-separated fragments such as:
ohwxperson, woman, black jacket, outdoors, looking at viewer, upper body
They are compact and easy to shuffle. They work well for trainers and base models accustomed to booru-like tagging.
The tradeoff is weaker description of relationships. Tags can list “person, dog, bench” without clearly saying who is sitting where.
Natural-language captions
Natural language describes relationships:
A medium photo of ohwxperson wearing a black jacket, standing outdoors and looking at the camera.
This can suit text encoders and base models trained on sentence-like descriptions. Keep sentences factual. Decorative prose adds tokens without adding useful control.
Hybrid captions
A hybrid can combine a stable prefix with concise descriptors, but don't mix styles randomly from file to file. Consistency makes audits and debugging easier.
Match caption style to the base model and trainer. A popular caption format isn't automatically best for every architecture.
Step 3: Use a Trigger Token Deliberately
A trigger token gives you a compact prompt handle for the learned concept. Choose a token that is:
- uncommon enough to avoid a strong existing meaning;
- easy to type consistently;
- free of spaces if your workflow expects one token-like string;
- not a real artist, celebrity, brand, or loaded class name.
Pair it with a class word when useful, such as ohwxperson woman or zvproduct backpack. The class word anchors the broad category while the unique token identifies your concept.
Tokenization varies by text encoder. A typed string may split into multiple tokenizer pieces, so “unique-looking” doesn't guarantee one true token. The practical goal is a stable, low-conflict identifier, not a magic syllable.
Step 4: Put Variable Details in the Caption
For a person or character, caption:
- clothing and accessories that change;
- pose, expression, and gaze;
- framing and camera angle;
- environment and major background elements;
- unusual lighting or image treatment;
- other people.
For an object or product, caption:
- view and orientation;
- open, closed, folded, worn, or in-use state;
- background and support surface;
- nearby props;
- color if the product has multiple variants.
For a style dataset, caption subject matter thoroughly. If “red castle at sunset” is visible, name it. The style should be the shared unexplained signal, not the castle.
Step 5: Don't Overcaption Fixed Identity
More words aren't automatically better. Repeating a person's eye color, nose shape, and hairstyle in every caption can spread identity across generic attribute words.
Caption a supposedly fixed feature when:
- it changes across the dataset;
- you want it independently promptable;
- the class or target model needs the distinction;
- omitting it causes a known training problem.
Otherwise, let the concept token carry fixed identity. Test this assumption with checkpoints because model families respond differently.
Step 6: Keep Vocabulary Consistent
Pick one phrase for one idea. Don't alternate among full body, full-body shot, entire person, and wide portrait without a reason.
Standardize:
- class words;
- framing labels;
- view labels;
- recurring garments;
- background terms;
- trigger capitalization.
Consistency doesn't mean every caption has the same template. It means the same visible fact isn't named five incompatible ways.
Step 7: Understand Caption Shuffling
sd-scripts can shuffle comma-separated caption items with shuffle_caption. This reduces dependence on a fixed tag order. Its keep_tokens setting keeps a chosen number of leading items fixed; keep_tokens = 1 is a common way to preserve the first trigger item.
Shuffling is designed for separable tag captions. Don't shuffle ordinary sentence fragments. Rearranging natural-language clauses can create broken text and unclear relationships.
If tag order carries meaning in your format, disable shuffling or use separators and settings documented by the trainer.
Step 8: Use Dropout Only With a Goal
Caption or tag dropout can encourage the adapter not to depend on every descriptor appearing together. It can also remove useful distinctions and push omitted details into the concept.
Start simple. Establish a clean baseline before adding dropout. When you enable it, preserve any required trigger or class prefix and compare fixed validation prompts.
Don't copy a dropout percentage from an unrelated dataset. The correct behavior depends on caption density, concept type, and trainer implementation.
Step 9: Treat Auto-Captioning as a Draft
Automatic captioners save time on large datasets, but they make predictable mistakes:
- wrong hair, eye, clothing, or object colors;
- missed accessories and small product details;
- guessed identity, age, gender, or relationships;
- irrelevant quality tags;
- inconsistent synonyms;
- descriptions of defects you plan to remove.
OneTrainer includes automatic caption tooling, and other local workflows can use image captioning or tagger models. None removes the need for review.
Correct false claims first. Then normalize vocabulary and add the trigger consistently.
Step 10: Keep Caption Files Paired
Many local trainers use matching image and .txt stems:
image_001.png
image_001.txt
sd-scripts supports a configurable caption extension. Hugging Face's Diffusers ImageFolder workflow instead documents metadata.jsonl for mapping local files to captions. Use the format your selected script expects.
Check for orphan images, orphan captions, duplicate stems, blank files, and encoding problems before training.
A Caption Audit That Catches Real Problems
Review captions in three passes:
- Per-image truth: Is every included detail visible and correct?
- Dataset consistency: Are equivalent details named the same way?
- Correlation check: Is a repeated incidental detail missing from many captions?
Search the entire caption folder for the trigger and confirm the expected count. Then search for common attributes and inspect whether they are applied consistently.
Examples by Dataset Type
Character
nvycharacter, woman, blue coat, smiling, city street, night, medium shot
Product
zvpack backpack, rear view, open main compartment, on a wooden table
Style
a sailboat crossing dark water beneath a cloudy sky, wide composition
In the style example, the style trigger can be included according to the training strategy, while the scene receives a complete description.
LoRA Studio from LocalForge AI can guide caption setup locally; OneTrainer's caption tools, sd-scripts sidecars, and Diffusers metadata remain good options when you want to own the exact format.
Bottom Line
Caption for control, not completeness. Name changing details, keep the concept assignment clear, and remove every false or inconsistent descriptor.
The best captions feel plain because they are. Training needs accurate labels, not impressive sentences.