Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 18 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -231,9 +231,26 @@ cd core
./webui.sh --install --extra training # Windows: .\webui.bat --install --extra training
```

**Headless (no UI).** On a rented GPU or in a script, train from a folder of captioned stills or
clips with the same entry point the Trainer tab spawns:

```bash
cd core
python -m inline_core.training \
--dataset /path/to/clips \
--arch minimax-h3 \
--clip-seconds 1 \
--models-dir ./models \
--output ./models/loras/my_h3_clip.safetensors \
--steps 500 --resolution 512
```

See [Headless training (CLI)](TRAINING.md#headless-training-cli) for the dataset layout,
`metadata.jsonl` import, resume, and the stills-only path.

For a worked example, see [`inlineresearch/skin-lora-krea-2-raw`](https://huggingface.co/inlineresearch/skin-lora-krea-2-raw), a photorealistic skin LoRA trained here on the Krea 2 RAW base from the 26 image and caption pairs published as [`inlineresearch/krea2-skin-lora`](https://huggingface.co/datasets/inlineresearch/krea2-skin-lora).

**[TRAINING.md](TRAINING.md) is the full reference:** [which base to train on](TRAINING.md#architecture-and-base-model-modes) · [measured benchmarks](TRAINING.md#benchmark-results) · [datasets and outputs](TRAINING.md#datasets-and-outputs) · [stop and resume](TRAINING.md#stop-and-resume) · [trigger words](TRAINING.md#trigger-words) · [base precision](TRAINING.md#base-precision)
**[TRAINING.md](TRAINING.md) is the full reference:** [which base to train on](TRAINING.md#architecture-and-base-model-modes) · [training on clips](TRAINING.md#training-on-clips) · [headless CLI](TRAINING.md#headless-training-cli) · [measured benchmarks](TRAINING.md#benchmark-results) · [datasets and outputs](TRAINING.md#datasets-and-outputs) · [stop and resume](TRAINING.md#stop-and-resume) · [trigger words](TRAINING.md#trigger-words) · [base precision](TRAINING.md#base-precision)

## How it works

Expand Down
88 changes: 87 additions & 1 deletion TRAINING.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,8 @@ resolution.
**Contents:** [The graph](#the-graph) · [Datasets and outputs](#datasets-and-outputs) ·
[Stop and resume](#stop-and-resume) · [Trigger words](#trigger-words) ·
[Architecture and base model modes](#architecture-and-base-model-modes) · [Install](#install) ·
[Training on clips](#training-on-clips) · [**Benchmark results**](#benchmark-results) ·
[Training on clips](#training-on-clips) · [Headless training (CLI)](#headless-training-cli) ·
[**Benchmark results**](#benchmark-results) ·
[Dataset and adapter options](#dataset-and-adapter-options) · [Base precision](#base-precision)

## The graph
Expand Down Expand Up @@ -102,6 +103,91 @@ better than the first frame usually does. Write them by hand if you would rather
Audio is not trained. H3 generates video and its soundtrack jointly, but the trainer packs zero
audio rows, so an adapter changes what a clip looks like and never what it sounds like.

## Headless training (CLI)

The Trainer tab is optional. The same loop runs from the shell, which is what you want on a rented
GPU, in a Docker box, or when you already have a folder of captioned clips and do not want the SPA.

**Dataset layout.** A folder of media with captions beside them:

```text
dataset/
0000.mp4
0000.txt
0001.mp4
0001.txt
```

Stills (`.png` / `.jpg` / `.webp`) and clips (`.mp4` / `.mov` / `.webm` / `.mkv`) can mix. If the
folder has a Hugging Face-style `metadata.jsonl` (`file_name` + `text` / `caption`) and no
sidecars, the CLI materialises `NNNN.txt` for you. Put `minimax_h3_fl2va_bf16.safetensors` in
`models/diffusion_models/` first (about 124GB).

**Train a clip LoRA (motion):**

```bash
cd core
# after ./webui.sh --install --extra all (or --extra training + runtime)
python -m inline_core.training \
--dataset /path/to/clips \
--arch minimax-h3 \
--clip-seconds 1 \
--models-dir ./models \
--output ./models/loras/my_h3_clip.safetensors \
--steps 500 \
--resolution 512 \
--rank 16
```

`--clip-seconds 1` is the practical floor: H3 snaps to a 22-frame grid at 24fps (~0.92s). Longer
clips cost time per step, not VRAM. Omit images-only runs' clip flag if you prefer; any short video
in the set is still snapped to the grid when the arch is H3.

**Stills only (appearance, no motion):**

```bash
python -m inline_core.training \
--dataset /path/to/stills \
--arch minimax-h3 \
--models-dir ./models \
--output ./models/loras/my_h3_still.safetensors \
--steps 1500 \
--resolution 512
```

**Useful flags.** `--trigger TOKEN` prepends the token to every caption. `--work-dir DIR` holds the
staged dataset, checkpoints and `manifest.json` (resume with `--resume --work-dir DIR`).
`--dry-run` stages and prints the manifest without loading the GPU. `--gpu-ids 0` pins a card.
Progress is JSON lines on stdout (`type: progress|checkpoint|done|error`), the same protocol the
Trainer node already parses.

**Weights & Biases (optional).** Set `WANDB_API_KEY` and pass a project (or `WANDB_PROJECT`):

```bash
export WANDB_API_KEY=…
python -m inline_core.training \
--dataset /path/to/clips \
--arch minimax-h3 --clip-seconds 1 \
--wandb-project scrya-h3-iso-video \
--wandb-run-name scrya_iso_char_clips \
--wandb-log-every 5 \
```

The run URL is printed on stderr as `wandb: https://wandb.ai/…` and mirrored as a progress status
line. Loss, step, peak VRAM and cache phase strings are logged. Omit `--wandb-project` (and leave
`WANDB_PROJECT` unset) for a fully offline run.

**Manifest path (what the UI already uses).** You can still hand the subprocess a prebuilt file:

```bash
python -m inline_core.training /path/to/manifest.json
```

That shape is what `studio/training.py` writes; the flag path builds the same object via
`inline_core.training.manifest`.

## Install

If you installed with `--extra all` from [Get Started](README.md#get-started), the trainer is already set up - nothing more to do. To add it to a leaner install, its dependencies (PEFT, 8-bit Adam, the captioner) sit behind the `training` extra:
Expand Down
Loading