How to train a Wan 2.2 LoRA

Pick the model and the trainer

Wan 2.2 ships three generation checkpoints you'd train on: T2V-A14B (text to video), I2V-A14B (image to video) and TI2V-5B (both, in one smaller dense model). Train on the checkpoint you'll run. A LoRA for image-to-video belongs on I2V-A14B, because that model takes the first frame as conditioning and the text-to-video model doesn't.

The Wan team doesn't publish its own LoRA trainer. Its README points to DiffSynth-Studio as supporting Wan 2.2 LoRA training, full training, low-memory offload and FP8 quantisation. The README also says the Wan-Animate team does not recommend using LoRAs trained on Wan 2.2 with Wan-Animate.

Wan 2.2 high vs low noise: what you're training

The A14B models are mixture-of-experts. Per the Wan README, a high-noise expert handles the early denoising steps and sets the overall layout, and a low-noise expert takes the later steps and refines detail. Each is about 14B parameters, 27B in total, with 14B active per step. The switch point is set from the signal-to-noise ratio, and the official configs put it at a boundary of 0.875 for T2V and 0.900 for I2V.

You have three options, and all three trainers support at least two of them:

• Train both experts. The default in ai-toolkit's example (train_high_noise: true, train_low_noise: true). musubi-tuner does this when you pass the low-noise model as --dit and the high-noise model as --dit_high_noise. DiffSynth-Studio runs two separate jobs with complementary timestep ranges and saves two LoRAs.
• Train the low-noise expert only. It handles the detail steps, which is where texture and fine appearance get resolved. musubi-tuner says that with only the low-noise model loaded at inference, it's used for all timesteps.
• Train the high-noise expert only. It sets layout and broad structure early in sampling. Less common on its own.

The Wan team doesn't say which expert matters for which kind of LoRA. Going by its description of the split, motion and composition are decided mostly in the high-noise steps and appearance detail in the low-noise steps, so a style or motion LoRA usually wants both experts. Training both is the safe default; train one if you're short on memory or time.

Wan 2.2 LoRA dataset preparation

• Frame rate. The A14B models generate at 16 fps (sample_fps=16 in the shared config), and musubi-tuner recommends 16 fps footage for Wan. If your clips are 24 or 30 fps, musubi-tuner's source_fps setting drops frames to match; check the result with --debug_mode video when caching.
• Frame counts. Wan wants 4n+1 frames (17, 33, 49, 81). musubi-tuner truncates target_frames to the nearest 4n+1, and DiffSynth-Studio's num_frames must be a multiple of 4 plus 1. DiffSynth-Studio's examples train on 49 frames.
• Resolution. Trainer examples use 960×544 (musubi-tuner), 832×480 (DiffSynth-Studio) and buckets of 512, 768 and 1024 (ai-toolkit). Lower resolution and fewer frames are the main levers on training memory.
• Images work too. ai-toolkit's Wan 2.2 example trains on still images (num_frames: 1) and notes it works for video by raising the frame count. musubi-tuner accepts image and video datasets in one config. Stills are cheaper for appearance; motion needs clips.
• Clip content. Each clip should show the thing you want the LoRA to learn, without long stretches of unrelated footage. musubi-tuner's frame_extraction modes (head, chunk, slide, uniform, full) control which frames of a longer clip get used; its docs suggest full when each clip is one complete motion and the others for repeated actions.

None of the three trainers gives a minimum or recommended dataset size, and the Wan team doesn't either. Start small, sample during training, and add data where samples are weak.

Captioning

musubi-tuner and ai-toolkit read one caption file per clip with the same name (clip01.mp4 and clip01.txt). DiffSynth-Studio reads a metadata.csv with video and prompt columns. musubi-tuner and DiffSynth-Studio also accept JSONL metadata files.

Write captions the way you'll prompt. Wan's official prompt-rewriting instructions describe the subject, how the motion unfolds and any camera movement ("the camera pans up"), and for image-to-video they focus on motion rather than what's already visible in the frame. Captions in that style keep training and inference consistent. If you use a trigger word, put it in every caption. ai-toolkit's example warns that its trigger_word setting doesn't apply when you cache text embeddings, so write it into the caption files yourself.

video,promptclip01.mp4,"A woman in a red coat walks along a pier at dusk, the camera tracks beside her from the left."clip02.mp4,"Close-up of the same woman turning her head toward the camera, hair moving in the wind."

Training with musubi-tuner

musubi-tuner needs the two Wan 2.2 DiT files (it points to the Comfy-Org repackaged weights; fp16 works, fp8_scaled files don't), the Wan 2.1 VAE (Wan2.1_VAE.pth; the Wan 2.2 VAE is for the 5B model only) and the umt5-xxl T5 encoder. No CLIP model for Wan 2.2. Start with a dataset config:

[general]resolution = [960, 544]caption_extension = ".txt"batch_size = 1enable_bucket = true[[datasets]]video_directory = "/data/clips"cache_directory = "/data/cache"target_frames = [1, 25, 45]frame_extraction = "head"source_fps = 30.0

Cache latents and text-encoder outputs once, before training. For I2V, add --i2v to the latent cache step. --fp8_t5 helps on GPUs under about 16GB.

python src/musubi_tuner/wan_cache_latents.py --dataset_config dataset.toml \  --vae models/Wan2.1_VAE.pth --i2vpython src/musubi_tuner/wan_cache_text_encoder_outputs.py --dataset_config dataset.toml \  --t5 models/models_t5_umt5-xxl-enc-bf16.pth --batch_size 16

musubi-tuner's docs show a full training command for Wan 2.1 and describe the Wan 2.2 changes in prose. This is that command with those changes applied for I2V-A14B, training both experts:

accelerate launch --num_cpu_threads_per_process 1 --mixed_precision bf16 \  src/musubi_tuner/wan_train_network.py \  --task i2v-A14B \  --dit models/wan2.2_i2v_low_noise_14B_fp16.safetensors \  --dit_high_noise models/wan2.2_i2v_high_noise_14B_fp16.safetensors \  --dataset_config dataset.toml --sdpa --mixed_precision bf16 --fp8_base \  --optimizer_type adamw8bit --learning_rate 2e-4 --gradient_checkpointing \  --max_data_loader_n_workers 2 --persistent_data_loader_workers \  --network_module networks.lora_wan --network_dim 32 \  --timestep_sampling shift --discrete_flow_shift 3.0 \  --offload_inactive_dit \  --max_train_epochs 16 --save_every_n_epochs 1 --seed 42 \  --output_dir output --output_name my-wan22-lora

--timestep_boundary defaults to 0.9 for I2V and 0.875 for T2V, matching the official configs. --offload_inactive_dit keeps only the active expert on the GPU and can't be combined with --blocks_to_swap. musubi-tuner says the official inference shifts are 12.0 for T2V and 5.0 for I2V, that training shift doesn't have to match them, and that timestep_sampling and discrete_flow_shift need experimentation.

From our reading of the training script, the combined run trains one set of LoRA weights that's applied to whichever expert is active. If you want separate files per expert, run it twice, once per expert, with --min_timestep, --max_timestep and --preserve_distribution_shape.

Training with ai-toolkit

ai-toolkit's example config trains the T2V-A14B model on images in 24GB. The parts specific to Wan 2.2:

network:  type: "lora"  linear: 32  linear_alpha: 32datasets:  - folder_path: "/path/to/images/or/video/folder"    caption_ext: "txt"    caption_dropout_rate: 0.05    num_frames: 1    resolution: [512, 768, 1024]train:  steps: 2000  gradient_checkpointing: true  optimizer: "adamw8bit"  lr: 1e-4  switch_boundary_every: 10  cache_text_embeddings: truemodel:  name_or_path: "ai-toolkit/Wan2.2-T2V-A14B-Diffusers-bf16"  arch: "wan22_14b"  quantize: true  qtype: "uint4|ostris/accuracy_recovery_adapters/wan22_14b_t2i_torchao_uint4.safetensors"  quantize_te: true  qtype_te: "qfloat8"  low_vram: true  model_kwargs:    train_high_noise: true    train_low_noise: true

switch_boundary_every: 10 alternates training between the two experts every 10 steps. With low_vram, the expert not being trained is unloaded. The 4-bit qtype uses an accuracy recovery adapter to train on a quantised base. The config's comments put 500 to 4,000 steps as a good range and warn that training on video is very VRAM-intensive. For video, raise num_frames in the dataset and samples.

Training with DiffSynth-Studio

DiffSynth-Studio trains each expert in its own run, selecting it with a timestep range. Its I2V example, high-noise half:

accelerate launch examples/wanvideo/model_training/train.py \  --dataset_base_path data/my_dataset \  --dataset_metadata_path data/my_dataset/metadata.csv \  --height 480 --width 832 --num_frames 49 \  --dataset_repeat 100 \  --model_id_with_origin_paths "Wan-AI/Wan2.2-I2V-A14B:high_noise_model/diffusion_pytorch_model*.safetensors,Wan-AI/Wan2.2-I2V-A14B:models_t5_umt5-xxl-enc-bf16.pth,Wan-AI/Wan2.2-I2V-A14B:Wan2.1_VAE.pth" \  --learning_rate 1e-4 --num_epochs 5 \  --remove_prefix_in_ckpt "pipe.dit." \  --output_path "./models/train/Wan2.2-I2V-A14B_high_noise_lora" \  --lora_base_model "dit" --lora_target_modules "q,k,v,o,ffn.0,ffn.2" --lora_rank 32 \  --extra_inputs "input_image" \  --max_timestep_boundary 0.358 --min_timestep_boundary 0

The low-noise run swaps in low_noise_model/... and uses --max_timestep_boundary 1 --min_timestep_boundary 0.358. The script's comments say these values correspond to timesteps 900 to 1000 and 0 to 900. For T2V-A14B the split value is 0.417 (timestep 875). DiffSynth-Studio recommends a learning rate of 1e-4 for LoRA, saving by step with --save_steps, and judging by samples rather than loss, which it says has little relationship with actual results.

Wan 2.2 LoRA training VRAM

What the trainers document for the 14B models:

• musubi-tuner recommends 24GB or more for video training in general, and 64GB of system RAM. Its Wan docs say 720×1280 image training fits in 24GB with fp8 and block swap.
• ai-toolkit's example is built for 24GB: a 4-bit transformer, fp8 text encoder, cached text embeddings, and one expert loaded at a time.
• DiffSynth-Studio gives no figure. Its training docs list gradient checkpointing and checkpoint offload to RAM (recommended for video), fp8 storage for the DiT during LoRA training, and multi-GPU DeepSpeed.

Wan 2.2 LoRA training on 16GB VRAM

No trainer documents a 16GB setup for the 14B models. The levers that exist are the ones above, pushed further: --fp8_base, --blocks_to_swap and --fp8_t5 in musubi-tuner, gradient checkpointing with offload, one expert at a time, lower resolution, and stills or short clips instead of 81-frame video. Expect to lean on system RAM. musubi-tuner notes that on Windows, --offload_inactive_dit needs about 42GB of shared VRAM for the two experts even in fp8, which means roughly 96GB of system RAM; --blocks_to_swap uses less.

The other route is TI2V-5B, which ai-toolkit and DiffSynth-Studio both support and which the Wan team runs for inference on a 24GB GPU. Or train the 14B models on a rented GPU with 48GB or 80GB and skip most of the memory tricks.

Using the LoRA: high and low, and LoRA strength

At inference, load each LoRA onto the expert it was trained for. In musubi-tuner's generator that's --lora_weight and --lora_multiplier for the low-noise model and --lora_weight_high_noise and --lora_multiplier_high_noise for the high-noise model:

python src/musubi_tuner/wan_generate_video.py --task i2v-A14B \  --dit models/wan2.2_i2v_low_noise_14B_fp16.safetensors \  --dit_high_noise models/wan2.2_i2v_high_noise_14B_fp16.safetensors \  --vae models/Wan2.1_VAE.pth --t5 models/models_t5_umt5-xxl-enc-bf16.pth \  --image_path start.png --video_size 480 832 --video_length 81 \  --prompt "The woman turns toward the camera and smiles, the camera pushes in slowly." \  --lora_weight output/my-wan22-lora-low.safetensors --lora_multiplier 1.0 \  --lora_weight_high_noise output/my-wan22-lora-high.safetensors --lora_multiplier_high_noise 1.0 \  --fp8 --lazy_loading --blocks_to_swap 20 \  --save_path out.mp4

These file names assume two separate runs; after a combined musubi-tuner run, pass the same file to both. In DiffSynth-Studio, the validation script loads the high-noise LoRA onto pipe.dit and the low-noise LoRA onto pipe.dit2, both with alpha=1. In ComfyUI, the official Wan 2.2 14B workflow uses two Load Diffusion Model nodes, one per expert, so a LoRA loader goes after each; musubi-tuner says its LoRAs seem to work in ComfyUI, depending on the nodes.

Wan 2.2 LoRA strength

Every trainer example loads LoRAs at strength 1.0, and neither the Wan team nor the trainers recommend another value. Start at 1.0 on both experts. If the LoRA overpowers the prompt or the start frame, lower it, and try lowering the high-noise strength separately, since that expert sets composition and motion. If you trained only the low-noise expert, apply it there and leave the high-noise model without a LoRA.

Training a LoRA for other models: Z-Image and FLUX.2 klein

The workflow above is the same for any diffusion model: collect a small, consistent set of examples, caption them with the words you'll prompt with, train a low-rank adapter on a frozen base, then test it at a few strengths. What changes per model is which checkpoint you train on and which trainer supports it.

• Z-Image: train on Z-Image base, not Turbo. The Z-Image card marks base as fine-tunable and Turbo as not; the Z-Image README lists DiffSynth-Studio (community) for LoRA training. A LoRA trained on base won't necessarily behave the same on Turbo, so test it there before relying on it.
• FLUX.2 klein: BFL says to fine-tune the undistilled Base models (4B Base or 9B Base), not the 4-step distilled ones (README). 4B Base is Apache-2.0; 9B Base is non-commercial.
• Video models other than Wan: LTX-2.x ships its own trainer, covered in our VRAM guide.

ComfyUI LoRA training

Most people searching for ComfyUI LoRA training end up training in one of the trainers above and using ComfyUI to run the result: put the .safetensors file in ComfyUI's models/loras folder and add a LoRA loader node between the model loader and the sampler. For Wan 2.2 14B that means two loaders, one on the high-noise model and one on the low-noise model, as described in the inference section above.

Running the base model

Windpaint doesn't train or host LoRAs, and doesn't serve video yet; the base Wan 2.2 image-to-video model is in its catalogue as coming soon. For self-hosting figures for the rest of the family, see hardware requirements; for licences, the commercial use page (Wan 2.2 is Apache 2.0).