Instruction Tuning and DPO in H2O Cascade
- Generative AI
- H2O Cascade
This lesson walks through instruction tuning for domain-specific large language models in H2O Cascade, used when prompt engineering alone is not enough or a frontier model would be too costly for a given use case. It covers preparing labeled instruction and response training data, then configuring fine-tuning with LoRA adapters, learning rate schedules, batch sizes, and gradient accumulation, either by hand or through built in AutoML. It also covers monitoring loss curves and validation perplexity during training, evaluating the model on held out test instructions, and publishing a well performing model to Hugging Face.
Key takeaways
- Why instruction tuning a smaller domain-specific model can reduce cost compared to using a frontier model
- What training data instruction tuning requires, examples pairing an instruction with the desired response
- How LoRA adapters, learning rate schedules, batch sizes, and gradient accumulation control the fine-tuning process
- How AutoML can search fine-tuning configurations automatically instead of tuning every parameter by hand
- What metrics decide whether a fine-tuned model is ready to deploy, instruction following, output quality, and safety
Start owning your AI