The Fine Art of Fine-Tuning Large Language Models Talk at H2O World India
- Generative AI
- H2O Cascade
Overview
Recorded at H2O World India, this talk from H2O principal data scientist and Kaggle Grandmaster Pascal introduces H2O's open source LLM Studio and the three-stage path to building a custom LLM: a pretrained foundation model, fine-tuning on curated data, and optional reinforcement learning from human feedback. He demonstrates fine-tuning in LLM Studio, then shows the resulting chat model answering questions, including a live example of the limits fine-tuning has without a connected knowledge database.
Key takeaways
- The three-stage path to a usable LLM: a pretrained foundation model, fine-tuning on a small curated dataset, and optional reinforcement learning from human feedback
- Why H2O released LLM Studio as open source under Apache 2.0, including a 20 billion parameter fine-tuned model cleared for commercial use
- How to import a dataset and configure a fine-tuning run in LLM Studio's UI, including hyperparameter tuning and validation tracking
- How to test a fine-tuned model's chat behavior directly in LLM Studio, including follow-up questions that rely on conversational memory
- Why fine-tuned LLMs still need a connected database or retrieval layer to ground answers in current, verifiable information rather than what's baked into training data
- What's next for LLM Studio: reward models, training custom foundation models, and better ways to automatically evaluate LLM output quality
Start owning your AI