Advanced Techniques & Optimization
Beyond the basics — ensemble methods, prompt optimisation, and distillation
Session Topics
Model Ensembling
Combining multiple models or multiple outputs for higher accuracy and reliability. Majority voting, weighted averaging, and mixture-of-experts.
Prompt Optimisation
Automated prompt search, DSPy, gradient-based prompt tuning — going beyond manual iteration to systematically find better prompts.
Knowledge Distillation
Training smaller, faster models to match larger models using synthetic data — how GPT-4 teaches GPT-3.5.
Retrieval Optimisation
Hybrid search (dense + sparse), HyDE (Hypothetical Document Embeddings), re-ranking with cross-encoders, and contextual compression.
Multi-Armed Bandits & Online Learning
Adaptive systems that learn from user feedback in real time — personalisation, exploration-exploitation, and reward modelling.
Optimize Your Hiring AI System
Choose an advanced technique and design how you'd implement it for your hiring AI. Measure expected impact and trade-offs. Techniques available: multi-armed bandits, online learning, federated learning, differential privacy, model distillation, or quantisation.
An optimisation design document with implementation plan, expected impact metrics, and trade-off analysis.
Key Takeaways
- No single model or technique wins on every task — ensembles reduce variance
- Automated prompt optimisation outperforms manual iteration at scale
- Distillation is how companies serve 80% of quality at 10% of the cost
- Retrieval quality has more impact than generation quality on RAG system output