Thank you for sending your enquiry! One of our team members will contact you shortly.
Thank you for sending your booking! One of our team members will contact you shortly.
Duration 35 hours
Course Outline
Introduction and Diagnostic Fundamentals
- Understanding failure modes in LLM systems and Ollama-specific challenges
- Setting up reproducible experiments and controlled testing environments
- Mastering the debugging toolkit: local logs, request/response captures, and sandboxing
Reproducing and Isolating Failures
- Methods for generating minimal failing examples and seeds
- Distinguishing stateful from stateless interactions to isolate context-related issues
- Managing determinism, randomness, and controlling non-deterministic behaviors
Behavioral Evaluation and Metrics
- Applying quantitative metrics: accuracy, ROUGE/BLEU variants, calibration, and perplexity proxies
- Conducting qualitative assessments: human-in-the-loop scoring and rubric design
- Verifying task-specific fidelity and defining acceptance criteria
Automated Testing and Regression
- Writing unit tests for prompts and components, as well as scenario and end-to-end tests
- Establishing regression suites and golden example baselines
- Integrating CI/CD for Ollama model updates and implementing automated validation gates
Observability and Monitoring
- Implementing structured logging, distributed tracing, and correlation IDs
- Tracking key operational metrics: latency, token usage, error rates, and quality signals
- Configuring alerts, dashboards, and SLIs/SLOs for model-backed services
Advanced Root Cause Analysis
- Tracing through complex prompt graphs, tool calls, and multi-turn interactions
- Performing comparative A/B diagnosis and ablation studies
- Analyzing data provenance, debugging datasets, and resolving dataset-induced errors
Safety, Robustness, and Remediation Strategies
- Applying mitigations: filtering, grounding, retrieval augmentation, and prompt scaffolding
- Using rollback, canary, and phased rollout patterns for model updates
- Conducting post-mortems, capturing lessons learned, and driving continuous improvement loops
Summary and Next Steps
Requirements
- Proven experience in developing and deploying LLM applications
- Knowledge of Ollama workflows and model hosting practices
- Proficiency with Python, Docker, and foundational observability tools
Target Audience
- AI engineers
- MLOps professionals
- QA teams managing production LLM systems
Custom Corporate Training
Training solutions designed exclusively for businesses.
- Customized Content: We adapt the syllabus and practical exercises to the real goals and needs of your project.
- Flexible Schedule: Dates and times adapted to your team's agenda.
- Format: Online (live), In-company (at your offices), or Hybrid.
Price per private group, online live training, starting from 8000 € + VAT*
Contact us for an exact quote and to hear our latest promotions