Thank you for sending your enquiry! One of our team members will contact you shortly.
Thank you for sending your booking! One of our team members will contact you shortly.
Course Outline
Introduction to LLMOps
- Differences between LLMOps and MLOps: unique challenges of operating LLMs.
- The LLM application lifecycle: prompting, evaluation, deployment, and monitoring.
- Checklist for production readiness of GenAI applications.
Prompt Management and Versioning
- Systems for prompt templating and variable injection.
- Semantic versioning for prompts with automated regression testing.
- Prompt registries and collaboration workflows.
LLM Evaluation at Scale
- Evaluation dimensions: accuracy, relevance, safety, and groundedness.
- LLM-as-judge metrics and human evaluation pipelines.
- Automated evaluation frameworks: RAGAS, DeepEval, and custom evaluators.
- Integrating quality gates into CI/CD for LLM deployments.
Safety Guardrails and Content Governance
- Input and output guardrails: NeMo Guardrails and Guardrails AI.
- PII detection, toxicity filtering, and defining topic boundaries.
- Strategies for defending against jailbreaks and prompt injections.
- Conducting red-teaming exercises for LLM applications to ensure safety assurance.
LLM Observability and Monitoring
- Telometry focusing on token usage, latency, cost, and quality metrics.
- Detecting drift in LLM outputs and embedding spaces.
- Session-level tracing for multi-turn agent conversations.
- Setting up dashboards and alerting using LangSmith, Arize, and OpenTelemetry.
AI Gateway and Model Orchestration
- Multi-provider routing using LiteLLM and Portkey.
- Implementing fallback strategies, retry logic, and circuit breakers.
- Aware model selection based on cost and load balancing techniques.
- Managing rate limits, quotas, and API key governance.
Performance Optimization
- Semantic caching using vector stores and exact-match strategies.
- Enforcing structured outputs through constrained decoding.
- Utilizing batching, streaming, and concurrency patterns.
- Optimizing latency across various model providers.
Governance, Compliance, and Audit
- Creating LLM audit trails: maintaining logs for prompts, responses, and decision provenance.
- Addressing data residency and privacy considerations for LLM APIs.
- Implementing policy-as-code for LLM usage within organizations.
- Developing an internal playbook for LLM operations.
Requirements
- Experience in building or integrating LLM-powered applications.
- Familiarity with Python and REST APIs.
- Basic understanding of prompt engineering concepts.
Audience
- ML engineers and MLOps practitioners transitioning to LLM operations.
- Platform engineers responsible for managing LLM infrastructure.
- Technical leads overseeing production GenAI deployments.
14 Hours
Custom Corporate Training
Training solutions designed exclusively for businesses.
- Customized Content: We adapt the syllabus and practical exercises to the real goals and needs of your project.
- Flexible Schedule: Dates and times adapted to your team's agenda.
- Format: Online (live), In-company (at your offices), or Hybrid.
Price per private group, online live training, starting from 3200 € + VAT*
Contact us for an exact quote and to hear our latest promotions
Testimonials (2)
use of proper and effective prompt
Marses Pacaldo
Course - Generative AI and Prompt Engineering for Corporate Professionals
The interactive style, the exercises
