Bedankt voor uw aanvraag! Een van onze medewerkers neemt binnenkort contact met u op
Bedankt voor uw boeking! Een van onze medewerkers neemt binnenkort contact met u op.
Duur 14 uren
Cursusaanbod
Deployment Strategies
- Trade-offs between Hosted APIs and self-hosting
- Patterns for on-premises, edge, and air-gapped environments
- Single model versus portfolio management
- Cost-per-decision analysis
Hardware Sizing and Capacity Planning
- Memory usage relative to parameter count
- CPU-only versus GPU-accelerated processing
- Planning for concurrency and throughput
- Allowing headroom for increasing task lengths
Serving Infrastructure
- Evaluating vLLM and SGLang options
- Execution via llama.cpp and GGUF
- Using Ollama for local development
- Selecting tools based on the shape of the decision workload
Quantisation While Preserving Calibration
- Precision formats and resulting memory savings
- Measuring quality degradation and calibration loss
- Per-tensor and importance-aware quantisation variants
- Running acceptance tests post-quantisation
Latency Optimisation
- Distinguishing median versus tail latency
- Impact of context length on latency
- Implementing warm starts and model caching
- Batching multiple decision queries
Edge and Constrained Environments
- Running small decision models on CPU
- Deployment on Apple Silicon and embedded systems
- Offline operation and update strategies
- Distributing weights in air-gapped settings
Containerisation and Scaling
- Designing images for model serving
- Managing volumes and caches
- Horizontal scaling and routing strategies
- Setting resource limits and sharing GPUs
Observability
- Tracking request, latency, and error metrics
- Monitoring confidence distributions
- Logging decision outcomes and escalations
- Setting alerts for calibration drift
Versioning and Rollout
- Versioning models and adapters
- Shadow and canary deployment methods
- Pinning model files for audit purposes
- Establishing rollback procedures
Security and Multi-Tenancy
- Implementing authentication and rate limiting
- Ensuring tenant isolation for prompts and logs
- Managing network isolation and egress controls
- Maintaining audit logs of decisions
Vereisten
- Experience with Linux administration and containers.
- Knowledge of model inference servers and GPU management.
- Foundational understanding of REST APIs, load balancing, and monitoring.
Target Audience
- MLOps and platform engineers managing inference infrastructure.
- SRE teams supporting AI services.
- AI engineers deploying decision models on-premises.
- Architects designing air-gapped or regulated AI deployments.
Aangepaste bedrijfsopleiding
Opleidingsoplossingen ontworpen exclusief voor bedrijven.
- Aangepaste inhoud: We passen de syllabus en praktijkopdrachten aan naar de echte doelen en behoeften van uw project.
- Voor flexibel schema: Datums en tijden aangepast aan het rooster van uw team.
- Formaat: Online (live), In-company (bij uw kantoren) of Hybride.
Prijs per privégroep, online live training, startend vanaf 3200 € + BTW*
Neem contact met ons op voor een exacte offerte en om onze laatste promoties te horen
