Thank you for sending your enquiry! One of our team members will contact you shortly.
Thank you for sending your booking! One of our team members will contact you shortly.
Course Outline
Recap of Apache Airflow Fundamentals
- Core concepts: DAGs, operators, and execution flow
- Airflow architecture and components
- Understanding advanced use cases and workflows
Creating Custom Operators
- Understanding the anatomy of an Airflow operator
- Developing custom operators for specific tasks
- Testing and debugging custom operators
Custom Hooks and Sensors
- Implementing hooks for external system integration
- Creating sensors for monitoring external triggers
- Enhancing workflow interactivity with custom sensors
Developing Airflow Plugins
- Understanding the plugin architecture
- Designing plugins to extend Airflow functionality
- Best practices for managing and deploying plugins
Integrating Airflow with External Systems
- Connecting Airflow to databases, APIs, and cloud services
- Using Airflow for ETL workflows and real-time data processing
- Managing dependencies between Airflow and external systems
Advanced Debugging and Monitoring
- Using Airflow logs and metrics for troubleshooting
- Configuring alerts and notifications for workflow issues
- Leveraging external monitoring tools with Airflow
Optimizing Performance and Scalability
- Scaling Airflow with Celery and Kubernetes Executors
- Optimizing resource utilization in complex workflows
- Strategies for high availability and fault tolerance
Case Studies and Real-World Applications
- Exploring advanced use cases in data engineering and DevOps
- Case study: Custom operator implementation for large-scale ETL
- Best practices for managing enterprise-level workflows
Summary and Next Steps
Requirements
- Strong understanding of Apache Airflow basics, including DAGs, operators, and execution architecture
- Proficiency in Python programming
- Experience with integrating data systems and workflow orchestration
Audience
- Data engineers
- DevOps engineers
- Software architects
21 Hours
Delivery Options
Private Group Training
Our identity is rooted in delivering exactly what our clients need.
- Pre-course call with your trainer
- Customisation of the learning experience to achieve your goals -
- Bespoke outlines
- Practical hands-on exercises containing data / scenarios recognisable to the learners
- Training scheduled on a date of your choice
- Delivered online, onsite/classroom or hybrid by experts sharing real world experience
Private Group Prices RRP from €6840 online delivery, based on a group of 2 delegates, €2160 per additional delegate (excludes any certification / exam costs). We recommend a maximum group size of 12 for most learning events.
Contact us for an exact quote and to hear our latest promotions
Public Training
Please see our public courses
