Apache Airflow on AWS EKS: The Hands-On Guide
How to set up Apache Airflow on AWS EKS with the Kubernetes Executor and be production-ready!
Created by Marc Lamberti | 9 hours on-demand video course
This course is designed to guide you through the different steps of creating a real world architecture:
- Configuring the EKS cluster following best practices
- Deploying automatically changes with GitOps
- Using Helm to configure and set up Airflow on Kubernetes
- Configuring the official Helm chart of Airflow to use the Kubernetes Executor and many different features
- Deploying DAGs in Airflow with Git-Sync and AWS EFS
- Deploying DAGs/Airflow through CI/CD pipelines with AWS CodePipeline
- Testing your DAGs automatically
- Securing your credentials and sensitive data in a Secret Backend
- Enabling remote logging with AWS S3
- Creating 3 different environments dev/staging and prod
- Making the production environment scalable and highly available
- and more!
What you’ll learn
- How to Set Up a Production Ready Architecture for Airflow on AWS EKS From A-Z
- How to deploy DAGs from Git (public and private)
- How to Create CI/CD Pipelines with AWS CodePipeline Deploy DAGs
- How to Share DAGs and Store Logs with AWS EFS
- How to Enable Remote Logging with AWS S3 in EKS
- How to Test your DAGs in CI/CD pipelines
- How to Store Sensitive Data in AWS Secret Manager
Recommended Course by Marc Lamberti
Apache Airflow: The Hands-On Guide
The Complete Hands-On Introduction to Apache Airflow
Apache Airflow: The Operators Guide
Udemy Promotional Code - September 2023
This will also bring up a list of coupons and promo codes that you can use to get a discount on Udemy courses
Get ahead, stay ahead. Online courses as low as $13.99.