Apache Airflow on AWS EKS: The Hands-On Guide
Deal Score+1
Apache Airflow on AWS EKS: The Hands-On Guide
How to set up Apache Airflow on AWS EKS with the Kubernetes Executor and be production-ready!
Created by Marc Lamberti | 9 hours on-demand video course
This Apache Airflow on AWS EKS: The Hands-On Guide course is designed to guide you through the different steps of creating a real world architecture:
- Configuring the EKS cluster following best practices
- Deploying automatically changes with GitOps
- Using Helm to configure and set up Airflow on Kubernetes
- Configuring the official Helm chart of Airflow to use the Kubernetes Executor and many different features
- Deploying DAGs in Airflow with Git-Sync and AWS EFS
- Deploying DAGs/Airflow through CI/CD pipelines with AWS CodePipeline
- Testing your DAGs automatically
- Securing your credentials and sensitive data in a Secret Backend
- Enabling remote logging with AWS S3
- Creating 3 different environments dev/staging and prod
- Making the production environment scalable and highly available
- and more!
What you’ll learn
- How to Set Up a Production Ready Architecture for Airflow on AWS EKS From A-Z
- How to deploy DAGs from Git (public and private)
- How to Create CI/CD Pipelines with AWS CodePipeline Deploy DAGs
- How to Share DAGs and Store Logs with AWS EFS
- How to Enable Remote Logging with AWS S3 in EKS
- How to Test your DAGs in CI/CD pipelines
- How to Store Sensitive Data in AWS Secret Manager
Recommended Course by Marc Lamberti
The Complete Hands-on Introduction to Airbyte
The Complete Hands-on Introduction to Airbyte Get started with Airbyte and learn how to use it with Apache …
Apache Airflow: The Hands-On Guide Best seller
Apache Airflow: The Hands-On Guide Master Apache Airflow from A to Z. Hands-on videos on Airflow with AWS, …
The Complete Hands-On Introduction to Apache Airflow
The Complete Hands-On Introduction to Apache Airflow Learn to author, schedule and monitor data pipelines …
Apache Airflow: The Operators Guide
Apache Airflow: The Operators Guide Step up with Apache Airflow! Master Airflow Operators and create incredible …
Who this course is for:
- Data engineers
- Software Engineers
- DevOps
