Apache Airflow on AWS EKS: The Hands-On Guide

How to set up Apache Airflow on AWS EKS with the Kubernetes Executor and be production-ready!

Created by Marc Lamberti | 9 hours on-demand video course

This course is designed to guide you through the different steps of creating a real world architecture:

1. Configuring the EKS cluster following best practices
2. Deploying automatically changes with GitOps
3. Using Helm to configure and set up Airflow on Kubernetes
4. Configuring the official Helm chart of Airflow to use the Kubernetes Executor and many different features
5. Deploying DAGs in Airflow with Git-Sync and AWS EFS
6. Deploying DAGs/Airflow through CI/CD pipelines with AWS CodePipeline
7. Testing your DAGs automatically
8. Securing your credentials and sensitive data in a Secret Backend
9. Enabling remote logging with AWS S3
10. Creating 3 different environments dev/staging and prod
11. Making the production environment scalable and highly available
12. and more!

What you’ll learn

• How to Set Up a Production Ready Architecture for Airflow on AWS EKS From A-Z
• How to deploy DAGs from Git (public and private)
• How to Create CI/CD Pipelines with AWS CodePipeline Deploy DAGs
• How to Share DAGs and Store Logs with AWS EFS
• How to Enable Remote Logging with AWS S3 in EKS
• How to Test your DAGs in CI/CD pipelines
• How to Store Sensitive Data in AWS Secret Manager

Recommended Course by Marc Lamberti

Apache Airflow: The Hands-On Guide

The Complete Hands-On Introduction to Apache Airflow

Apache Airflow: The Operators Guide

Redeem Offer

Click to open site

Udemy

success 100%