Azure Data Factory +Synapse Analytics End to End ETL project

October 12, 2023 IT & Software, Udemy CoursesUdemy

Deal Score+8

REDEEM

Deal Score+8

REDEEM

Azure Data Factory +Synapse Analytics End to End ETL project

Complete end to end Project on OTT platform using Azure Data Factory and Azure Synapse Analytics from Scratch with Labs

Created by Shanmukh Sattiraju | 5 hours on-demand video course

Are you looking to build an end-to-end ETL project using Azure data engineering services? Look no further than this comprehensive course on Azure Data Factory and Azure Synapse Analytics. This course is designed to guide you through the process of creating all the necessary services from scratch, before building an ETL project from start to finish.

Throughout this course, you’ll gain practical, hands-on experience with Azure Data Factory and Azure Synapse Analytics, learning how to use these powerful data engineering tools to create a highly effective ETL solution. You’ll explore the many features and capabilities of these platforms, as well as their integration with other Azure services like

Azure SQL Database
Azure Synapse Analytics
Azure Key Vault
Azure Data Factory for Orchestration,
Azure Storage solutions (Azure Datalake Gen2)
Microsoft Power BI
Azure Logic Apps

In addition to the core ETL project, this course also includes an additional section on CICD (Continuous Integration and Continuous Deployment) on Azure Data Factory, helping you to fully automate your data engineering workflow.

Whether you’re a beginner or an experienced data engineer, this course is designed to help you gain a comprehensive understanding of Azure Data Factory and Azure Synapse Analytics. By the end of the course, you’ll be able to confidently create and manage your own ETL projects on the Azure cloud.

What you’ll learn

Create incremental pipeline based on last modified date
Calling Azure Synapse notebook from Azure Data Factory
Configure CICD from Scratch in Azure Data Factory
Build end to end project using Azure Data Engineering Services
Create a Self Hosted Integration Runtime and ingest data from On-premise Environment
How to transform data with Azure Synapse Analytics
Create and configure an Azure Data Lake
Project Oriented Learning of Azure Data Engineering services
Create an Azure Key vault service
Using Secrets of Key vault in Linked services
Create and configure an Azure SQL Database
Writing PySpark transformation logic in Synapse Notebook
Use of Triggers in Azure Data Factory
Create an incremental data load pipeline to ingest file in daily basis
Build an Automated pipeline in Azure Data Factory
Setup CICD environment from Scratch
Create incremental pipeline based on file name and pick today’s file
Perform Data Analysis Using PySpark Code using Synapse Notebook
Create and configure an Azure Data Factory
Create and configure an Azure Synapse Analytics
Orchestrate all pipelines in Azure Data Factory
Loading Data into Azure SQL Database
Ingest Data from On-premise to Azure Datalake in Azure Cloud
Send an Automatic Alert email when a pipeline is failed
Transform only Today’s file in Azure Synapse Analytics
Report transformed data in Power BI
Complete hands-on project using Azure Data Engineering Services
Provides insights on services that needed to clear DP-203
Understand how GetMetaData, ForEach, If Condition Activities works