Prophecy for Data Engineering: self-service Spark
Prophecy for Data Engineering: self-service Spark
Low-code Spark and Databricks
Created by Mei Long, Richard Marcus, Anya Bida, Maciej Szpakowski | 5 hours on-demand video course
This Prophecy for Data Engineering: self-service Spark course is designed to help data engineers and analysts to build a medallion architecture on a data lakehouse. It is created with the intention of helping you embark on your data engineering journey with Spark and Prophecy.
We will start by staging the ingested data from application platforms like Salesforce, operational databases with CDC transactional data, and machine generated data like logs and metrics. We’re going to clean and normalize the ingested tables to prepare a complete, clean, and efficient data model. From that data model, we’re going to build four projects creating consumption applications for different real-world use-cases.
What you’ll learn
- Learn and design the data lakehouse paradigm for a e-commerce company
- Hands-on lab environment is provided with this course
- Implement and deploy a medallion architecture using Prophecy running on Databricks
- Understand Apache Spark and its best practices with real-life use cases
- Share and extend Pipeline components with data practitioners and analysts
- Deploy Pipelines to production and CI/CD and best practices
- Utilize version control and change management in data engineering
- Deploy data quality checks and unit tests
Recommended Course
Azure Databricks & Spark For Data Engineers (PySpark / SQL)
Databricks Certified Data Engineer Associate – Preparation
Azure Databricks and Spark SQL (Python)
Databricks Certified Data Engineer Professional -Preparation Best seller
Who this course is for:
- data engineers, data scientists, data analysts, data architects, data leads, data engineering leads
