Thank you for sending your enquiry! One of our team members will contact you shortly.
Thank you for sending your booking! One of our team members will contact you shortly.
Duration 21 hours (3 days)
Course Outline
Introduction:
- Apache Spark within the Hadoop Ecosystem
- Brief overview of Python and Scala
Foundational Theory:
- Architecture
- RDD (Resilient Distributed Datasets)
- Transformations and Actions
- Stages, Tasks, and Dependencies
Practical Exploration via Databricks (Hands-on Workshop):
- Exercises utilizing the RDD API
- Core action and transformation functions
- Working with PairRDD
- Join operations
- Caching strategies
- Exercises utilizing the DataFrame API
- SparkSQL
- DataFrame operations: select, filter, group, and sort
- UDF (User Defined Functions)
- Overview of the DataSet API
- Streaming concepts
Deployment Mastery via AWS (Hands-on Workshop):
- Fundamentals of AWS Glue
- Distinguishing between AWS EMR and AWS Glue
- Implementing example jobs in both environments
- Evaluating the advantages and limitations of each
Additional Topics:
- Introduction to Apache Airflow for orchestration
Requirements
Programming skills (preferably in Python or Scala)
Foundational knowledge of SQL
Testimonials (3)
Having hands on session / assignments
Poornima Chenthamarakshan - Intelligent Medical Objects
Course - Apache Spark in the Cloud
1. Right balance between high level concepts and technical details. 2. Andras is very knowledgeable about his teaching. 3. Exercise
Steven Wu - Intelligent Medical Objects
Course - Apache Spark in the Cloud
Get to learn spark streaming , databricks and aws redshift