All for Joomla All for Webmasters

Spark & Scala

asdf1

Spark Training Course Reviews

AWS-Course-ReviewsAverage of 4.9 out of 5 based on 87 Votes

About the Course

Ethans is market leader in providing trainings to working professionals, we are dedicated and commited to provide value to all our students. Our Spark and Scala course is expected to take two months with total 14-16 classes which includes practical Scala and Spark training, each class is having three-four hours training. It can take lesser time if the number of hours per day is increased.

 

Duration:40 hours class room program, 7 Weekends

Prerequisites: Good to have Basic Knowledge of UNIX, SQL or Any Programming Language (We conduct Free Java Classes on Saturday and Sunday)  

Lab: 40 hours’ lab sessions + 60 plus home assignments  + 10 POC's (Mini Projects)

Who get this training?

  • Data analysts and scientist
  • Big Data professionals
  • Developers
  • IOT
  • Operation Professionals
  • Automation Engineers
  • Robotics Engineers
  • College Students
  • Project Managers

 

Syllabus

Introduction

  • Introduction to Scala and Spark

Scala Programming: Level One

  • Arithmetic and Numbers
  • Values and Variables
  • Booleans and Comparison Operators
  • Strings and Basic Regex
  • Tuples
  • Scala Basics - Assessment Test Exercises

Collections

  • Intro to Collections
  • Lists
  • Arrays
  • Sets
  • Maps
  • Collections - Assessment Test Exercise

Scala Programming: Level Two

  • Flow Control
  • For Loops
  • While Loops
  • Functions
  • Scala Programming Exercises

Spark programming basics

  • The Resilient Distributed Dataset
  • Ratings Histogram Walkthrough
  • Spark Internals
  • Key / Value RDD's, and the Average Friends by Age example
  • [POC] Running the Average Friends by Age Example
  • Filtering RDD's, and the Minimum Temperature by Location Example
  • [POC] Running the Minimum Temperature Example, and Modifying it for Maximum
  • [POC] Counting Word Occurrences using Flatmap()
  • [POC] Improving the Word Count Script with Regular Expressions
  • [POC] Sorting the Word Count Results
  • [Exercise] Find the Total Amount Spent by Customer and Sort Them by Total Amount Spent

Advanced Examples of Spark Programs

  • [POC] Find the Most Popular Movie
  • [POC] Use Broadcast Variables to Display Movie Names
  • [POC] Find the Most Popular Superhero in a Social Graph
  • Superhero Degrees of Separation: Introducing Breadth-First Search
  • Superhero Degrees of Separation: Accumulators, and Implementing BFS in Spark
  • Superhero Degrees of Separation: Review the code, and run it!
  • Item-Based Collaborative Filtering in Spark, cache(), and persist()
  • [POC] Running the Similar Movies Script using Spark's Cluster Manager
  • [Exercise] Improve the Quality of Similar Movies

Running Spark on a Cluster

  • [POC] Using spark-submit to run Spark driver scripts
  • [POC] Packaging driver scripts with SBT
  • Introducing Amazon Elastic MapReduce
  • Creating Similar Movies from One Million Ratings on EMR
  • Partitioning
  • Best Practices for Running on a Cluster
  • Troubleshooting, and Managing Dependencies

Spark DataFrames and Datasets

  • Introduction to Spark DataFrames
  • DataFrames Overview
  • Spark DataFrame Operations
  • GroupBy and Aggregate Functions
  • Missing data
  • Date and Timestamps
  • DataFrame Project Exercises
  • [POC] Using DataSets instead of RDD's

Regression with Spark

  • Introduction to Linear Regression
  • Introduction to Regression Section
  • Linear Regression Documentation Example
  • Alternate Linear Regression Data CSV File
  • Linear Regression Walkthrough Part 1
  • Linear Regression Walkthrough Part 2
  • Linear Regression Exercise Project

Classification with Spark

  • Introduction to Classification
  • Classification Documentation Example
  • Spark Classification - Logistic Regression Example - Part 1
  • Spark Classification - Logistic Regression Example - Part 2
  • Logistic Regression Project Exercise

Model Evaluation

  • Model Evaluation Overview
  • Spark Model Evaluation - Documentation Example
  • Spark - Model Evaluation - Regression Example

Clustering with Spark

  • Introduction to Clustering with Spark
  • KMeans Theory Lecture
  • Example of KMeans with Spark
  • Clustering Project Exercise Overview

PCA with Spark

  • PCA Theory Overview
  • PCA with Spark - Project Exercise
  • PCA with Spark - Documentation Example

DataBricks and Spark

  • Databricks Overview
  • Introduction to Spark Recommendation Systems
  • Spark Recommender System Implementation
  • Zeppelin Notebooks on AWS Elastic MapReduce.

Intro to Spark Streaming

  • Spark Streaming Overview
  • [POC] Set up a Twitter Developer Account, and Stream Tweets
  • Structured Streaming

Projects

  • Live Twitter Sentiments Analysis using Spark
  • Yelp Data Processing Using Spark and Hive Part 1
feedback