Spark Installation on Linux

Here are the steps to install Hadoop 2.x on Linux machine

Step 1: Install Java

Check Java installation on your machine

$ java -version 

java version "1.8.0_66"
Java(TM) SE Runtime Environment (build 1.8.0_66-b17)
Java HotSpot(TM) 64-Bit Server VM (build 25.66-b17, mixed mode)

If you don’t have Java installed on your system, use below link to install the java.

https://www.java.com/en/download/help/linux_x64_install.xml

Step 2: Install Scala

Check Scala installation on your machine

$ scala -version
scala: command not found 

2.1 Download and Install Scala

$ cd /opt/hadoop
$ wget http://downloads.lightbend.com/scala/2.12.1/scala-2.12.1.tgz
$ tar -xzf scala-2.12.1.tgz
$ mv scala-2.12.1 scala
$ chown -R hadoop scala

2.2 Set PATH

$ export SCALA_HOME=/opt/hadoop/scala
$ export PATH=$SCALA_HOME/bin:$PATH

2.3 Verify Scala Installation

$ scala -version
Scala code runner version 2.12.1 -- Copyright 2002-2016, LAMP/EPFL and Lightbend, Inc.
$

Step 3: Install Spark

Download and install Spark

cd /opt/hadoop
$ wget http://d3kbcqa49mib13.cloudfront.net/spark-2.1.0-bin-hadoop2.6.tgz
$ tar -xzf spark-2.1.0-bin-hadoop2.6.tgz
$ mv spark-2.1.0-bin-hadoop2.6 spark
$ chown -R hadoop spark

Set PATH

$ export SPARK_HOME=/opt/hadoop/spark
$ export PATH=$SPARK_HOME/bin:$PATH

Step 3: Verify Spark Installation 

Run spark-shell

$ spark-shell

Exit scala shell

scala> :q
$