Apache Sqoop Installation on Linux

Step 1: Install JAVA and Hadoop

Apache Hive required java 6 or later version. We also need to install hadoop first before installing apache hive on our system. Use below links to install them.

Check Java installation on your machine

$ java -version 

java version "1.8.0_66"
Java(TM) SE Runtime Environment (build 1.8.0_66-b17)
Java HotSpot(TM) 64-Bit Server VM (build 25.66-b17, mixed mode)

If you don’t have Java installed on your system, use below link to install the java.

https://www.java.com/en/download/help/linux_x64_install.xml

Refer below article for Hadoop installation.
http://hadoopnalgos.blogspot.in/2016/04/hadoop-installation-on-linux.html

Step 2: Download Sqoop Archive

After configuring hadoop successfully on your linux system. lets start sqoop setup. First download latest hive source code http://mirrors.ibiblio.org/apache/sqoop/ and extract archive using following commands.

$ cd /opt/hadoop
$ wget 
http://mirrors.ibiblio.org/apache/sqoop/1.4.6/sqoop-1.4.6.bin__hadoop-2.0.4-alpha.tar.gz
$ tar xzf sqoop-1.4.6.bin__hadoop-2.0.4-alpha.tar.gz
$ mv sqoop-1.4.6.bin__hadoop-2.0.4-alpha sqoop
$ chown -R hadoop sqoop

Step 3: Setup Environment Variables

After extracting sqoop archive file, switch to hadoop user and setup following environment variables.

$ su - hadoop
$ export SQOOP_HOME=/usr/lib/sqoop
$ export PATH=$SQOOP_HOME/bin:$PATH

Step 4: Configure Sqoop

We need to configure Sqoop with Hadoop, edit the sqoop-env.sh file, which is placed in the $SQOOP_HOME/conf directory.

$ cd $SQOOP_HOME/conf
$ mv sqoop-env-template.sh sqoop-env.sh

Open sqoop-env.sh file and edit the following:

export HADOOP_COMMON_HOME=/usr/local/hadoop 
export HADOOP_MAPRED_HOME=/usr/local/hadoop

Step 5: Download and Configure mysql-connector

We can download mysql-connector-java-6.0.4.tar.gz file from here : http://ftp.ntu.edu.tw/MySQL/Downloads/Connector-J/ and move mysql-connector-java-6.0.4-bin.jar to /usr/lib/sqoop/lib directory.

$ wget http://ftp.ntu.edu.tw/MySQL/Downloads/Connector-J/mysql-connector-java-6.0.4.tar.gz  
$ tar -zxf mysql-connector-java-6.0.4.tar.gz
$ cd mysql-connector-java-6.0.4
$ mv mysql-connector-java-6.0.4-bin.jar /usr/lib/sqoop/lib

Step 5.1 Download Oracle JDBC Driver if required.

You can download the JDBC Driver from the Oracle website, for example http://www.oracle.com/technetwork/database/enterprise-edition/jdbc-112010-090769.html. You must accept the license agreement before you can download the driver. Download the ojdbc6.jar file and copy it to the /var/lib/sqoop/lib directory:

$ wget http://download.oracle.com/otn/utilities_drivers/jdbc/11204/ojdbc6.jar  
$ mv ojdbc6.jar /var/lib/sqoop/lib

Note:

Please note that ojdbc7.jar Certified with JDK 7 and JDK 8; It contains the JDBC driver classes except classes for NLS support in Oracle Object and Collection types. We need to use ojdbc7.jar for JDK 7 or JDK 8.

Step 5.2 Download Microsoft SQL Server Driver if required.

Download the Microsoft SQL Server JDBC driver from http://www.microsoft.com/en-us/download/details.aspx?displaylang=en&id=11774 and copy it to the /var/lib/sqoop/ directory. For example:

$ curl -L 'http://download.microsoft.com/download/0/2/A/02AAE597-3865-456C-AE7F-613F99F850A8/sqljdbc_4.0.2206.100_enu.tar.gz' | tar xz
$ sudo cp sqljdbc_4.0/enu/sqljdbc4.jar /var/lib/sqoop/

Step 6: Verify Sqoop

Run below command to verify the Sqoop installation

$ cd $SQOOP_HOME/bin 
$ sqoop-version