In this blog I discuss Sqoop deployment, Sqoop stands for SQL to Hadoop.
SQL is a tool which can import / export data from RDBMS
Sqoop
- Comes bundled with special connectors to Many RDBMS
- Is not a cluster, can be installed on one node only
- should be installed on edge node and not any of cluster nodes
On Edge Node
1. Download Sqoop [As root]
curl http://www-eu.apache.org/dist/sqoop/1.4.7/sqoop-1.4.7.bin__hadoop-2.6.0.tar.gz -o /sqoop-1.4.7.bin__hadoop-2.6.0.tar.gz
2. Add user [As root]
# groupadd -g 1000 hadoop
# useradd -u 1012 -g hadoop sqoop
# passwd sqoop
3. Untar sqoop [As root]
cd /usr/local
tar -xzf /tmp/sqoop-1.4.7.bin__hadoop-2.6.0.tar.gz
4. Create Soft Link
# ln -s /usr/local/sqoop-1.99.7-bin-hadoop200 /usr/local/sqoop
5. Set environment variables
vi /etc/profile.d/profile.sh (append)
export SQOOP_HOME=/usr/local/sqoop ;
export PATH=$PATH:$SQOOP_HOME/bin
source /etc/profile.d/profile.sh
6. Set environment for sqoop [As root]
cd $SQOOP_HOME/conf
mv sqoop-env-template.sh sqoop-env.sh
export HADOOP_COMMON_HOME=/usr/local/hadoop
export HADOOP_MAPRED_HOME=/usr/local/hadoop
7. Verify Install and find out sqoop version. [As sqoop]
sqoop-version
18/04/19 05:47:24 INFO sqoop.Sqoop: Running Sqoop version: 1.4.7
Sqoop 1.4.7
git commit id 2328971411f57f0cb683dfb79d19d4d19d185dd8
Compiled by maugli on Thu Dec 21 15:59:58 STD 2017
SQL is a tool which can import / export data from RDBMS
Sqoop
- Comes bundled with special connectors to Many RDBMS
- Is not a cluster, can be installed on one node only
- should be installed on edge node and not any of cluster nodes
On Edge Node
1. Download Sqoop [As root]
curl http://www-eu.apache.org/dist/sqoop/1.4.7/sqoop-1.4.7.bin__hadoop-2.6.0.tar.gz -o /sqoop-1.4.7.bin__hadoop-2.6.0.tar.gz
2. Add user [As root]
# groupadd -g 1000 hadoop
# useradd -u 1012 -g hadoop sqoop
# passwd sqoop
3. Untar sqoop [As root]
cd /usr/local
tar -xzf /tmp/sqoop-1.4.7.bin__hadoop-2.6.0.tar.gz
4. Create Soft Link
# ln -s /usr/local/sqoop-1.99.7-bin-hadoop200 /usr/local/sqoop
5. Set environment variables
vi /etc/profile.d/profile.sh (append)
export SQOOP_HOME=/usr/local/sqoop ;
export PATH=$PATH:$SQOOP_HOME/bin
source /etc/profile.d/profile.sh
6. Set environment for sqoop [As root]
cd $SQOOP_HOME/conf
mv sqoop-env-template.sh sqoop-env.sh
export HADOOP_COMMON_HOME=/usr/local/hadoop
export HADOOP_MAPRED_HOME=/usr/local/hadoop
7. Verify Install and find out sqoop version. [As sqoop]
sqoop-version
18/04/19 05:47:24 INFO sqoop.Sqoop: Running Sqoop version: 1.4.7
Sqoop 1.4.7
git commit id 2328971411f57f0cb683dfb79d19d4d19d185dd8
Compiled by maugli on Thu Dec 21 15:59:58 STD 2017
No comments:
Write comments